I'm running Snort which detects some P2P activity, specifically the BitTorrent announce request. I see the HTTP GET /announce.php?info_hash=XXX... request and I'm trying to convert this XXX into a proper SHA1 hash to try and get an idea of what is being downloaded.
I've read various things that say this is URL encoded, and others that say just remove the % character - however I am unable to reproduce this.
Can anyone suggest how to do this?
info_hash is an SHA1 hash. It's a binary hash, URL-encoded for inclusion in a URL.
If you want to turn it into a hex-encoded hash, you will need to extract it from the URL, URL-decode, and hex-encode. For example in Python:
>>> '%00%01%02%20%25ABC+XYZabc%7F%80%81%FE%FF'
'%00%01%02%20%25ABC+XYZabc%7F%80%81%FE%FF'
>>> urllib.unquote_plus(_)
'\x00\x01\x02 %ABC XYZabc\x7f\x80\x81\xfe\xff'
>>> _.encode('hex')
'00010220254142432058595a6162637f8081feff'
Okay, know I know. info_hash is an SHA1 hash. And an example of it is: %5d%97%dbA%d7a%2b%92%f5%c2%ef%dcv%bf%e7%e6%03%24%85%0a. If you use $_GET['info_hash'], it will not work, because of the %s.
You need to use $_SERVER['QUERY_STRING'].
Code example how to get SHA1 hash of info_hash in PHP:
$arye = $_SERVER['QUERY_STRING'];
$arye = explode('info_hash=', $arye)[1];
$arye = explode('&', $arye)[0];
$arye = explode('%', $arye);
$arp = '';
foreach($arye as $ara) {
if (strlen($ara) == 2) {
$arp .= $ara;
}else{
$e1 = substr($ara, 0, 2);
$e2 = substr($ara, 2, 1);
$e2 = unpack('H*', $e2)[1];
$arp .= $e1;
$arp .= $e2;
}
}
echo $arp; // This will be your SHA1 hash
Hash: %5d%97%dbA%d7a%2b%92%f5%c2%ef%dcv%bf%e7%e6%03%24%85%0a -> 5d97db41d7612b92f5c2efdc76bfe7e60324850a
No, you can,
infohash == sha1(torrentfile["info"])
But you could use info_hash as key to search it on DHT network
Related
I have the following code:
buff=esp.flash_read(esp.flash_user_start(),50)
print(buff)
I get the following output from print:
bytearray(b'{"ssid": "mySSID", "password": "myPASSWD"}\xff\xff\xff\xff\xff\xff')
What I want to do is get the json in buff. What is the correct "Python-way" to do that?
buff is a Python bytes object, as shown by the print output beginning with b'. To convert this into a string you need to decode it.
In standard Python you could use
buff.decode(errors='ignore')
Note that without specifying errors=ignore you would get a UnicodeDecodeError because the \xff bytes aren't valid in the default encoding, which is UTF-8; presumably they're padding and you want to ignore them.
If that works on the ESP8266, great! However this from the MicroPython docs suggests the keyword syntax might not be implemented - I don't have an ESP8266 to test it. If not then you may need to remove the padding characters yourself:
textLength = find(buff, b'\xff')
text = buff[0:textLength].decode()
or simply:
text = buff[0:buff.find(b'\xff')].decode()
If decode isn't implemented either, which it isn't in the online MicroPython interpreter, you can use str:
text = str(buff[0:find(buff, b'\xff')], 'utf-8')
Here you have to specify explicitly that you're decoding from UTF-8 (or whatever encoding you specify).
However if what you're really after is the values encoded in the JSON, you should be able to use the json module to retrieve them into a dict:
import json
j = json.loads(buff[0:buff.find(b'\xff')])
ssid = j['ssid']
password = j['password']
I use the Magento 2 REST API and want to handle errors that get thrown in C++.
Example error response:
{
"message": "Could not save category: %1",
"parameters": [
"URL key for specified store already exists."
]
}
I can retrieve both of these into a String and std::vector which leaves me with the question:
How am I able to return a String that is formatted by filling the placeholders?
With a fixed size, I could do something along the lines of this
char* buffer = new char[200];
String message = "Could not save category: %1";
std::vector<String> parameters = {"URL key for specified store already exists."};
String result = sprintf(buffer,message.c_str(),parameters[0]);
But, alas, I do not know the size beforehand.
How should I go about doing this? Are there stl functions that could help, should I be using self-written templates(no experience with this), can I convert the std::vector to a va_list, or are there other solutions?
Edit: I didn't notice this asks for a C++ dialect and not Standard C++. Leaving it for now, might be of use for other people.
Nothing standard exists that will do that automatically. That being said, if your interpolation format is just %<number> it might be easy enough to write:
string message = "Could not save category: %1";
std::vector<string> parameters = {"URL key for specified store already exists."};
for (size_t i = 0; i < parameters.size(); ++i) {
const auto interp = "%" + std::to_string(i+1);
const size_t pos = message.find(interp);
if (pos != std::string::npos)
message.replace(pos, interp.size(), parameters[i]);
}
This will put the result in the message string. Of course this implementation is rather limited and not particularly efficient, but again, doing this properly requires a library-sized solution, not SO-answer-sized one.
live version.
Let's imagine I have the following parts of a URL:
val url_start = "http://example.com"
val url_part_1 = "&fields[...]&" //This part of url can be in the middle of url or in the end
val url_part_2 = "&include..."
And then I try to concatenate the resulting URL like this:
val complete_url = url_start + url_part_2 + url_part_1
In this case I'd get http://example.com&include...&fields[...]& (don't consider syntax here), which is one & symbol between URL parts which means that concatenation was successful, BUT if I use different concat sequence in a different request like this:
val complete_url = url_start + url_part_1 + url_part_2
I'd get http://example.com&fields[...]&&include..., to be specific && in this case. Is there a way to ensure that concatenation is safer?
To keep you code clean use an array or object to keep your params and doin't keep "?" or "&" as part of urlStart or params. Add these at the end. e.g.
var urlStart = "http://example.com"
var params=[]
params.push ('a=1')
params.push ('b=2')
params.push ('c=3', 'd=4')
url = urlStart + '?' + params.join('&')
console.log (url) // http://example.com?a=1&b=2&c=3&d=4
First, you should note that it is invalid to have query parameters just after domain name; it should be something like http://example.com/?include...&fields[...] (note the /? part, you can replace it with / to make it a path parameter, but it's not likely that the router of the website supports parameters like this). Refer, for example, to this article: https://www.talisman.org/~erlkonig/misc/lunatech%5Ewhat-every-webdev-must-know-about-url-encoding/ to know more about what URLs can be valid.
For the simple abstract approach, you can use Kotlin's joinToString():
val query_part = arrayOf(
"fields[...]",
"include..."
).joinToString("&")
val whole_url = "http://example.com/?" + query_part
print(whole_url) // http://example.com/?fields[...]&include...
This approach is abstract because you can use joinToString() not only for URLs, but for whatever strings you want. This also means that if there will be an & symbol in one of the input strings itself, it will become two parameters in the output string. This is not a problem when you, as a programmer, know what strings will be joined, but if these strings are provided by user, it can become a problem.
For URL-aware approach, you can use URIBuilder from Apache HttpComponents library, but you'll need to import this library first.
the current hash is
{\"report_name\"=>\"Study/Control: ABIRATERONE ACETATE - 20151413355\", \"left_mue_start_date\"=>\"02-26-2015\", \"left_mue_end_date\"=>\"03-19-2015\", \"right_mue_start_date\"=>\"02-26-2015\", \"right_mue_end_date\"=>\"03-19-2015\", \"report_formulary_id\"=>\",7581\", \"mue\"=>\"true\", \"mue_type\"=>\"study/control\", \"chain_id\"=>\"1\", \"left_mue_formulary_ids\"=>[\"7581\"], \"action\"=>\"create_report\", \"controller\"=>\"informatics\", \"user_id\"=>339}
now I need to convert it in proper hash like
{"report_name" => "Study/Control: ABIRATERONE ACETATE - 20151413355"}
so I am trying to get it with JSON.parse but I am getting error like:
JSON::ParserError: 757: unexpected token at '{
So if someone know about that so please help me.
and I am using Rails 3.2
What you have is a hash printed as String. To convert it into a Hash use eval.
ch = "{\"report_name\"=>\"Study/Control: ABIRATERONE ACETATE - 20151413355\", \"left_mue_start_date\"=>\"02-26-2015\", \"left_mue_end_date\"=>\"03-19-2015\", \"right_mue_start_date\"=>\"02-26-2015\", \"right_mue_end_date\"=>\"03-19-2015\", \"report_formulary_id\"=>\",7581\", \"mue\"=>\"true\", \"mue_type\"=>\"study/control\", \"chain_id\"=>\"1\", \"left_mue_formulary_ids\"=>[\"7581\"], \"action\"=>\"create_report\", \"controller\"=>\"informatics\", \"user_id\"=>339}"
hash = eval(ch)
# => {"report_name"=>"Study/Control: ABIRATERONE ACETATE - 20151413355", "left_mue_start_date"=>"02-26-2015", "left_mue_end_date"=>"03-19-2015", "right_mue_start_date"=>"02-26-2015", "right_mue_end_date"=>"03-19-2015", "report_formulary_id"=>",7581", "mue"=>"true", "mue_type"=>"study/control", "chain_id"=>"1", "left_mue_formulary_ids"=>["7581"], "action"=>"create_report", "controller"=>"informatics", "user_id"=>339}
PS: A JSON string should look as follows, meaning what you have is not JSON and hence you got JSON::ParserError for using JSON.parse on a non-JSON string :
"{\"report_name\":\"Study/Control: ABIRATERONE ACETATE - 20151413355\",\"left_mue_start_date\":\"02-26-2015\",\"left_mue_end_date\":\"03-19-2015\",\"right_mue_start_date\":\"02-26-2015\",\"right_mue_end_date\":\"03-19-2015\",\"report_formulary_id\":\",7581\",\"mue\":\"true\",\"mue_type\":\"study/control\",\"chain_id\":\"1\",\"left_mue_formulary_ids\":[\"7581\"],\"action\":\"create_report\",\"controller\":\"informatics\",\"user_id\":339}"
To avoid using eval you could use JSON.parse ch.gsub('=>', ':') this way you will get a HASH from your HASH stored as STRING
Last time when I got this issue, since the json file that I got from a API that contains a BOM
UTF-8 BOM is a sequence of bytes (EF BB BF)
What's different between UTF-8 and UTF-8 without BOM?
at the beginning of that string, but you know that part wouldn't display or as readable when we got the string from response. I try to use Ruby JSON to parse it, but I failed, I got the same exception with yours. just a reminder for others, when you get a Json response. By the way, this would be no problem while you are handling that in javascript, but with problems in Python or Ruby languages.
I ran into a similar problem, though it was failing while parsing \"\". This is in regards to using Pact.IO. I mention it here since this is the highest ranked Google result while looking for the error I encountered. The solution in my case was to change the body of a POST in my C# application so that it wasn't using empty string, but a null string. Basically I added this before my HTTP call.
if (request.Method == HttpMethod.Post && request.Content!=null && request.Content.GetType()==typeof(StringContent) && request.Content.ReadAsStringAsync().Result == String.Empty)
request.Content = new StringContent(null);
Here's a challenge to all mathematica tag followers. Let's make it a lot more convenient to insert images into SO post from Mathematica by creating an imgur uploader.
How can we create a function imgur[g_] that will rasterize its argument (making sure that the final size is not wider than the width of StackOverflow posts), convert it to PNG, upload it to imgur, and return a ready to be pasted MarkDown line such as  ?
Useful references:
Imgur API
Example of using POST request from Mathematica on WRI blog (posting to Twitter) by ragfield
Example of using POST requests from Mathematica on SO (uploading to ifile.it)
I failed to adapt this latter method to uploading an image without exporting it to a file first.
Warning, use with care! StackOverflow uses a separate imgur installation that keep images indefinitely. If you use the main imgur, the images will disappear after 6 months if no one views them. Unfortunately as of 2011 November there seems to be no official way to upload images to StackOverflow programmatically.
Update: See below a solution for uploading to StackOverflow directly.
A little bird just informed me of a Mathematica solution to this question (the underlying implementation still uses JLink, but this answer hides all the java related code):
imgur[expr_] := Module[
{url, key, image, data, xml, imgurUrl},
url = "http://api.imgur.com/2/upload";
key = "c07bc3fb59ef878d5e23a0c4972fbb29";
image = Fold[ExportString, expr, {"PNG", "Base64"}];
xml = Import[url,
"XML", "RequestMethod" -> "POST",
"RequestParameters" -> {"key" -> key, "image" -> image}];
imgurUrl = Cases[xml, XMLElement["original", {}, {string_}] :> string,
Infinity][[1]];
"![Mathematica graphic](" <> imgurUrl <> ")"
]
This is V8 only and the XML import options "RequestMethod" and "RequestParameters" are undocumented and experimental (and therefore subject to change).
Note: Get an ready-made palette with this functionality here.
Arnoud's solution got me excited and impatient, so here's an improvement to it. I couldn't have done this without studying his code. This version seems to be somewhat more reliable and less prone to timeout errors, but to be honest, I know no Java at all, so any improvements are welcome.
Most importantly: this version uploads to stack.imgur.com directly, so it's safe to use here on StackOverflow, without having to worry that uploaded images will disappear after a while.
I provide three functions:
stackImage uploads the expression, exported as PNG, and returns the URL
stackMarkdown returns the markdown, ready to be copied
stackCopyMarkdown copies the markdown to the clipboard
Next step: create a palette button that does this automatically for the selected graphic in the notebook. Improvements to the code are very welcome.
Needs["JLink`"]
stackImage::httperr = "Server returned respose code: `1`";
stackImage::err = "Server returner error: `1`";
stackImage[g_] :=
Module[
{getVal, url, client, method, data, partSource, part, entity, code,
response, error, result},
(* this function attempts to parse the response fro the SO server *)
getVal[res_, key_String] :=
With[{k = "var " <> key <> " = "},
StringTrim[
First#StringCases[First#Select[res, StringMatchQ[#, k ~~ ___] &],
k ~~ v___ ~~ ";" :> v],
"'"]
];
data = ExportString[g, "PNG"];
JavaBlock[
url = "https://stackoverflow.com/upload/image";
client = JavaNew["org.apache.commons.httpclient.HttpClient"];
method = JavaNew["org.apache.commons.httpclient.methods.PostMethod", url];
partSource = JavaNew["org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource", "mmagraphics.png", MakeJavaObject[data]#toCharArray[]];
part = JavaNew["org.apache.commons.httpclient.methods.multipart.FilePart", "name", partSource];
part#setContentType["image/png"];
entity = JavaNew["org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity", {part}, method#getParams[]];
method#setRequestEntity[entity];
code = client#executeMethod[method];
response = method#getResponseBodyAsString[];
]
If[code =!= 200, Message[stackImage::httperr, code]; Return[$Failed]];
response = StringTrim /# StringSplit[response, "\n"];
error = getVal[response, "error"];
result = getVal[response, "result"];
If[StringMatchQ[result, "http*"],
result,
Message[stackImage::err, error]; $Failed]
]
stackMarkdown[g_] := "![Mathematica graphics](" <> stackImage[g] <> ")"
stackCopyMarkdown[g_] := Module[{nb, markdown},
markdown = Check[stackMarkdown[g], $Failed];
If[markdown =!= $Failed,
nb = NotebookCreate[Visible -> False];
NotebookWrite[nb, Cell[markdown, "Text"]];
SelectionMove[nb, All, Notebook];
FrontEndTokenExecute[nb, "Copy"];
NotebookClose[nb];
]
]
Update:
Here's a button that will show a preview of the selection and will offer uploading (or cancelling). It requires the previous functions to be defined.
Button["Upload to SO",
Module[{cell = NotebookRead#InputNotebook[], img},
If[cell =!= {}, img = Rasterize[cell];
MessageDialog[
Column[{"Upload image to StackExchange sites?",
img}], {"Upload and copy MarkDown" :> stackCopyMarkdown[img],
"Cancel" :> Null}, WindowTitle -> "Upload to StackExchange"]]]]
Unfortunately I can't put the button in a palette (CreatePalette) because the palette dimensions will influence the rasterization. Solutions to this problem are welcome.
Update 2:
Based on the answer to this question, here's a working Windows-only palette button:
button = Button["Upload to SO",
Module[{sel},
FrontEndExecute[
FrontEndToken[FrontEnd`SelectedNotebook[], "CopySpecial", "MGF"]];
sel = Cases[NotebookGet#ClipboardNotebook[],
RasterBox[data_, ___] :>
Image[data, "Byte", ColorSpace -> "RGB", Magnification -> 1],
Infinity];
If[sel =!= {},
With[{img = First[sel]},
MessageDialog[
Column[{"Upload image to StackExchange sites?",
img}], {"Upload and copy MarkDown" :> stackCopyMarkdown[img],
"Cancel" :> Null}, WindowTitle -> "Upload to StackExchange"]
]
]
]
]
CreatePalette[button]
Warning: it destroys the clipboard contents even if you click cancel in the preview box.
Note: This is using the anonymous imgur uploader with my anonymous key. The imgur site restricts uploads to 50 uploads/hour which should be fine normally, but this may cause a problem if a lot of people try this simultaneously. So please get your own anonymous key here:
http://imgur.com/register/api_anon
And then replace the key in the code below with your own key (thanks!).
The trickiest part to code was the conversion from a Mathematica expression to PNG image to Base64 encoding to URL encoding. There are about a 1,000 ways to do it wrong and I think I managed to try them all.
The code breaks down into a few pieces:
Construct the POST url
Make the HTTP connection
Send the POST url
Read back the result, which is XML
Extract the imgur url from the XML
Format the imgur url as markdown (or as a Mathematica Hyperlink function).
Here is the code:
imgur[expr_] :=
Module[{url, key, image, data, jUrl, jConn, jWriter, jInput, buffer,
byte, xml, imgurUrl},
Needs["JLink`"];
JLink`JavaBlock[
JLink`LoadJavaClass["java.net.URLEncoder"];
url = "http://api.imgur.com/2/upload";
key = "c07bc3fb59ef878d5e23a0c4972fbb29";
image = ExportString[ExportString[expr, "PNG"], "Base64"];
data =
URLEncoder`encode["key" , "UTF-8"] <> "=" <>
URLEncoder`encode[ key , "UTF-8"] <> "&" <>
URLEncoder`encode["image" , "UTF-8"] <> "=" <>
URLEncoder`encode[ image , "UTF-8"] ;
jUrl = JLink`JavaNew["java.net.URL", url];
jConn = jUrl#openConnection[];
jConn#setDoOutput[True];
jWriter =
JLink`JavaNew["java.io.OutputStreamWriter",
jConn#getOutputStream[]];
jWriter#write[data];
jWriter#flush[];
jInput = jConn#getInputStream[];
buffer = {};
While[(byte = jInput#read[]; byte >= 0), AppendTo[buffer, byte]];
];
xml = ImportString[FromCharacterCode[buffer], "XML"];
imgurUrl =
Cases[xml,
XMLElement["original", {}, {string_}] :>
string, \[Infinity]][[1]];
"![Mathematica graphic](" <> imgurUrl <> ")"
]
Testing:
In[]:= g = Graphics[{Blue, Disk[]}, PlotRange -> 1.2, ImageSize -> Small];
pic = Overlay[{Blur[Binarize#g, 10], g}];
imgur[pic]
Out[]= 
And the actual image: