Here's a challenge to all mathematica tag followers. Let's make it a lot more convenient to insert images into SO post from Mathematica by creating an imgur uploader.
How can we create a function imgur[g_] that will rasterize its argument (making sure that the final size is not wider than the width of StackOverflow posts), convert it to PNG, upload it to imgur, and return a ready to be pasted MarkDown line such as ![Mathematica graphic](http://i.imgur.com/ZENa4.jpg) ?
Useful references:
Imgur API
Example of using POST request from Mathematica on WRI blog (posting to Twitter) by ragfield
Example of using POST requests from Mathematica on SO (uploading to ifile.it)
I failed to adapt this latter method to uploading an image without exporting it to a file first.
Warning, use with care! StackOverflow uses a separate imgur installation that keep images indefinitely. If you use the main imgur, the images will disappear after 6 months if no one views them. Unfortunately as of 2011 November there seems to be no official way to upload images to StackOverflow programmatically.
Update: See below a solution for uploading to StackOverflow directly.
A little bird just informed me of a Mathematica solution to this question (the underlying implementation still uses JLink, but this answer hides all the java related code):
imgur[expr_] := Module[
{url, key, image, data, xml, imgurUrl},
url = "http://api.imgur.com/2/upload";
key = "c07bc3fb59ef878d5e23a0c4972fbb29";
image = Fold[ExportString, expr, {"PNG", "Base64"}];
xml = Import[url,
"XML", "RequestMethod" -> "POST",
"RequestParameters" -> {"key" -> key, "image" -> image}];
imgurUrl = Cases[xml, XMLElement["original", {}, {string_}] :> string,
Infinity][[1]];
"![Mathematica graphic](" <> imgurUrl <> ")"
]
This is V8 only and the XML import options "RequestMethod" and "RequestParameters" are undocumented and experimental (and therefore subject to change).
Note: Get an ready-made palette with this functionality here.
Arnoud's solution got me excited and impatient, so here's an improvement to it. I couldn't have done this without studying his code. This version seems to be somewhat more reliable and less prone to timeout errors, but to be honest, I know no Java at all, so any improvements are welcome.
Most importantly: this version uploads to stack.imgur.com directly, so it's safe to use here on StackOverflow, without having to worry that uploaded images will disappear after a while.
I provide three functions:
stackImage uploads the expression, exported as PNG, and returns the URL
stackMarkdown returns the markdown, ready to be copied
stackCopyMarkdown copies the markdown to the clipboard
Next step: create a palette button that does this automatically for the selected graphic in the notebook. Improvements to the code are very welcome.
Needs["JLink`"]
stackImage::httperr = "Server returned respose code: `1`";
stackImage::err = "Server returner error: `1`";
stackImage[g_] :=
Module[
{getVal, url, client, method, data, partSource, part, entity, code,
response, error, result},
(* this function attempts to parse the response fro the SO server *)
getVal[res_, key_String] :=
With[{k = "var " <> key <> " = "},
StringTrim[
First#StringCases[First#Select[res, StringMatchQ[#, k ~~ ___] &],
k ~~ v___ ~~ ";" :> v],
"'"]
];
data = ExportString[g, "PNG"];
JavaBlock[
url = "https://stackoverflow.com/upload/image";
client = JavaNew["org.apache.commons.httpclient.HttpClient"];
method = JavaNew["org.apache.commons.httpclient.methods.PostMethod", url];
partSource = JavaNew["org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource", "mmagraphics.png", MakeJavaObject[data]#toCharArray[]];
part = JavaNew["org.apache.commons.httpclient.methods.multipart.FilePart", "name", partSource];
part#setContentType["image/png"];
entity = JavaNew["org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity", {part}, method#getParams[]];
method#setRequestEntity[entity];
code = client#executeMethod[method];
response = method#getResponseBodyAsString[];
]
If[code =!= 200, Message[stackImage::httperr, code]; Return[$Failed]];
response = StringTrim /# StringSplit[response, "\n"];
error = getVal[response, "error"];
result = getVal[response, "result"];
If[StringMatchQ[result, "http*"],
result,
Message[stackImage::err, error]; $Failed]
]
stackMarkdown[g_] := "![Mathematica graphics](" <> stackImage[g] <> ")"
stackCopyMarkdown[g_] := Module[{nb, markdown},
markdown = Check[stackMarkdown[g], $Failed];
If[markdown =!= $Failed,
nb = NotebookCreate[Visible -> False];
NotebookWrite[nb, Cell[markdown, "Text"]];
SelectionMove[nb, All, Notebook];
FrontEndTokenExecute[nb, "Copy"];
NotebookClose[nb];
]
]
Update:
Here's a button that will show a preview of the selection and will offer uploading (or cancelling). It requires the previous functions to be defined.
Button["Upload to SO",
Module[{cell = NotebookRead#InputNotebook[], img},
If[cell =!= {}, img = Rasterize[cell];
MessageDialog[
Column[{"Upload image to StackExchange sites?",
img}], {"Upload and copy MarkDown" :> stackCopyMarkdown[img],
"Cancel" :> Null}, WindowTitle -> "Upload to StackExchange"]]]]
Unfortunately I can't put the button in a palette (CreatePalette) because the palette dimensions will influence the rasterization. Solutions to this problem are welcome.
Update 2:
Based on the answer to this question, here's a working Windows-only palette button:
button = Button["Upload to SO",
Module[{sel},
FrontEndExecute[
FrontEndToken[FrontEnd`SelectedNotebook[], "CopySpecial", "MGF"]];
sel = Cases[NotebookGet#ClipboardNotebook[],
RasterBox[data_, ___] :>
Image[data, "Byte", ColorSpace -> "RGB", Magnification -> 1],
Infinity];
If[sel =!= {},
With[{img = First[sel]},
MessageDialog[
Column[{"Upload image to StackExchange sites?",
img}], {"Upload and copy MarkDown" :> stackCopyMarkdown[img],
"Cancel" :> Null}, WindowTitle -> "Upload to StackExchange"]
]
]
]
]
CreatePalette[button]
Warning: it destroys the clipboard contents even if you click cancel in the preview box.
Note: This is using the anonymous imgur uploader with my anonymous key. The imgur site restricts uploads to 50 uploads/hour which should be fine normally, but this may cause a problem if a lot of people try this simultaneously. So please get your own anonymous key here:
http://imgur.com/register/api_anon
And then replace the key in the code below with your own key (thanks!).
The trickiest part to code was the conversion from a Mathematica expression to PNG image to Base64 encoding to URL encoding. There are about a 1,000 ways to do it wrong and I think I managed to try them all.
The code breaks down into a few pieces:
Construct the POST url
Make the HTTP connection
Send the POST url
Read back the result, which is XML
Extract the imgur url from the XML
Format the imgur url as markdown (or as a Mathematica Hyperlink function).
Here is the code:
imgur[expr_] :=
Module[{url, key, image, data, jUrl, jConn, jWriter, jInput, buffer,
byte, xml, imgurUrl},
Needs["JLink`"];
JLink`JavaBlock[
JLink`LoadJavaClass["java.net.URLEncoder"];
url = "http://api.imgur.com/2/upload";
key = "c07bc3fb59ef878d5e23a0c4972fbb29";
image = ExportString[ExportString[expr, "PNG"], "Base64"];
data =
URLEncoder`encode["key" , "UTF-8"] <> "=" <>
URLEncoder`encode[ key , "UTF-8"] <> "&" <>
URLEncoder`encode["image" , "UTF-8"] <> "=" <>
URLEncoder`encode[ image , "UTF-8"] ;
jUrl = JLink`JavaNew["java.net.URL", url];
jConn = jUrl#openConnection[];
jConn#setDoOutput[True];
jWriter =
JLink`JavaNew["java.io.OutputStreamWriter",
jConn#getOutputStream[]];
jWriter#write[data];
jWriter#flush[];
jInput = jConn#getInputStream[];
buffer = {};
While[(byte = jInput#read[]; byte >= 0), AppendTo[buffer, byte]];
];
xml = ImportString[FromCharacterCode[buffer], "XML"];
imgurUrl =
Cases[xml,
XMLElement["original", {}, {string_}] :>
string, \[Infinity]][[1]];
"![Mathematica graphic](" <> imgurUrl <> ")"
]
Testing:
In[]:= g = Graphics[{Blue, Disk[]}, PlotRange -> 1.2, ImageSize -> Small];
pic = Overlay[{Blur[Binarize#g, 10], g}];
imgur[pic]
Out[]= ![Mathematica graphic](http://i.imgur.com/eGOlL.png)
And the actual image:
Related
For some reason the iOS Safari browser does not allow you to see xml content returned via a server.
So to try and get around this, I thought I’d try to take the distinctive xml characters ‘>’ and ‘<‘ with something else, which is unlikely to be challenged e.g ‘~’.
I’ve tried a number of different ways , and while I can use the following to find/replace letters, when I try it with special characters, i can’t seem to get it to work.
Can anyone help ?
local xmltest = "<XML Test>"
local t = {< = "~", > = "~"}
local result = string.gsub(xmltest, "<>", t)
print(result)
Many thanks
Here is the answer, thanks #lhf
local xmltest = "<XML Test>"
local t = {["<"] = "~", [">"] = "~"}
local result = string.gsub(xmltest, "[<>]", t)
print(result)
How and why a non-English word is converted to weird characters like پاکستان to پاکستان, is there any way back to get پاکستان from پاکستان. It happens in browser shown code and received requests at server
Background:
I get lot of requests at my Non-English content (urdu) website with urls like
پاکستان
I tried to know what that means but search engines don't help. I tried things like
Decode this 'mystring'
What ecoding is this 'mystring'
I thought it might be corrupted/spam url, from this link
Weird characters in URL
Problem explanation/example
But when I viewed one my js file in browser (while having look on working js file). It is showing me same wired characters in browser, even at localhost
'pakistan': {'eng': 'Pakistan', 'ur': 'پاکستان'},
//But actually source code for above line is following
'pakistan': {'eng': 'Pakistan', 'ur': 'پاکستان'},
But in browser its showing me following for same line,
My knowledge
I only know about Encoding/Decoding, which seems unrelated here with best of my knowledge as?
encodeURI and decodeURI in JS or quote and unquote in python and same for other languages. But what they do for me is only
`پاکستان` to `%D9%BE%D8%A7%DA%A9%D8%B3%D8%AA%D8%A7%D9%86` and vise versa
Why needed?
I don't want to miss the requests received with those malformed urls, there must be some things to undo as all browsers chrome/firefox/edge showing those characters same, If their translation/conversion method and result is same then there should be some technique available to reverse it as well
Thanks to Giacomo Catenazzi and then I be greatful to the following answer
How to decode cp1252 string?
A very custom and still imperfect solution to my problem.
This algo needs to be improved Only by experiment I came to know, this algo works as its not working for me when string is long or including - (hyphens)
So I made changes according to my requirement and its working fair enough, so that I could guess what the actual string was.
import re, itertools
from lxml.builder import unicode
def specific_my_required_processing(received_string):
starting_characters_in_encoded_string_in_my_case = ['Ø', 'Ã', 'Ù', 'Ù', 'Ú']
arr = received_string.split('-')
res = []
missed = []
for string_item in arr:
decoded_string = guess_decode_string_without_hyphens(string_item)
if decoded_string and decoded_string[:1] not in starting_characters_in_encoded_string_in_my_case:
res.append(decoded_string)
else:
missed.append({string_item: decoded_string})
resulting_urdu_string = '-'.join(res)
print('\n\nResult', resulting_urdu_string)
print('\nCould not be decoded', missed)
def guess_decode_string_without_hyphens(s):
encodings = ['cp1251', 'cp1252', 'utf8']
for steps in range(2, 10, 2):
for encs in itertools.product(encodings, repeat=steps):
r = s
try:
for enc in encs:
r = r.encode(enc) if isinstance(r, unicode) else r.decode(enc)
except (UnicodeEncodeError, UnicodeDecodeError) as e:
continue
if re.match(u'^[\w\sа-яА-Я]+$', r):
res = str(r)
print('Encoding => ', encs, ' Conversion = ' + s + ' => ' + res)
return res
sample_encoded_string = 'اسلام-آباد-Ûائیکورٹ-ای-ÙˆÛŒ-ایم-قانون-سازی-کالعدم-قرار-دینے-Ú©ÛŒ-درخواست-نامکمل-قرار'
specific_my_required_processing(sample_encoded_string)
I use the Magento 2 REST API and want to handle errors that get thrown in C++.
Example error response:
{
"message": "Could not save category: %1",
"parameters": [
"URL key for specified store already exists."
]
}
I can retrieve both of these into a String and std::vector which leaves me with the question:
How am I able to return a String that is formatted by filling the placeholders?
With a fixed size, I could do something along the lines of this
char* buffer = new char[200];
String message = "Could not save category: %1";
std::vector<String> parameters = {"URL key for specified store already exists."};
String result = sprintf(buffer,message.c_str(),parameters[0]);
But, alas, I do not know the size beforehand.
How should I go about doing this? Are there stl functions that could help, should I be using self-written templates(no experience with this), can I convert the std::vector to a va_list, or are there other solutions?
Edit: I didn't notice this asks for a C++ dialect and not Standard C++. Leaving it for now, might be of use for other people.
Nothing standard exists that will do that automatically. That being said, if your interpolation format is just %<number> it might be easy enough to write:
string message = "Could not save category: %1";
std::vector<string> parameters = {"URL key for specified store already exists."};
for (size_t i = 0; i < parameters.size(); ++i) {
const auto interp = "%" + std::to_string(i+1);
const size_t pos = message.find(interp);
if (pos != std::string::npos)
message.replace(pos, interp.size(), parameters[i]);
}
This will put the result in the message string. Of course this implementation is rather limited and not particularly efficient, but again, doing this properly requires a library-sized solution, not SO-answer-sized one.
live version.
CONCLUSION:
For some reason the flow wouldn't let me convert the incoming message to a BLOB by changing the Message Domain property of the Input Node so I added a Reset Content Descriptor node before the Compute Node with the code from the accepted answer. On the line that parses the XML and creates the XMLNSC Child for the message I was getting a 'CHARACTER:Invalid wire format received' error so I took that line out and added another Reset Content Descriptor node after the Compute Node instead. Now it parses and replaces the Unicode characters with spaces. So now it doesn't crash.
Here is the code for the added Compute Node:
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
DECLARE NonPrintable BLOB X'0001020304050607080B0C0E0F101112131415161718191A1B1C1D1E1F7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECFD0D1D2D3D4D5D6D7D8D9DADBDCDDDEDFE0E1E2E3E4E5E6E7E8E9EAEBECEDEEEFF1F2F3F4F5F6F7F8F9FAFBFCFDFEFF';
DECLARE Printable BLOB X'20202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020';
DECLARE Fixed BLOB TRANSLATE(InputRoot.BLOB.BLOB, NonPrintable, Printable);
SET OutputRoot = InputRoot;
SET OutputRoot.BLOB.BLOB = Fixed;
RETURN TRUE;
END;
UPDATE:
The message is being parsed as XML using XMLNSC. Thought that would cause a problem, but it does not appear to be.
Now I'm using PHP. I've created a node to plug into the legacy flow. Here's the relevant code:
class fixIncompetence {
function evaluate ($output_assembly,$input_assembly) {
$output_assembly->MRM = $input_assembly->MRM;
$output_assembly->MQMD = $input_assembly->MQMD;
$tmp = htmlentities($input_assembly->MRM->VALUE_TO_FIX, ENT_HTML5|ENT_SUBSTITUTE,'UTF-8');
if (!empty($tmp)) {
$output_assembly->MRM->VALUE_TO_FIX = $tmp;
}
// Ensure there are no null MRM fields. MessageBroker is strict.
foreach ($output_assembly->MRM as $key => $val) {
if (empty($val)) {
$output_assembly->MRM->$key = '';
}
}
}
}
Right now I'm getting a vague error about read only messages, but before that it wasn't working either.
Original Question:
For some reason I am unable to impress upon the senders of our MQ
messages that smart quotes, endashes, emdashes, and such crash our XML
parser.
I managed to make a working solution with SQL queries, but it wasted
too many resources. Here's the last thing I tried, but it didn't work
either:
CREATE FUNCTION CLEAN(IN STR CHAR) RETURNS CHAR BEGIN
SET STR = REPLACE('–',STR,'–');
SET STR = REPLACE('—',STR,'—');
SET STR = REPLACE('·',STR,'·');
SET STR = REPLACE('“',STR,'“');
SET STR = REPLACE('”',STR,'”');
SET STR = REPLACE('‘',STR,'&lsqo;');
SET STR = REPLACE('’',STR,'’');
SET STR = REPLACE('•',STR,'•');
SET STR = REPLACE('°',STR,'°');
RETURN STR;
END;
As you can see I'm not very good at this. I have tried reading about
various ESQL string functions without much success.
So in ESQL you can use the TRANSLATE function.
The following is a snippet I use to clean up a BLOB containing non-ASCII low hex values so that it then be cast into a usable character string.
You should be able to modify it to change your undesired characters into something more benign. Basically each hex value in NonPrintable gets translated into its positional equivalent in Printable, in this case always a full-stop i.e. x'2E' in ASCII. You'll need to make your BLOB's long enough to cover the desired range of hex values.
DECLARE NonPrintable BLOB X'000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F';
DECLARE Printable BLOB X'2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E2E';
SET WorkBlob = TRANSLATE(WorkBlob, NonPrintable, Printable);
BTW if messages with invalid characters only come in every now and then I'd probably specify BLOB on the input node and then use something similar to the following to invoke the XMLNSC parser.
CREATE LASTCHILD OF OutputRoot DOMAIN 'XMLNSC'
PARSE(InputRoot.BLOB.BLOB CCSID InputRoot.Properties.CodedCharSetId ENCODING InputRoot.Properties.Encoding);
With the exception terminal wired up you can then correct the BLOB's of any messages containing parser breaking invalid characters before attempting to reparse.
Finally my best wishes as I've had a number of battles over the years with being forced to correct invalid message content in the "Integration Layer" after all that's what it's meant to do.
I really hope you will be able to help me out on this one.
I am new to pdf.js so for the moment, I am playing around with the pre-built version to see if I can integrate this into my web app.
My problem:
I am using tcpdf to generate a pdf file which I would like to visualize using pdf.js without having to save it to a file on the server.
I have a php file (generate_document.php) that I use to generate the pdf. The file ends with the following:
$pdf->Output('test.pdf', 'I');
according to the tcpdf documentation, the second parameter can be used to generate the following formats:
I: send the file inline to the browser (default). The plug-in is used if available. The name given by name is used when one selects the "Save as" option on the link generating the PDF.
D: send to the browser and force a file download with the name given by name.
F: save to a local server file with the name given by name.
S: return the document as a string (name is ignored).
FI: equivalent to F + I option
FD: equivalent to F + D option
E: return the document as base64 mime multi-part email attachment (RFC 2045)
Then, I would like to view the pdf using pdf.js without creating a file on the server (= not using 'F' as a second parameter and passing the file name to pdf.js).
So, I thought I could simply create an iframe and call the pdf.js viewer pointing to the php file:
<iframe width="100%" height="100%" src="/pdf.js_folder/web/viewer.html?file=get_document.php"></iframe>
However, this is not working at all....do you have any idea what I am overlooking? Or is this option not available in pdf.js?
I have done some research and I have seen some posts here on converting a base64 stream to a typed array but I do not see how this would be a solution to this problem.
Many thanks for your help!!!
EDIT
#async, thanks for your anwer.
I got it figured out in the meantime, so I thought I'd share my solution with you guys.
1) In my get_document.php, I changed the output statement to convert it directly to base64 using
$pdf_output = base64_encode($pdf->Output('test_file.pdf', 'S'));
2) In viewer.js, I use an XHR to call the get_document.php and put the return in a variable (pdf_from_XHR)
3) Next, I convert what came in from the XHR request using the solution that was already mentioned in a few other posts (e.g. Pdf.js and viewer.js. Pass a stream or blob to the viewer)
pdf_converted = convertDataURIToBinary(pdf_from_XHR)
function convertDataURIToBinary(dataURI) {
var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
var base64 = dataURI.substring(base64Index);
var raw = window.atob(base64);
var rawLength = raw.length;
var array = new Uint8Array(new ArrayBuffer(rawLength));
for (i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i);
}
return array;
}
et voilà ;-)
Now i can inject what is coming from that function into the getDocument statement:
PDFJS.getDocument(pdf_converted).then(function (pdf) {
pdfDocument = pdf;
var url = URL.createObjectURL(blob);
PDFView.load(pdfDocument, 1.5)
})