I've made an iPad app that posts data to a ASP script. Data is then stored in UTF-8 in the MySQL db. Today one of the users posted data which made an error:
Data posted:
Jeanette Sjösvärd, Uttke Renata, Håkan Giljam
Data saved in log and db:
Jeanette Sjösvärd, Uttke Renata, Håkan Giljam
When reading data from the database, the text is full of "Ã¥ ä ö" (should be "å ä ö")
The log also saves the original post data the way it arrives to the server in percentage format:
Jeanette%20Sj%C3%B6sv%C3%A4rd%2C%20Uttke%20Renata
When posting all the data again (copy and paste that percentage-encoded block) from an ASP page, the data gets saved without any encoding issues.
Facts
Full posted data is about 18kB
All ASP pages contains <%#LANGUAGE="VBSCRIPT" CODEPAGE="65001"%> on top
Questions/ideas
Why does the ASP script read the data in different ways, depending on the sender?
Why did it just happen 1 out of 100 times?
Is there any encoding information that is/should be sent along with a POST request?
Could it be depending on some single special character in the data?
Does the iPad use any other encoding than UTF-8 as a standard? (The iPad is set to Swedish)
Edit
Found this thread: What determines the encoding of then data you receive from an HTTP POST?
Will look into the encoding header of the post request. But how could I detect if posted data isn't UTF-8 and in that case convert it?
Follow-up in this post: https://stackoverflow.com/questions/15656710/how-to-detect-post-encoding-and-convert-to-utf-8-in-asp
Related
I have a Rails/Ember one-page app. Burp reports that
The value of the 'content_type' JSON parameter is copied into the HTML
document as plain text between tags. The payload
da80balert(1)4f31e was submitted in the content_type
JSON parameter. This input was echoed unmodified in the application's
response.
I can't quite parse this message referring to "is copied into" and "was submitted" in, but basically what is happening is:
A PUT or POST from the client contains ...<script>...</script>... in some field.
The server handles this request, and sends back the created object in JSON format, which includes the string in question
The client then displays that string, using the standard Embers/Handlebars {{content_type}}, which HTML-escapes the string and inserts it into the DOM, so the browser displays it on the screen as originally entered (and of course does NOT execute it).
So yes, the input was indeed echoed unmodified in the application's response. However, the application's response was not HTML, in which case there would indeed be a problem, but JSON, containing strings which when referred to by Handlebars will always be escaped properly for proper display in the browser.
So my question is, is this in fact a vulnerability? I have taken great care with my Ember app and can prove that no data from JSON objects is ever inserted "raw" into the DOM. Or is this a false positive given rise to by the mere fact the unescaped string may be found in the response if looked for using an unintelligent string comparison, not taking into account the fact that the JSON will be processed/escaped by the client-side framework?
To put it a different way, in a classic webapp spitting out HTML from the server, we know that user input such as the above must be escaped/sanitized properly. Unsanitized data "on the wire" in and of itself represents a vulnerability. However, in a one-page app based on JSON coming back from the server, the escaping/sanitization occurs in the client; the JSON on the "wire" may contain unsanitized data, and this is as expected. Am I missing something here?
There are subtle ways in which you can trick IE9 and older into treating JSON as HTML. So even if the server's response has a Content-Type header of application/json, IE will second guess it. This is called content type sniffing, and can be disabled by adding the X-Content-Type-Options: nosniff header.
JSON is not an executable format so your understanding is correct.
I did a demo of this exact problem in my talk on securing single page web apps at OWASP AppSec EU 2013 which someone put up on youtube here: http://m.youtube.com/watch?v=Femsrx0m9bU
I've built a DataSnap REST server using Delphi XE2 and I've added a server method for uploading files via TStream :
function TServerMethods.updateUploadFile(sFilename: string; UploadStream: TStream): string;
I want to be able to call this from a number of different clients (Android, iOS etc) and I've been testing the method using various REST clients such as Postman (Chrome plugin). However so far I cannot get it to accept the content for the HTTP POST body. Whenever I send the POST command I always get the following response :
{"error":"Message content is not a valid JSON value."}
I've tried using various different 'Content-Type' settings but nothing seems to work. It looks as though DataSnap is insisting on the content being JSON? I've tried pasting some valid JSON into the content area but this also gave the same error response.
Has anybody successfully used TStream as an input parameter for a DataSnap server method? Should I be doing it another way? I've used TStream as an output parameter for downloading files many times and it works well, this is my first attempt at uploading.
UPDATE
I made a quick Delphi DataSnap client to test the uploadFile server method and this all works great. I then used Fiddler to examine the POST command the Delphi client uses to send the TStream in the content body, and noticed it is a JSON array of integers (bytes) e.g. [37,80,68,70,45,49,46,51,13,10]. So I can see that I could modify my Android/iOS clients to convert the binary data to this byte array format before POSTing, but this is an overhead I could do without. If DataSnap streams raw binary when TStream is a return parameter, why can't it stream raw binary as an input parameter?
It seems when adding content data to the request body in a POST command, DataSnap server insists that this data is always JSON. This is why when using TStream as an input parameter, the stream data is converted to a JSON array of integers (bytes) by the Delphi DataSnap client. This format is very size inefficient as with upto 3 digits per byte, plus the comma, the size of the data being uploaded can grow by as much as 4 times.
What I have therefore done instead is to encode the data to upload in Base64. My server method now looks like this :
function TServerMethods.updateUploadFile(sFilename: string; Base64Data: TJSONObject): string;
Notice I'm wrapping the Base64 string in a TJSONObject. This is because if you just specify a String type, the Delphi DataSnap client will call the method with a GET and attempt to put the whole Base64 string in the URL path, causing a 'Connection Closed Gracefully' error. Using a TJSONObject forces DataSnap to use a POST and put the data in the content body. The JSON object passed is a single pair object :
{"UploadedData":"JVBERi0xLjMNCiXi48B5SiWGTK3PaY.........."}
This way the size of the data uploaded is much smaller and faster to transfer. I'd still prefer to be able to stream the raw data in the content body but DataSnap doesn't allow this.
Is there any way to tell JSoup to post data using iso-8859-1 rather than utf-8 ?
I tried posting a parameter that contains the letter 'è' the my webserver receives the character with hex code C3A8 but I want to send E8.
The code I wrote is
Document document = Jsoup.connect("https://somesite.com/test") .data("parameter1","\u00E8").
header("Content-Type","application/x-www-form-urlencoded;charset=UTF-8")
.method(Method.POST)
.execute()
.parse();
As already said, on the other hand I get a 2 bytes data (C3A8) rather than a single byte with E8 inside.
Thanks in advance for your help.
Trying to replicate a successful POST request with JSoup - data posted to server does not get decoded
Has an accepted answer, saying it is not possible to do it. You will have to find a way to post your code into utf-8. If you do not succeed in that, just open a new question about it.
I have a basic form that asks the user to enter some text in a regular html input control and I am also using jquery-tokeninput to allow users to choose tags from a pre-filled list.
One of the tags in the pre-filled list happens to be the word café, which I have got from the server and populating the tag list by calling
mytaglist.push({id: 'café', name: 'café'});
The problem is that when they attempt to enter a word like 'café' as a tag, asp.net mvc rejects the input saying that:
A potentially dangerous Request.Form value was detected from the client (articleTags="café").
Inspection using firebug shows my post data to be something like:
UserName=neo&category=&Title=caf%C3%A9&Text=sometext&articleTags=caf%26%23233%3B&IsAgreedTerms=true
As you can see, Title has value caf%C3%A9 which is correct but articleTags has value caf%26%23233%3B which I was not expecting.
I absolutely need to make sure that café (and not some encoded value) appears on the screen.
How can I make sure that I send the correct post data and still display café everytime?
Should I change how my server sends the text?
sending the data via Html.Raw solves the problem..
in my application ,i am sending data from my application to database
i am getting some odd characters in my database like this
i am sending my data like this
var
w:widestring;
u:utf8string;
begin
w:=data //data is function to get some info(string)
u:=utf8encode(w);
sendfn(u);
end;
i am using utf8_decode(my get data) in my php code before adding to my database.
and my database and tables collation is utf8_general_ci
can anyone help me in this issue
It's an educated guess, but does the Data function return an UTF-8 string instead of a WideString? I think the error could be in this Data function that you're calling, which returns the data in the wrong string format.
The php function utf8_decode converts characters from utf-8 to ISO-8859-1. If the path your data take (the browser or whatever component you use to send your data to the web server (http request, your php installation, your webpage and your database connection) from your delphi app to your database that is behind you webpage are able to support and configured to use utf-8 data you don't need the utf8_decode function, you can just insert your data the way it comes.
If you haven't already configured php to work with UTF-8, be aware that it is difficult and never works 100% (for me at least it never did), so maybe it would be better for you to use data in your locale encoding.