We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.
Related
please note that i had asked this question a while back but i was unable to attend to it because of some personal issue, so it got deleted, meta user suggested i reask the question and so here it is
Recently i encountered a website which doesn't send any dynamic information when you try to login, not even Username and Password !, what i mean is that after i use Fiddler and Http Analyzer i can see that 4 event happens :
All of these happens in the web browser :
1- A simple Get for the login main page
2- A Post which sends some data (These data do not include my Username and Password and they seem static! and even though there is a __USERCONTROLPATH which can be extracted from the step 1 response, it doesn't change, meaning even if i try to login multiple times using different browser at different times everything including the __USERCONTROLPATH is the same)
3- A Post which is of JSON type and the request is empty and the response only contains a single line as you can see at the image below:
4- A Get happens which is the login main page but if you look at HTML you can see that the user is logged in!
I tried doing these steps one by one, but i am unable to successfully log in to the website and i am unable to understand how my Username and Password is sent!
Here you can see some of the code i have written:
//for the sake of cookies and ... i do a Get on the main page
idHttp.get('MainLogPage');
//i send the first post which contains some information which never changes, they seems static and do not include the username and password
idhttp.post('someURL', requestList);
jsonRawRequest := TJSONObject.Create;
jsonRequest := TStringStream.Create(jsonRawRequest.ToString, TEncoding.UTF8);
/here i try to do the post that you saw in the image, no value is sent so i send an empty json request.
idhttp.post('someOtherURL', jsonRequest)
In the end my question is how can i login to such a site ?, what am i missing ?, currently i get error on Step 3, when i post the JSON, here is a picture of the error:
Update 01 :
I meant to update the whole post and explain every step in more detail but as i was messing with fiddler and checking the posts again i noticed something, when the Browser sends the POST "https://www.somewebsite.com//Index.aspx/LoginUser" the type is application/json and in the "HTTP analyzer" i can see that it send and receive a JSON but in the Fiddler the Webforms is empty! so i check the TextView thats where i saw it !, i again went and check HTTP analyzer's Raw stream, and in there i also saw a user and password being sent, i don't know how the post contains this string that you can see below, but in fiddler you can see it in "TextView" and in http analyzer you can see it in raw stream, now that i know this, can you tell me how can i imitate such a post ?, by the way the user and password are encrypted which i presume is a matter of its own, and i probably have to contact the site's admin for the algorithm ?, anyway, first things first, how can i send such a POST ?
In Fiddler :
HTTP Analyzer :
Update 02:
I tried what Remy Lebeau said,
But the JSON in the site was not standard as you can see in:
The input is not a valid Base-64 string Error, My JSON uses double quotation marks instead of single quotations
After that i tried a hard coded solution and i read the JSON from a file as you can see below:
loader := TStringList.Create;
loader.LoadFromFile('jsonWithCustomFormat.txt');
jsonRequest := TStringStream.Create(loader[0], TEncoding.UTF8);
After that i set the refer to the login page:
idHttp.Request.Referer := 'myLoginPageURL';
idHttp.Request.ContentType := 'application/json'
And Finally i send the POST:
idhttp.Post(URL, jsonRequest, ms);
The result of what i did was an alert error as you can in the image below:
I am very confused as to what the problem is, i have to solve this so any hint or help is much appreciated.
As you can see that postman returns expected result
but res.asString() gives [] in the blow code,can you tell me why?
def "simple test"(){
String url="http://xxx.xxx.xxx/assessment/api/Test.html"
when:""
io.restassured.response.Response res=RestAssured.given().header("Content-Type", "application/x-www-form-urlencoded").formParam("Action", "getDiagnosisList").formParam("Data", "[{\"subject\":\"冠心病\",\"option\":\"是\"}]").post(url)
then:""
res.prettyPrint()=="[\"身体健康状态不良\",\"医疗处置\"]"
}
It turns out that Chinese characters can't be encoded correctedly by default,after adding blow code,everything worked as expected:
RestAssured.given().config(RestAssured.config().encoderConfig(EncoderConfig.encoderConfig().defaultContentCharset("UTF-8")))
Maybe the request did via postman has not been cached, and on the other hand the same request via restassured is using some kind of cache. Recently I was having a similar issue because of it was hitting the varnish server. I'd recommend you to take a look at the response headers from both postman and restassured.
Been having some difficulty with an attachment issue on my site. At the moment our iOS app is pointing at an API endpoint for attachments, and sending a request similar to this:
POST /api/v2/attachments HTTP/1.1
--Boundary+0xAbCdEfGbOuNdArY
Content-Disposition: form-data; name="attachment"; filename="attachment.jpg"
Content-Type: image/jpeg
...image data...
--Boundary+0xAbCdEfGbOuNdArY--
Now, the request succeeds and the image is in fact uploaded, but it's turning out to be an invalid image because the boundary data is written to file. It basically looks like this:
file = Tempfile.new('attachment')
attachment_data = request.body.read
attachment_data.force_encoding('UTF-8')
file << attachment_data
attachment.asset = file
attachment.save!
Obviously request.body.read is including the entire request, Boundaries and all. We do actually have a stripping method that runs through each line of the file and strips out non-image data, but that's obviously not performant at all.
In an ideal world, we would just be getting the image data itself and using that to populate the tempfile, but I'm afraid I'm completely stumped about the best way to go about that.
Thoughts welcome. Thank you!
I'm trying out http requests to download a pdf file from google docs using google document list API and OAuth 1.0. I'm not using any external api for oauth or google docs.
Following the documentation, I obtained download URL for the pdf which works fine when placed in a browser.
According to documentation I should send a request that looks like this:
GET https://doc-04-20-docs.googleusercontent.com/docs/secure/m7an0emtau/WJm12345/YzI2Y2ExYWVm?h=16655626&e=download&gd=true
However, the download URL has something funny going on with the paremeters, it looks like this:
https://doc-00-00-docs.googleusercontent.com/docs/securesc/5ud8e...tMzQ?h=15287211447292764666&\;e=download&\;gd=true
(in the url '&\;' is actually without '\' but I put it here in the post to avoid escaping it as '&').
So what is the case here; do I have 3 parameters h,e,gd or do I have one parameter h with value 15287211447292764666&ae=download&gd=true, or maybe I have the following 3 param-value pairs: h = 15287211447292764666, amp;e = download, amp;gd = true (which I think is the case and it seems like a bug)?
In order to form a proper http request I need to know exectly what are the parameters names and values, however the download URL I have is confusing. Moreover, if the params names are h,amp;e and amp;gd, is the request containing those params valid for obtaining file content (if not it seems like a bug).
I didn't have problems downloading and uploading documents (msword docs) and my scope for downloading a file is correct.
I experimented with different requests a lot. When I treat the 3 parameters (h,e,gd) separetaly I get Unauthorized 401. If I assume that I have only one parameter - h with value 15287211447292764666&ae=download&gd=true I get 500 Internal Server Error (google api states: 'An unexpected error has occurred in the API.','If the problem persists, please post in the forum.').
If I don't put any paremeters at all or I put 3 parameters -h,amp;e,amp;gd, I get 302 Found. I tried following the redirections sending more requests but I still couldn't get the actual pdf content. I also experimented in OAuth Playground and it seems it's not working as it's supposed to neither. Sending get request in OAuth with the download URL responds with 302 Found instead of responding with the PDF content.
What is going on here? How can I obtain the pdf content in a response? Please help.
I have experimented same issue with oAuth2 (error 401).
Solved by inserting the oAuth2 token in request header and not in URL.
I have replaced &access_token=<token> in the URL by setRequestHeader("Authorization", "Bearer <token>" )
I am working on a flex application which communicates with a Rails backened.
When i request for some data, It sends back xml feed.
In some cases, if given parameters are not valid, then rails return an error feed with status code = 422 as following
email is wrong
But I dont get this feed in FaultEvent of Flex, How could i read error feed?
Thanks
Are you getting the result in ResultEvent in such cases? I am not sure for what all HTTP error codes FaultEvent will get invoke(I know only it goes for 404 and 500). May be its still going to ResultEvent as a valid result!
You can use HTTPService instead of URLLoader.
Flex HTTP results will not include the actual underlying HTTP response codes. It just doesn't work. (TM)