I'm running rails application with Puma server and every response has some hex numbers at the start (looks like body length) and the end (always zero) of each response.
HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-UA-Compatible: chrome=1
Content-Type: application/json; charset=utf-8
ETag: "8e6d795d26a3839c23c0b28f7b2e3c08"
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: [skip]; path=/; HttpOnly
X-Request-Id: 741cdbde-39a8-4b99-9af7-15a7e65ff1ac
X-Runtime: 0.202731
Transfer-Encoding: chunked
f <-- THIS
{"response":{}}
0 <-- AND THIS
What is this?
That's the way chunked transfer encoding works. It sends the body in pieces, preceded by a line containing the length of the piece in hexadecimal. The end of the body is indicated by a chunk with 0 length.
So what you're showing is that the first chunk is 15 characters long, and there's no second chunk.
A proper HTTP client or API should strip these chunk headers out, and just return the contents.
See Wikipedia for more details, or RFC 2616 for the official spec.
Related
Our Rails web app generates PDFs using wkhtmltopdf and sends them to the client. This works in every web browser we've tested it with except Edge.
We've tried rendering the response in a couple of different ways, this is how it was originally:
kit = PDFKit.new(#html_content)
render text: kit.to_pdf, content_type: 'application/pdf'
This opens the PDF viewer with the PDF displaying correctly in every browser that we tested with except Edge where the browser displays: Something's keeping this PDF from opening.
In our application logs, there is the POST request which is the form submission and I can see our app send the pdf file response, then there are subsequent GET requests to the form submission url which error because it's not expecting any GET request to that url. I've no idea what's going on here.
The response headers for the request are:
Cache-Control: max-age=0, private, must-revalidate
Connection: Keep-Alive
Content-Length: 34865
Content-Type: application/pdf; charset=utf-8
Date: Thu, 18 Jun 2015 14:35:30 GMT
Etag: "4baf297d1866339e60e8e893300909a0"
Server: WEBrick/1.3.1 (Ruby/2.0.0/2013-06-27)
Set-Cookie: _APP_session=<long cookie>; path=/; HttpOnly
X-Request-Id: 617580a8-4d7d-43c4-8e49-aeaeafba7b79
X-Runtime: 21.868098
X-XSS-Protection: 1; mode=block
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-ua-compatible: chrome=1
I have also tried using send_data like this:
send_data kit.to_pdf, type: 'application/pdf', disposition: 'inline'
Which results in the following response headers but ultimately the same problem:
Cache-Control: private
Connection: Keep-Alive
Content-Disposition: inline
Content-Length: 34866
Content-Transfer-Encoding: binary
Content-Type: application/pdf
Date: Thu, 18 Jun 2015 14:39:42 GMT
Etag: "11db49f1a26444a38fa2b51f3c3336ed"
Server: WEBrick/1.3.1 (Ruby/2.0.0/2013-06-27)
Set-Cookie: _APP_session=<long cookie>; path=/; HttpOnly
X-Request-Id: 501d9832-b07e-4764-8ecc-f1c1e9a6421e
X-Runtime: 7.054236
X-XSS-Protection: 1; mode=block
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-ua-compatible: chrome=1
If I remove the Content-Disposition: inline header from the above it brings up the save file prompt and downloading the file works fine. We need it to load in the browser window though.
I don't believe it to be a duplicate of this question because it works in IE 9, 10 and 11 and is only a problem with Edge.
We've been having what sounds like the same problem with PDF reports that we generate on the server and dispatch inline - the new tab that opens for the viewer appears to re-issue a request for the content instead of displaying the content from the response. Since we use a synthetic one-time-use path (for largely historical reasons to ensure a new version of the report is fetched), the report isn't actually there for the new tab's GET request.
Since we're using 20.10240, I'm not convinced it was actually fixed in 10158.
As with the OP, this seems to apply only to "Content-Disposition: inline"; if we use "attachment" instead, a temporary file is saved locally and the temporary file is opened in the viewer.
It was a bug but Microsoft have fixed it in build 10158! :)
I'm building an API on Rails version 4.1.7/Nginx that responds to request from an iOS app. We're seeing some weird caching on the client and we think it has something to do with a small difference in the response that Rails is sending back. My questions...
1) I want to understand why, for the exact same request (with only the Authorization header value changed), Rails sends back transfer-encoding: chunked sometimes and Content-Length: <number> sometimes? I thought that maybe it had something to do with the response size, but in the example responses whose headers I've pasted below, the data returned in the body is EXACTLY the same.
2) Is there a way to force it to use Content-Length? We think that that will fix our caching issues in our iOS app.
Response #1
HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Wed, 18 Mar 2015 00:59:31 GMT
ETag: "86f277ea63295460d4f3bed9a073eaa2"
Server: nginx/1.6.2
Status: 200 OK
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: dd36f139-1986-4da6-9645-4438d41e74b0
X-Runtime: 0.123865
X-XSS-Protection: 1; mode=block
transfer-encoding: chunked
Connection: keep-alive
Request #2
HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Wed, 18 Mar 2015 00:59:36 GMT
ETag: "86f277ea63295460d4f3bed9a073eaa2"
Server: nginx/1.6.2
Status: 200 OK
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 0cfd7705-157b-41b5-aa36-739bc6f8302e
X-Runtime: 0.092672
X-XSS-Protection: 1; mode=block
Content-Length: 2234
Connection: keep-alive
Both responses are valid according to HTTP 1.1, so you need to fix your client code that it can handle both. It is a bad idea to try to fix the server so that that it behave in a way that it does not trigger a bug in the client.
The next version of nginx may behave differently, you users may even have proxies that change the transfer, maybe only when they do roaming and use a different provider.
If you want to do some finger-printing on the header, the ETag-header may help you, as the ETag should stay constant when the content of the response is not changed, regardless of the transfer.
The server typically sends in chunks when it calls a dynamic page, because it then does not need to create a buffer for the whole page and wait till all of the page is generated.
The server often send the response in one go if it has the buffer already, for example because it is in cache or the content is on a file and is not to big. Sending in one go is more efficient, on the other hand, an extra copy of the data to buffer the output needs more memory and is less efficient. So the server may even decide this according to the available memory.
I am viewing an HTML page in a standard iOS UIWebView. Inside this page I have a standard YouTube embed, something like this:
<iframe id="video-play" width="624" height="350" src="http://www.youtube.com/embed/hBLf_N-T0vI" allowfullscreen class="hide fade"></iframe>
This works fine in all cases on the browser, and in most cases in a UIWebView. But for some videos I get the (I think misleading) message:
"The uploader has not made this video available in your country."
The problem is somewhere in the get_video_info call, and related to the fact that from a UIWebView it seems the eurl paramter is set to "unknown"
So this request works from within a UIWebView:
http://www.youtube.com/get_video_info?html5=1&video_id=hBLf_N-T0vI&eurl=unknown&ps=native&el=embedded&hl=en_GB
This fails, with errorcode 150 and the error message I mentioned above (it will work fine in a browser so use the raw HTTP request below)
http://www.youtube.com/get_video_info?html5=1&video_id=DldaCQby3j4&eurl=unknown&ps=native&el=embedded&hl=en_GB
If I change the eurl=unkown into eurl=http://rubbish.com/ then it works again:
http://www.youtube.com/get_video_info?html5=1&video_id=DldaCQby3j4&eurl=http://rubbish.com&ps=native&el=embedded&hl=en_GB
I've tried to look at the properties of the videos that are failing but can't figure out what is causing the difference between the good ones and the bad ones. I've looked at embed settings, privacy and tracking settings and anything else I can think of.
My other option is to figure out how to add an eurl paramter to the request.
Help!
Jon
If anyone wants the full HTTP requests/response:
BAD REQUEST
GET /get_video_info?html5=1&video_id=DldaCQby3j4&eurl=unknown&ps=native&el=embedded&hl=en_GB HTTP/1.1
Host: www.youtube.com
BAD RESPONSE
HTTP/1.1 200 OK
Date: Fri, 23 Nov 2012 15:42:47 GMT
Server: gwiseguy/2.0
X-Content-Type-Options: nosniff
Access-Control-Allow-Origin: *
Set-Cookie: use_hitbox=d5c5516c3379125f43aa0d495d100d6ddAEAAAAw; path=/; domain=.youtube.com
Set-Cookie: VISITOR_INFO1_LIVE=fH943IGDAFc; path=/; domain=.youtube.com; expires=Sun, 21-Jul-2013 15:42:47 GMT
Expires: Tue, 27 Apr 1971 19:44:06 EST
Cache-Control: no-cache
P3P: CP="This is not a P3P policy! See //support.google.com/accounts/bin/answer.py?answer=151657&hl=en-US for more info."
Content-Type: application/x-www-form-urlencoded
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Transfer-Encoding: chunked
status=fail&errorcode=150&reason=The+uploader+has+not+made+this+video+available+in+your+country.&storyboard_spec=http%3A%2F%2Fi1.ytimg.com%2Fsb%2FDldaCQby3j4%2Fstoryboard3_L%24L%2F%24N.jpg%7C48%2327%23100%2310%2310%230%23default%23w3GCNZfS0BvXcAQIB1BBnUJRlrY%7C80%2345%23112%2310%2310%232000%23M%24M%23VDmT14lMI4g0sdAxTbIujmEIkkI%7C160%2390%23112%235%235%232000%23M%24M%23c1uKcYcKallke_fsXgoUOkSHnwA&errordetail=0
GOOD REQUEST
GET /get_video_info?html5=1&video_id=DldaCQby3j4&eurl=http%3A%2F%2Frubbish.com%2F&ps=native&el=embedded&hl=en_GB HTTP/1.1
Host: www.youtube.com
GOOD RESPONSE
HTTP/1.1 200 OK
Date: Fri, 23 Nov 2012 15:42:34 GMT
Server: gwiseguy/2.0
X-Content-Type-Options: nosniff
Access-Control-Allow-Origin: *
Set-Cookie: use_hitbox=d5c5516c3379125f43aa0d495d100d6ddAEAAAAw; path=/; domain=.youtube.com
Set-Cookie: VISITOR_INFO1_LIVE=zG2n4ZwVAdY; path=/; domain=.youtube.com; expires=Sun, 21-Jul-2013 15:42:33 GMT
Expires: Tue, 27 Apr 1971 19:44:06 EST
Cache-Control: no-cache
P3P: CP="This is not a P3P policy! See //support.google.com/accounts/bin/answer.py?answer=151657&hl=en-US for more info."
Content-Type: application/x-www-form-urlencoded
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Transfer-Encoding: chunked
account_playback_token .... (chopped for brevity).
Could something to do with "claimed" videos. A contact at YouTube got back to us:
"The videos [that worked] are claimed and the others aren't, so
something in the claiming process is causing the issue."
In trying to upload a scorm package via the REST API the upload is not working.
PUT /d2l/api/lr/(D2LVERSION: version)/objects/
http://docs.valence.desire2learn.com/res/lor.html
We are always refused based on 403 permission.
Are there any settings I should look at? Also looking for a trace of this working.
As long as the account you are authenticated with can perform the function in the web UI it is expected that the API will also function.
If you are getting keys back from the login process but this call is failing you could try a basic call from the sample such as "whoami". If that works your keys are correct.
If the keys are correct and some GET calls are working, you may need to pass a different method to the signing call in the sdk. (The method is part of the signature).
Here is an example of a working trace. 403 errors often have a body that will provide additional information. Note the security parameters are: x_a,x_b,x_c,x_d,x_t parameters on the call.
PUT http://smihai-7:44459/d2l/api/LR/1.0/objects/?repositoryId=1&x_t=1339183935&x_a=L2Hd9WvDTcyiyu5n2AEgpg&x_c=tfJFhSUaczOeOGqDFPXPq8NSBPj2sOhz4U3RacqWRMY&x_b=TestToken&x_d=BEl7kdCcrjenkpBt9ri5dkt4bdEgCo6xfZDWIpkKctA HTTP/1.1
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp 101.3.0.0
Host: smihai-7:44459
Content-Type: multipart/form-data; boundary=-----------------------------28947758029299
Content-Length: 1203
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
-------------------------------28947758029299
Content-Disposition: form-data; name="Resource"; filename="Hello World Module.zip"
Content-Type: application/zip
PK???u?h?]G?g???????????Hello World Topic.htmlM??
?#??A??wwo?xH+???h????b
[o+ ????oD??RYW9u??????}J,?q
d2?[!%E|Crj?Wo{34??Xg??s???L?3??+??/?????*??W?W3?Fyb w?>?cR?Zrf?*???b??PK???u?h?N2?T??n?????imsmanifest.xml?V??0?#?Q??m#TI?.b ?XF??5?????k,?$~???6I??Vt??s?=g?x???%7)??J?i4 P?f\???????????DR?W`]?(?WL???g??d???s?,.&i?q????r??jT?kI??E?C?fsmd6-?q??G? .?f?i??4???!??v\???o?7$\jH%
??K??~P??m?`E?¥a)?C????v???6????#???U~?????x???[fe?.?3????~W???;B_?,???V#B?HE???:??q?e???s??_E? 1wK??<R????T??.9YE??SkP`?????*UT??3???j??#'??#2?;?e_c?#g.????}?p?>?c??????\?
????~,u????\s?M*L?U???E??
??????Kzp\E?X#?%\p???\??R
X-????%??C??????7?|??/&?=???h????l?\?\???????P???s??))??Td??K?????{?Y?+????v?gTN??h$?
?E'E?aB?UD????PK-????u?h?]G?g?????????????????????????Hello World Topic.htmlPK-????u?h?N2?T??n???????????????????imsmanifest.xmlPK??????????E????
-------------------------------28947758029299--
And the response is like this:
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Length: 69
Content-Type: application/json; charset=UTF-8
Expires: -1
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
X-XSS-Protection: 0
Date: Fri, 08 Jun 2012 19:32:22 GMT
{"IdentId":4,"Version":1,"ExecutionMessage":null,"ExecutionStatus":0}
By default, Asp.Net MVC (at least my setup) seems to send a header with the value
Cache-Control: private, s-maxage=0
I need to remove the s-maxage=0 part. The reason is that IE6 seems to handle the content-disposition: attachment header wrong if this header is present.
The end effect I observe is that if the user chooses to Open the downloaded file instead of saving it, it is not actually saved. The difference between the working non-mvc version and the non-working mvc version is just this header.
Working response:
HTTP/1.1 200 OK
Server: ASP.NET Development Server/9.0.0.0
Date: Thu, 10 Feb 2011 19:35:47 GMT
X-AspNet-Version: 2.0.50727
Content-Disposition: attachment; filename=results.txt
Cache-Control: private
Content-Type: text/plain; charset=iso-8859-1
Content-Length: 210
Connection: Close
<<DATA>>
Non-working response:
HTTP/1.1 200 OK
Server: ASP.NET Development Server/9.0.0.0
Date: Thu, 10 Feb 2011 20:24:04 GMT
X-AspNet-Version: 2.0.50727
X-AspNetMvc-Version: 1.0
Content-Disposition: attachment; filename=results.txt
Cache-Control: private, s-maxage=0
Content-Type: text/plain
Content-Length: 90
Connection: Close
<<DATA>>