I have recently moved my site over to a new host (VPS). Periodically when I browse to the site I see a string of data, such as --
��[�Rۺ� O���)��q.J�!l(�[�m2��8 �aɹ�ݙ�� ϓ�%�N���v�Mv'�-i]>----It�M�|�����p�B7���}����i��:�k\����~�Q.�մ��*BJ�soG���~�_ȸ���a���<�#HS}���\��R]ݕ ����\�\��A�caǬ(�QP��NE�� ����>����6����0��ZԹG��+oc���p�1]״�(�训錽oc�Z��%�Hwr��[����EX���G*o9p�� vڵ �HHW�C#{eT���#� =*U�%�Cz��J��jIAO"��q}�Q`U"�I�f��k�J�r����G��qcs[k�P��%�Bml�s��욃mh9"�1w�T��~�am���!+�p�K+i =J�t-%�O ީpU6l�P;�U��#T� 6l<�� �e���q�. N�E����Z�E���ў�1��.�j� ����4�/��pU&D�o�xW+��G��ہ����;�w���_ur�k�E�,#��Ġ���Д��.�A��t��/ ����\���{Ͼ=9�w�{jf��i������̹�C~�[���gc�;^��6��5�z)K��S��s����ku��l�7j�뇆�ɯr�6��WŜ�:7w��y}�e�1��;���������<�e|yV+l]7O�5u[?����V�`ҟ=�:\�^�}��_iY�����}9=���WEZ.l�ʞ�l���c�e+_82//h�!��V��o7g[��W�L��UU�����NtqG� b��b�]�i:�7�B��c���8wyUk}��k�����qʭ�I�xx�;��3]�Tw� ~-�w���m�o��K�4�ꃛsvsܻy,5���Qw�;|0�?�lY탃�˞�أ��p�g��枞36k����c��Z�{[�˂xO������K~|>�φ�֧Vùj�U�-/��� ѦL� �2}���� �R��-�b���ph� %?�֧��Q�]���B��kw�N���z#���e��F�S��~}�zUE�j�i�T�a��~'^3^�:k_�&ds{���6�r�����l.W�R����IZ�� z��ȈD�Z;pt)��:�S(31�2\=�!M��>�XD�֔��;gD_��IoDfCG�F�D�a�G!Μl 9ա��SX��N�T� �,��$<�j�kl~�Tp܅����X��>�p ���v�FڮO�L�����k��V��.����:�wo��P=��o���?��W��Q<8������uݏ��HkQ��?��1��SiaF.0�(;�7M�P�n4��4�x�IR�<3�����6rF�Ɇx���~ӄ=}pߑ�u�)�K�&���ړH�<+��&̎ y���& � D��I��Kܦ(?���(|� w6-c���N��ڞF�b۶|��!�%�l�|"���j6���-^A���X�w�pڦ�X��(��������f�h�ؐqb'όtI#<�-�="" r#d��w���w,okd�+nd�c�����="" 1k�w="" [�="" u�a��="" �v="" 4i;4�c���0���yo�]��z="">,��!��C��Q��>���zF;�:=�=���~�x�} c��Z!�tp��x�ǽ�oc���3�m��!Q~��ĵF�$̌N�ʂ��C�[���BIQ��}���q�j��I�:PR5H���RGC#/FSgU�V�Xz��$4�����~�w�5�n=ջ��}s|�]n�t/B��NC"�k��FE����GK� �#�>'��<���1b^Y�"p|Ԫ)HޞT���*:���W! �����]���f`6+["<�t'��d�h+��ܤ~q����Ȥ(="">N-��X�++R��`ŷcj2hӌ�OyGg JhZB�A*XY/+iث+�s%�1u��>2e���h(z�c#�N��A�e��}�{V�4��lR�}�jc�#��nGm�3���#�0"?�-�F5�#�"&?�&�Ԙ��d�W�����2���j�f�f�H� �b4zSs�w�B�$�3��:K��R�Ի*'�Ri3!�(F�skd+��]�R�u��'�J�|�eѯ��N�I1�:�����&j���;��Ҽ�eQ=Җ�zf�� �� �-�P�D�#n�h?�x�Q#��(&6�|���G�w�#�+;���*ͩL���:��Wr�I�f�aR�A�T#Oy��.R�#��!�.0�c�Z�Nbc�zU!z��eղ�e�W�X3�"�ӎ��Rh�ȇ~��܈��;:�s�t�Qڏ��Q�P��KE�������B��Ɖ�s�л�� V,�Z.$�W���K#�OBҨM���|�:��rC�'��.7i��F�s�T/R��"�^�����J�֞(�v��fC�8G��l/t���ƣ�ľ�����>�`�s�4�K�G5�a�{��D����<ԯ���K�O��sD�$�c�_d�7���{� �� �W�w>.-=�F�Z�W�;F��$�e-4�m�y����<�IĽv���C�c���M�v��O6l���ml�[َ�D��c�<د����p0�3�8���O�/�:�Ih���!<������c��ny=��9��jeee��|ֱ|'�8D|y zz�і��3�u��u����b�y�*jG������< ��Z4���v��c��I{�h�FB�����â���'sŴ,a����G5�%l��( #1�k3s���j�r͔L-���._ƶG� X<��C��y�߷DΒ��I�}#Y��#dJ����NxcJ���~���V���&��7"
Here is the header as seen in Firebug --
Response Headers
Cache-Control no-cache, must-revalidate, post-check=0, pre-check=0
Connection close
Content-Type text/html; charset=utf-8
Date Sun, 04 Mar 2012 17:43:55 GMT
Etag "1330883035"
Expires Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified Sun, 04 Mar 2012 17:43:55 +0000
Server Apache/2.2.15 (CentOS)
**Transfer-Encoding chunked**
Vary Cookie
**X-Drupal-Cache MISS**
X-Generator Drupal 7 (http://drupal.org)
X-Powered-By PHP/5.3.3
Request Headers
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie __utma=162614910.1950275874.1326920494.1328073656.1328201988.4; __utmz=162614910.1326920494.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); __utma=142212834.1354567149.1327029129.1329407721.1330721009.6; __utmz=142212834.1327029129.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESS9cac77df1227f59f7a6f85a5ce79093a=g49om8oemr508pag9qni57oj53; __qca=P0-556640154-1328084416252; DRUPAL_UID=1
Host domain.com
User-Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
I'm at a loss here. I have searched for a solution with no luck. I hope I have provided enough information to diagnose this problem. Thanks.
UPDATE: when the header when display is normal (differences in **)--
Response Headers
**Cache-Control public, max-age=0**
Connection close
Content-Encoding gzip
Content-Length 3354
Content-Type text/html; charset=utf-8
Date Sun, 04 Mar 2012 19:24:29 GMT
Etag "1330883035-1"
Expires Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified Sun, 04 Mar 2012 17:43:55 +0000
Server Apache/2.2.15 (CentOS)
**Vary Cookie,Accept-Encoding**
**X-Drupal-Cache HIT**
X-Generator Drupal 7 (http://drupal.org)
X-Powered-By PHP/5.3.3
Request Headers
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie __utma=162614910.1950275874.1326920494.1330883191.1330887981.6; __utmz=162614910.1330887981.6.2.utmcsr=xz.allanbendy.com|utmccn=(referral)|utmcmd=referral|utmcct=/; __utma=142212834.1354567149.1327029129.1329407721.1330721009.6; __utmz=142212834.1327029129.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESS9cac77df1227f59f7a6f85a5ce79093a=g49om8oemr508pag9qni57oj53; __qca=P0-556640154-1328084416252; DRUPAL_UID=1; has_js=1; __utmc=162614910; __utmb=162614910.6.10.1330887981
Host domain.com
**If-None-Match "1330883035-1"**
User-Agent Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
The Content-Type header here clearly says that the data is UTF-8 encoded. If it is not, then mess like the one in the example is to be expected, if there are any non-Ascii characters in the document.
Find out the actual encoding. If possible, convert the file to UTF-8 encoding. If that is not possible for some reason, check out whether you can affect the HTTP headers using a .htaccess file. Failing even that, you need to change all non-Ascii characters to character references or entity references.
It looks like the answer is compressed with gzip but the header "Content-Encoding: gzip" is missing in the first case.
Related
I'm quite new to caching so I've been trying some different ways of caching my website. I've settled on HTTP caching now, because it's the most appropriate with sporadic updates and lots of users perusing the same pages over and over.
I'm struggling to get it working however. The site shows different content based on whether you're logged in or not, so I have to invalidate cache based on current_user as well as the latest update on the collection of models.
If I look in chrome inspect the ETag and the modified_since are the same, but the server returns a 200 instead of a 304. My code works in development environment, so I'm lost in how to troubleshoot it. Also a different page that only invalidates based on the collection of models (similar on latest update), does work as expected.
Code from the controller:
def index
...#some code
# HTTTP caching:
last_mod = #scraps.order("updated_at").last.updated_at
user = current_user ? current_user.id : 0
fresh_when etag: user.to_s, last_modified: last_mod, public: false
end
Output from chrome inspect
Response Headers:
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Status: 200 OK
Last-Modified: Sun, 23 Jul 2017 20:40:53 GMT
Cache-Control: max-age=0, private, must-revalidate
ETag: W/"6e92592bdb6c3cf610020e2b076e64b4"
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Runtime: 3.187090
X-Request-Id: c698c0c6-8a0d-44ba-8ca9-3f162b766478
Date: Mon, 24 Jul 2017 14:49:38 GMT
Set-Cookie: ... [edited out]; path=/; HttpOnly
X-Powered-By: Phusion Passenger 5.0.30
Server: nginx/1.10.1 + Phusion Passenger 5.0.30
Content-Encoding: gzip
Request Headers:
GET /scraps?page=3&price_max=100&price_min=0&producer=silk+scraps HTTP/1.1
Host: www.picture-scraps.com
Connection: keep-alive
Accept: text/html, application/xhtml+xml, application/xml
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
X-XHR-Referer: https://www.picture-scraps.com/scraps?page=4&price_max=100&price_min=0&producer=silk+scraps
Referer: https://www.picture-scraps.com/scraps?page=4&price_max=100&price_min=0&producer=silk+scraps
Accept-Encoding: gzip, deflate, br
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4,af;q=0.2
Cookie: ... [edited out]
If-None-Match: W/"6e92592bdb6c3cf610020e2b076e64b4"
If-Modified-Since: Sun, 23 Jul 2017 20:40:53 GMT
I can imagine some additional information is needed, so please request and I'll add to the question.
Figured it out today. This post provides the answer. I saw the server used weak etags while in the dev environment strong etags were used. The latter is as expected as weak etags were only introduced from rails 5 forward.
If you use Nginx with rails 4 you might experience the same problem. Installing rails_weak_etags gem solved it for me.
How do you configure nginx to return a valid Content-Length header when responding to a HTTP HEAD request? Currently my server returns this:
curl --head http://example.com/myfile.xml
HTTP/1.1 200 OK
Date: Tue, 23 Aug 2016 13:49:46 GMT
Content-Type: text/xml
Set-Cookie: __cfduid=da2daeaa59809916192f7ac0645d3a3e91471960186; expires=Wed, 23-Aug-17 13:49:46 GMT; path=/; domain=.example.com; HttpOnly
Last-Modified: Mon, 22 Aug 2016 16:20:26 GMT
Vary: Accept-Encoding
ETag: W/"57bb264a-5442b26a"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Access-Control-Allow-Origin: *
Access-Control-Request-Method: *
Cache-Control: public
Server: cloudflare-nginx
CF-RAY: 9dac40-LHR
X-Cache: MISS from Squid
Via: 1.1 Squid (squid/3.2.14)
I must send the Content-Length header with the response to HEAD (if I don't, the service that checks that URL will never see that the file was changed, and will not download the new version). How do you set it up?
I might be wrong, but nginx probably isn't showing the content-length because of nature of dynamic content generated by Rails's application server.
You can ask Rails to send that in response header. In your Rails application's config/application.rb add the following middleware:
config.middleware.use "Rack::ContentLength"
This should return the content-length header in response.
I've got a strange issue and I'm running out of ideas. In my Rails(4.2)-App I'm using the fresh_when-Method to invalidate client caches for my blog pages:
def show
#post = Post.find(params[:id])
fresh_when #post
end
With curl everything works out fine, sending the matching Etag gives me a 304 response:
celmare$ curl -i -H 'If-None-Match: "3b4dd96aac692c03ce623db459c9cef2"' https://grosse.io/blog
Response:
HTTP/1.1 304 Not Modified
Connection: keep-alive
Status: 304 Not Modified
Last-Modified: Sun, 04 Oct 2015 10:41:08 GMT
Cache-Control: max-age=0, private, must-revalidate
Strict-Transport-Security: max-age=31536000
X-XSS-Protection: 1; mode=block
X-Request-Id: 68a0ecd2-3fac-4004-ac1e-fd6d14780f61
ETag: "3b4dd96aac692c03ce623db459c9cef2"
X-Frame-Options: SAMEORIGIN
X-Runtime: 0.006207
X-Content-Type-Options: nosniff
Date: Thu, 29 Oct 2015 13:17:02 GMT
X-Powered-By: Phusion Passenger 5.0.15
Server: nginx/1.8.0 + Phusion Passenger 5.0.15
When I open the page in the browser (e.g. Chrome Version 47.0.2526.35 beta (64-bit)) I always get 200 although the Etag still matches:
Request headers:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate, sdch
Accept-Language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Cookie:_gat=1; _syscfg_net_v2_session=eGFlYk83Z0kwUE9IYmtUVHg1Z1ppbHF2eFBrUitiTDBsRG1Kbml2bW8vQVZ6YW4xM0ZuRTNOS0w2VmVLM1ZaN0czZno3N0Y2MWpiUWNjQUV0YkVlaXhCZUJyZlJWWEVIZVpPclFaaHZxdFNncjNBVVg3MFR2SE0yWDRUaklsSlRMbmw4OVQrQmlDRHBIbmRSMS9VVml3PT0tLTYvUGdURTRaRjNXSU9WOTdOY1F3OEE9PQ%3D%3D--3bafbda7d522c61cd9fd04898c2c6a4bac06131b; _ga=GA1.2.235147781.1445350582
Host:grosse.io
If-Modified-Since:Sun, 04 Oct 2015 10:41:08 GMT
If-None-Match:W/"3b4dd96aac692c03ce623db459c9cef2"
Referer:https://grosse.io/blog
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.35 Safari/537.36
Response headers:
Cache-Control:max-age=0, private, must-revalidate
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html; charset=utf-8
Date:Thu, 29 Oct 2015 13:16:48 GMT
ETag:W/"3b4dd96aac692c03ce623db459c9cef2"
Last-Modified:Sun, 04 Oct 2015 10:41:08 GMT
Server:nginx/1.8.0 + Phusion Passenger 5.0.15
Set-Cookie:_syscfg_net_v2_session=MUtjWlQyY1ZFZnF2TzlvTDJkdnpmMDhqVmhoVld5YkJDdHl5NUtIdXJTY1VZQ1AzV1NVMjF1alFDSE9NKzliOGhzcmc4S3FLajRmNGFZUjltQzdPNDg4SW51aUxGU2xDd0FxVi82UFZneE5YU1FnTjJVSFhpL3RCQkNYdjlFVTlyZVRRU0ZPdG83UFNVbjVyckJmZ0R3PT0tLXh4Zzg0cjhBSTZKbVpkayttanpwUFE9PQ%3D%3D--dc404af2428a17085bea4b40a3f4f0fc6ef01e50; path=/; secure; HttpOnly
Status:200 OK
Strict-Transport-Security:max-age=31536000
Transfer-Encoding:chunked
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
X-Powered-By:Phusion Passenger 5.0.15
X-Request-Id:0633095f-b95d-4339-8e62-8b15683c2d8c
X-Runtime:0.034172
X-XSS-Protection:1; mode=block
And on top: In my local env it's working with the same browser. I can hardly imagine that it is a NGINX thing because the everything is configured very defaulty. Could it be something with HTTPS?
Any ideas? Thanks in advance.
Ok, I found the cause. It's a problem with NGINXs gzip compression in combination with weak "W/" Etags. Will try upgrading NGINX or using a Patch.
Adding etag on; after gzip on; in nginx.conf fixed the problem from 1.7.4 and newer.
Website making request to clients1.google.com/ocsp with POST.
Its making 2 requests in the same page. and both requests are identical.
What its for? and why it make same 2 requests?
Request Header:-
(Request-Line) POST /ocsp HTTP/1.1
Host clients1.google.com
User-Agent Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language en-US,en;q=0.5
Accept-Encoding gzip, deflate
Content-Length 107
Content-Type application/ocsp-request
Connection keep-alive
Response Header :
(Status-Line) HTTP/1.1 200 OK
Content-Type application/ocsp-response
Date Sat, 21 Sep 2013 14:08:40 GMT
Expires Wed, 25 Sep 2013 14:08:40 GMT
Cache-Control public, max-age=345600
Server ocsp_responder
Content-Length 463
X-XSS-Protection 1; mode=block
X-Frame-Options SAMEORIGIN
Alternate-Protocol 80:quic
It's probably checking certificate status of various certs in the browser.
I develop an internal web application for my company. It is used by our field technicians, all of whom carry a BlackBerry 8330 running 4.5. I would consider myself fortunate to have such a consistent target platform, if it wasn't BB 4.5...
I've noticed a lot of request overhead in loading the site, and know that if only my CSS resources were cached, the load time would be cut dramatically. The BB always requests a full copy of the CSS file, no matter if the Expires, Cache-Control, or Last-Modified headers are set. It doesn't hit its cache, it doesn't send an If-Modified-Since, nothing.
Anyone run into this or know what I can do to workaround it? I'd really like to avoid inlining my CSS if I don't have to.
EDIT: I just noticed that it is always requesting the page twice. Below are diffs between to 2 requests
GET /css/bb.css HTTP/1.1 |GET /css/bb.css HTTP/1.1
User-Agent: BlackBerry8330/4.5.0.77|User-Agent: BlackBerry8330/4.5.0.77
profile: http://www.blackberry.net/|profile: http://www.blackberry.net/
-----------------------------------|Accept: application/vnd.rim.html,te
-----------------------------------|Connection: close
Referer: http://10.7.2.167/page.php|Referer: http://10.7.2.167/page.php
-----------------------------------|Accept-Charset: ISO-8859-1,UTF-8,US
Host: 10.7.2.167 |Host: 10.7.2.167
-----------------------------------|Accept-Language: en-US,en;q=0.5
-----------------------------------|x-wap-profile: "http://www.blackber
Cookie: PHPSESSID=xxxxxxxx; token=x|Cookie: PHPSESSID=xxxxxxxx; token=x
-----------------------------------|Via: MDS_5.0.0.86
|
HTTP/1.1 200 OK |HTTP/1.1 200 OK
Date: Sun, 05 Sep 2010 09:34:52 GMT|Date: Sun, 05 Sep 2010 09:34:54 GMT
Server: Apache/2.2.15 (Debian) |Server: Apache/2.2.15 (Debian)
Last-Modified: Sat, 04 Sep 2010 02:|Last-Modified: Sat, 04 Sep 2010 02:
ETag: "10426-d64-48f65ab39bf80" |ETag: "10426-d64-48f65ab39bf80"
Accept-Ranges: bytes |Accept-Ranges: bytes
Content-Length: 3428 |Content-Length: 3428
Cache-Control: max-age=12960000 |Cache-Control: max-age=12960000
Expires: Wed, 02 Feb 2011 09:34:52 |Expires: Wed, 02 Feb 2011 09:34:54
Vary: Accept-Encoding |Vary: Accept-Encoding
-----------------------------------|Connection: close
Content-Type: text/css |Content-Type: text/css