There appears to be an issue with Twitter's Search API and cookies. The
following yields no results found:
GET http://search.twitter.com/search.json?q=%23tweetz HTTP/1.1
Accept: */*
Accept-Language: en-us
Content-Type: application/x-www-form-urlencoded
UA-CPU: AMD64
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Win64;
x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET
CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E)
Host: search.twitter.com
Connection: Keep-Alive
Cookie: k=68.40.12.103.1280961938551221; guest_id=128035711958013764;
lang=en;
_twitter_sess=BAh7CDoPY3JlYXRlZF9hdGwrCJWulT8qASIKZmxhc2hJQzonQWN0aW9uQ29u
%250AdHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsABjoKQHVzZWR7ADoHaWQiJWQ3%250AMWIxNzJlZmUzNThmM2Y1NGRkYzYwZTRjZmM1YmNh--0871d612503b63999c65c8c6abefb83f6bed9643
The response is:
HTTP/1.0 200 OK
Date: Thu, 05 Aug 2010 00:32:14 GMT
Server: hi
Status: 200 OK
X-Served-From: sjc1r085
X-Runtime: 0.02556
Content-Type: application/json; charset=utf-8
X-Timeline-Cache-Hit: Hit
X-Served-By: sjc1v012.prod.twitter.com
Cache-Control: max-age=15, must-revalidate, max-age=300
Expires: Thu, 05 Aug 2010 00:37:14 GMT
Content-Length: 251
Vary: Accept-Encoding
X-Varnish: 2325786061
Age: 0
Via: 1.1 varnish
X-Cache-Svr: sjc1v012.prod.twitter.com
X-Cache: MISS
Connection: close
{"results":[],"max_id":20340216330,"since_id":
20294384707,"refresh_url":"?since_id=20340216330&q=
%23tweetz","results_per_page":15,"page":1,"completed_in":
0.017987,"warning":"adjusted since_id to 20294384707 due to temporary
error","query":"%23tweetz"}
If I issue the same request without the cookies (used Fiddler), the response contains matching tweets.
I can't remove the cookies since they're server issued (I'm running in
a browser). Any thoughts?
This is because you logged into twitter in the same browser session.
Because of that you received a cookie with a default language filter "lang=en"
The fix (or workaround, whatever you prefer) is to overrule the default filter with lang=all like:
http://search.twitter.com/search.json?lang=all&q=%23tweetz
Related
I'm quite new to caching so I've been trying some different ways of caching my website. I've settled on HTTP caching now, because it's the most appropriate with sporadic updates and lots of users perusing the same pages over and over.
I'm struggling to get it working however. The site shows different content based on whether you're logged in or not, so I have to invalidate cache based on current_user as well as the latest update on the collection of models.
If I look in chrome inspect the ETag and the modified_since are the same, but the server returns a 200 instead of a 304. My code works in development environment, so I'm lost in how to troubleshoot it. Also a different page that only invalidates based on the collection of models (similar on latest update), does work as expected.
Code from the controller:
def index
...#some code
# HTTTP caching:
last_mod = #scraps.order("updated_at").last.updated_at
user = current_user ? current_user.id : 0
fresh_when etag: user.to_s, last_modified: last_mod, public: false
end
Output from chrome inspect
Response Headers:
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Status: 200 OK
Last-Modified: Sun, 23 Jul 2017 20:40:53 GMT
Cache-Control: max-age=0, private, must-revalidate
ETag: W/"6e92592bdb6c3cf610020e2b076e64b4"
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Runtime: 3.187090
X-Request-Id: c698c0c6-8a0d-44ba-8ca9-3f162b766478
Date: Mon, 24 Jul 2017 14:49:38 GMT
Set-Cookie: ... [edited out]; path=/; HttpOnly
X-Powered-By: Phusion Passenger 5.0.30
Server: nginx/1.10.1 + Phusion Passenger 5.0.30
Content-Encoding: gzip
Request Headers:
GET /scraps?page=3&price_max=100&price_min=0&producer=silk+scraps HTTP/1.1
Host: www.picture-scraps.com
Connection: keep-alive
Accept: text/html, application/xhtml+xml, application/xml
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
X-XHR-Referer: https://www.picture-scraps.com/scraps?page=4&price_max=100&price_min=0&producer=silk+scraps
Referer: https://www.picture-scraps.com/scraps?page=4&price_max=100&price_min=0&producer=silk+scraps
Accept-Encoding: gzip, deflate, br
Accept-Language: nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4,af;q=0.2
Cookie: ... [edited out]
If-None-Match: W/"6e92592bdb6c3cf610020e2b076e64b4"
If-Modified-Since: Sun, 23 Jul 2017 20:40:53 GMT
I can imagine some additional information is needed, so please request and I'll add to the question.
Figured it out today. This post provides the answer. I saw the server used weak etags while in the dev environment strong etags were used. The latter is as expected as weak etags were only introduced from rails 5 forward.
If you use Nginx with rails 4 you might experience the same problem. Installing rails_weak_etags gem solved it for me.
I try to use Office365 REST API to list user's files (OneDrive for Business or SharePoint Online), and it can access Office365 administrator account,
but when I try to access other normal active users (who are created by adminstrator account in the Office365 admin center), it could not login, the OAuth page display "Sorry, but we're having trouble signing you in. We received a bad request."
Request headers:
POST https://login.microsoftonline.com/common/login HTTP/1.1
Accept: */*
Cookie: flight-uxoptin=true; stsservicecookie=ests; x-ms-gateway-slice=productiona
Referer: https://login.microsoftonline.com/common/oauth2/authorize?client_id=2d4d11a2-f814-46a7-890a-274a72a7309e&response_type=code&redirect_uri=https%3A%2F%2Fwww.contoso.com%2Fcontoso.htm&resource=https%3A%2F%2Fapi.office.com%2Fdiscovery%2F
Accept-Language: en-US
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: login.microsoftonline.com
Content-Length: 1032
Connection: Keep-Alive
Cache-Control: no-cache
login=example#contoso.onmicrosoft.com&passwd=contoso&ctx=rQIIAdNiNtQztFIxNjY3S00yT9NNTLM01zUxMzLWTUoBcpMMTBOTTZMMTdOSE4uEuAT6rjdUsz8977Uh-YA6807eOasYNTJKSgqKrfT1y8vL9coz86oyC_SS83P1k3PyS1OKU4vKMpNTi_UySnJ3MDJeYGR8wch4i4nf37G0JMMIROQXZValPsIQmcTMn58IEtEDkcn5KamrmBVhNiUWZOrlp6UBDQbblJJZnJxfllpUqb-JmQ0okJuft4tZxSTJ2MTI2NxI18DQwkjXxNI8WdcyOTlR19jCxNLA0Nw4MTEl5QYz4wUWxl2cRPsBAA2&flowToken=AAABAAEAiL9Kn2Z27UubvWFPbm0gLRx6Haiauf3FLxlg3g9UORjSbd3XCCl2tfMhe0xEbC2UOYMMPyfYW9aXLIkJZC5nAK7l4G0_2DB-BItK0rx9ueogAA&n1=109320&n2=-1442391510000&n3=-1442391510000&n4=109320&n5=109320&n6=109320&n7=109320&n8=NaN&n9=109320&n10=110226&n11=111404&n12=111404&n13=111406&n14=113261&n15=27&n16=113500&n17=113502&n18=113511&n19=1235&n20=1&n21=0&n22=0&n23=0&n24=0&n25=0&n26=0&n27=0&n28=0&n29=-1442391623258&n30=-1442391623258&n31=0&n32=0&n33=0&n34=0&n35=0&n36=0&n37=0&n38=0&n39=0&n40=0&n41=0&n42=1442391623292&n43=1442391623511&type=11&LoginOptions=3&NewUser=1&idsbho=1&PwdPad=&sso=&vv=&uiver=1&i12=1&i13=MSIE&i14=9.0&i15=520&i16=770&i20=
Response headers:
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Expires: -1
Vary: Accept-Encoding
Server: Microsoft-IIS/8.5
x-ms-request-id: ab3393a8-0cde-4767-bd12-a2f4cd0be32c
client-request-id: 7b80d78e-e507-4acf-b063-c02703b90d9c
x-ms-gateway-service-instanceid: ESTSFE_IN_164
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000; includeSubDomains
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
Set-Cookie: x-ms-gateway-slice=productiona; path=/; secure; HttpOnly
Set-Cookie: stsservicecookie=ests; path=/
X-Powered-By: ASP.NET
Date: Wed, 16 Sep 2015 08:20:32 GMT
Content-Length: 2206
I follow the below OAuth flow
https://msdn.microsoft.com/en-us/library/azure/dn645542.aspx
Any idea what the problem is and how this can be resolved? I would really appreciate somebody's help with this. Thanks a lot!
Finally, I found a solution in the blow link
http://blog.beecomedigital.com/2015/06/08/aadsts90093-calling-principal-cannot-consent-due-to-lack-of-permissions/
BTW,
It is difficult to understand why the "Bad Request" message is displayed without anyother error message in the OAuth page.
In my case it was the scope parameter missing that caused this issue. Looks like you must include scope when sending authorization request.
A security scanning app picked up a risk against a signalR link in my asp.net MVC 5 website.
the X-Frame-Options response header is missing, which may allow
Cross-Frame Scripting attacks
Any one can tell me what's this about?
And How to solve it?
ASP.NET SignalR Input Validation Flaw Permits Cross-Site Scripting Attacks
Should not be the problem since I am using SignalR 2.1.x
The request is :
POST ***/signalr/send?transport=serverSentEvents&clientProtocol=1.4&connectionToken=bla**bla** HTTP/1.1
Host: ****
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Accept: text/plain, */*; q=0.01
Accept-Language: en-us,en;q=0.5
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: myhost
Pragma: no-cache
Cache-Control: no-cache
Cookie: authentication token
Content-Length: 113
data=********
The Response is :
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8
Expires: -1
Server: Microsoft-IIS/8.0
X-Content-Type-Options: nosniff
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Some html body
I think this warning can safely be ignored. The X-Frame-Options header is used to prevent clickjacking. SignalR responses don't have any links or any other clickable content.
However, it might be a good idea to set an X-Frame-Options header on every response to be extra safe. You can do this via IIS manager or web.config. If you are not using IIS, you can use OWIN middleware instead.
Website making request to clients1.google.com/ocsp with POST.
Its making 2 requests in the same page. and both requests are identical.
What its for? and why it make same 2 requests?
Request Header:-
(Request-Line) POST /ocsp HTTP/1.1
Host clients1.google.com
User-Agent Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0
Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language en-US,en;q=0.5
Accept-Encoding gzip, deflate
Content-Length 107
Content-Type application/ocsp-request
Connection keep-alive
Response Header :
(Status-Line) HTTP/1.1 200 OK
Content-Type application/ocsp-response
Date Sat, 21 Sep 2013 14:08:40 GMT
Expires Wed, 25 Sep 2013 14:08:40 GMT
Cache-Control public, max-age=345600
Server ocsp_responder
Content-Length 463
X-XSS-Protection 1; mode=block
X-Frame-Options SAMEORIGIN
Alternate-Protocol 80:quic
It's probably checking certificate status of various certs in the browser.
I have a form with a kendo upload control and a save button along with a kendo grid. When I open the form using Chrome or Firefox, the upload control works every time. When I open the form in Internet Explorer 10 and leave the form open for more than 1 minute without doing anything, the upload stops working.
I am working on an ASP.net MVC 4 web application.
In Fiddler I see 2 entries.
**POST /PRODUCT/FOLDER/FOLDER2/ProcessSubmit HTTP/1.1**
Accept: */*
Content-Type: multipart/form-data; boundary=---------------------------7dd36d1cb0ce0
Referer: http://localhost:82/PRODUCT/Admin/?contentUrl=/PRODUCT/FOLDER/FOLDER2/Page?cbuId=42aff2e4-33bb-4f73-8500-596f6d73ce3a
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Connection: Keep-Alive
DNT: 1
Host: localhost:82
Pragma: no-cache
Cookie: .MVCRETURNURL=/PRODUCT/Areas/FOLDER/Views/FOLDER2/Page.cshtml; .MVCAUTH=; .ASPXAUTH=F39BF9AA8D958D1783D6A939F3A6B499B81089434C09AC65C168BEF1E298B480935446E77B12A2044F4A7622C82B91DE7C1C6D8E09C4408A9D7C519E6EBB7ACD481735D851AA1A9E65A05CE34AFAE77FCB5E86DDD2EA2F05988C36BEBF5B30994ACEE33557FF3027D875A14ADE2128D2C9C8F20E
Authorization: Negotiate YHcGBisGAQUFAqBtMGugMDAuBgorBgEEAYI3AgIKBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHqI3BDVOVExNU1NQAAEAAACXsgjiBQAFADAAAAAIAAgAKAAAAAYBsR0AAAAPQ0tEUDA1MTRFQUVEMQ==
Content-Length: 0
HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
WWW-Authenticate: Negotiate oYIBHDCCARigAwoBAaEMBgorBgEEAYI3AgIKooIBAQSB/k5UTE1TU1AAAgAAAAoACgA4AAAAFcKJ4tPeOOS7ZmVeEKH7AQAAAAC8ALwAQgAAAAYBsR0AAAAPRQBBAEUARAAxAAIACgBFAEEARQBEADEAAQAQAEMASwBEAFAAMAA1ADEANAAEACQAcABlAHQAcgBvAGwAZQB1AG0AcABsAGEAYwBlAC4AYwBvAG0AAwA2AEMASwBEAFAAMAA1ADEANAAuAHAAZQB0AHIAbwBsAGUAdQBtAHAAbABhAGMAZQAuAGMAbwBtAAUAJABwAGUAdAByAG8AbABlAHUAbQBwAGwAYQBjAGUALgBjAG8AbQAHAAgAiqpF+oprzgEAAAAA
Date: Mon, 17 Jun 2013 18:46:28 GMT
Content-Length: 341
Proxy-Support: Session-Based-Authentication
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>
------------------------------------------------------------------
**POST /PRODUCT/FOLDER/FOLDER2/ProcessSubmit HTTP/1.1**
Accept: */*
Content-Type: multipart/form-data; boundary=---------------------------7dd36d1cb0ce0
Referer: http://localhost:82/PRODUCT/Admin/?contentUrl=/PRODUCT/FOLDER/FOLDER2/Page?cbuId=42aff2e4-33bb-4f73-8500-596f6d73ce3a
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Proxy-Connection: Keep-Alive
Content-Length: 12435
DNT: 1
Host: localhost:82
Pragma: no-cache
Cookie: .MVCRETURNURL=/PRODUCT/Areas/FOLDER/Views/FOLDER2/Page.cshtml; .MVCAUTH=; .ASPXAUTH=F39BF9AA8D958D1783D6A939F3A6B499B81089434C09AC65C168BEF1E298B480935446E77B12A2044F4A7622C82B91DE7C1C6D8E09C4408A9D7C519E6EBB7ACD481735D851AA1A9E65A05CE34AFAE77FCB5E86DDD2EA2F05988C36BEBF5B30994ACEE33557FF3027D875A14ADE2128D2C9C8F20E
Authorization: Negotiate oXcwdaADCgEBoloEWE5UTE1TU1AAAwAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAABXCiOIGAbEdAAAADwjVcbbDNKZpdbLdjWcqj76jEgQQAQAAAPUXp1AtIpqEAAAAAA==
------------------------------------------------------------------
If I press the save button on the same page, I see almost identical fiddler entries.
POST /PRODUCT/FOLDER/FOLDER2/UpdateMajorCodes HTTP/1.1
Accept: */*
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:82/PRODUCT/Admin/?contentUrl=/PRODUCT/FOLDER/FOLDER2/Page?cbuId=42aff2e4-33bb-4f73-8500-596f6d73ce3a
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Host: localhost:82
Content-Length: 0
DNT: 1
Connection: Keep-Alive
Pragma: no-cache
Cookie: .MVCRETURNURL=/PRODUCT/; .MVCAUTH=; .ASPXAUTH=9079B8150CB469C41BA155173534C32ABF863F0369000D0DCE3DB0A2AFF6124AB0878537E11EC70C59B118F8957B657E1E0D6D9C4233F39D2435040ADAFF397EEF703D4535AAA153753A4888BB6FF4DD053A303FA65C506ECB2F2F01A17E1A27F9C9EA2D468F5E77BD2F971698427989BB5372E7
Authorization: Negotiate YHcGBisGAQUFAqBtMGugMDAuBgorBgEEAYI3AgIKBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHqI3BDVOVExNU1NQAAEAAACXsgjiBQAFADAAAAAIAAgAKAAAAAYBsR0AAAAPQ0tEUDA1MTRFQUVEMQ==
HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
WWW-Authenticate: Negotiate oYIBHDCCARigAwoBAaEMBgorBgEEAYI3AgIKooIBAQSB/k5UTE1TU1AAAgAAAAoACgA4AAAAFcKJ4pCCkvtzwTQ5oJ/7AQAAAAC8ALwAQgAAAAYBsR0AAAAPRQBBAEUARAAxAAIACgBFAEEARQBEADEAAQAQAEMASwBEAFAAMAA1ADEANAAEACQAcABlAHQAcgBvAGwAZQB1AG0AcABsAGEAYwBlAC4AYwBvAG0AAwA2AEMASwBEAFAAMAA1ADEANAAuAHAAZQB0AHIAbwBsAGUAdQBtAHAAbABhAGMAZQAuAGMAbwBtAAUAJABwAGUAdAByAG8AbABlAHUAbQBwAGwAYQBjAGUALgBjAG8AbQAHAAgA6r5c6ZdrzgEAAAAA
Date: Mon, 17 Jun 2013 20:19:03 GMT
Content-Length: 341
Proxy-Support: Session-Based-Authentication
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>
------------------------------------------------------------------
POST /PRODUCT/FOLDER/FOLDER2/UpdateMajorCodes HTTP/1.1
Accept: */*
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:82/PRODUCT/Admin/?contentUrl=/PRODUCT/FOLDER/FOLDER2/Page?cbuId=42aff2e4-33bb-4f73-8500-596f6d73ce3a
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Host: localhost:82
Content-Length: 13798
DNT: 1
Connection: Keep-Alive
Pragma: no-cache
Cookie: .MVCRETURNURL=/PRODUCT/; .MVCAUTH=; .ASPXAUTH=9079B8150CB469C41BA155173534C32ABF863F0369000D0DCE3DB0A2AFF6124AB0878537E11EC70C59B118F8957B657E1E0D6D9C4233F39D2435040ADAFF397EEF703D4535AAA153753A4888BB6FF4DD053A303FA65C506ECB2F2F01A17E1A27F9C9EA2D468F5E77BD2F971698427989BB5372E7
Authorization: Negotiate oXcwdaADCgEBoloEWE5UTE1TU1AAAwAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAAAAAAABYAAAAAAAAAFgAAAAAAAAAWAAAABXCiOIGAbEdAAAAD93bE7wOkB2UOLRe3+N5TJ6jEgQQAQAAAPUXp1AtIpqEAAAAAA==
The save works and makes a call in to the server to save the data. The upload server call never happens. I am trying to figure out how to get the upload to work in IE 10 after the authentication timeout.
thanks
Kent Pigott