I have build a number of REST interfaces in app engine and while debugging a request on my iPhone client and via Chrome, I noticed that two identical requests had different responses sizes:
"GET /card?omid=2 HTTP/1.1" 200 1468 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36"
"GET /card?omid=2 HTTP/1.1" 200 10669 - "Null Society 1.83 (iPhone; iPhone OS 6.1.3; en_US)"
Note that the identical request from the iPhone client is 7x larger than the one from Chrome. These requests were a minute apart, and my server-side code does not distinguish between clients for determining responses. In this particular request, the server merely reads the query variable, "omid", and returns the appropriate data.
The only thing I can think of is that app engine natively compresses data by client? Seems weird though. Any help would be appreciated!
In the response section of the GAE docs it notes:
If the client sends HTTP headers with the request indicating that the
client can accept compressed (gzipped) content, App Engine compresses
the response data automatically and attaches the appropriate response
headers. It uses both the Accept-Encoding and User-Agent request
headers to determine if the client can reliably receive compressed
responses. Custom clients can indicate that they are able to receive
compressed responses by specifying both Accept-Encoding and User-Agent
headers with a value of gzip.
https://developers.google.com/appengine/docs/python/#Python_Responses
That sounds like it could explain what you are seeing.
Related
I am trying to get OpenID Connect authentication working for my legacy ASP.NET MVC application. My ASP.NET MVC application will be the Relaying Party and a business partner of ours will serve as the Identity Provider.
To get familiar with what I'll need to do I created an account on Auth0 and created a new App for a Web Application. I then downloaded their ASP.NET MVC OWIN quickstart from GitHub. I got everything setup and I am able to authenticate successfully with Microsoft Edge and Firefox. But with Chrome the workflow goes like this:
Visit localhost:3000
Attempt to access a protected resource, which redirects me to localhost:3000/Account/Login
/Account/Login creates the challenge, which does two things: (1) Creates the Nonce cookie, and (2) redirects the user to Auth0's /authorize endpoint
I successfully login on Auth0's login screen
A POST request is made to the /callback endpoint on localhost:3000
I get a Yellow Screen of Death with the following message:
IDX21323: RequireNonce is 'System.Boolean'. OpenIdConnectProtocolValidationContext.Nonce was null, OpenIdConnectProtocol.ValidatedIdToken.Payload.Nonce was not null. The nonce cannot be validated. If you don't need to check the nonce, set OpenIdConnectProtocolValidator.RequireNonce to 'false'. Note if a 'nonce' is found it will be evaluated.
Examining the HTTP traffic I see that the issue with Chrome is that in step (3) - when the server sets the Nonce cookie in the 302 Redirect - Chrome is not saving it. Therefore, when step (5) happens the browser does not send any Nonce information to the server and validation fails.
This is evidenced by the HTTP traffic at step (3) and (5). Here is the localhost response on step (3). You can see that it is telling the browser to store the Nonce cookie:
HTTP/1.1 302 Found
Cache-Control: private
Location: https://whatever.us.auth0.com/authorize?client_id=gYb3FOL5OWK419L8...
Set-Cookie: OpenIdConnect.nonce.fRunx5CPoGdhTRM3mgqpn62m9SFkH4AszKWpOOk8LV0%3D=T1NPQjNlYTgtQ...; path=/; expires=Sat, 18-Jul-2020 20:47:59 GMT; HttpOnly; SameSite=None
But after I am redirected to Auth0 I can check Chrome's cookies and it does not have the Nonce cookie in its cookies collection for localhost. Moreover, when step (5) hits, the browser request looks like so - no mention of the Nonce cookie:
POST http://localhost:3000/callback HTTP/1.1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36
code=DANVniZ296OzQW...
This results in the error aforementioned.
(When I examine the HTTP traffic using Edge or Firefox, in step (5) I see the browser sends the Nonce cookie, whereas it's missing entirely from Chrome.)
I am using Chrome version 84 and Windows 10. I also tried this on an old computer at home with Windows 7 (and Chrome 84) and experienced the exact same behavior.
What is going on here and, more importantly, how do I get it to work? My initial assumption was that maybe it was a SameSite cookie issue, but I don't think that's the case because I can see that the cookie isn't get created in the first place (it's not that it exists, but just isn't being sent on the redirect to localhost). Moreover, the Nonce cookie has SameSite=None, so that shouldn't matter, right?
Thanks
Figured it out with the help of a colleague...
Chrome won't save the cookie when using SameSite=None if the traffic is over HTTP. Needed to setup Visual Studio to use HTTPS. Once I did that, things worked as expected.
I Use Youtube API V3 , my web page request bots request so my api result is too many Request this Ip
Bots Example :
1 Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help#moz.com)
2 Mozilla/5.0 (compatible; AlphaBot/3.2; +http://alphaseobot.com/bot.html)
this type many bots request my website www.watchyoutubevideo.com
Example bots Google,Bing,Yahoo etc bots ....
how to solve
help me
thank you
Google Natural Language API has been working in my iOS app up until yesterday. The API started returning "permission denied" errors as of this morning. E.g:
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
Example request:
POST /v1/documents:analyzeEntities?key=..... HTTP/1.1
Host: language.googleapis.com
Content-Type: application/json
Connection: keep-alive
X-Ios-Bundle-Identifier: .....
Accept: */*
Accept-Language: en-us
Content-Length: 291
Accept-Encoding: gzip, deflate
User-Agent: CardScanner/1 CFNetwork/808.2.16 Darwin/15.6.0
{"encodingType":"UTF8","document":{"type":"PLAIN_TEXT","content":"....."}}
Billing is enabled for the account (with a balance of $0). The account also has 36 days left on the trial period.
The key matches the value in the Google Cloud Platform API dashboard. I have also tried regenerating the key, and using the new key in the app.
I have also tried enabling key restrictions for iOS devices, and included the "X-Ios-Bundle-Identifier" header with the app bundle identifier.
The app also uses the Google Vision API which works without issues. Calls to the Vision API do respond to changes to the key restrictions.
Calls made from the demo page also show a permissions error message. Calls from the API explorer do work however.
Edit:
The error is also happening on the demo on the product web page. Tracing the error in Charles shows the same "permission denied" response being returned to the web page:
Edit:
Below is an example of the HTTP request and response captured from the demo page. The request and resulting error is almost identical to my app, except that the demo seems to be using http 2, whereas my app is using http 1.
HTTP request:
:method: POST
:authority: language.googleapis.com
:scheme: https
:path: /v1/documents:analyzeEntities?key=.....
content-length: 250
origin: https://cloud.google.com
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
content-type: text/plain;charset=UTF-8
accept: */*
referer: https://cloud.google.com/natural-language/
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.8
{"document":{"type":"PLAIN_TEXT","content":"Google, headquartered in Mountain View, unveiled the new Android phone at the Consumer Electronic Show. Sundar Pichai said in his keynote that users love their new Android phones."},"encodingType":"UTF16"}
HTTP response:
:status: 403
vary: Origin
vary: X-Origin
vary: Referer
content-type: application/json; charset=UTF-8
content-encoding: gzip
date: Sun, 26 Feb 2017 14:52:24 GMT
server: ESF
cache-control: private
content-length: 128
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
access-control-allow-origin: https://cloud.google.com
access-control-expose-headers: content-encoding,date,server,content-length
alt-svc: quic=":443"; ma=2592000; v="35,34"
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
We are aware of this issue and should be fixed now. Let us know if you still see the same problem.
I have experienced the same behaviour and I believe it is an undocumented change at Google's end.
I can now only make the call work via OAuth, despite the documentation suggesting that an API key is sufficient. Up until very recently it was possible to make a call with an API key only and no restrictions configured on the key.
My "answer" for what it's worth is to provide feedback on the documentation page to complain that the documentation doesn't match the behaviour. I think the wording on the page may have changed recently, but it still suggests an API key should work for testing purposes.
"TRY NATURAL LANGUAGE API" button from Google Cloud Vision API demo page doesn't work either! Looks like some bug, hope they fix it soon...
I see these invalid HTTP requests in the server log.
The request URI includes scheme+hostname+port.
1.2.3.4 [13/Jan/2017:04:20:01 +0000] GET http://www.DOMAIN.hu:80/munkaugyi-segedanyagok/minimalber-2017-kormanyrendelet HTTP/1.1 403 http://m.facebook.com Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92 [FBAN/FBIOS;FBAV/75.0.0.48.61;FBBV/45926345;FBRV/46260258;FBDV/iPhone8,1;FBMD/iPhone;FBSN/iOS;FBSV/10.2;FBSS/2;FBCR/TelekomHU;FBID/phone;FBLC/en_US;FBOP/5]
All other requests from the same visitor suggest a legitimate user.
Could it be the Facebook app for iPhone?
It turns out most webservers support absolute request URI-s.
https://www.rfc-editor.org/rfc/rfc2616#section-3.2.1
An HTTP client may send requests with absolute request URI-s like the ones in proxy requests but with the same domain name.
HTTP digest authentication no longer works in our app since iOS 10 due to wrong nonce-count in Authorization: Digest header generated by NSURLSession.
The same code works in iOS 9, but fail to auth in iOS 10
Create a POST request with NSURLRequest
Fire it with NSURLSession
Handle NSURLAuthenticationMethodHTTPDigest in urlSession(_:didReceive:completionHandler:) delegate
The server responds with a 401 and qop="auth" string as expected
The app requests again with the Authorization: Digest header set.
According to RFC2617:
nonce-count
This MUST be specified if a qop directive is sent (see above), and
MUST NOT be specified if the server did not send a qop directive in
the WWW-Authenticate header field. The nc-value is the hexadecimal
count of the number of requests (including the current request)
that the client has sent with the nonce value in this request. For
example, in the first request sent in response to a given nonce
value, the client sends "nc=00000001". The purpose of this
directive is to allow the server to detect request replays by
maintaining its own copy of this count - if the same nc-value is
seen twice, then the request is a replay. See the description
below of the construction of the request-digest value.
However, the nonce-count start at "nc=00000002" even for the first request in iOS 10, which cause the server to reject it.
Expect server response 200 OK
iOS 9 & Before:
POST /Tunnel/Message.aspx HTTP/1.1
Host: 172.18.70.12:3454
Accept: */*
Content-Type: application/xml
User-Agent: iViewer/1 CFNetwork/758.5.3 Darwin/15.6.0
Connection: keep-alive
Cookie:
AuthType: digest
Accept-Language: zh-tw
Content-Length: 69
Accept-Encoding: gzip, deflate
Authorization: Digest username="admin", realm="ND8422P",
nonce="cc17a78cdd96d54e012eadefe7d13d82", uri="/Tunnel/Message.aspx",
response="51587db4bcf6eeece68c4ec21108f170",
cnonce="47b8df8a980f280038834b7817250e90", nc=00000001, qop="auth"
<?xml version="1.0" encoding="UTF-8"?><GetServerInfo></GetServerInfo>
HTTP/1.0 200 OK
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Content-Type: text/xml
Content-Length: 1127
iOS 10:
POST /Tunnel/Message.aspx HTTP/1.1
Host: 172.18.70.12:3454
Accept: */*
Content-Type: application/xml
User-Agent: iViewer/1 CFNetwork/808.0.2 Darwin/16.0.0
Connection: keep-alive
Cookie:
AuthType: digest
Accept-Language: en-us
Content-Length: 69
Accept-Encoding: gzip, deflate
Authorization: Digest username="admin", realm="ND8422P",
nonce="4b8bf8549da0c3010f031472e95f387d", uri="/Tunnel/Message.aspx",
response="91cf44bc0aadf2f743164d03b5c708c7",
cnonce="b5f9e6c69e19c1b396298d68f2aefe7e", nc=00000002, qop="auth"
<?xml version="1.0" encoding="UTF-8"?><GetServerInfo></GetServerInfo>
HTTP/1.0 401 Unauthorized
WWW-Authenticate: Digest qop="auth", realm="ND8422P", nonce="8e8b0538bb08876ac4d8203f1d14e9ac"
CSeq: 0
Is anyone facing the same issue?
The only related post I could find is:
Apple Developer Forums : Problem of the digest authentication, but no further information.
How do you fix it or get workaround on client app side without asking server side to ignore the wrong nonce-count?
Thanks.
Apple Developer Technical Support confirm that is a bug of iOS 10.
Hope it will be fixed soon.
Thank you for contacting Apple Developer Technical Support (DTS).
We believe this issue is a bug. Please file a bug report using the Bug Reporter tool https://developer.apple.com/bug-reporting/.
Update:
Apple fixed this issue in iOS 10.2 Beta 3
Chances are, the OS is sending a HEAD request first, and your server-side code isn't getting it. I would try running Charles Proxy to verify that this is what's happening.
That said, skipping a nonce count is not inherently an indication of any sort of attack. It could occur even in iOS 9 if a request got lost somehow (e.g. a network error). What's important is to ensure that the count doesn't go backwards. So I would argue that your server code is buggy and should not be rejecting that to begin with.
We have the same problem in our company as described here:
Cordova app can't connect with Dynamics NAV Web-Service (ODATA) after update to iOS 10
We can reproduce the issue both in our App and the Safari Browser with iOS 10 devices. There does not seem to be a simple client side workaround. We opened a Bug Report with Apple.
In our case the problem was solved with the 10.2 Beta release.