Please excuse me if this is normal, but I am attempting to send a post request from iOS using AFNetworking. Using Charles to monitor the request, I see that a GET is sent:
GET /api/updateTeamAlert/ HTTP/1.1
Host: www.******r.co
Accept-Encoding: gzip, deflate
Accept: */*
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
Connection: keep-alive
User-Agent: ****** Alerts/1.0 (iPhone Simulator; iOS 6.1; Scale/2.00)
Is this normal? I am trying to find out why my POST parameters are empty at the server - could this be the cause?
I am creating my request like this:
NSDictionary *params = #{#"device_id":#"test-device", #"innings":#6, #"team":#"WashingtonNationals"};
[_client postPath:#"updateTeamAlert"
parameters:params
success:^(AFHTTPRequestOperation *operation, id responseObject)
{
NSString *responseStr = [[NSString alloc] initWithData:responseObject encoding:NSUTF8StringEncoding];
NSLog(#"Request Successful, response '%#'", responseStr);
}
failure:^(AFHTTPRequestOperation *operation, NSError *error)
{
NSLog(#"[HTTPClient Error]: %#", error.localizedDescription);
}];
UPDATE
Well, all I had to do to get this working was change the postPath to include the trailing '/' - perhaps this is obvious to most, but I would love an explanation for the accepted answer.
Well, all I had to do to get this working was change the postPath to include the trailing '/' - perhaps this is obvious to most, but I would love an explanation for the accepted answer.
PHP applications often have misconfigured servers that lose information (like HTTP method) when doing a redirect. In your case, adding the / resolved to the canonical path for your particular web service, but in redirecting to that endpoint, the POST was changed into a GET.
Another way to potentially solve this issue is to use AFURLConnectionOperation -setRedirectResponseBlock, and ensure that the redirect request has the correct verb.
#mattt's answer above is incorrect.
HTTP/1.0 302's worked as you'd logically expect "POST /FOO" 302'd to /BAR implied that you'd get "POST /BAR". However, little to no clients implemented it this way, and commonly changed the method to GET. This is somewhat understandable as the new, redirected resource is unknown to the user, and one shouldn't POST willy nilly to an unknown resource.
HTTP/1.1 clears this up - 302's should let the user know about the redirection. 307's make the redirection work as you would expect, maintaining the method.
AFNetworking is setup to act like all the other naughty clients out there - 302's alter the method to GET, giving the client a chance to alert the user. I just had this issue with AFNetworking myself: I set up some breakpoints, stepped through, and watched the method change before my eyes.
I haven't yet tested that 307's worked as defined with AFNetworking, but regardless, 302's are behaving in the odd way that HTTP/1.1 defined them to work.
tl;dr - In HTTP/1.1 use 307's to redirect and maintain the method, not 302's.
I ran into a similar issue where every POST request was being interpreted as a GET request. It turns out, similar to servers messing up when you're missing the trailing slash, the request type can get lost when your DNS redirects www.site.com to site.com or vice versa.
In my case, my DNS forces site.com to redirect to www.site.com, but I had set my base URLs to point to site.com. Thus, when I sent a request to my API at site.com/api, the request was redirected to www.site.com/api and the request type was lost, and the server defaulted to a GET request. So I added the www to my base URL, sent my request directly to www.site.com/api (avoiding the DNS redirect), and my POST requests started working again.
TL;DR: By adding the www (or removing it, depending on your DNS) I eliminated the redirect and fixed the problem.
I facing same problem, I using Laravel for server.
If my request is "http://localhost/api/abc/" then POST method send from client will became GET method in Server.
But if my request is "http://localhost/api/abc" (without "/"), then server will receive POST method.
I founded root cause is because of Redirect rule in .htaccess file:
In my .htaccess have these lines:
# Redirect Trailing Slashes If Not A Folder...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ /$1 [L,R=301]
These lines will change POST method to GET method if request have "/" in the end.
To fix this issue, I just comment out these lines.
Hope this help.
Another case you will encounter this issue is that when you set up redirection from HTTP to HTTPS automatically, the POST request will be redirected as GET request if this redirection setup is at work. Just updating your API endpoint to HTTPS resolves this case.
Related
Please bare with me for a question for which it's nearly impossible to create a reproducible example.
I have an API setup with FastAPI using Docker, Serverless and deployed on AWS API Gateway. All routes discussed are protected with an api-key that is passed into the header (x-api-key).
I'm trying to accomplish a simple redirect from one route to another using fastapi.responses.RedirectResponse. The redirect works perfectly fine locally (though, this is without api-key), and both routes work perfectly fine when deployed on AWS and connected to directly, but something is blocking the redirect from route one (abc/item) to route two (xyz/item) when I deploy to AWS. I'm not sure what could be the issue, because the logs in CloudWatch aren't giving me much to work with.
To illustrate my issue let's say we have route abc/item that looks like this:
#router.get("/abc/item")
async def get_item(item_id: int, request: Request, db: Session = Depends(get_db)):
if False:
redirect_url = f"/xyz/item?item_id={item_id}"
logging.info(f"Redirecting to {redirect_url}")
return RedirectResponse(redirect_url, headers=request.headers)
else:
execution = db.execute(text(items_query))
return convert_to_json(execution)
So, we check if some value is True/False and if it's False we redirect from abc/item to xyz/item using RedirectResponse(). We pass the redirect_url, which is just the xyz/item route including query parameters and we pass request.headers (as suggested here and here), because I figured we need to pass along the x-api-key to the new route. In the second route we again try a query in a different table (other_items) and return some value.
I have also tried passing status_code=status.HTTP_303_SEE_OTHER and status_code=status.HTTP_307_TEMPORARY_REDIRECT to RedirectResponse() as suggested by some tangentially related questions I found on StackOverflow and the FastAPI discussions, but that didn't help either.
#router.get("/xyz/item")
async def get_item(item_id: int, db: Session = Depends(get_db)):
execution = db.execute(text(other_items_query))
return convert_to_json(execution)
Like I said, when deployed I can successfully connect directly to both abc/item and get a return value if True and I can also connect to xyz/item directly and get a correct value from that, but when I pass a value to abc/item that is False (and thus it should redirect) I get {"message": "Forbidden"}.
In case it can be of any help, I try debugging this using a "curl" tool, and the headers I get returned give the following info:
Content-Type: application/json
Content-Length: 23
Connection: keep-alive
Date: Wed, 27 Jul 2022 08:43:06 GMT
x-amzn-RequestId: XXXXXXXXXXXXXXXXXXXX
x-amzn-ErrorType: ForbiddenException
x-amz-apigw-id: XXXXXXXXXXXXXXXX
X-Cache: Error from cloudfront
Via: 1.1 XXXXXXXXXXXXXXXXXXXXXXXXX.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: XXXXX
X-Amz-Cf-Id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
So, this is hinting at a CloudFront error. Unfortunately I don't see anything slightly hinting at this API when I look into my CloudFront dashboard on AWS, there literally is nothing there (I do have permissions to view the contents though...)
The API logs in CloudWatch look like this:
2022-07-27T03:43:06.495-05:00 Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.495-05:00 [INFO] 2022-07-27T08:43:06.495Z Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.496-05:00 2022-07-27 08:43:06,496 INFO sqlalchemy.engine.Engine ROLLBACK
2022-07-27T03:43:06.496-05:00 [INFO] 2022-07-27T08:43:06.496Z ROLLBACK
2022-07-27T03:43:06.499-05:00 END RequestId: 6f449762-6a60189e4314
2022-07-27T03:43:06.499-05:00 REPORT RequestId: 6f449762-6a60189e4314 Duration: 85.62 ms Billed Duration: 86 ms Memory Size: 256 MB Max Memory Used: 204 MB
I have been wondering if my issue could be related to something I need to add to somewhere in my serverless.yml, perhaps in the functions: part. That currently looks like this for these two routes:
events:
- http:
path: abc/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
- http:
path: xyz/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
Finally, it's probably good to note that I have added custom middleware to FastAPI to handle the two different database connections I need for connecting to other_items and items tables, though I'm not sure how relevant this is, considering this functions fine when redirecting locally. For this I implemented the solution found here. This custom middleware is the reason for the redirect in the first place (we change connection URI based on route with that middleware), so I figured it's good to share this bit of info as well.
Thanks!
As noted here and here, it is mpossible to redirect to a page with custom headers set. A redirection in the HTTP protocol doesn't support adding any headers to the target location. It is basically just a header in itself and only allows for a URL (a redirect response though could also include body content, if needed—see this answer). When you add the authorization header to the RedirectResponse, you only send that header back to the client.
A suggested here, you could use the set-cookie HTTP response header:
The Set-Cookie HTTP response header is used to send a cookie from the
server to the user agent (client), so that the user agent can send it back to
the server later.
In FastAPI—documentation can be found here and here—this can be done as follows:
from fastapi import Request
from fastapi.responses import RedirectResponse
#app.get("/abc/item")
def get_item(request: Request):
redirect_url = request.url_for('your_endpoints_function_name') #e.g., 'get_item'
response = RedirectResponse(redirect_url)
response.set_cookie(key="fakesession", value="fake-cookie-session-value", httponly=True)
return response
Inside the other endpoint, where you are redirecting the user to, you can extract that cookie to authenticate the user. The cookie can be found in request.cookies—which should return, for example, {'fakesession': 'fake-cookie-session-value-MANUAL'}—and you retrieve it using request.cookies.get('fakesession').
On a different note, request.url_for() function accepts only path parameters, not query parameters (such as item_id in your /abc/item and /xyz/item endpoints). Thus, you can either create the URL in the way you already do, or use the CustomURLProcessor suggested here, here and here, which allows you to pass both path and query parameters.
If the redirection takes place from one domain to another (e.g., from abc.com to xyz.com), please have a look at this answer.
I think I understand CORS pretty well, but I'm still a bit puzzled about the browser's behavior when it comes to the preflight requests.
Let's say the browser issues this preflight request:
OPTIONS http://myserver.local:7000/api/order/4 HTTP/1.1
Host: myserver.local:7000
Connection: keep-alive
Accept: */*
Access-Control-Request-Method: POST
Access-Control-Request-Headers: x-my-custom-header
Origin: http://localhost:5000
Sec-Fetch-Mode: cors
Referer: http://localhost:5000/
and my API returns:
HTTP/1.1 204 No Content
Date: Wed, 09 Mar 2022 12:52:50 GMT
Server: Kestrel
Access-Control-Allow-Headers: x-my-custom-header
Access-Control-Allow-Methods: PUT,DELETE
Access-Control-Allow-Origin: http://localhost:5000
Vary: Origin
Note that the server allows methods PUT and DELETE in the response to the preflight request, but not POST, which is the method of the actual CORS request.
Should the browser not block this request due to the mismatch between the actual request's method and the methods listed in the Access-Control-Allow-Methods header? Or is it enough that the server respond with a 20x status code for the browser to accept the preflight and then send the actual request?
My assumptions was that the browser would compare the allow-methods and block the request if the requested method did no match... Am I missing something?
TL;DR
No, the browser doesn't require the server to explicitly allow the POST method, because the latter, as a so-called CORS-safelisted method, gets a free pass.
More details
What the spec says
The answer, as always, lies in the Fetch standard (section 4.8), which specifies how CORS works:
Let methods be the result of extracting header list values given Access-Control-Allow-Methods and response’s header list.
And further down:
If request’s method is not in methods, request’s method is not a CORS-safelisted method, and request’s credentials mode is "include" or methods does not contain *, then return a network error.
(my emphasis)
What is a CORS-safelisted method? The term is defined in section 2.2.1:
A CORS-safelisted method is a method that is GET, HEAD, or POST.
Interpretation
If the method of the CORS request is one of GET, HEAD, or POST, the browser doesn't require the server to explicitly list that method in the Access-Control-Allow-Methods header for CORS preflight to succeed.
Experiment
I've found Jake Archibald's CORS playground useful for testing my (mis)understanding of CORS. Running this particular instance in your browser may convince you that the POST method doesn't need to be explicitly allowed for CORS preflight to succeed.
I discovered that six years ago, the previous developer commented out this line of code (Ruby, Rails):
#protect_from_forgery
I replaced it with the default:
protect_from_forgery with: :exception
and now I mysteriously get the following error when I try to add items to my cart while logged out:
Access to XMLHttpRequest at 'https://id.foo-staging.com/openid/checklogin?return_to=http%3A%2F%2Flocalhost.foo-staging.com%3A3000%2Fcart%2Fitems' (redirected from 'http://localhost.foo-staging.com:3000/cart/items') from origin 'http://localhost.foo-staging.com:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.
I've pinned down that this is happening because of the following lines:
def get_user_identity_from_open_id_server
redirect_to "#{OPEN_ID_PROVIDER_URL}/openid/checklogin?return_to=#{Rack::Utils.escape(request.url)}"
end
def open_id_authentication
#stuff
get_user_identity_from_open_id_server
end
before_filter :open_id_authentication
I understand what causes a preflight request, at a very high level, thanks to the documentation. But I don't think I'm doing any of those things.
* the request method is anything other than GET, HEAD, or POST
* you’ve set custom request headers other than Accept, Accept-Language, Content-Language, Content-Type, DPR, Downlink, Save-Data, Viewport-Width, or Width
* the Content-Type request header has a value other than application/x-www-form-urlencoded, multipart/form-data, or text/plain
So my initial question is how do I determine what is triggering the preflight request, and then maybe I can figure out how to prevent it from happening. Is this a situation that I can change on my end, or does something need to change on id.foo-staging.com (which I don't have access to, but could probably ask the right person to fix it for me).
I've been Googling all day, and nothing seems to make any sense to me, especially because I can't pin down precisely what's wrong.
I can solve the issue with this code:
skip_before_filter :open_id_authentication, :only => [:create], :if => :current_user and :anonymous_cart
But I have to assume that this is unsafe, from a security standpoint?
ETA: This is what I see on the Network tab for this request:
General:
Request URL: https://id.foo-staging.com/openid/checklogin?return_to=http%3A%2F%2Flocalhost.foo-staging.com%3A3000%2Fcart%2Fitems
Referrer Policy: no-referrer-when-downgrade
Request Headers:
Provisional headers are shown
Access-Control-Request-Headers: x-requested-with
Access-Control-Request-Method: GET
Origin: http://localhost.foo-staging.com:3000
Referer: http://localhost.foo-staging.com:3000/p/Product0/1?id=1&slug=Product0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Query String Parameters:
return_to: http://localhost.foo-staging.com:3000/cart/items
I presume the problem is the x-requested-with request header. But I don't know how to resolve this.
EATA:
Many JavaScript frameworks such as JQuery will automatically send this header along with any AJAX requests. This header cannot be sent cross-domain:
I guess my only option is to figure out how to rewrite this without AJAX?
To avoid the preflight request you have to remove x-requested-with header but the error that you have is because the response location in the preflight call is different from the Origin.
To fix the problem, update your code to use the new URL as reported by the redirect, thereby avoiding the redirect.
I have a grails 2.2.4 application. I wanted to enable CORS
So I installed cors plugin by having the following line in build config.
plugins {
runtime ':cors:1.1.8'
}
Then in the config.groovy
cors.headers = ['Access-Control-Allow-Origin': '*']
But after this when I run the application, CORS in not enabled. So I debugged the CORS plugin. The issue seems to be in CorsFilter class in the following method
private boolean checkOrigin(HttpServletRequest req, HttpServletResponse resp) {
String origin = req.getHeader("Origin");
if (origin == null) {
//no origin; per W3C spec, terminate further processing for both preflight and actual requests
return false;
}
The origin parameter in the above line is always null as the request does not have the parameter 'Origin'. Is there something i'm doing wrong? I'm not looking for the answer which says add a manual header with the name "Origin" since that is not exactly a proper fix
I'm quite new to CORS so appriciate the help.
In addition to Access-Control-Allow-Origin, and in addition to setting the Origin header on request, you probably need to specify these response headers as well:
Access-Control-Allow-Headers: accept
Access-Control-Allow-Headers: origin
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Method: GET
Access-Control-Allow-Method: POST
Also make sure you respond to HTTP OPTIONS requests with these headers and a blank 200 OK response.
For now, let's assume that RestClient is sending the Origin header properly. It may still be getting stripped by your application. You can prevent this using the Access-Control-Allow-Headers: Origin header.
Most of the problems I have had with my web services is that the right headers are being sent, but they are stripped from the message by my web server. So I tend to adopt a shotgun approach of "allow everything" and then one by one remove what I don't need. My allow-headers header usually is pretty long and I end up having to include stuff like Content-Type, X-Requested-With and other junk before my requests will finally go through.
I further recommend that you test using something besides RestClient, if only as a sanity check. I use Postman, a free Chrome app, for all my messaging tests. It looks to me like the problem is with RestClient not sending the proper Origin header.
Our application makes use of RESTful service calls using NSURLSession. The calls themselves are routed through a reverse proxy, to aide in session management, security, etc. on the backend. The only problem we're having is related to authentication. When a user attempts to access a protected resource -- either through a browser or a REST call -- and they are not authenticated, the reverse proxy displays an HTML login page.
The problem is, we'd like to leverage NSURLSession's ability to handle authentication challenges automatically. However, NSURLSession is unable to recognize that we're getting back an authentication challenge, because no HTTP 401 or other error code is being returned to the client. The reverse proxy sends back a 200, because the HTML was delivered successfully. We're able to recognize that it is the login page by inspecting the HTML within the client, but we'd like to be able to handle the authentication using a custom NSURLAuthenticationChallenge, if at all possible, so that NSURLSession can retry requests after authentication is successful.
Is there any way for us to recognize that we're getting back a login page within our completionHandler and tell NSURLSession that we're really getting back an authentication challenge? Or does NSURLSession require that we receive back an HTTP error code (401, etc.)
A couple of possibilities come to mind.
If you're using the delegate rendition of NSURLSession, you might be able to detect the redirect to the authentication failure page via NSURLSessionTaskDelegate method willPerformHTTPRedirection.
You can probably also detect the redirect by examining the task's currentRequest.URL and see if a redirect happened there, too. As the documentation says, currentRequest "is typically the same as the initial request (originalRequest) except when the server has responded to the initial request with a redirect to a different URL."
Assuming that the RESTful service generally would not be returning HTML, you can look at the NSURLResponse, confirm that it's really a NSHTTPURLResponse subclass, and then look at allHeaderFields to confirm whether text/html appears in the Content-Type of the response. This obviously only works if the authentication page returns a Content-Type that includes text/html and the rest of your responses don't (e.g. they're application/json or something like that).
Anyway, that might look like:
NSString *contentType = [self headerValueForKey:#"Content-Type" response:response];
if ([contentType rangeOfString:#"text/html"].location != NSNotFound) {
// handle it here
}
Where, headerValueForKey might be defined as follows:
- (NSString *)headerValueForKey:(NSString *)searchKey response:(NSURLResponse *)response
{
if (![response isKindOfClass:[NSHTTPURLResponse class]])
return nil;
NSDictionary *headers = [(NSHTTPURLResponse *) response allHeaderFields];
for (NSString *key in headers) {
if ([searchKey caseInsensitiveCompare:key] == NSOrderedSame) {
return headers[key];
}
};
return nil;
}
At worst, you could detect the HTML response and parse it using something like HPPLE and programmatically detect the authentication HTML response.
See Wenderlich's article How to Parse HTML on iOS.
Frankly, though, I would prefer to see a REST response to report the authentication error with a REST response (or a proper authentication failure challenge or a non 200 response) rather than a redirect to a HTML page. If you have an opportunity to change the server response, that would be my personal preference.
Authentication challenges are triggered by the WWW-Authenticate header in a 40x response.
From the documentation :
The URL loading system classes do not call their delegates to handle request challenges unless the server response contains a WWW-Authenticate header. Other authentication types, such as proxy authentication and TLS trust validation do not require this header.
Unfortunately the proxy authentication referred to above does not apply in your reverse proxy/accelerator scenario.
Have you tried simply setting the username and password to the authorization header of NSMutableURLRequest as such.
NSString *authorizationString = [[[NSString stringWithFormat:#"%#:%#",user, password] dataUsingEncoding:NSUTF8StringEncoding] base64Encoding];
[yourRequest setValue:[NSString stringWithFormat:#"Basic %#", authorizationString] forHTTPHeaderField:#"Authorization"];
This should work.
You could try using an HTML Parser library (like https://github.com/nolanw/HTMLReader).
Then in the completion handler block, you can evaluate the response to check if it is an HTML page, and if it contains a login form.
I don't know this library, but I guess you can use it like this:
HTMLDocument *document = [HTMLDocument documentWithString:responseObjectFromServer];
HTMLNode *node = [document firstNodeMatchingSelector:#"form.login"];
if(node) {
// Perform second NSURLSession call with login data
}