Using the URLSession family of classes, is there a way to check the validity of a response? Specifically, I have an HTTP response whose Cache-Control header specifies no-cache, so that any cached response will have to be submitted for validation before it can be used. I can retrieve the CachedURLResponse object from URLCache.shared, but none of URLSession, URLCache, or CachedURLResponse seem to have any methods for determining whether such a cached response is still valid. Such methods are also absent from URLSessionDelegate and URLSessionTaskDelegate.
Is there any way to do this other than initiating the actual validation request myself? Presumably this is done somewhere in the URLSession stack (although I'm not sure of this), but it looks as if this functionality may just not be exposed by the public API.
The question is why do you need to check the validity. If you want only to check the validity you may use a probably uncommon method by using URLProtocol and writing a custom NSURLProtocolClient. The client has only empty methods except:
func urlProtocol(_ protocol: URLProtocol, cachedResponseIsValid cachedResponse: CachedURLResponse) {
// Cached response is valid. Store this information in a appropriate way.
protocol.stopLoading()
}
Now, you create a protocol with that client
let myClient = ...
let protocol = NSURLProtocol(request, cachedResponse, client: myClient)
protocol.startLoading()
protocol.stopLoading() // Stop, if urlProtocol(_: cachedResponseIsValid:) was not called
As far as I know, the caching of an HTTP response is handled automatically. (Just to be sure, I tried it again just before answering this question).
The default caching policy on a URLRequest is useProtocolCachePolicy. So, on all the subsequent requests to that url, the HTTP headers will contain the If-None-Match key, with the latest Etag as its value.
If this is not working for you automatically, make sure that the server is sending the Etag header in its response. Also, that the server acknowledges the If-None-Match header on all the requests.
TL;DR
CHECK:
That you get the Cache-Control and the Etag header in your
responses from the server.
That the server receives an If-None-Match header with the
correct Etag value, in all the subsequent requests.
That the server is actually configured to handle the cached
responses (As in, it responds to the If-None-Match header
correctly).
References:
https://developer.apple.com/documentation/foundation/nsurlrequest.cachepolicy
https://devcenter.heroku.com/articles/ios-network-caching-http-headers
Related
Background:
Let's have a WebAssembly (wasm) originating from .net code.
This wasm uses HttpClient and HttpClientHandler to access a backend API at https://api.uri.
The actual backend API location might change in time (like https://api.uri/version-5), but there is still this fixed endpoint, which provides redirection (3xx response) to the current location (which is in the same domain).
The API allows CORS, meaning it sends e.g. Access-Control-Allow-Origin: * headers in the responses.
In the normal (non-wasm) world, one just:
Plainly GETs the https://api.uri with no additional headers (CORS safe).
Retrieve the Location: header (containing e.g. https://api.uri/version-5) from the 3xx response as the final URI.
GETs/POSTs the final URI with additional headers (as needed, e.g. custom, auth, etc.).
Note: In ideal world, the redirection is handled transparently and the first two steps can just be omitted.
Although in the wasm world:
You are not allowed to (let the wasm/browser) send the OPTIONS pre-flight requests to a redirecting endpoint (https://api.uri).
You can't send any non-cors headers, when wanting to prevent pre-flight requests (reason for two stages, plain and full, described above).
You can't see the Location: header value (like https://api.uri/version-5) when trying the manual redirection (HttpClientHandler.AllowAutoRedirect = false), because the response is just artificially crafted with HTTP status code of 0 and ReasonPhrase == "opaqueredirect" - adoption to browser's Fetch API. What a nonsense! #1...
You can't see the auto-followed Location: header value in response.RequestMessage?.RequestUri, when trying the (default) automatic redirection (HttpClientHandler.AllowAutoRedirect = true), because there is still the original URI (https://api.uri) instead of the very expected auto-followed one (https://api.uri/version-5). What a nonsense! #2...
You can't send the full blown request with all the headers and rely on the automatic redirection, because it would trigger pre-flight, which is sill not allowed on redirecting endpoint.
So, the obvious question is:
Is there ANY way, how to handle such simple scenario from the Web Assembly?
(and not crash on CORS)
GET https://api.uri => 3xx, Location: https://api.uri/version-5
GET https://api.uri/version-5, Authorization: Basic BlaBlaBase64= ; Custom: Cool-Value => 200
Note: All this has been discovered within the Uno Platform wasm head, but I believe it applies for any .net wasm.
Note: I also guess "disabled" CORS (on the request side, via Sec-Fetch-Mode: no-cors) wouldn't help either, as then such request is not allowed to have additional headers/methods, right?
I have a main REST web app where I have an endpoint: POST /api/v1/my_endpoint
I want to allow it to be called:
1) via a browser via ajax from my other web apps on different domains
2) from a server side via HTTP client library.
In the case of ajax call - #1 - I'll have to include "Allow-Control-Allow-Origin" and the similar headers to my response to let a browser receive a response.
In the case #2 - I won't need to include those headers.
However, there's no reliable way to distinguish between #1 and #2.
How can I solve that?
On the server, you check for the presence of an Origin header. This header is sent by the browser as part of the CORS protocol. To explain how it works, below is a filter (in other frameworks, this is also known as middleware) used by a Jersey application. Jersey is a Java REST framework. Sorry I don't know Rails. But you should still be able to follow along with this explanation,.
How this filter works is that the ContiainerRequestFilter is called before the backend controller method is called, then the controller method is called, then the ContainerResponseFilter is called. See the commented notes above the methods to see which method are for which.
public class CorsFilter implements ContainerRequestFilter, ContainerResponseFilter {
#Override
// ******************************************
// implementation for ContainerRequestFilter
// ******************************************
public void filter(ContainerRequestContext request) throws IOException {
if (isPreflightRequest(request)) {
request.abortWith(Response.ok().build());
return;
}
}
private static boolean isPreflightRequest(ContainerRequestContext request) {
return request.getHeaderString("Origin") != null
&& request.getMethod().equalsIgnoreCase("OPTIONS");
}
#Override
// ******************************************
// implementation for ContainerResponseFilter
// ******************************************
public void filter(ContainerRequestContext request, ContainerResponseContext response)
throws IOException {
if (request.getHeaderString("Origin") == null) {
return;
}
if (isPreflightRequest(request)) {
response.getHeaders().add("Access-Control-Allow-Credentials", "true");
response.getHeaders().add("Access-Control-Allow-Methods",
"GET, POST, PUT, DELETE, OPTIONS, HEAD");
response.getHeaders().add("Access-Control-Allow-Headers",
"Origin, X-Requested-With, Content-Type, Accept, Authorization, X-CSRF-Token, " +
"Accept-Version, Content-Length, Content-MD5, Date, X-Api-Version, X-File-Name");
}
response.getHeaders().add("Access-Control-Allow-Origin", "*");
}
}
So when the request comes in, the filter checks to see if it is Preflight request. There are two types of cross origin request: a normal one, and preflight. The preflight request occurs before the actual request is made to the server. This is an OPTIONS request where the browser sends some CORS request headers to check with the server to see if the request is allowed. The server should respond back with CORS response headers telling the server that the request is allowed.
So with this filter, is it a preflight request, we abort the request (the controller method will not be called) and then the code execution goes to the ContainerResponseFilter where we set the CORS response headers. You can see in the code, we check the same isPreflightRequest() method.
If the request is not an OPTIONS request, and the Origin header is present, then it is a "normal" cross origin request, in which case, all that is required in in the response headers is Access-Control-Allow-Origin.
Notice that if there is no Origin header, then no CORS headers are added. This is basically how you will differentiate from an AJAX client and other HTTP clients. Only AJAX requests will add the Origin header, and this is done automatically by the browser when it detects a cross origin request.
I hope you understand everything I am talking about, even though the code is Java. It think it's pretty straight forward though, even if you have never used Java. The naming or methods and variables should make it easy to follow. If you understand the flow of the code, then you should pretty much understand the flow of the CORS protocol.
Aside
As an aside, it doesn't hurt if you add the Access-Control-Allow-Origin header on all responses, whether it be an AJAX request or an HTTP client request. If you want to be lazy and implement it this way, it won't hurt. It is required for the AJAX client, but not for the HTTP client. But if you include it in the HTTP client, nobody will die. The world will go on business as usual.
I have a grails 2.2.4 application. I wanted to enable CORS
So I installed cors plugin by having the following line in build config.
plugins {
runtime ':cors:1.1.8'
}
Then in the config.groovy
cors.headers = ['Access-Control-Allow-Origin': '*']
But after this when I run the application, CORS in not enabled. So I debugged the CORS plugin. The issue seems to be in CorsFilter class in the following method
private boolean checkOrigin(HttpServletRequest req, HttpServletResponse resp) {
String origin = req.getHeader("Origin");
if (origin == null) {
//no origin; per W3C spec, terminate further processing for both preflight and actual requests
return false;
}
The origin parameter in the above line is always null as the request does not have the parameter 'Origin'. Is there something i'm doing wrong? I'm not looking for the answer which says add a manual header with the name "Origin" since that is not exactly a proper fix
I'm quite new to CORS so appriciate the help.
In addition to Access-Control-Allow-Origin, and in addition to setting the Origin header on request, you probably need to specify these response headers as well:
Access-Control-Allow-Headers: accept
Access-Control-Allow-Headers: origin
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Method: GET
Access-Control-Allow-Method: POST
Also make sure you respond to HTTP OPTIONS requests with these headers and a blank 200 OK response.
For now, let's assume that RestClient is sending the Origin header properly. It may still be getting stripped by your application. You can prevent this using the Access-Control-Allow-Headers: Origin header.
Most of the problems I have had with my web services is that the right headers are being sent, but they are stripped from the message by my web server. So I tend to adopt a shotgun approach of "allow everything" and then one by one remove what I don't need. My allow-headers header usually is pretty long and I end up having to include stuff like Content-Type, X-Requested-With and other junk before my requests will finally go through.
I further recommend that you test using something besides RestClient, if only as a sanity check. I use Postman, a free Chrome app, for all my messaging tests. It looks to me like the problem is with RestClient not sending the proper Origin header.
In Delphi, I'm using Indy's TIdHTTPWebBrokerBridge coupled with TIdHTTP to send/receive data via HTTP. On the Server, I don't have any fancy handling, I always just respond with a simple content stream. If there's any issues, I only return information about that issue in the response content (such as authentication failed, invalid request, etc.). So, on the client side, can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
I'm wondering because on the client, the requests are wrapped inside functions which return just a boolean for the success of the request.
Inside this function:
IdHTTP.Get(SomeURL, AStream);
Result:= IdHTTP.ResponseCode = 200;
This function handles any and every request which could possibly fetch data. If there were any issues in the request, This function should return False. In my scenario, since I always return some sort of content on the server, would the client always receive a response code of 200 in this function?
I guess the real question is, if I always return some sort of content and handle all exceptions on the server, then will the server always return status code of 200 to each request?
"Does every successful HTTP request always return status code 200?"
See w3.org: HTTP/1.1 Status Code Definitions (RFC 2616)
The answer is No. All 2xx are considered successful.
That may depend on the HTTP method used.
Should your web-server application always return 200 upon success? That may as well depend on the request method and the signal it intends for the client . e.g.
for PUT method (emphasis is mine):
If an existing resource is modified, either the 200 (OK) or 204 (No
Content) response codes SHOULD be sent to indicate successful
completion of the request.
for POST method:
The action performed by the POST method might not result in a resource
that can be identified by a URI. In this case, either 200 (OK) or 204
(No Content) is the appropriate response status, depending on whether
or not the response includes an entity that describes the result.
If a resource has been created on the origin server, the response
SHOULD be 201 (Created) and contain an entity which describes the
status of the request and refers to the new resource, and a Location
header (see section 14.30). Responses to this method are not
cacheable, unless the response includes appropriate Cache-Control or
Expires header fields. However, the 303 (See Other) response can be
used to direct the user agent to retrieve a cacheable resource.
As you can learn from the RCF, every method SHOULD have it's own success status codes, depending on the implementation.
Your other question:
"can I assume that every successful request I make to this server will always have a response code of 200 (OK)?"
You can always expect Status code 200, if your web server always responds with Status 200. Your web server application controls what response it returns to the client.
That said, Status code 200 is the Standard response for successful HTTP requests (The actual response will depend on the request method used), and in the real world of web servers, SHOULD be set as default upon successful request, unless told otherwise (As explained in Remy's answer).
To answer your specific question:
can I assume that every successful request I make to this server will always have a response code of 200 (OK)?
The answer is Yes, because TIdHTTPWebBrokerBridge wraps TIdHTTPServer, which always sets the default response code to 200 for every request, unless you overwrite it with a different value yourself, or have your server do something that implicitly replies with a different response code (like Redirect() which uses 302, or SmartServeFile() which uses 304), or encounter an error that causes TIdHTTPServer to assign a 4xx or 5xx error response code.
However, in general, what others have told you is true. On the client side, you should handle any possible HTTP success response code, not just 200 by itself. Don't make any assumptions about the server implementation.
In fact, TIdHTTP already handles that for you. If TIdHTTP encounters a response code that it considers to be an error code, it will raise an EIdHTTPProtocolException exception into your code. So if you don't get an exception, assume the response is successful. You don't need to check the response code manually.
If there is a particular response code that normally raises an exception but you do not want it to, you can specify that value in the optional AIgnoreReplies parameter of TIdHTTP.Get() or TIdHTTP.DoRequest(). Or, if you are are using an up-to-date Indy 10 SVN revision, a new hoNoProtocolErrorException flag was recently added to the TIdHTTP.HTTPOptions property so the EIdHTTPProtocolException exception is not raised for any response code.
Successful resposes are 2xx List_of_HTTP_status_codes
i did the following. Process straight all 200`s and LOG exceptions. worked, not a single non 200 - except unauthorized and timeouts (password or sometimes unavaliable server). but many/all responses will be considered for a wide range of mainstream apps.
while (iRedo < 3) do begin
s := Self.HTTPComponent.Get( sUrl );
if self.HTTPComponent.ResponseCode = 200 then begin
break;
end;
// IDEIA - log what happend if not 200
logWhatHappend( s, HTTPComponent ); // then log content, headers, etc
inc( iRedo ); sleep( 5 );
end;
I'm implementing a REST API using ASP.NET MVC, and a little stumbling block has come up in the form of the Expect: 100-continue request header for requests with a post body.
RFC 2616 states that:
Upon receiving a request which
includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
This sounds to me like I need to make two responses to the request, i.e. it needs to immediately send a HTTP 100 Continue response, and then continue reading from the original request stream (i.e. HttpContext.Request.InputStream) without ending the request, and then finally sending the resultant status code (for the sake of argument, lets say it's a 204 No Content result).
So, questions are:
Am I reading the specification right, that I need to make two responses to a request?
How can this be done in ASP.NET MVC?
w.r.t. (2) I have tried using the following code before proceeding to read the input stream...
HttpContext.Response.StatusCode = 100;
HttpContext.Response.Flush();
HttpContext.Response.Clear();
...but when I try to set the final 204 status code I get the error:
System.Web.HttpException: Server cannot set status after HTTP headers have been sent.
The .NET framework by default always sends the expect: 100-continue header for every HTTP 1.1 post. This behavior can be programmatically controlled per request via the System.Net.ServicePoint.Expect100Continue property like so:
HttpWebRequest httpReq = GetHttpWebRequestForPost();
httpReq.ServicePoint.Expect100Continue = false;
It can also be globally controlled programmatically:
System.Net.ServicePointManager.Expect100Continue = false;
...or globally through configuration:
<system.net>
<settings>
<servicePointManager expect100Continue="false"/>
</settings>
</system.net>
Thank you Lance Olson and Phil Haack for this info.
100-continue should be handled by IIS. Is there a reason why you want to do this explicitly?
IIS handles the 100.
That said, no it's not two responses. In HTTP, when the Expect: 100-continue comes in as part of the message headers, the client should be waiting until it receives the response before sending the content.
Because of the way asp.net is architected, you have little control over the output stream. Any data that gets written to the stream is automatically put in a 200 response with chunked encoding whenever you flush, be it that you're in buffered mode or not.
Sadly all this stuff is hidden away in internal methods all over the place, and the result is that if you rely on asp.net, as does MVC, you're pretty much unable to bypass it.
Wait till you try and access the input stream in a non-buffered way. A whole load of pain.
Seb