Is it possible to force response caching if it contains neither Expires or Cache-Control: max-age?
I've came across this article, but unfortunately URLSession:dataTask:willCacheResponse:completionHandler: never gets called in my AFHTTPSessionManager subclass.
Any help appreciated.
You can force the caching by implementing your own NSURLProtocol that does not follow the standard HTTP caching rules. A complete tutorial is here, which persists the data using Core Data, but the basic steps are:
Subclass NSURLProtocol
Register your subclass with +registerClass:
Return YES in your +canInitWithRequest: method if this is the first time you've seen request, or NO if it isn't
You now have two choices:
Implement your own cache storage (in which case, follow the tutorial linked above)
Inject the cache control headers that you wish the URL loading system to follow
Assuming you want #2, override connection:didReceiveResponse: in your protocol subclass to create a response that has the cache control headers you want to emulate:
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSHTTPURLResponse *)response {
// Create a dictionary with the headers you want
NSMutableDictionary *newHeaders = [response.allHeaderFields mutableCopy];
newHeaders[#"Cache-Control"] = #"no-transform,public,max-age=300,s-maxage=900";
// Create a new response
NSHTTPURLResponse *newResponse = [[NSHTTPURLResponse alloc] initWithURL:response.URL
statusCode:response.statusCode
HTTPVersion:#"HTTP/1.1"
headerFields:newHeaders];
[self.client URLProtocol:self
didReceiveResponse:newResponse
cacheStoragePolicy:NSURLCacheStorageAllowed];
}
This will cause the response to be cached as if the server had provided these headers.
For URL sessions only, you need to set the session configuration's protocolClasses. Since you're using AFNetworking, that looks like:
[AFHTTPSessionManager sharedManager].session.configuration.protocolClasses = #[[MyURLProtocol class]]
There are some caveats, so make sure you read the protocolClasses documentation.
A few notes:
If there's any way to fix this by having your server send the appropriate headers, please, please do that instead.
For the sake of brevity I hardcoded "HTTP/1.1", but technically you should pull this out of the response.
AFNetworking uses the standard URL Loading System, and is mostly unrelated to this issue.
Related
I would like to determine if the response from NSURLSessionDataTask came from cache, or was served from server
I'am creating my NSURLSessionDataTask from
request.cachePolicy = NSURLRequestUseProtocolCachePolicy;
Two easy options come to mind:
Call [[NSURLCache sharedURLCache] cachedResponseForRequest:request] before you make the request, store the cached response, then do that again after you finish receiving data, and compare the two cached responses to see if they are the same.
Make an initial request with the NSURLRequestReturnCacheDataDontLoad cache policy, and if that fails, make a second request with a more sane policy.
The first approach is usually preferable, as the second approach will return data that exists in the cache even if it is stale. However, in some rare cases (e.g. an offline mode), that might be what you want, which is why I mentioned it.
In order to know whether an URLSessionDataTask response comes from the cache or the network, you have to use a custom URLSession and provide it with an URLSessionTaskDelegate which has to implement the following method:
func urlSession(_ session: URLSession, task: URLSessionTask, didFinishCollecting metrics: URLSessionTaskMetrics)
You'll find very useful information in metrics, especially the list of transactions metrics for the request. Each transaction metrics has a property resourceFetchType which can either be .localCache, .networklLoad, .serverPush or .unknown.
More information here: Apple documentation regarding URLSessionTaskDelegate
If your information need is only out of curiosity you can view your network usage the Xcode runtime network metics. Change the policy to a different setting and observe the difference.
If you want to use caching then it's not a good answer to disable caching on the client.
if the server has set cache headers (etag, cache-control="no-store") then NSURLSession will revalidate and serve you cached / fresh response based on the 200 / 304 response from the server. However in your code you will always see statusCode 200 regardless of if NSUrlSession received 200 or 304. This is limiting, because you may want to skip parsing, re-creating objects etc when the response hasnt changed.
THe workaround I did, is to use the etag value in the response to determine if something has changed
NSHTTPURLResponse *httpResp = (NSHTTPURLResponse *)response;
NSString *etag = (httpResp && [httpResp isKindOfClass:[NSHTTPURLResponse class]]) ? httpResp.allHeaderFields[#"Etag"] : nil;
BOOL somethingHasChanged = [etag isEqualToString:oldEtag];
Our application makes use of RESTful service calls using NSURLSession. The calls themselves are routed through a reverse proxy, to aide in session management, security, etc. on the backend. The only problem we're having is related to authentication. When a user attempts to access a protected resource -- either through a browser or a REST call -- and they are not authenticated, the reverse proxy displays an HTML login page.
The problem is, we'd like to leverage NSURLSession's ability to handle authentication challenges automatically. However, NSURLSession is unable to recognize that we're getting back an authentication challenge, because no HTTP 401 or other error code is being returned to the client. The reverse proxy sends back a 200, because the HTML was delivered successfully. We're able to recognize that it is the login page by inspecting the HTML within the client, but we'd like to be able to handle the authentication using a custom NSURLAuthenticationChallenge, if at all possible, so that NSURLSession can retry requests after authentication is successful.
Is there any way for us to recognize that we're getting back a login page within our completionHandler and tell NSURLSession that we're really getting back an authentication challenge? Or does NSURLSession require that we receive back an HTTP error code (401, etc.)
A couple of possibilities come to mind.
If you're using the delegate rendition of NSURLSession, you might be able to detect the redirect to the authentication failure page via NSURLSessionTaskDelegate method willPerformHTTPRedirection.
You can probably also detect the redirect by examining the task's currentRequest.URL and see if a redirect happened there, too. As the documentation says, currentRequest "is typically the same as the initial request (originalRequest) except when the server has responded to the initial request with a redirect to a different URL."
Assuming that the RESTful service generally would not be returning HTML, you can look at the NSURLResponse, confirm that it's really a NSHTTPURLResponse subclass, and then look at allHeaderFields to confirm whether text/html appears in the Content-Type of the response. This obviously only works if the authentication page returns a Content-Type that includes text/html and the rest of your responses don't (e.g. they're application/json or something like that).
Anyway, that might look like:
NSString *contentType = [self headerValueForKey:#"Content-Type" response:response];
if ([contentType rangeOfString:#"text/html"].location != NSNotFound) {
// handle it here
}
Where, headerValueForKey might be defined as follows:
- (NSString *)headerValueForKey:(NSString *)searchKey response:(NSURLResponse *)response
{
if (![response isKindOfClass:[NSHTTPURLResponse class]])
return nil;
NSDictionary *headers = [(NSHTTPURLResponse *) response allHeaderFields];
for (NSString *key in headers) {
if ([searchKey caseInsensitiveCompare:key] == NSOrderedSame) {
return headers[key];
}
};
return nil;
}
At worst, you could detect the HTML response and parse it using something like HPPLE and programmatically detect the authentication HTML response.
See Wenderlich's article How to Parse HTML on iOS.
Frankly, though, I would prefer to see a REST response to report the authentication error with a REST response (or a proper authentication failure challenge or a non 200 response) rather than a redirect to a HTML page. If you have an opportunity to change the server response, that would be my personal preference.
Authentication challenges are triggered by the WWW-Authenticate header in a 40x response.
From the documentation :
The URL loading system classes do not call their delegates to handle request challenges unless the server response contains a WWW-Authenticate header. Other authentication types, such as proxy authentication and TLS trust validation do not require this header.
Unfortunately the proxy authentication referred to above does not apply in your reverse proxy/accelerator scenario.
Have you tried simply setting the username and password to the authorization header of NSMutableURLRequest as such.
NSString *authorizationString = [[[NSString stringWithFormat:#"%#:%#",user, password] dataUsingEncoding:NSUTF8StringEncoding] base64Encoding];
[yourRequest setValue:[NSString stringWithFormat:#"Basic %#", authorizationString] forHTTPHeaderField:#"Authorization"];
This should work.
You could try using an HTML Parser library (like https://github.com/nolanw/HTMLReader).
Then in the completion handler block, you can evaluate the response to check if it is an HTML page, and if it contains a login form.
I don't know this library, but I guess you can use it like this:
HTMLDocument *document = [HTMLDocument documentWithString:responseObjectFromServer];
HTMLNode *node = [document firstNodeMatchingSelector:#"form.login"];
if(node) {
// Perform second NSURLSession call with login data
}
I'm hoping someone can shed some light on a few things I've been researching but not making much progress on.
I'd like to take advantage of NSURLCache to return cached responses for API calls that I make within an iOS App. When the device is online, I'd like to return the cached response if it's recent enough, otherwise fetch from remote. When the device is offline I'd like to immediately return the cached response (if any) regardless of it's age.
I'm using AFNetworking. The API calls that I'm making are to a server I control. Protocol is HTTPS. The Cache-Control response header is currently "max-age=0, public". I'm not currently setting cache related request headers (should I?). I set the request's cache policy to be NSURLRequestReturnCacheDataDontLoad when offline and use the default NSURLRequestUseProtocolCachePolicy when online. And I can see the requests and their responses in the default Cache.db on disk. However when I'm offline all requests fail (no cached responses (despite appearing to have been cached) are being used/returned.
Per http://nshipster.com/nsurlcache/ I initialize a sharedURLCache in didFinishLaunchingWithOptions and set the AFNetworking setCacheResponse block to something like this:
NSMutableDictionary *mutableUserInfo = [[cachedResponse userInfo] mutableCopy];
NSMutableData *mutableData = [[cachedResponse data] mutableCopy];
NSURLCacheStoragePolicy storagePolicy = NSURLCacheStorageAllowed;
return [[NSCachedURLResponse alloc] initWithResponse:[cachedResponse response] data:mutableData userInfo:mutableUserInfo storagePolicy:storagePolicy];
I've read these and other posts on the topic:
http://petersteinberger.com/blog/2012/nsurlcache-uses-a-disk-cache-as-of-ios5/
http://blackpixel.com/blog/2012/05/caching-and-nsurlconnection.html
I'm wondering if anyone out there has successfully achieved this functionality before using the standard NSURLCache (also interested in success stories involving SDURLCache but Peter S. says as of iOS5 disk caching is supported therefore SDURLCache is no longer needed, ie I'd like to use the default built in cache).
Thanks in advance!
Did you see this post?
AFNetworking (AFHttpClient) offline mode not working with NSURLRequestReturnCacheDataDontLoad policy
Looks like it might be a bug with iOS 6. Here is what Robert Mao had to say in the post:
A summarized work around is read the response from cache, instead of
use NSURLRequestReturnCacheDataDontLoad policy:
NSCachedURLResponse *cachedResponse = [[NSURLCache sharedURLCache] cachedResponseForRequest:request];
if (cachedResponse != nil &&
[[cachedResponse data] length] > 0)
{
// Get cached data
....
}
Unless all of your calls are 100% GET and free of side effects or time dependency then this is dangerous.
I'm loading certain images from a certain server asynchronously. I'm firing a number of requests at once using NSURLConnection connectionWithRequest method and receive the data using NSURLConnectionDelegate didReceiveData.
At didReceiveData, how do I know which request this data matches? At didReceiveResponse I can use the URL method of the response given as a parameter, but in didReceiveData I only have the received data.
It seemed like the perfect answer would be to use NSURLConnection sendAsynchronousRequest, as the completion handler has all the required parameters: (NSURLResponse*, NSData*, NSError*). I can use [response URL] to make a match to the original request... except in one case: not all the images I try to download exist. In that case, the request is redirected to a generic error page and the URL of that generic page is received, so I can't match the response to a request I've made. This I could handle with connectionWithRequest, but I can't get the best of both worlds.
In
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
you can use
NSURLRequest *request = [connection originalRequest];
to get the request that the connection was started with.
(This method is available since iOS 5.0, but I could not find it in my Xcode iOS 5.1 Library. You find it in the iOS 6 Library, in the header file NSURLConnection.h or here: http://developer.apple.com/library/ios/#documentation/Cocoa/Reference/Foundation/Classes/NSURLConnection_Class/Reference/Reference.html#//apple_ref/occ/instm/NSURLConnection/originalRequest).
More methods to manage multiple connections can be found in this thread: Managing multiple asynchronous NSURLConnection connections.
If you use sendAsynchronousRequest:queue:completionHandler: then you can just use the request parameter in the completion block.
At connection:didReceiveData: the first parameter is the NSURLConnection instance. So I don't understand where the problem is. You create a connection, then you send a request to that connection and the delegate receive the connection:didReceiveData with the connection value.
If you are using the same delegate for all the request you have to check the connection so you can say which request is associated to.
Perhaps you have to maintain a table of connection/request pairs.
I have a tried and tested use of NSURLRequest (and accompaniments) implementation, that works great for GETs, and POSTs for a given URL.
However, I want to now move the target of the URL without changing the URL used by the app, so I'm intending to use a webhop redirect via my DNS provider.
This works fine for the GET requests, but the POSTs just hang... no connection response is received.
The relevant iOS method for handling a redirect is,
-(NSURLRequest *)connection:(NSURLConnection *)connection
willSendRequest:(NSURLRequest *)request
redirectResponse:(NSURLResponse *)redirectResponse
According to Apple's documentation for (handling redirects),
If the delegate doesn't implement connection:willSendRequest:redirectResponse:, all canonical changes and server redirects are allowed.
Well, that's not my experience, because leaving this method out does not work for me. The request just hangs without a response.
Apple also suggests an implementation, of willSendRequest (see above linked Apple documentation), again this doesn't work for me. I see the invocations, but the resulting requests just hang.
My current implementation of willSendRequest is as follows (see below). This follows the redirect, but the handles the request as if it was a GET, rather than a POST.
I believe the problem is that the redirection is losing the fact that the HTTP request is a POST (there may be more problems, such as carrying the request Body forward too?).
I'm not sure what I should be doing here. So any advice on how to correctly handle a POST that receives a redirect would be appreciated. Thanks.
-(NSURLRequest *)connection:(NSURLConnection *)connection
willSendRequest:(NSURLRequest *)request
redirectResponse:(NSURLResponse *)redirectResponse
{
NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *) redirectResponse;
int statusCode = [httpResponse statusCode];
NSLog (#"HTTP status %d", statusCode);
// http statuscodes between 300 & 400 is a redirect ...
if (httpResponse && statusCode >= 300 && statusCode < 400)
{
NSLog(#"willSendRequest (from %# to %#)", redirectResponse.URL, request.URL);
}
if (redirectResponse)
{
NSMutableURLRequest *newRequest = [request mutableCopy]; // original request
[newRequest setURL: [request URL]];
NSLog (#"redirected");
return newRequest;
}
else
{
NSLog (#"original");
return request;
}
}
ADDITIONAL INFORMATION 1
The HTTP code received by willSendRequest is 301 - 'Moved Permanently.
Using allHTTPHeaderFields to extract the header fields, I see that he request I originally submit has the header
HTTP header {
"Content-Length" = 244;
"Content-Type" = "application/json";
}
...and the copied / redirected request has the header,
Redirect HTTP header {
Accept = "*/*";
"Accept-Encoding" = "gzip, deflate";
"Accept-Language" = "en-us";
"Content-Type" = "application/json";
}
...which doesn't look like a copy of the original request, or even a superset.
Keep your original request, then provide your own willSendRequest:redirectResponse: to customize that request, rather than working with the one Apple provides you.
- (NSURLRequest *)connection: (NSURLConnection *)connection
willSendRequest: (NSURLRequest *)request
redirectResponse: (NSURLResponse *)redirectResponse;
{
if (redirectResponse) {
// The request you initialized the connection with should be kept as
// _originalRequest.
// Instead of trying to merge the pieces of _originalRequest into Cocoa
// touch's proposed redirect request, we make a mutable copy of the
// original request, change the URL to match that of the proposed
// request, and return it as the request to use.
//
NSMutableURLRequest *r = [_originalRequest mutableCopy];
[r setURL: [request URL]];
return r;
} else {
return request;
}
}
By doing this, you're explicitly ignoring some aspects of the HTTP spec: Redirects should generally be turned into GET requests (depending on the HTTP status code). But in practice, this behaviour will serve you better when POSTing from an iOS application.
See also:
iOS Developer Library, URL Loading System Programming Guide: Handling Redirects and Other Request Changes
The HTTP specification for handling the 3xx class of status codes is very unfriendly towards protocols other than GET and HEAD. It expects some kind of user interaction for at the intermediary step of the redirection, which has lead to a plethora of incompatible client and server implementations, as well as a serious headache for web service developers.
From an iOS NSURL point of view, one of the things you might want to verify is that the original POST body is included in the new, redirect request.
Based off your comments on my original answer, and the edits to your question, it would appear that the URL you are trying to access has been updated permanently (301 status code). In which case you can actually avoid the redirects altogether by using the new URL.