iOS NSURLCache making up HTTP 404 when going through NSURLProtocol - ios

If you are serious about answering this question, please clone the mini app i have created and see if it misbehaves for you the same way, before speculating about the answer, thank you :)
https://github.com/pavel-zdenek/nsurlprotocol-testbed
A very simple browser: UIWebView with all requests going through bare bones NSURLProtocol reimplementation. It elapses the loading time. The switch un/registers the protocol handler at runtime. Now load one specific site: http://www.rollingstone.com. Besides tons of other resources, the page GETs
http://sjc.ads.nexage.com/js/admax/admax_api.js
which produces an XHR request for
http://sjc.mediation.nexage.net/admax/adMaxApi.do?dcn=<somehash>
PZProtocolHandler OFF: XHR loads instantly. Observed as HTTP 200 on the wire(shark), reflected as successful load in Safari Web Inspector. On further loads, it's not on the wire anymore but still HTTP 200 in Web Inspector. In other words, cached.
PZProtocolHandler ON: Web Inspector reports 3 tries, timing out after 10s each, resulting in HTTP 404. No change with further reloads. Now the interesting part: according to wire(shark), the request goes out and is responded with proper HTTP 200. Furthermore, all 3 tries are sequenced on one and the same TCP channel. "Someone" is making up 404 along the way to UIWebView.
Now, even before reading clever blogs, i found out that iOS URL caching goes thermonuclear when the server response does not contain Cache-Control header. And this particular GET request is such case. But it should not result in 404 failure, should it? Also i have tried to hibernate the default caching through various methods: implementing caching-related delegate calls in NSURLProtocol, creating new request with cache prohibiting flags, using expectedly top knowledge in SDURLCache, even replacing NSURLCache with what i think should be a "null cache". See everything in the project sources.
The 404 persists for me. I doubt that the mere fact of overriding NSURLProtocol would break things so much. I could live without an opportunity to fill a radar for that, thank you. I still hope that i am doing something wrong.

The issue isn't exactly solved, more like narrowed and cleared a little bit. If a new question develops, it will be most probably very different, so i'm self-answering this one.
So it probably isn't a cache issue. The site loads normally when explicit NSOperationQueue is assigned:
_connection = [[NSURLConnection alloc] initWithRequest:request
delegate:self
startImmediately:NO];
// comment out for sure stalling
[_connection setDelegateQueue:[NSOperationQueue mainQueue]];
[_connection start];
I would be tempted to prefer currentQueue, but that's nil at the moment of [NSURLProtocol startLoading]. Which according to Apple doc should mean that startLoading isn't a NSOperation itself, hence there is no queue.
Explicit routing to main thread obviously brings a different performance bottleneck: the delegate calls are stalled when UI is being interacted with, most exemplary when partially loaded page is attempted to be scrolled. While this is also an issue, it's something specific i can fill Apple TSI with. Any new knowledge will go to the github repo.
UPDATE:
Now the issue is fully solved by doing yet another simple thing: create a new vanilla NSOperationQueue and set the NSURLConnectionDelegate queue to it:
[_connection setDelegateQueue:_theNewlyCreatedQueue];
You aren't going to see this in any of the numerous NSURLProtocol howtos on the internets. It would be inappropriately self confident to claim that i know why it improves things so much, but now the offending website loads almost as smoothly as in Mobile Safari (within expected JavaScript execution speed difference) so the problem is solved for me.

Related

App architecture when 'state changing' APNS fails

I've seen several questions on this topic. But all simply say you just have to recover from other means. But none explain what the other means are! I couldn't find an answer on SO. This is also a follow up from the comments of this question.
Let's say I'm working on a Uber app. Drivers need to know passenger locations.
A passenger sets a pickup location for 123 XYZStreet.
2 minutes later she decides to cancel the entire pickup. So now I need
to inform the driver. This is an important state changing update.
The first thought that comes to mind is:
Send a notification that has content-available:1 so I can update the app as soon as the notification arrives, and in the didReceiveNotification I call GET(PassengerInfoModel) and also have include "alert" : "Pickup has been canceled. Don't go there' So the driver would also be visually informed. Obviously tapping on the notification is not what manages the updates. The content-available being set to 1 will manage that.
But doing that, still what happens when the arrival of that notification fails—completely? Well then the latest GET(PassengerInfoModel) won't happen. As a solution I've heard of a HEAD request:
The HEAD method is identical to GET except that the server MUST NOT
return a message-body in the response. The metainformation contained
in the HTTP headers in response to a HEAD request SHOULD be identical
to the information sent in response to a GET request. This method can
be used for obtaining metainformation about the entity implied by the
request without transferring the entity-body itself. This method is
often used for testing hypertext links for validity, accessibility,
and recent modification.
Not sure what happens if using a HEAD request we figured out that there was an update!? Do we then make a GET request in the success case of the HEAD's completion Handler?
Question1 How should we handle the HEAD request response? (I'm guessing that for the server to be able to route HEAD requests, there must be some changes, but let's just assume that's outside the scope of the question).
Question2 How often do we have to do this request? Based on this comment one solution could be to set a repeating timer in the viewDidAppear e.g. make a HEAD request every 2 minutes. Is that a good idea?
Question3 Now let's say we did that HEAD request, but the GET(PassengerInfoModel) is requested from 2 other scenes/viewControllers as well. The server can't differentiate between the different scenes/viewControllers. I'm guessing a solution would be have all our app's network requests managed through a singleton NetworkHandler. Is that a good idea?
I understand that this question is broad, but believe the issue needs to be addressed as a whole
Question1 How should we handle the HEAD request response? (I'm guessing that for the server to be able to route HEAD requests, there must be some changes, but let's just assume that's outside the scope of the question).
You probably don't need to deal with HEAD requests. Using Etags is a standard mechanism which lets you make a GET request and the server can just return an empty body with 304 response if nothing has changed, or the actual new content if something has.
Question2 How often do we have to do this request? Based on this comment one solution could be to set a repeating timer in the viewDidAppear e.g. make a HEAD request every 2 minutes. Is that a good idea?
I think this is reasonable, especially if you want to inform your user when you are unable to make that request successfully. You might also consider using Apple's Reachability code to detect when you can or cannot talk to your server.
Question3 Now let's say we did that HEAD request, but the GET(PassengerInfoModel) is requested from 2 other scenes/viewControllers as well. The server can't differentiate between the different scenes/viewControllers. I'm guessing a solution would be have all our app's network requests managed through a singleton NetworkHandler. Is that a good idea?
Yes, I think having a singleton is reasonable, though I'm not sure why the server cares what view controller is making the request. Like can't they just request different urls?

NSURLProtocol fails for video request

I'm having some issues with video requests handled through a special protocol scheme in a NSURLProtocol subclass. Every other resource (images/text) are getting handled correctly, however, when a video request is sent, I only get a call to 'canInitWithRequest' and no follow up. So, my video resource doesn't get resolved. Now, I've looked around and I found no definite solution for this. Some people use instead an embedded HTTP server, but that seems an overkill. Does anyone know if this is a bug or if not, why is this limitation, is there an workaround for it?
A similar issue: Custom NSURLProtocol class for WebView doesn't work when loading video in HTML5 document , but unfortunately without an answer.
#Meda, I was facing the similar issue. Here what I found and hope it is useful to you.
I assume that you are using NSUrlProtocol because you want to intercept the video requests.
I was using web view which makes request for video over HTTP. It goes to NSURLProtocol and makes the request. When it receives the data, webView loads the video rendering plugin (looking at the mime type in HTTP header). The plugin needs the data to come as Partial HTTP response (Response code 206). Further, the plugin does not use NSURLProtocol class but uses network layer below it. So requests that plugin makes, do not go thru NSURLProtocol. Considering this, there could be 2 problems in your case.
1. HTTP server you are using may not be supporting partial responses.
2. HTTP server is not reachable directly (Can you access the video from safari or any other
browser on your device?)
You can verify both the cases by taking network trace. use tcpdump (available on Mac) to take network trace and see what is happening there.

NSURLRequest with App Level ProxyHost/ProxyPort Settings

I'm looking for a way to use NSMutableURLRequest with app level proxyHost/Port settings, essentially a replacement for ASIHTTPRequest lib with proxyHost/proxyPort. I've tried modifying the CFReadStream (from NSURLRequest HTTPBodyStream), but it SIGSEGs when setting the proxy settings. I would rather not have to rewrite my app with CFNetworking, and it looks like AFNetwork lib doesn't include this feature yet either.
Has anyone successfully done this with NSMutableURLRequest?
The real answer appears to be creating a custom NSURLProtocol. Should be a straight forward derivation, and add the appropriate proxyHost/proxyPort to the request (along with any other values such as a customized User Agent string). Then, supposedly, all requests will be routed through this custom protocol (including UIWebView requests..direct or derived).
relevant samples:
https://developer.apple.com/library/ios/samplecode/CustomHTTPProtocol/Introduction/Intro.html#//apple_ref/doc/uid/DTS40013653
http://eng.42go.com/customizing-uiwebview-requests-with-nsurlprotocol/
I'll post more when I have the thing operational.
Things of note with this implementation.
Initially started using CFNetworking as the "wedge" in my custom
NSURLPRotocol, but quickly found I was rewriting the same code that
was in ASIHTTPRequest. So I just implemented the wedge with
ASIHTTPRequest.
The items that are not documented well (or at all), is the
interaction of UIWebView with NSURLProtocol callbacks, vs
NSURLRequest/Conenction with NSURLProtocol. Some Findings:
a) All dependent page resources are loaded automatically by UIWebView
(which we knew), and they all go through NSURLProtocol, so it is an
excellent place to put in code to modify all requests.
b) The UIWebView sets the Referer header. On a redirect, the only
way to get the UIWebView to update it's Referer from the original URL
to the new redirect URL is with the [[self client] URLProtocol:self
wasRedirectedToRequest:redirectRequest
redirectResponse:tmpHttpResponse]; callback.
c) when the above redirect callback is received by UIWebView, it
generates a new NSURLRequest (essentially the one you sent back to
it). So if you have a wedge that likes to do the redirect
internally, you have to cancel the Request that it attempts to make,
in favor of the new one from UIWebView.
You have to be careful with which callbacks you implement from ASIHTTPRequestDelegate. e.g. implementing didReceiveData will disable the built in gzip processing. willRedirectToURL disables most of the built in redirect processing (even if you call [request redirectToURL:newURL]; as recommended in the comments).

How to check if UIWebView loads from NSURLCache?

I actually have more than one question regarding this... I'm still new to NSURLCache and UIWebViews but somehow got the basics down I think.
Anyway, I was able to successfully implement Caching for my UIWebView(I think). I initialized the NSURLCache. My UIWebView loads its contents with the loadRequest: method. I assigned the ViewController that owns the UIWebView as it's delegate. I also had the ViewController implement NSURLConnectionDataDelegate. I implemented the connection:willCacheResponse: method just to log a message in the console so I could check that caching was done. I also logged the current disk and memory usage. It increases after the caching so I can see that it did work.
However, when I load the page again, I think the request is sent again to the server because the connection delegate methods like connection:didReceiveResponse: and connection:willCacheResponse are being called.
I also wanted to try getting the cachedResponse from the cache by using cachedResponseForRequest:. I called this method in webViewDidFinishLoad: but it always returns null. Did caching really work?
How can you tell if the WebView loaded from the cache? Am I misusing cachedResponseForRequest:? And on another note, what proper Control-Cache header values should be present in the very least for caching? I'm testing on google's homepage URL and it returns the private value for its Cache Control header but I can see that caching works because the connection:willCacheResponse: is called.
Help please?
If you run a HTTP proxy like Charles or Fiddler you can keep tabs on what network traffic is actually going out over the wire(less).
You can also inspect the request/response headers to see whether the response is indeed destined for the cache.
Note that in some cases you might see a conditional-GET (returns a 304) meaning the client asked for content that it has in its cache, but only wants to download the content if the server has a more recent version.

RestKit handling different HTTP status codes

I've just started to try out RestKit for an iOS app i'm building. I normally use ASIHttpRequest, but I want to test out RestKit mostly for its object mapping between JSON and CoreData. There are some great things about RestKit, but I've run into an issue that really makes it feel deficient, unless I'm doing something wrong or have missed something. I hope someone here can guide me on that.
I'm using RKObjectLoader to make async & sync calls to a REST API. My service is designed to send back proper HTTP status codes, along with some sort of description, a 401 being an example of when the API needs an authenticated user.
My problem is that RestKit stops acting normally if i get a 401 error back. The RKResponse object has a status code of 0, even though it has a payload in it. I'm pretty sure this comes down to NSURLConnection's poor handling of HTTP statuses, but I would expect RestKit to wrap around this somehow. Especially since the RKResponse class has quite a few wrapper functions to determine the status code of the response (isOK, isCreated, isForbidden, isUnauthorized, etc.).
In comparison, ASIHttpRequest doesn't use NSURLConnection, but instead uses the lower level CFNetwork code. ASIHttpRequest allows me to see exactly what came back over HTTP without sending out errors left & right.
Question is, am I doing something wrong, or is this the expected behavior out of RestKit? Has anyone successfully been able to make a calls to [RKResponse isAuthenticated]? Although its inconclusive to me, is there any difference between running in async and sync mode in this regard. I did read somewhere that NSURLConnection run in sync mode will act a bit differently, even though the underlying code is just calling the async operations. Does this have more to do with me using RKObjectLoader as opposed to just RKRequest? Perhaps the fact that the payload can't map to a model causes anger, but it seems that the code is breaking earlier within RKRequest.sendSynchronously, prior to when mapping actually takes place.
Bottom line is my code needs to be able to freely read HTTP status codes. Any guidance would be most appreciated.
Haider
The common way for RestKit 0.20.x is to subclass RKObjectRequestOperation.
I wrote a blog article about this problem which can be found here:
http://blog.higgsboson.tk/2013/09/03/global-request-management-with-restkit/
See http://groups.google.com/group/restkit/msg/839b84452f4b3e26
"... when authentication fails, the authentication challenge gets cancelled and that effectively voids the request."
UPDATE:
RestKit already includes a delegate method for this:
(void)request:(RKRequest *)request didFailAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge;
Triggers before
(void)objectLoader:(RKObjectLoader *)objectLoader didFailWithError:(NSError *)error
When HTTP Basic authentication fails, so we can use this instead.

Resources