NSURLSession URL Response is not cached when larger than a few kilobytes - ios

I am pretty new to programming in IOS with Cocoa and I am using Swift. I fetch data from a JSON API using an NSURLSession data session with custom delegate, not closures. The reason for using custom delegate is that I have to do basic authentication and I also inject a custom cache-control header to control caching behavior (my API doesn’t include any caching related headers in the response at all).
All this works perfectly but only for requests for which the URLSession:dataTask:didReceiveData: method is called only once. As soon as I get larger responses (some 20-30kBytes) that call the didReceivedData method several times, the URLSession:dataTask:willCacheResponse:completionHandler: method doesn’t get called at all, and therefore my response doesn’t get cached. Re-issuing the same request within 5 minutes will issue a request to the server again, which doesn’t happen for requests whose responses only call didReceiveData one single time. The URLSession:task:didCompleteWithError: method is correctly called and proceeded in all cases.
The documentation of the URLSession:dataTask:willCacheResponse:completionHandler: method (https://developer.apple.com/library/IOs/documentation/Foundation/Reference/NSURLSessionDataDelegate_protocol/index.html#//apple_ref/occ/intfm/NSURLSessionDataDelegate/URLSession:dataTask:willCacheResponse:completionHandler:) says this method is only called if the NSURLProtocol handling the request decides to do so, but I don’t really understand what to do to make this happen.
Any feedback and ideas are very welcome!
This is the code that issues the HTTP request:
let config = NSURLSessionConfiguration.defaultSessionConfiguration()
config.URLCache = NSURLCache.sharedURLCache()
//config.URLCache = NSURLCache(memoryCapacity: 512000000, diskCapacity: 1000000000, diskPath: "urlCache")
let urlString = apiUrlForFilter(filter, withMode: mode, withLimit: limit, withOffset: offset)
let url = NSURL(string: urlString)
var policy: NSURLRequestCachePolicy?
if ignoreCache == true {
policy = .ReloadIgnoringLocalCacheData
} else {
policy = .UseProtocolCachePolicy
}
let request = NSURLRequest(URL: url!, cachePolicy: policy!, timeoutInterval: 20)
let session = NSURLSession(configuration: config, delegate: self, delegateQueue: nil)
let task = session.dataTaskWithRequest(request)
task.resume()
I have the following delegate functions implemented:
URLSession:didReceiveChallenge:completionHandler: to handle SSL certificate trust,
URLSession:task:didReceiveChallenge:completionHandler: to handle basic authentication
URLSession:dataTask:didReceiveResponse:completionHandler: to read some specific headers and save them as instance variables
additionally, the important ones with code:
URLSession:dataTask:didReceiveData: to accumulate data as it arrives for larger HTTP request responses:
func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, didReceiveData data: NSData) {
if receivedData == nil {
receivedData = NSMutableData()
}
receivedData!.appendData(data)
println("did receive data: \(receivedData!.length) bytes")
}
URLSession:dataTask:willCacheResponse:completionHandler: to inject my own Cache-Control header:
func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, willCacheResponse proposedResponse: NSCachedURLResponse, completionHandler: (NSCachedURLResponse!) -> Void) {
println("willCacheResponse was called")
let response: NSURLResponse = proposedResponse.response
let httpResponse = response as NSHTTPURLResponse
var headers = httpResponse.allHeaderFields
var modifiedHeaders = headers
modifiedHeaders.updateValue("max-age=300", forKey: "Cache-Control")
let modifiedResponse = NSHTTPURLResponse(URL: httpResponse.URL!, statusCode: httpResponse.statusCode, HTTPVersion: "HTTP/1.1", headerFields: modifiedHeaders)
let cachedResponse = NSCachedURLResponse(response: modifiedResponse!, data: proposedResponse.data, userInfo: proposedResponse.userInfo, storagePolicy: proposedResponse.storagePolicy)
completionHandler(cachedResponse)
}
URLSession:task:didCompleteWithError: to check the complete response for errors and call a callback closure this class gets by initialization to further proceed the result data:
func URLSession(session: NSURLSession, task: NSURLSessionTask, didCompleteWithError error: NSError?) {
session.finishTasksAndInvalidate()
if error != nil {
var string: String?
if errorString != nil {
string = errorString
} else {
string = Helpers.NSURLErrorDomainErrorForCode(error!.code)
}
errorCallback(string!)
return
}
if receivedData == nil {
errorCallback("the query returned an empty result")
return
}
var jsonError: NSError?
let results: AnyObject! = NSJSONSerialization.JSONObjectWithData(receivedData!, options: NSJSONReadingOptions.AllowFragments, error: nil)
if results == nil {
errorCallback("the data returned was not valid JSON")
return
}
let jsonParsed = JSONValue.fromObject(results)
if let parsedAPIError = jsonParsed!["error"]?.string {
errorCallback("API error: \(parsedAPIError)")
return
}
callback(jsonParsed!, self.serverTime!)
}

Related

How to open HTTP stream on iOS using ndjson

Does anyone have experience opening HTTP stream on iOS? I have tried multiple solutions without any luck (examples bellow).
For better context, here's example of endpoint that will stream values (as ndjson) upon opening connection:
GET /v2/path/{id}
Accept: application/x-ndjson
Attempt #1:
Issue: The completion handler is never called
let keyID = try keyAdapter.getKeyID(for: .signHash)
let url = baseURL.appendingPathComponent("/v2/path/\(keyID)")
var urlRequest = URLRequest(url: url)
urlRequest.httpMethod = "GET"
urlRequest.setValue("application/x-ndjson", forHTTPHeaderField: "Accept")
session.dataTask(with: urlRequest) { data, response, error in
// This never gets called.
// I would expect that the completion is called every time backend emits new value.
}.resume()
Attempt #2:
Issue: Debugger displays this message: Connection 0: encountered error(12:1)
private var stream: URLSessionStreamTask? = nil
func startStream() {
let keyID = try keyAdapter.getKeyID(for: .signHash)
let url = baseURL.appendingPathComponent("/v2/path/\(keyID)")
let stream = session.streamTask(withHostName: url, port: 443)
// Not sure how to set headers.
// Header needs to be set so backend knows client wants to connect a stream.
self.stream = stream
stream.startSecureConnection()
startRead(stream: stream)
}
private func startRead(stream: URLSessionStreamTask) {
stream.readData(ofMinLength: 1, maxLength: 4096, timeout: 120.0) { data, endOfFile, error in
if let error = error {
Logger.shared.log(level: .error, "Reading data from stream failed with error: \(error.localizedDescription)")
} else if let data = data {
Logger.shared.log(level: .error, "Received data from stream (\(data.count)B)")
if !endOfFile {
self.startRead(stream: stream)
} else {
Logger.shared.log(level: .info, "End of file")
}
} else {
Logger.shared.log(level: .error, "Reading stream endup in unspecified state (both data and error are nil).")
}
}
}
Does anyone have experience with this? How can I keep HTTP connection open and listen to a new values that backend is streaming?
iOS can connect to HTTP stream using now deprecated API URLConnection. The API was deprecated in iOS 9, however it's still available for use (and will be in iOS 16 - tested).
First you need to create URLRequest and setup the NSURLConnection:
let url = URL(string: "\(baseURL)/v2/path/\(keyID)")!
var urlRequest = URLRequest(url: url)
urlRequest.setValue("application/x-ndjson", forHTTPHeaderField: "Accept")
let connnection = NSURLConnection(request: urlRequest, delegate: self, startImmediately: true)
connnection?.start()
Notice that the argument for delegate in the code above is of type Any which doesn't help to figure out what protocol(s) to implement. There are two - NSURLConnectionDelegate and NSURLConnectionDataDelegate.
Let's receive data:
public func connection(_ connection: NSURLConnection, didReceive data: Data) {
let string = String(data: data, encoding: .utf8)
Logger.shared.log(level: .debug, "didReceive data:\n\(string ?? "N/A")")
}
Then implement a method for catching errors:
public func connection(_ connection: NSURLConnection, didFailWithError error: Error) {
Logger.shared.log(level: .debug, "didFailWithError: \(error)")
}
And if you have custom SSL pinning, then:
public func connection(_ connection: NSURLConnection, willSendRequestFor challenge: URLAuthenticationChallenge) {
guard let certificate = certificate, let identity = identity else {
Logger.shared.log(level: .info, "No credentials set. Using default handling. (certificate and/or identity are nil)")
challenge.sender?.performDefaultHandling?(for: challenge)
return
}
let credential = URLCredential(identity: identity, certificates: [certificate], persistence: .forSession)
challenge.sender?.use(credential, for: challenge)
}
There is not much info on the internet, so hopefully it will save someone days of trial and error.

what am I doing wrong Swift 3 http request and response

I'm having big problems in a project I'm currently working with.
I've been reading about URLSession various places but all of them seem to be outdated and refers to NSURLSession I thought that they would be fairly similar and they probably are but for a newbie like me I'm lost. what I do is not working and I do not like solutions I find because they all do their work in a controller..
http://swiftdeveloperblog.com/image-upload-with-progress-bar-example-in-swift/
this one for instance. I'm using the PHP script but wanted to make a networking layer I could invoke and use at will. but I'm lacking a good resource from where I could learn about how to use this api.
every place I find is similar to the link above or older. the few newer seem to also follow the pattern without really explaining how to use this api.
at the same time I'm new to the delegate pattern in fact I only know that it is something that is heavily used in this Api but I have no IDEA how or why.
Basically I need help finding my way to solve this problem here:
I've tried to do something like this:
public class NetworkPostRequestor:NSObject,NetworkPostRequestingProtocol,URLSessionTaskDelegate,URLSessionDataDelegate
{
public var _response:HTTPURLResponse
public override init()
{
_response = HTTPURLResponse()
}
public func post(data: Data, url: URL)
{
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("Keep-Alive", forHTTPHeaderField: "Connection")
let configuration = URLSessionConfiguration.default
let session = URLSession(configuration: configuration,delegate: self, delegateQueue: OperationQueue.main)
let task = session.uploadTask(with: request, from: data)
task.resume()
}
public func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: #escaping (URLSession.ResponseDisposition) -> Void)
{
_response = response as! HTTPURLResponse
}
}
however I never even hit the PHPserver. the server when hit will say something like this in the terminal:
[Tue Mar 7 11:43:20 2017] 192.168.250.100:64265 [200]: /
[Tue Mar 7 11:43:20 2017] 192.168.250.100:64266 [404]: /favicon.ico - No such file or directory
Well that is when I hit it with my browser and there is no image with it. but alt least I know that it will write something with the terminal if it hits it. Nothing happens And without a resource to teach me this api I'm afraid I will never learn how to fix this or even if I'm doing something completely wrong.
I'm using Swift 3 and Xcode 8.2.1
Edit:
I've added this method to the class and found that I hit it every single time.
public func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
{
_error = error.debugDescription
}
the debug description have this string "some"
I never used this exact procedure with tasks but rather use the methods with callback. I am not sure if in the background there should be much of a difference though.
So to generate the session (seems pretty close to your):
let configuration = URLSessionConfiguration.default
let session = URLSession(configuration: configuration, delegate: nil, delegateQueue: OperationQueue.main)
Then I generate the request which stupidly enough needs an URL in the constructor:
var request = URLRequest(url: URL(string: "www.nil.com")!) // can't initialize without url
request.url = nil
Adding url with query parameters (you can just set the URL in your case, I have a tool to handle a few cases):
fileprivate func injectQueryParameters(request: inout URLRequest) {
if let query = queryParameters.urlEncodedString {
let toReturn = endpoint.url + "?" + query
if let url = URL(string: toReturn) {
request.url = url
} else {
print("Cannot prepare url: \(toReturn)")
}
} else {
let toReturn = endpoint.url
if let url = URL(string: toReturn) {
request.url = url
} else {
print("Cannot prepare url: \(toReturn)")
}
}
}
Then the form parameters. We mostly use JSON but anything goes here:
fileprivate func injectFormParameters( request: inout URLRequest) {
if let data = rawFormData {
request.httpBody = data
} else if let data = formParameters.urlEncodedString?.data(using: .utf8) {
request.httpBody = data
}
}
And the headers:
fileprivate func injectHeaders(request: inout URLRequest) {
headers._parameters.forEach { (key, value) in
if let stringValue = value as? String {
request.setValue(stringValue, forHTTPHeaderField: key)
}
}
}
So in the end the whole call looks something like:
class func performRequest(request: URLRequest, callback: (([String: Any]?, NSError?) -> Void)?) {
let configuration = URLSessionConfiguration.default
let session = URLSession(configuration: configuration, delegate: nil, delegateQueue: OperationQueue.main)
let task = session.dataTask(with: request) { data, response, error in
// Response is sent here
if let data = data {
callback?((try? JSONSerialization.jsonObject(with: data, options: .allowFragments)) as [String: Any]?, error)
} else {
callback?(nil, error)
}
}
task.resume()
}
I hope this puts you on the right track. In general you do have a few open source libraries you might be interested in. Alamofire is probably still used in most cases.

Download image directly to a photo album

Currently I'm using Alamofire for network requests. How can I download an image directly to a photo album that already exists using the PHPhotoLibrary from Photos.framework?
P.S.: I don't mind to have a solution using NSURLSession by itself.
Considerations: The file can't be stored in disk temporarily. I want the data in memory and save it once into the disk using the Photos.framework.
I figure it out by myself. I implemented it with a dataTaskWithRequest from NSURLSession.
Initially I thought using the NSData(contentsOfURL:) but the contentsOfURL method does not return until the entire file is transferred. It should only be used for local files and not for remote ones because if you're loading a large content and the device has a slow connection then the UI will be blocked.
// Avoid this!
let originalPhotoData = NSData(contentsOfURL: NSURL(string: "http://photo.jpg")!)
So, it is possible to load the content into a NSData with a data task. One solution can be like:
let request = NSMutableURLRequest(URL: NSURL(string: urlString.URLString)!)
request.allHTTPHeaderFields = [
"Content-Type": "image/jpeg"
]
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) { data, response, error in
if let e = error {
print(e.description)
return
}
guard let image = UIImage(data: data) else {
print("No image")
return
}
// Success
// Note: does not save into an album. I removed that code to be more concise.
PHPhotoLibrary.sharedPhotoLibrary().performChanges {
let creationRequest = PHAssetChangeRequest.creationRequestForAssetFromImage(image)
}, completionHandler: { (success : Bool, error : NSError?) -> Void in
if success == false, let e = error){
print(e)
}
}
}
task.resume()
The data is only available in the final stage but you can instead access the data and see the progress using a session delegate.
class Session: NSObject, NSURLSessionDelegate, NSURLSessionTaskDelegate, NSURLSessionDataDelegate {
let configuration = NSURLSessionConfiguration.defaultSessionConfiguration()
let queue = NSOperationQueue.mainQueue()
var session = NSURLSession()
var buffer = NSMutableData()
var expectedContentLength = 0
override init() {
super.init()
session = NSURLSession(configuration: configuration, delegate: self, delegateQueue: queue)
}
func dataTaskWithRequest(request: NSURLRequest) -> NSURLSessionDataTask {
// The delegate is only called if you do not specify a completion block!
return session.dataTaskWithRequest(request)
}
func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, didReceiveResponse response: NSURLResponse, completionHandler: (NSURLSessionResponseDisposition) -> Void) {
buffer = NSMutableData()
expectedContentLength = Int(response.expectedContentLength)
print("Expected content length:", expectedContentLength)
completionHandler(NSURLSessionResponseDisposition.Allow)
}
func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, didReceiveData data: NSData) {
buffer.appendData(data)
let percentageDownloaded = Float(buffer.length) / Float(expectedContentLength)
print("Progress: ", percentageDownloaded)
}
func URLSession(session: NSURLSession, task: NSURLSessionTask, didCompleteWithError error: NSError?) {
print("Finished with/without error \(error)")
// Use the buffer to create the UIImage
}
}

Would a file with a response header `Cache-Control:Private` be prevented from being cached in a NSURLCache?

Would a file with a response header Cache-Control:Private be prevented from being cached in a NSURLCache? Either a shared cache (as in setSharedCache and NSURLCache.sharedCache() ) or a custom one?
To expand, I have UIWebView that I need to access when offline. The source of this WebView has multiple external CSS and JS files associated with it. I can cache most of the site (the CSS, etc looks in place), however it seems to not be caching a specific JavaScript file that provides important information for the site. A difference that I noted between the file that won't cache and the rest is that it's Cache-Control is set to private (the others are public). However, from what I have read, setting the cache control to private is to prevent caching on proxies. Would it affect caching on iOS?
Setting up the cache
func application(application: UIApplication, didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool {
let URLCache: NSURLCache = NSURLCache(memoryCapacity: 10 * 1024 * 1024,
diskCapacity: 50 * 1024 * 1024,
diskPath: nil)
NSURLCache.setSharedURLCache(URLCache)
println("Disk cache usage: \(NSURLCache.sharedURLCache().currentDiskUsage)")
// http://stackoverflow.com/questions/21957378/how-to-cache-using-nsurlsession-and-nsurlcache-not-working
sleep(1)
return true
}
Using the cache
func getWebPage(onCompletion: (NSString, NSURL) -> Void) {
let url = getApplicationSelectorURL()
let request = NSURLRequest(URL: url, cachePolicy: .ReturnCacheDataElseLoad, timeoutInterval: 10.0)
let queue = NSOperationQueue()
NSURLConnection.sendAsynchronousRequest(request, queue: queue, completionHandler: { response, data, error in
println("Web page task completed")
var cachedResponse: NSCachedURLResponse
if (error != nil) {
println("NSURLConnection error: \(error.localizedDescription)")
if let cachedResponse = NSURLCache.sharedURLCache().cachedResponseForRequest(request) {
if let htmlString = NSString(data: cachedResponse.data, encoding: NSUTF8StringEncoding) {
onCompletion(htmlString, url)
} else {
println("htmlString nil")
}
} else {
println("cacheResponse nil")
}
} else {
cachedResponse = NSCachedURLResponse(response: response, data: data, userInfo: nil, storagePolicy: .Allowed)
NSURLCache.sharedURLCache().storeCachedResponse(cachedResponse, forRequest: request)
if let htmlString = NSString(data: data, encoding: NSUTF8StringEncoding) {
onCompletion(htmlString, url)
} else {
println("htmlString nil")
}
}
})
}
Populating the UIWebView
APICommunicator.sharedInstance.getWebPage({ htmlString, url in
dispatch_async(dispatch_get_main_queue(),{
self.webView.loadHTMLString(htmlString, baseURL: url)
})
})
I ended up creating a method similar to the NSURLConnectionDelegate method willCacheResponse, and replacing the Cache-Control:private header.
willCacheResponse method
func willCacheResponse(cachedResponse: NSCachedURLResponse) -> NSCachedURLResponse?
{
let response = cachedResponse.response
let HTTPresponse: NSHTTPURLResponse = response as NSHTTPURLResponse
let headers: NSDictionary = HTTPresponse.allHeaderFields
var modifiedHeaders: NSMutableDictionary = headers.mutableCopy() as NSMutableDictionary
modifiedHeaders["Cache-Control"] = "max-age=604800"
let modifiedResponse: NSHTTPURLResponse = NSHTTPURLResponse(
URL: HTTPresponse.URL!,
statusCode: HTTPresponse.statusCode,
HTTPVersion: "HTTP/1.1",
headerFields: modifiedHeaders)!
let modifiedCachedResponse = NSCachedURLResponse(
response: modifiedResponse,
data: cachedResponse.data,
userInfo: cachedResponse.userInfo,
storagePolicy: cachedResponse.storagePolicy)
return modifiedCachedResponse
}
calling method
if let cachedResponse = self.willCacheResponse(
NSCachedURLResponse(response: response,
data: data,
userInfo: nil,
storagePolicy: .Allowed)) {
NSURLCache.sharedURLCache().storeCachedResponse(cachedResponse, forRequest: request)
}
And now it displays correctly when offline. What a journey.
Yes, responses with the private cache control policy are not being cached by the NSURLCache. The RFC #2616 says
private:
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the
response are intended for only one user and are not a valid response for requests by other users. A private (non-shared) cache MAY cache the response.
Well, NSURLCache uses sharedCache which you even set up in the code you posted. I guess it explains pretty much everything.
Solution is either to change the server behaviour, or to override some methods of the NSURLCache class. (You can e.g. rewrite the header client-side, but this should be quite an awful hack.)

Swift - downcast NSURLResponse to NSHTTPURLResponse in order to get response code

I'm building rest queries in SWIFT using NSURLRequest
var request : NSURLRequest = NSURLRequest(URL: url)
var connection : NSURLConnection = NSURLConnection(request: request, delegate: self, startImmediately: false)!
connection.start()
My question is how to do i get response code out of the response that is returned:
func connection(didReceiveResponse: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
//...
}
According to Apple: NSHTTPURLResponse which is a subclass of NSURLResponse has a status code but I'm not sure how to downcast my response object so i can see the response code.
This doesn't seem to cut it:
println((NSHTTPURLResponse)response.statusCode)
Thanks
Use an optional cast (as?) with optional binding (if let):
func connection(didReceiveResponse: NSURLConnection!, didReceiveResponse response: NSURLResponse!) {
if let httpResponse = response as? NSHTTPURLResponse {
println(httpResponse.statusCode)
} else {
assertionFailure("unexpected response")
}
}
or as a one-liner
let statusCode = (response as? NSHTTPURLResponse)?.statusCode ?? -1
where the status code would be set to -1 if the response is not an HTTP response
(which should not happen for an HTTP request).

Resources