The example below from Apple documentation, shows how manually create a CachedURLResponse for a UrlSession.
One of the parameters is storagePolicy (in the example set to .allowedInMemoryOnly).
If set such parameter to .allowed (instead of .allowedInMemoryOnly, the cache should be stored to hard disk. Does this mean that it persists in-between app launches?
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask,
willCacheResponse proposedResponse: CachedURLResponse,
completionHandler: #escaping (CachedURLResponse?) -> Void) {
if proposedResponse.response.url?.scheme == "https" {
let updatedResponse = CachedURLResponse(response: proposedResponse.response,
data: proposedResponse.data,
userInfo: proposedResponse.userInfo,
storagePolicy: .allowedInMemoryOnly)
completionHandler(updatedResponse)
} else {
completionHandler(proposedResponse)
}
}
thanks
The URLCache class implements the caching of responses to URL load
requests, by mapping NSURLRequest objects to CachedURLResponse
objects. It provides a composite in-memory and on-disk cache, and lets
you manipulate the sizes of both the in-memory and on-disk portions.
You can also control the path where cache data is persistently stored.
case allowed Storage in URLCache is allowed without restriction.
case allowedInMemoryOnly Storage in URLCache is allowed; however
storage should be restricted to memory only.
Yes.
Related
I have an app which uses URLSession-based networking and URLCache for storing network requests on disk. I noticed that when the storage size of URLCache reaches the diskCapacity, the eviction strategy seems to be to remove all entries, which is a problem in my use case. So I decided to write an URLCache subclass which would provide a custom storage for cached responses and which would implement LRU eviction strategy with better control.
As URLCache's documentation states, subclassing for this purpose should be supported:
The URLCache class is meant to be used as-is, but you can subclass it when you have specific needs. For example, you might want to screen which responses are cached, or reimplement the storage mechanism for security or other reasons.
However, I ran into problems with trying to use this new URLCache subclass with URLSession networking.
I have a test resource which I fetch using HTTP GET. The response headers contain:
Cache-Control: public, max-age=30
Etag: <some-value>
When using the standard, non-subclassed URLCache, the first request loads the data from network as expected (verified with HTTP proxy). The second request doesn't go to the network, if done within first 30 seconds, as expected. Subsequent requests after 30 seconds cause conditional GETs with Etag, as expected.
When using a URLCache subclass, all requests load the data from network - max-age doesn't seem to matter, and no conditional GETs are made.
It seems that the URLCache does something special to the CachedURLResponse instances after they're loaded from its internal storage, and this something affects how URLSession handles the HTTP caching logic. What am I missing here?
I've written a very minimal URLCache subclass implementation to demonstrate this problem. This class stores and loads CachedURLResponse instances using NSKeyedArchiver / NSKeyedUnarchiver, and it supports only zero or one response. Note that there are no calls to super - this is by design, since I want to use my own storage.
class CustomURLCache: URLCache {
let cachedResponseFileURL = URL(filePath: NSTemporaryDirectory().appending("entry.data"))
// MARK: Internal storage
func read() -> CachedURLResponse? {
guard let data = try? Data(contentsOf: cachedResponseFileURL) else { return nil }
return try! NSKeyedUnarchiver.unarchiveTopLevelObjectWithData(data) as! CachedURLResponse
}
func store(_ cachedResponse: CachedURLResponse) {
try! (try! NSKeyedArchiver.archivedData(withRootObject: cachedResponse, requiringSecureCoding: false)).write(to: cachedResponseFileURL)
}
// MARK: URLCache Overrides
override func cachedResponse(for request: URLRequest) -> CachedURLResponse? {
read()
}
override func getCachedResponse(for dataTask: URLSessionDataTask, completionHandler: #escaping (CachedURLResponse?) -> Void) {
completionHandler(read())
}
override func storeCachedResponse(_ cachedResponse: CachedURLResponse, for request: URLRequest) {
store(cachedResponse)
}
override func storeCachedResponse(_ cachedResponse: CachedURLResponse, for dataTask: URLSessionDataTask) {
store(cachedResponse)
}
}
My test case:
func test() {
let useEvictingCache = false
let config = URLSessionConfiguration.default
if useEvictingCache {
config.urlCache = CustomURLCache()
} else {
config.urlCache = URLCache(memoryCapacity: 0, diskCapacity: 1024 * 1024 * 100)
}
self.urlSession = URLSession(configuration: config)
let url = URL(string: "https://example.com/my-test-resource")!
self.urlSession?.dataTask(with: URLRequest(url: url), completionHandler: { data, response, error in
if let data {
print("GOT DATA with \(data.count) bytes")
} else if let error {
print("GOT ERROR \(error)")
}
}).resume()
}
Tested on iOS 16.2.
Received a response for my question at Apple's Developer Forums from Quinn “The Eskimo”:
My experience is that subclassing Foundation’s URL loading system classes puts you on a path of pain [1]. If I were in your shoes, I’d do your custom caching above the Foundation URL loading system layer.
[1] Way back in the day the Foundation URL loading system was implemented in Objective-C and existed within the Foundation framework. In that world, subclasses mostly worked. Shortly thereafter — and I’m talking before the introduction of NSURLSession here — the core implementation changed languages and moved to CFNetwork. Since then, the classes you see in Foundation are basically thin wrappers around (private) CFNetwork types. That’s generally OK, except for the impact on subclassing.
So it sounds like one should read the URLCache documentation re: subclassing with a grain of salt.
What is difference between URLSession vs DispatchQueue.global().async + Data(contentsOf: ) in terms of download images from image urls?
func loadImageWithUrlSession() {
guard let url = URL(string: IMAGE_URL) else { return }
URLSession.shared.dataTask(with: url) { (data, response, error) in
if let error = error {
print(error.localizedDescription)
return
}
guard let data = data else { return }
let image = UIImage(data: data)
DispatchQueue.main.async { [weak self] in
guard let self = self else { return }
self.urlSessionImageView.image = image
}
}.resume()
}
func loadImageWithGCD() {
DispatchQueue.global(qos: .background).async {
guard
let url = URL(string: self.IMAGE_URL),
let data = try? Data(contentsOf: url) else {
return
}
let image = UIImage(data: data)
DispatchQueue.main.async { [weak self] in
guard let self = self else { return }
self.gcdImageView.image = image
}
}
}
I know that URLSession can cancel or suspend task.
But if I use Rx instead, I can do the same thing as above as well.
I had an experiment that and it was depending on which QoS I'm using.
By the way, .userInitiated QoS was way faster than URLSession.
Which one are you guys use for something like downloading task with a background thread and why?
Is there any kind-teacher can help me specifically?
URLSession offers far greater configuration control, diagnostics of failures, cancelation, background sessions, ability to download directly to persistent storage to minimize peak memory usages, etc. URLSession and Data(contentsOf:) just are not comparable on feature set.
The synchronous Data(contentsOf:) unnecessarily blocks GCD worker threads and is also susceptible to misuse. It also is quite limiting and you will easily find yourself regretting the decision in the future (e.g. you later add some authentication process; you want to customize the cache behaviors, you want to parse and act upon status codes in the responses, you need cancelation capabilities because you are retrieving images for collection or table views, etc.).
It’s illuminating to look at the documentation for one of the init with a URL methods for Data, where it warns us:
Important
Don't use this synchronous initializer to request network-based URLs. For network-based URLs, this method can block the current thread for tens of seconds on a slow network, resulting in a poor user experience, and in iOS, may cause your app to be terminated.
Instead, for non-file URLs, consider using the dataTask(with:completionHandler:) method of the URLSession class. See Fetching Website Data into Memory for an example.
Yes, dispatching this to a background thread addresses many of the above concerns, but Apple didn’t just suggest “just dispatch this to some background queue,” but rather explicitly advised to use URLSession instead. While your use of GCD global queue avoids some of issues that Apple warns us of above, it also imposes many unnecessarily limitations. If you use Data(contentsOf:), this is a decision that you’ll likely regret/refactor in the future. You might as well use URLSession now.
Regarding Data(contentsOf:) being appreciably faster when using .userInitiated, vs .default or URLSession approach, usually the network latency and transmission time dwarfs any queue priority related factors, so I find that claim hard to believe. In fact, I just tested download of 50 images via GCD (using both .default and .userInitiated) and the speed was not appreciably different than URLSession approach.
I use a URLSession data task to download several JPG images from a backend. As the images are rather large in size (~500 KB) I want to cache the respective responses until they have expired (i.e. they have exceeded their max-age).
This is the code I use for downloading the images:
let request = URLRequest(url: url,
cachePolicy: .useProtocolCachePolicy,
timeoutInterval: 10.0)
let task = URLSession.shared.dataTask(with: request) { (data, _, error) in
// Error:
guard let imageData = data, error == nil, let image = UIImage(data: imageData) else {
DispatchQueue.main.async {
completion(nil)
}
return
}
// Success:
DispatchQueue.main.async {
completion(image)
}
}
task.resume()
Curiously, this works great with caching for all images except for one. For some reason, this particular image is always downloaded again – its response is not cached.
The only difference between the responses that I can spot is that the image whose corresponding response is not cached has the biggest file size. While all other images are < 500 kB, this particular image is slightly > 500 kB.
I've played around with the shared cache size and set it to a ridiculously high value, with no effect:
URLCache.shared = URLCache(memoryCapacity: 1000 * 1024 * 1024,
diskCapacity: 1000 * 1024 * 1024,
diskPath: nil)
I've checked that the Cache-Control header field is correctly set in the response:
Cache-Control: public, max-age=86400
and the Age header field is always below max-age, for example:
Age: 3526
What could be the reason for a single response not to be cached?
How can I fix this?
This is not an answer to the question why the shared URLSession does not cache the image and I'm still grateful for any hints or answers to that question.
However, after experimenting some time with my code I figured out that (for whatever reason) the response is always being cached when I use a custom URL session with a default configuration rather than the default shared URL session:
let urlSession = URLSession(configuration: .default)
So if I use:
let task = urlSession.dataTask(with: request) { ... }
instead of
let task = URLSession.shared.dataTask(with: request) { ... }
the caching works as expected – whatever black magic is responsible for that. 🤔
I found a little hint in the docs for URLSession.shared though:
When working with a shared session, you should generally avoid
customizing the cache, ...
In other words, if you’re doing anything with caches, cookies,
authentication, or custom networking protocols, you should probably be
using a default session instead of the shared session.
If I run the following code and let the app in background, the download is still continuing. Finally, when the download is finished, I can get the right callback.
let configuration = NSURLSessionConfiguration.backgroundSessionConfigurationWithIdentifier(SessionProperties.identifier)
let backgroundSession = NSURLSession(configuration: configuration, delegate: self.delegate, delegateQueue: nil)
let url = NSURLRequest(URL: NSURL(string: data[1])!)
let downloadTask = backgroundSession.downloadTaskWithRequest(url)
downloadTask.resume()
But I have a requirement, that is I have to judge what the server returns to me, if it is a json, I don't do the download, so I want to get the response header first, then if it needs to download, I change the data task to download task, so I did as the following code
let configuration = NSURLSessionConfiguration.backgroundSessionConfigurationWithIdentifier(SessionProperties.identifier)
let backgroundSession = NSURLSession(configuration: configuration, delegate: self.delegate, delegateQueue: nil)
let url = NSURLRequest(URL: NSURL(string: data[1])!)
//I change the downloadTaskWithRequest to dataTaskWithRequest
let downloadTask = backgroundSession.dataTaskWithRequest(url)
downloadTask.resume()
Then I can get the response header in the callback, and if it needs to download file, I can change the data task to download task, as following
func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, didReceiveResponse response: NSURLResponse, completionHandler: (NSURLSessionResponseDisposition) -> Void) {
if let response = response as? NSHTTPURLResponse {
let contentType = response.allHeaderFields["Content-Type"] as! String
if contentType == "image/jpeg" {
//change the data task to download task
completionHandler(.BecomeDownload)
return
}
}
completionHandler(.Allow)
}
So far so good. When I run the app in the foreground, the effect is like what I thought. But after the app runs in background, the download is stoped, then when I open the app, the console says "Lost connection to background transfer service".
I thought Apple is so smart, he gives us many useful callbacks, but now, I didn't know where I am wrong, and I also see the source code about the AFNetworking and Alamofire, but I didn't find the referring thing.
I also think it is a common requirement, but I can't find any helpful information on the internet, it is too odd.
So hope you can help me out, thanks a billion.
Enable Background Mode in
Xcode->Target->Capabilities->On Background Mode and select the option Background Fetch.
The main issue I see is that you're calling the completionHandler twice. You need to return out of your content-type conditional like so:
if contentType == "image/jpeg" {
//change the data task to download task
completionHandler(.BecomeDownload)
return
}
Otherwise it appears that you are using the logic correctly. Hope that helps.
The problem is evident from your own answer. It's not a bug, you simply couldn't use data tasks for background transfers just download tasks.
Here is the correct full answer.
I am trying to download images but it is crucial that the images I download are in a specific order. I am using the following code to download the image:
func downloadImage(url: NSURL, completionHandler: (response: UIImage) -> ()){
print("Started downloading \"\(url.URLByDeletingPathExtension!.lastPathComponent!)\".")
manager.getDataFromUrl(url) { (data, response, error) in
dispatch_async(dispatch_get_main_queue()) { () -> Void in
guard let data = data where error == nil else { return }
print("Finished downloading \"\(url.URLByDeletingPathExtension!.lastPathComponent!)\".")
completionHandler(response: UIImage(data: data)!)
}
}
}
and I am using this code to call downloadImage
self.downloadImage(NSURL(string: self.url)!, completionHandler: { response in
dispatch_async(dispatch_get_main_queue()) {
self.images.append(response)
}
})
The problem is that the images start downloading in the correct order however the response isn't (this is obviously because the size of the images are different and one comes in faster). So what is happening is that all images start downloading, and whichever comes first appends to the images : [UIImage] array. How can I make it so that the images in the images array is in order?
I've also tried to remove the main block when calling downloadImage function
self.downloadImage(NSURL(string: self.url)!, completionHandler: { response in
self.images.append(response)
})
You cannot control the download order in the sense that all the requests to the same server will be pipelined no matter what the order you create the URL objects in. Also, some URLs may be cached while others may need to go to the remote server. What you need to do is maintain a mutable array or dictionary that contains the url to actual data mapping, then wait until all the urls have been completely downloaded and then iterate in a known order.
The simplest method is that you can save every image in Dictionary with their url after downloading. Like var imageData = [String: NSData](). Later you can sort it or use it by keys(url).