Using URLCache subclasses with URLSession - ios

I have an app which uses URLSession-based networking and URLCache for storing network requests on disk. I noticed that when the storage size of URLCache reaches the diskCapacity, the eviction strategy seems to be to remove all entries, which is a problem in my use case. So I decided to write an URLCache subclass which would provide a custom storage for cached responses and which would implement LRU eviction strategy with better control.
As URLCache's documentation states, subclassing for this purpose should be supported:
The URLCache class is meant to be used as-is, but you can subclass it when you have specific needs. For example, you might want to screen which responses are cached, or reimplement the storage mechanism for security or other reasons.
However, I ran into problems with trying to use this new URLCache subclass with URLSession networking.
I have a test resource which I fetch using HTTP GET. The response headers contain:
Cache-Control: public, max-age=30
Etag: <some-value>
When using the standard, non-subclassed URLCache, the first request loads the data from network as expected (verified with HTTP proxy). The second request doesn't go to the network, if done within first 30 seconds, as expected. Subsequent requests after 30 seconds cause conditional GETs with Etag, as expected.
When using a URLCache subclass, all requests load the data from network - max-age doesn't seem to matter, and no conditional GETs are made.
It seems that the URLCache does something special to the CachedURLResponse instances after they're loaded from its internal storage, and this something affects how URLSession handles the HTTP caching logic. What am I missing here?
I've written a very minimal URLCache subclass implementation to demonstrate this problem. This class stores and loads CachedURLResponse instances using NSKeyedArchiver / NSKeyedUnarchiver, and it supports only zero or one response. Note that there are no calls to super - this is by design, since I want to use my own storage.
class CustomURLCache: URLCache {
let cachedResponseFileURL = URL(filePath: NSTemporaryDirectory().appending("entry.data"))
// MARK: Internal storage
func read() -> CachedURLResponse? {
guard let data = try? Data(contentsOf: cachedResponseFileURL) else { return nil }
return try! NSKeyedUnarchiver.unarchiveTopLevelObjectWithData(data) as! CachedURLResponse
}
func store(_ cachedResponse: CachedURLResponse) {
try! (try! NSKeyedArchiver.archivedData(withRootObject: cachedResponse, requiringSecureCoding: false)).write(to: cachedResponseFileURL)
}
// MARK: URLCache Overrides
override func cachedResponse(for request: URLRequest) -> CachedURLResponse? {
read()
}
override func getCachedResponse(for dataTask: URLSessionDataTask, completionHandler: #escaping (CachedURLResponse?) -> Void) {
completionHandler(read())
}
override func storeCachedResponse(_ cachedResponse: CachedURLResponse, for request: URLRequest) {
store(cachedResponse)
}
override func storeCachedResponse(_ cachedResponse: CachedURLResponse, for dataTask: URLSessionDataTask) {
store(cachedResponse)
}
}
My test case:
func test() {
let useEvictingCache = false
let config = URLSessionConfiguration.default
if useEvictingCache {
config.urlCache = CustomURLCache()
} else {
config.urlCache = URLCache(memoryCapacity: 0, diskCapacity: 1024 * 1024 * 100)
}
self.urlSession = URLSession(configuration: config)
let url = URL(string: "https://example.com/my-test-resource")!
self.urlSession?.dataTask(with: URLRequest(url: url), completionHandler: { data, response, error in
if let data {
print("GOT DATA with \(data.count) bytes")
} else if let error {
print("GOT ERROR \(error)")
}
}).resume()
}
Tested on iOS 16.2.

Received a response for my question at Apple's Developer Forums from Quinn “The Eskimo”:
My experience is that subclassing Foundation’s URL loading system classes puts you on a path of pain [1]. If I were in your shoes, I’d do your custom caching above the Foundation URL loading system layer.
[1] Way back in the day the Foundation URL loading system was implemented in Objective-C and existed within the Foundation framework. In that world, subclasses mostly worked. Shortly thereafter — and I’m talking before the introduction of NSURLSession here — the core implementation changed languages and moved to CFNetwork. Since then, the classes you see in Foundation are basically thin wrappers around (private) CFNetwork types. That’s generally OK, except for the impact on subclassing.
So it sounds like one should read the URLCache documentation re: subclassing with a grain of salt.

Related

IOS app crashes on a line of code if there's no internet connection, how can I prevent this

The code this and it crashes on "try!", but I don't know how to catch the error and it has it be explicit otherwise it won't work.
func downloadPicture2(finished: () -> Void) {
let imageUrlString = self.payments[indexPath.row].picture
let imageUrl = URL(string: imageUrlString!)!
let imageData = try! Data(contentsOf: imageUrl)
cell.profilePicture.image = UIImage(data: imageData)
cell.profilePicture.layer.cornerRadius = cell.profilePicture.frame.size.width / 2
cell.profilePicture.clipsToBounds = true
}
The short answer is don't use try! - Use do/try/catch and recover from the problem in the catch clause.
For example -
func downloadPicture2(finished: () -> Void) {
cell.profilePicture.image = nil
if let imageUrlString = self.payments[indexPath.row].picture,
let imageUrl = URL(string: imageUrlString) {
do {
let imageData = try Data(contentsOf: imageUrl)
cell.profilePicture.image = UIImage(data: imageData)
}
catch {
print("Error fetching image - \(error)")
}
}
cell.profilePicture.layer.cornerRadius = cell.profilePicture.frame.size.width / 2
cell.profilePicture.clipsToBounds = true
}
Now you have code that won't crash if the url is invalid or there is no network, but there are still some serious issues with this code.
Data(contentsOf:) blocks the current thread while it fetches the data. Since you are executing on the main thread this will freeze the user interface and give a poor user experience.
Apple specifically warns not to do this
Important
Don't use this synchronous initializer to request network-based URLs. For network-based URLs, this method can block the current thread for tens of seconds on a slow network, resulting in a poor user experience, and in iOS, may cause your app to be terminated.
Rather, you should use an asynchronous network operations, such as a dataTask.
This code operates on cell - an external property. Once you move to asynchronous code you will probably be fetching images for multiple cells simultaneously. You should pass the relevant cell to this function to avoid clashes.
The use of the network isn't particularly efficient either; assuming this is part of a table or collection view, cells are reused as the view scrolls. You will repeatedly fetch the same image as this happens. Some sort of local caching would be more efficient.
If it is possible to use external frameworks in your project (i.e. your employer doesn't specifically disallow it) then I strongly suggest you look at a framework like SDWebImage or KingFisher. They will make this task much easier and much more efficient.

Does CachedURLResponse persists in between app launches?

The example below from Apple documentation, shows how manually create a CachedURLResponse for a UrlSession.
One of the parameters is storagePolicy (in the example set to .allowedInMemoryOnly).
If set such parameter to .allowed (instead of .allowedInMemoryOnly, the cache should be stored to hard disk. Does this mean that it persists in-between app launches?
func urlSession(_ session: URLSession, dataTask: URLSessionDataTask,
willCacheResponse proposedResponse: CachedURLResponse,
completionHandler: #escaping (CachedURLResponse?) -> Void) {
if proposedResponse.response.url?.scheme == "https" {
let updatedResponse = CachedURLResponse(response: proposedResponse.response,
data: proposedResponse.data,
userInfo: proposedResponse.userInfo,
storagePolicy: .allowedInMemoryOnly)
completionHandler(updatedResponse)
} else {
completionHandler(proposedResponse)
}
}
thanks
The URLCache class implements the caching of responses to URL load
requests, by mapping NSURLRequest objects to CachedURLResponse
objects. It provides a composite in-memory and on-disk cache, and lets
you manipulate the sizes of both the in-memory and on-disk portions.
You can also control the path where cache data is persistently stored.
case allowed Storage in URLCache is allowed without restriction.
case allowedInMemoryOnly Storage in URLCache is allowed; however
storage should be restricted to memory only.
Yes.

What is difference between URLSession vs GCD in terms of download image from image url?

What is difference between URLSession vs DispatchQueue.global().async + Data(contentsOf: ) in terms of download images from image urls?
func loadImageWithUrlSession() {
guard let url = URL(string: IMAGE_URL) else { return }
URLSession.shared.dataTask(with: url) { (data, response, error) in
if let error = error {
print(error.localizedDescription)
return
}
guard let data = data else { return }
let image = UIImage(data: data)
DispatchQueue.main.async { [weak self] in
guard let self = self else { return }
self.urlSessionImageView.image = image
}
}.resume()
}
func loadImageWithGCD() {
DispatchQueue.global(qos: .background).async {
guard
let url = URL(string: self.IMAGE_URL),
let data = try? Data(contentsOf: url) else {
return
}
let image = UIImage(data: data)
DispatchQueue.main.async { [weak self] in
guard let self = self else { return }
self.gcdImageView.image = image
}
}
}
I know that URLSession can cancel or suspend task.
But if I use Rx instead, I can do the same thing as above as well.
I had an experiment that and it was depending on which QoS I'm using.
By the way, .userInitiated QoS was way faster than URLSession.
Which one are you guys use for something like downloading task with a background thread and why?
Is there any kind-teacher can help me specifically?
URLSession offers far greater configuration control, diagnostics of failures, cancelation, background sessions, ability to download directly to persistent storage to minimize peak memory usages, etc. URLSession and Data(contentsOf:) just are not comparable on feature set.
The synchronous Data(contentsOf:) unnecessarily blocks GCD worker threads and is also susceptible to misuse. It also is quite limiting and you will easily find yourself regretting the decision in the future (e.g. you later add some authentication process; you want to customize the cache behaviors, you want to parse and act upon status codes in the responses, you need cancelation capabilities because you are retrieving images for collection or table views, etc.).
It’s illuminating to look at the documentation for one of the init with a URL methods for Data, where it warns us:
Important
Don't use this synchronous initializer to request network-based URLs. For network-based URLs, this method can block the current thread for tens of seconds on a slow network, resulting in a poor user experience, and in iOS, may cause your app to be terminated.
Instead, for non-file URLs, consider using the dataTask(with:completionHandler:) method of the URLSession class. See Fetching Website Data into Memory for an example.
Yes, dispatching this to a background thread addresses many of the above concerns, but Apple didn’t just suggest “just dispatch this to some background queue,” but rather explicitly advised to use URLSession instead. While your use of GCD global queue avoids some of issues that Apple warns us of above, it also imposes many unnecessarily limitations. If you use Data(contentsOf:), this is a decision that you’ll likely regret/refactor in the future. You might as well use URLSession now.
Regarding Data(contentsOf:) being appreciably faster when using .userInitiated, vs .default or URLSession approach, usually the network latency and transmission time dwarfs any queue priority related factors, so I find that claim hard to believe. In fact, I just tested download of 50 images via GCD (using both .default and .userInitiated) and the speed was not appreciably different than URLSession approach.

Large image URL response not cached by URLSession: Why?

I use a URLSession data task to download several JPG images from a backend. As the images are rather large in size (~500 KB) I want to cache the respective responses until they have expired (i.e. they have exceeded their max-age).
This is the code I use for downloading the images:
let request = URLRequest(url: url,
cachePolicy: .useProtocolCachePolicy,
timeoutInterval: 10.0)
let task = URLSession.shared.dataTask(with: request) { (data, _, error) in
// Error:
guard let imageData = data, error == nil, let image = UIImage(data: imageData) else {
DispatchQueue.main.async {
completion(nil)
}
return
}
// Success:
DispatchQueue.main.async {
completion(image)
}
}
task.resume()
Curiously, this works great with caching for all images except for one. For some reason, this particular image is always downloaded again – its response is not cached.
The only difference between the responses that I can spot is that the image whose corresponding response is not cached has the biggest file size. While all other images are < 500 kB, this particular image is slightly > 500 kB.
I've played around with the shared cache size and set it to a ridiculously high value, with no effect:
URLCache.shared = URLCache(memoryCapacity: 1000 * 1024 * 1024,
diskCapacity: 1000 * 1024 * 1024,
diskPath: nil)
I've checked that the Cache-Control header field is correctly set in the response:
Cache-Control: public, max-age=86400
and the Age header field is always below max-age, for example:
Age: 3526
What could be the reason for a single response not to be cached?
How can I fix this?
This is not an answer to the question why the shared URLSession does not cache the image and I'm still grateful for any hints or answers to that question.
However, after experimenting some time with my code I figured out that (for whatever reason) the response is always being cached when I use a custom URL session with a default configuration rather than the default shared URL session:
let urlSession = URLSession(configuration: .default)
So if I use:
let task = urlSession.dataTask(with: request) { ... }
instead of
let task = URLSession.shared.dataTask(with: request) { ... }
the caching works as expected – whatever black magic is responsible for that. 🤔
I found a little hint in the docs for URLSession.shared though:
When working with a shared session, you should generally avoid
customizing the cache, ...
In other words, if you’re doing anything with caches, cookies,
authentication, or custom networking protocols, you should probably be
using a default session instead of the shared session.

Create a Class that Makes HTTPRequest in Swift

I am making an app that has the need to make HTTP Request in many of its ViewControllers.
I ended up copy and pasting these codes in to each of the ViewControllers and listen to the delegates callback of NSURLConnectionDelegate and NSURLConnectionDataDelegate
func makeRequest()
{
//Base64
var username = "testUsername";
var password = "testPassword";
var loginString = NSString(format: "%#:%#", username, password);
var loginData: NSData = loginString.dataUsingEncoding(NSUTF8StringEncoding)!;
var base64LoginString = loginData.base64EncodedStringWithOptions(nil);
var url: NSURL = NSURL(string: self.HTTP_REQUEST_STRING)!;
var urlRequest: NSMutableURLRequest = NSMutableURLRequest(URL: url);
urlRequest.setValue("Basic \(base64LoginString)", forHTTPHeaderField: "Authorization");
urlRequest.HTTPMethod = "POST";
urlConnection = NSURLConnection(request: urlRequest, delegate: self)!;
}
func connection(connection: NSURLConnection!, didReceiveData data: NSData!)
{
self.resultData.appendData(data);
}
func connectionDidFinishLoading(connection: NSURLConnection!)
{
//Do Something
}
func connection(connection: NSURLConnection, didFailWithError error: NSError)
{
//Do Something
}
I am wondering if there is a better approach than this, rather than copy and pasting the codes into every ViewControllers?
Is it possible to put these codes into a class? But then, how do we know if the connection has finished loading?
I am sorry for this question, I lack the knowledge of good Object Oriented design.
Thank you
To be up to date, you should be using NSURLSession as your request class. You're not necessarily required to listen in for the delegation callbacks as there is a closure callback that will provide you with and error and data according to how you configured your session. Regarding placement, it depends on your code and what you want. You can place this code in the viewController if it makes sense, some people create proxy classes to make all their requests and report statuses back to them. It all depends on flexibility, robustness and structure of your application. If you're making the same network request from 3 different viewControllers, it's likely that your should be placing the network request in a type of proxy class to prevent duplicate code. Look into the proxy and singleton design patterns if you'd like to know more about code design.
Here's a nice tutorial on NSURLSession to get you started:
Raywenderlich tutorial
Proxy Design Pattern
Singleton Design Pattern
Like others have mentioned, you have far more learning to do than a single answer on SO can provide. In the mean time, take a look at Alamofire, a fairly comprehensive networking library built around NSURLSession. In it you will find code for creating requests and much more. Alamofire provides a prebuilt, OO way of performing network requests, suitable for one off requests or an entire network manager class.

Resources