My app makes a lot of URL requests (calling a web service) in a loop for an extended period of time. When I watch the app in the Allocations tool, I see memory consumption going up continuously during the run of that loop. For testing purposes, I've reduced the loop to the following, which exhibits the same behavior:
NSURL *myUrl = [[NSURL alloc] initWithString:#"http://my.server.com/webservice"];
NSURLRequest request = [[NSURLRequest alloc] initWithURL:myUrl cachePolicy:NSURLRequestReloadIgnoringCacheData timeoutInterval:30];
while (1)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *rawResponse = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
[[NSURLCache sharedURLCache] removeAllCachedResponses];
[pool release];
}
When I first discovered the problem, I figured it was because my URL requests/responses were being cached. That's when I added the [[NSURLCache sharedURLCache] removeAllCachedResponses]. I was expecting that to clear out the cache after each call to the web service and free up any memory being used for URL caching purposes. No luck.
I must be doing something wrong, but I can't find it. Am I barking up the wrong tree in thinking it's the URL caching? What else could it be?
Related
I'm using this line to get the contents of a URL:
NSString *result=[[NSString alloc] initWithContentsOfURL:[NSURL URLWithString:URL]
encoding:NSUTF8StringEncoding
error:nil];
The problem is when there's a bad connection, it loads the contents from cache. Is there a way to avoid this behaviour? For example, clearing the cache or something.
First, it's not recommended to use initWithContentsOfURL:encoding:error to load data from a network resource.
Second, if you want to control caching behavior, you should be using an NSURLRequest. NSURLRequest allows you to customize the caching behavior of the request by setting the cachePolicy of the request. In your case, you want to use NSURLRequestReloadIgnoringLocalCacheData. An example of doing this synchronously using NSURLConnection would be:
NSString *result = nil;
NSData *data = nil;
NSURLResponse *response = nil;
NSURLError *error = nil;
NSURLRequest *request = [[NSURLRequest alloc] initWithURL:URL cachePolicy:NSURLRequestReloadIgnoringLocalCacheData timeoutInterval:20L];
data = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
if (response != nil && [data length] > 0){
result = [NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
}
Note that this is a very naive implementation that does not check the HTTP status code returned, the mime-type of the response, or perform any error handling. It is also not a recommended practice to load network resources synchronously or to do so from the main thread. A better implementation would use sendAsynchronousRequest:completion: or NSURLSession.
However, it does demonstrate at a high level what you would need to do to answer your question: The NSURLRequest specifies that this request should never use the local cache, and the returned data is used to create an instance of NSString.
Simple cache-buster dummy parameter with random value added to URL should work.
And as #Josh-Caswell said, use NSURLRequest. Although in case of proxy servers, just using NSURLRequest may not help and you will still need cache-buster.
I have a test app setup and it successfully downloads content from the network even if the user switches apps while a download is in progress. Great, now I have background downloads in place. Now I want to add caching. There is no point to me downloading images more than once, b/c of system design, given an image URL I can tell you the content behind that URL will never change. So, now I want to cache the results of my download using apple's built in in-memory/on-disk cache that I've read so much about (as opposed to me saving the file manually in NSCachesDirectory and then checking there before making new request, ick). In an attempt to get caching working on top of this working code, I added the following code:
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
// Override point for customization after application launch.
// Set app-wide shared cache (first number is megabyte value)
[NSURLCache setSharedURLCache:[[NSURLCache alloc] initWithMemoryCapacity:60 * 1024 * 1024
diskCapacity:200 * 1024 * 1024
diskPath:nil]];
return YES;
}
When I create my session, I've added two NEW lines (URLCache and requestCachePolicy).
// Helper method to get a single session object
- (NSURLSession *)backgroundSession
{
static NSURLSession *session = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfiguration:#"com.example.apple-samplecode.SimpleBackgroundTransfer.BackgroundSession"];
configuration.URLCache = [NSURLCache sharedURLCache]; // NEW LINE ON TOP OF OTHERWISE WORKING CODE
configuration.requestCachePolicy = NSURLRequestReturnCacheDataElseLoad; // NEW LINE ON TOP OF OTHERWISE WORKING CODE
session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
});
return session;
}
Then, just to be ultra redundant in an attempt to see caching success I switched my NSURLRequest line from
// NSURLRequest *request = [NSURLRequest requestWithURL:downloadURL]; // Old line, I've replaced this with...
NSURLRequest *request = [NSURLRequest requestWithURL:downloadURL cachePolicy:NSURLRequestReturnCacheDataElseLoad timeoutInterval:2*60]; // New line
Now, when I go to download the item a 2nd time, the experience is exaclty like the first!! Takes a long time to download and progress bar is animated slow and steady like an original download. I want the data in the cache immediately!! What am I missing???
----------------------------UPDATE----------------------------
Okay, thanks to Thorsten's answer, I've added the following two lines of code to my didFinishDownloadingToURL delegate method:
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)downloadURL {
// Added these lines...
NSLog(#"DiskCache: %# of %#", #([[NSURLCache sharedURLCache] currentDiskUsage]), #([[NSURLCache sharedURLCache] diskCapacity]));
NSLog(#"MemoryCache: %# of %#", #([[NSURLCache sharedURLCache] currentMemoryUsage]), #([[NSURLCache sharedURLCache] memoryCapacity]));
/*
OUTPUTS:
DiskCache: 4096 of 209715200
MemoryCache: 0 of 62914560
*/
}
This is great. It confirms the cache is growing. I presume since I'm using a downloadTask (downloads to file as opposed to memory), that that's why DiskCache is growing and not memory cache first? I figured everything would go to memory cache until that overflowed and then disk cache would be used and that maybe memory cache was written to disk before the OS kills the app in the background to free up memory. Am I misunderstanding how Apple's cache works?
This is a step forward for sure, but the 2nd time I download the file it takes just as long as the first time (maybe 10 seconds or so) and the following method DOES get executed again:
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didWriteData:(int64_t)bytesWritten totalBytesWritten:(int64_t)totalBytesWritten totalBytesExpectedToWrite:(int64_t)totalBytesExpectedToWrite
{
// This shouldn't execute the second time around should it? Even if this is supposed to get executed a second time around then shouldn't it be lightning fast? It's not.
// On all subsequent requests, it slowly iterates through the downloading of the content just as slow as the first time. No caching is apparent. What am I missing?
}
What do you make of my edits above? Why am I not seeing the file returned very quickly on subsequent requests?
How can I confirm if the file is being served from the cache on the 2nd request?
Note that the following SO post helped me solve my problem: Is NSURLCache persistent across launches?
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
// Set app-wide shared cache (first number is megabyte value)
NSUInteger cacheSizeMemory = 500*1024*1024; // 500 MB
NSUInteger cacheSizeDisk = 500*1024*1024; // 500 MB
NSURLCache *sharedCache = [[NSURLCache alloc] initWithMemoryCapacity:cacheSizeMemory diskCapacity:cacheSizeDisk diskPath:#"nsurlcache"];
[NSURLCache setSharedURLCache:sharedCache];
sleep(1); // Critically important line, sadly, but it's worth it!
}
In addition to the sleep(1) line, also note the size of my cache; 500MB.
According to docs you need a cache size that is way bigger than what you're trying to cache.
The response size is small enough to reasonably fit within the cache.
(For example, if you provide a disk cache, the response must be no
larger than about 5% of the disk cache size.)
So for example if you want to be able to cache a 10MB image, then a cache size of 10MB or even 20MB will not be enough. You need 200MB.
Honey's comment below is evidence that Apple is following this 5% rule. For an 8Mb he had to set his cache size to minimum 154MB.
Solution - first get all info u need it something like this
- (void)loadData
{
if (!self.commonDataSource) {
self.commonDataSource = [[NSArray alloc] init];
}
[self setSharedCacheForImages];
NSURLSession *session = [self prepareSessionForRequest];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:[BaseURLString stringByAppendingPathComponent:#"app.json"]]];
[request setHTTPMethod:#"GET"];
__weak typeof(self) weakSelf = self;
NSURLSessionDataTask *dataTask = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
if (!error) {
NSArray *jsonResponse = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:&error];
weakSelf.commonDataSource = jsonResponse;
dispatch_async(dispatch_get_main_queue(), ^{
[weakSelf updateDataSource];
});
}
}];
[dataTask resume];
}
- (void)setSharedCacheForImages
{
NSUInteger cashSize = 250 * 1024 * 1024;
NSUInteger cashDiskSize = 250 * 1024 * 1024;
NSURLCache *imageCache = [[NSURLCache alloc] initWithMemoryCapacity:cashSize diskCapacity:cashDiskSize diskPath:#"someCachePath"];
[NSURLCache setSharedURLCache:imageCache];
}
- (NSURLSession *)prepareSessionForRequest
{
NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration defaultSessionConfiguration];
[sessionConfiguration setHTTPAdditionalHeaders:#{#"Content-Type": #"application/json", #"Accept": #"application/json"}];
NSURLSession *session = [NSURLSession sessionWithConfiguration:sessionConfiguration];
return session;
}
After you need to download each file - in my case - make parsing of response and download images. Also before making request you need to check if cache already have response for your request - something like this
NSString *imageURL = [NSString stringWithFormat:#"%#%#", BaseURLString ,sourceDictionary[#"thumb_path"]];
NSURLSession *session = [self prepareSessionForRequest];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:imageURL]];
[request setHTTPMethod:#"GET"];
NSCachedURLResponse *cachedResponse = [[NSURLCache sharedURLCache] cachedResponseForRequest:request];
if (cachedResponse.data) {
UIImage *downloadedImage = [UIImage imageWithData:cachedResponse.data];
dispatch_async(dispatch_get_main_queue(), ^{
cell.thumbnailImageView.image = downloadedImage;
});
} else {
NSURLSessionDataTask *dataTask = [session dataTaskWithRequest:request completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
if (!error) {
UIImage *downloadedImage = [UIImage imageWithData:data];
dispatch_async(dispatch_get_main_queue(), ^{
cell.thumbnailImageView.image = downloadedImage;
});
}
}];
[dataTask resume];
}
After that you can also check result with xCode Network Analyzer.
Also note as mentionted by #jcaron and documented by Apple
NSURLSession won't attempt to cache a file larger than 5% of the cache
size
Result something like
Once you set the cache and the session, you should use the session-methods to download your data:
- (IBAction)btnClicked:(id)sender {
NSString *imageUrl = #"http://placekitten.com/1000/1000";
NSURLSessionDataTask* loadDataTask = [session dataTaskWithRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:imageUrl]] completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
UIImage *downloadedImage = [UIImage imageWithData:data];
NSLog(#"ImageSize: %f, %f", downloadedImage.size.width, downloadedImage.size.height);
NSLog(#"DiskCache: %i of %i", [[NSURLCache sharedURLCache] currentDiskUsage], [[NSURLCache sharedURLCache] diskCapacity]);
NSLog(#"MemoryCache: %i of %i", [[NSURLCache sharedURLCache] currentMemoryUsage], [[NSURLCache sharedURLCache] memoryCapacity]);
}];
[loadDataTask resume]; //start request
}
After the first call, the image is cached.
I've written a short iOS application that contacts a server via the usual:
NSString *URLString = #"http://mysite/script.php?foo=1";
NSURL *getURL = [NSURL URLWithString:URLString];
NSURLRequest *getRequest = [NSURLRequest requestWithURL:getURL];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *responseData = [NSURLConnection sendSynchronousRequest:getRequest returningResponse:&response error:&error];
This works OK, but as I access the web site relatively frequently with various values of foo, and as we plan to migrate to https, people have raised the question about the overhead on the phone of setting up and tearing down the NSURLConnection every time. I see this answer saying that the class handles caching, but is that true for https too?
According to my tests, yes, it does appear to be true that the NSURLConnection class does caching of https to some extent. I did like the look of AFHTTPClient as mentioned in the comment, but due to various issues I was unable to use it in this project, however.
I'm currently doing this when populating core data from a JSON file:
NSString *urlString = [value objectForKey:#"url"];
NSURL *url = [NSURL URLWithString:urlString];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *dataResponse = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
[managedObject setValue:dataResponse forKey:#"image"];
Is there a better (asynchronous) way to do this with AFNetworking? What is the best method for this case? Does it have to be synchronous because we're dealing with CoreData?
UPDATE: Trying this now:
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[NSURLConnection sendAsynchronousRequest:request queue:queue completionHandler:^(NSURLResponse *response, NSData *data, NSError *error)
{
[managedObject setValue:data forKey:#"image"];
}];
For some reason when I access the managed object later, the image attribute is always null, even though *data above is not null in the completion handler. The image gets saved fine in the synchronous method. What am I missing?
NSURLConnection can deal with async too.
The method that you can use is (iOS >= 5) is
+ sendAsynchronousRequest:queue:completionHandler:
If you need to target iOS < 5 then use the delegate pattern for NSURLConnection. A good wrapper for this can be found in NSURLConnection and grand central dispatch.
About Core Data, I would say it depends. If data you need to store is cheap, do it in the main thread. On the contrary you have three different ways to do it:
(1) use new Core Data queue-based API (iOS >= 5)
(2) kick off a NSOperation within a NSOperationQueue and do the long work in background
(3) use GDC
Pay attention to Core Data constraints (threads constraints) when you deal with (2) or (3).
Hope that helps.
P.S. If you want to know something else let me know.
There's a sendAsynchronousRequest:queue:completionHandler: message of NSURLConnection.
I want to download a lot of pictures from the server. How can I do this as fast as possible? Currently I am using:
UIImage* myImage = [UIImage imageWithData:
[NSData dataWithContentsOfURL:
[NSURL URLWithString: #"http://example.com/image.jpg"]]];
It is painfully slow. Is there any speed increase in downloading multiple images at the same time (asynchronously), and if so how many is too many?
You won't get a definitive answer to the optimal number of connections, because there is none. It just depends on several variables such as bandwidth, image size or your own patience. You need to measure this by yourself to get it right.
Doing asynchronous requests won't increase the downloading speed, but the user experience is way better. Seriously, you should consider doing it for any download that takes more than a second.
I always recommend using ASIHTTPRequest, it makes implementing things such as queues and progress bars easy.
Here's the simplest example from an asynchronous request using the library:
- (IBAction)grabURLInBackground:(id)sender
{
NSURL *url = [NSURL URLWithString:#"http://allseeing-i.com"];
ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:url];
[request setDelegate:self];
[request startAsynchronous];
}
- (void)requestFinished:(ASIHTTPRequest *)request
{
// Use when fetching text data
NSString *responseString = [request responseString];
// Use when fetching binary data
NSData *responseData = [request responseData];
}
- (void)requestFailed:(ASIHTTPRequest *)request
{
NSError *error = [request error];
}
Update: This library will no longer be supported. From 1:
"Please note that I am no longer working on this library - you may
want to consider using something else for new projects. :)"
Nowadays I use AFNetworking for most of my projects.