Download Multiple Images Sequentially using NSURLSession downloadTask in Objective C - ios

Question

My app offers the option to download 3430 high resolution images from our server, each image of size 50k - 600k bytes.
The original approach was to just download all of them - but we realized that gave a lot of NSURLErrorTimedOut errors and crashed our program. We then implemented it such that we download all of the images, but in batches of 100 images at a time. Someone on SO suggested we actually implement our download like this:
Create a list of all file URLs that need to be downloaded.
Write your code so that it downloads these URLs sequentially. I.e. do
not let it start downloading a file until the previous one has
finished (or failed and you decided to skip it for now).
Use NSURLSession's support for downloading an individual file to a
folder, don't use the code to get an NSData and save the file
yourself. That way, your application doesn't need to be running while
the download finishes.
Ensure that you can tell whether a file has already been downloaded or
not, in case your download gets interrupted, or the phone is restarted
in mid-download. You can e.g. do this by comparing their names (if
they are unique enough), or saving a note to a plist that lets you
match a downloaded file to the URL where it came from, or whatever
constitutes an identifying characteristic in your case.
At startup, check whether all files are there. If not, put the missing
ones in above download list and download them sequentially, as in #2.
Before you start downloading anything (and that includes downloading
the next file after the previous download has finished or failed), do
a reachability check using the Reachability API from Apple's
SystemConfiguration.framework. That will tell you whether the user has
a connection at all, and whether you're on WiFi or cellular (in
general, you do not want to download a large number of files via
cellular, most cellular connections are metered).
We create a list of all images to download here:
- (void)generateImageURLList:(BOOL)batchDownloadImagesFromServer
{
NSError* error;
NSFetchRequest* leafletURLRequest = [[[NSFetchRequest alloc] init] autorelease];
NSEntityDescription* leafletURLDescription = [NSEntityDescription entityForName:#"LeafletURL" inManagedObjectContext:managedObjectContext];
[leafletURLRequest setEntity:leafletURLDescription];
numberOfImages = [managedObjectContext countForFetchRequest:leafletURLRequest error:&error];
NSPredicate* thumbnailPredicate = [NSPredicate predicateWithFormat:#"thumbnailLocation like %#", kLocationServer];
[leafletURLRequest setPredicate:thumbnailPredicate];
self.uncachedThumbnailArray = [managedObjectContext executeFetchRequest:leafletURLRequest error:&error];
NSPredicate* hiResPredicate = [NSPredicate predicateWithFormat:#"hiResImageLocation != %#", kLocationCache];
[leafletURLRequest setPredicate:hiResPredicate];
self.uncachedHiResImageArray = [managedObjectContext executeFetchRequest:leafletURLRequest error:&error];
}
We use NSURLSession to download an individual image to a folder by calling hitServerForUrl and implementing didFinishDownloadingToURL:
- (void)hitServerForUrl:(NSURL*)requestUrl {
NSURLSessionConfiguration *defaultConfigurationObject = [NSURLSessionConfiguration defaultSessionConfiguration];
NSURLSession *defaultSession = [NSURLSession sessionWithConfiguration:defaultConfigurationObject delegate:self delegateQueue: nil];
NSURLSessionDownloadTask *fileDownloadTask = [defaultSession downloadTaskWithURL:requestUrl];
[fileDownloadTask resume];
}
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)location {
if (isThumbnail)
{
leafletURL.thumbnailLocation = kLocationCache;
}
else
{
leafletURL.hiResImageLocation = kLocationCache;
}
// Filename to write to
NSString* filePath = [leafletURL pathForImageAtLocation:kLocationCache isThumbnail:isThumbnail isRetina:NO];
// If it's a retina image, append the "#2x"
if (isRetina_) {
filePath = [filePath stringByReplacingOccurrencesOfString:#".jpg" withString:#"#2x.jpg"];
}
NSString* dir = [filePath stringByDeletingLastPathComponent];
[managedObjectContext save:nil];
NSError* error;
[[NSFileManager defaultManager] createDirectoryAtPath:dir withIntermediateDirectories:YES attributes:nil error:&error];
NSURL *documentURL = [NSURL fileURLWithPath:filePath];
NSLog(#"file path : %#", filePath);
if ([[NSFileManager defaultManager] fileExistsAtPath:filePath]) {
//Remove the old file from directory
}
[[NSFileManager defaultManager] moveItemAtURL:location
toURL:documentURL
error:&error];
if (error){
//Handle error here
}
}
This code calls loadImage, which calls `hitServer:
-(void)downloadImagesFromServer{
[self generateImageURLList:NO];
[leafletImageLoaderQueue removeAllObjects];
numberOfHiResImageLeft = [uncachedHiResImageArray count];
for ( LeafletURL* aLeafletURL in uncachedHiResImageArray)
{
//// Do the same thing again, except set isThumb = NO. ////
LeafletImageLoader* hiResImageLoader = [[LeafletImageLoader alloc] initWithDelegate:self];
[leafletImageLoaderQueue addObject:hiResImageLoader]; // do this before making connection!! //
[hiResImageLoader loadImage:aLeafletURL isThumbnail:NO isBatchDownload:YES];
//// Adding object to array already retains it, so it's safe to release it here. ////
[hiResImageLoader release];
uncachedHiResIndex++;
NSLog(#"uncached hi res index: %ld, un cached hi res image array size: %lu", (long)uncachedHiResIndex, (unsigned long)[uncachedHiResImageArray count]);
}
}
- (void)loadImage:(LeafletURL*)leafletURLInput isThumbnail:(BOOL)isThumbnailInput isBatchDownload:(BOOL)isBatchDownload isRetina:(BOOL)isRetina
{
isRetina_ = isRetina;
if (mConnection)
{
[mConnection cancel];
[mConnection release];
mConnection = nil;
}
if (mImageData)
{
[mImageData release];
mImageData = nil;
}
self.leafletURL = leafletURLInput;
self.isThumbnail = isThumbnailInput;
NSString* location = (self.isThumbnail) ?leafletURL.thumbnailLocation :leafletURL.hiResImageLocation;
//// Check if the image needs to be downloaded from server. If it is a batch download, then override the local resources////
if ( ([location isEqualToString:kLocationServer] || (isBatchDownload && [location isEqualToString:kLocationResource])) && self.leafletURL.rawURL != nil )
{
//NSLog(#"final loadimage called server");
//// tell the delegate to get ride of the old image while waiting. ////
if([delegate respondsToSelector:#selector(leafletImageLoaderWillBeginLoadingImage:)])
{
[delegate leafletImageLoaderWillBeginLoadingImage:self];
}
mImageData = [[NSMutableData alloc] init];
NSURL* url = [NSURL URLWithString:[leafletURL pathForImageOnServerUsingThumbnail:self.isThumbnail isRetina:isRetina]];
[self hitServerForUrl:url];
}
//// if not, tell the delegate that the image is already cached. ////
else
{
if([delegate respondsToSelector:#selector(leafletImageLoaderDidFinishLoadingImage:)])
{
[delegate leafletImageLoaderDidFinishLoadingImage:self];
}
}
}
Currently, I'm trying to figure out how to download the images sequentially, such that we don't call hitServer until the last image is finished downloading. Do I need to be downloading in the background? Thank you for suggestions!

Answer 1

My app offers the option to download 3430 high resolution images from our server, each image of size 50k - 600k bytes.
This seems like a job for on-demand resources. Just turn these files into on-demand resources obtained from your own server, and let the system take care of downloading them in its own sweet time.

Answer 2

This sounds very much like an architectural issue. If you fire off downloads without limiting them of course you're going to start getting timeouts and other things. Think about other apps and what they do. Apps that give the user the ability to do multiple downloads often limit how may can occur at once. iTunes for example can queue up thousands of downloads, but only runs 3 at a time. Limiting to just one at a time will only slow things down for your users. You need a balance that consider your user's available bandwidth.
The other part of this is to again consider what your users want. Does every one of your uses want every single image? I don't know what you are offering them, but in most apps which access resources like images or music, it's up to the user what and when they download. Thus they only download what they are interested in. So I'd recommend only downloading what the users are viewing or have somehow requested they want to download.