I'm trying to create a simple downloader for our app. The app will download large files from our server (at least 100 MB per file), so I want to implement a downloader that will work even if the app is inactive.
I'm using NSURLSession to handle this requests. Here's how I initialised the session:
#property (nonatomic) NSURLSession *downloadSession;
- (void)viewDidLoad {
[super viewDidLoad];
// Initialise the session
NSURLSessionConfiguration *sessionConfig = [NSURLSession backgroundSessionConfigurationWithIdentifier: #"backgroundSessionIdentifier"];
[sessionConfig setHTTPMaximumConnectionsPerHost:10];
_downloadSession = [NSURLSession sessionWithConfiguration: sessionConfig delegate: self delegateQueue: nil];
}
I also implement the NSURLSessionDownloadDelegate function below to check the progress of each download:
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didWriteData:(int64_t)bytesWritten totalBytesWritten:(int64_t)totalBytesWritten totalBytesExpectedToWrite:(int64_t)totalBytesExpectedToWrite {
float totalProgress = (float)totalBytesWritten / (float)totalBytesExpectedToWrite;
NSLog(#"URL: %#, progress: %f", downloadTask.originalRequest.URL.absoluteString, totalProgress);
}
I have a UICollectionView and each cell has a button that will create a download task when tapped:
// A delegate function of the custom UICollectionViewCell
- (void)beginDownloadAtIndex:(NSInteger)fileIndex {
NSString *fileURL = [self.urls objectAtIndex: fileIndex];
NSURLSessionDownloadTask *downloadTask = [self.downloadSession downloadTaskWithURL: [NSURL URLWithString: fileURL]];
[downloadTask resume];
}
Now say that I have 10 items in the UICollectionView. I set the HTTPMaximumConnectionsPerHost to 10 so that I can download them simultaneously.
What happens is that the first 3 download tasks I created will begin downloading simultaneously, but the other 7 will not. When one of the first 3 download tasks is complete, one of the "dormant" download task will then begin downloading.
I may be doing something wrong because when I tried using the default session configuration [NSURLSessionConfiguration defaultSessionConfiguration], all 10 download tasks will begin downloading simultaneously.
I'd really appreciate it if anyone can help me to identify what's wrong with my code.
Background sessions happen out of process in a separate background daemon. It seems very likely that Apple deliberately limited the ability to change that parameter to prevent an unusually high number of transfers from being in flight while your app isn't running.
You can file a bug with Apple, but I suspect that they'll send it back as "Behaves correctly".
That said, I'm not sure what you're trying to achieve by using a large number of parallel downloads. Barring unusually high packet loss or deliberate throttling by the ISP (e.g. burst mode), there shouldn't be much difference between running ten large downloads three at a time and running them ten at a time, on average.
Related
How do photo apps upload EVERYTHING from the CameraRoll in the background?
I have the need to upload 100s of photos in the background based on date range. My app is currently using NSURLSession with the following code (I think...) But for this to work, my task scheduler has to copy the JPG to a file in App storage (see: Background Upload With Stream Request Using NSUrlSession in iOS8) before the App goes into background. For 100s of photos this takes too much time and storage.
Is there a way to use a "streams" approach, or to reliably schedule additional NSURLSession tasks from the background? My developer says that CameraRoll photos that are potentially in iCloud would cause background scheduling to fail.
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
didSendBodyData:(int64_t)bytesSent
totalBytesSent:(int64_t)totalBytesSent totalBytesExpectedToSend:(int64_t)totalBytesExpectedToSend {
NSString *identifier = task.originalRequest.allHTTPHeaderFields[#"X-Image-Identifier"];
NSDictionary *d = [self sessionInfosDictionary];
NSURLSessionTaskInfo *info = d[identifier];
double p = (double)totalBytesSent/(double)totalBytesExpectedToSend;
info.progress = p;
[self saveSessionInfos:d];
for (id<PhotosUploaderDelegate>delegate in _delegates) {
if ([delegate respondsToSelector:#selector(photoUploader:didUploadDataForAssetWithIdentifier:totalBytesSent:totalBytesExpectedToSend:)]) {
[delegate photoUploader:self didUploadDataForAssetWithIdentifier:identifier totalBytesSent:totalBytesSent totalBytesExpectedToSend:totalBytesExpectedToSend];
}
}
}
The task is not trivial, there is a lot of work to do.
Start from [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:] and +[NSURLSession uploadTaskWith...] methods.
You'll see that the tricky part is recovering from upload errors. You'll need to track each background upload in your application, by checking -[NSURLSession getTasksWithCompletionHandler:]. But first start from the beginning, with background session configurations and upload tasks.
I haven't tried it but maybe you could copy the file and start a new upload task in the background session callback? This way you might be able to copy and upload the files one at a time.
- (void)application:(UIApplication *)application handleEventsForBackgroundURLSession:(NSString *)identifier completionHandler:(void (^)())completionHandler {
// copy file and schedule new upload task
}
Everyone,
I have searched a while and it appears I could not get answer to the following question:
-(void)application:(UIApplication *)application performFetchWithCompletionHandler:(void (^)(UIBackgroundFetchResult))fetchCompletionHandler
{
.....
NSURLSessionConfiguration * config=[NSURLSessionConfiguration backgroundSessionConfiguration:#"10-MIN-COREDATA-UPDATE" ];
config.timeoutIntervalForRequest=150.0;
config.timeoutIntervalForResource=2500.0;
NSURLSession * session=[NSURLSession sessionWithConfiguration:config delegate:nil delegateQueue: nil ];
NSURLSessionDownloadTask *bgDownloadTask=[session downloadTaskWithURL:MyURL completionHandler:^(NSURL *location, NSURLResponse *response, NSError *error) {
// use the downloaded file to update a coredata big table
[self processFileAndUpdateCoreData]; // this may take long time
}] ;
[bgDownloadTask resume];
.....
}
I know background Fetch has a wall clock of 30 seconds but I read Background data transfer (NSURLSession) can download very large files and has no time limits in background mode, if I combine both of these "background" modes into one call, how is the 30-second wall clock calculated? Suppose my code finishes under 30 seconds before calling NSURLsession download (which takes long time to download AND Processing), is my background fetch call still under 30-seconds or I have well overused the allocated 30 seconds background call limit? In another way, how do I calculate time here? Should I exclude NSURL session time or should I include it for the total time elapse for background Fetch call?
Any suggestions and advices are greatly appreciated.
}
I tried MANY times and launched a background NSURL session in app background refresh call, in NSURL session delegate I use the newly downloaded file to update a big core data schema, the app always gets killed in about 90 seconds. So the answer is: 30 seconds wall clock applies no matter what you do.
I am trying to upload some 100 images to S3 in the background with AFURLSessionManager in small batches of 10 like what is being done here- Manage the number of active tasks in a background NSURLSession
I am using a shared NSURLSession and adding tasks according more tasks to this when some tasks are completed. Average size of each file is about 1.6 MB and the number of tasks that are guaranteed to run per a task queue is 5
Here is my method for adding the tasks:
(also available as an easier-to-read gist)
- (void) addTasksToSessionWithTaskObject:(Task*)taskObject withSessionInitialisationNeeded:(BOOL) needed{
NSString *filePath = [[NSBundle mainBundle] pathForResource:pathForResourceFile ofType:resourceFileType];
S3PutObjectRequest *putObjectRequest = [[S3PutObjectRequest alloc] initWithKey:targetFileKey
inBucket:_bucketname];
putObjectRequest.cannedACL = [S3CannedACL publicReadWrite];
putObjectRequest.filename = filePath;
putObjectRequest.contentType = [resourceFileType isEqualToString:#"MOV"] ? #"movie/mov" : #"image/jpg";
putObjectRequest.endpoint = #"http://s3.amazonaws.com";
putObjectRequest.contentLength=[[[NSFileManager defaultManager]
attributesOfItemAtPath:filePath error:nil] fileSize];
putObjectRequest.delegate = self;
[putObjectRequest configureURLRequest];
NSMutableURLRequest *request = [s3Client signS3Request:putObjectRequest];
NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc]initWithURL:[NSURL URLWithString:[NSString stringWithFormat:#"http://s3.amazonaws.com/UploadTest/%#",taskObject.fileKey]]];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];
if(needed) {
sharedSession = [self backgroundSession];
}
NSURLSessionUploadTask *task = [sharedSession uploadTaskWithRequest:request2 fromFile:forFileUrl];
task.taskDescription = pathForResourceFile;
[currentlyActiveTaskIdArray addObject:#([task taskIdentifier])];
[task resume];
}
And this what I've done with the delegate
- (void)URLSession:(NSURLSession *)sessionI task:(NSURLSessionTask *)task
didCompleteWithError:(NSError *)error{
dispatch_async(dispatch_get_main_queue(), ^{
__block UIBackgroundTaskIdentifier bgTaskI = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
[[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
}];
if([currentlyActiveTaskIdArray containsObject:#([task taskIdentifier])]){
[currentlyActiveTaskIdArray removeObject:#([task taskIdentifier])];
}
if(currentlyActiveTaskIdArray.count < LOWER_SLAB_FOR_TASKS + 1){
[self initiateS3UploadForSetOfTasksIsItBeginningOfUpload:NO];
}
[[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
});
}
Here is the Code to add more tasks
- (void) initiateS3UploadForSetOfTasksIsItBeginningOfUpload:(BOOL)beginning{
int i=0;
for(Task *eachTaskObject in tasksArray){
if(i < numberOfTasksTobeAdded){
[self addTasksToSessionWithTaskObject:eachTaskObject WithSessionInitialisationNeeded:NO];
i++;
}
}
}
I've been running tests with 100 files in Foreground mode and Background mode. In Foreground mode, it uploads the files at a consistant, steady and constant speed, it completes 90 files in the first 3 minutes, and the remaining 10 files in 20 seconds.
When I run the app in Background mode, I would expect it to upload the first 90 files just as fast as it did in the 3 minute Foreground window, and then slow down after that... but that's not the case. In Background mode, it uploads 15 files in the first minute, then it starts slowing down... a lot. It starts uploading in 8 file batches in slower and slower intervals: 1 minute, 3 minutes, 5 minutes, 10 minutes, and now 17 minutes. We're at 65 files 46 minutes in.
Is there a way to either keep it fast for at least the fist 3 minutes, or keep consistent speed in the background?
UPDATE: Following the comments from Clay here Ive switched back to NSURLSession from AFURLSessionManager because as he points out using block based callbacks is an extremely risky business with NSURLSession. Further I've played with HTTPMaximumConnectionsPerHost and set this around 10-this has given better results but nowhere near what I would want to be.
From what I can tell, setTaskDidCompleteBlock: is not an Apple API, NSURLSession-associated method. It is an AFURLSessionManager method (docs). If you are using AFNetworking on this, then you need to be announcing that bold, top, front and center. That is not the same, at all, as using NSURLSession. I would guess AFNetworking's background NSURLSession-based implementation comes with its own foibles and idiosyncrasies.
For my part, whatever success I've had with sustained background NSURLSession uploads are using only the stock API.
Addressing questions, etc.
Regarding AFNetworking: we use it for general web api I/O. At the time NSURLSession came out, AFNetworking really didn't robustly support app-in-background ops, so I didn't use it. Perhaps because I went through the background NSURLSession pain & hazing, I look askance at AFNetworking backgrounding under the rubric of "Now you have two problems". But maybe they have cracked the nut by now.
I strive for one NSURLSession. I started out being cavalier about creation & destruction of sessions, but found this made for some truly gnarly problems. Experiences seem to vary on this.
I use the default HTTPMaximumConnectionsPerHost, no problems there. The Apple docs are silent on the default value, but here's what lldb tells me in the random particular device/OS I chose:
(lldb) p [config HTTPMaximumConnectionsPerHost]
(NSInteger) $0 = 4
If you are having troubles with backgrounding slowing down, I doubt tweaking this is on the right track.
FWIW, background NSURLSessions do not support the block interfaces, delegate only.
I'm trying to upload my iOS photo's from the camera roll to an external server via a background fetch call. To accomodate this, i loop through ALAssetsLibrary in the background fetch to look for new photos. When new photos are found or all photo's in case of a new device, i initiate a background transfer for that photo. My plan was to start a NSURLSession and add a few tasks per fetch.
This works. The files get uploaded. But the callbacks are inconsistent. After simulating a lot of background fetches, one in a hundred times the didCompleteWithError callback isn't fired. But the biggest problem is that the Tasks don't match a lot of the times. when i check the task Identifier after creating a single task with:
NSURL *theURL = [NSURL fileURLWithPath:fullFileName isDirectory:NO];
NSURLSessionUploadTask *uploadTask = [_session uploadTaskWithRequest:request fromFile:theURL];
NSLog(#"Task id at start: %d", [uploadTask taskIdentifier]);
And in the callback:
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
{
NSLog(#"Didcomplete - task_id: %d", [task taskIdentifier]);
...
My output is:
2013-12-31 14:02:36.628 Project[18685:1303] Didcomplete - task_id: 30
2013-12-31 14:02:36.768 Project[18685:70b] Task id at start: 31
I guess i already read somewhere that the output does not have to be linear because of the background tasks. But because the identifiers differ, i can't match the output to the task and proccess the background task correctly after it's finished.
Anyone has an idea what could cause this behavior? Or what i could try?
This seems to be a simulator issue. When testing on a real device, the task identifiers match up in the didcomplete callback and proper processing of the request is possible.
While my app is in the background, I want to upload many files using NSURLSessionUploadTask.
With a suitably configured NSURLSession object, the API for queueing background uploading is:
NSURLSessionUploadTask *dataTask = [sessionObj uploadTaskWithRequest:urlRequest
fromFile:localFilePath];
[dataTask resume];
When the upload completes -- success or failure -- I get this callback in the background:
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
During the above callback, I queue another upload with
-uploadTaskWithRequest:fromFile:.
But after some files are uploaded, the callbacks stop, and so does the uploading.
Is there something I'm missing to keep uploads going? E.g. do I need to put some extra code on this callback to keep the uploads going?
-(void)URLSessionDidFinishEventsForBackgroundURLSession:(NSURLSession *)session {
AppDelegate *appDelegate = (id)[[UIApplication sharedApplication] delegate];
if (appDelegate.backgroundSessionCompletionHandler) {
void (^completionHandler)() = appDelegate.backgroundSessionCompletionHandler;
appDelegate.backgroundSessionCompletionHandler = nil;
completionHandler();
}
}
Note: I have already read this related SO question, but it didn't help.
Keep the task queue topped off
I've found that, in the background, you should avoid letting the task queue reach 0 tasks until you are really "done". Instead, you get better continuity if you always keep few tasks in the queue at all times. For instance, when the number of tasks gets down to 3, add 3 more.
Protip: for a count of active tasks, you're better off with your own tally, rather than clumsy async -[NSURLSession getTasksWithCompletionHandler:]. I add/remove the background task ID (as #(taskID)) to an NSSet (e.g. NSSet *activeTaskIDs), and use the count from that. I spell this out here.
Bracket async calls
If you do anything async during the the didComplete... callback, you must surround that with UIApplication's -beginBackgroundTaskWithExpirationHandler: and -endBackgroundTask:. Otherwise, it looks like you've popped the stack, and iOS will kill you.