I'm trying to upload my iOS photo's from the camera roll to an external server via a background fetch call. To accomodate this, i loop through ALAssetsLibrary in the background fetch to look for new photos. When new photos are found or all photo's in case of a new device, i initiate a background transfer for that photo. My plan was to start a NSURLSession and add a few tasks per fetch.
This works. The files get uploaded. But the callbacks are inconsistent. After simulating a lot of background fetches, one in a hundred times the didCompleteWithError callback isn't fired. But the biggest problem is that the Tasks don't match a lot of the times. when i check the task Identifier after creating a single task with:
NSURL *theURL = [NSURL fileURLWithPath:fullFileName isDirectory:NO];
NSURLSessionUploadTask *uploadTask = [_session uploadTaskWithRequest:request fromFile:theURL];
NSLog(#"Task id at start: %d", [uploadTask taskIdentifier]);
And in the callback:
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
{
NSLog(#"Didcomplete - task_id: %d", [task taskIdentifier]);
...
My output is:
2013-12-31 14:02:36.628 Project[18685:1303] Didcomplete - task_id: 30
2013-12-31 14:02:36.768 Project[18685:70b] Task id at start: 31
I guess i already read somewhere that the output does not have to be linear because of the background tasks. But because the identifiers differ, i can't match the output to the task and proccess the background task correctly after it's finished.
Anyone has an idea what could cause this behavior? Or what i could try?
This seems to be a simulator issue. When testing on a real device, the task identifiers match up in the didcomplete callback and proper processing of the request is possible.
Related
I'm trying to create a simple downloader for our app. The app will download large files from our server (at least 100 MB per file), so I want to implement a downloader that will work even if the app is inactive.
I'm using NSURLSession to handle this requests. Here's how I initialised the session:
#property (nonatomic) NSURLSession *downloadSession;
- (void)viewDidLoad {
[super viewDidLoad];
// Initialise the session
NSURLSessionConfiguration *sessionConfig = [NSURLSession backgroundSessionConfigurationWithIdentifier: #"backgroundSessionIdentifier"];
[sessionConfig setHTTPMaximumConnectionsPerHost:10];
_downloadSession = [NSURLSession sessionWithConfiguration: sessionConfig delegate: self delegateQueue: nil];
}
I also implement the NSURLSessionDownloadDelegate function below to check the progress of each download:
- (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didWriteData:(int64_t)bytesWritten totalBytesWritten:(int64_t)totalBytesWritten totalBytesExpectedToWrite:(int64_t)totalBytesExpectedToWrite {
float totalProgress = (float)totalBytesWritten / (float)totalBytesExpectedToWrite;
NSLog(#"URL: %#, progress: %f", downloadTask.originalRequest.URL.absoluteString, totalProgress);
}
I have a UICollectionView and each cell has a button that will create a download task when tapped:
// A delegate function of the custom UICollectionViewCell
- (void)beginDownloadAtIndex:(NSInteger)fileIndex {
NSString *fileURL = [self.urls objectAtIndex: fileIndex];
NSURLSessionDownloadTask *downloadTask = [self.downloadSession downloadTaskWithURL: [NSURL URLWithString: fileURL]];
[downloadTask resume];
}
Now say that I have 10 items in the UICollectionView. I set the HTTPMaximumConnectionsPerHost to 10 so that I can download them simultaneously.
What happens is that the first 3 download tasks I created will begin downloading simultaneously, but the other 7 will not. When one of the first 3 download tasks is complete, one of the "dormant" download task will then begin downloading.
I may be doing something wrong because when I tried using the default session configuration [NSURLSessionConfiguration defaultSessionConfiguration], all 10 download tasks will begin downloading simultaneously.
I'd really appreciate it if anyone can help me to identify what's wrong with my code.
Background sessions happen out of process in a separate background daemon. It seems very likely that Apple deliberately limited the ability to change that parameter to prevent an unusually high number of transfers from being in flight while your app isn't running.
You can file a bug with Apple, but I suspect that they'll send it back as "Behaves correctly".
That said, I'm not sure what you're trying to achieve by using a large number of parallel downloads. Barring unusually high packet loss or deliberate throttling by the ISP (e.g. burst mode), there shouldn't be much difference between running ten large downloads three at a time and running them ten at a time, on average.
How do photo apps upload EVERYTHING from the CameraRoll in the background?
I have the need to upload 100s of photos in the background based on date range. My app is currently using NSURLSession with the following code (I think...) But for this to work, my task scheduler has to copy the JPG to a file in App storage (see: Background Upload With Stream Request Using NSUrlSession in iOS8) before the App goes into background. For 100s of photos this takes too much time and storage.
Is there a way to use a "streams" approach, or to reliably schedule additional NSURLSession tasks from the background? My developer says that CameraRoll photos that are potentially in iCloud would cause background scheduling to fail.
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
didSendBodyData:(int64_t)bytesSent
totalBytesSent:(int64_t)totalBytesSent totalBytesExpectedToSend:(int64_t)totalBytesExpectedToSend {
NSString *identifier = task.originalRequest.allHTTPHeaderFields[#"X-Image-Identifier"];
NSDictionary *d = [self sessionInfosDictionary];
NSURLSessionTaskInfo *info = d[identifier];
double p = (double)totalBytesSent/(double)totalBytesExpectedToSend;
info.progress = p;
[self saveSessionInfos:d];
for (id<PhotosUploaderDelegate>delegate in _delegates) {
if ([delegate respondsToSelector:#selector(photoUploader:didUploadDataForAssetWithIdentifier:totalBytesSent:totalBytesExpectedToSend:)]) {
[delegate photoUploader:self didUploadDataForAssetWithIdentifier:identifier totalBytesSent:totalBytesSent totalBytesExpectedToSend:totalBytesExpectedToSend];
}
}
}
The task is not trivial, there is a lot of work to do.
Start from [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:] and +[NSURLSession uploadTaskWith...] methods.
You'll see that the tricky part is recovering from upload errors. You'll need to track each background upload in your application, by checking -[NSURLSession getTasksWithCompletionHandler:]. But first start from the beginning, with background session configurations and upload tasks.
I haven't tried it but maybe you could copy the file and start a new upload task in the background session callback? This way you might be able to copy and upload the files one at a time.
- (void)application:(UIApplication *)application handleEventsForBackgroundURLSession:(NSString *)identifier completionHandler:(void (^)())completionHandler {
// copy file and schedule new upload task
}
I am trying to upload some 100 images to S3 in the background with AFURLSessionManager in small batches of 10 like what is being done here- Manage the number of active tasks in a background NSURLSession
I am using a shared NSURLSession and adding tasks according more tasks to this when some tasks are completed. Average size of each file is about 1.6 MB and the number of tasks that are guaranteed to run per a task queue is 5
Here is my method for adding the tasks:
(also available as an easier-to-read gist)
- (void) addTasksToSessionWithTaskObject:(Task*)taskObject withSessionInitialisationNeeded:(BOOL) needed{
NSString *filePath = [[NSBundle mainBundle] pathForResource:pathForResourceFile ofType:resourceFileType];
S3PutObjectRequest *putObjectRequest = [[S3PutObjectRequest alloc] initWithKey:targetFileKey
inBucket:_bucketname];
putObjectRequest.cannedACL = [S3CannedACL publicReadWrite];
putObjectRequest.filename = filePath;
putObjectRequest.contentType = [resourceFileType isEqualToString:#"MOV"] ? #"movie/mov" : #"image/jpg";
putObjectRequest.endpoint = #"http://s3.amazonaws.com";
putObjectRequest.contentLength=[[[NSFileManager defaultManager]
attributesOfItemAtPath:filePath error:nil] fileSize];
putObjectRequest.delegate = self;
[putObjectRequest configureURLRequest];
NSMutableURLRequest *request = [s3Client signS3Request:putObjectRequest];
NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc]initWithURL:[NSURL URLWithString:[NSString stringWithFormat:#"http://s3.amazonaws.com/UploadTest/%#",taskObject.fileKey]]];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];
if(needed) {
sharedSession = [self backgroundSession];
}
NSURLSessionUploadTask *task = [sharedSession uploadTaskWithRequest:request2 fromFile:forFileUrl];
task.taskDescription = pathForResourceFile;
[currentlyActiveTaskIdArray addObject:#([task taskIdentifier])];
[task resume];
}
And this what I've done with the delegate
- (void)URLSession:(NSURLSession *)sessionI task:(NSURLSessionTask *)task
didCompleteWithError:(NSError *)error{
dispatch_async(dispatch_get_main_queue(), ^{
__block UIBackgroundTaskIdentifier bgTaskI = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
[[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
}];
if([currentlyActiveTaskIdArray containsObject:#([task taskIdentifier])]){
[currentlyActiveTaskIdArray removeObject:#([task taskIdentifier])];
}
if(currentlyActiveTaskIdArray.count < LOWER_SLAB_FOR_TASKS + 1){
[self initiateS3UploadForSetOfTasksIsItBeginningOfUpload:NO];
}
[[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
});
}
Here is the Code to add more tasks
- (void) initiateS3UploadForSetOfTasksIsItBeginningOfUpload:(BOOL)beginning{
int i=0;
for(Task *eachTaskObject in tasksArray){
if(i < numberOfTasksTobeAdded){
[self addTasksToSessionWithTaskObject:eachTaskObject WithSessionInitialisationNeeded:NO];
i++;
}
}
}
I've been running tests with 100 files in Foreground mode and Background mode. In Foreground mode, it uploads the files at a consistant, steady and constant speed, it completes 90 files in the first 3 minutes, and the remaining 10 files in 20 seconds.
When I run the app in Background mode, I would expect it to upload the first 90 files just as fast as it did in the 3 minute Foreground window, and then slow down after that... but that's not the case. In Background mode, it uploads 15 files in the first minute, then it starts slowing down... a lot. It starts uploading in 8 file batches in slower and slower intervals: 1 minute, 3 minutes, 5 minutes, 10 minutes, and now 17 minutes. We're at 65 files 46 minutes in.
Is there a way to either keep it fast for at least the fist 3 minutes, or keep consistent speed in the background?
UPDATE: Following the comments from Clay here Ive switched back to NSURLSession from AFURLSessionManager because as he points out using block based callbacks is an extremely risky business with NSURLSession. Further I've played with HTTPMaximumConnectionsPerHost and set this around 10-this has given better results but nowhere near what I would want to be.
From what I can tell, setTaskDidCompleteBlock: is not an Apple API, NSURLSession-associated method. It is an AFURLSessionManager method (docs). If you are using AFNetworking on this, then you need to be announcing that bold, top, front and center. That is not the same, at all, as using NSURLSession. I would guess AFNetworking's background NSURLSession-based implementation comes with its own foibles and idiosyncrasies.
For my part, whatever success I've had with sustained background NSURLSession uploads are using only the stock API.
Addressing questions, etc.
Regarding AFNetworking: we use it for general web api I/O. At the time NSURLSession came out, AFNetworking really didn't robustly support app-in-background ops, so I didn't use it. Perhaps because I went through the background NSURLSession pain & hazing, I look askance at AFNetworking backgrounding under the rubric of "Now you have two problems". But maybe they have cracked the nut by now.
I strive for one NSURLSession. I started out being cavalier about creation & destruction of sessions, but found this made for some truly gnarly problems. Experiences seem to vary on this.
I use the default HTTPMaximumConnectionsPerHost, no problems there. The Apple docs are silent on the default value, but here's what lldb tells me in the random particular device/OS I chose:
(lldb) p [config HTTPMaximumConnectionsPerHost]
(NSInteger) $0 = 4
If you are having troubles with backgrounding slowing down, I doubt tweaking this is on the right track.
FWIW, background NSURLSessions do not support the block interfaces, delegate only.
I have a requirement to download a number of files (around 500). I have an array containing all the urls of these files, I wanted to use NSURLSession so that i can support background downloading too.
I cant think of correct way to achieve this. If i am initiating next file download after one is completed then background downloading will not work.
shall I creating multiple downloading tasks and initiate?
Please suggest me how to achieve this.
Edit:
First, sorry for late response and here is a solution for your problem. Begin with downloading Apple's Simple Background Transfer sample. Then you will see the URLSessionDidFinishEventsForBackgroundURLSession method in view controller. You can modify this method for calling another download task like below sample and I think this is what you want to do.
There is also a comment over this method like this the session delegate will receive this message to indicate that all messages previously enqueued for this session have been delivered. So creating a queue for your requests could be better solution then this.
- (void)URLSessionDidFinishEventsForBackgroundURLSession:(NSURLSession *)session
{
APLAppDelegate *appDelegate = (APLAppDelegate *)[[UIApplication sharedApplication] delegate];
if (appDelegate.backgroundSessionCompletionHandler) {
void (^completionHandler)() = appDelegate.backgroundSessionCompletionHandler;
appDelegate.backgroundSessionCompletionHandler = nil;
completionHandler();
[self start:nil];
}
NSLog(#"All tasks are finished");
}
I was having problems with this. My app had to update itself and download news videos in the background using BACKGROUND FETCH to get json list of files then firing off n number of webservice calls to download these files using BACKGROUND TRANSFER
[NSURLSessionConfiguration backgroundSessionConfiguration:
For each file I was creating one NSSession and one NSURLSessionDownloadTask.
file 1 - NSSession1 > NSURLSessionDownloadTask1
file 2 - NSSession2 > NSURLSessionDownloadTask2
file 3 - NSSession3 > NSURLSessionDownloadTask3
This woke fine when the app was in the foreground.
But I had problems when the app was in background and woken by BACKGROUND FETCH
One file would download and then it would halt.
It was like only the first NSSession1 was executed.
It may have been that iOS was waiting till device was idle again to run next session but this was too slow
I got it working by having one NSSession and attaching all NSURLSessionDownloadTask3
NSURLSession * backgroundSession_ =
for(url to call){
create NSURLSessionDownloadTask1 (set its session:backgroundSession_)
create NSURLSessionDownloadTask2 (set its session:backgroundSession_)
create NSURLSessionDownloadTask3 (set its session:backgroundSession_)
}
Be careful when doing this
call NSSession finishTasksAndInvalidate not invalidateAndCancel
//[session invalidateAndCancel];
[session finishTasksAndInvalidate];
invalidateAndCancel will stop the session and not finish the other download tasks
While my app is in the background, I want to upload many files using NSURLSessionUploadTask.
With a suitably configured NSURLSession object, the API for queueing background uploading is:
NSURLSessionUploadTask *dataTask = [sessionObj uploadTaskWithRequest:urlRequest
fromFile:localFilePath];
[dataTask resume];
When the upload completes -- success or failure -- I get this callback in the background:
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
During the above callback, I queue another upload with
-uploadTaskWithRequest:fromFile:.
But after some files are uploaded, the callbacks stop, and so does the uploading.
Is there something I'm missing to keep uploads going? E.g. do I need to put some extra code on this callback to keep the uploads going?
-(void)URLSessionDidFinishEventsForBackgroundURLSession:(NSURLSession *)session {
AppDelegate *appDelegate = (id)[[UIApplication sharedApplication] delegate];
if (appDelegate.backgroundSessionCompletionHandler) {
void (^completionHandler)() = appDelegate.backgroundSessionCompletionHandler;
appDelegate.backgroundSessionCompletionHandler = nil;
completionHandler();
}
}
Note: I have already read this related SO question, but it didn't help.
Keep the task queue topped off
I've found that, in the background, you should avoid letting the task queue reach 0 tasks until you are really "done". Instead, you get better continuity if you always keep few tasks in the queue at all times. For instance, when the number of tasks gets down to 3, add 3 more.
Protip: for a count of active tasks, you're better off with your own tally, rather than clumsy async -[NSURLSession getTasksWithCompletionHandler:]. I add/remove the background task ID (as #(taskID)) to an NSSet (e.g. NSSet *activeTaskIDs), and use the count from that. I spell this out here.
Bracket async calls
If you do anything async during the the didComplete... callback, you must surround that with UIApplication's -beginBackgroundTaskWithExpirationHandler: and -endBackgroundTask:. Otherwise, it looks like you've popped the stack, and iOS will kill you.