I'm using AFDownloadRequestOperation + AFNetworking to download and resume a list of files from a server. The code is working great at downloading and resuming multiple files at a time. But how to queue all operations inside an operations queue and execute the operation one by one?
Here's my current code
// request the video file from server
NSString *downloadURL = [NSString stringWithFormat:#"%#%#", [recipe download_url], [step valueForKey:#"video"]];
NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:downloadURL]];
AFDownloadRequestOperation *operation = [[AFDownloadRequestOperation alloc] initWithRequest:request targetPath:videoFile shouldResume:YES];
// done saving!
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSLog(#"Done downloading %#", videoFile);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error: %ld", (long)[error code]);
}];
// set the progress
[operation setProgressiveDownloadProgressBlock:^(AFDownloadRequestOperation *operation, NSInteger bytesRead, long long totalBytesRead, long long totalBytesExpected, long long totalBytesReadForFile, long long totalBytesExpectedToReadForFile) {
float progress = ((float)totalBytesReadForFile) / totalBytesExpectedToReadForFile;
[progressBar setProgress:progress];
}];
[operation start];
You'll either need to add the operations to AFHTTPClient's operationQueue, or add them to a NSOperationQueue you've made yourself.
Then set the number of maximum concurrent operations for the queue your using to 1
Change
[operation start];
to
[[YourAFHTTPClientSubclass sharedInstance] enqueueHTTPRequestOperation:operation];
By default, this will have a maximum number of operations "determined dynamically by the NSOperationQueue object based on current system conditions".
If you really want to force everything to one at a time, do:
[YourAFHTTPClientSubclass sharedInstance].operationQueue.maxConcurrentOperationCount = 1;
This will block all network operations until the operation is finished.
Of course, you can make your own operation queue as Audun suggested, but it's probably better to let the system decide what to do based on current conditions.
Depending on your use case, you may want to set the priority of the operations of the video downloads to low:
operation.queuePriority = NSOperationQueuePriorityLow
This will allow other network operations to be placed in the queue at a higher priority than your video downloads.
Related
I'm using AFNetworking to download more or less 200 images. The problem is that the main thread is blocked during the download, not during the success/failure block.
Here is my code:
imageDownloads=[[NSMutableArray alloc]init];
for(NSString *url in liens){
NSString *totalURL = [NSString stringWithFormat:#"http://%#", url];
[imageDownloads addObject:[[ImageDownload alloc] initWithURL:[NSURL URLWithString:totalURL] filename:nil]];
}
for (int i=0; i < imageDownloads.count; i++)
{
ImageDownload *imageDownload = imageDownloads[i];
[self downloadImageFromURL:imageDownload];
}
- (void)downloadImageFromURL:(ImageDownload *)imageDownload
{
NSURLRequest *request = [NSURLRequest requestWithURL:imageDownload.url];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead) {
imageDownload.totalBytesRead = totalBytesRead;
imageDownload.totalBytesExpected = totalBytesExpectedToRead;
[self updateProgressView];
}];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSAssert([responseObject isKindOfClass:[NSData class]], #"expected NSData");
imageDownload.totalBytesExpected = imageDownload.totalBytesRead;
[self updateProgressView];
//all kind of basic stuff here I left out: I get store the data inside CoreData
NSLog(#"finished %#", imageDownload);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"error %#", error);
}];
[operation start];
}
Basically, when I launch the code, the thread is blocked for like 30-40 seconds (the pictures are about 100MB in total), and then suddenly I can see all the NSLog logs appear with the "Finished"... text. So that part if really quick. But I thought AFNetworking wasn't supposed to block the main thread while I was downloading? This also doesn't allow me to track the progress of the download...Am I doing something wrong or misinterpreting something?
You're updating the progress view in the progress block. Because AFNetworking is inherently async anyway, each of these requests will stack and run at the same time. If you're running 200 of them, that's going to freeze up the app. Try using NSOperationQueue's maxConcurrentOperationCount to limit the number of concurrent threads.
Alternatively, you could save all the trouble and just use sdwebimage.
NSURLSession has NSURLSessionDataTask and NSURLSessionDownloadTask and I'm more of a fan of data, as I don't need to retrieve it from the file system.
The NSURLSessionDownloadTaskDelegate has a method for retrieving progress on the download, though it doesn't seem the data counterpart does. If I want download progress, am I required to use NSURLSessionDownloadTask?
If so, I understand NSURLSessionDownloadTask stores the file downloaded and then you retrieve it. Does this directory where files are stored need to emptied occasionally, or is it just a temp store that gets removed as soon (or soon after) accessing it?
I'd use AFNetworking for that. The AFHTTPRequestOperation allows you to set a block for tracking progress.
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:YOUR_URL_REQUEST];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject)
{
// Completion code.
}
failure:^(AFHTTPRequestOperation *operation, NSError *error)
{
// Error handling code.
}];
Here is how you can track progress:
[operation setDownloadProgressBlock:^(NSInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite)
{
float progress = ((float)((int)totalBytesWritten) / (float)((int)totalBytesExpectedToWrite));
// Track the progress here
}];
I download asynchronously some object, I store it in array. Next for each object I download some coordinates with geocoding (it is also asynchronously), and update my database for each object with new parameters which is coordinate. My method looks like this:
- (void)downloadObjectsWithTitle:(NSString *)title andHandler:(void(^)(NSMutableDictionary *result))handler {
AFHTTPClient *httpClient = [[AFHTTPClient alloc] initWithBaseURL:url];
NSMutableURLRequest *request = [httpClient requestWithMethod:#"GET"
path:nil
parameters:nil];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[httpClient registerHTTPOperationClass:[AFHTTPRequestOperation class]];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
//I get here array of objects
//now for each object I want to download geocoding localization so i called another asynchronyous method getLocationWithTitle:andHandler;
for(int i = 0; i < resutArray.count; i++) {
[self downloadLocationWithString:[dictionary objectForKey:#"string"] andHandler:^(NSMutableDictionary *result) {
//update database;
}];
}
handler(dictionary);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error: %#", error);
}];
[operation start];
}
My question is how to downalod coordinates for each object and that fire:
handler(dictionary);
so wait for each coordinates download (for each object) before quit method (fire handler).
Thnaks for all sugestions.
Maintain a count of all the tasks. When it's zero you're done.
Assuming you're using dispatch_async in downloadLocationWithString: on a concurrent queue:
dispatch_barrier_async(queue, ^{
// will only be called after all the blocks submitted to queue have finished.
}];
(If you're using serial queue, simply call handler at the last line of the last block)
Try a global flag. set NO first. In download block, after download complete set flag to yes. You can check that flag.
I am doing a lot of URL requests (about 60 small images) and I have started to do them Asynchronously. My code adds another image (little downloading thing) and then sets a Request going.
When the request is done I want "data" to be put in the location which was originally added for it, however, I can not see how to pass "imageLocation" to the block for it to store the image in the correct location.
I have replaced the 3rd line with below which seems to work but I am not 100% it is correct (it is very hard to tell as the images are nearly identical). I am also thinking that it is possible to pass "imageLocation" at the point where the block is declared.
Can any confirm any of this?
__block int imageLocation = [allImages count] - 1;
// Add another image to MArray
[allImages addObject:[UIImage imageNamed:#"downloading.png"]];
imageLocation = [allImages count] - 1;
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:urlString]];
[request setTimeoutInterval: 10.0];
[NSURLConnection sendAsynchronousRequest:request
queue:[NSOperationQueue currentQueue]
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {
if (data != nil && error == nil)
{
//All Worked
[allImages replaceObjectAtIndex:imageLocation withObject:[UIImage imageWithData:data]];
}
else
{
// There was an error, alert the user
[allImages replaceObjectAtIndex:imageLocation withObject:[UIImage imageNamed:#"error.png"]];
}];
Dealing with asynchronous methods is a pain ;)
In your case its guaranteed that the completion block will execute on the specified queue. However, you need to ensure that the queue has a max concurrent operations count of 1, otherwise concurrent access to shared resources is not safe. That's a classic race http://en.wikipedia.org/wiki/Race_condition. The max concurrent operations of a NSOperationQueue can be set with a property.
In general, completion handlers may execute on any thread, unless otherwise specified.
Dealing with asynchronous methods gets a lot easier when using a concept called "Promises" http://en.wikipedia.org/wiki/Promise_(programming). Basically, "Promises" represent a result that will be evaluated in the future - nonetheless the promise itself is immediately available. Similar concepts are named "futures" or "deferred".
There is an implementation of a promise in Objective-C on GitHub: RXPromise. When using it you also get safe access from within the handler blocks to shared resources. An implementation would look as follows:
-(RXPromise*) fetchImageFromURL:(NSString*)urlString queue:(NSOperationQueue*) queue
{
#autoreleasepool {
RXPromise* promise = [[RXPromise alloc] init];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:urlString]];
[NSURLConnection sendAsynchronousRequest:request
queue:queue
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {
if (data != nil) {
[promise fulfillWithValue:data];
}
else { // There was an error
[promise rejectWithReason:error];
};
}];
return promise;
}
}
Then call it:
- (void) fetchImages {
...
for (NSUInteger index = 0; index < N; ++index)
{
NSString* urlString = ...
[self fetchImageFromURL:urlString, self.queue]
.then(^id(id data){
[self.allImages replaceObjectAtIndex:index withObject:[UIImage imageWithData:data]];
return #"OK";
},
^id(NSError* error) {
[self.allImages replaceObjectAtIndex:index withObject:[UIImage imageNamed:#"error.png"]];
return error;
});
}
}
A couple of thoughts:
If you want to download 60 images, I would not advise using a serial queue for the download (e.g. do not use an operation queue with maxConcurrentOperationCount of 1), but rather use a concurrent queue. You will want to synchronize the updates to make sure your code is thread-safe (and this is easily done by dispatching the updates to a serial queue, such as the main queue, for that final update of the model), but I wouldn't suggest using a serial queue for the download itself, as that will be much slower.
If you want to use the NSURLConnection convenience methods, I'd suggest something like the following concurrent operation request approach (where, because it's on a background queue, I'm using sendSynchronousRequest rather than sendAsynchronousRequest), where I'll assume you have an NSArray, imageURLs, of NSURL objects for the URLs of your images:
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
queue.maxConcurrentOperationCount = 4;
CFAbsoluteTime start = CFAbsoluteTimeGetCurrent();
NSOperation *completionOperation = [NSBlockOperation blockOperationWithBlock:^{
NSLog(#"all done %.1f", CFAbsoluteTimeGetCurrent() - start);
NSLog(#"allImages=%#", self.allImages);
}];
[imageURLs enumerateObjectsUsingBlock:^(NSURL *url, NSUInteger idx, BOOL *stop) {
NSOperation *operation = [NSBlockOperation blockOperationWithBlock:^{
NSURLResponse *response = nil;
NSError *error = nil;
NSData *data = [NSURLConnection sendSynchronousRequest:[NSURLRequest requestWithURL:url] returningResponse:&response error:&error];
if (!data) {
NSLog(#"%s sendSynchronousRequest error: %#", __FUNCTION__, error);
} else {
UIImage *image = [UIImage imageWithData:data];
if (image) {
dispatch_sync(dispatch_get_main_queue(), ^{
[self.allImages replaceObjectAtIndex:idx withObject:image];
});
}
}
}];
[queue addOperation:operation];
[completionOperation addDependency:operation];
}];
[[NSOperationQueue mainQueue] addOperation:completionOperation];
A couple of asides: First, I'm using an operation queue rather than a GCD concurrent queue, because it's important to be able to constrain the degree on concurrency. Second, I've added a completion operation, because I assume it would be useful to know when all the downloads are done, but if you don't need that, the code is obviously simper. Third, that benchmarking code using CFAbsoluteTime is unnecessary, but useful solely for diagnostic purposes if you want to compare the performance using a maxConcurrentOperationCount of 4 versus 1.
Better than using the NSURLConnection convenience methods, above, you might want to use a NSOperation-based network request. You can write your own, or better, use a proven solution, like AFNetworking. That might look like:
CFAbsoluteTime start = CFAbsoluteTimeGetCurrent();
NSOperation *completionOperation = [NSBlockOperation blockOperationWithBlock:^{
NSLog(#"all done %.1f", CFAbsoluteTimeGetCurrent() - start);
NSLog(#"allImages=%#", self.allImages);
}];
AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];
manager.responseSerializer = [AFImageResponseSerializer serializer];
manager.operationQueue.maxConcurrentOperationCount = 4;
[imageURLs enumerateObjectsUsingBlock:^(NSURL *url, NSUInteger idx, BOOL *stop) {
NSOperation *operation = [manager GET:[url absoluteString] parameters:nil success:^(AFHTTPRequestOperation *operation, id responseObject) {
[self.allImages replaceObjectAtIndex:idx withObject:responseObject];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"%s image request error: %#", __FUNCTION__, error);
}];
[completionOperation addDependency:operation];
}];
[[NSOperationQueue mainQueue] addOperation:completionOperation];
Because AFNetworking dispatches those completion blocks back to the main queue, that solves the synchronization issues, while still enjoying the concurrent network requests.
But the main take-home message here is that an NSOperation-based network request (or at least one that uses NSURLConnectionDataDelegatemethods) opens additional opportunities (e.g. you can cancel all of those network requests if you have to, you can get progress updates, etc.).
Frankly, having walked through two solutions that illustrate how to download the images efficiently up-front, I feel compelled to point out that this is an inherently inefficient process. I might suggest a "lazy" image loading process, that requests the images asynchronously as they're needed, in a just-in-time (a.k.a. "lazy") manner. The easiest solution for this is to use a UIImageView category, such as provided by AFNetworking or SDWebImage. (I'd use AFNetworking's if you're using AFNetworking already for other purposes, but I think that SDWebImage's UIImageView category is a little stronger.) These not only seamlessly load the images asynchronously, but offer a host of other advantages such as cacheing, more efficient memory usage, etc. And, it's as simple as:
[imageView setImageWithURL:url placeholder:[UIImage imageNamed:#"placeholder"]];
Just a few thoughts on efficiently performing network requests. I hope that helps.
In my iOS App,I have hundreds of image download.
I use AFNetworking to get those images.I want to manage the request number of AFWorking.
This is my code:
The problem is:It will block my UI.
THX for help me!
dispatch_group_t group = dispatch_group_create();
dispatch_queue_t queue = dispatch_queue_create(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_semaphore_t semaphore = dispatch_semaphore_create(5);
for (NSString *urlString in self.downloadImageList) {
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
dispatch_group_async(group, queue, ^{
NSURL *url = [NSURL URLWithString:urlString];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSString *filename = url.lastPathComponent;
NSURL *outputFileURL = APPLICATION_DOCUMENTS_DIRECTORY;
outputFileURL = [outputFileURL URLByAppendingPathComponent:[NSString stringWithFormat:#"%#/images/%#",self.boardId,filename]];
dispatch_group_async(group, dispatch_get_main_queue(), ^{
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
operation.outputStream = [NSOutputStream outputStreamWithURL:outputFileURL append:YES];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
[self.downloadImageList removeObject:urlString];
NIDINFO(#"download success %#",filename);
dispatch_semaphore_signal(semaphore);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NIDERROR(#"download image error:%#\n%#",error,urlString);
dispatch_semaphore_signal(semaphore);
}];
[operation start];
});
});
}
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
dispatch_release(semaphore);
dispatch_release(group);
The problem is: It will block my UI.
Well yeah, that's sort of the point of dispatch_semaphore_t. If you remove those calls, your AFNetworking calls will execute asynchronously in the background without affecting your main / UI thread.
I don't know the full context of where this code is being used, but it looks like you may want to consider having the method that calls this take a block parameter, that can be called once all of the requests have finished. You may also want to take a look at AFHTTPClient's batch operations features, which allows you to track the progress of a group of operations, getting callbacks for each individually, and when all of then finish.