Timeout when issuing too many AFNetworking requests - ios

I have this code to download 40 json
NSMutableArray *mutableOperations = [NSMutableArray array];
for (NSDictionary *dict in general_URL) {
NSURL *url = [dict objectForKey:#"url"];
NSString *key = [dict objectForKey:#"key"];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
operation.responseSerializer = [AFHTTPResponseSerializer serializer];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
[self.all_data setObject:[self parseJSONfile:responseObject] forKey:key];
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error: %#", error);
}];
[mutableOperations addObject:operation];
}
NSArray *operations = [AFURLConnectionOperation batchOfRequestOperations:mutableOperations progressBlock:^(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations) {
NSLog(#"progress:%f", (float)numberOfFinishedOperations / totalNumberOfOperations);
} completionBlock:^(NSArray *operations) {
NSLog (#"all done");
}];
[manager.operationQueue addOperations:operations waitUntilFinished:NO];
As you can see I use a manager to have a queue of request. The problem is that suddenly, it go in timeout with -1001 code.
It happens only in EDGE mode, in wifi and 3g it don't happen.
What's the problem?

If you specify the maxConcurrentOperationCount of the operation queue, that will control how many concurrent operations are attempted, thus mitigating any timeouts resulting from the fact that iOS limits how many simultaneous network connections are permitted:
manager.operationQueue.maxConcurrentOperationCount = 4;
[manager.operationQueue addOperations:operations waitUntilFinished:NO];
In the absence of this, when you submit your 40 operations, all of them are likely to attempt to start NSURLConnection objects, even though only 4 or 5 can really run at at a time. On slow connections, this can result in some of your latter requests timing out.
If you specify the maxConcurrentOperationCount, it won't attempt to start the latter connections until the prior connections have completed. You'll still enjoy the performance benefit of concurrent requests, but you won't be making a bunch of requests that will timeout because of the throttling of concurrent NSURLConnection requests that iOS enforces.

Related

ios - AFNetworking blocking main thread during download

I'm using AFNetworking to download more or less 200 images. The problem is that the main thread is blocked during the download, not during the success/failure block.
Here is my code:
imageDownloads=[[NSMutableArray alloc]init];
for(NSString *url in liens){
NSString *totalURL = [NSString stringWithFormat:#"http://%#", url];
[imageDownloads addObject:[[ImageDownload alloc] initWithURL:[NSURL URLWithString:totalURL] filename:nil]];
}
for (int i=0; i < imageDownloads.count; i++)
{
ImageDownload *imageDownload = imageDownloads[i];
[self downloadImageFromURL:imageDownload];
}
- (void)downloadImageFromURL:(ImageDownload *)imageDownload
{
NSURLRequest *request = [NSURLRequest requestWithURL:imageDownload.url];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead) {
imageDownload.totalBytesRead = totalBytesRead;
imageDownload.totalBytesExpected = totalBytesExpectedToRead;
[self updateProgressView];
}];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSAssert([responseObject isKindOfClass:[NSData class]], #"expected NSData");
imageDownload.totalBytesExpected = imageDownload.totalBytesRead;
[self updateProgressView];
//all kind of basic stuff here I left out: I get store the data inside CoreData
NSLog(#"finished %#", imageDownload);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"error %#", error);
}];
[operation start];
}
Basically, when I launch the code, the thread is blocked for like 30-40 seconds (the pictures are about 100MB in total), and then suddenly I can see all the NSLog logs appear with the "Finished"... text. So that part if really quick. But I thought AFNetworking wasn't supposed to block the main thread while I was downloading? This also doesn't allow me to track the progress of the download...Am I doing something wrong or misinterpreting something?
You're updating the progress view in the progress block. Because AFNetworking is inherently async anyway, each of these requests will stack and run at the same time. If you're running 200 of them, that's going to freeze up the app. Try using NSOperationQueue's maxConcurrentOperationCount to limit the number of concurrent threads.
Alternatively, you could save all the trouble and just use sdwebimage.

AFNetworking 2.0 download multiple images with completion

I'm trying to figure out a way to download multiple images with AFNewtorking 2.0. I've read a lot of posts here in SO, but can't find the answer I'm looking for, hope you guys can help me.
The problem is that I want to know when all of the downloads finished and if all images where downloaded.
So I have an array with image URL's ant trying to do something like this.
for(NSString *photoUrlString in self.photos){
NSURL *url = [NSURL URLWithString:photoUrlString];
AFHTTPRequestOperation *requestOperation = [[AFHTTPRequestOperation alloc] initWithRequest:[NSURLRequest requestWithURL:url]];
requestOperation.responseSerializer = [AFImageResponseSerializer serializer];
[requestOperation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Image error: %#", error);
}];
[requestOperation start];
}
I've found some answers with putting these requests into a queue and setting max concurrent operations to 1. But don't know how that works really.
Any help is appreciated, thanks in advance!
for(Photo *photo in array){
//form the path where you want to save your downloaded image to
NSString *constPath = [photo imageFullPath];
//url of your photo
NSURL *url = [NSURL URLWithString:photo.serverPath];
AFHTTPRequestOperation *op = [[AFHTTPRequestOperation alloc] initWithRequest:[NSURLRequest requestWithURL:url]];
op.responseSerializer = [AFImageResponseSerializer serializer];
op.outputStream = [NSOutputStream outputStreamToFileAtPath:constPath append:NO];
op.queuePriority = NSOperationQueuePriorityLow;
[op setDownloadProgressBlock:^(NSUInteger bytesRead, long long totalBytesRead, long long totalBytesExpectedToRead){
}];
op.completionBlock = ^{
//do whatever you want with the downloaded photo, it is stored in the path you create in constPath
};
[requestArray addObject:op];
}
NSArray *batches = [AFURLConnectionOperation batchOfRequestOperations:requestArray progressBlock:^(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations) {
} completionBlock:^(NSArray *operations) {
//after all operations are completed this block is called
if (successBlock)
successBlock();
}];
[[NSOperationQueue mainQueue] addOperations:batches waitUntilFinished:NO];
Try this:
// _group, _queue are iVar variable
dispatch_group_t *_group = dispatch_group_create();
dispatch_queue_t *_queue = dispatch_queue_create("com.company.myqueue2", NULL);
// all files download
for(int i = 0 ; i < numberOfFileDownloads; i++){
dispatch_group_async(_group, _queue, ^{
// here is background thread;
// download file
});
}
// all files are download successfully, this method is called
dispatch_group_notify(_group, _queue, ^{
}
Check out +[AFURLConnectionOperation batchOfRequestOperations:progressBlock:completionBlock:]
Although it's not documented, implementation is self-explanatory. Also it allows you to monitor the progress.
You will need to have an array of HTTP operations prior to using this method (this is if you decided to stick to NSURLConnection-based implementation of AFNetworking).

iOS: manage a AFHTTPRequestOperationManager

In my app I should download some JSON files, then I store these URL in a plist as you ca see in my code. After I create an 'AFHTTPRequestOperationManager' and I create a loop where I add some operation for the numbers of my 'url_list'.
NSString* plistPath = [[NSBundle mainBundle] pathForResource:#"url_json" ofType:#"plist"];
NSArray *url_list = [NSArray arrayWithContentsOfFile:plistPath];
self.manager = [AFHTTPRequestOperationManager manager];
for (id element in url_list){
NSURL *url = [NSURL URLWithString:element];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
AFHTTPRequestOperation *op = [[AFHTTPRequestOperation alloc] initWithRequest:request];
op.responseSerializer = [AFHTTPResponseSerializer serializer];
[op setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSLog(#"JSON: %#", responseObject);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error: %#", error);
}];
[self.manager.operationQueue addOperation:op];
}
Now this code should be fine, but I want to have two information:
what's the way to know the progress value of my 'manager'?, because I want to know the state of all operation in a single progress value
I want to know when an operation finish, because when an operation finish I should pass 'responseObject' to a method that parse this data
Can you help me?
Take a look at AFNetworking batching documentation:
https://github.com/AFNetworking/AFNetworking#batch-of-operations
It gives you an option to assign progress block which is called on single operation completion and on top of that you can assign completion block which will be called when all operations are completed.
If you need you can still assign completion block to single operation to parse responseObjects.

Bulk Video Upload with AFNetworking

I am attempting to get AFNetworking (1.x) to batch upload some long videos to my server, however when I use the standard functions the application's memory spikes to 400MB and then quits.
NSMutableArray *requests = [[NSMutableArray alloc] init];
NSMutableURLRequest *request = [[MyServerClient sharedClient] multipartFormRequestWithMethod:#"POST" path:URL parameters:#{ #"json": jsonString } constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
[formData appendPartWithFileData:mediaData name:idString fileName:fileName mimeType:mimeType];
}];
[request setTimeoutInterval:10800];
[requests addObject:operation];
[[MYServerClient sharedClient] enqueueBatchOfHTTPRequestOperationsWithRequests:requests progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {
NSLog(#"Number - %d %d", numberOfCompletedOperations, totalNumberOfOperations);
} completionBlock:^(NSArray *operations) {
NSLog(#"Completion block");
}];
The MyServerClient is a pretty standard subclass of AFHTTPClient:
- (id)initWithBaseURL:(NSURL *)url {
self = [super initWithBaseURL:url];
[self.operationQueue setMaxConcurrentOperationCount:1];
[self registerHTTPOperationClass:[AFJSONRequestOperation class]];
return self;
}
I'm guessing the memory crash might have something to do with the application failing to throttle the amount of concurrent operations, but in light of setting setMaxConcurrentOperationCount, I'm not sure. Does anyone have any ideas why this would be happening?
EDIT: My guess is that the crash is due to the app attempting to load the media into memory prior to attaching it to the multipart request. Is there some way to stream the upload from disk within this POST usage scenario?
For anyone struggling with the same issue, the following appears to fix my crash:
NSMutableURLRequest *request = [[MyServerClient sharedClient] multipartFormRequestWithMethod:#"POST" path:URL parameters:#{ #"json": jsonString } constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
[formData appendPartWithFileURL:mediaURL name:idString error:nil];
}];

How to batch request with AFNetworking 2?

So I'm rewriting an app for iOS 7 with AFNetworking 2.0 and I'm running into the issue of sending a batch of requests at once and tracking their progress. In the old AFNetworking there was the enqueueBatchOfHTTPRequestOperations:progressBlock:completionBlock: method on AFHTTPClient, this is clearly refactored out and I'm a bit confused on how to enqueue multiple requests.
I have created a subclass of AFHTTPSessionManager and I'm using the POST:... and GET:... methods to communicate with the server. But I can't find anything in the code and/or docs to enqueue multiple requests at once like with the old AFHTTPClient.
The only thing I can find is the undocumented batchOfRequestOperations:progressBlock:completionBlock: method on AFURLConnectionOperation, but that looks like the iOS 6 way of doing this.
Clearly I'm missing something in the new NSURLSession concept that I should use to batch requests or looking over a new AFNetworking feature. Hope someone can help me on the right track here!
tl;dr: How can I send a batch of requests with my AFHTTPSessionManager subclass?
Thanks Sendoa for the link to the GitHub issue where Mattt explains why this functionality is not working anymore. There is a clear reason why this isn't possible with the new NSURLSession structure; Tasks just aren't operations, so the old way of using dependencies or batches of operations won't work.
I've created this solution using a dispatch_group that makes it possible to batch requests using NSURLSession, here is the (pseudo-)code:
// Create a dispatch group
dispatch_group_t group = dispatch_group_create();
for (int i = 0; i < 10; i++) {
// Enter the group for each request we create
dispatch_group_enter(group);
// Fire the request
[self GET:#"endpoint.json"
parameters:nil
success:^(NSURLSessionDataTask *task, id responseObject) {
// Leave the group as soon as the request succeeded
dispatch_group_leave(group);
}
failure:^(NSURLSessionDataTask *task, NSError *error) {
// Leave the group as soon as the request failed
dispatch_group_leave(group);
}];
}
// Here we wait for all the requests to finish
dispatch_group_notify(group, dispatch_get_main_queue(), ^{
// Do whatever you need to do when all requests are finished
});
I want to look write something that makes this easier to do and discuss with Matt if this is something (when implemented nicely) that could be merged into AFNetworking. In my opinion it would be great to do something like this with the library itself. But I have to check when I have some spare time for that.
Just updating the thread... I had the same problem and after some researches I found some good solutions, but I decided to stick with this one:
I am using the project called Bolts. So, for the same sample above posted by #Mac_Cain13, it would be:
[[BFTask taskWithResult:nil] continueWithBlock:^id(BFTask *task) {
BFTask *task = [BFTask taskWithResult:nil];
for (int i = 0; i < 10; i++) {
task = [task continueWithBlock:^id(BFTask *task) {
return [self executeEndPointAsync];
}];
}
return task;
}] continueWithBlock:^id(BFTask *task) {
// Everything was executed.
return nil;
}];;
- (BFTask *) executeEndPointAsync {
BFTaskCompletionSource *task = [BFTaskCompletionSource taskCompletionSource];
[self GET:#"endpoint.json" parameters:nil
success:^(NSURLSessionDataTask *task, id responseObject) {
[task setResult:responseObject];
}
failure:^(NSURLSessionDataTask *task, NSError *error) {
[task setError:error];
}];
}];
return task.task;
}
Basically, it's stacking all of the tasks, waiting and unwrapping until there is no more tasks, and after everything is completed the last completion block is executed.
Another project that does the same thing is RXPromise, but for me the code in Bolts was more clear.
For request which can be post or get, you can use AFNetworking 2.0 for batch operation as firstly you need to create operation like this:
//Request 1
NSString *strURL = [NSString stringWithFormat:#"your url here"];
NSLog(#"scheduleurl : %#",strURL);
NSDictionary *dictParameters = your parameters here
NSMutableURLRequest *request = [[AFHTTPRequestSerializer serializer] requestWithMethod:#"POST" URLString:strURL parameters:dictParameters error: nil];
AFHTTPRequestOperation *operationOne = [[AFHTTPRequestOperation alloc] initWithRequest:request];
operationOne = [AFHTTPResponseSerializer serializer];
[operationOne setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject)
{
//do something on completion
}
failure:^(AFHTTPRequestOperation *operation, NSError *error)
{
NSLog(#"%#",[error description]);
}];
//Request 2
NSString *strURL1 = [NSString stringWithFormat:#"your url here"];
NSLog(#"scheduleurl : %#",strURL);
NSDictionary *dictParameters1 = your parameters here
NSMutableURLRequest *request1 = [[AFHTTPRequestSerializer serializer] requestWithMethod:#"POST" URLString:strURL1 parameters:dictParameters1 error: nil];
AFHTTPRequestOperation *operationTwo = [[AFHTTPRequestOperation alloc] initWithRequest:request1];
operationTwo = [AFHTTPResponseSerializer serializer];
[operationTwo setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject)
{
//do something on completion
}
failure:^(AFHTTPRequestOperation *operation, NSError *error)
{
NSLog(#"%#",[error description]);
}];
//Request more here if any
Now perform batch operation like this :
//Batch operation
//Add all operation here
NSArray *operations = [AFURLConnectionOperation batchOfRequestOperations:#[operationOne,operationTwo] progressBlock:^(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations)
{
NSLog(#"%i of %i complete",numberOfFinishedOperations,totalNumberOfOperations);
//set progress here
yourProgressView.progress = (float)numberOfFinishedOperations/(float)totalNumberOfOperations;
} completionBlock:^(NSArray *operations)
{
NSLog(#"All operations in batch complete");
}];
[[NSOperationQueue mainQueue] addOperations:operations waitUntilFinished:NO];
On AFNetworking 2.0, AFHTTPClient has been split on AFHTTPRequestOperationManager and AFHTTPSessionManager, so probably you could start with the first, which has operationQueue property.
Currently, NSURLSession tasks are not suitable for the same kind of patterns request operations use. See the answer from Mattt Thompson here regarding this issue.
Direct answer: if you need dependencies or batches, you'll still need to use request operations.

Resources