I've searched a lot within SO but I can't find the right answer for my question.
Here the problem:
I'm figuring out the right mechanism to send multiple upload requests within an NSOperation subclass. In particular, this class performs two different operation within its main method:
First it retrieves data from a local db
Then it sends the composed data to a web server
Since, these two operations can take time to executed I wrapped them, as already said, within an NSOperation.
To upload data I decided to adopt a sync pattern (I need to sync my application with the number of upload requests that has been successfully submitted to the web server).
To perform a similar upload I'm using ASIHttpRequest in a synch fashion like the following.
for (int i = 0; i < numberOfUploads; i++) {
// 1-grab data here...
// 2-send data here
ASIHTTPRequest *request = [ASIHTTPRequest requestWithURL:url];
[request startSynchronous];
NSError *error = [request error];
if (!error) {
int response = [request responseStatusCode];
if(response == 200)
uploads++;
}
}
So, my questions are:
is this a valid solution to upload data to a web server?
is it valid to create ASIHTTPRequest *request within a background thread?
do I have to use an async pattern? If yes, how to?
Note I'm using ASIHttpRequest for sync requests but I think the same pattern could be applied with NSUrlConnection class through
sendSynchronousRequest:returningResponse:error:
Thank you in advance.
To answer your questions directly:
Yes, calling an NSUrlConnection (in your case, ASI wrapper) with a sync call is valid in an NSOperation.
You can create NSUrlConnections in background threads, but there are a couple of things to remember here:
If you use it on a background thread, you have to either call the synchronous methods, or you have to keep the thread alive yourself. Using the async in an NSOperation is explained pretty well here: How do I do an Asychronous NSURLConnection inside an NSOperation? I have used this pattern and it works well.
NSUrlConnnection Delegate callbacks call back to the thread that the NSUrlConnection was created on. Just something to remember.
You do not have to use the async pattern, but you can. The async pattern provides more flexibility. For example, if you need to cancel the operation, you have the ability to cancel the NSUrlConnection request with the async pattern. With the sync pattern, you are forced to let it run (unless you kill the thread explicitly).
One note: I would reconsider using ASI as it is no longer supported. AFNetworking seems to be the most popular replacement, though I chose to start using NSUrlConnection directly.
whenever you want to call using ASIHTTPRequest in background thread you have to call synchronous call only, because threads will close the request as soon they are sent,and about your questions
1, this is valid solution but call using synchronours only
2. you can call ASIHTTPRequest in background thread or you can call it using
nsurlconnection sendSynchronousRequest:returningResponse:error:
async pattern will not work for background thread you have to use them on main thread only .
hope it helps .
Related
If an iOS app has to make hundreds of server requests in background and save the result in local mobile database, which approach would be better in terms of performance (less crashes)?
Passing all requests as 1 block in Global Background Queue
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
for (i=0;i<users;i++)
{
Call Api 1
Call Api 2
Call Api 3
}
});
OR
Creating 'n' number of User Serial Queues and adding all 3 api calls as individual blocks in each serial queue.
for (i=0;i<users;i++)
{
dispatch_queue_t myQueue = dispatch_queue_create([[users objectAtIndex:i] UTF8String], DISPATCH_QUEUE_SERIAL);
dispatch_async(myQueue, ^{
Call Api 1
});
dispatch_async(myQueue, ^{
Call Api 2
});
dispatch_async(myQueue, ^{
Call Api 3
});
}
Edit: In each Call Api, I am using NSOperationQueue.
queue = [[NSOperationQueue alloc] init];
[NSURLConnection sendAsynchronousRequest:request queue:queue completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {..}
I would suggest using NSOperations rather. Create 'n' NSOperations one for each API. Create a NSOperatioQueue, add the NSOperations to queue and sit back and relax while iOS takes the burden of deciding how many operations to run concurrently and all other complicated task ascociated with thread and memory for you :)
The major difference between GCD and NSOperations is the ability to pause and resume the operations :) In case of GCD once submitted operation is bound to happen and there is no way you can skip them or pause them :)
Adding the dependencies between multiple operations in GCD is cumbersome where as adding dependencies and prioritising tasks in NSOperations is just a matter of few statements :)
EDIT
As per your edit you are already using NSOperation for each API. So there is absolutely no need to call the NSOperations in dispatch_async again :) Rather consider creating an NSOperationQueue and adding these NSOperations to queue :)
QA for your comments
1.What if I create new NSOperationQueue every time instead of adding NSOperations to single NSOperationQueue?
Creating a seperate queue for each NSOperation is never a great idea :)
NSOperationQueue was suggested only with the intention of reducing the complication you will have to bear with multiple threads manually :)
When you submit an NSOperation to NSOperationQueue you can specify how many concurrent operations can be performed by NSOperationQueue.
Notice that it is just an upper value :) If you specify maximum concurrent operation allowed to 10 it means when iOS has resources like memory and CPU cycles it may carry out 10 operations.
You being a developer may not be always in a great position to decide what is the optimal number of threads that system can afford :) but OS can always do it efficiently. So it is always advisable to transfer these burden on OS as much as possible :)
But if you want to have a separate thread for each API calls rather then creating a separate queue you can consider executing the NSOpertaions individually by calling [NSOperation start] Creating an NSOperationQueue for each NSOperation is overhead.
I believe if you have any experience with JAVA you must have came across ExecutorPool concept. NSOperationQueue does exactly what ExecutorPool does for JAVA :)
2.Should I use [NSURLConnection sendAsynchronousRequest:request queue:[NSOperationQueue mainQueue] .. instead of [NSURLConnection sendAsynchronousRequest:request queue:[[NSOperationQueue alloc] init]
I believe you are aware that all you UI operations are performed in main thread and main thread makes use of Main Queue to dispatch the events. Performing the lengthy operations on Main Queue and keeping it busy will lead to a very bad user experience.
Calling 100 API's on Main queue may lead to user uninstalling your app and giving the worst possible rating :)
I guess you now know the answer for your question now :) to be specific use [[NSOperationQueue alloc] init] for your situation.
First you should read this: GCD Practicum.
Second you shouldn't roll your own solution here. Instead use AFNetworking, and just make the requests as needed. It has its own operation queue already setup so you don't need to deal with that. Then set the maximum number of concurrent requests to some value that you tune by hand. Start with four.
Setup
I have two areas in my program - branch 1 and branch 2 - where network requests are made asynchronously 1 concurrent GET request at a time for each area. Requests are sent 1 at a time because the server has a tiny few milliseconds of a grace period for any user making requests to the server. Running 1 concurrent request at a time aims to aid that grace period.
The operation has been made such that if any request fails in any branch - there is a fail safe to just repeat the request again.
Problem:
When I run these branches separately from each other, i.e. not at the same time, the server is happy. However, as soon as I allow for both operations to take place at the same time the server throws a 429 error which is the error letting the user know that there are too many requests coming in at any one time. What then happens is that half of the requests fails, and then because of the fail safe the requests are being sent again for processing until they're all completed.
It's a waste of time and resources on behalf of the user's data package.
Scenario
Branch 1 sends out 10 requests (normally its within the thousands, processing one at a time, but for simplicity purposes we'll keep it at 10).
These 10 requests will be processed concurrently 1 at a time very quickly and return with their appropriate data objects.
Now if we run branch 1 operation again - 10 requests concurrently and then add branch 2's 10 requests, this will create 20 request operations - twice as many - GOING OUT to the server which will cause roughly half of the requests from both branch 1 and 2 to fail, with the fail-safe then kicking in to resend those request operations. boohoo.
Question
Is there a way to funnel these request operations coming out of my application regardless of where these two branches lay within my app so that we can control and bottleneck the request operations going out to the server so that the server won't be bombarded with requests only for half of the requests to fail?
Possible solution
One idea I had was to create a singleton that would act as a queue as well as the bottle neck required that would allow me to add request operations from ANY branch anywhere within my code base - hence why i thought a singleton would be a great idea for this - and then as soon as it realises a request operation has been added to the queue to then to have that queue run 1 concurrent request at at time.
This I think will work great for my purposes, I just didn't know how to handle the completion blocks of the requests since the request operation from Branch 1 are different from the completion blocks of the request operations from Branch 2.
How would you manage that? Is there anything we can do?
Edit 1 - What I'm currently doing
In Branch 1 - Using AFHTTPOperation:
1) For each process I create a for loop that iterates 10 times adding 10 AFHTTPRequestOperations to a local array called multipleOperations of type NSMutableArray.
2) I then add the multipleOperations array to a batchOfRequestOperations method like so:
NSArray *operations = [AFURLConnectionOperation batchOfRequestOperations:muiltipleOperations
progressBlock:^(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations){
//Do some stuff here for each iteration
}
completionBlock:^(NSArray*operations){
//Call the method that repeats this process again.
}];
3) I then add the operations to the mainQueue like so
[[NSOperationQueue mainQueue] addOperations:operations waitUntilFinished:NO];
I'm now wondering whether this process in branch one even has it set to 1 concurrenncy, cant see that it is :\
In Branch 2 using NSURLSessionDataTask (just realised, wouldnt it be easier if they were all using the same subclass? lol
I notice that I am using a AFHTTPSessionManager that creates NSURLSessionDataTasks.. which doesnt run in a for loop but instead calls its own method for another request to be sent out EVERY TIME the request executes its own completion block. So it will forever keep calling itself as soon as the previous request in branch 2 is done. This is how I know its running only 1 request at a time concurrently. So thats good.
Is there anyway to make this more cleaner?
No where am I using HTTPRequestOperationManager :( Can someone show me how things can be done properly?
Thank you.
Based on your edit, I can see a some things you'll probably want to change. First off, yes, it will be much easier if you use either AFHTTPOperations or NSURLSessionDataTasks, but not both.
If you're going to use AFHTTPOperation, then don't queue the operations on [NSOperationQueue mainQueue]. The reason being is that that is the main UI queue and you don't want your network operations blocking it. What you should be doing is creating your own NSOperationQueue and setting its concurrency to 1.
NSOperationQueue *networkQueue = [[NSOperationQueue alloc] init];
networkQueue.maxConcurrentOperationCount = 1;
Using your idea of a singleton to store that queue somewhere you can access from anywhere in your app would be a good way to go about making sure you send all your network operations through the same queue.
That being said, I'd recommend using NSURLSessionDataTask with AFHTTPSessionManager instead. The docs for AFNetworking state:
Developers targeting iOS 7 or Mac OS X 10.9 or later that deal
extensively with a web service are encouraged to subclass
AFHTTPSessionManager, providing a class method that returns a shared
singleton object on which authentication and other configuration can
be shared across the application.
That sounds like a good way to accomplish what you want. Basically you'll create a subclass of AFHTTPSessionManager with a class method that creates an instance initialized with a custom NSURLSessionConfiguration. You can configure the NSURLSessionConfiguration however is appropriate for your app, but the main property you'll be interested in for this particular issue is HTTPMaximumConnectionsPerHost:
This property determines the maximum number of simultaneous
connections made to each host by tasks within sessions based on this
configuration.
With that set to 1, you can let AFHTTPSessionManager (well, actually, NSURLSession) worry about making sure that only one request is made to your server at any given time.
Something along these lines:
#interface AppHTTPSessionManager : AFHTTPSessionManager
+ (instancetype)appSession;
#end
#implementation AppHTTPSessionManager
+ (instancetype)appSession {
static AppHTTPSessionManager *_appSession = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
configuration.HTTPMaximumConnectionsPerHost = 1;
_appSession = [[AppHTTPSessionManager alloc] initWithSessionConfiguration:configuration];
});
return _appSession;
}
#end
Then, whenever you want to make a network call, make sure you create your NSURLSessionDataTask from [AppHTTPSessionManager appSession]. If you do, those tasks should automatically be limited to one request to the server at a time.
I have a class that sends webservice calls and delivers the response via delegation.
I now want to add a caching layer in between the views and my webservice class.
This new class should serialize the calls in a way that every call is delayed until the callback of the previous call has finished.
I tried to realize that with GCD:
- (void)callWebserviceFunctionX {
dispatch_group_notify(self.serviceGroup, self.serialQueue, ^{
dispatch_group_enter(self.serviceGroup);
// call webservice
});
}
And in the callback:
-(void)callbackFunctionXWithResponse:(id)response {
// do something
dispatch_group_leave(self.serviceGroup);
}
The idea to group each call and its callback with dispatch_group_enterand dispatch_group_leave and wait for previous groups using dispatch_group_notify.
However, this approach does not seem to work as I intended.
Is there a better way to achieve this?
UPDATE:
I tried every combination of dispatch_group_enter, dispatch_group_leave, dispatch_group_notifyand dispatch_group_async I can think of without success.
I also thought about NSOperationand NSOperationQueue, but - if I understood correctly - that would force me to write a separate class for every webservice call.
Is there another alternative I did not think of yet?
I think you'd be better off using NSOperation, dependencies between them to ensure serialisation and NSOperationQueue to run them.
To avoid creating a NSOperation subclass for each request you could use the builtin NSBlockOperation, you provide a block to each instance and adding dependencies between the NSBlockOperation instances should give you the aimed serialisation.
Hope this helps.
Regards
You could use MKNetworkKit as your Networking solution. This uses NSOperationQueue under the hood and you can use NSOperation dependencies to serialize your request / responses. MKNetworkKit also supports caching of the responses so might help with your caching implementation also.
MKNetworkKit Overview
http://blog.mugunthkumar.com/products/ios-framework-introducing-mknetworkkit/
Someone had a similar problem using MKNetworkKit and GCD
MKNetworkKit and GCD dispatch_group_t
What are the differences among following types calling Queues,which one is the best?
A)
[NSURLConnection sendAsynchronousRequest:urlRequest queue:[NSOperationQueue mainQueue] completionHandler:^(NSURLResponse *response, NSData *data, NSError *error)
B)
NSOperationQueue *myQueue = [[NSOperationQueue alloc] init];
[myQueue addOperationWithBlock:^{
// Background work
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// Main thread work (UI usually)
}];
}];
C)Adding NSoperation which has subclassed to NSoperationQueue for example http://www.cimgf.com/2008/02/16/cocoa-tutorial-nsoperation-and-nsoperationqueue/
D)
[[NSOperation mainQueue] addOperation:myOperation];
IS it right approach? because this code adds NSoperation to mainQueue.This is not good for
background task.Usually mainQueue will be used for updating UI Only.
E)If I have missed anything except above call methods for Queue , please mention them also in answer.
None of those examples is best, and they're not even very different from each other. As the name implies, NSOperationQueue is a queue, i.e. a first in first out (FIFO) data structure, that contains operations. You can create your own operation queues, or you can use an existing one.
Example A is a reasonable example of using an existing queue, the main queue. You wouldn't want to put a synchronous network request on the main thread (which is what you get with +mainQueue) because it would block the user interface, but it's not necessarily a bad choice here because the request is asynchronous and the operation queue being passed in is used only to run the completion handler. Indeed, the completion handler might need to manipulate the user interface, and that should be done from the main queue.
Example B illustrates creating a new operation queue and scheduling an operation on that queue. That operation in turn schedules another operation on the main queue. This is a pretty typical scenario -- again, you should only manipulate the UI from the main thread, so it's common to have a background operation create an operation that runs on the main thread in order to update the UI.
Example C is similar to B, except that the operation in question is a subclass rather than one created from a block. NSOperation existed before Grand Central Dispatch and blocks came along, and it used to be that the only way to create an operation that did something interesting was to subclass NSOperation and override -main. The blog post you linked says it was posted Feb. 16, 2008, which would certainly have been from that era. Creating operations from blocks is a newer and often more concise style, but there's nothing wrong with subclassing especially if you might need to perform the same kind of operation in several places. Note also that that article uses -performSelectorOnMainThread:withObject:waitUntilDone:, which is an easier way to run something on the main thread than creating another NSOperation subclass.
Example D is too vague to really comment on -- it merely shows how to add some unspecified operation to the main queue. You're right that you wouldn't want to do that for long-running operations -- those should be scheduled on a background queue instead to avoid blocking the UI. But without knowing what myOperation does, you can't say that it's right or wrong.
So, none of your examples are incorrect, and none is really better than another. To the extent that they're different, it's because they're used in different situations. For example, NSURLConnection takes an NSOperationQueue and a block as parameters because it needs to wait until the connection is done before it schedules an operation created from the completion block. Once the connection is done, though, NSURLConnection will do pretty much what you see in examples B and D.
My example on iOS 6:
10 Multi-Part requests need to be sent (in order) to the server.
(so the request forms a queue)
progress should be shown.
if one request fails all following should fail
a request queue should be cancellable
Can AFNetworking help me with this? Or should I try to build something with NSOperations and run the loops myself?
If I need to pass context data between theses requests for example a transaction id produced by the first request. Are there any considerations about thread visibility I need to consider?
AFNetworking can do this. I recommend that you use AFHTTPRequestOperationManager (which itself uses NSOperation), rather than AFHTTPSessionManager. There are ways to do it with AFHTTPSessionManager, but none as elegant as with operations.
Under the hood, here's what you'd do without the manager:
You will use a request serializer to make your NSMutableURLRequest (for example, [AFHTTPRequestSerializer -multipartFormRequestWithMethod:URLString:parameters:constructingBodyWithBlock:error:]; there's a similar JSON request serializer too).
Once you have a URL Request, make the operation with [AFHTTPRequestOperation -initWithRequest:]. You should also set its completion blocks.
Finally, add your operation to [AFHTTPRequestOperationManager manager].operationQueue and start it.
Now that you understand how this is basically all working together, here's a simpler approach:
Subclass AFHTTPRequestOperationManager, optionally setting the requestSerializer if you don't like the default
Override (or copy with new implementation) -POST:parameters:constructingBodyWithBlock:success:failure:] - what you want to do is NOT start your operation right away.
Set the NSOperation dependency chains
start the first one