I'm using the AFNetworking library, which is excellent, however I'm having trouble keeping track of operations in the NSOperationQueue. I am adding NSOperation objects to the NSOperationQueue, and I need to keep track of progress - so update a UIProgressView to show how far the queue is to completion and then also execute a block of code once the queue is complete.
I've tried KVO - using the answer here: Get notification when NSOperationQueue finishes all tasks however I come across the problem (elaborated on the second answer down there) where sometimes operations in the queue may complete fast enough to temporarily decrement the operationCount property to 0 - which then cause issues with the code in the accepted answer - i.e. prematurely execute the code to be executed after all objects in the queue have finished and progress tracking will not be accurate as a result.
A variation I've tried is checking for operationCount == 0 in the success block of each NSOperation that I add to the NSOperationQueue and then executing code based on that, e.g.
[AFImageRequestOperation *imgRequest = [AFImageRequestOperation imageRequestOperationWithRequest:urlRequest success:^(UIImage *image) {
//Process image & save
if(operationQ.operationCount == 0){
// execute completion of Queue code here
}
else {
// track progress of the queue here and update UIProgressView
}
}];
However, I come up with the same issue as I do with KVO.
I've thought about using GCD with a dispatch queue using a completion block - so asynchronously dispatch an NSOperationQueue and then execute the completion block but that doesn't solve my issue with regard to keeping track of the queue progress to update UIProgressView.
Also not used
AFHttpClient enqueueBatchOfHTTPRequestOperations:(NSArray *) progressBlock:^(NSUInteger numberOfFinishedOperations, NSUInteger totalNumberOfOperations)progressBlock completionBlock:^(NSArray *operations)completionBlock
since my images are coming from a few different URLs (rather than one base url).
Any suggestions or pointers will be appreciated. Thanks.
Just a final update:
Solved this issue using the AFHTTPClient enqueueBatchOfHTTPRequestOperations in the end with the help of Matt (see accepted answer) and note the comments as well.
I did come across another solution that does not make use of AFHTTPClient but just NSOperationQueue on its own. I've included this as well in case it's of any use to anyone, but if you're using the AFNetworking Library I'd recommend the accepted answer (since it's most elegant and easy to implement).
AFHTTPClient -enqueueBatchOfHTTPRequestOperations:progressBlock:completionBlock: is the correct way to do this. The method takes an array of request operations, which can be constructed from any arbitrary requests—not just ones sharing a domain.
Another (not as elegant) solution, if you're only using NSOperationQueue and not the AFHTTPClient, is the following (assuming the following code will be in some loop to create multiple requests and add to the NSOperationQueue).
[AFImageRequestOperation *imgRequest = [AFImageRequestOperation imageRequestOperationWithRequest:urlRequest success:^(UIImage *image) {
//Process image & save
operationNum++
//initially operationNum set to zero, so this will now increment to 1 on first run of the loop
if(operationNum == totalNumOperations){
//totalNumOperations would be set to the total number of operations you intend to add to the queue (pre-determined e.g. by [array count] property which would also be how many times the loop will run)
// code to execute when queue is finished here
}
else {
// track progress of the queue here and update UIProgressView
float progress = (float)operationNum / totalNumOperations
[progView setProgress:progress] //set the UIProgressView.progress property
}
}];
Adding these NSOperation objects to the NSOperationQueue will ensure the success block of each operation will complete before executing the queue completion code which is embedded in the success block of each NSOperation object. Note NSOperationQueue.operationCount property isn't used since it is not reliable on fast operations since there may be an state in between an operation exiting a queue and just before the next one is added where the operationCount is zero and so if we compared NSOperationQueue.operationCount = 0 instead then the completion code for the queue would execute prematurely.
Related
I'm writing an app where I've got a long running server-syncronization task running in the background, and I'd like to use NSOperation and NSOperationQueue for this. I'm leaning this way, since I need to ensure only one synchronisation operation is running at once.
My question arises since my architecture is built around NSNotifications; my synchronisation logic proceeds based on these notifications. From what I can see, NSOperation logic needs to be packed into the main method. So what I'm wondering is if there is any way to have an NSOperation finish when a certain notification is received. I suspect this is not the case, since I haven't stumbled upon any examples of this usage, but I figured I'd ask the gurus in here. Does an NSOperation just finish when the end of the main method is reached?
There is no reason a NSOperation cannot listen for a notification on the main thread, but either the finish logic must be thread safe, or the operation must keep track of its current thread.
I would recommend a different approach. Subclass NSOperation to support a method like -finishWithNotification: Have a queue manager that listens for the notification. It can iterate through its operations finishing any operations which respond to -finishWithNotification:.
- (void)handleFinishNotification:(NSNotification *)notification
{
for (NSOperation *operation in self.notificationQueue) {
if ([operation isKindOfClass:[MYOperation class]]) {
dispatch_async(self.notificationQueue.underlyingQueue), ^{
MYOperation *myOperation = (MYOperation *)operation;
[myOperation finishWithNotification:notification];
});
}
}
}
If I understood you correctly concurrent NSOperation is what you need.
Concurrent NSOperation are suitable for long running background/async tasks.
NSOperation Documentation
See: Subclassing Notes & Operation Dependencies Section
EDIT:(Adding more explanation)
Basically concurrent operations do not finish when main method finishes. Actually what concurrent operation mean is that the control will return to calling code before the actual operation finishes. The typical tasks that are done in start method of concurrent operation are: Mark operation as executing, start the async work(e.g. NSURLConnection async call) or spawn a new thread which will perform bulk of the task. And RETURN.
When the async task finishes mark the operation as finished.
I seem to have some confusion between dispatch_queue_t and NSOperationQueue queues.
By default, AFNetworking's AFImageRequestOperation will execute the success callback block on the application's main thread. To change this, AFHTTPRequestOperation has the property successCallbackQueue which lets you choose on which queue to run the callback.
I'm trying to execute the success callback on the same background queue / background threads which already did the HTTP request. Instead of returning to the main thread, the NSOperationQueue which ran the HTTP request should run the callback as well, since there are some heavy calculations I need to do using some of the returned images.
My first try was to set successCallbackQueue to the NSOperationQueue instance on which the AFImageRequestOperation ran. However, the successCallbackQueue property is of type dispatch_queue_t, so I need a way to get the underlying dispatch_queue_t of my NSOperation instance, if there is such a thing.
Is that possible, or do I need to create a separate dispatch_queue_t?
The reason I ask: It's somewhat strange that AFNetworking inherits from NSOperation, but expects us to use dispatch_queue_t queues for the callbacks. Kind of mixing the two paradigmas dispatch_queue_t and NSOperationQueue.
Thanks for any hints!
There is no such thing, there isn't a one-to-one correspondence of an NSOperationQueue and a dispatch_queue_t, the queueing concepts in the two APIs are very different (e.g. NSOperationQueue does not have strict FIFO queueing like GCD does).
The only dispatch queue used by NSOperationQueue to execute your code is the default priority global concurrent queue.
NSOperationQueue is not your bottleneck with AFNetworking. Request operations are bound by network, not CPU or memory. All of the work is done asynchronously in dispatch queues, which are accessible as properties in AFHTTPRequestOperation.
It is not advisable to use the network thread to do any processing. This will not improve performance in any way.
Instead, if you're noticing performance issues, try limiting the maximum number of concurrent operations in the operation queue, as a way to indirectly control the amount of work being done by those background processing queues.
It is interesting that AFHTTPClient uses an NSOperationQueue to run the AFHTTPRequestOperations but GCD dispatch_queues to handle the results.
Regarding NSOperationQueue the Apple docs say:
Note: In iOS 4 and later, operation queues use Grand Central Dispatch to execute operations.
but there doesn't seem to be a public API to get the dispatch_queue for a given operation.
If it's not that important to you that the success callback has to be on exactly the same queue/thread that the original operation was executed why not set:
successCallbackQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND,0);
XCode 6.4 for iOS 8.4, ARC enabled
1) "...so I need a way to get the underlying dispatch_queue_t of my NSOperation instance, if there is such a thing."
There is a property of NSOperationQueue that can help:
#property(assign) dispatch_queue_t underlyingQueue
It can be used as follows to assign to NSOperationQueue:
NSOperationQueue *concurrentQueueForServerCommunication = [[NSOperationQueue alloc] init];
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
concurrentQueueForServerCommunication.underlyingQueue = concurrentQueue;
Or assign from NSOperationQueue:
NSOperationQueue *concurrentQueueForServerCommunication = [[NSOperationQueue alloc] init];
dispatch_queue_t concurrentQueue = concurrentQueueForServerCommunication.underlyingQueue;
Not sure if the API you are using for network communication updates your UI after the network task's completion, but just in case it does not, then you must know to get back onto the main queue when the completion block is executed:
dispatch_sync(dispatch_get_main_queue(), ^{
//Update your UI here...
}
Hope this helps! Cheers.
First of all, it's a good behaviour to execute the success of the AFImageRequestOperation on the main thread because the main usage of this operation is to download an image in background and display it on the UI (which should be on the main thread), but in order to satisfy the needs of those user (your case too) who want to execute the callback on other threads, there is also a successCalbackQueue.
So you can create your own dispatch_queue_t with dispatch_queue_create method or the recommended way, you should use dispatch_get_global_queue to get the main queue.
On each case, make sure that in your success block, if you are making some changes to the UI, put them inside dispatch_async(dispatch_get_main_queue(), ^{ // main op here});
Swift 3 code, based on #serge-k's answer:
// Initialize the operation queue.
let operationQueue = OperationQueue()
operationQueue.name = "com.example.myOperationQueue"
operationQueue.qualityOfService = .userInitiated
// Initialize a backing DispatchQueue so we can reuse it for network operations.
// Because no additional info is give, the dispatch queue will have the same QoS as the operation queue.
let operationQueueUnderlyingQueue = DispatchQueue(label: "com.example.underlyingQueue")
operationQueue.qualityOfService.underlyingQueue = operationQueueUnderlyingQueue
You can then use this in Alamofire (or AFNetworking) in the following manner:
Alamofire.request("https://example.com/get", parameters: nil).validate().responseJSON(queue: operationQueue.underlyingQueue) { response in
response handler code
}
A warning here, from Apple's documentation on setting the OperationQueue's underlying queue:
The value of this property should only be set if there are no operations in the queue; setting the value of this property when operationCount is not equal to 0 raises an invalidArgumentException. The value of this property must not be the value returned by dispatch_get_main_queue(). The quality-of-service level set for the underlying dispatch queue overrides any value set for the operation queue's qualityOfService property.
I have an NSOperationQueue that handles importing data from a web server on a loop. It accomplishes this with the following design.
NSURLConnect is wrapped in an NSOperation and added to the Queue
On successful completion of the download (using a block), the data from the request is wrapped in another NSOperation that adds the relevant data to Core Data. This operation is added to the queue.
On successful completion (using another block), (and after a specified delay) I call the method that started it all and return to step 1. Thus, i make another server call x seconds later.
This works great. I'm able to get data from the server and handle everything on the background. And because these are just NSOperations I'm able to put everything in the background, and perform multiple requests at a time. This works really well.
The ONLY problem that I currently have is that I'm unable to successfully cancel the operations once they are going.
I've tried something like the following :
- (void)flushQueue
{
self.isFlushingQueue = YES;
[self.operationQueue cancelAllOperations];
[self.operationQueue waitUntilAllOperationsAreFinished];
self.isFlushingQueue = NO;
NSLog(#"successfully flushed Queue");
}
where self.isFlushingQueue is a BOOL that I use to check before adding any new operations to the queue. This seems like it should work, but in fact it does not. Any ideas on stopping my Frankenstein creation?
Edit (Solved problem, but from a different perspective)
I'm still baffled about why exactly I was unable to cancel these operations (i'd be happy to keep trying possible solutions), but I had a moment of insight on how to solve this problem in a slightly different way. Instead of dealing at all with canceling operations, and waiting til queue is finished, I decided to just have a data structure (NSMutableDictionary) that had a list of all active connections. Something like this :
self.activeConnections = [NSMutableDictionary dictionaryWithDictionary:#{
#"UpdateContacts": #YES,
#"UpdateGroups" : #YES}];
And then before I add any operation to the queue, I simply ask if that particular call is On or Off. I've tested this, and I successfully have finite control over each individual server request that I want to be looping. To turn everything off I can just set all connections to #NO.
There are a couple downsides to this solution (Have to manually manage an additional data structure, and every operation has to start again to see if it's on or off before it terminates).
Edit -- In pursuit of a more accurate solution
I stripped out all code that isn't relevant (notice there is no error handling). I posted two methods. The first is an example of how the request NSOperation is created, and the second is the convenience method for generating the completion block.
Note the completion block generator is called by dozens of different requests similar to the first method.
- (void)updateContactsWithOptions:(NSDictionary*)options
{
//Hard coded for ease of understanding
NSString *contactsURL = #"api/url";
NSDictionary *params = #{#"sortBy" : #"LastName"};
NSMutableURLRequest *request = [self createRequestUsingURLString:contactsURL andParameters:params];
ConnectionCompleteBlock processBlock = [self blockForImportingDataToEntity:#"Contact"
usingSelector:#selector(updateContactsWithOptions:)
withOptions:options andParsingSelector:#selector(requestUsesRowsFromData:)];
BBYConnectionOperation *op = [[BBYConnectionOperation alloc] initWithURLRequest:request
andDelegate:self
andCompletionBlock:processBlock];
//This used to check using self.isFlushingQueue
if ([[self.activeConnections objectForKey:#"UpdateContacts"] isEqualToNumber:#YES]){
[self.operationQueue addOperation:op];
}
}
- (ConnectionCompleteBlock) blockForImportingDataToEntity:(NSString*)entityName usingSelector:(SEL)loopSelector withOptions:(NSDictionary*)options andParsingSelector:(SEL)parseSelector
{
return ^(BOOL success, NSData *connectionData, NSError *error){
//Pull out variables from options
BOOL doesLoop = [[options valueForKey:#"doesLoop"] boolValue];
NSTimeInterval timeInterval = [[options valueForKey:#"interval"] integerValue];
//Data processed before importing to core data
NSData *dataToImport = [self performSelector:parseSelector withObject:connectionData];
BBYImportToCoreDataOperation *importOperation = [[BBYImportToCoreDataOperation alloc] initWithData:dataToImport
andContext:self.managedObjectContext
andNameOfEntityToImport:entityName];
[importOperation setCompletionBlock:^ (BOOL success, NSError *error){
if(success){
NSLog(#"Import %#s was successful",entityName);
if(doesLoop == YES){
dispatch_async(dispatch_get_main_queue(), ^{
[self performSelector:loopSelector withObject:options afterDelay:timeInterval];
});
}
}
}];
[self.operationQueue addOperation:importOperation];
};
}
Cancellation of an NSOperation is just a request, a flag that is set in NSOperation. It's up to your NSOperation subclass to actually action that request and cancel it's work. You then need to ensure you have set the correct flags for isExecuting and isFinished etc. You will also need to do this in a KVO compliant manner. Only once these flags are set is the operation finished.
There is an example in the documentation Concurrency Programming Guide -> Configuring Operations for Concurrent Execution. Although I understand that this example may not correctly account for all multi-threaded edge cases. Another more complex example is provided in the sample code LinkedImageFetcher : QRunLoopOperation
If you think you are responding to the cancellation request correctly then you really need to post your NSOperation subclass code to examine the problem any further.
Instead of using your own flag for when it is ok to add more operations, you could try the
- (void)setSuspended:(BOOL)suspend
method on NSOperationQueue? And before adding a new operation, check if the queue is suspended with isSuspended?
I'm googling like crazy and still confused about this.
I want to download an array of file urls to disk, and I want to update my view based on the bytes loaded of each file as they download. I already have something that will download a file, and report progress and completion via blocks.
How can I do this for each file in the array?
I'm ok doing them one at a time. I can calculate the total progress easily that way:
float progress = (numCompletedFiles + (currentDownloadedBytes / currentTotalBytes)) / totalFiles)
I mostly understand GCD and NSOperations, but how can you tell an operation or dispatch_async block to wait until a callback is called before being done? It seems possible by overriding NSOperation, but that seems like overkill. Is there another way? Is it possible with just GCD?
I'm not sure if I understand you correctly, but perhaps you need dispatch semaphores to achieve your goal. In one of my projects I use a dispatch semaphore to wait until another turn by another player is completed. This is partially the code I used.
for (int i = 0; i < _players.count; i++)
{
// a semaphore is used to prevent execution until the asynchronous task is completed ...
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
// player chooses a card - once card is chosen, animate choice by moving card to center of board ...
[self.currentPlayer playCardWithPlayedCards:_currentTrick.cards trumpSuit:_trumpSuit completionHandler:^ (WSCard *card) {
BOOL success = [self.currentTrick addCard:card];
DLog(#"did add card to trick? %#", success ? #"YES" : #"NO");
NSString *message = [NSString stringWithFormat:#"Card played by %#", _currentPlayer.name];
[_messageView setMessage:message];
[self turnCard:card];
[self moveCardToCenter:card];
// send a signal that indicates that this asynchronous task is completed ...
dispatch_semaphore_signal(sema);
DLog(#"<<< signal dispatched >>>");
}];
// execution is halted, until a signal is received from another thread ...
DLog(#"<<< wait for signal >>>");
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
dispatch_release(sema);
DLog(#"<<< signal received >>>");
dispatch groups are the GCD facility designed to track completion of a set of independent or separately async'd blocks/tasks.
Either use dispatch_group_async() to submit the blocks in question, or dispatch_group_enter() the group before triggering the asynchronous task and dispatch_group_leave() the group when the task has completed.
You can then either get notified asynchronously via dispatch_group_notify() when all blocks/tasks in the group have completed, or if you must, you can synchronously wait for completion with dispatch_group_wait().
I just wanted to note that I did get it working by subclassing NSOperation and making it a "concurrent" operation. (Concurrent in this context means an async operation that it should wait for before marking it as complete).
http://www.dribin.org/dave/blog/archives/2009/05/05/concurrent_operations/
Basically, you do the following in your subclass
override start to begin your operation
override isConcurrent to return YES
when you finish, make sure isExecuting and isFinished change to be correct, in a key-value compliant manner (basically, call willChangeValueForKey: and didChangeValueForKey: for isFinished and isExecuting
And in the class containing the queue
set the maxConcurrentOperationCount on the NSOperationQueue to 1
add an operation after all your concurrent ones which will be triggered once they are all done
I'm trying to synchronize the following code in iOS5:
an object has a method which makes an HTTP request from which it
gets some data, including an URL to an image
once the data arrives, the textual data is used to populate a
CoreData model
at the same time, a second thread is dispatched async to download
the image; this thread will signal via KVO to a viewController when
the image is already cached and available in the CoreData model.
since the image download will take a while, we immediately return
the CoreData object which has all attributes but for the image to
the caller.
Also, when the second thread is done downloading, the CoreData model
can be saved.
This is the (simplified) code:
- (void)insideSomeMethod
{
[SomeHTTPRequest withCompletionHandler:
^(id retrievedData)
{
if(!retrievedData)
{
handler(nil);
}
// Populate CoreData model with retrieved Data...
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
NSURL* userImageURL = [NSURL URLWithString:[retrievedData valueForKey:#"imageURL"]];
aCoreDataNSManagedObject.profileImage = [NSData dataWithContentsOfURL:userImageURL];
});
handler(aCoreDataNSManagedObject);
[self shouldCommitChangesToModel];
}];
}
- (void)shouldCommitChangesToModel
{
dispatch_barrier_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
NSError *error = nil;
if(![managedObjectContext save:&error])
{
// Handle error
}
});
}
But what's going on is that the barrier-based save-block is always executed before the the image-loading block. That is,
dispatch_barrier_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
NSError *error = nil;
if(![managedObjectContext save:&error])
{
// Handle error
}
});
Executes before:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
NSURL* userImageURL = [NSURL URLWithString:[retrievedData valueForKey:#"imageURL"]];
aCoreDataNSManagedObject.profileImage = [NSData dataWithContentsOfURL:userImageURL];
});
So obviously I'm not really dispatching the image-loading block before the barrier, or the barrier would wait until the image-loading block is done before executing (which was my intention).
What am I doing wrong? how do I make sure the image-loading block is enqueued before the barrier block?
At first glance the issue may be that you are dispatching the barrier block on a global concurrent queue. You can only use barrier blocks on your own custom concurrent queue. Per the GCD docs on dispatch_barrier_async, if you dispatch a block to a global queue, it will behave like a normal dispatch_async call.
Mike Ash has a good blog post on GCD barrier blocks: http://www.mikeash.com/pyblog/friday-qa-2011-10-14-whats-new-in-gcd.html
Good luck
T
You need to create your own queue and not dispatch to the global queues as per the ADC Docs
The queue you specify should be a concurrent queue that you create
yourself using the dispatch_queue_create function. If the queue you
pass to this function is a serial queue or one of the global
concurrent queues, this function behaves like the dispatch_async
function.
from https://developer.apple.com/library/ios/documentation/Performance/Reference/GCD_libdispatch_Ref/Reference/reference.html#//apple_ref/c/func/dispatch_barrier_async .
You can create tons of your own GCD queues just fine. gcd queues are very small and you can create tons of them without issue. You just need to free them when you're done with them.
For what you seem to be trying to solve, dispatch_barrier_async may not be the best solution.
Have a look at the Migrating Away From Threads section of the Concurrency Programming Guide. Just using dispatch_sync on a your own serial queue may solve your synchronization problem.
Alternatively, you can use NSOperation and NSOperationQueue. Unlike GCD, NSOperation allows you to easily manage dependancies (you can do it using GCD, but it can get ugly fast).
I'm a little late to the party, but maybe next time you could try using dispatch_groups to your advantage. http://www.raywenderlich.com/63338/grand-central-dispatch-in-depth-part-2