I am using NSOperationQueue and AFNetworking to implement a multi-download feature for my project. I also use Core Data to persist downloaded information but I am OK with that part. My problem is when it comes to actually create the download tasks. So here is how I did that:
// create a NSOperationQueue
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
queue.name = #"DownloadQueue";
queue.maxConcurrentOperationCount = 1;
// create an operation (AFDownloadRequestOperation is a subclass of AFHTTPRequestOperation)
AFDownloadRequestOperation *operation = // some initializations
// add the operation to the queue
[queue addOperation:operation];
// later some more operations are created and added to the queue
Then say I added 3 operations to the queue. Since I set the maxConcurrentOperationCount to 1, so only 1 operation in the queue is running, the other 2 are waiting:
operation 1: running
operation 2: waiting
operation 3: waiting
Then I use pause(which is a method of AFURLConnectionOperation) to pause the operation that is running, and I expect to see one of the 2 waiting operations start running, but it does not happen, the 2 waiting operations remain waiting:
operation 1: paused
operation 2: waiting
operation 3: waiting
I still have to resume operation 1 and wait till it finishes, then operation 2 or 3 will start running.
My question is how to make the other operations in the queue running after I pause a running operation like I explained above?
Also, I would appreciate it very much if you could explain a better way to implement multi-downloading (specifically related to manage the max number of download tasks, pause/resume a download task)
Solution:
Simply use cancel to pause an operation (You will need to handle the situation when the cancel flag is "YES" in your operation to actually let the operation quit). Then all the operations in the queue behaviors as I expected (cancel one operation, then another waiting one gets to run)
And when you need to resume downloading, simply create a new operation that fetches contents from web to the same local file path. (with the help of AFDownloadRequestOperation, resuming downloading works well)
Now my multi downloading system works well, comment if you need info about implementation.
Related
I want to download and process a image from Firebase to local directory on secondary thread and then update the UI. The problem is firebase returns in completion on main thread and not on the thread on which my NSOperation is being executed. I want to switch back to the thread on which my NSOperation is working. Is there a way I can achieve it ?
Download Meta Data from Firebase Realtime DB
Process and Store in Local Db
Update UI
Download image From Firebase to temp location
Once properly downloaded move to appropriate Directory
Update UI
Below is the sample code and have mentioned thread on which the completion is called.
[photosDetailReferenceDB observeEventType:FIRDataEventTypeValue withBlock:^(FIRDataSnapshot * _Nonnull snapshot) {
//Main Thread
[self.downloadQueue addOperationWithBlock:^{
//Bg Thread 2
[[[FirebaseHelper sharedManager].storageuserRef child:serverPath] writeToFile:[NSURL fileURLWithPath:tempPath] completion:^(NSURL * _Nullable URL, NSError * _Nullable error) {
//Main Thread
// Here I want to switch back to Thread 2.
// Since yet I have to move the image back to proper directory and update the image status in core data.
}];
}];
}];
It helps to stop thinking in terms of threads and start thinking in terms of queues.
Specifically, the thread of execution is, beyond the main thread, irrelevant. What is relevant, though, is the queue upon which the code is executed and whether or not that queue concurrent or serial execution.
So, in your case, the FirebaseHelper's callback to the main queue would immediately dispatch a block or operation to your background computation queue.
It may be as simple as dispatch_async() in the pure GCD queue case or, since you are using NSOperation, adding an operation to the appropriate NSOperationQueue.
If I add an operation to NSOperationQueue it would be different then
the one already executing. Firebase will be returning 100's of object
that means there would be 100's of NSOperations already and this would
lead to create another sub operation for each of them
If your operation queue is serial, then the operations will be executed one after the other, effectively as if it were the same thread (the actual thread of execution is irrelevant).
However, you don't want to have 100s or thousands of operations in flight, even in a serial queue.
You need to separate the work from the execution of the work. That is, you need to store a representation of the work that needs to be done in your data model and then pick off bits of work to be executed as your work queue or work queues (if parallelization makes sense) are emptied.
I need to make some API calls and I want to ensure that they come back in the order that they went out. Is this the proper flow to have that happen?
Create NSOperationQueue, set max concurrent operations to 1
Create URL String to API
Create NSOperation block, call method to call API, pass URL string
Add NSOperation to NSOperationQueue
This is where I get confused. Setting the max concurrent operations to 1 essentially makes NSOperationQueue into a synchronous queue, only 1 operation gets called at a time. However, each operation is going to make a NSURLSession call, which is async. How can I ensure that the next operation doesn't run until I have finished with the first? (By finish I want to store the returned JSON in a NSArray, adding each additional returned JSON to that array).
The proper way to ensure that NSOperations run in order is to add dependencies. Dependencies are powerful as they allow ordering of different operations on different queues. You can make an API call or data processing on a background queue; when complete, a dependent operation can update the UI on the main thread.
let operation1 = NSBlockOperation {
print("Run First - API Call")
}
let operation2 = NSBlockOperation {
print("Run Second - Update UI")
}
operation2.addDependency(operation1)
let backgroundQueue = NSOperationQueue()
backgroundQueue.addOperation(operation1)
NSOperationQueue.mainQueue().addOperation(operation2)
// operation1 will finish before operation2 is called, regardless of what queue they're in
See Apple Docs on addDependency in NSOperation here: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/NSOperation_class/index.html#//apple_ref/occ/cl/NSOperation
Also, be careful with assuming that maxConcurrentOperationCount = 1 as all that it does is ensure that only 1 operation will run at a time. This does NOT ensure the order of the queue. An operation with a higher priority will likely run first.
I've got an occasional crash that has to do with the improperly finished task on a concurrent thread while an app is transitioning to background.
So I have 3 threads:
A (main).
B (managed by GCD).
C (manually created to process intensive socket operations).
The scenario is the following:
In the applicationDidEnterBackground: handler (which is certainly executed on thread A) a long-running task is begun on thread B to complete all ongoing operations (save an application state, close a socket, etc.). In this task I need to wait until a socket properly finishes its work on thread C and only after that to continue with this long-running task.
Below is simplified code:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// Some synchronous task.
[stateManager saveState];
// Here I need to wait until the socket finishes its task.
...
// Continuing of the long-running task.
...
}
What is the acceptable way to accomplish this task. Is it OK if I do something like this?
while (socket.isConnected)
{
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];
}
// Continuing of the long-running task.
Or maybe something wrong in my current architecture and I need to use NSOperation to serialize asynchronous tasks somehow for example?
update: The problem has been solved by using dispatch_semaphore APIs as #Rob Napier suggested.
First, do not think of these as threads, and if you're creating a thread with NSThread or performSelectorInBackground: (or even worse, pthreads), don't do that. Use GCD. GCD creates queues. Queues order blocks, which eventually do run on threads, but the threads are an implementation detail and there is not a 1:1 mapping of queues to threads. See Migrating Away from Threads for more discussion on that.
To the question of how to wait for some other operation, the tool you probably want is a dispatch_semaphore. You create the semaphore and hand it to both operations. Then you call dispatch_semaphore_wait when you want to wait for something, and dispatch_sempahore_signal when you want to indicate that that something has happened. See Using Dispatch Semaphores to Regulate the Use of Finite Resources for more example code. The "finite resource" in this case is "the socket." You want to wait until the other part is done using it and returned it to the pool.
Semaphores will work even if you are using manual threading, but I can't emphasize enough that you should not be doing manual threading. All your concurrency should be managed through GCD. This is an important tool to overall concurrency management in iOS.
I would use NSOperation with dependencies.
So, you have tasks
A - main thread - aka 'entry point'
B - heavy boy to run in background
C - something else heavy to run after socket finished
Your heavy task from "B" is OperationB
Assume your socket framework capable of running syncronous in current thread? - then this is your OperationSend
Do the rest to do in background - OperationC
there you have a chain of operations, dependent on each other:
OperationB -> OperationSend -> OperationC
I'm writing an app where I've got a long running server-syncronization task running in the background, and I'd like to use NSOperation and NSOperationQueue for this. I'm leaning this way, since I need to ensure only one synchronisation operation is running at once.
My question arises since my architecture is built around NSNotifications; my synchronisation logic proceeds based on these notifications. From what I can see, NSOperation logic needs to be packed into the main method. So what I'm wondering is if there is any way to have an NSOperation finish when a certain notification is received. I suspect this is not the case, since I haven't stumbled upon any examples of this usage, but I figured I'd ask the gurus in here. Does an NSOperation just finish when the end of the main method is reached?
There is no reason a NSOperation cannot listen for a notification on the main thread, but either the finish logic must be thread safe, or the operation must keep track of its current thread.
I would recommend a different approach. Subclass NSOperation to support a method like -finishWithNotification: Have a queue manager that listens for the notification. It can iterate through its operations finishing any operations which respond to -finishWithNotification:.
- (void)handleFinishNotification:(NSNotification *)notification
{
for (NSOperation *operation in self.notificationQueue) {
if ([operation isKindOfClass:[MYOperation class]]) {
dispatch_async(self.notificationQueue.underlyingQueue), ^{
MYOperation *myOperation = (MYOperation *)operation;
[myOperation finishWithNotification:notification];
});
}
}
}
If I understood you correctly concurrent NSOperation is what you need.
Concurrent NSOperation are suitable for long running background/async tasks.
NSOperation Documentation
See: Subclassing Notes & Operation Dependencies Section
EDIT:(Adding more explanation)
Basically concurrent operations do not finish when main method finishes. Actually what concurrent operation mean is that the control will return to calling code before the actual operation finishes. The typical tasks that are done in start method of concurrent operation are: Mark operation as executing, start the async work(e.g. NSURLConnection async call) or spawn a new thread which will perform bulk of the task. And RETURN.
When the async task finishes mark the operation as finished.
I'm googling like crazy and still confused about this.
I want to download an array of file urls to disk, and I want to update my view based on the bytes loaded of each file as they download. I already have something that will download a file, and report progress and completion via blocks.
How can I do this for each file in the array?
I'm ok doing them one at a time. I can calculate the total progress easily that way:
float progress = (numCompletedFiles + (currentDownloadedBytes / currentTotalBytes)) / totalFiles)
I mostly understand GCD and NSOperations, but how can you tell an operation or dispatch_async block to wait until a callback is called before being done? It seems possible by overriding NSOperation, but that seems like overkill. Is there another way? Is it possible with just GCD?
I'm not sure if I understand you correctly, but perhaps you need dispatch semaphores to achieve your goal. In one of my projects I use a dispatch semaphore to wait until another turn by another player is completed. This is partially the code I used.
for (int i = 0; i < _players.count; i++)
{
// a semaphore is used to prevent execution until the asynchronous task is completed ...
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
// player chooses a card - once card is chosen, animate choice by moving card to center of board ...
[self.currentPlayer playCardWithPlayedCards:_currentTrick.cards trumpSuit:_trumpSuit completionHandler:^ (WSCard *card) {
BOOL success = [self.currentTrick addCard:card];
DLog(#"did add card to trick? %#", success ? #"YES" : #"NO");
NSString *message = [NSString stringWithFormat:#"Card played by %#", _currentPlayer.name];
[_messageView setMessage:message];
[self turnCard:card];
[self moveCardToCenter:card];
// send a signal that indicates that this asynchronous task is completed ...
dispatch_semaphore_signal(sema);
DLog(#"<<< signal dispatched >>>");
}];
// execution is halted, until a signal is received from another thread ...
DLog(#"<<< wait for signal >>>");
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
dispatch_release(sema);
DLog(#"<<< signal received >>>");
dispatch groups are the GCD facility designed to track completion of a set of independent or separately async'd blocks/tasks.
Either use dispatch_group_async() to submit the blocks in question, or dispatch_group_enter() the group before triggering the asynchronous task and dispatch_group_leave() the group when the task has completed.
You can then either get notified asynchronously via dispatch_group_notify() when all blocks/tasks in the group have completed, or if you must, you can synchronously wait for completion with dispatch_group_wait().
I just wanted to note that I did get it working by subclassing NSOperation and making it a "concurrent" operation. (Concurrent in this context means an async operation that it should wait for before marking it as complete).
http://www.dribin.org/dave/blog/archives/2009/05/05/concurrent_operations/
Basically, you do the following in your subclass
override start to begin your operation
override isConcurrent to return YES
when you finish, make sure isExecuting and isFinished change to be correct, in a key-value compliant manner (basically, call willChangeValueForKey: and didChangeValueForKey: for isFinished and isExecuting
And in the class containing the queue
set the maxConcurrentOperationCount on the NSOperationQueue to 1
add an operation after all your concurrent ones which will be triggered once they are all done