I'm developing a sort of fast image scan application.
In the -captureOutput:didOutputSampleBuffer:fromConnection: method I pick up the CVPixelBuffer and add them into an NSOperation subclass. The NSOperation takes the buffer and transform it into an image saved on the filesystem.
The -isConcurrent method of NSOperation returns YES. After the operation is created, is added to an NSOperationQueue.
Everything runs fine except for one issue that is affecting the scan frame rate.
Using time profiler I've found that some of the NSOperation are run on the same thread of the AVCaputureVideoOutput delegate that I've created:
dispatch_queue_t queue = dispatch_queue_create("it.CloudInTouchLabs.avsession", DISPATCH_QUEUE_SERIAL);
[dataOutput setSampleBufferDelegate:(id)self queue:queue];
When an operation is running on the same thread of the AV session queue, it affects the frame rate, it happens only with some them, probably GCD is deciding under the hood dispatching on active threads. The only method I've found to solve that issue, is to create a separate thread and pass it to the the single operations forcing them to run on it.
Is there another way, to force operations run on a different thread?
[EDIT]
Following the #impcc suggestions I made some test.
Results are pretty interesting even if sort of inconsistent.
Test platform is an iPhone 5 connected in debugging mode through an MBP with 16gb of RAM quad core i7. The session has a 60fps output tested with the algorithm in RosyWriter apple sample code.
AVSession queue sync targeted to high priority queue
26 fps where sometimes the thread of the queue is shared with the one
of the operation. The time taken inside the delegate methods has an
average of 0.02s
14 fps with a thread created just for the operations, if the main
method is not called on this thread, the start will be forced to perform
on that specific thread. This thread is create one time and kept
alive with a dummy port.. The time taken inside the delegate is
0.008.
AVSession queue concurrent targeted to high priority queue
13.5 fps where sometimes the thread of the queue is shared with the one of the operation. The time taken inside the delegate methods has
an average of 0.02s, substacially equal to the counterpart with sync
queue.
14 fps with a thread created just for the operations, if the main
method is not called on this thread, the start will be forced to perform
on that specific thread. This thread is create one time and kept
alive with a dummy port. The time taken inside the delegate is
0.008.
Conclusion
Concurrent or serial queue doesn't seems to make a big difference, however concurrent for me is not ok cause of the timestamps that I need to take to preserve the sequence of single picture. The fact that is surprising me is the frame drop using an ad-hoc thread, even if the time taken inside the delegate method is considerably less, the frame rate drops about 10fps. Just trusting in GCD the frame rate is better but the delegate methods takes more than 2 times to finish, this probably due to the fact that sometimes the AV queue is used by also the nsoperations. I can't really get why the time taken inside the delegate doesn't seems to be correlated to the fps. Shouldn't be the faster the better?
By further investigations it really seems that what is stealing time si in the process of adding and probably executing operation in queue.
I think you might be misunderstanding the meaning of isConcurrent. This is totally understandable, since it's incredibly poorly named. When you return YES from -isConcurrent what it really means is "I will handle any concurrency needs associated with this operation, and will otherwise behave asynchronously." In this situation, NSOperationQueue is free to call -start on your operation synchronously on the thread from which you add the operation. The NSOperationQueue is expecting that, because you've declared that you will manage your own concurrency, that your -start method will simply kick off an asynchronous process and return right away. I suspect this is the source of your problem.
If you implemented your operation by overriding -main then you almost certainly want to be returning NO from isConcurrent. To make matters more complicated, the behaviors associated with isConcurrent have changed over the years (but all that is covered in the official docs.) A fantastic explanation of how to properly implement a returns-YES-from-isConcurrent NSOperation can be found here.
The way I'm reading your question here, it doesn't sound like your NSOperation subclass actually needs to "manage its own concurrency", but rather that you simply want it to execute asynchronously, "on a background thread", potentially "concurrently" with other operations.
In terms of assuring the best performance for your AVCaptureVideoDataOutputSampleBufferDelegate callbacks, I would suggest making the queue you pass in to -setSampleBufferDelegate:queue: be (itself) concurrent and that it target the high-priority global concurrent queue, like this:
dispatch_queue_t queue = dispatch_queue_create("it.CloudInTouchLabs.avsession", DISPATCH_QUEUE_CONCURRENT);
dispatch_set_target_queue(queue, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0));
[dataOutput setSampleBufferDelegate:(id)self queue:queue];
Then you should make the delegate callback methods as lightweight as possible -- nothing more than packaging up the information you need to make the NSOperation and add it to the NSOperationQueue.
This should ensure that the callbacks always take precedence over the NSOperations. (It's my understanding that NSOperationQueue targets either the main queue (for the NSOperationQueue that's associated with the main thread and run loop) or the default priority background queue.) This should allow your callbacks to fully keep up with the frame rate.
Another important thing to take away here (that another commenter was getting at) is that if you're doing all your concurrency using GCD, then there is only one special thread -- the main thread. Other than that, threads are just a generic resource that GCD can (and will) use interchangeably with one another. The fact that a thread with a thread ID of X was used at one point to service your delegate callback, and at another point was used to do your processing is not, in and of itself, indicative of a problem.
Hope this helps!
Related
I'm an Android engineer trying to port some iOS code that uses 5 SERIAL dispatch queues. I want to make sure I'm thinking about things the right way.
dispatch_sync to a SERIAL queue is basically using the queue as a synchronized queue- only one thread may access it and the block that gets executed can be thought of as a critical region. It happens immediately on the current thread- its the equivalent of
get_semaphore()
queue.pop()
do_block()
release_semaphore()
dispatch_async to a serial queue- performs the block on another thread and lets the current thread return immediately. However since its a serial queue it promises only one of these asynchronous thread is going to execute at a time (the next call to dispatch_async will wait until all other threads are finished). That block can also be thought of as a critical region, but it will occur on another thread. So the same code as above, but its passed to a worker thread first.
Am I off in any of that, or did I figure it out correctly?
This feels like an overly complicated way of thinking of it and there are lots of little details of that description that aren't quite right. Specifically, "it happens immediately on the current thread" is not correct.
First, let's step back: The distinction between dispatch_async and dispatch_sync is merely whether the current thread waits for it or not. But when you dispatch something to a serial queue, you should always imagine that it's running on a separate worker thread of GCD's own choosing. Yes, as an optimization, sometimes dispatch_sync will use the current thread, but you are in no way guaranteed of this fact.
Second, when you discuss dispatch_sync, you say something about it running "immediately". But it's by no means assured to be immediate. If a thread does dispatch_sync to some serial queue, then that thread will block until (a) any block currently running on on that serial queue finish; (b) any other queued blocks for that serial queue run and complete; and (c) obviously, the block that thread A itself dispatched runs and completes.
Now, when you use a serial queue for synchronization for, some thread-safe access to some object in memory, often that synchronization process is very quick, so the waiting thread will generally be blocked for a negligible amount of time as its dispatched block (and any prior dispatched blocks) to finish. But in general, it's misleading to say that it will run immediately. (If it always could run immediately, then you wouldn't need a queue to synchronize access).
Now your question talks about a "critical region", to which I assume you're talking about some bit of code that, in order to ensure thread-safety or for some other reason like that, must be synchronized. So, when running this code to be synchronized, the only question re dispatch_sync vs dispatch_async is whether the current thread must wait. A common pattern, for example, is to say that one may dispatch_async writes to some model (because there's no need to wait for the model to update before proceeding), but dispatch_sync reads from some model (because you obviously don't want to proceed until the read value is returned).
A further optimization of that sync/async pattern is the reader-writer pattern, where concurrent reads are permissible but concurrent writes are not. Thus, you'll use a concurrent queue, dispatch_barrier_async the writes (achieving serial-like behavior for the writes), but dispatch_sync the reads (enjoying concurrent performance with respect to other read operations).
To pick nits, dispatch_sync doesn't necessarily run the code on the current thread, but if it doesn't, it still blocks the current thread until the task completes. The distinction is only potentially important if you're relying on thread IDs or thread-local storage.
But otherwise, yes, unless I missed something subtle.
Recently, i am learning concurrency in swift. According to apple's document in NSOperation class reference :
When you add an operation to an operation queue, the queue ignores the value of the asynchronous property and always calls the start method from a separate thread. Therefore, if you always run operations by adding them to an operation queue, there is no reason to make them asynchronous.
does it mean synchronous in a separate thread is the same as asynchronous? and when i do the test with the following code, the operation indeed doesn't block the current main thread.
let operationQueue = NSOperationQueue()
let operation = NSBlockOperation(){
//do some task here
}
operationQueue.addOperation(operation)
so if it is true, then why should we create concurrency subclass of NSOperation?
Oh, NSOperation. Such a bizarre history you have.
NSOperation is relatively old (in iOS terms; fairly modern in ObjC terms). It was added in OS X 10.5. Before OS X 10.6/iOS 4, there were no NSBlockOperation objects. There were no blocks at all. So the only way to make an operation was to subclass or use NSInvocationOperation. Both approaches are cumbersome, but were still easier and more powerful than the older approach of using NSThread directly.
(This was right at the time when multi-core became a thing. 10.5 was famous for adding Core Animation which was I believe the first major preemptive multitasking framework in Cocoa. Before 10.5, most things were done with the runloop and cooperative multitasking, which is actually very efficient and effective for single-core systems. But it doesn't scale well to multi-core systems. Tools like NSOperation were provided to help us write better multi-core code, but GCD was so much more powerful that it completely dominated how multitasking code is written in Cocoa.)
When you subclass NSOperation, you needed to tell the system whether your operation is asynchronous. This isn't a request to run you asynchronously. This is a promise that your start method will not block. It's up to your start method to make sure the operation really is asynchronous.
This is only necessary in the case that your NSOperation is being started manually, and even then it was often not needed. If you put it onto an NSOperationQueue (and you really should always do that), this property is irrelevant. I remember it creating a lot of confusion at the time.
It's become even more irrelevant since the introduction of blocks. It is almost always much easier to use an NSBlockOperation (or dispatch_async) than to subclass NSOperation, which was always a bit tricky to get quite right.
Just in case you haven't already read it, if you want to study Cocoa concurrency, you definitely want to start with the Concurrency Programming Guide.
Asynchronous is always defined relative to the thread that makes a request. So a request is asynchronous relative to thread A if thread A makes a request that runs in thread B such that thread A is able to do other work while thread B is running the request.
If thread B in turn farms out the request to thread C such that thread B is able to do other work while thread C is running the request then that second request is an asynchronous relative to thread B.
It doesn't make sense to just keep farming out the same element of work asynchronously over and over again, of course. But assume the work delegated by thread A to thread B described above can be split up into multiple smaller elements of work. It would be reasonable for thread B to invoke those smaller elements of work asynchronously on threads C, D, etc. This might happen if B provides a service to A such that A doesn't want/need to know the details of how the work gets done; it just wants the work done asynchronously. B knows the details and can decide if/how to accomplish the work via smaller parallel units.
I want to understand something about GCD and Threads.
I have a for loop in my view controller which asks my model to do some async network request.
So if the loop runs 5 times, the model sends out 5 network requests.
Is it correct to state that 5 threads have been created by my model considering the fact that I'm using NSURLConnection's sendAsyncRequest and the completion handlers will be called on an additional 5 threads ?
Now, If I ask my view controller to execute this for loop on a different thread and in every iteration of the loop, the call to the model should be dependent on the previous iteration, would I be creating an "Inception" of threads here ?
Basically, I want the subsequent async requests to my server only if the previous thread has completed entirely (By entirely I mean all of its sub threads should have finished executing too.)
I can't even frame the question properly because I'm massively confused myself.
But if anybody could help with anything, that would be helpful.
It is not correct to state that five threads have been created in the general case.
There is no one-to-one mapping between threads and blocks. GCD is an implementation of thread pooling.
A certain number of threads are created according to the optimal setup for that device — the cost of creating and maintaing threads under that release of the OS, the number of processor cores available, the number of threads it already has but which are presently blocked and any other factors Apple cares to factor in may all be relevant.
GCD will then spread your blocks over those threads. Or it may create new threads. But it won't necessarily.
Beyond that queues are just ways of establishing the sequencing between blocks. A serial dispatch queue does not necessarily own its own thread. All concurrent dispatch queues do not necessarily own their own threads. But there's no reason to believe that any set of queues shares any threads.
The exact means of picking threads for blocks has changed between versions of the OS. E.g. iOS 4 was highly profligate in thread creation, in a way that iOS 5+ definitely haven't been.
GCD will just try to do whatever is best in the circumstances. Don't waste your time trying to second guess it.
"Basically, I want the subsequent async requests to my server only if the previous thread has completed entirely (By entirely I mean all of its sub threads should have finished executing too.)"
Only focusing on the above statement to avoid confusion. Simple solution would be create a queue. feed the queue with 5 loops. Each loop will be making network request synchronously(you can use sendSynchronousRequest: method available in NSURLConnection), performing the operations after request completion and then start the next loop. queue following FIFO will execute the your requests subsequently.
GCD : Think of this as a simple queue that can accept tasks. Tasks are blocks of your code. You can put in as many tasks as you want in a queue (permitting system limits). Queues come in different flavors. Concurrent vs Serial. Main vs Global. High Priority vs Low Priority. A queue is not a thread.
Thread : It is a single line of execution of code in sequence. You can have multiple threads working on your code at the same time. A thread is not a queue.
Once you separate the 2 entities things start become clear.
GCD basically uses the threads in the process to work on tasks. In a serial queue everything is processed in sequence. So you don't need to have synchronization mechanisms in your code, the very nature of serial queue ensures synchronization. If this is a concurrent queue (i.e. 2 or more tasks being processed at the same time, then you need to ensure critical sections of your code are protected with synchronization).
Here is how you queue work to be done.
dispatch_async(_yourDispatchQueue, ^() {
NSLog (#"work queued");
});
The above NSLog will now get executed in a background thread in a near future time, but in a background thread.
If you notice when we put a request in we use dispatch_async. The other variation is dispatch_sync. The different between the 2 is after you put the request in the queue, the async variation will move on. The sync variation will not !!
If you are going to use a GCD for NSURLConnection you need to be careful on which thread you start the connection. Here is a SO link for more info. GCD with NSURLConnection
I'm loving NSOperationQueue but I'm having some issues understanding some portions of it.
In the second issue of objc.io they go over NSOperationQueue and mention that it has two kinds of queues, the main queue which runs on the main thread, and the background queues. They mention you can access the main queue with [NSOperation mainQueue] and then add manipulate it.
You normally would not want to do this, correct? If it's running on the main thread, will it not block the main thread for other tasks? Wouldn't it not run concurrently with other tasks?
It also mentions you can add to the background queues (which I understand would be better?) by creating instances of NSOperation (subclasses potentially).
Do I save a reference to the NSOperationQueue that I create for operations in order to have for creating more operations? I assume there's no singleton for background queues like there is for mainQueue, so how do I manage adding tasks to background queues?
It also mentions you can control the amount of operations running concurrently with the maxConcurrentOperationCount property.
I know normally you set it to NSOperationQueueDefaultMaxConcurrentOperationCount, but if I set it to a specific number manually, does it correspond to the maximum number of threads that can be run at once? For example if the processor on the iPhone can run 4 threads at once and I set that property to 8, what happens?
You ask:
You normally would not want [to add operations to the mainQueue], correct? If it's running on the main thread, will it not block the main thread for other tasks? Wouldn't it not run concurrently with other tasks?
Yes, you would never want to add anything slow to the main queue. But that doesn't mean that you don't use the main queue. For some operations (e.g. UI updates) it's critical.
The typical pattern is to create a operation queue for tasks that you want to run in the background, but if you subsequently need to do something that needs to run on the main queue (e.g. UI updates, updating the model, etc.), you would go ahead and do that, for example:
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[queue addOperationWithBlock:^{
// do some time consuming stuff in the background
// when done, you might update the UI in the main queue
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// update the UI here
}];
];
You ask:
Do I save a reference to the NSOperationQueue that I create for operations in order to have for creating more operations? I assume there's no singleton for background queues like there is for mainQueue, so how do I manage adding tasks to background queues?
Yes, if you want to add more operations to that same queue later, yes, you want to maintain a reference to that queue. You can do this by adding it to the app delegate, some central view controller, or a singleton.
But yes, there's no built-in singleton for background queues (because you can, conceivably have different queues for different operations, e.g. one for network operations, one for image processing, etc.). But you can write your own singleton for each queue of each type, if you want.
You also ask:
I know normally you set it to NSOperationQueueDefaultMaxConcurrentOperationCount, but if I set it to a specific number manually, does it correspond to the maximum number of threads that can be run at once? For example if the processor on the iPhone can run 4 threads at once and I set that property to 8, what happens?
One should set maxConcurrentOperationCount to be whatever you think is appropriate for the type of queue. For network operation queue, you generally wouldn't exceed 4, but for other types of queues, you might easily have more. I believe that there is a maximum of 64 worker threads (which concurrent queues avail themselves as they need threads).
If you attempt to use more than that, the app won't start your operation until a worker thread becomes available. Apple advises, though, that one refrain from using up all of the worker threads. So use a reasonable number appropriate for your queue's function. Frankly, one of the advantages of operation queues over dispatch queues is that you can constrain the maximum number of worker threads that will be used at any given time to better manage the device's limited resources.
References
WWDC 2012 video Asynchronous Design Patterns with Blocks, GCD, and XPC is an excellent primer on some GCD patterns and touches upon the "too many threads" question.
The Building Concurrent User Interfaces on iOS video walks through some practical implications of building concurrent iOS apps.
The About Threaded Programming section of the Threading Programming Guide touches upon the relationship between cores and threads.
The Concurrency and Application Design section of the Concurrency Programming Guide is an articulate discussion of the relationships between threads and operation/dispatch queues.
Generally you don't want to use the main queue. Any operation there will run on the main thread.
When you create an operation queue, create it for a purpose. Like it will be used for all server requests. So you can control how many concurrent requests are in progress. So don't add algorithmic processing operations to this queue because they have a different purpose. Keep a reference to the queue so you can add operations in future (and pause / cancel operations).
There is no 'normal' setting for maxConcurrentOperationCount - it should be set based on the purpose.
If you set it to 8 then the queue will run up to 8 at the same time. This may not be the most efficient option. Always keep the purpose of the queue in mind.
First of all, you will have to keep in mind that you always separate the main thread with the background thread. Only the operations which involve updating the UI must be performed in the main thread and rest of the operations must be performed in the Background thread. e.g if you are dealing with the multiple downloads, then you have to take care of all the network based operations in the background queue, and you will have to perform the UI update in the main queue.
//e.g for updating UI in main thread.
[self performSelectorOnMainThread:#selector(updateUI) withObject:nil waitUntilDone:YES];
Also when you use set maxConcurrentOperationCount property as NSOperationQueueDefaultMaxConcurrentOperationCount, it means the operationQueue takes the number of concurrent operations depending on the system environment.
Useful Links:
http://mobile.tutsplus.com/tutorials/iphone/nsoperationqueue/
http://www.raywenderlich.com/19788/how-to-use-nsoperations-and-nsoperationqueues
http://www.cimgf.com/2008/02/16/cocoa-tutorial-nsoperation-and-nsoperationqueue/
When selecting which queue to run dispatch_async on, dispatch_get_global_queue is mentioned a lot. Is this one special background queue that delegates tasks to a certain thread? Is it almost a singleton?
So if I use that queue always for my dispatch_async calls, will that queue get full and have to wait for things to finish before another one can start, or can it assign other tasks to different threads?
I guess I'm a little confused because when I'm choosing the queue for an NSOperation, I can choose the queue for the main thread with [NSOperationQueue mainQueue], which seems synonymous to dispatch_get_main_queue but I was under the impression background queues for NSOperation had to be individually made instances of NSOperationQueue, yet GCD has a singleton for a background queue? (dispatch_get_global_queue)
Furthermore - silly question but wanted to make sure - if I put a task in a queue, the queue is assigned to one thread, right? If the task is big enough it won't split it up over multiple threads, will it?
When selecting which queue to run dispatch_async on,
dispatch_get_global_queue is mentioned a lot. Is this one special
background queue that delegates tasks to a certain thread?
A certain thread? No. dispatch_get_global_queue retrieves for you a global queue of the requested relative priority. All queues returned by dispatch_get_global_queue are concurrent, and may, at the system's discretion, dispatch work to many different threads. The mechanics of this are an implementation detail that is opaque to you as a consumer of the API.
In practice, and at the risk of oversimplifying it, there is one global queue for each priority level, and at the time of this writing, based on my experience, each of those will at any given time be dispatching work to between 0 and 64 threads.
Is it almost a singleton?
Strictly no, but you can think of them as singletons where there is one singleton per priority level.
So if I use that queue always for my dispatch_async calls, will that
queue get full and have to wait for things to finish before another
one can start, or can it assign other tasks to different threads?
It can get full. Practically speaking, if you are saturating one of the global concurrent queues (i.e. more than 64 background tasks of the same priority in flight at the same time), you probably have a bad design. (See this answer for more details on queue width limits)
I guess I'm a little confused because when I'm choosing the queue for
an NSOperation, I can choose the queue for the main thread with
[NSOperationQueue mainQueue], which seems synonymous to
dispatch_get_main_queue
They are not strictly synonymous. Although NSOperationQueue uses GCD under the hood, there are some important differences. For instance, in a single pass of the main run loop, only one operation enqueued to +[NSOperationQueue mainQueue] will be executed, whereas more than one block submitted to dispatch_get_main_queue might be executed on a single run loop pass. This probably doesn't matter to you, but they are not, strictly speaking, the same thing.
but I was under the impression background
queues for NSOperation had to be individually made instances of
NSOperationQueue, yet GCD has a singleton for a background queue?
(dispatch_get_global_queue)
In short, yes. It sounds like you're conflating GCD and NSOperationQueue. NSOperationQueue is not just a "trivial wrapper" around GCD, it's its own thing. The fact that it's implemented on top of GCD should not really matter to you. NSOperationQueue is a task queue, with an explicitly settable width, that you can create instances of "at will." You can make as many of them as you like. At some point, all instances of NSOperationQueue are, when executing NSOperations, pulling resources from the same pool of system resources as the rest of your process, including GCD, so yes, there are some interactions there, but they are opaque to you.
Furthermore - silly question but wanted to make sure - if I put a task
in a queue, the queue is assigned to one thread, right? If the task is
big enough it won't split it up over multiple threads, will it?
A single task can only ever be executed on a single thread. There's not some magical way that the system would have to "split" a monolithic task into subtasks. That's your job. With regard to your specific wording, the queue isn't "assigned to one thread", the task is. The next task from the queue to be executed might be executed on a completely different thread.