Pretty simple question that I haven't found anywhere in documentation or tutorials on GCD: What happens if I'm submitting work to queues faster than it's being processed and removed? I'm aware that GCD queues have no size limit, would work just pile up until the program runs out of memory? Is there any way to properly handle this situation?
What happens if I'm submitting work to queues faster than it's being processed and removed?
It depends.
If dispatching tasks to a single/shared serial queue, they will just be added to the queue and it will process them in a FIFO manner. No problem. Memory is your only constraint.
If dispatching tasks to a concurrent queue, though, you end up with “thread explosion”, and you will quickly exhaust the limited number of worker threads available for that quality-of-service (QoS). This can result in unpredictable behaviors should the OS need to avail itself of a queue of the same QoS. Thus, you must be very careful to avoid this thread explosion.
See a discussion on thread explosion WWDC 2015 Building Responsive and Efficient Apps with GCD and again in WWDC 2016 Concurrent Programming With GCD in Swift 3.
Is there any way to properly handle this situation?
It is hard to answer that in the abstract. Different situations call for different solutions.
In the case of thread explosion, the solution is to constrain the degree of concurrency using concurrentPerform (limiting the concurrency to the number of cores on your device). Or we use operation queues and their maxConcurrentOperationCount to limit the degree of concurrency to something reasonable. There are other patterns, too, but the idea is to constrain concurrency to something suitable for the device in question.
But if you're just dispatching a large number of tasks to a serial queue, there's not much you can do (other than looking for parallelism opportunities, to make efficient use of all of CPU’s cores). But that's OK, as that is the whole purpose of a queue, to let it perform tasks in the order they were submitted, even if the queue can't keep up. It wouldn’t be a “queue” if it didn’t follow this FIFO sort of pattern.
Now if dealing with real-time data that cannot be processed quickly enough, you have a different problem. In that case, you might want to decouple the capture of the input from the processing and decide how to you want to handle it. E.g. if you can't keep up with real-time processing of a video, for example, you have a choice. Either you start dropping frames or process the data asynchronously/later. You just have to decide what is right for your use case. We cannot answer this question in the abstract.
Related
I have 2 tasks that need to be performed:
One is synchronized with refresh rate via the Present call; does fancy graphics.
The other does a bunch of computations on a virtually infinite workload; does not need to be synchronous with the first task; really does not like being interrupted (encourages coarser workload granularity).
Is there a way to optimally use the GPU in this situation with DirectX?
Perhaps the solution would:
issue Dispatch (or Draw) calls in a way that allows them to run/finish asynchronously.
signal the current shader to stop.
use hardware or driver scheduling.
Right now my soultion is to try and predict how long it would take to run the shaders, which is unreliable, unless I add a bunch of downtime...
Trying to avoid the th**ad word as it means a different thing on GPUs
Create two separate D3D11 devices. Use one for the rendering, and another one (driven from another CPU thread with lower priority) for the computations.
Rework your low-priority computations making each Dispatch() to take a couple milliseconds of GPU time to complete. Don’t submit many compute calls at once: use 2 queries or a single fence to never dispatch more than 2 pending compute calls. Dispatch 2 calls initially, when the first is complete dispatch the 3-rd one, etc.
While 3D rendering on your main thread, lock an std::mutex, release once you rendered the scene before Present. On the background thread, lock that mutex when submitting more compute tasks, but keep it unlocked while waiting for a query or fence.
You still gonna have some interference between these two tasks, but it might be good enough for your use case.
Ideally, consider using timestamp queries to measure GPU time spent computing your background tasks. Then adjust size of the single task dynamically based on these numbers, this should allow to achieve ideal granularity of these tasks regardless on the GPU performance. Don’t forget to apply some rolling average over the last 5-10 completed tasks before using the number for these adjustments.
I know that the max number of threads spawned cannot exceed 66 through the response to this question. But is there a way to limit the thread count to a value which an user has defined?
From my experience and work with GCD under various circumstances, I believe this is not possible.
Said that, it is very important to understand, that by using GCD, you spawn queues, not threads. Whenever a call to create a queue is made from your code, GCD subsystem in its turn checks OS condition and seeks for available resources. New threads are then created under the hood based on these conditions – in the order and with the resources allocated, not controlled by you. This is clearly explained in official documentation:
When it comes to adding concurrency to an application, dispatch queues
provide several advantages over threads. The most direct advantage is
the simplicity of the work-queue programming model. With threads, you
have to write code both for the work you want to perform and for the
creation and management of the threads themselves. Dispatch queues let
you focus on the work you actually want to perform without having to
worry about the thread creation and management. Instead, the system
handles all of the thread creation and management for you. The
advantage is that the system is able to manage threads much more
efficiently than any single application ever could. The system can
scale the number of threads dynamically based on the available
resources and current system conditions. In addition, the system is
usually able to start running your task more quickly than you could if
you created the thread yourself.
Source: Dispatch Queues
There is no way you can control resources consumption with GCD, like by setting some kind of threshold. GCD is a high-level abstraction over low-level things, such as threads, and it manages it for you.
The only way you can possibly influence how many resources particular task within your application should take, is by setting its QoS (Quality of Service) class (formerly known simply as priority, extended to a more complex concept). To be brief, you can classify tasks within your application based on their importance, this way helping GCD and your application be more resource- and battery- efficient. Its employment is highly encouraged in complex applications with vast concurrency usage.
Even still, however, this kind of regulation from developer end has its limits and ultimately does not address the goal to control threads creation:
Apps and operations compete to use finite resources—CPU, memory,
network interfaces, and so on. In order to remain responsive and
efficient, the system needs to prioritize tasks and make intelligent
decisions about when to execute them.
Work that directly impacts the user, such as UI updates, is extremely
important and takes precedence over other work that may be occurring
in the background. This higher priority work often uses more energy,
as it may require substantial and immediate access to system
resources.
As a developer, you can help the system prioritize more effectively by
categorizing your app’s work, based on importance. Even if you’ve
implemented other efficiency measures, such as deferring work until an
optimal time, the system still needs to perform some level of
prioritization. Therefore, it is still important to categorize the
work your app performs.
Source: Prioritize Work with Quality of Service Classes
To conclude, if you are deliberate in your intent to control threads, don't use GCD. Use low-level programming techniques and manage them yourself. If you use GCD, then you agree to leave this kind of responsibility to GCD.
Can someone give a rundown of the benefits and drawbacks of these 3 systems in how they relate to thread safety?
From watching more recent WWDC videos, I get the feeling that Apple is pushing the usage of GCD to create performant reader-writer that are thread safe.
What's the idea/backing behind this? Is it the time to access a lock having to enter the kernel that leads to this GCD push, and shying away from #synchronized and NSLock?
Are #synchronized and NSLock being pushed out of what would be considered best practice, or is there still a place for them?
There are many many details that could be discussed at great length in regards to this. But, at the core:
These always require a lock to be taken somewhere or somehow:
#synchronized(...) { ... }
[lock lock];
Locks are very expensive for the reasons you mention; they necessarily consume kernel resources. (The #synchronized() case actually may avoid kernel locks these days, but it is a hash based exclusion mechanism and that, in itself, is expensive).
And these do not always require a lock (but sometimes maybe do):
dispatch_sync(...concurrent q...., ^{ ... });
dispatch_async(...queue of any kind...., ^{ ... });
There is a fast path through the dispatch functions that are effectively lockless (though they will use test-and-set atomic primitives that can cause performance issues under load).
The end result is that a synchronous dispatch to a concurrent queue can effectively be treated as "execute this on this thread right now". A synchronous dispatch to a serial queue can do the atomic test-and-set to test if the queue is processing, mark it as busy, and, if it wasn't busy, execute the block on the calling thread immediately.
Asynchronous dispatches can be similarly as fast, though asynchronous dispatch requires copying the block (which can be very cheap, but something to think about).
In general, GCD can do anything a lock can do, can do it at least -- if not more -- efficiently, and you can use the GCD APIs to go way beyond just simple locking (using a semaphore as a computation throttle, for example).
BTW: If your tasks are relatively coarse grained, have a look at NSOperationQueue and NSOperation.
In what cases would you prefer to use NSOperationQueue over GCD?
From my limited experience of these two, I take it that with NSOperationQueue you basically have control over how many concurrent operations there are.
With GCD you can't do this, since you are using a queue. Except you can somehow simulate this with a multi core processor, although still I think there's no way to control it.
NSOperationQueue is built on GCD as of iOS 4. Use the simplest API for the task at hand.Measure if it's a performance problem and then reevaluate if needed.dispatch_async is lower level, usually C-type stuff (but not limited to), and is good for one-shot and sequential type deals. NSOperationQueues are higher level, Objective-C stuff, and are good if you are adding a lot of operations at various points in your code, and/or need to manage concurrency, priorities and dependencies.
I assume by NSPriorityQueue you mean NSOperationQueue? The main reasons to use NSOperationQueue over GCD are if you need its additional features:
Older OS support
KVO on operation properties
Dependencies
Queue width limiting (although you can do this fairly easily in GCD with dispatch_semaphore_t)
Otherwise, unless you're working with an API that takes an NSOperationQueue, GCD is probably a better bet
Being at higher level,I found NSOperationQueue more elegant to manage tasks/operations instead of handling them at lower level using GCD.
I'm studying OS synchronization problems. I understand semaphores and their use in reader-writer and producer-consumer situations. I'm not getting the concept of monitors, though. Can someone help me understand them?
Super simple high level answer:
A semaphore counts how many are using a resource (or a pool of resources) and stops when a limit is reached. (For example, you could have a semaphore of 3 then the first 3 would be able to use the resource and then any additional would be locked out till a resource is released -- only 3 can have a lock at once.)
A monitor only allows a single lock -- by one process. When something is using it nothing else can.
A semaphore that counts to 1 is the same as a monitor.
Because a semaphore is designed to do more, a semaphore that only counted to 1 would not be efficient. (That is, when one implements a monitor it is more efficient than a semaphore which counts to 1 because a monitor has less requirements).