How to parallelize many (100+) tasks without hitting global GCD limit? - ios

The problem:
When lazy-loading a list of 100+ icons in the background, I hit the GCD thread limit (64 threads), which causes my app to freeze up with a semaphore_wait_trap on the main thread. I want to restructure my threading code to prevent this from happening, while still loading the icons asynchronous to prevent UI blocking.
Context:
My app loads a screen with SVG icons on it. The amount differs on average from 10-200. The icons get drawn by using a local SVG image or a remote SVG image (if it has a custom icon), then they get post-processed to get the final image result.
Because this takes some time, and they aren't vital for the user, I want to load and post-process them in the background, so they would pop in over time. For every icon I use the following:
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
//code to be executed in the background
SVGKImage *iconImage = [Settings getIconImage:location];
dispatch_async(dispatch_get_main_queue(), ^{
//code to be executed on the main thread when background task is finished
if (iconImage) {
[iconImgView setImage:iconImage.UIImage];
}
});
});
The getIconImage method handles the initial loading of the base SVG, which reads it synchronized with [NSInputStream inputStreamWithFileAtPath:path] if local, and [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&errorWithNSData] if it should load remotely. This all happens synchronous.
Then there is some post-processing of recoloring the SVG, before it gets returned and put in the UIImageView on the main thread.
Question:
Is there a way to structure my code to allow for parallelized background loading but prevent deadlock because of too many threads?
Solution EDIT:
_iconOperationQueue = [[NSOperationQueue alloc]init];
_iconOperationQueue.maxConcurrentOperationCount = 8;
// Code will be executed on the background
[_iconOperationQueue addOperationWithBlock:^{
// I/O code
SVGKImage *baseIcon = [Settings getIconBaseSVG:location];
// CPU-only code
SVGKImage *iconImage = [Settings getIconImage:location withBaseSVG:baseIcon];
UIImage *svgImage = iconImage.UIImage; // Converting SVGKImage to UIImage is expensive, so don't do this on the main thread
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
// Code to be executed on the main thread when background task is finished
if (svgImage) {
[iconImgView setImage:svgImage];
}
}];
}];

Instead of directly using GCD with a concurrent queue, use an NSOperationQueue. Set its maxConcurrentOperationCount to something reasonable, like 4 or 8.
If you can, you should also separate I/O from pure computation. Use the width-restricted operation queue for the I/O. The pure computation you can use an unrestricted operation queue or pure GCD for.
The reason is that I/O blocks. GCD detects that the system is idle and spins up another worker thread and starts another task from the queue. That blocks in I/O, too, so it does that some more until it hits its limit. Then, the I/O starts completing and the tasks unblock. Now you have oversubscribed the system resources (i.e. CPU) because there are more tasks in flight than cores and suddenly they are actually using CPU instead of being blocked by I/O.
Pure computation tasks don't provoke this problem because GCD sees that the system is actually busy and doesn't dequeue more tasks until earlier ones have completed.

You can stay with GCD by using a semaphore something like this running the whole operation in the background otherwise waiting for the semaphore will stall the UI:
dispatch_semaphore_t throttleSemaphore = dispatch_semaphore_create(8);
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
for /* Loop through your images */ {
dispatch_semaphore_wait(throttleSemaphore, DISPATCH_TIME_FOREVER);
dispatch_async(concurrentQueue, ^{
//code to be executed in the background
SVGKImage *iconImage = [Settings getIconImage:location];
dispatch_async(dispatch_get_main_queue(), ^{
//code to be executed on the main thread when background task is finished
if (iconImage) {
[iconImgView setImage:iconImage.UIImage];
}
dispatch_semaphore_signal(throttleSemaphore);
});
});
}

Related

Running multiple background threads iOS

Is it possible to run multiple background threads to improve performance on iOS . Currently I am using the following code for sending lets say 50 network requests on background thread like this:
dispatch_async(dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void){
// send 50 network requests
});
EDIT:
After updating my code to something like this no performance gain was achieved :( Taken from here
dispatch_queue_t fetchQ = dispatch_queue_create("Multiple Async Downloader", NULL);
dispatch_group_t fetchGroup = dispatch_group_create();
// This will allow up to 8 parallel downloads.
dispatch_semaphore_t downloadSema = dispatch_semaphore_create(8);
// We start ALL our downloads in parallel throttled by the above semaphore.
for (NSURL *url in urlsArray) {
dispatch_group_async(fetchGroup, fetchQ, ^(void) {
dispatch_semaphore_wait(downloadSema, DISPATCH_TIME_FOREVER);
NSMutableURLRequest *headRequest = [NSMutableURLRequest requestWithURL:url cachePolicy: NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0];
[headRequest setHTTPMethod: #"GET"];
[headRequest addValue: cookieString forHTTPHeaderField: #"Cookie"];
NSOperationQueue *queue = [[[NSOperationQueue alloc] init] autorelease];
[NSURLConnection sendAsynchronousRequest:headRequest
queue:queue // created at class init
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error){
// do something with data or handle error
NSLog(#"request completed");
}];
dispatch_semaphore_signal(downloadSema);
});
}
// Now we wait until ALL our dispatch_group_async are finished.
dispatch_group_wait(fetchGroup, DISPATCH_TIME_FOREVER);
// Update your UI
dispatch_sync(dispatch_get_main_queue(), ^{
//[self updateUIFunction];
});
// Release resources
dispatch_release(fetchGroup);
dispatch_release(downloadSema);
dispatch_release(fetchQ);
Be careful not to confuse threads with queues
A single concurrent queue can operate across multiple threads, and GCD never guarantees which thread your tasks will run on.
The code you currently have will submit 50 network tasks to be run on a background concurrent queue, this much is true.
However, all 50 of these tasks will be executed on the same thread.
GCD basically acts like a giant thread pool, so your block (containing your 50 tasks) will be submitted to the next available thread in the pool. Therefore, if the tasks are synchronous, they will be executed serially. This means that each task will have to wait for the previous one to finish before preceding. If they are asynchronous tasks, then they will all be dispatched immediately (which begs the question of why you need to use GCD in the first place).
If you want multiple synchronous tasks to run at the same time, then you need a separate dispatch_async for each of your tasks. This way you have a block per task, and therefore they will be dispatched to multiple threads from the thread pool and therefore can run concurrently.
Although you should be careful that you don't submit too many network tasks to operate at the same time (you don't say specifically what they're doing) as it could potentially overload a server, as gnasher says.
You can easily limit the number of concurrent tasks (whether they're synchronous or asynchronous) operating at the same time using a GCD semaphore. For example, this code will limit the number of concurrent operations to 6:
long numberOfConcurrentTasks = 6;
dispatch_semaphore_t semaphore = dispatch_semaphore_create(numberOfConcurrentTasks);
for (int i = 0; i < 50; i++) {
dispatch_async(concurrentQueue, ^{
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
[self doNetworkTaskWithCompletion:^{
dispatch_semaphore_signal(semaphore);
NSLog(#"network task %i done", i);
}];
});
}
Edit
The problem with your code is the line:
dispatch_queue_t fetchQ = dispatch_queue_create("Multiple Async Downloader", NULL);
When NULL is passed to the attr parameter, GCD creates a serial queue (it's also a lot more readable if you actually specify the queue type here). You want a concurrent queue. Therefore you want:
dispatch_queue_t fetchQ = dispatch_queue_create("Multiple Async Downloader", DISPATCH_QUEUE_CONCURRENT);
You need to be signalling your semaphore from within the completion handler of the request instead of at the end of the request. As it's asynchronous, the semaphore will get signalled as soon as the request is sent off, therefore queueing another network task. You want to wait for the network task to return before signalling.
[NSURLConnection sendAsynchronousRequest:headRequest
queue:queue // created at class init
completionHandler:^(NSURLResponse *response, NSData *data, NSError *error){
// do something with data or handle error
NSLog(#"request completed");
dispatch_semaphore_signal(downloadSema);
}];
Edit 2
I just noticed you are updating your UI using a dispatch_sync. I see no reason for it to be synchronous, as it'll just block the background thread until the main thread has updated the UI. I would use a dispatch_async to do this.
Edit 3
As CouchDeveloper points out, it is possible that the number of concurrent network requests might be being capped by the system.
The easiest solution appears to be migrating over to NSURLSession and configuring the maxConcurrentOperationCount property of the NSOperationQueue used. That way you can ditch the semaphores altogether and just dispatch all your network requests on a background queue, using a callback to update the UI on the main thread.
I am not at all familiar with NSURLSession though, I was only answering this from a GCD stand-point.
You can send multiple requests, but sending 50 requests in parallel is usually not a good idea. There is a good chance that a server confronted with 50 simultaneous request will handle the first few and return errors for the rest. It depends on the server, but using a semaphore you can easily limit the number of running requests to anything you like, say four or eight. You need to experiment with the server in question to find out what works reliably on that server and gives you the highest performance.
And there seems to be a bit of confusion around: Usually all your network requests will run asynchronously. That is you send the request to the OS (which goes very quick usually), then nothing happens for a while, then a callback method of yours is called, processing the data. Whether you send the requests from the main thread or from a background thread doesn't make much difference.
Processing the results of these requests can be time consuming. You can process the results on a background thread. You can process the results of all requests on the same serial queue, which makes it a lot easier to avoid multithreading problems. That's what I do because it's easy and even in the worst case uses one processor for intensive processing of the results, while the other processor can do UI etc.
If you use synchronous network requests (which is a bad idea), then you need to dispatch each one by itself on a background thread. If you run a loop running 50 synchronous network requests on a background thread, then the second request will wait until the first one is completely finished.

Performing UI updates on main thread synchronously from a concurrent queue

As far as I have understood GCD UI operations should always be performed on the main thread/main queue asynchronously. But the following code seems to also work without any problem. Can someone please explain why ?
I am passing 2 blocks synchronously to a dispatch_async. One block downloads an image and the other displays it on the view.
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
__block UIImage *image = nil;
dispatch_sync(concurrentQueue, ^{
/* Download the image here */
});
dispatch_sync(dispatch_get_main_queue(), ^{
/* Show the image to the user here on the main queue */
});
});
The queue is important (it has to be the main queue) but whether the gcd calls are synchronous or asynchronous is irrelevant - that just affects how the rest of your code around the gcd calls is timed. Once a block is running on a queue it doesn't matter how it was scheduled.
Synchronous dispatch can simplify your code (since it won't return until the block is executed) but does come with the risk of locking if you end up waiting for things to finish.

what the benefit of using dispatch_sync ? ios

Is there a difference between these two implementation :
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
__block UIImage *image = nil;
dispatch_sync(concurrentQueue, ^{
/* Download the image here sync downloading */
});
dispatch_sync(dispatch_get_main_queue(), ^{
/* Show the image to the user here on the main queue */
});
});
and
dispatch_queue_t concurrentQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
__block UIImage *image = nil;
/* Download the image here sync downloading */
dispatch_sync(dispatch_get_main_queue(), ^{
/* Show the image to the user here on the main queue */
});
});
In the first code, I download the image using a block by dispatch sync, and in the second one I download it without block!
I think I should get deadlock in the first implementation because Apple says: dispatch_sync->Calling this function and targeting the current queue results in deadlock
I think I should get deadlock in the first implementation because apple say :dispatch_sync->Calling this function and targeting the current queue results in deadlock
It's true if the queue is a serial queue. However, in this case, you are using global queue, thus deadlock never happen. So the difference is which thread will do Download the image here sync downloading in the global queue thread pool.
First implementation
main queue -----------+---------------------------------+-----------+------
async(global)| ^ |
v sync(main)| v
global queue thread1 +-----+BLOCKED +-----------+BLOCKED +------
sync(global)| ^
v |
global queue thread2 +---------------+
Second implementation
main queue -----------+---------------------------------+-----------+------
async(global)| ^ |
v sync(main)| v
global queue thread1 +---------------------------------+BLOCKED +------
But dispatch_sync is not good idea in this case. https://developer.apple.com/library/ios/documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html
Important: You should never call the dispatch_sync or dispatch_sync_f function from a task that is executing in the same queue that you are planning to pass to the function. This is particularly important for serial queues, which are guaranteed to deadlock, but should also be avoided for concurrent queues.
... Even though I can't find any evidence of dispatch_sync should also be avoided for concurrent queues from
http://opensource.apple.com/source/libdispatch/libdispatch-339.92.1/src/queue.c
dispatch_sync dispatch_get_main_queue() done the action immediately.Because we are doing the cation in main queue.Here we are doing the UI updates and all that stuffs in lazy loading and asynchronous download etc.
Its related to GCD for more details please refer AppleDevelopers forum.
The main benefits of using dispatch_sync is we can do the operation concurrently.

Issue with GCD and too many threads

I have an image loader class which provided with NSURL loads and image from the web and executes completion block. Code is actually quite simple
- (void)downloadImageWithURL:(NSString *)URLString completion:(BELoadImageCompletionBlock)completion
{
dispatch_async(_queue, ^{
// dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
UIImage *image = nil;
NSURL *URL = [NSURL URLWithString:URLString];
if (URL) {
image = [UIImage imageWithData:[NSData dataWithContentsOfURL:URL]];
}
dispatch_async(dispatch_get_main_queue(), ^{
completion(image, URLString);
});
});
}
When I replace
dispatch_async(_queue, ^{
with commented out
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
Images are loading much faster, wich is quite logical (before that images would be loaded one at a time, now a bunch of them are loading simultaneously). My issue is that I have perhaps 50 images and I call downloadImageWithURL:completion: method for all of them and when I use global queue instead of _queue my app eventually crashes and I see there are 85+ threads. Can the problem be that my calling dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0) 50 times in a row makes GCD create too many threads? I thought that gcd handles all the treading and makes sure the number of threads is not huge, but if it's not the case is there any way I can influence number of threads?
The kernel creates additional threads when workunits on existing GCD worker threads for a global concurrent queue are blocked in the kernel for a significant amount of time (as long as there is further work pending on the global queue).
This is necessary so that the application can continue to make progress overall (e.g. the execution of one of the pending blocks may be what allows the blocked threads to become unblocked).
If the reason for worker threads to be blocked in the kernel is IO (e.g. the +[NSData dataWithContentsOfURL:] in this example), the best solution is replace those calls with an API that will perform that IO asynchronously without blocking, e.g. NSURLConnection for networking or dispatch I/O for filesystem IO.
Alternatively you can limit the number of concurrent blocking operations manually, e.g. by using a counting dispatch semaphore.
The WWDC 2012 GCD session went over this topic in some detail.
Well from http://developer.apple.com/library/ios/#documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html
Concurrent queues (also known as a type of global dispatch queue) execute one or more tasks concurrently, but tasks are still started in
the order in which they were added to the queue. The currently
executing tasks run on distinct threads that are managed by the
dispatch queue. The exact number of tasks executing at any given point
is variable and depends on system conditions.
and
Serial queues (also known as private dispatch queues) execute one task
at a time in the order in which they are added to the queue. The
currently executing task runs on a distinct thread (which can vary
from task to task) that is managed by the dispatch queue.
By dispatching all your blocks to the high priority concurrent dispatch queue with
[NSData dataWithContentsOfURL:URL]
which is a synchronous blocking network operation, it looks like the default GCD behaviour will be to spawn a load of threads to execute your blocks ASAP.
You should be dispatching to DISPATCH_QUEUE_PRIORITY_BACKGROUND. These tasks are in no way "High Priority". Any image processing should be done when there is spare time and nothing is happening on the main thread.
If you want more control over how many of these things are happening at once i reccommend that you look into using NSOperation. You can take your blocks and embed them in an operation using NSBlockOperation and then you can submit these operations to your own NSOperationQueue. An NSOperationQueue has a - (NSInteger)maxConcurrentOperationCount and as an added benefit operations can also be cancelled after scheduling if needed.
You can use NSOperationqueue, which is supported by NSURLConnection
And it has the following instance method:
- (void)setMaxConcurrentOperationCount:(NSInteger)count

wait for GCD async to stop before new code

I have a GCD that goes on the background. I have a button that when pressed I want it to load a loading wait screen while the GCD finishes, and then execute the rest of the code on that button. Attached is the sample.
Mine does not work, I basically want to say, wait as long as it takes to finish GCD, and load a waiting message in the meantime, when done continue code.
Thank you
- (IBAction)btnTapped:(id)sender
{
shouldCancel=NO;
dispatch_queue_t existingQueque = dispatch_get_main_queue();//finds the current GCD, the one I created in a different method
dispatch_group_t group =dispatch_group_create();
dispatch_group_async(group, existingQueque, ^
{
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);//does not work, I guess group can't be created here.
[self performSelectorOnMainThread:#selector(showWaitViewWithMessage:) withObject:#"Loading" waitUntilDone:YES];//load this until GCD queque done
[self performSelector:#selector(performSearch) withObject:nil afterDelay:0];
});
}
A couple of thoughts:
You suggest that dispatch_get_main_queue() "finds the current GCD, the one I created in a different method". No, this just gets the main queue (the one that, if you use it, will block your user interface), not the queue that you created elsewhere through dispatch_create_queue. The dispatch_get_main_queue() just gets the main queue, and while your searching is happening, your UI will be blocked (e.g. UIActivityIndicatorView won't spin, whatever).
If you've dispatched a whole bunch of tasks to a background queue, if you want to wait for all of them to finish, that's when you use dispatch_group_t or dispatch_barrier, but given what you've shown doesn't require that (you have only one dispatched operation), you just don't need to go there. By the way, barriers are not recommended if you're using global queues.
The typical pattern for a single GCD background task is more simple than your question suggests. You (a) update your UI to say "loading" and show a UIActivityIndicatorView or something like that, so the user has a richer UX showing them that the app is working on something; (b) dispatch the search in the background; and (c) when done, dispatch the UI update back to the main queue. Thus, the typical pattern is:
- (IBAction)btnTapped:(id)sender
{
dispatch_queue_t backgroundQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
// or, if you've already created you own background queue just use that here,
// or just create one here. But don't use dispatch_get_main_queue, as that
// won't use a background queue.
//
// dispatch_queue_t backgroundQueue = dispatch_queue_create("org.yourdomain.yourapp.search", NULL);
[self showWaitViewWithMessage:#"Loading"];
dispatch_async(backgroundQueue, ^{
[self performSearch]; // do this in the background
dispatch_async(dispatch_get_main_queue(), ^{
[self updateUiAfterSearch]; // when done, dispatch UI update back to main queue
});
});
// if you created a queue, remember to release it
//
// dispatch_release(backgroundQueue);
}
As an aside, in your performSelectorOnMainThread, I see no reason to waitUntilDone. Don't wait unless there is some compelling reason to do so. As you see above, this construct isn't needed at all, but just a FYI.
By the way, it's important to know that many servers impose limits on how many concurrent requests a given client may make at a time. If it's possible that you might be initiating multiple requests (e.g. the user taps buttons and the server is slow to respond) and this allows them to run concurrently. In this scenario, it's worth pursuing NSOperationQueue, where you can set maxConcurrentOperationCount. If you use the block versions of the NSOperationQueue methods (e.g. addOperationWithBlock rather than GCD's dispatch_async), the code can be structured in the same way, but it let's you constrain the number of background operations.
Also, NSOperationQueue offers the ability to easily establish dependencies between the operations (e.g. a completion NSOperation that is dependent on all of the others finishing). I can outline that, but the code you posted doesn't necessitate that, so I'll spare you that unless you let me know you want to see what that would look like.
you have to save the queue you create, dont create it each time and if you only want one at a time, use a serial queue
#implementation DDAppDelegate {
dispatch_queue_t queue;
}
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
[self do];
[self performSelector:#selector(do) withObject:nil afterDelay:1];
}
- (void)do {
if(!queue)
queue = dispatch_queue_create("com.example.MyQueue", NULL);
dispatch_async(queue, ^{
//serialized
NSLog(#"1");
sleep(10);
});
}
#end
if you want a concurrent queue, use a global queue and dispatch_barrier_async
#implementation DDAppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
[self do];
[self performSelector:#selector(do) withObject:nil afterDelay:1];
}
- (void)do {
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0);
dispatch_barrier_async(queue, ^{
//serialized
NSLog(#"1");
sleep(10);
});
}

Resources