I have a loop which I run and in each iteration I have a block that I want run on an NSOperationQueue. The underlying queue is serial. This loop could add hundreds of potentially long running block tasks. When I set m_opQueue.suspended = YES the blocks will still keep executing.
I'm well aware that a single block cannot stop right in the middle, but I expected that pausing the NSOperationQueue would simply not execute the next operation until suspended was false.
Can anyone explain whether I'm wrong or how I achieve what I want?
dispatch_queue_t index_queue = dispatch_queue_create("someQueue", DISPATCH_QUEUE_SERIAL);
m_OpQueue = [[NSOperationQueue alloc] init];
m_OpQueue.underlyingQueue = index_queue;
for ( NSUInteger i = 0; i < total; i++ ) {
void (^block)(void) = ^void() {
// Do stuff.
NSLog(#"processing complete.");
};
// Effectively adds a NSBlockOperation.
[m_OpQueue addOperationWithBlock:block];
}
This curious behavior you describe (where previously enqueued operations will continue to start even after the queue has been suspended), is caused by how you created the serial queue.
Generally, you create a serial operation queue by setting maxConcurrentOperationCount:
m_OpQueue.maxConcurrentOperationCount = 1;
If you do that (no need to set underlyingQueue), you see the expected behavior.
Related
So I'm putting 10 tasks on a concurrent queue using dispatch_async. They do not block the next task, and gets processed in order. My UI is responsive.
for (int i = 0; i < 10; i++) {
dispatch_async(concurrencyQueue, ^() {
NSLog(#"..calling insertion method to insert record %d", i);
dispatch_sync(serialQueue, ^() {
//this is to simulate writing to database
NSLog(#"----------START %d---------", i);
[NSThread sleepForTimeInterval:1.0f];
NSLog(#"--------FINISHED %d--------", i);
});
});
}
Within each task, we simulate a write to database with a "1 sec sleep" on a serial Queue via dispatch_sync.
I always thought dispatch_sync blocks everyone, and syncs its tasks because that's how it behaves when I use it individually. However, in this situation, it does not block the main thread. Instead, it runs beautifully in the background like I want it.
Is it because whatever thread is associated with the queue is being affected?
For example, the main thread is executing the concurrent queue via dispatch_async and that's why it is not blocked.
The dispatch_sync only syncs and blocks against the background thread that's working on the concurrent queue. Hence, the dispatch_sync is associated with the background thread, thus never affecting my UI main thread.
Is my thinking correct?
thank you!
You never block the main thread because your code is running on either the threads of the concurrencyQueue or the thread of the serialQueue. None of those are the main thread.
All of the calls to sleep happen one by one on the thread of the serialQueue. So it is the thread of the serialQueue that is blocked.
However, since you dispatch to the serialQueue using dispatch_sync, you are also blocking each thread of the concurrent queue. This would be better pictured if you add another NSLog after the call to dispatch_sync.
for (int i = 0; i < 10; i++) {
dispatch_async(concurrencyQueue, ^() {
NSLog(#"..calling insertion method to insert record %d", i);
dispatch_sync(serialQueue, ^() {
//this is to simulate writing to database
NSLog(#"----------START %d---------", i);
[NSThread sleepForTimeInterval:1.0f];
NSLog(#"--------FINISHED %d--------", i);
});
NSLog(#"..called insertion method to insert record %d", i);
});
}
That 2nd NSLog after the dispatch_sync will show you better how the dispatch_sync is affecting the calls to dispatch_async.
Yes, you are right. dispatch_sync() blocks only the thread the queue is running on.
I just got a memory leak in my code after I updated my iPad to iOS9, which worked fine on iOS8 and iOS7.
I have an anonymous thread created by the following code:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
[self threadWork];
});
And the thread does a pair of malloc/free call like this:
- (void)threadWork {
// Create a serial queue.
dispatch_queue_t mySerialQueue = dispatch_queue_create("myQueue", NULL);
while (1) {
// Do a simple malloc.
int *foo = (int *)malloc(1024);
// Do free in serial queue.
dispatch_async(mySerialQueue, ^{
free(foo);
});
[NSThread sleepForTimeInterval:1.0 / 60.0];
}
}
This routing will keep the memory usage increasing and finally crashes device on iOS 9. The problem also happened on new/delete in Objective-C++.
I found some other way to do this without memory leak:
Use main queue or global queue to instead the serial queue.
Create concurrent queue instead the serial queue.
Use [NSThread detachNewThreadWithSelector:toTarget:withObject:] to create the thread instead GCD.
I don't understand why this simple routing causes this problem.
I've searched this on google but found nothing.
How can I do this with keeping serial queue and GCD anonymous thread?
Update:
I tried to put NSLog commands in my code to figure out when will the malloc/free be called. The result shows that both of them are called immediately and come in pair. I also tried to slow the thread down to once per second, but the problem still here.
The test code of thread:
- (void)threadWork {
uint64_t mallocCount = 0;
__block uint64_t freeCount = 0;
dispatch_queue_t mySerialQueue = dispatch_queue_create("MyQueue", NULL);
while (1) {
void *test = malloc(1024);
NSLog(#"malloc %llu", ++mallocCount);
dispatch_async(mySerialQueue, ^{
free(test);
NSLog(#"free %llu", ++freeCount);
});
[NSThread sleepForTimeInterval:1.0];
}
}
The console result:
...
2015-10-23 09:51:33.876 OS9MemoryTest[759:153135] malloc 220
2015-10-23 09:51:33.876 OS9MemoryTest[759:153133] free 220
2015-10-23 09:51:34.877 OS9MemoryTest[759:153135] malloc 221
2015-10-23 09:51:34.878 OS9MemoryTest[759:153133] free 221
2015-10-23 09:51:35.883 OS9MemoryTest[759:153135] malloc 222
2015-10-23 09:51:35.883 OS9MemoryTest[759:153133] free 222
I think I've found a better way to do this without leak problem rather than using dispatch_sync.
The point seems to be the setting of Quality of Service (QoS) class of serial queue.
Doing free in a queue which have QOS_CLASS_UNSPECIFIED QoS class causes this problem.
In my question, I free memory in a serial queue which was created by the following call :
dispatch_queue_t mySerialQueue = dispatch_queue_create("MyQueue", NULL);
Its QoS setting is QOS_CLASS_UNSPECIFIED which causes this problem.
If create a serial queue with dispatch_queue_attr_t object, which have QoS setting excepted QOS_CLASS_UNSPECIFIED, the code runs perfectly without leaking:
- (void)threadWork {
// Create a serial queue with QoS class.
dispatch_queue_attr_t attr = dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_DEFAULT, 0);
dispatch_queue_t mySerialQueue = dispatch_queue_create("myQueue", attr);
while (1) {
// Do a simple malloc.
int *foo = (int *)malloc(1024);
// Do free in serial queue.
dispatch_async(mySerialQueue, ^{
free(foo);
});
[NSThread sleepForTimeInterval:1.0 / 60.0];
}
}
I still don't understand why this problem would happened on iOS9,
but setting the QoS seems to make things work.
Hi I am building an app using Swift. I need to process notifications in a specific order. Therefore I am trying to use addOperations waitUntilFinished.
Here is what I did:
let oldify = NSOperation()
oldify.completionBlock = {
println("oldify")
}
let appendify = NSOperation()
appendify.completionBlock = {
println("appendify")
}
let nettoyify = NSOperation()
nettoyify.completionBlock = {
println("nettoyify")
}
NSOperationQueue.mainQueue().maxConcurrentOperationCount = 1
NSOperationQueue.mainQueue().addOperations([oldify, appendify, nettoyify], waitUntilFinished: true)
With this code none of the operations is being executed. When I try this instead:
NSOperationQueue.mainQueue().maxConcurrentOperationCount = 1
NSOperationQueue.mainQueue().addOperation(oldify)
NSOperationQueue.mainQueue().addOperation(appendify)
NSOperationQueue.mainQueue().addOperation(nettoyify)
The operations get executed but not in the right order.
Does anyone know what I'm doing wrong? I am getting confident in swift but completely new to NSOperations
A couple of issues:
You are examining behavior of the completion block handlers. As the completionBlock documentation says:
The exact execution context for your completion block is not guaranteed but is typically a secondary thread. Therefore, you should not use this block to do any work that requires a very specific execution context.
The queue will manage the operations themselves, but not their completion blocks (short of making sure that the the operation finishes before its completionBlock is started). So, bottom line, do not make any assumptions about (a) when completion blocks are run, (b) the relation of one operation's completionBlock to other operations or their completionBlock blocks, etc., nor (c) which thread they are performed on.
Operations are generally executed in the order in which they were added to the queue. If you add an array of operations, though, the documentation makes no formal assurances that they are enqueued in the order they appear in that array. You might, therefore, want to add the operations one at a time.
Having said that, the documentation goes on to warn us:
An operation queue executes its queued operation objects based on their priority and readiness. If all of the queued operation objects have the same priority and are ready to execute when they are put in the queue—that is, their isReady method returns YES—they are executed in the order in which they were submitted to the queue. However, you should never rely on queue semantics to ensure a specific execution order of operation objects. Changes in the readiness of an operation can change the resulting execution order. If you need operations to execute in a specific order, use operation-level dependencies as defined by the NSOperation class.
To establish explicit dependencies, you might do something like:
let oldify = NSBlockOperation() {
NSLog("oldify")
}
oldify.completionBlock = {
NSLog("oldify completion")
}
let appendify = NSBlockOperation() {
NSLog("appendify")
}
appendify.completionBlock = {
NSLog("appendify completion")
}
appendify.addDependency(oldify)
let nettoyify = NSBlockOperation() {
NSLog("nettoyify")
}
nettoyify.completionBlock = {
NSLog("nettoyify completion")
}
nettoyify.addDependency(appendify)
let queue = NSOperationQueue()
queue.addOperations([oldify, appendify, nettoyify], waitUntilFinished: false)
BTW, as you'll see above, you should not add operations to the main queue in conjunction with the waitUntilFinished. Feel free to add them to a different queue, but don't dispatch from a serial queue, back to itself, with the waitUntilFinished option.
I read the documentation and came to know that timer (dispatch_source_t) skips to fire if the handler is still in progress for previous iterations.
But this whole business of handler taking it longer makes this inaccurate. And I am observing that I am unable to stop the timer at intended times.
My code looks like this:
double secondsToFire = 1.0f;
dispatch_queue_t queue = dispatch_get_main_queue();
m_myTimer = CreateDispatchTimer(secondsToFire, queue,
^{
//Do some time consuming operation, taking any number of seconds
int retVal = DoSomeOperation();
if (retVal == 1)
{
cancelTimer(m_myTimer);
SomeOtherOperation();
}
});
dispatch_source_t CreateDispatchTimer(double interval, dispatch_queue_t queue, dispatch_block_t block)
{
dispatch_source_t timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, queue);
if (timer)
{
// dispatch_source_set_timer(timer, dispatch_time(DISPATCH_TIME_NOW, interval * NSEC_PER_SEC), interval * NSEC_PER_SEC, (1ull * NSEC_PER_SEC) / 10);
dispatch_source_set_timer(timer, dispatch_time(DISPATCH_TIME_NOW, 0), interval * NSEC_PER_SEC, 0);
dispatch_source_set_event_handler(timer, block);
dispatch_resume(timer);
}
return timer;
}
void cancelTimer(dispatch_source_t _timer)
{
if (_timer)
{
dispatch_source_cancel(_timer);
_timer = nil;
}
}
Note:
Inside DoSomeOperation(), I have code enclosed with #synchronized(array), in which I access an array who is being written by another private queue. But entire DoSomeOperation() is executed on main queue.
My question is, is this is the right and accurate timing model? I am posting here because I am facing lot of inaccuracies - timer doesn't fire every second, and it doesn't stop as intended too. I am able to observe that SomeOtherOperation() gets called when retVal == 1, but timer isn't done yet.
Another Note:
m_myTimer above is an iVar, and my project is ARC, if that could make any difference.
No, a dispatch timer doesn't get "skipped" if it fires while your handler is running, the handler will get re-invoked for the pending event right away once the previous invocation returns.
If multiple firings occur while the handler is running or enqueued (or while the source is suspended), they will get all get coalesced into a single handler invocation (as is the case for all edge-triggered source types).
You can check how many firings a given handler invocation is for with dispatch_source_get_data()
My concern was about accuracy in starting, firing and stopping, and not in correct reporting of things.
I finally ended up assigning one of my tasks off a private queue instead of main one. The advantage could be clearly visible in timer accuracy and prompt timer cancellation.
Conclusion:
More the same (esp. main) queue getting flogged by timer tasks, more they become inaccurate.
I have a really heavy task to perform, and I dont want to block the main thread. So I'm starting a separate Concurrent queue for it. There can be 4 instances of the said task.
-(dispatch_queue_t)getConcurrentQueue
{
if(concurrentQueue == nil)
{
concurrentQueue = dispatch_queue_create("com.myself.HeavyTask", DISPATCH_QUEUE_CONCURRENT);
}
return concurrentQueue;
}
Now to start the heavy task I have -
-(void)beginTask
{
//.....
//.....
__weak typeof(self) weakSelf = self;
dispatch_queue_t queue = [self getConcurrentQueue];
dispatch_async(queue, ^{
[weakSelf heavyTask];
});
}
Now the method heavyTask goes like this --
-(void)heavyTask
{
//...
dispatch_sync(dispatch_get_current_queue(), ^{
// Initialising code for heavy task
// This is the critical section. Only one of the 4 concurrent threads can enter this at a time
}
//....
while(condition)
{
// Perform meat of the task
}
//...
dispatch_sync(dispatch_get_current_queue(), ^{
// Teardown code. Freeing memory etc.
// This is also a critical section.
}
//...
}
The initialising code and teardown code uses some 3rd party C methods which are not thread safe. So making them thread safe is not the scope of the question.
Now I've put the "initialising code" and and the "teardown code" within -
dispatch_sync(dispatch_get_current_queue(), ^{
}
My code is crashing and I'm getting error messages stating that there is insufficient thread locking around the critical section code.
I read that dispatch_get_current_queue() is not safe, so I replaced it with concurrentQueue. I also tried replacing with dispatch_get_main_queue(). Still the code crashes, complaining about insufficient thread locking.
I know there is something wrong in my understanding of implementing critical section using GCD.
Can anyone clearly show me how to make my code work properly here??
Side question -- Can I use #synchronized { } blocks here??
There's a lot wrong with your code, including not adhering to naming conventions.
So basically, if you want to execute the same task concurrently with respect to each other, use the global concurrent queue to execute those tasks.
If you want to concurrently access shared resources from within these tasks (or from elsewhere), define a dedicated queue, say "sync_queue" where you exclusively access these resources. This "sync_queue" executes your "critical sections".
The "sync_queue" can be serial or concurrent.
If you use a serial queue, use dispatch_async(sync_queue, block) for write access and dispatch_sync(sync_queue, block) for read access to shared resources.
If you use a concurrent queue, use dispatch_barrier_async(sync_queue, block) for write access and dispatch_barrier_sync(sync_queue, block) for read access to shared resources.
Example:
// Read access using a serial sync_queue:
...
__block int counter;
dispatch_sync(sync_queue, ^{
counter = _counter;
});
// Write access using a serial sync_queue:
...
dispatch_async(sync_queue, ^{
_counter = counter;
});
// Read access using a concurrent sync_queue:
...
__block int counter;
dispatch_barrier_sync(sync_queue, ^{
counter = _counter;
});
// Write access using a concurrent sync_queue:
...
dispatch_barrier_async(sync_queue, ^{
_counter = counter;
});
Example for your "heavy task":
-(void)heavyTask
{
dispatch_barrier_async(sync_queue, ^{
// Initialize heavy task
...
// Continue with the task:
dispatch_async(dispatch_get_global_queue(0,0), ^{
BOOL condition = YES; // condition must be local to the block (it's not a shared resource!)
while(condition)
{
// Perform meat of the task
condition = ...;
}
dispatch_barrier_async(sync_queue, ^{
// Teardown code. Freeing memory etc.
// This is also a critical section.
...
}
});
}
}
You called it "getSerialQueue" but really you are creating a "concurrent" queue in it. Try to fix it substituting DISPATCH_QUEUE_CONCURRENT with DISPATCH_QUEUE_SERIAL in getSerialQueue.
Keep in mind that:
dispatch_sync means: I will wait here until this block finishes
dispatch_async means: I will not wait
This is not related to concurrent or serial. If two tasks in a concurrent queue call dispatch_sync(block), 'block' will be executed concurrently.
Hope this helps.