I'm developing an iOS application with latest SDK and I have to implement a LinkedBlockingQueue with Objective-C.
I have to do something like this code:
public boolean onEvent(final EventArgs e) {
if (e.getClass() != this.eventType) return false;
long now = android.os.SystemClock.uptimeMillis();
long diff = now - this.last;
final long threadExecutionTimeMs = now - lastThreadExecution;
if (executions.remainingCapacity() == 0)
{
executions.poll();
}
executions.add(threadExecutionTimeMs);
...
}
Probably I can use NSMutableArray but I don't know how to emulate blocking, etc.
Any advice?
There's no equivalent AFAIK, you'll need to roll your own.
I think I'd probably use a dispatch semaphore.
Create a class with an instance variable that is the list of items in the queue (NSMutableArray is probably a good candidate). The class should also have an instance variable that is a dispatch semaphore.
In -init initialise the array and the semaphore with an initial count of zero.
Create two methods, one to enqueue an element and one to dequeue an element. The enqueue must also signal the semaphore after adding the element. The dequeue must wait on the semaphore before removing the element.
The code would look something like this:
#implementation MyQueue
{
NSMutableArray* myArray;
dispatch_semaphore_t fd_sema;
}
-(void) enqueue (id) anObject
{
#synchronized(myArray)
{
[myArray addObject: anObject];
}
dispatch_semaphore_signal(fd_sema);
}
-(id) dequeue
{
dispatch_semaphore_wait(fd_sema, DISPATCH_TIME_FOREVER);
id ret = nil;
#synchronized(myArray)
{
ret = [myArray objectAtIndex: 0];
[myArray removeObjectAtIndex: 0];
}
return ret;
}
The semaphore effectively counts the number of objects in the array. dispatch_semaphore_signal() increments the semaphore. dispatch_semaphore_wait() decrements the semaphore unless it's already zero, in which case the thread stops until something else signals the semaphore.
Accesses to the array itself are synchronized because a) there would otherwise be a race condition on removing something from the queue and b) I can't be bothered to look up whether NSMutableArray is thread safe.
Related
I'm trying to build an array of dictionaries in a background thread while keeping access to the current array until the background operation is done. Here's a simplified version of my code:
#property (nonatomic, strong) NSMutableArray *data;
#property (nonatomic, strong) NSMutableArray *dataInProgress;
- (void)loadData {
self.dataInProgress = [NSMutableArray array];
dispatch_async(dispatch_get_global_queue(QOS_CLASS_UTILITY, 0), ^{
[self loadDataWorker];
});
}
- (void)loadDataWorker {
for (int i=0; i<10000; i++) {
[self addDataItem];
}
dispatch_async(dispatch_get_main_queue(), ^{
[self loadDataFinish]; // the crash occurs before we get to this point
});
}
- (void)addDataItem {
// first check some previously added data
int currentCount = (int)[self.dataInProgress count];
if (currentCount > 0) {
NSDictionary *lastItem = [self.dataInProgress objectAtIndex:(currentCount - 1)];
NSDictionary *checkValue = [lastItem objectForKey:#"key3"]; // this line crashes with EXC_BAD_ACCESS
}
// then add another item
NSDictionary *dictionaryValue = [NSDictionary dictionaryWithObjectsAndKeys:#"bar", #"foo", nil];
NSDictionary *item = [NSDictionary dictionaryWithObjectsAndKeys:#"value1", #"key1", #"value2", #"key2", dictionaryValue, #"key3", nil];
// as described in UPDATE, I think this is the problem
dispatch_async(dispatch_get_main_queue(), ^{
[dictionaryValue setObject:[self makeCustomView] forKey:#"customView"];
});
[self.dataInProgress addObject:item];
}
- (UIView *)makeCustomView {
return [[UIView alloc] initWithFrame:CGRectMake(0, 0, 0, 0)];
}
- (void)loadDataFinish {
self.data = [NSMutableArray arrayWithArray:self.dataInProgress];
}
This works fine in most cases, but when the dataset is large, I start to get crashes on the line indicated above. The likelihood of a crash is greater with more data or a device with less memory. On an iPhone 6 with 10,000 items, it happens about one in five times. So it looks like when memory gets tight, the dictionaries inside the data array are destroyed before I access them.
If I do everything in the main thread there are no crashes. I originally had this problem with non-ARC code, then I converted my project to ARC and the same problem remains.
Is there a way to ensure the objects added earlier in the build process are retained until I'm done? Or is there a better way to do what I'm doing?
Here's a stack trace:
thread #17: tid = 0x9c586, 0x00000001802d1b90 libobjc.A.dylib`objc_msgSend + 16, queue = 'com.apple.root.background-qos', stop reason = EXC_BAD_ACCESS (code=1, address=0x10)
frame #0: 0x00000001802d1b90 libobjc.A.dylib`objc_msgSend + 16
frame #1: 0x0000000180b42384 CoreFoundation`-[__NSDictionaryM objectForKey:] + 148
frame #2: 0x00000001002edd58 MyApp`-[Table addDataItem](self=0x000000014fd44600, _cmd="addDataItem", id=0x00000001527650d0, section=3, cellData=0x0000000152765050) + 1232 at Table.m:392
frame #4: 0x00000001002eca28 MyApp`__25-[Table loadData]_block_invoke(.block_descriptor=0x000000015229efd0) + 52 at Table.m:265
frame #5: 0x0000000100705a7c libdispatch.dylib`_dispatch_call_block_and_release + 24
frame #6: 0x0000000100705a3c libdispatch.dylib`_dispatch_client_callout + 16
frame #7: 0x0000000100714c9c libdispatch.dylib`_dispatch_root_queue_drain + 2344
frame #8: 0x0000000100714364 libdispatch.dylib`_dispatch_worker_thread3 + 132
frame #9: 0x00000001808bd470 libsystem_pthread.dylib`_pthread_wqthread + 1092
frame #10: 0x00000001808bd020 libsystem_pthread.dylib`start_wqthread + 4
UPDATE
I traced through my full code with the answers below in mind, particularly those about locking while multithreading, and realized that part of the data I'm adding to my data array is a UIView that I'm creating during the build process. Since it's bad to build views in a background thread, and I did see problems when doing that, I'm jumping back to the main thread for makeCustomView. See the lines of code I added above with "UPDATE" in the comment. This must be the problem now; when I skip adding the custom view, I have no more crashes.
I could rework the build workflow so that all the data except the custom views are added on the background thread, then I could make a second pass and add the custom views on the main thread. But is there a way to manage the threads in this workflow? I tried locking with NSLock before and after calling makeCustomView, but that made no difference. I also found an SO answer saying NSLock is basically outdated, so I didn't go further with that.
If I understood you correctly, concurrent accesses to the dataInProgress array causes the problem, because the array is filled in a background thread and used in the main thread. But NSMutableArray is not thread safe. This fits to my intention that the array itself is corrupted.
You could solve that with NSLock to serialize the accesses to the array, but this is akin of outdated and does not fit to the rest of your code, which uses the more modern (and better) GCD.
A. Situation
What you have:
a builder control flow, which has to run in background
a creation of views control flow, which has to run in the main queue (thread). (I'm not completely sure, whether the pure creation of a view has to be done in main thread, but i would do.)
both control flows accesses the same resource (dataInProgress)
B. GCD
With classical thread/lock approach you start async control flows and serialize them with locks, when they access concurrently a shared resource.
With GCD you start control flows concurrently to each other, but serialized for a given shared resource. (Basically, there are more features, more complexity, but this is, what we need here.)
C. Serializing
It is correct to start the builder in a background queue ("thread") to run it without blocking the main thread. Done.
It is correct to switch back to main thread, if you want to do something with UI elements, esp. creating a view.
Since both control flows accesses the same resource, you have to serialize the accesses. You do this by creating a (serial) queue for that resource:
…
#property dispatch_queue_t dataInProgressAccessQ;
…
// In init or whatever
self. dataInProgressAccessQ = dispatch_queue_create("com.yourcompany.dataInProgressQ", NULL);
After doing that, you put every access to the dataInProgress array in that queue. There is a simple example for that:
// [self.dataInProgress addObject:item];
dispatch_async( self.dataInProgressAccessQ,
^{
[self.dataInProgress addObject:item];
});
In this case it is very easy, because you have to switch the queue at the and of the code. If it is in the middle, you have two options:
a) Use the queue similar to a lock. Let's have an example:
// NSInteger currentCount = [self.dataInProgress count]; // Why int?
NSInteger currentCount;
dispatch_sync( self.dataInProgressAccessQ,
^{
currentCount = [self.dataInProgress count];
});
// More code using currentCount
Using dispatch_sync() will let the code execution wait, until accesses from other control flows are finished. (It is like a lock.)
Edit: As with locks, the access is guaranteed to be serialized. But there might be the problem, that another thread removes objects from the array. Let's have a look to such a situation:
// NSInteger currentCount = [self.dataInProgress count]; // Why int?
NSInteger currentCount;
dispatch_sync( self.dataInProgressAccessQ,
^{
currentCount = [self.dataInProgress count];
});
// More code using currentCount
// Imagine that the execution is stopped here
// Imagine that -makeCustomView removes the last item in meanwhile
// Imagine that the execution continues here
// -> currentCount is not valid anymore.
id lastItem = [self.dataInProgress objectAtIndex:currentCount]; // crash: index out of bounds
To prevent this, you really have to isolate your concurrent code. This highly depends on your code. However, in my example:
id lastItem;
dispatch_sync( self.dataInProgressAccessQ,
^{
NSInteger currentCount;
currentCount = [self.dataInProgress count];
lastItem = [self.dataInProgress objectAtIndex:currentCount]; // don't crash: bounds are not changed
});
// Continue with lastItem
As you can imagine, when getting the last item, if can be removed from the array in the very next moment after you read it. Maybe this causes problems of inconsistency in your code. It really depends on your code.
End of edit
b) Maybe you get performance problems, because is works like a lock (synch). If so, you have to analyze your code and extract parts, that can run concurrently again. The pattern looks like this:
// NSInteger currentCount = [self.dataInProgress count]; // Why int?
dispatch_async( self.dataInProgressAccessQ, // <-- becomes asynch
^{
NSInteger currentCount = [self.dataInProgress count];
// go back to the background queue to leave the access queue fastly
dispatch_async( dispatch_get_global_queue(),
^{
// use current count here.
});
});
dispatch_async( self.dataInProgressAccessQ,
^{
// Another task, that can run concurrently to the above
});
What you can do there, is a matter of your concrete code. Maybe it is a help for you, to have your own private builder queue instead of using the global queue.
But this is the basic approach: Move a task into a queue and do not wait, until it is finished, but add code at the end, that completes the task in another control flow.
Instead of
Code
--lock--
var = Access code
--unlock--
More Code using var
it is
Code
asynch {
var Access Code
asynch {
More code using var
}
}
Of course, you have to do the same inside -makeCustomView.
I agree with Phillip Mills. This looks like a thread safety issue around your self.dataInProgress object.
From Apple docs https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/Multithreading/ThreadSafetySummary/ThreadSafetySummary.html :
Mutable objects are generally not thread-safe. To use mutable objects in a threaded application, the application must synchronize access to them using locks. (For more information, see Atomic Operations). In general, the collection classes (for example, NSMutableArray, NSMutableDictionary) are not thread-safe when mutations are concerned. That is, if one or more threads are changing the same array, problems can occur. You must lock around spots where reads and writes occur to assure thread safety.
If addDataItem is being called from various background threads, you need to lock around reading and writing to self.dataInProgress.
I don't think you need deep copies. If the dictionaries aren't mutable, all you need is for them to not be released...and a copy of the array they're in will do that for you.
What I believe you need is synchronization around any access to self.data. I suggest creating a NSLock object for your class and wrapping each of the following two lines with lock/unlock method calls:
self.data = [NSMutableArray arrayWithArray:self.dataInProgress];
//...
NSDictionary *item = [self.data objectAtIndex:index];
Also, why does self.data need to be mutable? If it doesn't, self.data = [self.dataInProgress copy]; is simpler...and quite possibly more efficient for memory and performance.
The one thing that's worrying me is, what about the caller of getData. It may not know that the self.data array has changed. If the array becomes shorter, you're headed for an "index out of bounds" crash.
It would be good to only call getData when you know the array is going to be stable. (In other words, synchronize the data gets at a higher level.)
I would attempt at passing in a weak reference to self. I bet if you might have a strong retain cycle happening there somewhere. If I remember correctly, __weak doesn't up the retain count, and __block allows you to change the variable
- (void)loadData {
self.dataInProgress = [NSMutableArray array];
__weak __block SelfClassName *weakSelf = self;
dispatch_async(dispatch_get_global_queue(QOS_CLASS_UTILITY, 0), ^{
[weakSelf loadDataWorker];
});
}
- (void)loadDataWorker {
for (int i=0; i<10000; i++) {
[self addDataItem];
}
__weak __block SelfClassName *weakSelf = self;
dispatch_async(dispatch_get_main_queue(), ^{
[weakSelf loadDataFinish];
});
}
File Person.h
#interface Person:NSObject
+ (void)callBlock:(void (^)())block;
#end
File Person.m
#imp:
+ (void)callBlock:(void (^)())block
{
[NSThread sleepForTimeInterval:2];
block();
}
#end
Code in ViewDidLoad:
Person *p = [[Person alloc] init];
void (^block)() = ^{
NSLog(#"%#"，p);
[p release];
}
[Person callBlock:block];
finish!
MyQuestion:
in main function, block var is a stackblock,and this block assign to the function +callBlock:,and the block in Person.m is also a stackblock. Content of them are same.
In my opinion,the block in main will be free by system before called in Person.m, so I think this program will crash, but it run normal. why?
And i think my code is same as below.
...
void example_addBlockToArray(NSMutableArray *array) {
char b = 'B';
[array addObject:^{
printf("%cn", b);
}];
}
void example() {
NSMutableArray *array = [NSMutableArray array];
example_addBlockToArray(array);
void (^block)() = [array objectAtIndex:0];
block();
}
This programe crashed！ which the difference between them?
Sorry! both program use mrc!!! i did not write clearly!
As it seems, you're using manual memory management.
Therefore is an explanation:
Person object case
You create the object
You create the block
You call the block
Block logs the object out
Block frees the object
That's why there is no crash
Char log case
Since you're not using ARC, it goes this way:
You add the block that logs char
Once you leave the function that adds the block to an array, the char you've just created is going to be released from memory.
In case if you use ARC for memory management, it will keep this char alive in the memory, until this block exists. But once you remove it from an array and block's reference count equals to 0, this char also being released.
You take a block from an array
Call it, it references to a memory address of char that already released, crash appears. As explained in your error:
EXC_BAD_ACCESS (code=1, address=0x0)
Means, that you point to zero address (null pointer exception in other words).
So, that's it.
As pointed out in the comments, the first example does not crash because the block is defined in a function scope which doesn't end until after the block is called.
The second example, on the other hand, defines the block in a secondary function, which ends before the block is called. Thus, at the time of the block invocation, the stack has already been modified and the block has become invalid.
After main function executed the block in main will be free by system ，not before called in Person.m. I tried the second code in viewcontroller, it works fine, not crash.
I have written a recursive block following these guidelines:
NSMutableArray *groups = [NSMutableArray arrayWithArray:#[#"group1", #"group2", #"group3", #"group4"];
__block CommunicationCompletionHandler completion = [^{
[groups removeObjectAtIndex:0];
if ([groups count] > 0) {
// This will send some information to the network, and calls the completion handler when it receives a response
[mySocket saveGroup:groups[0] completion:completion];
}
} copy]; // Removing copy here doesn't work either
[mySocket saveGroup:groups[0] completion:completion];
In the saveGroup:completion: method, I add the completion handler to an array:
self.completionHandlers[SaveGroupCompletionHandlerKey] = [completion copy];
And when I receive a response, I call the following method (key is in this case SaveGroupCompletionHandlerKey):
- (void)performCompletionHandlerForKey:(NSString *)key {
if (self.completionHandlers[key]) {
((CommunicationCompletionHandler)self.completionHandlers[key])();
[self.completionHandlers removeObjectForKey:key];
}
}
The problem is that the completion handler only gets called once. The removeObjectForKey: line makes the block deallocate. If I uncomment that line, everything works fine. I'm not sure how the array has the last reference to this block, since I add a copy (which I believe is being optimized to a retain).
For clarity, the flow of the app is:
Send data for first group over network
Receive response
Call completion handler
In the completion handler, send data for next group (this is the recursive part).
Anybody here who can point out what I'm doing wrong?
In -performCompletionHandlerForKey: you remove the completion handler from your dictionary after executing the block, which means that the handler will always be removed from the dictionary after one run.
Instead, store the block in a temporary variable and remove it from the dictionary before executing the block.
By the way, the advice to remove the weak reference is wrong. As your code is written now, your block will never be deallocated. The typical block recursion pattern is this:
__weak __block MyBlock weakHandler;
MyBlock handler = ^ {
if (foo) {
MyBlock strongHandler = weakHandler;
[bar asyncOperationWithCompletion:strongHandler];
}
};
weakHandler = handler;
[bar asyncOperationWithCompletion:handler];
A popular way to avoid retain retain cycles is to create a weak reference to the object before defining the block, then create a strong reference inside the block and set it to that weak reference. This method is frequently used to avoid strongly capturing self inside of blocks:
- (void)someMethod {
__weak MyType *weakSelf = self;
[self someMethodWithABlockArg:^{
MyType *strongSelf = weakSelf;
[strongSelf someOtherMethod];
}];
}
The strong reference created inside the block prevents the object from being deallocated while the block is running. You can, of course, do the same with any object type.
Edit2: Looks like [someBlock copy] is indeed fine. Have you tried running Analyze on the code? It may be that completion is not yet initialized when it is referred to inside of the block.
I want to synchronize some data with a web service. For each item I have to make a asynchronous call.
I want to have a completion block witch is called, when each item was synchronized. For each item I am able to perform a completion block. Now, I don't know a good way how to do it.
This is the interface:
-(void) synchronizeItemsOnComplete:(CompleteBlock) block {
NSArray* items = // get items
for (int i = 0, n = [items count]; i < n; i++) {
[self synchronizeItem:[items objectAtIndex:i] onComplete:^{
// What do do here?
}];
}
// And/or here?
}
-(void) synchronizeItemOnComplete:(CompleteBlock) block {
// do something
block();
}
How can I wait for the synchronization and then perform the block?
I tried something like this:
NSArray* items = // get items
__block int countOfItemsUntilDone = [items count];
for (int i = 0, n = countOfItemsUntilDone; i < n; i++) {
[self synchronizeItem:[items objectAtIndex:i] onComplete:^{
countOfItemsUntilDone--;
}];
}
dispatch_queue_t queue = dispatch_queue_create("wait for syncing", DISPATCH_QUEUE_CONCURRENT);
dispatch_async(queue, ^{
while (countOfItemsUntilDone > 0) {
usleep(1000); // wait a little bit
}
block();
});
dispatch_release(queue);
But I think this is a quite bad way. Any ideas?
Instead of spinning in a loop waiting for the counter to equal zero, check the counter value each time you decrement it, then fire an event when it reaches zero.
-(void) synchronizeItemsOnComplete:(CompleteBlock) block {
NSArray* items = // get items
__block NSUInteger remaining = [items count];
for (ItemClass* item in items) {
[self synchronizeItemImage:item onComplete:^{
--remaining;
if (remaining == 0) {
block();
}
}];
}
}
To explain why it feels wrong, there are two things you're doing here that you should do either never or rarely:
Using background queues. This is difficult and bug-prone. Don't do it without reading up a lot about writing concurrent code. You also only really need to do this if an operation blocks for a substantial amount of time (eg., to read a file from disk, or perform an intensive calculation). Don't assume you need to do it unless you have a good reason (eg., a measurable performance problem).
Spinning in a loop, checking a variable for changes and calling sleep. You should never do this.
Also, if you're looping over the elements in an array, the for ... in syntax is much nicer (and potentially more efficient) calling objectAtIndex: on each index.
Never check or decrement shared memory in different threads like this, it can cause races. Use a dispatch group to do what you're doing.
dispatch_queue_t myBGQueue;
dispatch_group_t itemsGroup = dispatch_group_create();
for (ItemClass *item in items) {
dispatch_group_async(itemsGroup, myBGQueue, ^{
[self synchronizeItemImage:item];
});
}
/* execution will sleep here until all the blocks added in the `for` complete */
dispatch_group_wait(itemsGroup, DISPATCH_TIME_FOREVER);
dispatch_release(itemsGroup);
You can use these to use synchronously.
GCD and this
performSelector:waitUntilDone:YES
Let's say that I want to keep things nice and speedy in the main UI, so I break off slow parts into queues (using the global concurrent queues). Assume that selectedUser in this case remains static throughout.
In one View Controller I have something like this:
- (IBAction)buttonPressed:(id)sender {
User *selectedUser = [self getSelectedUser];
dispatch_queue_t queue;
queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(queue, ^{
#autoreleasepool {
[userManager doSomething:selectedUser withForceOption:YES];
}
});
}
And in another class I have a singleton defined (userManager), with a method like this:
- (void)doSomething:(User*)user withForceOption:(BOOL)force {
SAppDelegate *delegate = (SAppDelegate *)[UIApplication sharedApplication].delegate;
extlib_main_queue = delegate.extlib_main_queue;
dispatch_async(extlib_main_queue, ^{
#autoreleasepool {
extlib_status_t status;
user.opIsStarting = YES;
extlib_call_id callId = -1;
// this is the part that worries me:
extlib_str_t uri = extlib_str((char *) [[NSString stringWithFormat:#"http:%##%s", user.account,DOMAIN] UTF8String]);
status = extlib_call_make_call(0, &uri, 0, NULL, NULL, &callId);
}
});
}
My question is: is it safe to do this, or do I need to do something else to make sure that the passed User instance's parameters remain accessible to both blocks?
The User object will be retained by both blocks as long as they are alive. The only issue here is that the User object need to actually be safe to access from different threads.
You have nothing to worry about, because blocks retain the variables that they refer to.
The block in buttonPressed: retains selectedUser since the block refers to it, and the block in doSomething:withForceOption: retains user because the block in there refers to it as well.
Read this section of Blocks Programming Topics for more details on how this works.