I'm trying to update NSManagedObject's in the background based on properties from a network fetch. Having trouble wrapping my head around concurrency. What I have tried is
fetch NSManagedObjects(Asset) on a localcontext (NSPrivateQueueType)
enumerate over assets array perform a network GET request for each asset and add it to a dispatch_group
Within the completion block from the network call, map updated values to the asset (This is my problem)
My code looks something like this
__block dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
__block dispatch_group_t taskGroup = dispatch_group_create();
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext) {
NSArray *assets = [Asset MR_findAllInContext:localContext];
for (Asset *asset in assets) {
dispatch_group_enter(taskGroup);
dispatch_async(queue, ^{
[stockClient GET:#"/v1/public/yql" parameters:asset.assetSymbol success:^(NSURLSessionDataTask *task, id responseObject) {
//TODO:Check response
NSDictionary *stockData = [[[responseObject objectForKey:#"query"] objectForKey:#"results"] objectForKey:#"quote"];
[asset mapPropertiesFrom:stockData];//----------> How do I access the localcontext queue?
dispatch_group_leave(taskGroup);
} failure:^(NSURLSessionDataTask *task, NSError *error) {
//TODO:Error handling here
dispatch_group_leave(taskGroup);
}];
});
}
dispatch_group_notify(taskGroup, dispatch_get_main_queue(), ^{
NSLog(#"All Tasks are completed");
});
}];
}
When I try to update the asset(NSManagedObject) from the network completion block
[asset mapPropertiesFrom:stockData];
I receive
CoreData could not fulfill a fault for
I suspect this is because the completion block is on the main queue and my asset was fetched on a private queue and you can't access NSManagedObjects from different queues...but I'm not sure is this is the problem and if it is how to resolve it.
So,how could I complete such a task? Am I going about this completely wrong?
EDIT
Made some changes based on comments below, but I'm receiving error
//Setup Dispatch Group
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_group_t taskGroup = dispatch_group_create();
NSArray *assets = [Asset MR_findAll];
for (Asset *asset in assets) {
dispatch_group_enter(taskGroup);
dispatch_async(queue, ^{
NSManagedObjectContext *localContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
localContext.parentContext = [NSManagedObjectContext MR_defaultContext];
Asset *localAsset = [asset MR_inContext:localContext];
[stockClient updateStockDataFor:localAsset completionHandler:^(NSDictionary *stockData, NSError *error) {
NSManagedObjectContext *responseContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
responseContext.parentContext = [NSManagedObjectContext MR_defaultContext];
Asset *responseAsset = [localAsset MR_inContext:responseContext];
[responseAsset mapPropertiesFrom:stockData];
[responseContext MR_saveToPersistentStoreAndWait];
dispatch_group_leave(taskGroup);
}];
});
}
dispatch_group_notify(taskGroup, dispatch_get_main_queue(), ^{
NSLog(#"Tasks are Completed");
});
}
Error:
Can only use -performBlock: on an NSManagedObjectContext that was
created with a queue.
Wrap the calls that will read/write anything from core data in a perform block, using:
- (void)performBlock:(void (^)())block
From the docs:
You use this method to send messages to managed objects if the context
was initialized using NSPrivateQueueConcurrencyType or
NSMainQueueConcurrencyType.
This method encapsulates an autorelease pool and a call to
processPendingChanges.
As long as you use this method for your calls there shouldn't be any problems.
I don't understand the second part of your question but in my app im performing updates/reads from network calls all the time using this method. Alternatively you can just dispatch to the main queue just the core data parts. That is, do the network thing on the background and when you need core data use a dispatch queue to the main queue.
As far as MagicalRecord is concerned here, you are dispatching a background task in the background. Basically, your queues are out of sync. I suggest you use a single background queue to save your data.
Try something like this:
__block dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
__block dispatch_group_t taskGroup = dispatch_group_create();
NSArray *assets = [Asset MR_findAll];
for (Asset *asset in assets) {
dispatch_group_enter(taskGroup);
dispatch_async(queue, ^{
NSManagedObjectContext = //create a confinement context
Asset *localAsset = [asset MR_inContext:localContext];
[stockClient GET:#"/v1/public/yql" parameters:asset.assetSymbol success:^(NSURLSessionDataTask *task, id responseObject) {
//TODO:Check response
NSManagedObjectContext *responseContext = //create a confinement context
Asset *responseAsset = [localAsset MR_inContext:responseContext];
NSDictionary *stockData = [[[responseObject objectForKey:#"query"] objectForKey:#"results"] objectForKey:#"quote"];
[asset mapPropertiesFrom:stockData];//----------> How do I access the localcontext queue?
[responseContext MR_saveToPersistentStoreAndWait];
dispatch_group_leave(taskGroup);
} failure:^(NSURLSessionDataTask *task, NSError *error) {
//TODO:Error handling here
dispatch_group_leave(taskGroup);
}];
});
}
dispatch_group_notify(taskGroup, dispatch_get_main_queue(), ^{
NSLog(#"All Tasks are completed");
});
}
Related
I have a content that consists of main file and additional files.
So here is the problem: At first I need to download,unpack and insert into database additional files and only then do the same thing for main file. Additional files are needed to be downloaded serial, and main file must be downloaded after them.
What is the right way to do it?
Right now I'm doing it this way:
- (void)checkIfPlacesAreDownloaded:(NSArray *)places{
[SVProgressHUD showWithStatus:#"Downloading places"];
dispatch_group_t group = dispatch_group_create();
for(NSDictionary *place in places){
BOOL result = [IDGDatabaseManager checkIfPlaceIsDownloaded:place];
if(!result){
dispatch_group_enter(group);
[self downloadPlace:place withCompletionHandler:^{
[IDGDatabaseManager setPlaceDownloaded:[place objectForKey:#"place_ID"]
WithCompletionBlock:^(BOOL success, NSError *error) {
dispatch_group_leave(group);
}];
}];
}
}
dispatch_group_notify(group, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
[self downloadExcursionWithParams:self.excursionDownloadResponse];
});
}
It only works if there is one file in "places" array. If there is more than one file they start to be downloaded parallel and it is not suitable for me.
I think that downloadPlace:withCompletionHandler: method works asynchronously on the background concurrent queue. That is why the file downloads run in parallel. I'd use a private serial queue instead or simply do the next:
[SVProgressHUD showWithStatus:#"Downloading places"];
dispatch_group_t group = dispatch_group_create();
// create a serial background queue to run the file downloads
dispatch_queue_attr_t qosAttribute = dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_SERIAL, QOS_CLASS_UTILITY, 0);
dispatch_queue_t myQueue = dispatch_queue_create("com.YourApp.YourQueue", qosAttribute);
for(NSDictionary *place in places){
BOOL result = [IDGDatabaseManager checkIfPlaceIsDownloaded:place];
if(!result){
dispatch_group_enter(group);
// run the download async on the serial bg queue
__weak __typeof(self) weakSelf = self;
dispatch_async(myQueue, ^{
__typeof(self) strongSelf = self;
// we need a semaphore to wait for the download completion
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
[strongSelf downloadPlace:place withCompletionHandler:^{
[IDGDatabaseManager setPlaceDownloaded:[place objectForKey:#"place_ID"]
WithCompletionBlock:^(BOOL success, NSError *error) {
dispatch_semaphore_signal(sema);
}];
}];
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
dispatch_group_leave(group);
});
}
}
dispatch_group_notify(group, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
[self downloadExcursionWithParams:self.excursionDownloadResponse];
});
For example I have a method with three async blocks. Each block result is needed to perform next block to achieve final methods result. So, what I'm looking for is a nice GCD strategy to make'em perform in a strict order and without dead locks
__block id task1Result;
__block id task2Result;
__block id finalResult;
[self startTask1:^(id result) { task1Result = result }]
[self startTask2:task1Result block:^(id result) { task2Result = result }]
[self startTask3:task2Result block:^(id result) { finalResult = result }]
UPD. I have found a solution:
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block id task1Result;
__block id task2Result;
__block id finalResult;
[self startTask1:^(id result) {
task1Result = result;
dispatch_semaphore_signal(sem);
}];
dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
[self startTask2:task1Result block:^(id result) {
task2Result = result;
dispatch_semaphore_signal(sem);
}];
dispatch_semaphore_wait(sem, DISPATCH_TIME_FOREVER);
[self startTask3:task2Result block:^(id result) { finalResult = result }];
But in my case I faced a problem with some library method which brings app to deadlock. ><
Create a serial dispatch queue like described here:
https://developer.apple.com/library/ios/documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationQueues/OperationQueues.html
In a nutshell:
dispatch_queue_t queue;
queue = dispatch_queue_create("com.example.MyQueue", NULL);
dispatch_async(queue, ^{
printf("Do some work here.\n");
});
dispatch_async(queue, ^{
printf("When finished do next task.\n");
});
Be aware that you have to handle the queue yourself.
If each task directly consumes the result of the previous task, can’t you start each one from the completion callback of its predecessor? You’ll still need a dispatch group to wait for the last task to complete, though.
dispatch_group_t group = dispatch_group_create();
__block id result;
dispatch_group_enter(group);
[self startTask1:^(id task1Result) {
[self startTask2:task1Result block:^(id task2Result) {
[self startTask3:task2Result block:^(id finalResult) {
result = finalResult;
dispatch_group_leave(group);
}];
}];
}];
dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
One complication you may run into is whether attempting to enqueue tasks from a completion handler runs the risk of deadlock, i.e. if your completion handlers are invoked on the same serial queue that handles enqueueing tasks.
I have 3 blocks of code that must execute one by one after previous finished. My implementation not works. I need some help to do it. My code bellow.
for (NSString *i in items)
{
[[RequestAPI sharedInstance]downloadImage:i completion:^(AFHTTPRequestOperation *operation, UIImage *image, NSError *error) {
//here main thread I receive images and go to BG
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
// here I save image on disk and get path
NSString *path = [ImageManager saveImageToDisk:image toEntity:entity withparams:#{#"save" : #"lala"}];
__block NSMutableDictionary *attachments = [NSMutableDictionary dictionary];
__block NSMutableArray *photoPaths = [NSMutableArray array];
dispatch_async(dispatch_get_main_queue(), ^{
//block1. here I load entity and dictionary from it with NSKeyedUnarchiver from CD and set to it image path
if (entity.attachments)
{
attachments = [NSKeyedUnarchiver unarchiveObjectWithData:entity.attachments];
if (attachments[type])
{
photoPaths = attachments[type];
}
}
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
//block2. here I check all images equality ti themselves in entity
BOOL haveDublicate = NO;
NSData *i = [ImageManager imageDataFromPath:path];
NSArray *photoImages = [ImageManager imageDatasFromPaths:photoPaths];
for (NSData *saved in photoImages)
{
if ([saved isEqualToData: i])
{
haveDublicate = YES;
}
}
if (!photoPaths)
{
photoPaths = [NSMutableArray array];
}
dispatch_async(dispatch_get_main_queue(), ^{
//block3. and finally if all ok I save image path, change load counter and post notification
if (path.length
&& ![photoPaths containsObject:path]
&& !haveDublicate
)
{
[photoPaths addObject:path];
[savedLinks setObject:photoPaths forKey:type];
entity.attachments = [NSKeyedArchiver archivedDataWithRootObject:savedLinks];
[self saveContext];
}
[RequestAPI sharedInstance].downloadsCount -= 1;
[[NSNotificationCenter defaultCenter]postNotificationName:kReloadFeedData object:nil];
});
});
});
});
}];
As dispatch_async says they will be executed asynchronous and not synchronous as you expected. Use dispatch_sync instead.
If you want to execute your code on a separate thread simple do the following
// create your thread
dispatch_queue_t queue = dispatch_queue_create("My Other Queue", 0);
// execute your synchronous block on the thread you've just created
dispatch_sync(queue,^{
// add your implementation here to be executed on your separate thread
dispatch_sync(dispatch_get_main_queue()^{
// update your UI here. Don't forget you can only update UI on the main thread
});
});
I am new to blocks and am trying to figure out how to wait for the block to finish before performing my action (in this case a nslog) So how can I wait till the block is done before performing this nslog in the code below: NSLog(#"convertedPhotos::%#",convertedImages);
convertedImages = [[NSMutableArray alloc] init];
for (NSDictionary *photo in photos) {
// photo is a dictionary containing a "caption" and a "urlRep"
[photoUrls addObject:photo[#"urlRep"]];
}
if (photoUrls.count) {
for (id photos in photoUrls){
NSString *urlString = photos;
[self base64ImageAtUrlString:urlString result:^(NSString *base64) {
[jsonWithPhotos setObject:convertedImages forKey:#"photo64"];
NSError *error;
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:jsonWithPhotos
options:NSJSONWritingPrettyPrinted // Pass 0 if you don't care about the readability of the generated string
error:&error];
if (! jsonData) {
NSLog(#"Got an error: %#", error);
} else {
NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
NSLog(#"json::%#",jsonString);
}
}];
}
}
else {
NSLog(#"where are my urls?");
}
NSLog(#"convertedPhotos::%#",convertedImages);
}
}
this method/block is called from above
- (void)base64ImageAtUrlString:(NSString *)urlString result:(void (^)(NSString *))completion {
NSURL *url = [NSURL URLWithString:urlString];
ALAssetsLibrary *library = [[ALAssetsLibrary alloc] init];
[library assetForURL:url resultBlock:^(ALAsset *asset) {
// borrowing your code, here... didn't check it....
ALAssetRepresentation *representation = [asset defaultRepresentation];
CGImageRef imageRef = [representation fullResolutionImage];
//TODO: Deal with JPG or PNG
NSData *imageData = UIImageJPEGRepresentation([UIImage imageWithCGImage:imageRef], 0.1);
NSString *base64 = [imageData base64EncodedString];
completion(base64);
[convertedImages addObject:base64];
// NSLog(#"converted::%#",convertedImages);
} failureBlock:^(NSError *error) {
NSLog(#"that didn't work %#", error);
}];
}
I would use NSOperationQueue (or dispatch queue) and NSCondition (or dispatch group) to wait for operations to complete. It is also important to wrap blocks in #autoreleasepool to flush memory once you do not need it if you work with memory consuming objects like NSData.
example:
// create results array
__block NSMutableArray* results = [NSMutableArray new];
// create serial queue
dispatch_queue_t queue = dispatch_queue_create("myQueue", 0);
for(NSInteger i = 0; i < 10; i++) {
// enqueue operation in queue
dispatch_async(queue, ^{
// create semaphore
dispatch_semaphore_t sema = dispatch_semaphore_create(0);
// do something async, I do use another dispatch_queue for example
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
// wrap in autoreleasepool to release memory upon completion
// in your case wrap the resultBlock in autoreleasepool
#autoreleasepool {
// here for example the nested operation sleeps for two seconds
sleep(2);
// add the operation result to array
// I construct an array of strings for example
[results addObject:[NSString stringWithFormat:#"Operation %d has finished.", i]];
// signal that nested async operation completed
// to wake up dispatch_semaphore_wait below
dispatch_semaphore_signal(sema);
}
});
// wait until the nested async operation signals that its finished
dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
NSLog(#"Finished single operation.");
});
}
// will be called once all operations complete
dispatch_async(queue, ^{
NSLog(#"Finished all jobs.");
NSLog(#"Results: %#", results);
});
For any non-main queue use semaphores
dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);
dispatch_async(dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// some serious stuff here
...
dispatch_semaphore_signal(semaphore);
});
dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
In case you want to wait for async task execution being in the main queue - you wouldn't probably want to block it while waiting for a semaphore.
I use this construction which doesn't freeze the UI for the main queue only.
__block BOOL flag = NO;
dispatch_async(dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
// some serious stuff here
...
flag = YES;
});
// Run until 'flag' is not flagged (wait for the completion block to finish executing
while (CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0, true) && !flag){};
Your best option is not to use blocks directly. Instead, create instances of NSBlockOperation and add them to an operation queue. Then create one more NSBlockOperation and make it dependent upon all of the other operations. This ensures that the last operation is only run after all others are completed and allows you to control how many operations will run at any one time.
If you have nested block calls, or some API that you can't change to enable this then you can still do it if you create an NSOperation subclass such that the operation does not complete until all of the asynchronous operations are complete. Take a look at dribin.org concurrent operations.
I have some networking code with heavy JSON parsing going on. It needs to be done in the background to not block the main thread. The code looks like this :
-(void) getSomeDataWithParameters:(...)parameters completion:(void (^)(NSArray *data))completion
{
NSURLRequest *req = ...
AFJSONRequestOperation *op = [[AFJSONRequestOperation alloc] initWithRequest:req];
// sometimes I have more requests
// startOperations is a wrapper on AFHTTPClient enqueueBatchOfHTTPRequestOperations:progressBlock:completionBlock:
// that handles errors and loading views
[self startOperations:#[op] completionBlock:^(NSArray *operations) {
// getBgQueue = return dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(getBgQueue(), ^{
NSArray *data = [MyParserClass parseJSON:op.responseJSON inContext:self.localContext];
[self.localContext MR_saveToPersistentStoreWithCompletion:^(BOOL success, NSError *error) {
// this is executed on main thread
if(completion) completion(...);
}];
});
}];
}
(AFNetworking 1.x)
The above code works very fine, but it's a pain to setup and write. And often the whole method content is wrapped inside another block to fetch some required data first... basically the blocks just pile up and makes ugly code
I'm using enqueueBatchOfHTTPRequestOperations and not individual completion blocks on AFJSONRequestOperation because batch completion block would sometimes fire before all individual operations completion blocks... (I also read somewhere that Mattt discouraged doing this)
Any pointers on how to do better than this?
I'm not sure what you want here, but just like "longcat is long", it's somewhat inherent in the pattern: 'continuation-passing style is continuation-passing style'. If you want to flatten things out a bit, you could make local block variables, but to a certain degree, you're stuck because you need the completion for -MR_saveToPersistentStoreWithCompletion to close over data in order to pass it to the -getSomeDataWithParameters... completion, but data won't exist until the -startOperations completion is executed.
You could probably achieve a less-nested appearance by using a bunch of __block variables, and splitting the code into several local blocks, but to me that feels kind of like cutting off your nose to spite your face. This code is readily understandable the way it is.
By the way... I notice that you're closing over op in the -startOperations completion block. This is fine because you're enqueuing op by doing -startOperations: #[op] ... but it would arguably be cleaner to get op from the operations parameter to the completion. I tightened this up as much as seemed reasonable:
- (void)getSomeDataWithParameters:(...)parameters completion:(void (^)(NSArray *data))completion
{
NSURLRequest *req = ...;
AFJSONRequestOperation *op = [[AFJSONRequestOperation alloc] initWithRequest:req];
[self startOperations:#[op] completionBlock:^(NSArray *operations) {
for (AFJSONRequestOperation *op in operations) {
dispatch_async(getBgQueue(), ^{
NSArray *data = [MyParserClass parseJSON:op.responseJSON inContext:self.localContext];
void (^mrSaveCompletion)(BOOL, NSError*) = completion ? ^(BOOL success, NSError *error) { completion(data); } : nil;
[self.localContext MR_saveToPersistentStoreWithCompletion: mrSaveCompletion];
});
}
}];
}
This will fan out each response potentially to a different thread. If you want all responses to execute on a single background thread, just swap the nesting of the for loop and the dispatch_async.
From there, the only really "superfluous" code is the dispatch_async. You could eliminate that by making -startOperations:... take a queue parameter where you would pass in the queue you wanted the completion to be called. Maybe like this:
- (void)startOperations: (NSArray*)ops completionQueue: (dispatch_queue_t)queue completionBlock: (void (^)(NSArray*))completion
{
void (^completionWrapper)(NSArray*) = !completion ? nil : ^(NSArray* ops) {
if (queue)
dispatch_async(queue, ^{ completion(ops); });
else
completion(ops);
};
[self startOperations: ops completionBlock: completionWrapper];
}
- (void)getSomeDataWithParameters:(...)parameters completion:(void (^)(NSArray *data))completion
{
NSURLRequest *req = ...;
AFJSONRequestOperation *op = [[AFJSONRequestOperation alloc] initWithRequest:req];
[self startOperations:#[op] completionQueue: getBgQueue() completionBlock:^(NSArray *operations) {
for (AFJSONRequestOperation *op in operations) {
NSArray *data = [MyParserClass parseJSON:op.responseJSON inContext:self.localContext];
void (^mrSaveCompletion)(BOOL, NSError*) = !completion ? nil : ^(BOOL success, NSError *error) { completion(data); };
[self.localContext MR_saveToPersistentStoreWithCompletion: mrSaveCompletion];
});
}];
}