How to create multiple objects in background? - ios

I'm using MagicalRecord 2.0.3 and I can't really figure out how to save data in the background.
According to the documentation, something like this should work:
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
// Do this hundreds of times
[MyObject createInContext:localContext];
}];
However, nothing is saved to the database. I've seen multiple people posting solutions similar to this:
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
// Do this hundreds of times
[MyObject createInContext:localContext];
} completion:^{
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
[[NSManagedObjectContext defaultContext] saveNestedContexts];
}];
}];
This does save my data in the database, however since the save happens on the main thread, my application is unresponsive for a while (with my dataset, about 3 seconds which is way too long).
I've also tried this, but it also blocks up while saving:
self.queue = [[NSOperationQueue alloc] init];
[self.queue addOperationWithBlock:^{
NSManagedObjectContext *localContext = [NSManagedObjectContext contextForCurrentThread];
// Do this hundreds of times
[MyObject createInContext:localContext];
[localContext saveNestedContexts];
}];
And lastly, same blocking effect with this code:
dispatch_queue_t syncQueue = dispatch_queue_create("Sync queue", NULL);
dispatch_async(syncQueue, ^{
NSManagedObjectContext *localContext = [NSManagedObjectContext contextForCurrentThread];
// Do this hundreds of times
[MyObject createInContext:localContext];
[[NSManagedObjectContext contextForCurrentThread] saveNestedContexts];
});
So, what is the best way to solve this? I need to create hundreds of objects in the background and the app needs to remain responsive.

MagicalRecord uses a child context when doing work in the background. This works fine for small changes, but will create excessive main thread blocking when importing large amounts of data.
The way to do it is to use a parallel NSManagedObjectContext and to do the merging yourself with the NSManagedObjectContextDidSaveNotification notification and the mergeChangesFromContextDidSaveNotification method. See performance tests here: http://floriankugler.com/blog/2013/5/11/backstage-with-nested-managed-object-contexts
When saving a nested contexts everything has to be copied to the parent context. As opposed to this, objects that have not been fetched (in the context into which you are merging) will not be merged by mergeChangesFromContextDidSaveNotification. This is what makes it faster.
You might encounter problems if you want to display these results right away after saving in batches and using an NSFetchResultsController. See the following question for a solution:
NSFetchedResultsController with predicate ignores changes merged from different NSManagedObjectContext
For more performance tips take a look at this question: Implementing Fast and Efficient Core Data Import on iOS 5
Create your own context.
NSManagedObjectContext *importContext = [[NSManagedObjectContext alloc]
initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[importContext setPersistentStoreCoordinator:yourPersistentStoreCoordinator];
[importContext setUndoManager:nil]; // For importing you don't need undo: Faster
// do your importing with the new importContext
// …
NSError* error = nil;
if(importContext.hasChanges) {
if(![importContext save:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
}
}
Make sure you are listening for the saves to managed object contexts.
[[NSNotificationCenter defaultCenter]
addObserver:singleton
selector:#selector(contextDidSave:)
name:NSManagedObjectContextDidSaveNotification object:nil];
In the contextDidSave:you merge the change yourself.
- (void) contextDidSave:(NSNotification*) notification
{
if(![notification.object isEqual:self.mainContext]) {
dispatch_async(dispatch_get_main_queue(), ^{
[self.mainContext mergeChangesFromContextDidSaveNotification:notification];
});
}
}

Managed object contexts are not thread safe so if you ever need to do any kind of background work with your Coredata objects (i.e. a long running import/export function without blocking the main UI) you will want to do that on a background thread.
In these cases you will need to create a new managed object context on the background thread, iterate through your coredata operation and then notify the main context of your changes.
You can find an example of how this could work here
Core Data and threads / Grand Central Dispatch

Related

Multicontext Core Data: merging to unsaved context

I'm trying to implement this core data stack:
PSC <--+-- MainMOC
|
+-- BackgroundPrivateMOC
There are some things I'm actually don't understand. Perhaps we have an object in our Persisten Store and we fetch it from the main MOC to do some changes (user change it manually). At the same time my BG MOC is doing some changes with the same one object and save the changes to PS. After the saving is done we must merge the BG MOC to the MAIN MOC (this is a common practice). What I expect after the merging is that the MAIN MOC contains changes from the BG MOC (because the changes were done a bit later than the MAIN ones). But this actually doesn't happened. All I have after the merging is finished is a dirty refreshedObjects = 1 in my MAIN MOC and if I fetch that object again through the MAIN MOC, I don't see any changes made through the BG MOC.
How should I correctly propagate BG changes to MAIN MOC while the
MAIN MOC was not saved prior the BG changes was made?
How to handle
the situation when my MAIN MOC has non-zero refreshedObjects after merging is completed, and
how to push these objects in the MAIN MOC to make them available to
fetch and with?
I believe my sample code can help you to understand my problem more clearly. You can just download the project (https://www.dropbox.com/s/1qr50zto5j4hj40/ThreadedCoreData.zip?dl=0) and run XCTest, that I prepared.
Here is the failing test code:
#implementation ThrdCoreData_Tests
- (void)setUp
{
[super setUp];
/**
OUR SIMPLE STACK:
PSC <--+-- MainMOC
|
+-- BackgroundPrivateMOC
*/
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
// main context (Main queue)
_mainMOC = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
[_mainMOC setPersistentStoreCoordinator:coordinator];
[_mainMOC setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];
// background context (Private Queue)
_bgMOC = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
_bgMOC.persistentStoreCoordinator = self.persistentStoreCoordinator;
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(mergeBGChangesToMain:)
name:NSManagedObjectContextDidSaveNotification
object:_bgMOC];
u_int32_t value = arc4random_uniform(3000000000); // simply generate new random values for the test
_mainMOCVlaue = [NSString stringWithFormat:#"%u" , value];
_expectedBGValue = [NSString stringWithFormat:#"%u" , value/2];
Earthquake * mainEq = [Earthquake MR_findFirstInContext:self.mainMOC];
if (!mainEq){ // At the very first time the test is running, create one single test oject.
Earthquake * mainEq = [Earthquake MR_createEntityInContext:self.mainMOC];
mainEq.location = nil; // initial value will be nil
[self.mainMOC MR_saveOnlySelfAndWait];
}
}
- (void)testThatBGMOCSuccessfullyMergesWithMain
{
_expectation = [self expectationWithDescription:#"test finished"];
// lets change our single object in main MOC. I expect that the value will be later overwritten by `_expectedBGValue`
Earthquake * mainEq = [Earthquake MR_findFirstInContext:self.mainMOC];
NSLog(#"\nCurrently stored value:\n%#\nNew main value:\n%#", mainEq.location, _mainMOCVlaue);
mainEq.location = _mainMOCVlaue; // the test will succeed if this line commented
// now change that object in BG MOC by setting `_expectedBGValue`
[_bgMOC performBlockAndWait:^{
Earthquake * bgEq = [Earthquake MR_findFirstInContext:_bgMOC];
bgEq.location = _expectedBGValue;
NSLog(#"\nNew expected value set:\n%#", _expectedBGValue);
[_bgMOC MR_saveToPersistentStoreAndWait]; // this will trigger the `mergeBGChangesToMain` method
}];
[self waitForExpectationsWithTimeout:3 handler:nil];
}
- (void)mergeBGChangesToMain:(NSNotification *)notification {
dispatch_async(dispatch_get_main_queue(), ^{
[self.mainMOC mergeChangesFromContextDidSaveNotification:notification];
// now after merge done, lets find our object with expected value `_expectedBGValue`:
Earthquake * expectedEQ = [Earthquake MR_findFirstByAttribute:#"location" withValue:_expectedBGValue inContext:self.mainMOC];
if (!expectedEQ){
Earthquake * eqFirst = [Earthquake MR_findFirstInContext:self.mainMOC];
NSLog(#"\nCurrent main MOC value is:\n%#\nexptected:\n%#", eqFirst.location, _expectedBGValue);
}
XCTAssert(expectedEQ != nil, #"Expected value not found");
[_expectation fulfill];
});
}
First, when posting core data code, I suggest you not post code that depends on a third party library, unless that third party library is directly related to your problem. I assume MR is magical record, but I don't use it, and it seems to just muddy the waters of the post because who knows what it is (or is not) doing under the covers.
In other words, try to trim examples down to as little as code as necessary... and no more... and only include third-party libraries when absolutely necessary.
Secondly, when writing unit tests for your core data usage, I suggest using an in-memory stack. You always start empty and it can be initialized however you want. Much easier to use for testing.
That said, your problem is a misunderstanding of what mergeChangesFromContextDidSaveNotification does (and does not do).
Basically, you have an object in a Core Data persistent store. You have two different MOCs attached to the store via the same PSC.
Your test then loads the object into main MOC, and changes the value without saving to the PSC. A second MOC then loads the same object, and changes its value to something different (i.e., the store, and both MOCs all have a different value for a particular attribute of the same object).
Now, when we save the MOC, if there are conflicts, the conflicts will be handled as instructed by the mergePolicy. However, the merge policy does not apply to mergeChangesFromContextDidSaveNotification.
You can think of mergeChangesFromContextDidSaveNotification as inserting any new objects, deleting any deleted objects, and "refreshing" any updated objects while preserving any local changes.
In your test, if you add another attribute (e.g., "title") and change both "title" and "location" in the BG MOC but only change "location" in the main MOC, you will see that the "title" gets merged from the BG MOC into the main MOC as expected.
However, as you note in your question, the "location" appears to not get merged. In actuality, it does get merged, but any local change will override what's in the store... and this is exactly what you want to happen because the user likely made that change, and does not want it to be changed behind their back.
Basically, any pending local changes will override changes from the to-be-merged-MOC.
If you want something different, you have to implement that behavior when you do the merge, like this...
- (void)mergeBGChangesToMain:(NSNotification*)note {
NSMutableSet *updatedObjectIDs = [NSMutableSet set];
for (NSManagedObject *obj in [note.userInfo objectForKey:NSUpdatedObjectsKey]) {
[updatedObjectIDs addObject:[obj objectID]];
}
[_mainMOC performBlock:^{
for (NSManagedObject *obj in [_mainMOC updatedObjects]) {
if ([updatedObjectIDs containsObject:obj.objectID]) {
[_mainMOC refreshObject:obj mergeChanges:NO];
}
}
[_mainMOC mergeChangesFromContextDidSaveNotification:note];
}];
}
That code first collects the ObjectIDs of each object that was updated in the merged-from-MOC.
Prior to doing the merge, we then look at each of the updated objects in the merge-to-MOC. If we are merging an object into our MOC, and our merge-to-MOC has also changed that object, then we want to allow the values in the merged-from-MOC to override those in the merged-to-MOC. Thus, we refresh the local object from the store, basically discarding any local changes (there are side effects, e.g., causing the object to become a fault, releasing references to any relationships, and releasing any transient properties - see documentation of refreshObject:mergeChanges:).
Consider the following category, which addresses your situation, and a common problem when using observers like NSFetchedResultsController.
#interface NSManagedObjectContext (WJHMerging)
- (void)mergeChangesIntoContext:(NSManagedObjectContext*)moc
withDidSaveNotification:(NSNotification*)notification
faultUpdatedObjects:(BOOL)faultUpdatedObjects
overrideLocalChanges:(BOOL)overrideLocalChanges
completion:(void(^)())completionBlock;
#end
#implementation NSManagedObjectContext (WJHMerging)
- (void)mergeChangesIntoContext:(NSManagedObjectContext *)moc
withDidSaveNotification:(NSNotification *)notification
faultUpdatedObjects:(BOOL)faultUpdatedObjects
overrideLocalChanges:(BOOL)overrideLocalChanges
completion:(void (^)())completionBlock {
NSAssert(self == notification.object, #"Not called with");
NSSet *updatedObjects = notification.userInfo[NSUpdatedObjectsKey];
NSMutableSet *updatedObjectIDs = nil;
if (overrideLocalChanges || faultUpdatedObjects) {
updatedObjectIDs = [NSMutableSet setWithCapacity:updatedObjects.count];
for (NSManagedObject *obj in updatedObjects) {
[updatedObjectIDs addObject:[obj objectID]];
}
}
[moc performBlock:^{
if (overrideLocalChanges) {
for (NSManagedObject *obj in [moc updatedObjects]) {
if ([updatedObjectIDs containsObject:obj.objectID]) {
[moc refreshObject:obj mergeChanges:NO];
}
}
}
if (faultUpdatedObjects) {
for (NSManagedObjectID *objectID in updatedObjectIDs) {
[[moc objectWithID:objectID] willAccessValueForKey:nil];
}
}
[moc mergeChangesFromContextDidSaveNotification:notification];
if (completionBlock) {
completionBlock();
}
}];
}
#end

New thread + NSManagedObjectContext

I'm trying to separate my application work when there is a bigger work to do to optimize performance. My problem is about a NSManagedObjectContext used in another thread than the main one.
I'm calling:
[NSThread detachNewThreadSelector:#selector(test:) toTarget:self withObject:myObject];
On the test method there are some stuff to do and I have a problem here:
NSArray *fetchResults = [moc
executeFetchRequest:request
error:&error];
Here is my test method:
-(void) test:(MyObject *)myObject{
#autoreleasepool {
//Mycode
}
}
The second time I call the test method, my new thread is blocked when the executeFetchRequest is called.
This problem arrived when my test method is called more than one time in succession. I think the problem comes from the moc but I can't really understand why.
Edit:
With #Charlie's method it's almost working. Here is my code to save my NSManagedObjectContext (object created on my new thread).
- (void) saveContext:(NSManagedObjectContext *) moc{
NSError *error = nil;
if ([moc hasChanges] && ![moc save:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
}
}
This method is called on the new thread. My problem now is that with this save, I have a deadlock and I don't really understand why. Without it's perfectly working.
Edit2
I'm working on this issue but I still can't fix it. I changed my code about the detachNewThreadSelector. Here is my new code:
NSManagedObjectContext* context = [[NSManagedObjectContext alloc]
initWithConcurrencyType:NSPrivateQueueConcurrencyType];
context.persistentStoreCoordinator = self.persistentStoreCoordinator;
context.undoManager = nil;
[context performBlock:^
{
CCImages* cachedImage;
NSManagedObjectContext *childContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
childContext.parentContext = context;
cachedImage=[CCImages getCCImageForKey:path inManagedObjectContext:childContext];
UIImage *image = [self getImageFromCacheWithPath:path andCachedImage:cachedImage atDate:now];
if (image != nil){
if(![weakSelf.delegate respondsToSelector:#selector(CacheCacheDidLoadImageFromCache:)])
[weakSelf setDelegate:appDelegate.callbacksCollector];
//[weakSelf useCallbackCollectorForDelegate:weakSelf inMethod:#"initPaginatorForListMoments"];
[weakSelf.delegate CacheCacheDidLoadImageFromCache:image];
}
}
- (UIImage*) getImageFromCacheWithPath:(NSString*) path andCachedImage:(CCImages *) cachedImage atDate: (NSDate *) now{
NSURL* localURL=[NSURL URLWithString:cachedImage.path relativeToURL:[self imageCacheDirectory]];
UIImage * image;
//restore uiimage from local file system
if (localURL) {
image=[UIImage imageWithContentsOfFile:[localURL path]];
//update cache
[cachedImage setLastAccessedAt:now];
[self saveContext];
if(image)
return image;
}
return nil;
}
Just after that, I'm saving my contexts (manually for now)
[childContext performBlock:^{
NSError *error = nil;
if (![childContext save:&error]) {
DDLogError(#"Error during context saving when getting image from cache : %#",[error description]);
}
else{
[context performBlock:^{
NSError *error = nil;
if (![context save:&error]) {
DDLogError(#"Error during context saving when getting image from cache : %#",[error description]);
}
}];
}
}];
There is a strange problem. My call back method is called without any problem on my controller (which implements the CacheCacheDidLoadImageFromCache: method). On this method I attest the reception of the image (DDLogInfo) and say that I want my spinner to stop. It does not directly but only 15secondes after the callback method was called.
My main problem is that my context (I guess) is still loading my image from the cache while it was already found. I said 'already' because the callback method has been called and the image was present. There is no suspicious activity of the CPU or of the memory. Instruments didn't find any leak.
I'm pretty sure that I'm using wrongly the NSManagedObjectContext but I can't find where.
You are using the old concurrency model of thread confinement, and violating it's rules (as described in the Core Data Concurrency Guide, which has not been updated yet for queue confinement). Specifically, you are trying to use an NSManagedObjectContext or NSManagedObject between multiple threads.
This is bad.
Thread confinement should not be used for new code, only to maintain the compatibility of old code while it's being migrated to queue confinement. This does not seem to apply to you.
To use queue confinement to solve your problem, first you should create a context attached to your persistent store coordinator. This will serve as the parent for all other contexts:
+ (NSManagedObjectContent *) parentContextWithPersistentStoreCoordinator:(NSPersistentStoreCoordinator *)coordinator {
NSManagedObjectContext *result = nil;
result = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[result setPersistentStoreCoordinator:coordinator];
return result;
}
Next, you want the ability to create child managed object contexts. You will use these to perform work on the data, wether reading or writing. An NSManagedObjectContext is a scratchpad of the work you are doing. You can think of it as a transaction. For example, if you're updating the store from a detail view controller you would create a new child context. Or if you were performing a multi-step import of a large data set, you would create a child for each step.
This will create a new child context from a parent:
+ (NSManagedObjectContext *) childContextWithParent:(NSManagedObjectContext *)parent {
NSManagedObjectContext *result = nil;
result = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[result setParent:parent];
return result;
}
Now you have a parent context, and you can create child contexts to perform work. To perform work on a context, you must wrap that work in performBlock: to execute it on the context's queue. I do not recommend using performBlockAndWait:. That is intended only for re-rentrant methods, and does not provide an autorelease pool or processing of user events (user events are what drives nearly all of Core Data, so they're important. performBlockAndWait: is an easy way to introduce bugs).
Instead of performBlockAndWait: for your example above, create a method that takes a block to process the results of your fetch. The fetch, and the block, will run from the context's queue - the threading is done for you by Core Data:
- (void) doThingWithFetchResults:(void (^)(NSArray *results, NSError *error))resultsHandler{
if (resultsHandler != nil){
[[self context] performBlock:^{
NSArray *fetchResults = [[self context] executeFetchRequest:request error:&error];
resultsHandler(fetchResults, error);
}];
}
}
Which you would call like this:
[self doThingsWithFetchResults:^(NSArray *something, NSError *error){
if ([something count] > 0){
// Do stuff with your array of managed objects
} else {
// Handle the error
}
}];
That said, always prefer using an NSFetchedResultsController over using executeFetch:. There seems to be a belief that NSFetchedResultsController is for powering table views or that it can only be used from the main thread or queue. This is not true. A fetched results controller can be used with a private queue context as shown above, it does not require a main queue context. The delegate callbacks the fetched results controller emits will come from whatever queue it's context is using, so UIKit calls need to be made on the main queue inside your delegate method implementations. The one issue with using a fetched results controller this way is that caching does not work due to a bug.
Again, always prefer the higher level NSFetchedResultsController to executeFetch:.
When you save a context using queue confinement you are only saving that context, and the save will push the changes in that context to it's parent. To save to the store you must recursively save all the way. This is easy to do. Save the current context, then call save on the parent as well. Doing this recursively will save all the way to the store - the context that has no parent context.
Example:
- (void) saveContextAllTheWayBaby:(NSManagedObjectContext *)context {
[context performBlock:^{
NSError *error = nil;
if (![context save:&error]){
// Handle the error appropriately.
} else {
[self saveContextAllTheWayBaby:[context parentContext]];
}
}];
}
You do not, and should not, use merge notifications and mergeChangesFromContextDidSaveNotification: with queue confinement. mergeChangesFromContextDidSaveNotification: is a mechanism for the thread confinement model that is replaced by the parent-child context model. Using it can cause a whole slew of problems.
Following the examples above you should be able to abandon thread confinement and all of the issues that come with it. The problems you are seeing with your current implementation are only the tip of the iceberg.
There are a number of Core Data sessions from the past several years of WWDC that may also be of help. The 2012 WWDC Session "Core Data Best Practices" should be of particular interest.
if you want to use managed object context in background thread, there are two approaches,
1 Create a new context set concurrency type to NSPrivateQueueConcurrencyType and set the parentContext to main thread context
2 Create a new context set concurrency type to NSPrivateQueueConcurrencyType and set persistentStoreCoordinator to main thread persistentStoreCoordinator
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void) {
NSManagedObjectContext *privateContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
privateContext.persistentStoreCoordinator = mainManagedObjectContext.persistentStoreCoordinator;
[[NSNotificationCenter defaultCenter] addObserverForName:NSManagedObjectContextDidSaveNotification object:nil queue:nil usingBlock:^(NSNotification* note) {
NSManagedObjectContext *moc = mainManagedObjectContext;
if (note.object != moc) {
[moc mergeChangesFromContextDidSaveNotification:note];
}
}];
// do work here
// remember managed object is not thread save, so you need to reload the object in private context
});
before exist the thread, make sure remove the observer, bad thing can happen if you don't
for more details read http://www.objc.io/issue-2/common-background-practices.html

iOS, Core Data. Very slow with using of multithread

My code looks like something like this:
dispatch_async(background_save_queue, ^{
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] init];
[context setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];
[context setPersistentStoreCoordinator:coordinator];
[[NSNotificationCenter defaultCenter] addObserver:mainContext selector:#selector(mergeChangesFromContextDidSaveNotification:) name:NSManagedObjectContextDidSaveNotification object:context];
//code to create objects with core data in context ("context" variable)
[context save:nil];
[context release];
}
And the similar code without multithread (with one context only) works much better.
Is my code wrong and are there are other examples of filling core data objects with multithreading?
Finally I checked that another thread is not slower than the main one. The problem is there are a lot of records in a data source and I need to perform a check for each record if this record already written in database. So the speed of my database creation process depends greatly on a count of the records and a database structure.

CoreData deadlock with multiple threads

I'm experiencing the same deadlock issue (that is quite common on SO) that occurs in the multiple NSManagedObjectContexts & multiple threads scenario. In some of my view controllers, my app uses background threads to get data from a web service, and in that same thread it saves it. In others, where it makes sense to not progress any further without saving (e.g. persist values from a form when they hit "Next"), the save is done on the main thread. AFAIK there should be nothing wrong with this in theory, but occasionally I can make the deadlock happen on a call to
if (![moc save:&error])
...and this seems to be always on the background thread's save when the deadlock occurs. It doesn't happen on every call; in fact it's quite the opposite, I have to use my app for a couple of minutes and then it'll happen.
I've read all the posts I could find as well as the Apple docs etc, and I'm sure I'm following the recommendations. To be specific, my understanding of working with multiple MOCs/threads boils down to:
Each thread must have its own MOC.
A thread's MOC must be created on that thread (not passed from one thread to another).
A NSManagedObject cannot be passed, but a NSManagedObjectID can, and you use the ID to inflate a NSManagedObject using a different MOC.
Changes from one MOC must be merged to another if they are both using the same PersistentStoreCoordinator.
A while back I came across some code for a MOC helper class on this SO thread and found that it was easily understandable and quite convenient to use, so all my MOC interaction is now thru that. Here is my ManagedObjectContextHelper class in its entirety:
#import "ManagedObjectContextHelper.h"
#implementation ManagedObjectContextHelper
+(void)initialize {
[[NSNotificationCenter defaultCenter] addObserver:[self class]
selector:#selector(threadExit:)
name:NSThreadWillExitNotification
object:nil];
}
+(void)threadExit:(NSNotification *)aNotification {
TDAppDelegate *delegate = (TDAppDelegate *)[[UIApplication sharedApplication] delegate];
NSString *threadKey = [NSString stringWithFormat:#"%p", [NSThread currentThread]];
NSMutableDictionary *managedObjectContexts = delegate.managedObjectContexts;
[managedObjectContexts removeObjectForKey:threadKey];
}
+(NSManagedObjectContext *)managedObjectContext {
TDAppDelegate *delegate = (TDAppDelegate *)[[UIApplication sharedApplication] delegate];
NSManagedObjectContext *moc = delegate.managedObjectContext;
NSThread *thread = [NSThread currentThread];
if ([thread isMainThread]) {
[moc setMergePolicy:NSErrorMergePolicy];
return moc;
}
// a key to cache the context for the given thread
NSString *threadKey = [NSString stringWithFormat:#"%p", thread];
// delegate.managedObjectContexts is a mutable dictionary in the app delegate
NSMutableDictionary *managedObjectContexts = delegate.managedObjectContexts;
if ( [managedObjectContexts objectForKey:threadKey] == nil ) {
// create a context for this thread
NSManagedObjectContext *threadContext = [[NSManagedObjectContext alloc] init];
[threadContext setPersistentStoreCoordinator:[moc persistentStoreCoordinator]];
[threadContext setMergePolicy:NSErrorMergePolicy];
// cache the context for this thread
NSLog(#"Adding a new thread:%#", threadKey);
[managedObjectContexts setObject:threadContext forKey:threadKey];
}
return [managedObjectContexts objectForKey:threadKey];
}
+(void)commit {
// get the moc for this thread
NSManagedObjectContext *moc = [self managedObjectContext];
NSThread *thread = [NSThread currentThread];
if ([thread isMainThread] == NO) {
// only observe notifications other than the main thread
[[NSNotificationCenter defaultCenter] addObserver:[self class] selector:#selector(contextDidSave:)
name:NSManagedObjectContextDidSaveNotification
object:moc];
}
NSError *error;
if (![moc save:&error]) {
NSLog(#"Failure is happening on %# thread",[thread isMainThread]?#"main":#"other");
NSArray* detailedErrors = [[error userInfo] objectForKey:NSDetailedErrorsKey];
if(detailedErrors != nil && [detailedErrors count] > 0) {
for(NSError* detailedError in detailedErrors) {
NSLog(#" DetailedError: %#", [detailedError userInfo]);
}
}
NSLog(#" %#", [error userInfo]);
}
if ([thread isMainThread] == NO) {
[[NSNotificationCenter defaultCenter] removeObserver:[self class] name:NSManagedObjectContextDidSaveNotification
object:moc];
}
}
+(void)contextDidSave:(NSNotification*)saveNotification {
TDAppDelegate *delegate = (TDAppDelegate *)[[UIApplication sharedApplication] delegate];
NSManagedObjectContext *moc = delegate.managedObjectContext;
[moc performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:)
withObject:saveNotification
waitUntilDone:NO];
}
#end
Here's a snippet of the multi-threaded bit where it seems to deadlock:
NSManagedObjectID *parentObjectID = [parent objectID];
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0ul);
dispatch_async(queue, ^{
// GET BACKGROUND MOC
NSManagedObjectContext *backgroundContext = [ManagedObjectContextHelper managedObjectContext];
Parent *backgroundParent = (Parent*)[backgroundContext objectWithID:parentObjectID];
// HIT THE WEBSERVICE AND PUT THE RESULTS IN THE PARENT OBJECT AND ITS CHILDREN, THEN SAVE...
[ManagedObjectContextHelper commit];
dispatch_sync(dispatch_get_main_queue(), ^{
NSManagedObjectContext *mainManagedObjectContext = [ManagedObjectContextHelper managedObjectContext];
parent = (Parent*)[mainManagedObjectContext objectWithID:parentObjectID];
});
});
The conflictList in the error seems to suggest that it's something to do with the ObjectID of the parent object:
conflictList = (
"NSMergeConflict (0x856b130) for NSManagedObject (0x93a60e0) with objectID '0xb07a6c0 <x-coredata://B7371EA1-2532-4D2B-8F3A-E09B56CC04F3/Child/p4>'
with oldVersion = 21 and newVersion = 22
and old object snapshot = {\n parent = \"0xb192280 <x-coredata://B7371EA1-2532-4D2B-8F3A-E09B56CC04F3/Parent/p3>\";\n name = \"New Child\";\n returnedChildId = 337046373;\n time = 38;\n}
and new cached row = {\n parent = \"0x856b000 <x-coredata://B7371EA1-2532-4D2B-8F3A-E09B56CC04F3/Parent/p3>\";\n name = \"New Child\";\n returnedChildId = 337046373;\n time = 38;\n}"
);
I've tried putting in refreshObject calls as soon as I've gotten hold of a MOC, with the theory being that if this is a MOC we've used before (e.g. we used an MOC on the main thread before and it's likely that this is the same one that the helper class will give us), then perhaps a save in another thread means that we need to explicitly refresh. But it didn't make any difference, it still deadlocks if I keep clicking long enough.
Does anyone have any ideas?
Edit: If I have a breakpoint set for All Exceptions, then the debugger pauses automatically on the if (![moc save:&error]) line, so the play/pause button is already paused and is showing the play triangle. If I disable the breakpoint for All Exceptions, then it actually logs the conflict and continues - probably because the merge policy is currently set to NSErrorMergePolicy - so I don't think it's actually deadlocking on the threads. Here's a screehshot of the state of both threads while it's paused.
I do not recommend your approach at all. First, unless you are confined to iOS4, you should be using the MOC concurrency type, and not the old method. Even under iOS 5 (which is broken for nested contexts) the performBlock approach is much more sound.
Also, note that dispatch_get_global_queue provides a concurrent queue, which can not be used for synchronization.
The details are found here: http://developer.apple.com/library/ios/#documentation/cocoa/conceptual/CoreData/Articles/cdConcurrency.html
Edit
You are trying to manage MOCs and threading manually. You can do it if you want, but there be dragons in your path. That's why the new way was created, to minimize the chance for errors in using Core Data across multiple threads. Anytime I see manual thread management with Core Data, I will always suggest to change as the first approach. That will get rid of most errors immediately.
I don't need to see much more than you manually mapping MOCs and threads to know that you are asking for trouble. Just re-read that documentation, and do it the right way (using performBlock).

Core Data saving objects in background issue

What I'm trying todo in a nutshell is I am using a background queue to save JSON objects pulled from a web service to the Core Data Sqlite3 database. The saving takes place on a serialized background queue I've created via GCD, and saved to a secondary instance of NSManagedObjectContext that is created for that background queue. Once the save is complete I need to update the instance of NSManagedObjectContext that is on the main thread with the newly created/updated objects. The problem I am having though is the instance of NSManagedObjectContext on the main thread is not able to find the objects that were saved on the background context. Below is a list of actions I'm taking with code samples. Any thoughts on what I'm doing wrong?
Create a background queue via GCD, run all pre-processing logic and then save the background context on that thread:
.
// process in the background queue
dispatch_async(backgroundQueue, ^(void){
if (savedObjectIDs.count > 0) {
[savedObjectIDs removeAllObjects];
}
if (savedObjectClass) {
savedObjectClass = nil;
}
// set the thead name
NSThread *currentThread = [NSThread currentThread];
[currentThread setName:VS_CORE_DATA_MANAGER_BACKGROUND_THREAD_NAME];
// if there is not already a background context, then create one
if (!_backgroundQueueManagedObjectContext) {
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
if (coordinator != nil) {
_backgroundQueueManagedObjectContext = [[NSManagedObjectContext alloc] init];
[_backgroundQueueManagedObjectContext setPersistentStoreCoordinator:coordinator];
}
}
// save the JSON dictionary starting at the upper most level of the key path, and return all created/updated objects in an array
NSArray *objectIds = [self saveJSON:jsonDict objectMapping:objectMapping class:managedObjectClass managedObjectContext:_backgroundQueueManagedObjectContext level:0];
// save the object IDs and the completion block to global variables so we can access them after the save
if (objectIds) {
[savedObjectIDs addObjectsFromArray:objectIds];
}
if (completion) {
saveCompletionBlock = completion;
}
if (managedObjectClass) {
savedObjectClass = managedObjectClass;
}
// save all changes object context
[self saveManagedObjectContext];
});
The "saveManagedObjectContext" method basically looks at which thread is running and saves the appropriate context. I have verified that this method is working correctly so I will not place the code here.
All of this code resides in a singleton, and in the singleton's init method I am adding a listener for the "NSManagedObjectContextDidSaveNotification" and it calls the mergeChangesFromContextDidSaveNotification: method
.
// merge changes from the context did save notification to the main context
- (void)mergeChangesFromContextDidSaveNotification:(NSNotification *)notification
{
NSThread *currentThread = [NSThread currentThread];
if ([currentThread.name isEqual:VS_CORE_DATA_MANAGER_BACKGROUND_THREAD_NAME]) {
// merge changes to the primary context, and wait for the action to complete on the main thread
[_managedObjectContext performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:) withObject:notification waitUntilDone:YES];
// on the main thread fetch all new data and call the completion block
dispatch_async(dispatch_get_main_queue(), ^{
// get objects from the database
NSMutableArray *objects = [[NSMutableArray alloc] init];
for (id objectID in savedObjectIDs) {
NSError *error;
id object = [_managedObjectContext existingObjectWithID:objectID error:&error];
if (error) {
[self logError:error];
} else if (object) {
[objects addObject:object];
}
}
// remove all saved object IDs from the array
[savedObjectIDs removeAllObjects];
savedObjectClass = nil;
// call the completion block
//completion(objects);
saveCompletionBlock(objects);
// clear the saved completion block
saveCompletionBlock = nil;
});
}
}
As you can see in the method above I am calling the "mergeChangesFromContextDidSaveNotification:" on the main thread, and I have set the action to wait until done. According to the apple documentation the background thread should wait until that action is complete before it continues with the rest of the code below that call. As I mentioned above once I run this code everything seems to work, but when I try to print out the fetched objects to the console I don't get anything back. It seems that the merge is not in fact taking place, or possibly not finishing before the rest of my code runs. Is there another notification that I should be listening for to ensure that the merge has completed? Or do I need to save the main object context after the merge, but before the fecth?
Also, I apologize for the bad code formatting, but it seems that SO's code tags don't like method definitions.
Thanks guys!
UPDATE:
I've made the changes that were recommended below, but still having the same problem. Below is the updated code I have.
This is the code that invokes the background thread saving processes
// process in the background queue
dispatch_async(backgroundQueue, ^(void){
if (savedObjectIDs.count > 0) {
[savedObjectIDs removeAllObjects];
}
if (savedObjectClass) {
savedObjectClass = nil;
}
// set the thead name
NSThread *currentThread = [NSThread currentThread];
[currentThread setName:VS_CORE_DATA_MANAGER_BACKGROUND_THREAD_NAME];
// if there is not already a background context, then create one
if (!_backgroundQueueManagedObjectContext) {
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
if (coordinator != nil) {
_backgroundQueueManagedObjectContext = [[NSManagedObjectContext alloc] init];
[_backgroundQueueManagedObjectContext setPersistentStoreCoordinator:coordinator];
}
}
// save the JSON dictionary starting at the upper most level of the key path
NSArray *objectIds = [self saveJSON:jsonDict objectMapping:objectMapping class:managedObjectClass managedObjectContext:_backgroundQueueManagedObjectContext level:0];
// save the object IDs and the completion block to global variables so we can access them after the save
if (objectIds) {
[savedObjectIDs addObjectsFromArray:objectIds];
}
if (completion) {
saveCompletionBlock = completion;
}
if (managedObjectClass) {
savedObjectClass = managedObjectClass;
}
// listen for the merge changes from context did save notification
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(mergeChangesFromBackground:) name:NSManagedObjectContextDidSaveNotification object:_backgroundQueueManagedObjectContext];
// save all changes object context
[self saveManagedObjectContext];
});
This is the code that is called with by the NSManagedObjectContextDidSaveNotification notification
// merge changes from the context did save notification to the main context
- (void)mergeChangesFromBackground:(NSNotification *)notification
{
// kill the listener
[[NSNotificationCenter defaultCenter] removeObserver:self name:NSManagedObjectContextDidSaveNotification object:_backgroundQueueManagedObjectContext];
NSThread *currentThread = [NSThread currentThread];
// merge changes to the primary context, and wait for the action to complete on the main thread
[[self managedObjectContext] performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:) withObject:notification waitUntilDone:YES];
// dispatch the completion block
dispatch_async(dispatch_get_main_queue(), ^{
// get objects from the database
NSMutableArray *objects = [[NSMutableArray alloc] init];
for (id objectID in savedObjectIDs) {
NSError *error;
id object = [[self managedObjectContext] existingObjectWithID:objectID error:&error];
if (error) {
[self logError:error];
} else if (object) {
[objects addObject:object];
}
}
// remove all saved object IDs from the array
[savedObjectIDs removeAllObjects];
savedObjectClass = nil;
// call the completion block
//completion(objects);
saveCompletionBlock(objects);
// clear the saved completion block
saveCompletionBlock = nil;
});
}
UPDATE:
So I found the solution. Turns out that the way I was saving out the object IDs on the background thread and then trying to use them on the main thread to re-fetch them wasn't working out. So I ended up pulling the inserted/updated objects from the userInfo dictionary that is sent with the NSManagedObjectContextDidSaveNotification notification. Below is my updated code that is now working.
As before this code starts the pre-prossesing and saving logic
// process in the background queue
dispatch_async(backgroundQueue, ^(void){
// set the thead name
NSThread *currentThread = [NSThread currentThread];
[currentThread setName:VS_CORE_DATA_MANAGER_BACKGROUND_THREAD_NAME];
[self logMessage:[NSString stringWithFormat:#"(%#) saveJSONObjects:objectMapping:class:completion:", [managedObjectClass description]]];
// if there is not already a background context, then create one
if (!_backgroundQueueManagedObjectContext) {
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
if (coordinator != nil) {
_backgroundQueueManagedObjectContext = [[NSManagedObjectContext alloc] init];
[_backgroundQueueManagedObjectContext setPersistentStoreCoordinator:coordinator];
}
}
// save the JSON dictionary starting at the upper most level of the key path
[self saveJSON:jsonDict objectMapping:objectMapping class:managedObjectClass managedObjectContext:_backgroundQueueManagedObjectContext level:0];
// save the object IDs and the completion block to global variables so we can access them after the save
if (completion) {
saveCompletionBlock = completion;
}
// listen for the merge changes from context did save notification
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(mergeChangesFromBackground:) name:NSManagedObjectContextDidSaveNotification object:_backgroundQueueManagedObjectContext];
// save all changes object context
[self saveManagedObjectContext];
});
This is the modified method that handles the NSManagedObjectContextDidSaveNotification
- (void)mergeChangesFromBackground:(NSNotification *)notification
{
// kill the listener
[[NSNotificationCenter defaultCenter] removeObserver:self name:NSManagedObjectContextDidSaveNotification object:_backgroundQueueManagedObjectContext];
// merge changes to the primary context, and wait for the action to complete on the main thread
[[self managedObjectContext] performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:) withObject:notification waitUntilDone:YES];
// dispatch the completion block
dispatch_async(dispatch_get_main_queue(), ^{
// pull the objects that were saved from the notification so we can get them on the main thread MOC
NSDictionary *userInfo = [notification userInfo];
NSMutableArray *modifiedObjects = [[NSMutableArray alloc] init];
NSSet *insertedObject = (NSSet *)[userInfo objectForKey:#"inserted"];
NSSet *updatedObject = (NSSet *)[userInfo objectForKey:#"updated"];
if (insertedObject && insertedObject.count > 0) {
[modifiedObjects addObjectsFromArray:[insertedObject allObjects]];
}
if (updatedObject && updatedObject.count > 0) {
[modifiedObjects addObjectsFromArray:[updatedObject allObjects]];
}
NSMutableArray *objects = [[NSMutableArray alloc] init];
// iterate through the updated objects and find them in the main thread MOC
for (NSManagedObject *object in modifiedObjects) {
NSError *error;
NSManagedObject *obj = [[self managedObjectContext] existingObjectWithID:object.objectID error:&error];
if (error) {
[self logError:error];
}
if (obj) {
[objects addObject:obj];
}
}
modifiedObjects = nil;
// call the completion block
saveCompletionBlock(objects);
// clear the saved completion block
saveCompletionBlock = nil;
});
}
I'm going to throw this out there. Stop following the best practices for concurrency listed in the Core Data Programming Guide. Apple has not updated it since adding nested contexts which are MUCH easier to use. This video goes into full detail: https://developer.apple.com/videos/wwdc/2012/?id=214
Setup your primary context to use your main thread (appropriate for handling UI):
NSManagedObjectContext * context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
[context setPersistentStoreCoordinator:yourPSC];
For any object you create that may be doing concurrent operations, create a private queue context to use
NSManagedObjectContext * backgroundContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[backgroundContext setParentContext:context];
//Use backgroundContext to insert/update...
//Then just save the context, it will automatically sync to your primary context
[backgroundContext save:nil];
The QueueConcurrencyType refers to the queue the context will do it's fetch (save and fetch request) operations on. The NSMainQueueConcurrencyType context does all it's work on the main queue, which makes it appropriate for UI interaction. A NSPrivateQueueConcurrencyType does it on it's own private queue. So when you call save on the backgroundContext, it merges it's private data calling the parentContext using performBlock as appropriate automatically. You don't want to call performBlock on the private queue context in case it happens to be on the main thread which will cause a deadlock.
If you want to get really fancy, You can create a primary context as a private queue concurrency type (which is appropriate for background saving) with a main queue context for just your UI and then child contexts of your main queue context for background operations (like imports).
I see you've worked out an answer that works for you. But I have been having some similar issues and wanted to share my experience and see if it is at all helpful to you or others looking at this situation.
Multi-threaded Core Data stuff is always a little confusing to read, so please excuse me if I misread your code. But it appears that there could be a simpler answer for you.
The core issue you had in the first attempt is that you saved off managed object IDs (supposedly the object identifiers that can be passed between threads) to a global variable for use on the main thread. You did this on a background thread. The problem was that you did this BEFORE saving to the background thread's managed object context. Object IDs are not safe to pass to another thread/context pair prior to a save. They can change when you save. See the warning in the documentation of objectID: NSManagedObject reference
You fixed this by notifying your background thread of the save, and inside that thread, grabbing the now-safe-to-use-because-the-context-has-been-saved object IDs from the notification object. These were passed to the main thread, and the actual changes were also merged into the main thread with the call to mergeChangesFromContextDidSaveNotification. Here's where you might save a step or two.
You are registering to hear the NSManagedObjectContextDidSaveNotification on the background thread. You can register to hear that same notification on the main thread instead. And in that notification you will have the same object IDs that are safe to use on the main thread. The main thread MOC can be safely updated using mergeChangesFromContextDidSaveNotification and the passed notification object, since the method is designed to work this way: mergeChanges docs. Calling your completion block from either thread is now safe as long as you match the moc to the thread the completion block is called on.
So you can do all your main thread updating stuff on the main thread, cleanly separating the threads and avoiding having to pack and repack the updated stuff or doing a double save of the same changes to the persistent store.
To be clear - the Merge that happens is on the managed object contextand its in-memory state - the moc on the main thread is updated to match the one on the background thread, but a new save isn't necessary since you ALREADY saved these changes to the store on the background thread. You have thread safe access to any of those updated objects in the notification object, just as you did when you used it on the background thread.
I hope your solution is working for you and you don't have to re-factor - but wanted to add my thoughts for others who might see this. Please let me know if I've misinterpreted your code and I'll amend.
in your case because your writing to the background moc the notification for mergeChangesFromContextDidSaveNotification will come in on the background moc, not the foreground moc.
so you'll need to register for notifications on the background thread coming to the background moc object.
when you receive that call you can send a message to the main thread moc to mergeChangesFromContextDidSaveNotification.
andrew
update:
here's a sample that should work
//register for this on the background thread
NSNotificationCenter *nc = [NSNotificationCenter defaultCenter];
[nc addObserver:self selector:#selector(mergeChanges:) name:NSManagedObjectContextDidSaveNotification object:backgroundMOC];
- (void)mergeChanges:(NSNotification *)notification {
NSManagedObjectContext *mainThreadMOC = [singleton managedObjectContext];
//this tells the main thread moc to run on the main thread, and merge in the changes there
[mainThreadMOC performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:) withObject:notification waitUntilDone:YES];
}

Resources