I'm trying to use core data in a multi thread way.
I simply want to show the application with the previously downloaded data while downloading new data in background.
This should let the user access the application during update process.
I have a NSURLConnection which download the file asyncronously using delegate (and showing the progress), then i use an XMLParser to parse the new data and create new NSManagedObjects in a separate context, with its own persistentStore and using a separate thread.
The problem is that creating new objects in the same context of the old one while showing it can throws BAD_INSTRUCTION exception.
So, I decided to use a separate context for the new data, but I can't figure out a way to move all the objects to the other context once finished.
Paolo aka SlowTree
The Apple Concurrency with Core Data documentation is the place to start. Read it really carefully... I was bitten many times by my misunderstandings!
Basic rules are:
Use one NSPersistentStoreCoordinator per program. You don't need them per thread.
Create one NSManagedObjectContext per thread.
Never pass an NSManagedObject on a thread to the other thread.
Instead, get the object IDs via -objectID and pass it to the other thread.
More rules:
Make sure you save the object into the store before getting the object ID. Until saved, they're temporary, and you can't access them from another thread.
And beware of the merge policies if you make changes to the managed objects from more than one thread.
NSManagedObjectContext's -mergeChangesFromContextDidSaveNotification: is helpful.
But let me repeat, please read the document carefully! It's really worth it!
Currently [May 2015] the Apple Concurrency with Core Data documentation is, at best, very misleading as it doesn't cover any of the enhancements in iOS 5 and hence no longer shows the best ways to use core data concurrently. There are two very important changes in iOS 5 - parent contexts and new concurrency/threading types.
I have not yet found any written documentation that comprehensively covers these new features, but the WWDC 2012 video "Session 214 - Core Data Best Practices" does explain it all very well.
Magical Record uses these new features and may be worth a look.
The real basics are still the same - you can still only use managed objects the thread their managed object context was created on.
You can now use [moc performBlock:] to run code on the right thread.
There's no need to use mergeChangesFromContextDidSaveNotification: anymore; instead create a child context to make the changes, then save the child context. Saving the child context will automatically push the changes up into the parent context, and to save the changes to disk just perform a save on the parent context in it's thread.
For this to work you must create the parent context with a concurrent type, eg:
mainManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
Then on the background thread:
context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
[context setParentContext:mainManagedObjectContext];
<... perform actions on context ...>
NSError *error;
if (![context save:&error])
{
<... handle error ...>
}
[mainManagedObjectContext performBlock:^{
NSError *e = nil;
if (![mainContext save:&e])
{
<... handle error ...>
}
}];
I hope this can help all the peoples that meet problems using core data in a multithread environment.
Take a look at "Top Songs 2" in apple documentation. With this code i took the "red pill" of Matrix, and discovered a new world, without double free error, and without faults. :D
Hope this helps.
Paolo
p.s.
Many thanks to Yuji, in the documentation you described above I found that example.
Related
I have been reading about merge policies and it gives me conflicting ideas about when changes are merged.
I am having two contexts - one in background queue and one in main queue. Both have policies defined NSOverwriteMergePolicy which I think is outright wrong. I am already having problems in sync. I often witness that in background sync from server data my NSManagedObjects are often outdated, and I end up saving them, causing loss of data.
Is there any place I could visit all the rules & order of precedence for context save, context refresh with respect to overriding policies?
I have seen all the documentation around merge policies but I am confused whether they take effect upon SAVE or REFRESH.
Also, up to some extent, this is very confusing. For example, Apple Docs state this for NSMergeByPropertyObjectTrumpMergePolicy:
A policy that merges conflicts between the persistent store's version
of the object and the current in-memory version by individual
property, with the external changes trumping in-memory changes.
The merge occurs by individual property.
For properties that have been changed in both the external source and in memory, the in-memory
changes trump the external ones.
How to ensure that my desired properties get modified / remain unaffected when I choose to modify / not modify them on different contexts?
TL:DR: Merge policy is evil. You should only write to core-data in one synchronous way.
Full explanation: If an object is changed at the same time from two different contexts core-data doesn't know what to do. You can set a mergePolicy to set which change should win, but that really isn't a good solution, because you will lose data that way. The way that a lot of pros have been dealing with the problem for a long time was to have an operation queue to queue the writes so there is only one write going on at a time, and have another context on the main thread only for reads. This way you never get any merge conflicts. (see https://vimeo.com/89370886 for a great explanation on this setup).
Making this setup with NSPersistentContainer is very easy. In your core-data manager create a NSOperationQueue
_persistentContainerQueue = [[NSOperationQueue alloc] init];
_persistentContainerQueue.maxConcurrentOperationCount = 1;
And do all writing using this queue:
- (void)enqueueCoreDataBlock:(void (^)(NSManagedObjectContext* context))block{
void (^blockCopy)(NSManagedObjectContext*) = [block copy];
[self.persistentContainerQueue addOperation:[NSBlockOperation blockOperationWithBlock:^{
NSManagedObjectContext* context = self.persistentContainer.newBackgroundContext;
[context performBlockAndWait:^{
blockCopy(context);
[context save:NULL]; //Don't just pass NULL here. look at the error and log it to your analytics service
}];
}]];
}
When you call enqueueCoreDataBlock the block is enqueued to ensures that there are no merge conflicts. But if you write to the viewContext that would defeat this setup. Likewise you should treat any other contexts that you create (with newBackgroundContext or with performBackgroundTask) as readonly because they will also be outside of the writing queue.
sometimes I need to update an already persistend managed object with another temporary managed object created from a server response. The temporary object must be discarded and the other one should be saved immediately after the update operation. In MagicalRecord (MR) 2.3++ it is recommended to save objects like so:
- (void)updateObject:(NSManagedObject*)alreadyPersistedObject withDictionary:(NSDictionary*)dictionary {
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext) {
NSManagedObject *updatedObject = [NSManagedObject MR_createEntityInContext:localContext];
[MyParserHandler parseDictionary:dictionary intoManagedObject:updatedObject];
// update some properties of alreadyPersistedObject using updatedObject
}];
}
I know that we should initialize temporary objects using temporary local contexts in earlier versions of MR. Is that also true for MR 2.3 and higher?
If YES, can somebody please give me a code example for that and needs the temp context be a child of the [NSManagedObjectContext MR_defaultContext] and how to discard this context after usage?
If NO, what is the preferred technique to do this now?
Thanks a lot for helping!
Just to restate the problem, you are trying to update existing data from an external data source into your core data store. You create a background context and have temporary objects into which you import this external data, and then save. So what you want to do is have those new changes propagate to your existing in-memory objects.
If this is the case, then if you simply use the body of that method, the default context provided by MagicalRecord will have those updates upon the completion of that block. MagicalRecord is trying to do much of that merging work for you. If you just create your temporary object inside the saveWithBlock: method, the save will eventually merge those changes to the default context. So, if your objects are in that context, they will receive those changes as a result of the save.
If you find that you need some more control over your merging of data, I suggest you use the built-in Core Data merging mechanisms rather than trying to merge manually. In this case, you should use the parent/child context relationships, or the NSManagedObjectContextDidSaveNotification to merge in changes from another context. The code for those solutions is fairly common on the internet.
I've learned that generally, intensive tasks should take place on background threads, as if they're on the main thread they'll block user interaction and interface updates.
Does Core Data fall under that umbrella? I received a great answer to my question about loading images asynchronously in a UITableView, but I'm curious how to then work with Core Data as the backend.
I know Apple has a Core Data Concurrency guide, but I'm curious in which cases one is supposed to use Core Data in the background.
As a quick example, say I'm making a Twitter client and want to get all the tweet information (tweet text, username, user avatar, linked images, etc.). I asynchronously download that information and receive some JSON from the Twitter API that I then parse. Should I then do a dispatch_async(dispatch_get_main_queue()...) when I add the information to Core Data?
I also then want to download the user's avatar, but I want to do that separately from presenting the tweet, so I can present the tweet quickly as possible, then present the image when ready. So I asynchronously download the image. Should I then update that Core Data item asynchronously?
Am I in a situation where I don't need multi-threaded Core Data at all? If so, when would be a situation where I need it?
I've learned that generally, intensive tasks should take place on background threads, as if they're on the main thread they'll block user interaction and interface updates.
Does Core Data fall under that umbrella?
Yes, actually.
Core Data tasks can and should - where possible - be executed on background threads or on non-main queues.
It's important to note though, that each managed object is associated to a certain execution context (a thread or a dispatch queue). Accessing a managed object MUST be executed only from within this execution context. This association comes from the fact that each managed object is registered with a certain Managed Object Context, and this managed object context is associated to a certain execution context when it is created.
See also:
[NSManagedObjectContext initWithConcurrencyType](),
[NSManagedObjectContext performBlock]
[NSManagedObjectContext performBlockAndWait]
Consequently, when displaying properties of managed objects, this involves UIKit, and since UIKit methods MUST be executed on the main thread, the managed object's context must be associated to the main thread or main queue.
Otherwise, Core Data and user code can access managed objects from arbitrary execution contexts, as long as this is the one to which the managed object is associated with.
The below picture is an Instrument Time Profile which shows quite clearly how Core Data tasks can be distributed on different threads:
The highlighted "spike" in the CPU activity shows a task which performs the following:
Load 1000 JSON objects from origin (this is just a local JSON file),
Create managed objects in batches of 100 on a private context
Merge them into the Cora Data Stack main context
Save the context (each batch)
Finally, fetch all managed objects into the main context
As we can see, only 26.4% of the CPU load will be executed on the main thread.
Core Data does indeed fall under that umbrella. Particularly for downloading and saving data, but also possibly for fetching depending on the number of objects in the data store and the predicate to be applied.
Generally, I'd push all object creation and saving which is coming from a server onto a background thread. I'd only update and save objects on the main thread if they're user generated (because it would only be one, updated slowly and infrequently).
Downloading your twitter data is a good example as there will potentially be a good amount of data to process. You should process and save the data on a background thread and save it up to the persistent store there. The persistent store should then merge the changes down to the main thread context for you (assuming you have the contexts configured nicely - use a 3rd party framework for that like MagicalRecord).
Again, for the avatar update, you're already on a background thread so you might as well stay there :-)
You might not need to use multiple threads now. If you only download the 5 most recent tweets then you might not notice the difference. But using a framework can make the multi-threading relatively easy. And using a fetched results controller can make knowing when the UI should be updated on the main thread very easy. So, it's worthwhile taking the time to understand the setup and using it.
The best way to handle behavior like this is to use multiple NSManagedObjectContexts. The "main" context you create like so:
_mainManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
_mainManagedObjectContext.persistentStoreCoordinator = [self mainPersistentStoreCoordinator];
You're going to want to do any heavy writes on a different NSManagedObjectContext to avoid blocking your UI thread as you import (which can be quite noticeable on large or frequent operations to your Main context).
To achieve this, you would do something like this:
Retrieve your data asynchronously from the network
Spin up a temporary (or perhaps a permanent) background NSManagedObjectContext and set the "parent" to the main ManagedObjectContext (so it will merge the data you import when you save)
Use the performBlock: or performBlockAndWait: APIs to write to that context on its own private queue (in the background)
Example (uses AFNetworking):
[_apiClient GET:[NSString stringWithFormat:#"/mydata"]
parameters:nil
success:^(AFHTTPRequestOperation *operation, id responseObject) {
NSError* jsonError;
id jsonResponse = [NSJSONSerialization JSONObjectWithData:operation.responseData options:kNilOptions error:&jsonError];
if (!jsonError) {
NSArray* parsedData = (NSArray*)jsonResponse;
completionBlock(parsedShares, nil);
} else {
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
context.parentContext = YOUR_MAIN_CONTEXT; // when you save this context, it will merge its changes into your main context!
// the following code will occur in a background queue for you that CoreData manages
[context performBlock:^{
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
if (![context save:&saveError]) {
NSLog(#"Error saving context: %#", saveError);
} else {
NSLog(#"Saved data import in background!");
}
}];
}
}
failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"There was an error: %#", error)
}];
Docs: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSManagedObjectContext_Class/NSManagedObjectContext.html#//apple_ref/occ/instm/NSManagedObjectContext/performBlockAndWait:
As I alluded to in your other question, it's largely going to be a matter of how long the manipulations are going to take. If the computation required doesn't have a noticeable delay, by all means be lazy about and do your core data on the main thread.
In your other example, you're requesting 20 items from twitter, parsing 20 items and sticking them into CoreData isn't going to be noticeable. The best approach here is to probably continue to just fetch 20 at a time and update in the foreground as each chunk becomes available.
Downloading all items from twitter in one request will take a significant amount of time and computation and it's probably worth creating a separate ManagedObjectModel and synchronizing it with the foreground model. Since you really have one-way data (it always flows twitter->core data->user interface) the likelihood of clashes is minimal, so you can easily use NSManagedObjectContextDidSaveNotification and mergeChangesFromContextDidSaveNotification:
Last year I used RestKit 0.10 to seamlessly download and save core data objects in background. However, when I tried to use restkit in 2013, I noticed that they have taken out the ActiveRecord pattern, which I relied upon to abstract away all the unpleasantness of background saving.
I found that the ActiveRecord pattern exists in MagicalRecord framework, but most of the documentation I could find is for version 2.x, while my cocoapods install 3.x.
I spent the last 2 hours searching, and find a lot of answers that are really out of date and no longer work for these new frameworks.
This poses the question: what the standard/easiest way to deal with saving core data objects in background using frameworks available in 2013? Should I try some other framework?
If you don't use any external library like Magical Record or RestKit, but simply go for the all manual stuff, you can take advantage of the new NSManagedObjectContext APIs.
You can now have contexts nested with a parent-child relationship and you can also tell each context to perform a block in it's own thread. My advice, therefore is to have the following structure for your application:
1) A background saving context. This will be the only context that saves and reads data directly to/from the database.
2) A context initalized on the main thread that will be your point of access for everything you need to do in the application, especially updating the UI. This context will be a child of the saving context.
3) As needed, you'll create background contextes that perform work on background threads, e.g. loading data from the network and serialize this data in NSManagedObject instances. This contextes will be children of the main context.
4) Every time you call -[NSManagedObjectContext save:] on a context, you should also call the same method on it's parentContext. To do this you could have a convenience method in a category on NSManagedObjectContext that reads something like this:
- (void)saveSelfAndParent {
[self save:NULL];
[self.parentContext performBlock:^{
[self.parentContext saveSelfAndParent];
}];
}
This is already a thread safe configuration and your changes will propagate the changes up to the database. Note that as the saving context will have no parent (and thus self.parentContext will be nil), the performBlock: won't crash the app.
Here's an example what you need to do to create a new entity assuming you kick off your background work using Grand Central Dispatch (GCD):
dispatch_async(dispatch_async_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType: NSPrivateQueueConcurrencyType];
context.parentContext = mainContext;
// do some expensive job
...
// initialize a new NSManagedObject instance using the information we calculated
NSManagedObject *myObject = ...;
// once we're done, let's save the context
[context saveSelfAndParent];
});
Note that we initialized the context with a private queue concurrency type (NSPrivateQueueConcurrencyType) which tells the context that he's a background context. That is very important!
That's all! :)
For more information refer to the NSManagedObjectContext Class Reference.
I'm writing an iPad cookery application which is heavily based on CoreData.
In my app I have a screen for editing recipe with 2 types of information:
Lists with recipe category\subcategory and country
All other recipe-related information
Each of this list may be edited in popover. Changes of this list shall be persisted immediately (i.e. if user will add some recipe category to possible categories list, but cancels recipe creation, this category shall be available for all recipes).
It was descided to implement 2 separate NSManagedObjectContexts to handle each type of info: main context for recipes management and supporting context for lists.
All core data operations performed via MagicalRecord framework. Both of contexts having MagicalRecord's default context as parent. Each context observes changes in other one. All contexts are being created and used on main thread so it seems that this issue has nothing related to multithreading issues.
When I'm trying to create object in support context and persist changes in support context immediately after object's creation everything going OK. The problem comes when newly created object is being deleted right after creation - EXC_BAD_ACCES received.
However, the entity is being persisted correctly and on next launch it may be used and deleted without synchronization issues.
1 note: When object is being accessed from Main context by existingObjectWithID: method of NSManagedObjectContext it becomes possible to delete this object. However crash happens on main context's (parent context of both Main and Supporting contexts) saving then.
Here is my code:
Entity creation:
RecipeCategory* category = [RecipeCategory MR_createInContext:_supportingContext];
category.name = itemName;
[_supportingContext MR_saveToPersistentStoreAndWait];
Entity deletion:
[(RecipeCategory*)itemToRemove MR_deleteEntity];
[_supportingContext MR_saveToPersistentStoreAndWait];
Contexts creation and observing setup:
[_mainContext MR_stopObservingContext:_supportingContext];
[_supportingContext MR_stopObservingContext:_mainContext];
_mainContext = [NSManagedObjectContext MR_contextWithParent:[NSManagedObjectContext MR_defaultContext]];
_supportingContext = [NSManagedObjectContext MR_contextWithParent:[NSManagedObjectContext MR_defaultContext]];
[_mainContext MR_observeContextOnMainThread:_supportingContext];
[_supportingContext MR_observeContextOnMainThread:_mainContext];
Please advice what may cause this issue because now I'm confused even in which way shall I move to solve this. Changes management section in Core Data documentation gives nothing. Same results gave google.
Don't do that. A context watching another that is also watching the watcher...that's bad. You first need to understand the rules of nested contexts and how data flows from one to the other on save.
I your case, you may want to look for the MR_confinementContext method on NSMOC. This will create a context that uses the old thread confinement model. This may help you get around your thread crashes. But first, don't do the circular observing...data flows in