Is asynchronous Core Data needed in most cases? - ios

I've learned that generally, intensive tasks should take place on background threads, as if they're on the main thread they'll block user interaction and interface updates.
Does Core Data fall under that umbrella? I received a great answer to my question about loading images asynchronously in a UITableView, but I'm curious how to then work with Core Data as the backend.
I know Apple has a Core Data Concurrency guide, but I'm curious in which cases one is supposed to use Core Data in the background.
As a quick example, say I'm making a Twitter client and want to get all the tweet information (tweet text, username, user avatar, linked images, etc.). I asynchronously download that information and receive some JSON from the Twitter API that I then parse. Should I then do a dispatch_async(dispatch_get_main_queue()...) when I add the information to Core Data?
I also then want to download the user's avatar, but I want to do that separately from presenting the tweet, so I can present the tweet quickly as possible, then present the image when ready. So I asynchronously download the image. Should I then update that Core Data item asynchronously?
Am I in a situation where I don't need multi-threaded Core Data at all? If so, when would be a situation where I need it?

I've learned that generally, intensive tasks should take place on background threads, as if they're on the main thread they'll block user interaction and interface updates.
Does Core Data fall under that umbrella?
Yes, actually.
Core Data tasks can and should - where possible - be executed on background threads or on non-main queues.
It's important to note though, that each managed object is associated to a certain execution context (a thread or a dispatch queue). Accessing a managed object MUST be executed only from within this execution context. This association comes from the fact that each managed object is registered with a certain Managed Object Context, and this managed object context is associated to a certain execution context when it is created.
See also:
[NSManagedObjectContext initWithConcurrencyType](),
[NSManagedObjectContext performBlock]
[NSManagedObjectContext performBlockAndWait]
Consequently, when displaying properties of managed objects, this involves UIKit, and since UIKit methods MUST be executed on the main thread, the managed object's context must be associated to the main thread or main queue.
Otherwise, Core Data and user code can access managed objects from arbitrary execution contexts, as long as this is the one to which the managed object is associated with.
The below picture is an Instrument Time Profile which shows quite clearly how Core Data tasks can be distributed on different threads:
The highlighted "spike" in the CPU activity shows a task which performs the following:
Load 1000 JSON objects from origin (this is just a local JSON file),
Create managed objects in batches of 100 on a private context
Merge them into the Cora Data Stack main context
Save the context (each batch)
Finally, fetch all managed objects into the main context
As we can see, only 26.4% of the CPU load will be executed on the main thread.

Core Data does indeed fall under that umbrella. Particularly for downloading and saving data, but also possibly for fetching depending on the number of objects in the data store and the predicate to be applied.
Generally, I'd push all object creation and saving which is coming from a server onto a background thread. I'd only update and save objects on the main thread if they're user generated (because it would only be one, updated slowly and infrequently).
Downloading your twitter data is a good example as there will potentially be a good amount of data to process. You should process and save the data on a background thread and save it up to the persistent store there. The persistent store should then merge the changes down to the main thread context for you (assuming you have the contexts configured nicely - use a 3rd party framework for that like MagicalRecord).
Again, for the avatar update, you're already on a background thread so you might as well stay there :-)
You might not need to use multiple threads now. If you only download the 5 most recent tweets then you might not notice the difference. But using a framework can make the multi-threading relatively easy. And using a fetched results controller can make knowing when the UI should be updated on the main thread very easy. So, it's worthwhile taking the time to understand the setup and using it.

The best way to handle behavior like this is to use multiple NSManagedObjectContexts. The "main" context you create like so:
_mainManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
_mainManagedObjectContext.persistentStoreCoordinator = [self mainPersistentStoreCoordinator];
You're going to want to do any heavy writes on a different NSManagedObjectContext to avoid blocking your UI thread as you import (which can be quite noticeable on large or frequent operations to your Main context).
To achieve this, you would do something like this:
Retrieve your data asynchronously from the network
Spin up a temporary (or perhaps a permanent) background NSManagedObjectContext and set the "parent" to the main ManagedObjectContext (so it will merge the data you import when you save)
Use the performBlock: or performBlockAndWait: APIs to write to that context on its own private queue (in the background)
Example (uses AFNetworking):
[_apiClient GET:[NSString stringWithFormat:#"/mydata"]
parameters:nil
success:^(AFHTTPRequestOperation *operation, id responseObject) {
NSError* jsonError;
id jsonResponse = [NSJSONSerialization JSONObjectWithData:operation.responseData options:kNilOptions error:&jsonError];
if (!jsonError) {
NSArray* parsedData = (NSArray*)jsonResponse;
completionBlock(parsedShares, nil);
} else {
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
context.parentContext = YOUR_MAIN_CONTEXT; // when you save this context, it will merge its changes into your main context!
// the following code will occur in a background queue for you that CoreData manages
[context performBlock:^{
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
// THIS IS WHERE YOU IMPORT YOUR DATA INTO CORE DATA
if (![context save:&saveError]) {
NSLog(#"Error saving context: %#", saveError);
} else {
NSLog(#"Saved data import in background!");
}
}];
}
}
failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"There was an error: %#", error)
}];
Docs: https://developer.apple.com/library/mac/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSManagedObjectContext_Class/NSManagedObjectContext.html#//apple_ref/occ/instm/NSManagedObjectContext/performBlockAndWait:

As I alluded to in your other question, it's largely going to be a matter of how long the manipulations are going to take. If the computation required doesn't have a noticeable delay, by all means be lazy about and do your core data on the main thread.
In your other example, you're requesting 20 items from twitter, parsing 20 items and sticking them into CoreData isn't going to be noticeable. The best approach here is to probably continue to just fetch 20 at a time and update in the foreground as each chunk becomes available.
Downloading all items from twitter in one request will take a significant amount of time and computation and it's probably worth creating a separate ManagedObjectModel and synchronizing it with the foreground model. Since you really have one-way data (it always flows twitter->core data->user interface) the likelihood of clashes is minimal, so you can easily use NSManagedObjectContextDidSaveNotification and mergeChangesFromContextDidSaveNotification:

Related

NSFetchedResultsController feeding table view while background update of same persistent store causes deadlock

Still working on converting an app over from downloading information every time it uses or displays it, to caching it on-phone using CoreData (courtesy of MagicalRecord). This is on iOS 7
Because we don't have a data-push system set up to automatically update the phone's cached data whenever some data changes on the backend, I've been thinking over the last many months (as we worked on other aspects of the app) how to manage keeping a local copy of the data on the phone and being able to have the most up to date data in the cache.
I realized that as long as I still fetch the data every time :-( I can use the phone's CoreData backed cache of data to display and use, and just use the fetch of the data to update the on-phone database.
So I have been converting over the main data objects from being downloaded data making up a complete object, to these main data objects being light stand-in objects for CoreData objects.
Basically, each of the normal data objects in the app, instead of containing all the properties of the object internally, contains only the objectIDof the underlying CoreData object and maybe the app specific ID internally, and all other properties are dynamic and gotten from the CoreData object and passed through (most properties are read-only and updates are done through bulk-rewriting of the core data from passed in JSON)
Like this:
- (NSString *)amount
{
__block NSString *result = nil;
NSManagedObjectContext *localContext = [NSManagedObjectContext MR_newContext];
[localContext performBlockAndWait:^{
FinTransaction *transaction = (FinTransaction *)[localContext existingObjectWithID:[self objectID] error:nil];
if (nil != transaction)
{
result = [transaction.amount stringValue];
}
}];
return result;
}
Occasionally there is one that needs to be set and those look like this:
- (void)setStatus:(MyTransactionStatus)status
{
[MagicalRecord saveWithBlock:^(NSManagedObjectContext *localContext) {
FinTransaction *transaction = (FinTransaction *)[localContext existingObjectWithID:[self objectID] error:nil];
if (nil != transaction)
{
transaction.statusValue = status;
}
} completion:^(BOOL success, NSError *error){}];
}
Now, my issue is that I have a view controller that basically uses an NSFetchedResultsController to display stored data from the local phone's CoreData database in a table view. At the same time as this is happening, and the user may start to scroll through the data, the phone spins off a thread to download updates to the data and then starts updating the CoreData data store with the updated data, at which point it then runs an asynchronous GCD call back on the main thread to have the fetched results controller refetch its data and and tells the table view to reload.
The problem is that if a user is scrolling through the initial fetched results controller fetched data and table view load, and the background thread is updating the same Core Data objects in the background, deadlocks occur. It is not the exact same entities being fetched and rewritten (when a deadlock occurs), i.e., not that object ID 1 is being read and written, but that the same persistent data store is being used.
Every access, read or write, happens in a MR_saveWithBlock or MR_saveWithBlockAndWait (writes/updates of data) as the case may be, and a [localContext performBlock:] or [localContext performBlockAndWait:] as may be appropriate. Each separate read or write has its own NSManagedObjectContext. I have not seen any where there are stray pending changes hanging around, and the actual places it blocks and deadlocks is not always the same, but always has to do with the main thread reading from the same persistent store as the background thread is using to update the data.
The fetched results controller is being created like this:
_frController = [[NSFetchedResultsController alloc] initWithFetchRequest:fetchRequest
managedObjectContext:[NSManagedObjectContext MR_rootSavingContext]
sectionNameKeyPath:sectionKeyPath
cacheName:nil];
and then an performFetch is done.
How can I best structure this sort of action where I need to display the extent data in a table view and update the data store in the background with new data?
While I am using MagicalRecord for most of it, I am open to comments, answers, etc with or without (straight CD) using MagicalRecord.
So the way I'd handle this is to look at having two managed object contexts each with its own persistent store coordinator. Both of the persistent store coordinators talk to the same persistent store on disk.
This approach is outlined in some detail in Session 211 from WWDC 2013 — "Core Data Performance Optimization and Debugging", which you can get to on Apple's Developer Site for WWDC 2013.
In order to use this approach with MagicalRecord, you will need to look at using the upcoming MagicalRecord 3.0 release, with the ClassicWithBackgroundCoordinatorSQLiteMagicalRecordStack (yes, that name needs work!). It implements the approach outlined in the WWDC session, although you need to be aware that there will be changes needed to your project to support MagicalRecord 3, and that it's also not quite released yet.
Essentially what you end up with is:
1 x Main Thread Context: You use this to populate your UI, and for your fetched results controllers, etc. Don't ever make changes in this context.
1 x Private Queue Context: Make all of your changes using the block-based saved methods — they automatically funnel through this context and save to disk.
I hope that makes sense — definitely watch the WWDC session — they use some great animated diagrams to explain why this approach is faster (and shouldn't block the main thread as much as the approach you're using now).
I'm happy to go into more detail if you need it.

Small Core Data performance gains from background saving

I'm trying to squeeze every last performance gain from my app's use of Core Data. Using a large data set for testing, I've made some significant gains by refactoring the object graph and modifying various predicates used in fetching. The only thing I'm still not pleased with, in terms of performance, is saving the context after a large delete operation. As a result, I finally implemented threading using a parent-child context structure. However, I am underwhelmed by the performance gain.
Object graph: I'm only showing the relationships because I know that's what makes deleting expensive.
Country:
cookingStyles (to-many relationship)
CookingStyle:
country (inverse relationship, one-to-one)
recipes (to-many relationship)
Recipe:
cookingStyle (inverse relationship, one-to-one)
For the sake of argument, assume that each cooking style is unique for a country. Therefore, configuring the CookingStyle's country relationship as one-to-many will NOT de-duplicate data. The same goes for each recipe, i.e., each recipe is unique for a cooking style.
Performance Test:
Data set: 18 Country objects, each Country has 10 CookingStyle objects, each CookingStyle has 35 Recipe objects (total = 18 Country objects, 180 CookingStyle objects, 6,300 Recipe objects)
I'm using Time Profiler to measure performance on a 5G iPod Touch running iOS 7.1.
I'm using parent-child NSManagedObjectContext instances to implement threading.
The table view controller where the deletion occurs uses a NSFetchedResultsController.
Results: Saving the parent-child contexts after deleting one Country object still takes about 1000 ms. Prior to implementing threading, it took about 1200-1300 ms to delete a Country object.
When I dive deeper into the Time Profiler results, saving from the child context to the parent context is blocking the main thread. The latter save is not hitting the disk, so I assumed that it would be faster.
Question:
Are my expectations too high under the test case, or is there something else I should be doing to improve performance when deleting Country objects?
Code:
How I create the parent-child contexts:
_model = [NSManagedObjectModel mergedModelFromBundles:nil];
_coordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:_model];
_parentContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[_parentContext performBlockAndWait:^{
[_parentContext setPersistentStoreCoordinator:_coordinator];
[_parentContext setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];
}];
_context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
[_context setParentContext:_parentContext];
[_context setMergePolicy:NSMergeByPropertyObjectTrumpMergePolicy];
How I save the contexts (source: "Core Data", 2nd Ed., Marcus Zarra):
- (void)saveContext:(BOOL)wait
{
if (!self.context) return;
if ([self.context hasChanges]) {
[self.context performBlockAndWait:^{
NSError *error;
if ([self.context save:&error]) {
QuietLog(#"\nchild context SAVED changes to parent context");
} else {
QuietLog(#"\nchild context FAILED to save: %#", error);
[self showValidationError:error];
}
}];
}
void (^saveParentContext)(void) = ^{
NSError *error= nil;
if ([self.parentContext save:&error]) {
QuietLog(#"\nparent context SAVED changes to persistent store");
} else {
QuietLog(#"\nparent context FAILED to save: %#", error);
[self showValidationError:error];
}
};
if ([self.parentContext hasChanges]) {
if (wait) {
[self.parentContext performBlockAndWait:saveParentContext];
} else {
[self.parentContext performBlock:saveParentContext];
}
}
}
How I delete and save:
- (void)deleteButtonTapped:(UIBarButtonItem *)sender
{
AEBCoreDataHelper *coreDataHelper = [AEBCoreDataHelper sharedCoreDataHelper];
Country *country = ...
[coreDataHelper.context deleteObject:country];
// NSFetchedResultsController's delegate will delete the row for me
[coreDataHelper saveContext:NO]; // blocks main thread for too long
}
Update:
I thought it was the UI that was unresponsive while the parent-child contexts were saved. It's not. Instead, anything involving Core Data is unresponsive until the parent context (private queue) completes its save to the disk. The UI only seemed unresponsive because I tried to scroll the table view, which requires another batch of data from the fetched results controller. If I scroll through the entire table view before deleting an object, then scrolling is responsive soon after deleting. Multi-threading with Core Data feels like shenanigans.
You are blocking on saving the parent if the parent has changes but not blocking when it doesn't have changes. That is not going to gain you any performance.
If you don't have changes, there is nothing to save, it is effectively a no-opt.
If you have changes, you want them to run asynchronously so that your main thread can go back to servicing the user. There is no value in blocking the main thread as the main context already knows about the changes and anything it changes will not impact the save.
Lastly, using a private context will not be "faster". It will give a perception of being faster to the user because they get control back sooner. Threading is not directly linked to performance. The idea of offloading the save is so that the user can go back to using the app instead of waiting for the save. The save can easily take longer but the user doesn't care.
Update
If you are getting the UI blocked on a scroll then that is an indication that your save is taking too long.
ANY activity that hits the NSPersistentStoreCoordinator is going to lock it. A long fetch or a long save, doesn't matter. This is a fact of life. There are ways to avoid that rule but just throwing threads at the problem is not going to fix it. You need to work with the underlying issue of locking the NSPersistentStoreCoordinator.
First, I would look at breaking up the save into smaller units. Making the saves smaller allows the NSPersistentStoreCoordinator to weave in saves and fetches so that the human doesn't perceive the saves.
If you can't do that, consider doing your deletes against a separate NSPersistentStoreCoordinator. This is assuming you are using SQLite. You can then feed the changes back to your main NSManagedObjectContext via notifications. Note that the consumption of those notifications will cause a performance impact on the main NSManagedObjectContext and therefore on the UX.
You can also explore pre-faulting in the objects in the UITableView so that there is no need to go back out to disk. But then you are trading speed/performance for memory and on iOS you tend to lose when you do that.

iOS 6 What is the standard way to deal with background saving of managed objects (in 2013)?

Last year I used RestKit 0.10 to seamlessly download and save core data objects in background. However, when I tried to use restkit in 2013, I noticed that they have taken out the ActiveRecord pattern, which I relied upon to abstract away all the unpleasantness of background saving.
I found that the ActiveRecord pattern exists in MagicalRecord framework, but most of the documentation I could find is for version 2.x, while my cocoapods install 3.x.
I spent the last 2 hours searching, and find a lot of answers that are really out of date and no longer work for these new frameworks.
This poses the question: what the standard/easiest way to deal with saving core data objects in background using frameworks available in 2013? Should I try some other framework?
If you don't use any external library like Magical Record or RestKit, but simply go for the all manual stuff, you can take advantage of the new NSManagedObjectContext APIs.
You can now have contexts nested with a parent-child relationship and you can also tell each context to perform a block in it's own thread. My advice, therefore is to have the following structure for your application:
1) A background saving context. This will be the only context that saves and reads data directly to/from the database.
2) A context initalized on the main thread that will be your point of access for everything you need to do in the application, especially updating the UI. This context will be a child of the saving context.
3) As needed, you'll create background contextes that perform work on background threads, e.g. loading data from the network and serialize this data in NSManagedObject instances. This contextes will be children of the main context.
4) Every time you call -[NSManagedObjectContext save:] on a context, you should also call the same method on it's parentContext. To do this you could have a convenience method in a category on NSManagedObjectContext that reads something like this:
- (void)saveSelfAndParent {
[self save:NULL];
[self.parentContext performBlock:^{
[self.parentContext saveSelfAndParent];
}];
}
This is already a thread safe configuration and your changes will propagate the changes up to the database. Note that as the saving context will have no parent (and thus self.parentContext will be nil), the performBlock: won't crash the app.
Here's an example what you need to do to create a new entity assuming you kick off your background work using Grand Central Dispatch (GCD):
dispatch_async(dispatch_async_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
NSManagedObjectContext *context = [[NSManagedObjectContext alloc] initWithConcurrencyType: NSPrivateQueueConcurrencyType];
context.parentContext = mainContext;
// do some expensive job
...
// initialize a new NSManagedObject instance using the information we calculated
NSManagedObject *myObject = ...;
// once we're done, let's save the context
[context saveSelfAndParent];
});
Note that we initialized the context with a private queue concurrency type (NSPrivateQueueConcurrencyType) which tells the context that he's a background context. That is very important!
That's all! :)
For more information refer to the NSManagedObjectContext Class Reference.

How to avoid the UI freeze while a Managed Object Context is saving?

I want to implement a UI-responsive downloading and parsing of a large data set, saving it with Core Data.
My setup:
I display the downloaded content in a custom view controller. I don't use a NSFetchedResultsController.
There are 3 MOCs:
masterMOC (responsible for saving to disk, NSPrivateQueueConcurrencyType)
mainMOC (used by UI, NSMainQueueConcurrencyType, a child of the masterMOC)
backgroundMOC (responsible for the import from JSON, created in a separate thread, a child of the masterMOC)
I am importing in batches - every 50 items I perform the MOC saving in the following way:
NSError *error;
[backgroundMOC save:&error];
NSManagedObjectContext *masterMOC = backgroundMOC.parentContext; //set during initialization
[masterMOC performBlock:^{
NSError *parentContextError = nil;
[masterMOC save:&parentContextError];
}];
I expect the changes in the mainMOC to be made after the masterMOC is saved. If I try to access some relationship of a random managed object while the masterMOC is saving (saving takes some time), the UI hangs until the saving is completed.
Question: how to avoid the UI freeze while the masterMOC is saving?
Your problem probably is that the data store is blocking while you are writing to it. So, either make the data store non-blocking (this may or may not be possible in your case) or if not viable, make the accessor non-blocking. In the latter case the GUI will not hang, but it also will not update either until the result of the access comes back.

Core Data multi thread application

I'm trying to use core data in a multi thread way.
I simply want to show the application with the previously downloaded data while downloading new data in background.
This should let the user access the application during update process.
I have a NSURLConnection which download the file asyncronously using delegate (and showing the progress), then i use an XMLParser to parse the new data and create new NSManagedObjects in a separate context, with its own persistentStore and using a separate thread.
The problem is that creating new objects in the same context of the old one while showing it can throws BAD_INSTRUCTION exception.
So, I decided to use a separate context for the new data, but I can't figure out a way to move all the objects to the other context once finished.
Paolo aka SlowTree
The Apple Concurrency with Core Data documentation is the place to start. Read it really carefully... I was bitten many times by my misunderstandings!
Basic rules are:
Use one NSPersistentStoreCoordinator per program. You don't need them per thread.
Create one NSManagedObjectContext per thread.
Never pass an NSManagedObject on a thread to the other thread.
Instead, get the object IDs via -objectID and pass it to the other thread.
More rules:
Make sure you save the object into the store before getting the object ID. Until saved, they're temporary, and you can't access them from another thread.
And beware of the merge policies if you make changes to the managed objects from more than one thread.
NSManagedObjectContext's -mergeChangesFromContextDidSaveNotification: is helpful.
But let me repeat, please read the document carefully! It's really worth it!
Currently [May 2015] the Apple Concurrency with Core Data documentation is, at best, very misleading as it doesn't cover any of the enhancements in iOS 5 and hence no longer shows the best ways to use core data concurrently. There are two very important changes in iOS 5 - parent contexts and new concurrency/threading types.
I have not yet found any written documentation that comprehensively covers these new features, but the WWDC 2012 video "Session 214 - Core Data Best Practices" does explain it all very well.
Magical Record uses these new features and may be worth a look.
The real basics are still the same - you can still only use managed objects the thread their managed object context was created on.
You can now use [moc performBlock:] to run code on the right thread.
There's no need to use mergeChangesFromContextDidSaveNotification: anymore; instead create a child context to make the changes, then save the child context. Saving the child context will automatically push the changes up into the parent context, and to save the changes to disk just perform a save on the parent context in it's thread.
For this to work you must create the parent context with a concurrent type, eg:
mainManagedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
Then on the background thread:
context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
[context setParentContext:mainManagedObjectContext];
<... perform actions on context ...>
NSError *error;
if (![context save:&error])
{
<... handle error ...>
}
[mainManagedObjectContext performBlock:^{
NSError *e = nil;
if (![mainContext save:&e])
{
<... handle error ...>
}
}];
I hope this can help all the peoples that meet problems using core data in a multithread environment.
Take a look at "Top Songs 2" in apple documentation. With this code i took the "red pill" of Matrix, and discovered a new world, without double free error, and without faults. :D
Hope this helps.
Paolo
p.s.
Many thanks to Yuji, in the documentation you described above I found that example.

Resources