I am trying to understand a concept of backgroundContext in CoreData. Though I read several articles about it, I am not still sure about its purpose.
I have an app using CoreData that allows user create, update or delete records. User can also fetch the data he added. I want to ensure that if there are a lot of records, it will not influence a flow of the UI while fetching data.
So I studied a bit about backgroundContexts. I implemented following according to what I understood. I am not sure though whether it is a correct solution. My idea is - if I fetch in background, it cannot influence the main thread.
//I have an PersistentContainer created by Xcode.
//I create an backgroundContext.
self.backContext = (UIApplication.shared.delegate as! AppDelegate).persistentContainer.newBackgroundContext()
//Then if the user adds a new record, it's added to backContext and saved to mainContext
...
let newRecord = Record(context: self.backContext!)
...
self.backContext!.save()
self.context!.save() // mainContext
...
//If the user fetches the data I use:
self.backContext.perform {
...
}
//Since I want to show results in UI, I know these objects (from fetch) exist just in the background thread, so instead I fetch for IDs
.resultType = .managedObjectIDResultType
//Now I have IDs of fetched objects, so I restore objects in main thread using their IDs:
let object = try? self.backContext.existingObject(with: ID) as? Record
//and I can finally use fetched objects to update UI.
The question is:
Is this even correct what I am doing? (it works perfectly though)
Will this solve the problem of freezing UI if user fetches a large amount of data?
How do we use backgroundContexts correctly? Why is it not recommended to work directly with mainContext? How to prevent freezing UI while fetching big data?
One more question: If I use FetchedResultsConteroller - do I need to handle the problem of freezing UI? (while waiting on first (init) fetch result?)
Of course I am ignoring that while fetching data, my context is blocked, so I cannot create a new record
If you are fetching objects to be displayed on screen, you should absolutely be using fetch requests against the main thread context. Core Data is designed for this specific use case and you should not be experiencing slowdowns or freezes because of executing fetches. If you are having problems, then profile your app in instruments and find out where the actual slowdown is.
Background contexts are meant to be used if you are performing bulky or long-running work like processing large API responses which you've shown to be affecting main thread performance.
So I do not have to be afraid of freezing UI, even if my database will contain thousands of records? I can make fetch request with mainContext?
Yes
If I would like to do some special time consuming operations that would not be shown to UI, my code would be correct, right?
Yes, you'd normally create a background context, do work, save the background context - and then access those objects as normal from the main context.
And last but not least - why is it not recommended to work directly with mainContext when I add a new record?
I'm not sure where you've seen this recommendation, but quite a common pattern is to make a new main-queue (not background) child context to support the application workflow of adding a new object. Then if the user cancels the addition, you can just discard the editing context without needing to worry about undoing your work.
Related
I have an iOS app with master and detail layout.
In master, I managed my own NSManagedObjectContext, and also detail is, by this way:
NSPersistentStoreCoordinator *psc = ((AppDelegate *)UIApplication.sharedApplication.delegate).persistentStoreCoordinator;
_context = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType];
[_context setPersistentStoreCoordinator:psc];
In master, I display list that can be clicked by user to show the detail in detail layout.
Upon filling the detail by user, user can save the detail by clicking on button there.
However, I am trying to understand this:
Since there is a save button in detail, the save method will save the detail with detail's context and call the load list in master
Load list will remove all the NSMutableArray of the list [_activities removeAllObjects]; and re-fetch the data from Core Data and reload the tableview
Done that but the the re-fetch function seems to use old data and not the latest.
Why does re-fetch the data doesn't work if I use same context?
You are using outdated APIs to create your managed object contexts. After creating a context, instead of assigning the persistent store, you should set the parentContext.
If you display a "list", you should be using a context that is of type NSMainQueueConcurrencyType. The best way is a NSFetchedResultsController which will also help you manage memory and performance, and greatly simplify the updates you need for your UI. (You would avoid the verbosity and complexity of merging the data "manually" via NSManagedObjectContextDidSaveNotification.)
NB: Resetting the context is a crude and inefficient method for the task you are trying to accomplish. Due to the fact that it affects the entire context, it could change the behavior in other parts of your app.
You have two contexts going on here which is why the data is not being refreshed. If you call [context reset] then you will find that the data will refresh.
Explanation:
An NSManagedObjedctContext is a "scratchpad" which manages the Objective-C representation of SQLite (or XML) data in memory. When you fetch data the context retains the managed objects that are created by the fetch. If you then fetch from another context that context reads the data from the persistent store and creates new managed objects based on what it finds.
ALSO: When you perform a fetch the context checks to see if a managed object representation is ALREADY in existence and then returns it if it is. This means that two contexts can get out of sync quite quickly.
There are several ways around this:
1) Calling reset on a context returns it to it's "baseState" which means all the managed objects are "forgotten" or released by the context. Therefore, another fetch will force the context to read the data directly from the store.
2) Implementing the NSManagedObjectContextDidSaveNotification (see here) will allow you to incorporate changes between contexts.
I have found this article very useful in the past.
I'm trying to implement some basic UIManagedDocument import/export functionality into my app, mainly for dev so that I can easily inspect the document contents and more crucially preserve a set of test data when I start trying to iterate on my CoreData models.
All I am trying to do is load some JSON data from a local file and inject it into my apps UIManagedDocument. The UIManagedDocument's ManagedObjectContext contents are visualised in my app using some Core Data Table View Controllers from the Stanford iOS courses.
I thought I'd try to write this with some threading to keep the UI responsive and to learn how to do it. So I've done something like this
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
// read JSON data file
// parse JSON into dictionaries etc
// iterate over dictionaries
// try to create a new core data entity with my document's ManagedObjectContext
} );
At first I thought this was working. No errors, asserts, crashes triggered and upon looking at my CoreData TableViews I could see the newly added objects in my UI. Unfortunately the newly added objects were seemingly never saved back to the store. I even hooked up to listen to the NSManagedObjectContextDidSaveNotification from my UIDocument's managedObjectContext and saw it wasn't triggering on pressing the home button, like it usually does if it has some changes performed in the app with my UI pending. Infact even doing these operations in the UI wouldn't cause the notification and saving to occur so it was clearly not happy.
I unrolled the code from within the background queue and ran it on the main thread synchronously and everything worked ok, the new data was saved correctly.
I started reading about the complexities of threading and coredata, the documentation seemed to suggest using the UIDocument's ManagedObjectContext's parent ManagedObjectContext to perform operations on in the background so I tried doing the same code again using this parent context, so as follows
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
// read JSON data file
// parse JSON into dictionaries etc
// iterate over dictionaries
// try to create a new core data entity with my document's ManagedObjectContext parent ManagedObjectContext
} );
This time for some reason the CoreData TableView controllers no longer updated to show the newly injected objects. Even after explicitly calling save on the parent context, nothing appeared. However on quitting the app and reloading the app the newly injected objects did seem to be added correctly. Interestingly at this point i'd left a fetchrequest with a cachename specified and that threw up an error on this first run of the app after injecting the objects this way. I guess somehow the way the object had come from the parent context directly perhaps invalidated the cache somehow, that's still something I don't fully understand. Even changing the cache to nil didn't fix the issue of the table views not updated the same session as when the objects were injected into the parent context.
Looking elsewhere I've seen some uses of the managedObjectContext performBlock suggested. Another case where someone has said you must call
[document updateChangeCount:UIDocumentChangeDone]
after all changes to ensure the saving is performed, or perhaps using
- (void)autosaveWithCompletionHandler:(void (^)(BOOL success))completionHandler
instead. Though elsewhere I've seen mentioned that saving should be enough to push context contents through the hierarchy. Does saving only work from child -> parent and not from parent -> child.
Or am I just doing it wrong?
Anyone's time and help is really appreciated! Cheers.
please look at the 'Parent/Child Contexts' section in the Multi-Context CoreData.
Whenever a child MOC saves the parent learns about these changes and this causes the fetched results controllers to be informed about these changes as well. This does not yet persist the data however, since the background MOCs don’t know about the PSC. To get the data to disk you need an additional saveContext: on the main queue MOC.
I do saving to the PSC n following way:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
{
// read JSON data file
// parse JSON into dictionaries etc
// iterate over dictionaries
NSError* error;
if (self.managedObjectContext.hasChanges)
{
[self.managedObjectContext save: &error];
}
// Save main context
dispatch_sync(dispatch_get_main_queue(), ^
{
[[AXContextHelper sharedInstance] saveMainContext];
});
}
After looking into all the suggestions I could find for similar problems, none of them seemed to help.
The scenario i was describing in the end had the store stuff to file handled ok, but the UI not updating. At the time when I do this import/export I'm in a view ontop of the core data table view controller that doesn't update when I inject all these JSON objects, so in the end all I did was force that controller to re-fetch it's data. I believe the NSFetchedResultsController is meant to monitor the managedObjectContext and update the fetch request as required. For whatever reason, this wasn't working.
With the force re-fetch called, everything seems to work ok. The newly injected entities appear in my tables, and are saved to the store files.
Either there are some bugs in these objects, or I'm still using it wrong but I see other problems... my import/export data both work now on background threads with the parent context. What I've noticed today is that I can use the UI to properly insert and edit some objects using the child context on the main thread. I don't deliberately call anything at the moment after making these edits so i guess the edits are are still pending till core data decides to save. If i then go to my export feature and use it using the parent context then i find the newly edited or inserted objects aren't there.
Clearly there's no messaging happening in time from child -> parent, and messaging from background thread edits on the the parent itself just doesn't seem to work.. I never get any notifications for the edits i perform in a thread on the parent context, even though they seem to work. If I force the save to happen via the documents then the passed in dictionaries show no edits being made, even though the edits are saved correctly to store files somehow.
I guess i now need to force my document to save after every operation, or at least before every operation i'm about to do on another thread to ensure the parent/child are both in sync.
Still the problem I originally described I've managed to get around, but I'm still left wondering quite how I should be using the parent and child context of a UIManagedDocument to avoid encountering these issues.
I have a NSFetchedResultsController tied to my main managed object context. It is in charge of keeping data for a table view in my main view.
I have an NSOperation running on a background thread that refreshes/deletes the managed objects that the fetched results controller is keeping track of. I create a child context (private concurrency type / parent = main managed object context) in the nsoperation and insert/delete objects. When it is finished, it saves its context, as well as the parent context.
What's interesting and very frustrating is that this works fine in iOS 6. When I insert or delete objects, my fetched results controller is notified of the changes and everything works as expected. However, on iOS 5, everything works except for deletes. The fetched results controller is not notified of a delete. However... if I manually refresh the fetched results controller (making it nil and refetching the same predicate) then it will show the expected result.
Also, when I register for change/save notifications on the main context, I can see that the user info dictionary contains the objects that I've deleted... even in iOS 5!
One issue that I thought it may be, but I don't think holds because I've removed the factors, is that this object is in a many to one relationship with another object. The object I am deleting/inserting is an "employee" and it has a relationship with a "department". The employee is set to nullify and the department is set to cascade.
As I said, this works fine in iOS6 but not in iOS5.
Any tips would be very helpful.
This bug is due to saving to the persistent store. This child context saves itself, then calls perform block on it's parent, the main managed object context. When the main managed object context saves, it triggers a background context to write to the persistent store. When I removed the background context save, the fetched results controller updated as expected.
Something interesting that I found that was probably causing this was that the managed object was leaking every time I tried saving to the store. Not exactly sure how to fix this yet, but it's good to know the reason for it.
This has been asked before, but no solution described that is fast enough for my app needs.
In the communications protocol we have set up, the server sends down a new set of all customers every time a sync is performed. Earlier, we had been storing as a plist. Now want to use Core Data.
There can be thousands of entries. Deleting each one individually takes a long time. Is there a way to delete all rows in a particular table in Core Data?
delete from customer
This call in sqlite happens instantly. Going through each one individually in Core Data can take 30 seconds on an iPad1.
Is it reasonable to shut down Core Data, i.e. drop the persistence store and all managed object contexts, then drop into sqlite and perform the delete command against the table? No other activity is going on during this process so I don't need access to other parts of the database.
Dave DeLong is an expert at, well, just about everything, and so I feel like I'm telling Jesus how to walk on water. Granted, his post is from 2009, which was a LONG time ago.
However, the approach in the link posted by Bot is not necessarily the best way to handle large deletes.
Basically, that post suggests to fetch the object IDs, and then iterate through them, calling delete on each object.
The problem is that when you delete a single object, it has to go handle all the associated relationships as well, which could cause further fetching.
So, if you must do large scale deletes like this, I suggest adjusting your overall database so that you can isolate tables in specific core data stores. That way you can just delete the entire store, and possibly reconstruct the small bits that you want to remain. That will probably be the fastest approach.
However, if you want to delete the objects themselves, you should follow this pattern...
Do your deletes in batches, inside an autorelease pool, and be sure to pre-fetch any cascaded relationships. All these, together, will minimize the number of times you have to actually go to the database, and will, thus, decrease the amount of time it takes to perform your delete.
In the suggested approach, which comes down to...
Fetch ObjectIds of all objects to be deleted
Iterate through the list, and delete each object
If you have cascade relationships, you you will encounter a lot of extra trips to the database, and IO is really slow. You want to minimize the number of times you have to visit the database.
While it may initially sound counterintuitive, you want to fetch more data than you think you want to delete. The reason is that all that data can be fetched from the database in a few IO operations.
So, on your fetch request, you want to set...
[fetchRequest setRelationshipKeyPathsForPrefetching:#[#"relationship1", #"relationship2", .... , #"relationship3"]];
where those relationships represent all the relationships that may have a cascade delete rule.
Now, when your fetch is complete, you have all the objects that are going to be deleted, plus the objects that will be deleted as a result of those objects being deleted.
If you have a complex hierarchy, you want to prefetch as much as possible ahead of time. Otherwise, when you delete an object, Core Data is going to have to go fetch each relationship individually for each object so that it can managed the cascade delete.
This will waste a TON of time, because you will do many more IO operations as a result.
Now, after your fetch has completed, then you loop through the objects, and delete them. For large deletes you can see an order of magnitude speed up.
In addition, if you have a lot of objects, break it up into multiple batches, and do it inside an auto release pool.
Finally, do this in a separate background thread, so your UI does not pend. You can use a separate MOC, connected to a persistent store coordinator, and have the main MOC handle DidSave notifications to remove the objects from its context.
WHile this looks like code, treat it as pseudo-code...
NSManagedObjectContext *deleteContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateConcurrencyType];
// Get a new PSC for the same store
deleteContext.persistentStoreCoordinator = getInstanceOfPersistentStoreCoordinator();
// Each call to performBlock executes in its own autoreleasepool, so we don't
// need to explicitly use one if each chunk is done in a separate performBlock
__block void (^block)(void) = ^{
NSFetchRequest *fetchRequest = //
// Only fetch the number of objects to delete this iteration
fetchRequest.fetchLimit = NUM_ENTITIES_TO_DELETE_AT_ONCE;
// Prefetch all the relationships
fetchRequest.relationshipKeyPathsForPrefetching = prefetchRelationships;
// Don't need all the properties
fetchRequest.includesPropertyValues = NO;
NSArray *results = [deleteContext executeFetchRequest:fetchRequest error:&error];
if (results.count == 0) {
// Didn't get any objects for this fetch
if (nil == results) {
// Handle error
}
return;
}
for (MyEntity *entity in results) {
[deleteContext deleteObject:entity];
}
[deleteContext save:&error];
[deleteContext reset];
// Keep deleting objects until they are all gone
[deleteContext performBlock:block];
};
[deleteContext preformBlock:block];
Of course, you need to do appropriate error handling, but that's the basic idea.
Fetch in batches if you have so much data to delete that it will cripple memory.
Don't fetch all the properties.
Prefetch relationships to minimize IO operations.
Use autoreleasepool to keep memory from growing.
Prune the context.
Perform the task on a background thread.
If you have a really complex graph, make sure you prefetch all the cascaded relationships for all entities in your entire object graph.
Note, your main context will have to handle DidSave notifications to keep its context in step with the deletions.
EDIT
Thanks. Lots of good points. All well explained except, why create the
separate MOC? Any thoughts on not deleting the entire database, but
using sqlite to delete all rows from a particular table? – David
You use a separate MOC so the UI is not blocked while the long delete operation is happening. Note, that when the actual commit to the database happens, only one thread can be accessing the database, so any other access (like fetching) will block behind any updates. This is another reason to break the large delete operation into chunks. Small pieces of work will provide some chance for other MOC(s) to access the store without having to wait for the whole operation to complete.
If this causes problems, you can also implement priority queues (via dispatch_set_target_queue), but that is beyond the scope of this question.
As for using sqlite commands on the Core Data database, Apple has repeatedly said this is a bad idea, and you should not run direct SQL commands on a Core Data database file.
Finally, let me note this. In my experience, I have found that when I have a serious performance problem, it is usually a result of either poor design or improper implementation. Revisit your problem, and see if you can redesign your system somewhat to better accommodate this use case.
If you must send down all the data, perhaps query the database in a background thread and filter the new data so you break your data into three sets: objects that need modification, objects that need deletion, and objects that need to be inserted.
This way, you are only changing the database where it needs to be changed.
If the data is almost brand new every time, consider restructuring your database where these entities have their own database (I assume your database already contains multiple entities). That way you can just delete the file, and start over with a fresh database. That's fast. Now, reinserting several thousand objects is not going to be fast.
You have to manage any relationships manually, across stores. It's not difficult, but it's not automatic like relationships within the same store.
If I did this, I would first create the new database, then tear down the existing one, replace it with the new one, and then delete the old one.
If you are only manipulating your database via this batch mechanism, and you do not need object graph management, then maybe you want to consider using sqlite instead of Core Data.
iOS 9 and later
Use NSBatchDeleteRequest. I tested this in the simulator on a Core Data entity with more than 400,000 instances and the delete was almost instantaneous.
// fetch all items in entity and request to delete them
let fetchRequest = NSFetchRequest(entityName: "MyEntity")
let deleteRequest = NSBatchDeleteRequest(fetchRequest: fetchRequest)
// delegate objects
let myManagedObjectContext = (UIApplication.sharedApplication().delegate as! AppDelegate).managedObjectContext
let myPersistentStoreCoordinator = (UIApplication.sharedApplication().delegate as! AppDelegate).persistentStoreCoordinator
// perform the delete
do {
try myPersistentStoreCoordinator.executeRequest(deleteRequest, withContext: myManagedObjectContext)
} catch let error as NSError {
print(error)
}
Note that the answer that #Bot linked to and that #JodyHagins mentioned has also been updated to this method.
Really your only option is to remove them individually. I do this method with a ton of objects and it is pretty fast. Here is a way someone does it by only loading the managed object ID so it prevents any unnecessary overhead and makes it faster.
Core Data: Quickest way to delete all instances of an entity
Yes, it's reasonable to delete the persistent store and start from scratch. This happen fairly quick. What you can do is remove the persistent store (with the persistent store URL) from the persistent store coordinator, and then use the url of the persistent store to delete the database file from your directory folder. I did it using NSFileManager's removeItemAtURL.
Edit: one thing to consider: Make sure to disable/release the current NSManagedObjectContext instance, and to stop any other thread which might be doing something with a NSManagedObjectContext which is using the same persistent store. Your application will crash if a context tries to access the persistent store.
I'm building an interface to allow users to filter a set of Photos. The data model is above. I'd like the disable any controls that would result in 0 results.
To accomplish this, I'm running new fetch requests for every control that is off/not selected each time the user makes their own change. I add the data that the control represents to my NSCompoungPredicate, then remove it after I get the result. If the count of the result is 0, I disable that control.
I'm doing this all on the main thread so in some cases there is a bit of a lag in the app. Is there a better way to do this type of filtering with less overhead? Should I run these filter fetches on their own thread? I've never done that with CoreData and worm what I've read I need a separate context for that and I'm not sure how to go about setting up code for that.
A bit of code would help. Aside from that, here are a few suggestions.
First, on your fetch request, use countForFetchRequest:error: because it will just query the database, and return the count instead of object information.
Second, if you don't want to use threads, and the search is still too slow, you can do the initial query when the app starts. This will then enable/disable the various controls.
You can simple catch the context notifications that tell when data has changed, and update that information accordingly. Then, you don't have to do any queries at all. Just initialize, and update the status as objects are added/removed from the database.
If you want to use threads, then that's not really all that difficult.
It sounds like all you want is a thread that is just running queries. You setup a MOC...
NSManagedObjectContext *checkerMoc = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
checkerMoc.persistentStoreCoordinator = MyCurrentMoc.persistentStoreCoordinator;
Now, whenever you want to check the database...
[checkerMoc performBlock:^{
NSFetchRequest *fetchRequest = ...
// Do your fetch request... this block of code is running in the other thread
[checkerMoc fetch...];
// When the fetch request is done, do whatever you want in your UI...
dispatch_async(dispatch_get_main_queue(), ^{
// Now this code is running in the main thread... access your UI
self.myControl.enabled = fetchResultCount > 0;
});
}];
Note, you are using the same persistent store coordinator, so if the main thread tries to access the database, it will get stacked behind this request. You can also use a separate persistentStoreCoordinator for checkerMoc is this is an issue.