NSPersistentContainer concurrency for saving to core data - ios

I've read some blogs on this but I'm still confused on how to use NSPersistentContainer performBackgroundTask to create an entity and save it. After creating an instance by calling convenience method init(context moc: NSManagedObjectContext) in performBackgroundTask() { (moc) in } block if I check container.viewContext.hasChanges this returns false and says there's nothing to save, if I call save on moc (background MOC created for this block) I get errors like this:
fatal error: Failure to save context: Error Domain=NSCocoaErrorDomain Code=133020 "Could not merge changes." UserInfo={conflictList=(
"NSMergeConflict (0x17466c500) for NSManagedObject (0x1702cd3c0) with objectID '0xd000000000100000 <x-coredata://3EE6E11B-1901-47B5-9931-3C95D6513974/Currency/p4>' with oldVersion = 1 and newVersion = 2 and old cached row = {id = 2; ... }fatal error: Failure to save context: Error Domain=NSCocoaErrorDomain Code=133020 "Could not merge changes." UserInfo={conflictList=(
"NSMergeConflict (0x170664b80) for NSManagedObject (0x1742cb980) with objectID '0xd000000000100000 <x-coredata://3EE6E11B-1901-47B5-9931-3C95D6513974/Currency/p4>' with oldVersion = 1 and newVersion = 2 and old cached row = {id = 2; ...} and new database row = {id = 2; ...}"
)}
So I've failed to get the concurrency working and would really appreciate if someone could explain to me the correct way of using this feature on core data in iOS 10

TL:DR: Your problem is that you are writing using both the viewContext and with background contexts. You should only write to core-data in one synchronous way.
Full explanation: If an object is changed at the same time from two different contexts core-data doesn't know what to do. You can set a mergePolicy to set which change should win, but that really isn't a good solution, because you will lose data that way. The way that a lot of pros have been dealing with the problem for a long time was to have an operation queue to queue the writes so there is only one write going on at a time, and have another context on the main thread only for reads. This way you never get any merge conflicts. (see https://vimeo.com/89370886#t=4223s for a great explanation on this setup).
Making this setup with NSPersistentContainer is very easy. In your core-data manager create a NSOperationQueue
//obj-c
_persistentContainerQueue = [[NSOperationQueue alloc] init];
_persistentContainerQueue.maxConcurrentOperationCount = 1;
//swift
let persistentContainerQueue = OperationQueue()
persistentContainerQueue.maxConcurrentOperationCount = 1
And do all writing using this queue:
// obj c
- (void)enqueueCoreDataBlock:(void (^)(NSManagedObjectContext* context))block{
void (^blockCopy)(NSManagedObjectContext*) = [block copy];
[self.persistentContainerQueue addOperation:[NSBlockOperation blockOperationWithBlock:^{
NSManagedObjectContext* context = self.persistentContainer.newBackgroundContext;
[context performBlockAndWait:^{
blockCopy(context);
[context save:NULL]; //Don't just pass NULL here, look at the error and log it to your analytics service
}];
}]];
}
//swift
func enqueue(block: #escaping (_ context: NSManagedObjectContext) -> Void) {
persistentContainerQueue.addOperation(){
let context: NSManagedObjectContext = self.persistentContainer.newBackgroundContext()
context.performAndWait{
block(context)
try? context.save() //Don't just use '?' here look at the error and log it to your analytics service
}
}
}
When you call enqueueCoreDataBlock the block is enqueued to ensures that there are no merge conflicts. But if you write to the viewContext that would defeat this setup. Likewise you should treat any other contexts that you create (with newBackgroundContext or with performBackgroundTask) as readonly because they will also be outside of the writing queue.
At first I thought that NSPersistentContainer's performBackgroundTask had an internal queue, and initial testing supported that. After more testing I saw that it could also lead to merge conflicts.

Related

Core Data: having issue inserting data with relationship

I have two entities (Categories, Event) and they have two-way many-to-many relationship. A Category can have more than one Event and an Event can have more than one Category. Category to Event relationship is optional in a sense that Category can exist without an Event, but Event to Category relationship is mandatory (Event cannot exist without Category). I'm trying to insert events and adding categories to them but I'm getting NSValidationErrorValue=Relationship error. This is my code :
private func storeEventsXMLStream(_ xml: XMLIndexer) {
let managedObjectContext = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.privateQueueConcurrencyType)
managedObjectContext.persistentStoreCoordinator = (UIApplication.shared.delegate as! AppDelegate).persistentContainer.persistentStoreCoordinator
// Remove all data before inserting
// This line of code is necessary because data needs to be downloaded on daily basis.
// Otherwise, I will get redundant data.
removeAllExistingData("Event_Ottawa", managedObjectContext: managedObjectContext)
autoreleasepool { // Scoping is necessary to fix memory leak
for xmlcat in xml["events"]["event"]{
let event = NSEntityDescription.insertNewObject(forEntityName: "Event_Ottawa", into: managedObjectContext) as! Event_Ottawa
event.id = Int32((xmlcat.element?.attribute(by: "id")?.text)!)!
event.website_url_english = xmlcat["website_url_english"].element?.text
event.website_url_french = xmlcat["website_url_french"].element?.text
// setting other attributes of events here. Exactly like I did in above 3 line
// Just another attribute. Storing it a String in Coredata
var recur_rules = ""
for rule in xmlcat["recur_rules"]["recur_rule"] {
recur_rules += (rule.element?.attribute(by: "weekday")?.text)!
}
if !recur_rules.isEmpty {
event.recur_rules = recur_rules
}
do {
var predicateArray:[NSPredicate] = []
// Categories are inserted to the Coredata before this method call. So I'm fetching the applicable one here.
for category in xmlcat["categories"]["category"] {
let predicate = NSPredicate(format: "id = %#", (category.element?.attribute(by: "id")?.text)!)
predicateArray.append(predicate)
}
let requestCategory:NSFetchRequest<Category_Event_Ottawa> = Category_Event_Ottawa.fetchRequest()
requestCategory.predicate = NSCompoundPredicate.init(andPredicateWithSubpredicates: predicateArray)
let managedContext = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.privateQueueConcurrencyType)
managedContext.persistentStoreCoordinator = (UIApplication.shared.delegate as! AppDelegate).persistentContainer.persistentStoreCoordinator
let applicableCategories = try managedContext.fetch(requestCategory)
for category in applicableCategories {
event.addToCategory(category)
}
}
} catch {
print(error)
}
}
}
// only save once per batch insert
do {
try managedObjectContext.save()
} catch {
print(error)
}
managedObjectContext.reset() <-- I get EXC_BAD_ACCESS, when I use the same object context for fetching as insertion
}
And the error I receive :
Error Domain=NSCocoaErrorDomain Code=1560 "(null)" UserInfo={NSDetailedErrors=(
"Error Domain=NSCocoaErrorDomain Code=1550 \"The operation couldn\U2019t be completed. (Cocoa error 1550.)\" UserInfo={Dangling reference to an invalid object.=null, NSValidationErrorValue=Relationship 'category' on managed object (0x6100050963f0)
If use the same managed object context for both insertion and fetching, I don't get this error anymore. But I get EXC_BAD_ACCESS from the line that reset context object. I learnt it from another post that CoreData is not thread safe. So maybe that was an issue. But how do I resolve this issue? In case it's relevant, Deletion Rule for Event-Category relationship is Nullify and Category-Event is Cascade.
In this line:
requestCategory.predicate = NSCompoundPredicate.init(andPredicateWithSubpredicates: predicateArray)
you're creating a compound predicate with ANDs. You're asking Core Data to fetch you all of the categories that have id = 7 AND id = 8 AND ... etc. That's not going to work. The category can only have a single id. You actually want an 'or' predicate in this case.
However, I think the better way to do this is to load all of your categories into a dictionary keyed by their id before you start looping through your XML, and then just pull them out of the dictionary. That will be much more performant than fetching each time.
Also, you can't fetch categories in a separate context and then create relationships between objects from separate contexts. Core Data will crash if you try it.

Crashes during CoreData fetching on serial queue

I went through many discutions and subjects about CoreData, but I keep getting the same problem.
Here's the context : I have an application which have to do several access to CoreData. I decided, for simplifying, to declare a serial thread specifically for access (queue.sync for fetching, queue.async for saving). I have a structure that is nested three times, and for recreating the entire structure, I fetch subSubObject, then SubObject and finally Object
But sometimes (like 1/5000 recreation of "Object") CoreData crash on fetching results, with no stack trace, with no crash log, only a
EXC_BAD_ACCESS (code 1)
Objects are not in cause, and the crash is weird because all access are done in the same thread which is a serial thread
If anyone can help me, I will be very grateful !
Here's the structure of the code :
private let delegate:AppDelegate
private let context:NSManagedObjectContext
private let queue:DispatchQueue
override init() {
self.delegate = (UIApplication.shared.delegate as! AppDelegate)
self.context = self.delegate.persistentContainer.viewContext
self.queue = DispatchQueue(label: "aLabel", qos: DispatchQoS.utility)
super.init()
}
(...)
public func loadObject(withID ID: Int)->Object? {
var object:Object? = nil
self.queue.sync {
let fetchRequest = NSFetchRequest<NSManagedObject>(entityName: "Name")
fetchRequest.predicate = NSPredicate(format: "id == %#", NSNumber(value: ID))
do {
var data:[NSManagedObject]
// CRASH HERE ########################
try data = context.fetch(fetchRequest)
// ###################################
if (data.first != nil) {
let subObjects:[Object] = loadSubObjects(forID: ID)
// Task creating "object"
}
} catch let error as NSError {
print("CoreData : \(error), \(error.userInfo)")
}
}
return object
}
private func loadSubObjects(forID ID: Int)->[Object] {
var objects:[Object] = nil
self.queue.sync {
let fetchRequest = NSFetchRequest<NSManagedObject>(entityName: "Name")
fetchRequest.predicate = NSPredicate(format: "id == %#", NSNumber(value: ID))
do {
var data:[NSManagedObject]
// OR HERE ###########################
try data = context.fetch(fetchRequest)
// ###################################
if (data.first != nil) {
let subSubObjects:[Object] = loadSubObjects(forID: ID)
// Task creating "objects"
}
} catch let error as NSError {
print("CoreData : \(error), \(error.userInfo)")
}
}
return objects
}
(etc...)
TL;DR: get rid of your queue, replace it with an operation queue. Run fetches on the main thread with viewContext do writing in one synchronous way.
There are two issues. First is that managedObjectContexts are not thread safe. You cannot access a context (neither for reading or for writing) except from the single thread that it is setup to work with. The second issue is that you shouldn't be doing multiple writing to core-data at the same time. Simultaneous writes can lead to conflicts and loss of data.
The crash is cause by accessing viewContext from a thread that is not the main thread. The fact that there is a queue ensuring that nothing else is accessing core data at the same time doesn't fix it. When core-data thread safety is violated core-data can fail at any time and in any way. That means that it may crash with hard to diagnose crash reports even at points in the code where you are on the correct thread.
You have the right idea that core-data needs a queue to work well when saving data, but your implementation is flawed. A queue for core-data will prevent write conflicts caused by writing conflicting properties to an entity at the same time from different context. Using NSPersistentContainer this is easy to set up.
In your core-data manager create a NSOperationQueue
let persistentContainerQueue : OperationQueue = {
let queue = OperationQueue.init();
queue.maxConcurrentOperationCount = 1;
return queue;
}()
And do all writing using this queue:
func enqueueCoreDataBlock(_ block: #escaping (NSManagedObjectContext) -> Swift.Void){
persistentContainerQueue.addOperation {
let context = self.persistentContainer.newBackgroundContext();
context.performAndWait {
block(context)
do{
try context.save();
} catch{
//log error
}
}
}
}
For writing use enqueueCoreDataBlock: which will give you a context to use and will execute every block inside the queue so you don't get write conflicts. Make sure that no managedObject leave this block - they are attached to the context which will be destroyed at the end of the block. Also you can't pass managedObjects into this block - if you want to change a viewContext object you have to use the objectID and fetch in the background context. In order for the changes to be seen on the viewContext you have to add to your core-data setup persistentContainer.viewContext.automaticallyMergesChangesFromParent = true
For reading you should use the viewContext from the main thread. As you are generally reading in order to display information to the user you aren't gaining anything by having a different thread. The main thread would have to wait for the information in any event so it is faster just the run the fetch on the main thread. Never write on the viewContext. The viewContext does not use the operation queue so writing on it can create write conflicts. Likewise you should treat any other contexts that you create (with newBackgroundContext or with performBackgroundTask) as readonly because they will also be outside of the writing queue.
At first I thought that NSPersistentContainer's performBackgroundTask had an internal queue, and initial testing supported that. After more testing I saw that it could also lead to merge conflicts.

Changes on Background NSManagedObjectContext not visible on Main, using NSFetchedResultsController

This is a really bizarre issue, and I thought I understood Core Data.
I use a background context that has no parent. Hooked right into the Persistent Store Coordinator. I update objects on this background context then save it. I listen to the ContextDidSaveNotification and merge those changes into my main thread context. Those updated objects are not faults on the main thread as they are already used to populate table view cells. So I would expect those changes to actually merge. But they are not.
Without getting into the details of my data models, it suffices to say that an object has a property "downloadState". Once the parsing work is done on the background thread, downloadStateValue (an enum) gets set to "3", which corresponds to 'completed'.
I subscribe to the ContentWillSave notification now to inspect what's going on. I get this at the end of my parsing work:
2016-06-13 10:19:21.055 MyApp[29162:52855206] Going to save background context.
updated:{(
<QLUserPinnedCourse: 0x7fe195403c10> (entity: QLUserPinnedCourse; id: 0xd0000000002c0002 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLUserPinnedCourse/p11> ; data: {
course = "0xd000000000dc0008 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLCourse/p55>";
courseId = 2794;
/* other fields redacted */
}),
<QLCourse: 0x7fe1954cded0> (entity: QLCourse; id: 0xd000000000dc0008 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLCourse/p55> ; data: {
/* other fields redacted*/
contentDownloadState = 3;
courseId = 2794;
pinnedUserData = "0xd0000000002c0002 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLUserPinnedCourse/p11>";
})
The NSFetchedResultsController that is listenting to QLUserPinnedCourse objects gets the delegate calls, which triggers cell reloads in my tables.
The predicate is:
// Specify criteria for filtering which objects to fetch
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"pinned == %# && course.contentDownloadState IN %#",
#YES,
#[#(QLDownloadStateSucceeded), #(QLDownloadStateNotYetAttempted), #(QLDownloadStateFailed), #(QLDownloadStateIncomplete)]
];
Now when I get to the cell code, I have a QLUserPinnedCourse object to work with. I set a breakpoint in the debugger and get:
(lldb) po userCourse.course
<QLCourse: 0x7fe19568f740> (entity: QLCourse; id: 0xd000000000dc0008 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLCourse/p55> ; data: {
contentDownloadState = 1;
courseId = 2794;
pinnedUserData = "0xd0000000002c0002 <x-coredata://95821ADC-8A1F-4DAC-B20C-EDD8F8F413EA/QLUserPinnedCourse/p11>";
})
The question is, WHY is contentDownloadState not 3, but still 1 ?? I don't get it.
Shouldn't these changes have been merged??
Details as to my stack:
PSC -> Private Concurrent (saving context) -> Main Thread context
PSC -> Private Concurrent (import context)
ContextDidSave:
if the context was an import context, merge changes into both contexts above:
_contextSaveObserver = [[NSNotificationCenter defaultCenter] addObserverForName:NSManagedObjectContextDidSaveNotification
object:nil
queue:nil
usingBlock:^(NSNotification* note)
{
NSManagedObjectContext *contextSaved = note.object;
NSManagedObjectContext *moc = weakself.mainQueueContext;
// basically, if this was a background worker thread
DDLogDebug(#"updatedObjects:%#", note.userInfo[NSUpdatedObjectsKey]);
if ([contextSaved.userInfo[CoreDataUserInfoKeyIsWorkerContext] boolValue])
{
[weakself.privateSavingContext performBlock:^(){
for (NSManagedObject *object in note.userInfo[NSUpdatedObjectsKey]) {
[[weakself.privateSavingContext objectWithID:[object objectID]] willAccessValueForKey:nil];
}
[weakself.privateSavingContext mergeChangesFromContextDidSaveNotification:note];
[moc performBlock:^(){
for (NSManagedObject *object in note.userInfo[NSUpdatedObjectsKey]) {
[[moc objectWithID:[object objectID]] willAccessValueForKey:nil];
}
[moc mergeChangesFromContextDidSaveNotification:note];
}];
}];
}
}];
Note that I'm asking the userCourse.course object for its attribute, although my FRC is interested in QLUserPinnedCourse objects. I thought because I specify a keypath in the predicate that relates to a QLCourse object, these changes are refreshed.
It's a quirk of Core Data. You actually need to re-fault objects in the main context which were updated by the save operation.
Here's an example in Swift:
mainContext.performBlock {
let updatedObjects : Set<NSManagedObject> = notification.userInfo![NSUpdatedObjectsKey] as! Set<NSManagedObject>
for obj in updatedObjects {
self.mainContext.objectWithID(obj.objectID).willAccessValueForKey(nil)
}
self.mainContext.mergeChangesFromContextDidSaveNotification(notification)
}
The main part is the call to willAccessValueForKey:nil, which causes the object to be marked as a fault. This will cause NSFetchedResultsControllers in the main context to fire.
So I found a solution but I can't tell you why it works.
The problem I suppose is that I would have a method 'start downloading content' and I would update the property contentDownloadState on the main thread context to 'incomplete/downloading', then proceed to get all the content.
All the rest of the work was done on a background thread context. When finished I updated that value with 'succeeded'. It wasn't merging that change. I have no idea why.
Once I decided to do everything on the worker context. i.e. change its value then save the context to disk, the changes, ALL the changes were propagating.
So in the end I solved it, but really don't understand the problem.

How should I correctly manage a full Core Data stack in private queue?

I handle two complete Core Data stacks in my app:
The one that is provided by default in AppDelegate.
A second stack I fully create in order to perform NSManagedObject updates in a private queue, to avoid blocking the UI.
I have a class to create the second "auxiliary" Core Data stack, and I do this way:
class CoreDataStack: NSObject {
class func getPrivateContext() -> NSManagedObjectContext {
let bundle = NSBundle.mainBundle()
let modelURL = bundle.URLForResource("MyApp", withExtension: "momd")
let model = NSManagedObjectModel(contentsOfURL: modelURL!)!
let psc = NSPersistentStoreCoordinator(managedObjectModel: model)
let privateContext = NSManagedObjectContext(concurrencyType: NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
privateContext.persistentStoreCoordinator = psc
let documentsURL = CoreDataStack.applicationDocumentsDirectory()
let storeURL = documentsURL.URLByAppendingPathComponent("MyApp.sqlite")
let options = [NSMigratePersistentStoresAutomaticallyOption: true]
var error: NSError? = nil
let store: NSPersistentStore?
do {
store = try psc.addPersistentStoreWithType(NSSQLiteStoreType, configuration: nil, URL: storeURL, options: options)
} catch let error1 as NSError {
error = error1
store = nil
}
if store == nil {
print("Error adding persistent store: \(error)")
abort()
}
return privateContext
}
class func applicationDocumentsDirectory() -> NSURL {
let fileManager = NSFileManager.defaultManager()
let urls = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask)
return urls[0]
}
}
I need to firstly confirm/clarify some points:
A) Is it legal/correct to create a full Core Data stack to use a context in a private queue the way I'm doing?
B) Would creating even a new NSManagedObjectModel from the same resource and same .sqlite file than the one used in the AppDelegate default Core Data stack cause problems?
About managing both contexts I have (the default in AppDelegate, let's call it mainContext, and the one I create in a private queue, let's call it privateContext):
The mainContext is intended to show the NSManagedObject information throughout the app.
The privateContext is intended to be used to call web services to get updated data, create the new NSManagedObject with the received information, and compare this new objects with the ones the app already have.
My questions regarding this are:
Should the privateContext be always used by calling performBlock or performBlockAndWait? Does that include all related operations, such s reading/inserting objects to the privateContext, and clearing/saving it?
The mainContext is supposed to be associated to the main queue/thread, right? So then all its related operations should be performed in main thread...
Having into account that privateContext has its own full Core Data stack... if I save its objects, would they be stored at the same .sqlite file than the ones when saving the mainContext? Or would such file be some way "duplicated"?
If privateContext should save data from its private queue, and mainContext should be used in main thread, would it cause any problem to fetch from the mainContext the data that was saved from the privateContext?
I need help to understand and correctly manage Core Data concurrency in my app, I'm making a mess with all this persistence staff and I'm occasionally finding errors in operations that seemed to work... thanks so much in advance.

Magical Record add object, different context error

I'm using Magical Record in my app, and want to add the functionality for a user to add a 'Note', which is a child of 'entry'.
I added this code:
[MagicalRecord saveWithBlock: ^(NSManagedObjectContext *localContext) {
Note *newNote = [Note MR_createInContext: localContext];
newNote.content = noteContent;
newNote.name = #"User Note";
[self.entry addNotesObject: newNote];
}
completion: ^(BOOL success, NSError *error) {
if (error != nil)
{
// show alert
}
else if (success)
{
[[self tableView] reloadData];
}
}];
The error I keep getting on the last line is "Illegal attempt to establish a relationship 'entry' between objects in different contexts"
I tried setting the context of both 'entry' and 'newNote' to 'localContext', but I still get the same error.
What am I missing?
self.entry was created in different context, so you can't access it from this one.
Instead of:
[self.entry addNotesObject: newNote];
you should first find self.entry object in localContext:
[[self.entry MR_inContext:localContext] addNotesObject: newNote];
You can find an explanation of using MagicalRecord in a concurrent environment at Performing Core Data operations on Threads. Though it's quite short, so in my opinion it's worthwhile to read Core Data Programming Guide even though you don't use CD directly.

Resources