Part of a project I'm working in involves managing storages for products.
A Storage object contains StorageShelf objects, which define the amount of shelves. A StorageShelf, in turn, contains some other objects such as StorageLocation objects that determine the amount of products that can fit on a shelf. All These objects exist in CoreData. I also store Storages and StorageShelfs in a CloudKit database for synchronization purposes.
If I am running my app on two iPads, and add a storage with some shelves on one of the devices, it receives the notifications on the other and starts processing the updates it received.
I have a CloudOperationQueue in my app that has a maxConcurrentOperationCount of 1, in order to make sure only one update is made at the same time (This is because all objects of my app, also those out of the scope of this question are highly interdependent).
Inconsitently, this results in the following SIGABRT error message:
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Objects should not be both modified and additional'
Specifically, this error occurs when I save the context after making modifications to one of the shelf objects I received. There is no consistency in which shelf.
Question
What does this error mean, and how can one debug it?
I recently got this error, also inconsistently. The issue was connected to adding an object and then saving the CoreData context on a background thread (part of a drag and drop update), while also trying to update a view showing a list of the objects on the main thread.
The fix is to cause things to happen synchronously, for me the easy fix was moving the save operation to the main thread, like (in Swift):
DispatchQueue.main.async {
// save your context
}
I was also getting this error recently, and also inconsistently. I had several core data objects with a 1:many relation and randomly when I would add an object to that relation and save context I would receive this exception.
For me the problem turned out to be I was using the NSPersistentContainer.viewcontext
in a background thread and
to write to core data
whehn according to these docs https://developer.apple.com/documentation/coredata/nspersistentcontainer viewcontext should be read only for the main queue. Switching to NSpersistentContainer.newBackgroundContext instead fixed the issue.
Related
In my app, on button click,for every 1 second(timer1 call), I am inserting some data into the core data and for every 2 minutes(timer2 call) I am fetching inserted data to send it to the server(web service call).
Data insertion and retrieval is happening correctly but some times app is getting crashed and error says attempt to recursively call -save: on the context in core data.
is it fine to use single managedObjectContext for both fetching and insertion?
what concurrency type to be used for mangedObjectContext?(privateQueueConcurrency or mainQueueConcurrency)
[I am using Xcode 8.1 with IOS 10.1]
Hmm, you don't give us much to go on, a bit more code around how you store data might help. Are you using NSPersistentContainer? Do you wrap your operations in a context.perform() call?
To help locate where things go off the road, you might want to provide these two launching flags in your scheme:
-com.apple.CoreData.Logging.stderr 1
-com.apple.CoreData.ConcurrencyDebug 1
I'm in the middle of development of an iOS application after working quite some time with Core Data and Magical Record and having an error:
error: NULL _cd_rawData but the object is not being turned into a fault
I didn't know Core Data before this project and as it turns out I was very naive to think I can just work with Magical Record without worrying about concurrency, as I haven't dedicated any thoughts/work about managed contexts for the main thread and background threads.
After A LOT of reading about Core Data Managed Object Contexts and Magical Record, I understand that:
NSManagedObjects are not thread safe.
NSManagedObjectId IS thread safe.
I can use: Entity *localEntity = [entity MR_inContext:localContext] of Magical Record to work with an entity in a background thread's context.
I should use Magical Record's saveWithBlock:completion: and saveWithBlockAndWait: methods to get a managed context to use for background threads.
A little information regarding my application:
I'm using the latest version of Magical Record which is 2.2.
I have a backend server which my application talks to a lot.
Their communication is similar to Whatsapp, as it uses background threads for communicating with the server and updating managed objects upon successful responses.
I'm wrapping the model with DataModel objects that hold the managed objects in arrays for quick referencing of UI/background use.
Now - my questions are:
Should I fetch only from the UI thread? Is it okay that I'm holding the managed objects in DataModel objects?
How can I create a new entity from a background thread and use the newly created entity in the DataModel objects?
Is there a best design scenario that I should use? specifically when sending a request to the server and getting a response, should I create a new managed context and use it throughout the thread's activity?
Let me know if everything is clear. If not, I'll try and add clarity.
Any help or guidelines would be appreciated.
Thanks!
I'm not working with MagicalRecord, but these questions are more related to CoreData than to MagicalRecord, so I'll try to answer them :).
1) Fetching from main(UI) thread
There are many ways how to design app model, so two important things I've learned using CoreData for few years:
when dealing with UI, always fetch objects on main thread. As You correctly stated NSManagedObjects are not thread safe so you can't (well, can, but shouldn't) access their data from different thread. NSFetchedResultsController is Your best friend when you need to display long lists (eg. for messages – but watch out for request's batchSize).
you should design your storage and fetches to be fast and responsive. Use indexes, fetch only needed properties, prefetch relationships etc.
on the other hand if you need to fetch from large amount of data, you can use context on different thread and transfer NSManagedObjectIDs only. Let's say your user has huge amount of messages and you want to show him latest 10 from specific contact. You can create background context (private concurrency), fetch these 10 message IDs (NSManagedObjectIDResultType), store them in array (or any other suitable format for you), return them to your main thread and fetch those IDs only. Note that this approach speed things up if fetch takes long because of predicate/sortDescriptor not if the "problem" is in the process of turning faults into objects (eg. large UIImage stored in transformable attribute :) )
2) Creating entity in background
You can create the object in background context, store it's NSManagedObjectID after you save context (object has only temporary ID before save) and send it back to your main thread, where you can perform fetch by ID and get the object in your main context.
3) Working with background context
I don't know if it's exactly the best, but I'm pretty much satisfied with NSManagedObjectContext observation and merging from notifications. Checkout:
mergeChangesFromContextDidSaveNotification:
So, you create background context, add main context as observer for changes (NSManagedObjectContextObjectsDidChangeNotification) and background context automatically sends you notifications (every time you perform save) about all of it's changes – inserted/updated/deleted objects (no worries, you can just merge it by calling mergeChangesFromContextDidSaveNotification:). This has many advantages, such as:
everything is updated automatically (every object you have fetched in "observing context" gets updated/deleted)
every merge runs in memory (no fetches, no persisting on main thread)
if you implement NSFetchedResultsController's delegate method, everything updates automatically (not exactly everything – see below)
On the other side:
take care about merging policy (NSMangedObjectContext mergePolicy)
watchout for referencing managed objects that got deleted from background (or just another context)
NSFetchedResultsController updates only on changes to "direct" attributes (checkout this SO question)
Well, I hope it answers your questions. If everything comes up, don't hesitate to ask :)
Side note about child contexts
Also take a peek to child contexts. They can be powerful too. Basically every child context sends it's changes to parent context on save (in case of "base" context (no parent context), it sends it's changes to persistent coordinator).
For example when you are creating edit/add controller you can create child context from your main context and perform all changes in it. When user decides to cancel the operation, you just destroy (remove reference) the child context and no changes will be stored. If user decides to accept changes he/she made, save the child context and destroy it. By saving child context all changes are propagated to it's parent store (in this example your main context). Just be sure to also save the parent context (at some point) to persist these changes (save: method doesn't do the bubbling). Checkout documentation of managing parent store.
Happy coding!
We have been trying to debug a Core Data multiple-context/threading issue wherein merging a Core Data save notification into our main thread NSManagedObjectContext is sporadically crashing the app. This is crashing ~2% of our app sessions and we are at a loss as to how to solve this. We would really appreciate any guidance or general advice on what could possibly cause this crash.
We have a Core Data setup that looks like this:
N.B. This is the default Core Data stack in Magical Record v2.3 created from [MagicalRecord setupAutoMigratingCoreDataStack]
This is the scenario where our app is crashing:
HTTP request returns JSON
JSON is parsed into NSManagedObjects (Some new entities, some updated entities) on Root Saving Context
Root Saving Context saves to persistent store
NSManagedObjectContextDidSaveNotification is broadcast by Core Data. Default context on main queue observes this and calls mergeChangesFromContextDidSaveNotification: with the NSDictionary of changes on the main thread.
It crashes when objectID is sent to an invalid object (most likely NSManagedObject has been deallocated).
This is occurring inside the private implementation of NSManagedObjectContext mergeChangesFromContextDidSaveNotification: so it is impossible for us to see what has actually gone wrong here; all we can tell at this point that an object which should exist, does not.
This only happens on a small percent of Core Data saves, indicating that may not be a fundamental flaw in our Core Data → API stack. Moreover, there is no indication that the size or type of the changes (insertions/updates/deletions) in the context changes have any impact on the likelihood of the crash.
The documentation of NSManagedObjectContextDidSaveNotification says that:
"You can pass the notification object to mergeChangesFromContextDidSaveNotification: on another thread, however you must not use the managed object in the user info dictionary directly on another thread. For more details, see Concurrency with Core Data in Core Data Programming Guide."
Maybe this is the issue? I would make sure the object u get from the notification is getting saved on Default Context on the same thread it was posted by Root.
It's been some time now since this question was posted and after rediscovering it I'd like to answer my own question for the sake of others who find this thread.
In my circumstance I had migrated a large code base from sibling NSManagedObjectContexts updated via NSManagedObjectContextDidSaveNotification's. However the problem was not really anything to do with this, even though this did expose the issue.
The real cause of this were places that there were older parts of the code, setup by previous engineers, that had setup KVO on NSManagedObjects and their properties. It transpired that KVO on Core Data entities is in fact a very very bad idea.
More accurately, it appeared that this happened when KVO was setup on entities and either the object, or the target of a relationship on this object was deleted from the NSPersistentStore. This second condition seemed to not be the only cause of the issue, but was definitely a very prominent cause in my situation.
Lesson's learnt:
Use a fetched results controller when you need to. KVO is not a convenient shortcut and you shouldn't avoid migrating dodgy Core Data KVO code to NSFetchedResultsControllers or another sensible alternative as the procrastination will just hurt you.
Multi threaded Core Data is a difficult but very worthwhile skill to become an expert in. Knowing your Core Data stack and the nuances and limitations of Core Data multithreading is absolutely worth all the mental anguish.
One possibility is that your persistent store has become corrupted and is in an inconsistent state. If this happens an error code is generated which Magical Record does not necessarily deal with. This can be the source of a number of difficult-to-repeat apparently-random crashes related to Magical Record (and may or may not be considered a Magical Record bug).
It's worth reading the Magical Record issues threads here (same issue) and here (different issue, but could be similar cause). When I hit these problems I managed to make some temporary patch fixes following various hints in those threads, but ultimately I decided to remove my dependency on Magical Record, and I have had no problems since then.
I am searching for the best possible way to update a fairly large core-data based dataset in the background, with as little effect on the application UI (main thread) as possible.
There's some good material available on this topic including:
Session 211 from WWDC 2013 (Core Data Performance Optimization and Debugging, from around 25:30 onwards)
Importing Large Data Sets from objc.io
Common Background Practices from objc.io (Core Data in the Background)
Backstage with Nested Managed Object Contexts
Based on my research and personal experience, the best option available is to effectively use two separate core-data stacks that only share data at the database (SQLite) level. This means that we need two separate NSPersistentStoreCoordinators, each of them having it's own NSManagedObjectContext. With write-ahead logging enabled on the database (default from iOS 7 onwards), the need for locking could be avoided in almost all cases (except when we have two or more simultaneous writes, which is not likely in my scenario).
In order to do efficient background updates and conserve memory, one also needs to process data in batches and periodically save the background context, so the dirty objects get stored to the database and flushed from memory. One can use the NSManagedObjectContextDidSaveNotification that gets generated at this point to merge the background changes into the main context, but in general you don't want to update your UI immediately after a batch has been saved. You want to wait until the background job is completely done and than refresh the UI (recommended in both the WWDC session and objc.io articles). This effectively means that the application main context remains out of sync with the database for a certain time period.
All this leads me to my main question, which is, what can go wrong, if I changed the database in this manner, without immediately telling the main context to merge changes? I'm assuming it's not all sunshine an roses.
One specific scenario that I have in my head is, what happens if a fault needs to be fulfilled for an object loaded in the main context, if the background operation has in between deleted that object from the database? Can this for instance happen on a NSFetchedResultsController based table view that uses a batchSize to fetch objects incrementally into memory? I.e., an object that has not yet been fully fetched gets deleted but than we scroll up to a point where the object needs to get loaded. Is this a potential problem? Can other things go wrong? I'd appreciate any input on this matter.
Great question!
I.e., an object that has not yet been fully fetched gets deleted but
than we scroll up to a point where the object needs to get loaded. Is
this a potential problem?
Unfortunately it'll cause problems. A following exception will be thrown:
Terminating app due to uncaught exception 'NSObjectInaccessibleException', reason: 'CoreData could not fulfill a fault for '0xc544570 <x-coredata://(...)>'
This blog post (section titled "How to do concurrency with Core Data?") might be somewhat helpful, but it doesn't exhaust this topic. I'm struggling with the same problems in an app I'm working on right now and would love to read a write-up about it.
Based on your question, comments, and my own experience, it seems the larger problem you are trying to solve is:
1. Using an NSFetchedResultsController on the main thread with thread confinement
2. Importing a large data set, which will insert, update, or delete managed objects in a context.
3. The import causes large merge notifications to be processed by the main thread to update the UI.
4. The large merge has several possible effects:
- The UI gets slow, or too busy to be usable. This may be because you are using beginUpdates/endUpdates to update a tableview in your NSFetchedResultsControllerDelegate, and you have a LOT of animations queing up because of the large merge.
- Users can run into "Could not fulfill fault" as they try to access a faulted object which has been removed from the store. The managed object context thinks it exists, but when it goes to the store to fulfill the fault the fault it's already been deleted. If you are using reloadData to update a tableview in your NSFetchedResultsControllerDelegate, you are more likely to see this happen than when using beginUpdates/endUpdates.
The approach you are trying to use to solve the above issues is:
- Create two NSPersistentStoreCoordinators, each attached to the same NSPersistentStore or at least the same NSPersistentStore SQLite store file URL.
- Your import occurs on NSManagedObjectContext 1, attached to NSPersistentStoreCoordinator 1, and executing on some other thread(s). Your NSFetchedResultsController is using NSManagedObjectContext 2, attached to NSPersistentStoreCoordinator 2, running on the main thread.
- You are moving the changes from NSManagedObjectContext 1 to 2
You will run into a few problems with this approach.
- An NSPersistentStoreCoordinator's job is to mediate between it's attached NSManagedObjectContexts and it's attached stores. In the multiple-coordinator-context scenario you are describing, changes to the underlying store by NSManagedObjectContext 1 which cause a change in the SQLite file will not be seen by NSPersistentStoreCoordinator 2 and it's context. 2 does not know that 1 changed the file, and you will have "Could not fulfill fault" and other exciting exceptions.
- You will still, at some point, have to put the changed NSManagedObjects from the import into NSManagedObjectContext 2. If these changes are large, you will still have UI problems and the UI will be out of sync with the store, potentially leading to "Could not fulfill fault".
- In general, because NSManagedObjectContext 2 is not using the same NSPersistentStoreCoordinator as NSManagedObjectContext 1, you are going to have problems with things being out of sync. This isn't how these things are intended to be used together. If you import and save in NSManagedObjectContext 1, NSManagedObjectContext 2 is immediately in a state not consistent with the store.
Those are SOME of the things that could go wrong with this approach. Most of these problems will become visible when firing a fault, because that accesses the store. You can read more about how this process works in the Core Data Programming Guide, while the Incremental Store Programming Guide describes the process in more detail. The SQLite store follows the same process that an incremental store implementation does.
Again, the use case you are describing - getting a ton of new data, executing find-Or-Create on the data to create or update managed objects, and deleting "stale" objects that may in fact be the majority of the store - is something I have dealt with every day for several years, seeing all of the same problems you are. There are solutions - even for imports that change 60,000 complex objects at a time, and even using thread confinement! - but that is outside the scope of your question.
(Hint: Parent-Child contexts don't need merge notifications).
Two Persistent Store Coordinators (pscs) is certainly the way to go with large datasets. File locking is faster than the locking within core data.
There's no reason you couldn't use the background psc to create thread confined NSManagedObjectContexts in which each is created for each operation you do in the background. However, instead of letting core data manage the queueing you now need to create NSOperationQueues and/or threads to manage the operations based on what you're doing in the background. NSManagedObjectContexts are free and not expensive. Once you do this you can hang onto your NSManagedObjectContext and only use it during that one operation and/or threads life time and build as many changes as you want and wait until the end to commit them and merge them to the main thread how ever you decide. Even if you have some main thread writes you can still at crucial points in your operations life time refetch/merge back into your threads context.
Also it's important to know that if you're working on large sets of data don't worry about merging contexts so as long as you aren't touching something else. For example if you have class A and class B and you have two seperate opertions/threads to work on them and they have no direct relationship you do not have to merge the contexts if one changes you can keep on rolling with the changes. The only major need for merging background contexts in this fashion is if there are direct relationships faulting. It would be better to prevent this though through some sort of serialization whether it be NSOperationQueue or what ever else. So feel free to work away on different objects in the background just be careful about their relationships.
I've worked on a large scale core data projects and had this pattern work very well for me.
Indeed, this is the best core data scenario you can work with. Almost no Main UI staleness, and easy background management of your data. When you want to tell the Main Context (and maybe a currently running NSFetchedResultsController) you listen for save notifications of the backgroundContext like this:
[[NSNotificationCenter defaultCenter]
addObserver:self selector:#selector(reloadFetchedResults:)
name:NSManagedObjectContextDidSaveNotification
object:backgroundObjectContext];
Then, you can merge changes, but waiting for the Main Thread context to catch them before saving. When you receive the mergeChangesFromContextDidSaveNotification notification the changes are not yet saved. Hence the performBlockAndWait is mandatory, so the Main context gets the changes and then the NSFetchedResultsController updates its values correctly.
-(void)reloadFetchedResults:(NSNotification*)notification
{
NSManagedObjectContext*moc=[notification object];
if ([moc isEqual:backgroundObjectContext])
{
// Delete caches of fethcedResults if you have a deletion
if ([[theNotification.userInfo objectForKey:NSDeletedObjectsKey] count]) {
[NSFetchedResultsController deleteCacheWithName:nil];
}
// Block the background execution of the save, and merge changes before
[managedObjectContext performBlockandWait:^{
[managedObjectContext
mergeChangesFromContextDidSaveNotification:notification];
}];
}
}
There is a pitfall no one has noticed. You can get the save notification before the background context has actually saved the object you want to merge. If you want to avoid problems by a faster Main Context asking for an object that has not been saved yet by the background context, you should (you really should) call obtainPermanentIDsForObjects before any background save. Then you are safe to call the mergeChangesFromContextDidSaveNotification. This will ensure that the merge receives a valid permanent Id for merging.
I'm currently debugging a big project written by another developer. The project uses CoreData which i am very new to. I'm having a crash which happens due to the fact that some NSManagedObject is being a fault(i have poor understanding of what fault is) and i would like to convert the object into "not fault" and see if it helps. Reading documentation made me think that to make object not fault is equal to to fire fault (again i have poor understanding of what is "to fire"), so i decided to fire it by calling any method on it, and this method is hasChanges since it's not in the list of methods which don't fire fault. However even after calling this method the object still remained being fault. Can anyone give me an example of how to convert NSManagedObject into "not fault" state?
The exception you mentioned in a comment is:
Terminating app due to uncaught exception 'NSObjectInaccessibleException', reason: 'CoreData could not fulfill a fault for '0x1f0627a0 <x-coredata://E40418A0-A8E5-4340-865F-A9DA2E0095DD/CoreObject/p288>''
The "could not fulfill a fault" message indicates that this is not simply a problem with firing the fault. In fact there are no special steps to fire a fault-- you just access the attributes, and if necessary, the fault fires automatically.
What this error is telling you is that you're doing something that causes a fault to fire, but that Core Data can't find any record of the instance you're using. This can happen in certain scenarios, for example:
Fetch an object, and leave it as a fault (i.e. don't access any attributes)
Delete it from Core Data, but keep a reference to the object (maybe in an instance variable)
Save changes
Try to access an attribute on the object you fetched in step 1.
At this point Core Data would normally use the object to look up the attribute value. But you already deleted it, so the attribute data is gone. Core Data throws this exception.
There are some other scenarios that can trigger this-- calling reset on a managed object context while keeping previously fetched objects around, or removing the persistent store (so that the data still exists in the store file, but the file isn't loaded anymore). In general it means Core Data is trying to look up data on an object that's no longer valid.
What you need to do:
Figure out which object is causing this. Setting an exception breakpoint, so that the debugger loads just as the crash is happening, is probably a good idea.
Get rid of that object. Preventing the fault from firing might prevent this specific crash, but as long as you still have this object, it's like a land mine in your app. It will explode and crash the app as soon as you touch it.
Figure out why you have invalid managed objects lurking in memory. Maybe you should have gotten rid of them earlier? Maybe you're accidentally deleting something you don't want to delete? For some reason you're keeping managed objects around when they're no longer valid. This is the core problem that's causing your trouble.
Faulting is the process whereby an object and/or its properties are fetched from the database on-demand.
For example, if you attempt to access a property or relation on some object, person.name, that object may not have its data in-memory, and it has to be fetched from the underlying data store. This is a fault.
Faulting is a normal part of how Core Data works, and should not be related to crashes. An object will be unfaulted when you try to access its properties and relations.