I've done a lot of changes to my core data model. In the past we used the simple automatic migration. However this will fail this time. Since I really don't care about the data being migrated I just want to delete the persistent store if auto migration fails and set it up again. Is this a valid way to go? Any thing I have to be careful ? Could this get my app rejected ?
There are some definite problems with doing that, and you need to be careful with it. This answer had some good advice from the NSManagedObjectContext's documentation
A context always has a “parent” persistent store coordinator which
provides the model and dispatches requests to the various persistent
stores containing the data. Without a coordinator, a context is not
fully functional. The context’s coordinator provides the managed
object model and handles persistency. All objects fetched from an
external store are registered in a context together with a global
identifier (an instance of NSManagedObjectID) that’s used to uniquely
identify each object to the external store.
When faced with a similar situation in one of our apps, I opted to make a new persistent store, and deprecate the old one because our old store had been messed up on many of our devices by a previous bad migration. It ended up being a messier transition than I had hoped, but it did work.
The problems with your plan are not insurmountable, I'm just recommending caution. I liked Giao's advice of using NSManagedObjectContext's reset. When deleting and rebuilding, the persistent store coordinator could get confused. I worry because Apple seems to be doing so many things behind the scenes. I also worry because It seems like core data behaves differently on released apps than it does on our debug versions, especially in the upgrade process.
I think you are smart in recognizing that your automigrate is going to have trouble, and that you are looking for another path. In the recent past I've seen a group that really had to scramble for a month to deal with a failed data migration in their app.
Related
I've spent the better part of a workday trying to solve this.
Background
I have a simple core data model, with books and reading sessions. The books have covers (images) that are stored as binary data with "Allows External Storage".
On iOS 11.4 and below, everything works fine all the time. When I save a new session everything gets updated properly.
Problem
Since iOS 12, when I create a new reading session and link it to the book, about every second time, core data generates a SQL statement that also updates the book cover field, sometimes resulting in a bad reference (to file on disk) which often results in the cover being nil when restarting the app, and almost always creates duplicate copy of the cover on disk (as can be seen in Simulator's _EXTERNAL_DATA folder).
In-memory context and objects remain correct though (and everything in the UI is therefore OK), until the app is restarted, then the cover is often nil.
iOS 12 specific
On iOS 12, I can deterministically reproduce the error in the simulator, on physical devices, and users have reported the error as well. I cannot reproduce the error on iOS 11.4, and no users reported the error previous to iOS 12.
Steps taken
I've enabled "-com.apple.CoreData.ConcurrencyDebug 1", so it shouldn't be that I'm accessing anything from the wrong queue. I've also enabled "-com.apple.CoreData.SQLDebug 3" so that I can see exactly what gets written.
I've made sure the Book instance (and therefore the cover) is not modified by my code before the association with the new Session by checking hasChanges, just before I do newSession.book = book and context.save().
To be 100% sure I'm not touching the cover property on any thread I've short-circuited my getters and setters for that property. No improvement.
I've tried using objectID to request an instance of the book just before the association and save. No improvement.
I've even tried the option where the context keeps strong references to all objects, just to make sure it was not some kind of memory management issue. No improvement.
Question
Any ideas for next steps?
Status update
This is a defect in iOS 12. See accepted answer below for a detailed description of a resonable workaround.
Update: The underlying Core Data issue appears to be resolved in iOS 12.1 (verified in beta 4). We will keep the workaround described below in our app, and won't be recommending using the External Storage option any time soon.
After talking to Apple engineers and filing the Radar mentioned above, we couldn’t wait around for a fix, so we took the hit and switched to storing files on the filesystem and managing it directly ourselves.
Another alternative that we considered was migrating our model not to allow External Storage for BLOBs, but I don't know what impact that would have had on performance and I was also worried about a model migration at a time when this part of iOS seems to be unstable, especially after reading stories like this in the past: Core Data: don’t store large files as binary data – Alexander Edge – Medium
It wasn't too much of a pain to implement local storage ourselves. You just need to have a unique identifier for each record that you can use to create a filename so you can map files to records. We added an extension to our Managed Object subclass with methods for reading, writing and deleting the files. Now, instead of calling e.g. article.photo = image.pngData(), we now need to call something like article.savePhoto(image.pngData()) and then we do similar when we want to retrieve the image. You can also add some code to these methods to support backwards compatibility with any images that are currently stored in Core Data.
Deletion was a little more tricky because our objects are deleted from multiple places in the code, including cascading deletes. In the end I opted to do it in the managed object's prepareForDeletion method but it is not ideal. There is plenty of discussion of how best to implement this here: cocoa - How to handle cleanup of external data when deleting unsaved Core Data objects? - Stack Overflow
Finally, to prevent our app crashing when a non-Optional binary attribute has disappeared because of this bug, I override awakeFromFetch in my Managed Object subclass to ensure that any required attributes are not nil, and if they are, I set them to a placeholder image so that they can be saved without the validation failing.
In one of my apps I would like to enable sync via CloudKit. The app itself stores the data using CoreData. I cannot use CoreData+iCloud because I need to be able to track changes which were made which is not possible with the above mentioned setup.
The obvious challenge which I have relates to the fact that CloudKit operations will be happening on the background thread and so I need to support concurrency with my CoreData stack.
Lets assume that I have some CoreData entity which I want to sync with CloudKit. To do background sync I need to create a separate managed object context. When this context fetches changes made to my entity on other devices I want to save these changes.
However, the problem is that during the execution of fetch user could have changed some attributes of a given entity and I am now stuck with this entity having some important information on both contexts. I cannot just choose between which one to save - I need to merge the changes.
Unfortunately, I do not understand which approach I need to take to be able to manually sync these changes. I have already checked the possible techniques (parent-child, multiple contexts, multiple store coordinators) but I still don't understand which approach I need to choose.
Can someone help me to resolve this confusion?
The most ideal solution for me would be such that I would get a conflict whenever sync context tries to merge changes to the main context which also had some changes. In this case I could easily decide which attributes should be updated, but I really don't know how this can be done...
In Apple WWDC 2014 they did a session on just this in effect. Here is the transcript, you can find the video easily too; assuming your Apple Developer.
http://asciiwwdc.com/2014/sessions/231
In in short an Apple engineer goes over a very simple example of using change tokens; an industry standard solution to resolving problem of sorting out data change conflicts.
The Ray Wenderlich web site also has some excellent VIDEOs on cloud kit and core data, it is a subscription service, although well worth the money at just 20$ a month, and no I have no affiliation with him; I'm just a subscriber. Google him and your find his website.
We have been trying to debug a Core Data multiple-context/threading issue wherein merging a Core Data save notification into our main thread NSManagedObjectContext is sporadically crashing the app. This is crashing ~2% of our app sessions and we are at a loss as to how to solve this. We would really appreciate any guidance or general advice on what could possibly cause this crash.
We have a Core Data setup that looks like this:
N.B. This is the default Core Data stack in Magical Record v2.3 created from [MagicalRecord setupAutoMigratingCoreDataStack]
This is the scenario where our app is crashing:
HTTP request returns JSON
JSON is parsed into NSManagedObjects (Some new entities, some updated entities) on Root Saving Context
Root Saving Context saves to persistent store
NSManagedObjectContextDidSaveNotification is broadcast by Core Data. Default context on main queue observes this and calls mergeChangesFromContextDidSaveNotification: with the NSDictionary of changes on the main thread.
It crashes when objectID is sent to an invalid object (most likely NSManagedObject has been deallocated).
This is occurring inside the private implementation of NSManagedObjectContext mergeChangesFromContextDidSaveNotification: so it is impossible for us to see what has actually gone wrong here; all we can tell at this point that an object which should exist, does not.
This only happens on a small percent of Core Data saves, indicating that may not be a fundamental flaw in our Core Data → API stack. Moreover, there is no indication that the size or type of the changes (insertions/updates/deletions) in the context changes have any impact on the likelihood of the crash.
The documentation of NSManagedObjectContextDidSaveNotification says that:
"You can pass the notification object to mergeChangesFromContextDidSaveNotification: on another thread, however you must not use the managed object in the user info dictionary directly on another thread. For more details, see Concurrency with Core Data in Core Data Programming Guide."
Maybe this is the issue? I would make sure the object u get from the notification is getting saved on Default Context on the same thread it was posted by Root.
It's been some time now since this question was posted and after rediscovering it I'd like to answer my own question for the sake of others who find this thread.
In my circumstance I had migrated a large code base from sibling NSManagedObjectContexts updated via NSManagedObjectContextDidSaveNotification's. However the problem was not really anything to do with this, even though this did expose the issue.
The real cause of this were places that there were older parts of the code, setup by previous engineers, that had setup KVO on NSManagedObjects and their properties. It transpired that KVO on Core Data entities is in fact a very very bad idea.
More accurately, it appeared that this happened when KVO was setup on entities and either the object, or the target of a relationship on this object was deleted from the NSPersistentStore. This second condition seemed to not be the only cause of the issue, but was definitely a very prominent cause in my situation.
Lesson's learnt:
Use a fetched results controller when you need to. KVO is not a convenient shortcut and you shouldn't avoid migrating dodgy Core Data KVO code to NSFetchedResultsControllers or another sensible alternative as the procrastination will just hurt you.
Multi threaded Core Data is a difficult but very worthwhile skill to become an expert in. Knowing your Core Data stack and the nuances and limitations of Core Data multithreading is absolutely worth all the mental anguish.
One possibility is that your persistent store has become corrupted and is in an inconsistent state. If this happens an error code is generated which Magical Record does not necessarily deal with. This can be the source of a number of difficult-to-repeat apparently-random crashes related to Magical Record (and may or may not be considered a Magical Record bug).
It's worth reading the Magical Record issues threads here (same issue) and here (different issue, but could be similar cause). When I hit these problems I managed to make some temporary patch fixes following various hints in those threads, but ultimately I decided to remove my dependency on Magical Record, and I have had no problems since then.
when dealing with CoreData, I've run into a few problems I'm trying to nip in the bud for future proofing the system out of the gate. The simple fact of the matter is that I've never done anything like this before (work with CoreData that is). While I've managed to figure out how to work with it in the app, I need to know a decent practice to signal an app between versions that default data needs to be refresh on first app launch.
So right now, in my AppDelegate, I setup my managed object context, and I perform a fetch request to see if there are any records at all in a particular table/entity. I only want this to happen on first launch so im not constantly rewriting the contents of the DB every app launch. Anyways, so it goes ahead and uses Object Models to handle inserting of data amongst the entities in question (theres a few)
Now, for this version of the app, it's going into the store without an API (thats a far future thing), but between versions released to the app store, we may have to update specific information within the entities (for example: prices), again I only want this refresh to happen on app launch. Also, the schema MIGHT change, Im not sure if or when, but I'd like to make sure this can accomodate that just in case.
I figured, versioning the coredata "Add Model Version" would do the trick, set the new db version as the active version, but when I launch the app in the simulator, nothing happens which tells me that the data inside is being retained.
Any help towards what it is that I should do to accomodate this would be appreciated. Thank you!
You should find the Core Data Model Versioning and Data Migration guide useful:
https://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreDataVersioning/Articles/Introduction.html
You'll also probably find Method for import initial data with coredata useful.
I created a new App using UIManagedDocument. While on my devices everything is fine, I got a lot of bad ratings, because there are problems on other devices :(
After a lot of reading and testing, I decided to go back to the traditional Core Data stack.
But what is the best way to do this with an app, that is already in the app store?
How can I build this update? What should I take care of?
Thanks,
Stefan
I think you may be better off to determine your issues with UIManagedDocument and resolve them.
However, if you want to go to plain MOC, you only have a few things to worry about. The biggest is that the UIMD stores things in a file package, and depending on your options you may have to worry about change logs.
In the end, if you want a single sqlite file, and you want to reduce the possibility of confusion, you have a class that simply opens your UIManagedDocument, and fetches each object, then replicates it in the single sqlite file for your new MOC.
Now, you should not need a different object model, so you should not have any migration issues.
Then, just delete the file package that holds the UIManagedDocument, and only use your single file sqlite store.
Basically, on startup, you try to open the UIManagedDocument. If it opens, load every object and copy it into the new database. Then delete it.
From then, you should be good to go.
Note, however, that you may now experience some UI delays because all the database IO is happening on the main UI thread. To work around this, you may need to use a separate MOC, and coordinate changes via the normal COreData notification mechanisms. There are tons of documents, examples, and tutorials on that.
EDIT
Thanks for your answer. My problem with these issues is, that I'm not
able to reproduce them. All my Devices are working fine. But I got a
lot mails, about problems like this: - duplicate entries - no data
after stoping and restarting the app - some say, that the app works
fine for some days and stops working(no new data). These are all
strange things, that don't happen on my devices. So for me the best
way is to go back to plain MOC. My DB doesn't hold many user generated
data, all the data is loaded from a webservice, so it's no problem to
delete the data and start of using a new DB. – Urkman
Duplicate entries. That one sounds like the bug related to temporary/permanent IDs. There are lots of posts about that. Here is one: Core Data could not fullfil fault for object after obtainPermanantIDs
Not saving. Sounds like you are not using the right API for UIManagedDocument. You need to make sure to not save the MOC directly, and either use an undo manager or call updateChangeCount: to notify UIManagedDocument that it has dirty data that you want to be saved. Again, lots of posts about that as well. Search for updateChangeCount.
However, you know your app best, and it may just be better and easier to use plain Core Data.
Remember, if you are doing lots of imports from the web, to use a separate MOC, and have your main MOC watch for DidSave notifications to update itself with the newly imported data.
UIManagedDocument is a special kind of document, an UIDocument subclass, that stores its data using Core Data Framework. So it combines the power of document architecture and core data capabilities.
You can read more about document based architecture from Document Based App Programming Guide for iOS and I recommend WWDC2011 Storing Documents in iCloud using iOS5 session video. I also recommend Stanford CS193P: iPad and iPhone App Development (Fall 2011) Lecture 13.
What is created when you call saveToURL:forSaveOperation:completionHandler: is an implementation detail of UIManagedDocument and UIDocument and you should not really worry or depend on it. However in current implementation a folder containing an sqlite database file is being created.
No. All entities will be contained in a single database file also more generally called: a persistent store. It is possible to use more than one persistent store, but those are more advanced use cases and UIManagedDocument currently uses one.
UIManagedDocument's context refers to a NSManagedObjectContext from underlying Core Data Framework. UIManagedDocument actually operates two of those in parallel to spin off IO operations to a background thread. When it comes to the nature of a context itself here's a quote from Core Data Programming Guide:
You can think of a managed object context as an intelligent scratch pad. When you fetch objects from a persistent store, you bring temporary copies onto the scratch pad where they form an object graph (or a collection of object graphs). You can then modify those objects however you like. Unless you actually save those changes, however, the persistent store remains unaltered.
But it really is a good idea to take a look at the lectures and other material I posted above to get a general picture of the technologies used and their potential value to you as a developer in different situations.