i want to add tables/columns to a database during runtime.
Currently I'm using Core Data.
I know that there's a possibility to do so in XCode (add new data model version), but I definitely can't use that way, because I receive the database schema from a web service.
Is there any good possibility to run ddl commands during runtime when using Core Data, or is it just possible with directly using sqlite (or a wrapper/ormapper)?
If it's better to use a wrapper/ormapper please give me some suggestions about which should be used in this case.
Workflow should be:
start app
check if database is up to date
if new version of schema is available from a web service do DDL commands
continue with app workflow
PS: Please no answers which describe alternatives modifying the schema with XCode!
Can you modify the Core Data model at run time? Yes...but, it probably won't work the way you want it to work.
Core Data's API makes it possible to construct or modify every detail of a data model at run time. Xcode's model editor is a convenience, but you could skip it and do everything in code if you wanted. For example, NSEntityDescription's properties attribute (which covers both attributes and relationships) is writeable. You could create a new NSAttributeDescription and update the entity's properties to contain it. Bang, you just added a new attribute to the entity. Similarly, NSManagedObjectModel's entities property is writeable, so you could create a new NSEntityDescription and add it to the model. That gives you a new entity, created at run time.
But, and it's a big one: you can only do this before you load the data store. Once you load your persistent store, altering the model will throw an exception. When Core Data loads a persistent store, it compares the model file to the model used in the store file. They must match, and you can't do anything to change this fact after loading the store. Once you load the store, the model is fixed.
What's more, even if you modify your model before loading the persistent store, you can only load persistent stores that match the current version of the model-- unless, that is, you also write code to migrate the persistent store to the new model. How hard that is depends on the nature of the changes. At a minimum then, you would need to make any changes before loading previously saved data, and then also arrange to do model migration to update the persistent store to use the new model.
With Core Data the model (schema) and data are stored separately and matched up when the store is loaded. That's not how SQLite works internally but it's the approach that Core Data enforces.
Related
I am currently doing some tests with Ensembles, specifically testing Core Data light migration.
My current configuration is as follow:
Device-A running my app with data model 1
Device-B running my app with data model 2
data model 2 is based on data model 1 with one additional string property, which is optional
My scenario is as follow:
At the beginning, running my app with data model 1 on both Device-A, and Device-B, everything synced fine using Ensembles (iCloud configuration)
On Device-B, install and run my updated app using data model 2
On Device-A, keep running my old app using data model 1, and add a new record
The result: the new record added on Device-A is uploaded to iCloud and then synced to device-B
My question: can I configure Ensembles to prevent it from uploading changes to iCloud in case that related data model is not the latest one? (i.e. in my case, Device-A uploads an object based on data model 1 while iCloud is already based on data model 2)
Thanks in advance!
UPDATE 1:
Drew, thank you very much for your answer. I definitely agree that uploads can't (and probably shouldn't) be prevented as Ensembles is a decentralised, peer-to-peer system.
In the ideal case, I would like that the device with the new data model will ignore data that is based on the old data model. (in a similar way to the existing behavior where the device with the old data model will ignore any data based on the new data model). Is that supported?
If not, please consider the following scenario as an example:
The old data model has an entity called 'Book' with two properties: title, and author (both fields are non optional)
The new data model has a new optional property called titleFirstLetter that should hold the first letter of the title field.
Currently, when Ensembles is not involved, I have full control when saving new NSManagedObject to the persistence store. Therefore, the updated code of my app which responsible for adding a new book, will make sure to extract the first letter from the title field and save it to the new titleFirstLetter property. (i.e. a book titled Catch-22 will have C in the titleFirstLetter property when book is saved).
In addition, when light migration occurs on the core data stack, I detect that, and perform a one-time procedure where I iterate all existing books in the database, and set the titleFirstLetter according to the title value. From this point and on, the database is consistent and valid, while the new code will ensure that future books added to the database will keep database valid.
Regarding Ensembles, if I don't have any control on old data coming from devices with older data model, how can I fill the new property of titleFirstLetter, if my code is never being called?
Thank you for your kind assistance!
You can't prevent it, no. Ensembles is a decentralised, peer-to-peer system. There is really no way for one device to know the current state of another device, so you couldn't prevent an upload.
The updated device should be capable of handling the old data from the other device. The device with the old model will ignore any data based on the new model, until it too is updated. Then it will merge all of that ignored data.
It is best to avoid migrations where possible, and stick to simple stuff like adding properties or entities, rather than tricky refactors. If you need to make a lot of changes, consider simply starting with a new ensemble (e.g. change the ensembles identifier).
Our app is having a coredata store which is based on a single coredata model. There are read-only data as well as read-write data.
The read-only data are pre-loaded and bundled along with the app, so that on a fresh install, this database is copied to the application sandbox and from there on the data base will be updated via webservice (ie only the changed data will get updated from webservice so that less data is transferred).
Now we have situation where we need to add more attributes to the read-only entities.
Light weight migration will help in upgrading the schema easily, but the problem is about the new data, since we are adding new attributes to all the read-only entities, all the data records are changed and a webservice sync might take a lot of time to download and update data. To avoid this we are bundling the updated data along with the app ( this will solve the issue for a fresh install). But for the users which are upgrading the app is there a standard mechanism to copy the read-only entities from the bundled db and update those to the existing database in the sandbox so that they will get the updated read-only data and also their read-write data remains intact.
UPDATE
Here is the scenario,
I have X.sqlite bundled with the proj (which has new schema), if X.sqlite is not there in doc dir I then copy it and from there everything works OK. Now in the App update scenario, X.sqilte will be already present in doc dir and won’t be copied and the migration assistant will migrate the schema. So now we have X.sqlite with the new schema in doc dir but old data (without new attributes). Now what I want to know is if there is a way to merge the data from bundled X.sqlite with the one which is there in the doc dir. I want to know if there is a process for merging.
To be more precise
Below are the entities
*Store - ReadOnly
*Products - ReadOnly
*ProductGroups - ReadOnly
*ShopList - User based
All are in the same model and in the same store.
Now Store/ Products / ProductGroups have extra attributes.
Lightweight migrator will migrate the schema of X.Sqlite so that the DB will have the new attribute columns.
Now what I am concerned is the next step,
Lets take Store as an example. Store has two new attributes latitude and logitude. Now the question how to copy the data? the steps
Copying the bundled DB to doc dir with diff name?
Create a new persistance co-ordinator?
Read the bundled data and get the objects?
then iterate through the existing db?
If I have understood your question: you want to update read-only data during app update--while leaving read-write data that user has changed intact.
There are several ways to achieve this:
Have two separate databases. One database can have the read-write
data and another with read-only data. They can relate to each other
using fetched properties. During update, replace or update the
read-only database--while leaving the read-write one intact.
Update the database using a background thread. The update code will
have it's own ManagedObjectContext--but sharing the same
PersistentStore. Synch the main ManagedObjectContext from the
background thread using some protocol.
The second option of updating from a background thread might work well if you choose to update from your web service.
If I haven't understood your issue, please clarify.
Ok So finally after a lot of research I achieved my goal, below are the trails and solutions that i did
Sol 1
Have the read-only and read-write data in separate databases, so that I can safely delete the readonly db if there is any master data update and I can safeguard user's data, but considering the timelines and constraints that I have, it wont be possible for me. Posting here so that it might help others.
Sol 2
I thought rather trying to merge the new data from bundled DB to the existing DB, I thought of merging the user data from existing db to the new db. Below are the steps done.
--> Created a new datacontext.
--> Created a new persistent co-ordinator
--> Renamed the bundled db with _v2 and copied it to the Doc directory, now we have 2 DB in the doc dir
I took some app Importing large data sets
--> Now using the ManagedObject clone category, I copied all the user info data from the existing db to the new db _v2. Found the category here NSManagedObject+Clone
--> Worked fine, now I got my _v2 database with new readonly data and the user data from the old database.
--> Now I need to give control back to the default datacontext
--> I tried to change the PSC of the old context to the new PSC but system didnt allow me to do that.
--> I then tried to change the persistence store of the old context to the new store but I got error saying that database already exists. (migratePersistentStore:toURL:options:withType:error:)
--> I ran out of ideas here.
Sol 3
I then discussed my problems with some of my other colleagues and they suggested to provide the new data in a different format and that striked. As I already mentioned, my app has logic to download new data as JSON and merge it to core data, why can I provide a JSON file with the new data, along with my app?
I collected the new response from thew webservice and created a JSON (not big just 1.5MB) and attached with app bundle, and for users that update the app, instead of core data merging I will read the JSON data locally and do the initial merging to the core data DB, there by the data base will have the new readonly data and also user data intact. After the initial merge, everything will be taken care by online sync.
I am trying to use multiple (two) persistent stores with Core Data for the first time.
It seems quite simple to add a store; but once this is done, how do I specify that a request to write (or read) some information to (or from) an entity has to be performed on one store or the other?
Some sample code would be welcome, but I can’t find anything on the net.
Fetches always cover all of the persistent stores managed by the coordinator.
When adding data, you can do either of the following:
Use configurations in your data model and when adding persistent stores. Configurations define named subsets of your model that contain some but not all entities. If an entity only exists in a configuration that's only used with one persistent store file, then new instances will automatically go to that store.
If the above doesn't work for your app, you'll need to call assignObject:toPersistentStore: to tell the managed object context which store to use.
I want to add a new fetch request template to a core data model. I know I could do it programmatically, but all the other fetch request templates are present in the core data editor and it makes sense to add the new one alongside them.
My question is, since this is part of a point release for an app that has been in the store for quite some time already, is there any chance this will break existing installs? Does it count as some kind of migration, or not? Obviously we will QA it, but I'd like to know the answer in advance, and googling has been fruitless so far.
It's not stated explicitly anywhere, but in the NSManagedObjectModel documentation, it says the following:
Changing Models
Since a model describes the structure of the data in a persistent store, changing any parts of a model that alters the schema renders it incompatible with (and so unable to open) the stores it previously created. If you change your schema, you therefore need to migrate the data in existing stores to new version (see Core Data Model Versioning and Data Migration Programming Guide). For example, if you add a new entity or a new attribute to an existing entity, you will not be able to open old stores; if you add a validation constraint or set a new default value for an attribute, you will be able to open old stores.
It doesn't explicitly mention fetch requests, but these don't have anything to do with the schema, so I think you'll be fine.
First, Thank you for any help provided.
I have an iOS leveraging CoreData to retain various presentations, this data comes from a sqlite file and there is no server connection.
I will have to be able to provide App updates (via appstore), this update may add more data to the database.
The tricky part is that it can not simply overwrite the current database, there are a few user tables that I will not like touched.
Please provide any information I should consider when accomplishing this or any links are greatly appreciated.
Thank you.
Given your app has no server connection, you will have to rely on shipping data within the updated application itself. I would recommend using a plist file or define your own xml or json structure. You can then read this data to create/update core data nsmanagedobjects.
It looks like someone in the past was using plist->coredata on SO
Would you have relationships between user created data and shipped data?
If not, you might go the route of connecting two stored to the persistent store coordinator. The shipped store would be read-only. The store with user created data would be read-write. You can use this approach, too, if you have relationships between shipped and user-created objects, but it's a lot more complicated, since CoreData doesn't manage cross-store relationships for you, and you'll need to write your own logic (doable, but not straight forward).
If you need to have relationships between shipped and user-created objects, you can still ship a CoreData store. When the app launches for the first time (no user-created objects), you copy the store to the Documents folder and user this store to create your CoreData stack. User created objects will be added to this store. Once you have new 'shipped' objects (i.e. a new store in the app-bundle), you'll have to manually migrate that stores data into the store that the user has changed. You'll have to be able to find
(1) objects that need to be deleted
(2) objects that need to be updated (changed)
(3) objects that need to be added
If you mark your shipped objects with a special flag such that you can tell if it's a user created object or a shipped one, that would be doable. You also have to have some sort of ID to be able to tell which objects in the new store correspond to which ones in the existing (old) store.
You do not need to go the route of using plists. In fact, I'd recommend against it. You can easily open two stores at the same time. Either to use both stored, or just to migrate objects from one store to the other store.