Our app is having a coredata store which is based on a single coredata model. There are read-only data as well as read-write data.
The read-only data are pre-loaded and bundled along with the app, so that on a fresh install, this database is copied to the application sandbox and from there on the data base will be updated via webservice (ie only the changed data will get updated from webservice so that less data is transferred).
Now we have situation where we need to add more attributes to the read-only entities.
Light weight migration will help in upgrading the schema easily, but the problem is about the new data, since we are adding new attributes to all the read-only entities, all the data records are changed and a webservice sync might take a lot of time to download and update data. To avoid this we are bundling the updated data along with the app ( this will solve the issue for a fresh install). But for the users which are upgrading the app is there a standard mechanism to copy the read-only entities from the bundled db and update those to the existing database in the sandbox so that they will get the updated read-only data and also their read-write data remains intact.
UPDATE
Here is the scenario,
I have X.sqlite bundled with the proj (which has new schema), if X.sqlite is not there in doc dir I then copy it and from there everything works OK. Now in the App update scenario, X.sqilte will be already present in doc dir and won’t be copied and the migration assistant will migrate the schema. So now we have X.sqlite with the new schema in doc dir but old data (without new attributes). Now what I want to know is if there is a way to merge the data from bundled X.sqlite with the one which is there in the doc dir. I want to know if there is a process for merging.
To be more precise
Below are the entities
*Store - ReadOnly
*Products - ReadOnly
*ProductGroups - ReadOnly
*ShopList - User based
All are in the same model and in the same store.
Now Store/ Products / ProductGroups have extra attributes.
Lightweight migrator will migrate the schema of X.Sqlite so that the DB will have the new attribute columns.
Now what I am concerned is the next step,
Lets take Store as an example. Store has two new attributes latitude and logitude. Now the question how to copy the data? the steps
Copying the bundled DB to doc dir with diff name?
Create a new persistance co-ordinator?
Read the bundled data and get the objects?
then iterate through the existing db?
If I have understood your question: you want to update read-only data during app update--while leaving read-write data that user has changed intact.
There are several ways to achieve this:
Have two separate databases. One database can have the read-write
data and another with read-only data. They can relate to each other
using fetched properties. During update, replace or update the
read-only database--while leaving the read-write one intact.
Update the database using a background thread. The update code will
have it's own ManagedObjectContext--but sharing the same
PersistentStore. Synch the main ManagedObjectContext from the
background thread using some protocol.
The second option of updating from a background thread might work well if you choose to update from your web service.
If I haven't understood your issue, please clarify.
Ok So finally after a lot of research I achieved my goal, below are the trails and solutions that i did
Sol 1
Have the read-only and read-write data in separate databases, so that I can safely delete the readonly db if there is any master data update and I can safeguard user's data, but considering the timelines and constraints that I have, it wont be possible for me. Posting here so that it might help others.
Sol 2
I thought rather trying to merge the new data from bundled DB to the existing DB, I thought of merging the user data from existing db to the new db. Below are the steps done.
--> Created a new datacontext.
--> Created a new persistent co-ordinator
--> Renamed the bundled db with _v2 and copied it to the Doc directory, now we have 2 DB in the doc dir
I took some app Importing large data sets
--> Now using the ManagedObject clone category, I copied all the user info data from the existing db to the new db _v2. Found the category here NSManagedObject+Clone
--> Worked fine, now I got my _v2 database with new readonly data and the user data from the old database.
--> Now I need to give control back to the default datacontext
--> I tried to change the PSC of the old context to the new PSC but system didnt allow me to do that.
--> I then tried to change the persistence store of the old context to the new store but I got error saying that database already exists. (migratePersistentStore:toURL:options:withType:error:)
--> I ran out of ideas here.
Sol 3
I then discussed my problems with some of my other colleagues and they suggested to provide the new data in a different format and that striked. As I already mentioned, my app has logic to download new data as JSON and merge it to core data, why can I provide a JSON file with the new data, along with my app?
I collected the new response from thew webservice and created a JSON (not big just 1.5MB) and attached with app bundle, and for users that update the app, instead of core data merging I will read the JSON data locally and do the initial merging to the core data DB, there by the data base will have the new readonly data and also user data intact. After the initial merge, everything will be taken care by online sync.
Related
In my project I have pre-populated data in Core Data, ie. I have a sqlite file that's copied into user's folder on first launch, however I also have user data which user generates, in this case is there anyway I can use a single data model? I know I can create two models one static and one for user but just wondering if there's a better way.
Thanks!
update:
My concern is although it works for the first release, in subsequent releases if I need to update the static data, it would be a new sqlite file that's copied into user folder which will overwrite existing user data.
Question: [...] can I use a single data model?
Yes. There is the concept of Configurations with Core Data Models. You can learn about it here: NSManagedObjectModel. This allows you to partition one data model into several parts. For the use case described in the OP you want have two parts or configurations: "StaticData" and "UserData". "StaticData" holds entities and data that may change only with App updates (new releases). "UserData" holds user data. You need to design data and foreign keys accordingly in a manner that the entities of both data sets are "well linked" for the App's lifetime. (If you cannot ensure this from the beginning to the end you need to design additional data migration processing for each release).
Question/Use Case: Only update static data not overwrite user data.
Each "Configuration" (see 1.) will have a persistent core data store (SQLite file) for its own. For the use case specified the file with user data must not be excluded from the backup but the static data must be excluded from the backup. You can learn about how to exclude files and directories from backups here: NSURLIsExcludedFromBackupKey. After a new release the empty user data will be overwritten with the user data from the backup.
I am currently working on the app that is going to use database to store items. The first thing that everybody say, when it comes to store data in iOS is Core Data.
But, after few days of looking through tutorials and docs, I have a big question.
Let me explain architecture a little bit more. So we have a backend, where you can add items, also, we have iOS and Android application. I am creating a Core Data model for our database.
What we want,is to check if there is update for database and download it. The problem is that we don't use JSON or XML, we are using the new sqlite file.
Since Core data creates three files for database, which are:
db.sqlite
db.sqlite-wal
db.sqlite-shm
Is core data able to replace "db.sqlite" with the new one, that is downloaded from server?
Thought the idea of replacing the database file instead of importing objects to it is very tempting it's highly discouraged to mess up with sqlite database created by CoreData. You should never touch it manually, when you do, you'll very likely end up with broken DB or messed up data.
So no, CoreDate is not able to replace underlaying sqlite file. You should instead import your data using CoreData stack, that's how it's designed. Creating JSON/XML service is the best way you can go.
BUT IN THEORY and in case you would be able to keep CD internal information stored in the db untouched, it should be possible to replace the sqlite file. If your database is read-only for users, it might work, but if users are able to create or modify records, forget about it right now.
First, you'd have to tear down all CoreData stacks (Managed Object Context, Persistent Store Coordinator, Managed Object Model) that might be using it before you replace it, replace the file and re-create CoreData stack(s).
db.sqlite is the main database file, the other two are temporary files, Write ahead log (wal) and Shared memory (smh), so you don't need these two.
Remember you never MUST NOT change the structure of the database, just data in it!!!
However, as stated in the beginning, I do not recommend this approach at all.
I have live application on the app store. It uses SQLite data storage and ugly methods that save and get data form it. I have replaced the data model with core data. So right now I have one table that contains favorites list. I suppose user will lose their favorites data, and the second problem will be that previous SQLite data base will be in the app.
What's the best way to migrate that data to new data base. So code was ugly, I have started to write new project. It means that there will not be any SQLite data base as before with the same name, just new Core Data db. How usually we need to solve this problem.
I think I can add a method in app delegate that will check if data base with name in the application I will try to retrieve all data from that storage and convert it to the Core Data entities. Then when data will be converted with success, I will drop that database. When I will if all my active users will have this update. I will remove this feature from app delegate because all will up to date. Does it make sense?
Your approach is correct. In the new app version, check if there is an up-do-date Core Data persistent store, if not, read the old data, copy it to Core Data, delete the old sqlite file.
You can leave that code in there for a long time - it just won't run if there already is a valid Core Data installation.
I'm upgrading the current version of a dictionary app. The new version of the database has new schema and is prepopulated with new data.
However, there is a single attribute value I would like to add from the user's version of the database for about 12000 records found in 3 of the entities.
Edit: These are values that were created throughout use of the app, so I can't simply overwrite the previous database. I was hoping it was possible to load the new database, and then update certain values using data from the user's version.
How should I do this?
I need to load two versions of the dictionary with different model versions into memory and transfer across. Is this right?
You need to perform a core data lightweight migration. check for a tuto on the web if required - very simple
You can save with your project an sqlite or xml (depend on size of data) with data to be updated - OR - you can download from a web service of your own, the required updates.
i want to add tables/columns to a database during runtime.
Currently I'm using Core Data.
I know that there's a possibility to do so in XCode (add new data model version), but I definitely can't use that way, because I receive the database schema from a web service.
Is there any good possibility to run ddl commands during runtime when using Core Data, or is it just possible with directly using sqlite (or a wrapper/ormapper)?
If it's better to use a wrapper/ormapper please give me some suggestions about which should be used in this case.
Workflow should be:
start app
check if database is up to date
if new version of schema is available from a web service do DDL commands
continue with app workflow
PS: Please no answers which describe alternatives modifying the schema with XCode!
Can you modify the Core Data model at run time? Yes...but, it probably won't work the way you want it to work.
Core Data's API makes it possible to construct or modify every detail of a data model at run time. Xcode's model editor is a convenience, but you could skip it and do everything in code if you wanted. For example, NSEntityDescription's properties attribute (which covers both attributes and relationships) is writeable. You could create a new NSAttributeDescription and update the entity's properties to contain it. Bang, you just added a new attribute to the entity. Similarly, NSManagedObjectModel's entities property is writeable, so you could create a new NSEntityDescription and add it to the model. That gives you a new entity, created at run time.
But, and it's a big one: you can only do this before you load the data store. Once you load your persistent store, altering the model will throw an exception. When Core Data loads a persistent store, it compares the model file to the model used in the store file. They must match, and you can't do anything to change this fact after loading the store. Once you load the store, the model is fixed.
What's more, even if you modify your model before loading the persistent store, you can only load persistent stores that match the current version of the model-- unless, that is, you also write code to migrate the persistent store to the new model. How hard that is depends on the nature of the changes. At a minimum then, you would need to make any changes before loading previously saved data, and then also arrange to do model migration to update the persistent store to use the new model.
With Core Data the model (schema) and data are stored separately and matched up when the store is loaded. That's not how SQLite works internally but it's the approach that Core Data enforces.