I am building a project where I need to prepopulate a coredata database with existing data.
I built a parser to create the sqlite file in the iOS Simulator, everything works fine.
I am using a single entity, and one of the attribute is indexed.
Performance after parsing my data file into core data is great, everything is good.
Now I am using the generated sqlite file (~200Mb) in project with same data model, same index, etc... and on first startup I copy over the db file to prepopulate the data
NSString *defaultStorePath = [[NSBundle mainBundle] pathForResource:#"myproject" ofType:#"sqlite"];
NSString *storePath = [[[self applicationDocumentsDirectory] path] stringByAppendingPathComponent: #"myproject.sqlite"];
NSError *error;
if (![[NSFileManager defaultManager] fileExistsAtPath:storePath])
{
if ([[NSFileManager defaultManager] copyItemAtPath:defaultStorePath toPath:storePath error:&error])
NSLog(#"Copied starting data to %#", storePath);
else
NSLog(#"Error copying default DB to %# (%#)", storePath, error);
}
The copying works fine, and the data can be accessed normally.
However the performance is terrible, and the index is clearly not being used.
A look at the size of the sqlite file after the copy operation, it went from 200Mb to 120Mb.
Everything looks alright in the model, what needs to be indexed is checked as indexed.
1) Is there a way for the index data not to be removed when copying the sqlite over?
2) Is it possible to programmatically rebuild the index?
3) Any other thoughts?
Check at the Apple documentation about this issue:
Although Core Data supports SQLite as one of its persistent store types, the database format is private. You cannot create a SQLite database using native SQLite API and use it directly with Core Data (nor should you manipulate an existing Core Data SQLite store using native SQLite API). If you have an existing SQLite database, you need to import it into a Core Data store (see “Efficiently Importing Data”).
To sum up, don't do it. The database schema is private and may change.
I use CSV files to pre-load all my initial data to CoreData on background when the application starts for the first time. Beware of multithreading CoreData access, by the way.
Hope it helps.
Looks like the problem was that the project didn't get cleaned well in between tests, it could have been a bug in Xcode 4.3 I was using at the time.
The same methodology is working fine now.
Related
I am trying to browse through the data written by Core Data in an iOS app I am developing.
After the app ran for a while, and I assume collected some data, I now wish to look through the data and see what was written.
I have tried getting and browsing the .sqlite file through getting the app container from the device (Xcode > Devices > myApp > Download Container...).
I got the db files, myAppDB.sqlite, myAppDB.sqlite-shm and myAppDB.sqlite-wal.
When trying to look through them, it seems like the .sqlite is an empty table (except maybe some generic CoreData/sqlite stuff), and the -wal file has all the info.
The thing is I was only able to know that the wal has useful data when opening it with TextEdit, which din't show it in a very readable way, and when I tried to use an SQLite Manager app I an alert saying the wal is encrypted and I am asked to put a password...
For what it matters, I am writing a framework which handles the db (the model file and the code for writing data is inside the framework), then I have this framework running in an app I am developing. This is the code I use to create the store from within the framework (using MagicalRecord):
NSBundle *frameworkBundle = [NSBundle bundleForClass:[self class]];
[MagicalRecord setDefaultModelNamed:#"myAppStore.momd" inBundle:frameworkBundle];
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"myAppStoreDB.sqlite"];
[MagicalRecord setupCoreDataStackWithStoreAtURL:storeURL];
UPDATE: I managed to open the sqlite file with both Core-Data-Editor and CoreDataUtility but they both override and delete the contents of the .wal file, and show an empty table... It does have the model (entity names/properties etc.) but no data.
My wal file is 873KB but when I open the sqlite with one of these 2 tools it becomes 0Bytes...
tl;dr
How can I browse through the info written by Core Data of the app I am developing?
Well, for some reason I had to force not using WAL in my store (using #"journal_mode":#"DELETE" as explained here).
I then got only .sqlite file without the smh and wal files, and was able to open it and view the data using the 2 mentioned tools (Core-Data-Editor and CoreDataUtility).
My guess is that this is something to do with either the fact that I am dealing with CoreData from a framework (creating a moc, creating entities, saving etc.) and not from the application. Another guess is that it has something to do with the fact that I am using MagicalRecord.
Any insights regarding the cause would be appreciated...
I have a core data based application that uses Dropbox to backup and restore data. The way I backup is pretty straight forward. I copy the .sqlite file on the user's dropbox.
Now my backup and restore functionality are working fine. The issue is with the .sqlite file itself. It appears that the .sqlite file is incomplete.
I entered about 125 entries in my application and took a backup. The backup appeared in my dropbox but when I use a .sqlite explorer tool to see the contents, I only see records upto 117th entry.
I tried updating the first entry and then again observing .sqlite file but no changes again.
What's even more strange is that the app appears to have recorded all changes. When I add a new entry or update an existing one and restart the app, the newly added data seems to persist.
But this newly added data does not appear in my .sqlite file.
I am backing up using this code:
AppDelegate *appDelegate = [[UIApplication sharedApplication] delegate];
NSString *filePath = [[[appDelegate applicationDocumentsDirectory] path] stringByAppendingPathComponent:#"MyApp.sqlite"];
if (account) {
if ([filesystem isShutDown]) {
filesystem = [[DBFilesystem alloc] initWithAccount:account];
[DBFilesystem setSharedFilesystem:filesystem];
}
DBPath *newPath = [[DBPath root] childPath:[NSString stringWithFormat:#"Backup - %#.sqlite", [NSDate date]]];
DBFile *file = [[DBFilesystem sharedFilesystem] createFile:newPath error:nil];
[file writeContentsOfFile:filePath shouldSteal:NO error:nil];
[filesystem shutDown];
}
I also copied the .sqlite file from the Simulator's folder and tried seeing it in the .sqlite browser. It still exhibits the same behaviour. Any reason why this must be happening?
Starting with iOS 7 / OS X 10.9, Core Data uses "Write-Ahead Logging" (WAL) as default
journaling mode for the underlying SQLite store file. This is explained in
Technical Q&A QA1809: New default journaling mode for Core Data SQLite stores in iOS 7 and OS X Mavericks
With the WAL mode, Core Data keeps the main store file untouched and
appends transactions to a -wal file in the same location. After the
Core Data context is saved, the -wal file is not deleted, and the data
in that file is not merged to the store file either. Therefore, simply
making copies of the store file will likely cause data loss and
inconsistency.
That should explain why your .sqlite file alone is incomplete.
As as solution you can (also explained in that Technical Note):
Disable WAL-mode (and use the "old" rollback journaling mode) for the SQLite store by setting the
#{NSSQLitePragmasOption:#{#"journal_mode":#"DELETE"}};
option when adding the persistent store, or
Use the
- (NSPersistentStore *)migratePersistentStore:(NSPersistentStore *)store toURL:(NSURL *)URL options:(NSDictionary *)options withType:(NSString *)storeType error:(NSError **)error
method to make a backup copy of the Core Data store.
I have an app (say version 1.0) which is using Core data and now i have updated the app using sqlite and for the new update i am not using core data at all (we will call it version 1.1), however i have kept the name of my database and all the column names of the table similar to the previous core data app.
My query is that what major or minor effects will it have if the user updates to the new version (1.1) which is using sqlite. Will it have some kind of dataloss or any crashes if yes then please guide me out by providing any links or suggestions
To avoid dataloss i am planning to transfer data in chunk from core data to sqlite app as read in this post, so am i doing this right do let me know if i have missed any valuable steps.
Migrating user data stored in sqlite to core data in app upgrade
Thank you
If your actions will correct and grammar, users willn't lose any data.
How I understand, in your next version of app you will managing data by any sqlite framework, then you just need .sqlite data file. To get it .sqlite file path you can:
NSString *dataFilePath = [[[self applicationDocumentsDirectory] path]
stringByAppendingPathComponent: #"YourAppName.sqlite"];
NSURL *dataFileUrl = [NSURL fileURLWithPath:storePath];
When applicationDocumentsDirectory method implement like this:
- (NSURL *)applicationDocumentsDirectory
{
return [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject];
}
If I understand something incorrect, write it in comments.
I use coreData in my iOS App. It's possible, that the user Add, Delete Data into the Database.
I have to deliver default data ( some different data-sets ).
At the moment, I'm creating the database by first Application launch. I read data from a csv file an create the database with it.
The csv is in the Application sandbox; the coreData (managedDocument) is in ApplicationDocument (creation on runtime...).
It works perfect for me - but I ask me, will Apple allow that, if I push the App to the AppStore?
There is nothing wrong with this approach and it can't be a reason for rejection. There is also another way to do it. You can create the database the way you do it now, copy the .sqlite file and provide it as your default database. Then copy it on app first run. The following code will do it:
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent: #"YourDBName.sqlite"];
if (![fileManager fileExistsAtPath:storeURL.path]) {
NSString *defaultStorePath = [[NSBundle mainBundle] pathForResource:#"YourDBName" ofType:#"sqlite"];
if (defaultStorePath) {
[fileManager copyItemAtPath:defaultStorePath toPath:storeURL.path error:NULL];
}
}
With this approach you will not need to include your csv file in your bundle.
Yes, apple does allow shipping a database populated by default.
the standard way to do it is to ship a default database in your bundle, then at launch time check if there is a database in your application documents directory, and if it does not exist, then copy the database from your bundle to the documents directory.
I need my application work as follows:
1- User download and startup the application.
2- The application will automatically fill the (based core data) with specific records.
3- The application will works ok.
4- When the user close the application and restart it.
5_ The application will not automatically fill the core data with the specific records because it already there.
6- The user will not be able to add/remove/update the data.
Is there a good technique to do that using only core data and accepted by Apple.
Populate the data in the simulator
Make a copy of the SQLITE database from the simulator folder
Add the database as a resource to your project.
Do something like this before initializing your Core Data stack:
code:
// Put down default db if it doesn't already exist
NSFileManager *fileManager = [NSFileManager defaultManager];
if (![fileManager fileExistsAtPath:storePath]) {
NSString *defaultStorePath = [[NSBundle mainBundle]
pathForResource:#"MyDataFile" ofType:#"sqlite"];
if (defaultStorePath) {
[fileManager copyItemAtPath:defaultStorePath toPath:storePath error:NULL];
}
}
Not shown, but critical: all the files you prepopulate data for must be marked as "do not backup", or you will be rejected, per the App Store guidelines. See this SO question for some details on how to do this: Why was my application still rejected after excluding files from iCloud backup using this code?
I can tell you one very easy way to do it. In the app delegate, create a variable in NSUserDefaults and change its value when the application is loaded for the first time. Then depending upon the value of NSUserDefaults, fill the data you want in the Core Data store. And this will happen only once because the value of NSUserDefaults variable is not going to change again.