Another way to do an "initial sync" - ios

I'm working on an app that has to work entirely offline and then sync up to a server.
The server can (and probably will) change depending on the project that is currently underway.
I've written the synchronisation stuff and it works really well. Everything gets updated both ways.
However, the initial sync of a device is basically a big info dump of thousands of records and it takes a really long time. I have analysed and optimised this but the time is now just from the sheer volume of data being synced and the relationships between that data.
A normal sync (i.e. sending and getting updates) only takes about 5 seconds from start to finish including reads, uploads, downloads and writes.
Is there a way I can "plug into" a computer and just import a DB file into the app?
Is there any other way of doing this other than going through the sync process and downloading and installing all this stuff on the device?
I can't do it at build time as the app is independent of which project it is on and each project has a different set of data.

Import the sqlite db into your app, and link it in your core-data stack in the app delegate. This massive amount of data should be shipped with the app and then syncing would make changes to that initial dataset.
This is how you link the sqlite to your core-data stack
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator
{
if (_persistentStoreCoordinator != nil) {
return _persistentStoreCoordinator;
}
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"MyAppName.sqlite"];
//Connects to sqlite---------------------------------------
NSFileManager *fileManager = [NSFileManager defaultManager];
// If the expected store doesn't exist, copy the default store.
if (![fileManager fileExistsAtPath:[storeURL path]]) {
NSURL *defaultStoreURL = [[NSBundle mainBundle] URLForResource:#"MyAppName" withExtension:#"sqlite"];
if (defaultStoreURL) {
[fileManager copyItemAtURL:defaultStoreURL toURL:storeURL error:NULL];
}
}
//end sqlite connection ---------------------------------------
NSError *error = nil;
_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:nil error:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
abort();
}
return _persistentStoreCoordinator;
}

Related

Migrate persistent store to new location

I need to move my app's database to a new location and stop it from being shared through iCloud automatically.
From my understanding, Apple previously documented that a database should be placed in the Documents directory of an application, but now says that the database should be placed in the Library directory. This is because iOS now shares all data from the Documents directory through iCloud.
My issue is that I need to move my database from the Documents directory to the Library directory. Below is the code I use to do this.
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator {
if (_persistentStoreCoordinator != nil) {
return _persistentStoreCoordinator;
}
_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
// Get the URL of the persistent store
NSURL *oldURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"App.sqlite"];
bool oldStoreExists = [[NSFileManager defaultManager] fileExistsAtPath:[oldURL path]];
// Get the URL of the new App Group location
NSURL *newURL = [[self applicationLibraryDirectory] URLByAppendingPathComponent:#"App.sqlite"];
bool shouldMoveStore = false;
NSURL *storeURL = nil;
if (!oldStoreExists) {
//There is no store in the old location
storeURL = newURL;
} else {
storeURL = oldURL;
shouldMoveStore = true;
}
NSError *error = nil;
NSDictionary *options = #{ NSMigratePersistentStoresAutomaticallyOption:#YES, NSInferMappingModelAutomaticallyOption:#YES };
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:options error:&error]) {
NSLog(#"Error Moving Store: %ld\nDescription: %#\nReason: %#\nSuggested Fix:%#", (long)error.code, error.localizedDescription, error.localizedFailureReason, error.localizedRecoverySuggestion);
abort();
}
if (shouldMoveStore)
[self movePersistentStoreFrom:oldURL To:newURL];
return _persistentStoreCoordinator;
}
-(void) movePersistentStoreFrom:(NSURL *)oldURL To:(NSURL *)newURL {
// Get the reference to the current persistent store
NSPersistentStore *oldStore = [_persistentStoreCoordinator persistentStoreForURL:oldURL];
// Migrate the persistent store
NSError *error = nil;
[_persistentStoreCoordinator migratePersistentStore:oldStore toURL:newURL options:nil withType:NSSQLiteStoreType error:&error];
if (error) {
NSLog(#"Error Moving Store: %ld\nDescription: %#\nReason: %#\nSuggested Fix:%#", (long)error.code, error.localizedDescription, error.localizedFailureReason, error.localizedRecoverySuggestion);
abort();
}
}
This code seems to work in moving the persistent store, but on the next run of the application the old store is found again and the app attempts to migrate the store a second time. At this point a crash occurs, and I'm assuming it is due to this double migration call. In classic Xcode style, the stack trace is pretty much useless.
Update -
The migratePersistentStore function does not remove ("cut/paste") the old sqlite database, instead it just makes a "copy". It looks like I'll need to remove the old one programmatically.
I switched the code to check if the newStoreExists as opposed to the !oldStoreExists, and the migration does work.
My new issue is that the table rows of the migrated data have become randomized. Any help with this would be appreciated!
Update 2 -
As it turns out, any data migrations like moving the store location or updating the database will randomize your data if you are using Core Data. You need to put an attribute into your table and use a NSSortDescriptor inside of your NSFetchRequest to manually sort the data and keep it ordered.
If you never migrate the data (I previously had not) the core data fetch request always come in the order it was saved.

How to enable iCloud for Core Data?

This is the first time I'm trying to make this work. I'm following the iCloud Programming Guide for Core Data, the "Using the SQLite Store with iCloud" section, and in AppDelegate I have:
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator
{
// The persistent store coordinator for the application. This implementation creates and return a coordinator, having added the store for the application to it.
if (_persistentStoreCoordinator != nil) {
return _persistentStoreCoordinator;
}
// Create the coordinator and store
_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"MyApp.sqlite"];
NSError *error = nil;
NSString *failureReason = #"There was an error creating or loading the application's saved data.";
// To enable iCloud
NSDictionary *storeOptions = #{NSPersistentStoreUbiquitousContentNameKey: #"MyAppCloudStore"};
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:storeOptions error:&error]) {
// Report any error we got.
NSMutableDictionary *dict = [NSMutableDictionary dictionary];
dict[NSLocalizedDescriptionKey] = #"Failed to initialize the application's saved data";
dict[NSLocalizedFailureReasonErrorKey] = failureReason;
dict[NSUnderlyingErrorKey] = error;
error = [NSError errorWithDomain:#"YOUR_ERROR_DOMAIN" code:9999 userInfo:dict];
// Replace this with code to handle the error appropriately.
// abort() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
abort();
}
return _persistentStoreCoordinator;
}
Previously, I enabled iCloud services for this App ID, create the corresponding provisioning, and turn on the iCloud services in Xcode's capabilities (iCloud Documents and default container).
According to the iCloud Programming Guide for Core Data, at this point:
Test: Run the app on a device. If Core Data successfully created and configured an iCloud-enabled persistent store, the framework logs a message containing “Using local storage: 1,” and, later, another message containing “Using local storage: 0”.
I run the app on an iPhone, and persistentStoreCoordinator is retrieved, but I see nothing in Xcode's debug area.
What am I missing?
Thanks?
I found the problem: [[NSFileManager defaultManager] ubiquityIdentityToken] was returning nil, and it was because I had my device not updated to iCloud Drive.
I found the response in this post.

CoreData with multiple databases

I'm building an app that is going to rely on 3 separate .sqlite databases. What methods in my App Delegate am I going to have to edit to allow for this? Right now, my managedObjectContext and managedObjectModel haven't been touched from how the template created them. My persistantStoreCoordinator looks like this:
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator
{
if (_persistentStoreCoordinator != nil)
{
return _persistentStoreCoordinator;
}
NSURL *storeURLConfigA = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"dbA.sqlite"];
NSURL *storeURLConfigB = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"dbB.sqlite"];
NSURL *storeURLConfigC = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"dbC.sqlite"];
// // Pre-load .sqlite db in Project Navigator into app on first run after deleting app
if (![[NSFileManager defaultManager] fileExistsAtPath:[storeURLConfigA path]])
// dbA
{
NSURL *preloadURL = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:#"dbA" ofType:#"sqlite"]];
NSError* err = nil;
if (![[NSFileManager defaultManager] copyItemAtURL:preloadURL toURL:storeURLConfigA error:&err])
{
NSLog(#"Error preloading database A - %#",error.description);
}
}
if (![[NSFileManager defaultManager] fileExistsAtPath:[storeURLConfigB path]])
// dbB
{
NSURL *preloadURL = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:#"dbB" ofType:#"sqlite"]];
NSError* err = nil;
if (![[NSFileManager defaultManager] copyItemAtURL:preloadURL toURL:storeURLConfigB error:&err])
{
NSLog(#"Error preloading database B - %#",error.description);
}
}
if (![[NSFileManager defaultManager] fileExistsAtPath:[storeURLConfigC path]])
// dbC
{
NSURL *preloadURL = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:#"dbC" ofType:#"sqlite"]];
NSError* err = nil;
if (![[NSFileManager defaultManager] copyItemAtURL:preloadURL toURL:storeURLConfigC error:&err])
{
NSLog(#"Error preloading database C - %#",error.description);
}
}
// // Put the Configs into the PersistantStoreCoordinator and tie them to their database file
NSError *error = nil;
_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
NSDictionary *options = #{NSSQLitePragmasOption : #{#"journal_mode": #"DELETE"}};
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:#"Config_A" URL:storeURLConfigA options:options error:&error])
{
NSLog(#"Error setting up dbA - %#",error.description);
abort();
}
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:#"Config_B" URL:storeURLConfigB options:options error:&error])
{
NSLog(#"Error setting up dbB - %#",error.description);
abort();
}
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:#"Config_C" URL:storeURLConfigC options:options error:&error])
{
NSLog(#"Error setting up dbC - %#",error.description);
abort();
}
return _persistentStoreCoordinator;
}
Then, I have a single .xcdatamodelID in my mainBundle that has all of my tables/Entities in it, for all three databases. I used XCode's CoreData editor interface to add Configurations Config_A, Config_B, and Config_C, in addition to the Default Configuration. I've moved the Entities into the Configurations for the databases I want them to be in (each Entity should only be in one database).
I run the app and everything runs fine, and I can read and write data without any problem. The problem comes when I view the tables (through the terminal or Firefox's SQLite Manager, for example). All 3 of the databases contain all of the tables, but the tables only have data in the database they are supposed to exist in (there are 0 rows in that table in the other two databases).
Why are all of the tables in each database? How can I get each database to only include the couple tables I want them to have?
You have added all 3 of your databases (NSPersistentStores) to the same NSPersistentStoreCoordinator. Therefore, when you perform a save on a Managed Object Context, the store coordinator distributes those changes to all of its stores. Instead, create a persistent store coordinator for each database, perhaps keeping them in an NSDictionary, with a key #"A" for the A database persistent store coordinator, key #"B" for the B database coordinator, etc. Then when you create a new managed object context, you will have to determine which database (NSPersistentStoreCoordinator) it belongs to. This will allow you to keep your tables/changes separated and prevent all data going into all 3 databases.
I started with what Patrick Goley had in his answer, but I still couldn't get the behavior I was looking for. After reading what he had, I got the idea to just go and break my three databases into totally separate pieces everywhere I could think of doing so, and that seems to be working. If someone needs me to post my whole app delegate I will, but the gist of it is:
I have properties for 3 NSManagedObjectContexts, 3 NSManagedObjectModels, and 3 NSPersistantStoreCoordinators.
I copied the factory methods to create each (which were originally written when I created my app from a template) and hardcoded each to use the appropriate contexts/models/store coordinators.
I have 3 NSSets that holds the table names for it's respective database, and when I want to read from (or save to) a table, I pass the name of that table, and use the NSSets to get the appropriate context/model/store coordinator.
After spending a lot of time reading all the Apple docs and other threads I could find, I feel like it wasn't supposed to be this difficult to support separate databases, but this was the only thing that's actually doing what I want so far. I don't know what I was missing before, but this is working, so it will do for now. If I get more free time in the future I might look into it more and try to figure out the right way, and if I find it I'll update.

Core Data Encryption with MagicalRecord

Is it possible to configure the level of protection of the SQLite file generated for the Core Data?
I need to use the NSFileProtectionComplete level on it.
Any ideas?
Look for the line where you do addPersistentStoreWithType:configuration:URL:options:
NSURL *storeURL = ...;
NSError *error = nil;
persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:...];
if (![persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
configuration:nil
URL:storeURL
options:nil
error:&error])
{
NSLog(#"Add persistent store failed: %#", error);
}
Then add:
NSDictionary *attributes = #{NSFileProtectionKey: NSFileProtectionComplete};
if (![[NSFileManager defaultManager] setAttributes:attributes
ofItemAtPath:path
error:&error]) {
NSLog(#"File protection failed: %#", error);
}
Be aware that you can't use the database in the background, consider using NSFileProtectionCompleteUnlessOpen:
NSFileProtectionComplete:
The file is stored in an encrypted format on disk and cannot be read from or written to while the device is locked or booting.
NSFileProtectionCompleteUnlessOpen:
The file is stored in an encrypted format on disk. Files can be created while the device is locked, but once closed, cannot be opened again until the device is unlocked. If the file is opened when unlocked, you may continue to access the file normally, even if the user locks the device. There is a small performance penalty when the file is created and opened, though not when being written to or read from. This can be mitigated by changing the file protection to NSFileProtectionComplete when the device is unlocked.

Questions about CoreData and pre-populated models

To pre-populate CoreData in my app upon first launch, I have included a PreModel.sqlite file that was previously created by the app from data that it downloaded from a web service, which includes images.
To populate the model, I do this:
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator
{
if (_persistentStoreCoordinator != nil) {
return _persistentStoreCoordinator;
}
NSLog(#"creating new store");
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"PreModel.sqlite"];
if(![[NSFileManager defaultManager] fileExistsAtPath:[storeURL path]]) {
NSString *sqlitePath = [[NSBundle mainBundle] pathForResource:#"PreModel" ofType:#"sqlite"];
if (sqlitePath) {
NSError *error = nil;
[[NSFileManager defaultManager] copyItemAtPath:sqlitePath toPath:[storeURL path] error:&error];
if (error) {
NSLog(#"Error copying sqlite database: %#", [error localizedDescription]);
}
}
}
NSError *error = nil;
_persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[self managedObjectModel]];
if (![_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:storeURL options:nil error:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
abort();
}
return _persistentStoreCoordinator;
}
It seems to work. But I have 2 questions:
If the sqlite file is just a database file and does not actually contain any images, how is the app finding and loading the images on first launch?
Even on subsequent runs of the app I see "creating new store" logged every time. Why is _persistentStoreCoordinator always nil? I am clearly setting it in the code.
It's possible to store images in a database file, usually as binary blobs (which look like instances of NSData in Core Data). If you can provide more info about your model or the code that stores/loads the images, we can be more specific.
"Creating new store" is expected to get logged every time the app is launched in this instance. Even though the SQLite file is persistent on disk, you can't expect data structures in your code to stick around when your app is terminated - you need to create a new persistent store object for your program every time it launches.
Think of it like assigning NSInteger x = 10, then expecting to be able to quit and relaunch your program while maintaining that x has the value 10. That's not how programs work - you'd need to reassign x = 10 before you can expect to read x and get 10 back. The variable _persistentStoreCoordinator works the same way here.

Resources