Let me get this out of the way right now: yes, it was almost certainly a mistake to not use Core Data. However, I was new to iOS development when I made these decisions, and I had no idea I'd be hamstrung like this. Moreover, the app is intended to also run on Android (eventually), so I avoided platform-specific APIs wherever possible.
I have an iOS app that stores data in a local SQLite database file. The data stored in the file is provided by the user, so it's important that it be kept safe. I had plans to "do this later", and later is now here. I am quickly coming to the realization that it won't be as straightforward as I had hoped...
I now understand that it won't be possible to seamlessly synchronize data across devices, and I'm willing to accept that limitation until I manage to migrate to Core Data. However, in the meantime I'd at least like the SQLite database to be backed up periodically so users can feel safe using the app on a single device. I was thinking I would do this:
periodically (e.g. once a week) copy the SQLite file from local storage into cloud storage, thus ensuring it is backed up
when the app starts, if the local store is missing or corrupted but the file exists in the cloud storage, ask the user if they would like to copy it over
The biggest problem with this approach is that the user could run the app on multiple devices and therefore the data stored in iCloud could be from any one of those devices, but only one. To combat that, I thought I could just use a per-device, unique name for the file in cloud storage. I would generate this using UIDevice.identifierForVendor.
So my startup logic would be:
Determine the unique name for the cloud file.
Is the local file missing or corrupted, and if so, does the cloud file exist?
2.1. Ask the user if they would like to restore from the cloud file. Make it really hard for them to say no because doing so will lose all their data.
2.2. If they say yes, copy the cloud file to the local file storage.
Open the local database file.
And running in the background I would occasionally copy the database file from local to cloud storage.
I would like to know whether this a sensible approach until I do Core Data integration. Also, are there any hidden "gotchas" that I'm perhaps missing?
UPDATE: as #TomHarrington pointed out in a comment, it turns out my database file is already sitting in /Documents, which is backed up to iTunes and any iCloud account. So my question morphs into this:
Should I simply ensure my database has a device-specific name so that it is not clobbered by the app running on another device connected to the same iCloud account?
I'm going to answer my question, since I ended up going down this path and finding a MASSIVE blocker. There is a bug in the UIDevice.identifierForVendor API that causes it to regenerate every time a new version of the app is installed! See here. This of course rules out using it as a device identifier. sigh
I think I'm SOL with that approach. Instead, I might generate a GUID on first execution and use that as my identifier. Problem is, I need to store that somewhere that isn't backed up to iCloud.
Ugh, I may just give up here and say my app can't be run on multiple devices until Core Data integration is done.
UPDATE: I ended up generating an identifier on first run and storing it in the keychain (as a local entry only so it isn't backed up to iCloud).
Related
Can core data do the job or should I use sqlite?
In general, applications on iOS run in a sandboxed environment. That means, you can only manipulate with things in your sandbox. However, when app is removed, so is the sandbox. Even if you install the application again, it will be completely clean. The only storage which is persisted across multiple installations of the app is Keychain, but it is storage purposed for sensitive data like passwords, certificates etc.
CoreData is basically a SQLite wrapper, which means that the backing SQLite database (file) is stored in the application sandbox.
If you referring to reinstallation of the whole iOS operating system, you can first create a backup which you can recover later and the data should be restored.
But, if you are talking about persisting data while reinstalling your app, you need to look into how to store them on remote storage (can be iCloud, Realm Cloud etc, your custom API/backend) and synchronize them to some remote storage.
I recommend you to look into CloudKit (iCloud API) as it is Apple platform and possibly purposed for what you are looking for.
Core Data uses SQLite under the hood as a default, so your question is wrong a bit.
I will try to give you an answer, if deleting then installing the the ios means deleting application and installing it again. Your application runs in a sandbox, so all the data you have in your DB will be removed after uses reinstall it.
If you want to persist it:
You must have some remote storage
If chunk of data is small, you can store it in keychain.
Hope it helps
Given the recent addition of local datastore for iOS to Parse, it should be possible to rely exclusively on Parse to manage app's database, thus totally avoiding Core Data. Does this sound like a good idea? What would be the pros and cons of such an approach?
In particular, I am wondering whether it will be possible to pre-populate Parse local datastore with some data, and include this database as a part of the app when submitting to appstore.
UPDATE
From the comments that were posted, it seems that people misunderstood my intended use case. Sorry guys, I should have made my question more clear from the beginning. Let me clarify it now, anyway.
So, there is some amount of data in Parse database on the web, same for every user, e.g. a catalogue of books. It will be updated every now and then. What I want is to publish an app on App Store which is pre-populated with Parse data store, as it stands at the moment when the app is published. For that to happen, I'd like to pin all available data when building my app and ship that data store along with the app. The problem is that the pinned data will be stored on device's (or emulator's) file system, it won't be part of the project. That's why if I build the app and submit it to app store, the data won'd be included.
Any suggestions how to attach the local data store to the app?
The local data store is stored in the sandboxed part of the filesystem in iOS. When you package the store with the app, it'll live in the signed application folder, not in the location Parse expects it to be.
So, if you were looking to do this, you'd need to include your default local data store in the application on building/submission, and copy it into the location Parse expects it to be in (which is Library/Private Documents/Parse and the file is called ParseOfflineStore) when your application starts up. This must happen before you call enableLocalDatastore, or an empty one will be initialized.
It should be possible!
Read this in the docs. Parse has a highly resourceful and fully documented guide for their backend.
https://parse.com/docs/ios_guide#localdatastore
Per my comment above concerning didFinishLaunchingWithOptions; it has been a place for your to create objects on launch, I have been doing that for a long time. Especially with channels. However, by enabling the local data store you can access those objects you pinned or created with a simple query with no reachability per your concern. Either way they both are created on disk. Core Data has a lot more cons. Especially with NSFetchedResultsController and the flexibility it offers. It's all up to you what you want to do with your app. PFQueryTableViewController isn't bad but if your direction and vision for your app is to be exclusively Parse then why not. It's a great feature. However I didn't see anything in the docs about the local queries effecting your limit so I would suggest looking into that if you have a large audience performing numerous queries per second.
Take advantage of their docs. They do a great job at keeping us informed.
I have a CoreData app (using https://github.com/lhunath/UbiquityStoreManager), backed by iCloud. In one use case a user with a local store enables iCloud (where data already exists). I want to prompt the user to make a decision of whether to migrate the local data to iCloud or just use the iCloud version. As part of this, I'd like to display the device name and last sync date of the version in iCloud.
I've been tinkering around with my NSPersistentStore's metadata, but that doesn't appear to get synced to iCloud.
Any suggestions?
You could use iCloud's key-value store to store the device name & date of the last sync.
My no doubt unpopular suggestion is "don't". Trying to determine what is in iCloud at any given time puts you on pretty shaky ground. You may be able to get it to work most of the time, but there will always be circumstances where it breaks down.
If you really must import some data when first enabling iCloud, I suggest just always importing the data, and then deduping later as the iCloud data comes in. As ugly as it sounds, that's the only approach really guaranteed to work with Apple's approach.
It is worth taking a look at other Core Data sync frameworks like TICDS and Ensembles. They take a more sane approach to data identity, which means you can avoid the whole deduping step. (Disclosure: I develop Ensembles)
do a metadata query on the iCloud files and check the most recent transaction log file in iCloud. See the link below for a sample app that uses this approach to check whether the app is properly synchronised with iCloud.
http://ossh.com.au/design-and-technology/software-development/sample-library-style-ios-core-data-app-with-icloud-integration/
EDIT:
I just realised I don't get the actual device name, but once you have found the most recent log file then use this to get the device. Just be aware this call may be expensive.
NSFileVersion *openedVersion = [NSFileVersion currentVersionOfItemAtURL:fileURL];
return openedVersion.localizedNameOfSavingComputer;
I've been having trouble getting an app submitted to the App Store. This is due to the fact that that database, which is updatable, is too large for the iCloud backup limitations. Most of the data in the db is static, but one table records the user's schedule for reviewing words (this is a vocabulary quiz).
As far as I can tell, I have two or three realistic options. The first is to put the whole database into the Library/Cache directory. This should be accepted, because it's not backed up to iCloud. However, there's no guarantee that it will be maintained during app updates, per this entry in "Make App Backups More Efficient" at this url:
http://developer.apple.com/library/IOs/#documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/PerformanceTuning/PerformanceTuning.html
Files Saved During App Updates
When a user downloads an app update, iTunes installs the update in a new app directory. It then moves the user’s data files from the old installation over to the new app directory before deleting the old installation. Files in the following directories are guaranteed to be preserved during the update process:
<Application_Home>/Documents
<Application_Home>/Library
Although files in other user directories may also be moved over, you should not rely on them being present after an update.
The second option is to put the data into the NSDocuments or NSLibrary directory, as mark it with the skipBackupFlag. However, one problem is this flag doesn't work for iOS 5.0 and previous per this entry in "How do I prevent files from being backed up to iCloud and iTunes?" at
https://developer.apple.com/library/ios/#qa/qa1719/_index.html
Important The new "do not back up" attribute will only be used by iOS 5.0.1 or later. On iOS 5.0 and earlier, applications will need to store their data in <Application_Home>/Library/Caches to avoid having it backed up. Since this attribute is ignored on older systems, you will need to insure your app complies with the iOS Data Storage Guidelines on all versions of iOS that your application supports
This means that even if I use the "skipBackupFlag", I'll still have the problem that the database is getting backed up to the cloud, I think.
So, the third option, which is pretty much of an ugly hack, is to split the database into two. Put the updatable part into the NSLibrary or NSDocuments directory, and leave the rest in application resources. This would have the small, updatable part stored on the cloud, and leave the rest in the app resources directory. The problem is that this splits the db for no good reason, and introduces possible performance issues with having two databases open at once.
So, my question is, is my interpretation of the rules correct? Am I going to have to go with option 3?
p.s. I noticed in my last post cited urls were edited to links without the url showing. How do I do this?
Have you considered using external file references as described in https://developer.apple.com/library/IOS/#releasenotes/DataManagement/RN-CoreData/_index.html . Specifically, refer to "setAllowsExternalBinaryDataStorage:" https://developer.apple.com/library/IOS/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSAttributeDescription_Class/reference.html#//apple_ref/occ/instm/NSAttributeDescription/setAllowsExternalBinaryDataStorage: . Pushing out large data into a separate file can help reduce database size .
I store some data in my iOS app directly in a local .sqlite file. I chose to do this instead of CoreData because the data will need to be compatible with non-Apple platforms.
Now, I'm trying to come up with the best way to sync this file over iCloud. I know you can't sync it directly, for many reasons. I know CoreData is able to sync its DBs, but even ignoring that using CD would essentially lock this file into Apple platforms (I think? I've only looked into CD a bit), I need the iCloud syncing of this file to work across ALL of iCloud's supported platforms - which is supposed to include Windows. I have to assume that there won't be any compatibility for the CoreData files in the Windows API. Planning out the best way to accomplish this would be a lot easier if Apple would tell us any more than "There will be a Windows API [eventually?]"
In addition, I'll eventually need to implement at least one more sync service to support platforms that iCloud does not. It would be helpful, though not required, if the method I use for iCloud can be mostly reused for future services.
For these reasons, I don't think CoreData can help me with this. Am I correct in thinking this?
Moving on from there, I need to devise an algorithm for this, or find an existing one or an existing 3rd party solution. I haven't stumbled across anything yet. However, I have been mulling over a couple possible methods I could implement:
Method 1:
Do something similar to how CoreData syncs sqlite DBs: send "transaction logs" to iCloud instead and build each local sqlite file off of those.
I'm thinking each device would send a (uniquely named) text file listing all the sql commands that that device executed, with timestamps. The device would store how far along in each list of commands it has executed, and continue from that point each time the file is updated. If it received updates to multiple log files at once, it would execute each command in timestamp order.
Things could get 'interesting' efficiency-wise once these files get large, but it seems like a solvable problem.
Method 2:
Periodically sync a copy of the working database to iCloud. Have a modification timestamp field in every record. When an updated copy of the DB comes through, query all the records with newer timestamps than some reference time and update the record in the local DB from the new data.
I see many potential problems with this method:
-Have to implement something further to recognize record deletion.
-The DB file could get conflicts. It might be possible to deal with them by handling each conflict version in timestamp order.
-Determining the date to check each update from could be tricky, as it depends on which device the update is coming from.
There are a lot of potential problems with method 2, but method 1 seems doable to me...
Does anyone have any suggestions as to what might be the best course of action? Any better ideas than my "Method 1" (or reasons why it wouldn't work)?
Try those two solutions from Ray Wenderlich:
Exporting/Importing data through mail:
http://www.raywenderlich.com/1980/how-to-import-and-export-app-data-via-email-in-your-ios-app
File Sharing with iTunes:
http://www.raywenderlich.com/1948/how-integrate-itunes-file-sharing-with-your-ios-app
I found it quite complex but helped me a lot.
Both method 1 and method 2 seem doable. Perhaps a combination of the two in fact - use iCloud to send a separate database file that is a subset of data - i.e. just changed items. Or maybe another file format instead of sqlite db - XML/JSON/CSV etc.
Another alternative is to do it outside of iCloud - i.e. a simple custom web service for syncing. So each change gets submitted to a central server via JSON/XML over HTTP, and then other devices pull updates from that.
Obviously it depends how much data and how many devices you want to sync across, and whether you have access to an appropriate server and/or budget to cover running such a server. iCloud will do that for "free" but all it really does is transfer files. A custom solution allows you to define your syncing model as you wish, but you have to develop and manage it and pay for it.
I've considered the possibility of transferring a database file through iCloud but I think that I would run into classic problems of timing - slow start for the user - and corrupted databases if the app is run on multiple devices simultaneously. (iPad/iPhone for example).
Sooo. I've had to use the transaction logs method. It really is difficult to implement, but once in place, seems ok.
I am using Apple's SharedCoreData sample as the base for this work. This link requires an Apple Developer Account.
I did find a much much better solution from Tim Roadley however this only works for IOS and I needed both IOS and MacOS.
rant> iCloud development really has to get easier and more stable! /rant