The Situation
I have an app that stores a core data database in the documents directory. It seems to work well for the most part, except for the fact a few users (of a very large number) are complaining their data just 'disappeared'.
It's a carefully/well coded app, no weird errors or crashes coming from Core Data.
My Suspicion
iOS sometimes shows the word 'cleaning' beneath app icons when storage space is low. This cleans some directories to free up space.
Help!
Could this be the cause? If so, how can I stop this? Any light that can be shed on this would be much appreciated.
The documents directory is the recommended place to store a core data database and iOS will never "clean up" anything stored there.
Users can manually delete files in the Documents directory, by uninstalling the app or (if you've enabled it in info.plist) browsing their phone from in iTunes.
Most users do not expect their data to be destroyed when they uninstall an app (Macs and PCs would leave the data in place for example), so this is probably what's happening.
You should consider storing a second copy of the data on your server, or on the user's iCloud account. That way it won't be destroyed by an uninstall. If it's your server, then you can justify charging money for this feature (recurring revenue is good right?).
Backups to iTunes and iCloud will both include your database, so you can instruct users to restore to a recent backup to get their data back.
Also double check your code to see how it handles an out of disk space error when attempting to save changes to the database. Depending how you're using Core Data, this could go bad.
These days Core Data in iCloud or some other cloud solution is the best approach.
Related
Let me get this out of the way right now: yes, it was almost certainly a mistake to not use Core Data. However, I was new to iOS development when I made these decisions, and I had no idea I'd be hamstrung like this. Moreover, the app is intended to also run on Android (eventually), so I avoided platform-specific APIs wherever possible.
I have an iOS app that stores data in a local SQLite database file. The data stored in the file is provided by the user, so it's important that it be kept safe. I had plans to "do this later", and later is now here. I am quickly coming to the realization that it won't be as straightforward as I had hoped...
I now understand that it won't be possible to seamlessly synchronize data across devices, and I'm willing to accept that limitation until I manage to migrate to Core Data. However, in the meantime I'd at least like the SQLite database to be backed up periodically so users can feel safe using the app on a single device. I was thinking I would do this:
periodically (e.g. once a week) copy the SQLite file from local storage into cloud storage, thus ensuring it is backed up
when the app starts, if the local store is missing or corrupted but the file exists in the cloud storage, ask the user if they would like to copy it over
The biggest problem with this approach is that the user could run the app on multiple devices and therefore the data stored in iCloud could be from any one of those devices, but only one. To combat that, I thought I could just use a per-device, unique name for the file in cloud storage. I would generate this using UIDevice.identifierForVendor.
So my startup logic would be:
Determine the unique name for the cloud file.
Is the local file missing or corrupted, and if so, does the cloud file exist?
2.1. Ask the user if they would like to restore from the cloud file. Make it really hard for them to say no because doing so will lose all their data.
2.2. If they say yes, copy the cloud file to the local file storage.
Open the local database file.
And running in the background I would occasionally copy the database file from local to cloud storage.
I would like to know whether this a sensible approach until I do Core Data integration. Also, are there any hidden "gotchas" that I'm perhaps missing?
UPDATE: as #TomHarrington pointed out in a comment, it turns out my database file is already sitting in /Documents, which is backed up to iTunes and any iCloud account. So my question morphs into this:
Should I simply ensure my database has a device-specific name so that it is not clobbered by the app running on another device connected to the same iCloud account?
I'm going to answer my question, since I ended up going down this path and finding a MASSIVE blocker. There is a bug in the UIDevice.identifierForVendor API that causes it to regenerate every time a new version of the app is installed! See here. This of course rules out using it as a device identifier. sigh
I think I'm SOL with that approach. Instead, I might generate a GUID on first execution and use that as my identifier. Problem is, I need to store that somewhere that isn't backed up to iCloud.
Ugh, I may just give up here and say my app can't be run on multiple devices until Core Data integration is done.
UPDATE: I ended up generating an identifier on first run and storing it in the keychain (as a local entry only so it isn't backed up to iCloud).
The app I'm working on generally uses a single SQL store in Core Data to hold a working dataset for the app. However, there can be any number of small, separate stores which originated as either
In-app Purchased content, or
Exported subsets of the working dataset that may be reused from time to time.
These will always consist of a single sqlite file (WAL turned off). The In-App purchases can always be re-downloaded, but it would be possible for the exported data to be lost for good (since they might choose to permanently delete the data after it is exported).
Intuitively, since the exported data might not be possible to recreate, it should go in a subdirectory of the documents folder. But I'm not sure whether Apple would agree about that.
I have no idea where the downloaded purchased content should be saved, since technically, it can be re-downloaded at any time.
So, my question is, where in the iOS filesystem should I put these (iOS7 and beyond).
There is no official, public guidance on where content purchased via IAP should go on the filesystem. However, experience with Data Storage Guidelines feedback has indicated that:
NSDocumentsDirectory should only contain data created or edited by the user
(some) Non-user data can still be stored in NSDocumentsDirectory if it has the NSURLIsExcludedFromBackupKey attribute set. In general you should still try and avoid doing this.
In-App Purchase content is considered "restorable application data" and should be stored accordingly. The application support directory is one place where it could be stored.
If your exported data was exported by a user-initiated action and the data at that point "belongs" to the user, storing that exported data in the NSDocumentsDirectory should be fine. If you have any doubt, store it in the application support directory or caches directory. Be aware that in low space conditions the data may be purged by the system.
I have a CoreData app (using https://github.com/lhunath/UbiquityStoreManager), backed by iCloud. In one use case a user with a local store enables iCloud (where data already exists). I want to prompt the user to make a decision of whether to migrate the local data to iCloud or just use the iCloud version. As part of this, I'd like to display the device name and last sync date of the version in iCloud.
I've been tinkering around with my NSPersistentStore's metadata, but that doesn't appear to get synced to iCloud.
Any suggestions?
You could use iCloud's key-value store to store the device name & date of the last sync.
My no doubt unpopular suggestion is "don't". Trying to determine what is in iCloud at any given time puts you on pretty shaky ground. You may be able to get it to work most of the time, but there will always be circumstances where it breaks down.
If you really must import some data when first enabling iCloud, I suggest just always importing the data, and then deduping later as the iCloud data comes in. As ugly as it sounds, that's the only approach really guaranteed to work with Apple's approach.
It is worth taking a look at other Core Data sync frameworks like TICDS and Ensembles. They take a more sane approach to data identity, which means you can avoid the whole deduping step. (Disclosure: I develop Ensembles)
do a metadata query on the iCloud files and check the most recent transaction log file in iCloud. See the link below for a sample app that uses this approach to check whether the app is properly synchronised with iCloud.
http://ossh.com.au/design-and-technology/software-development/sample-library-style-ios-core-data-app-with-icloud-integration/
EDIT:
I just realised I don't get the actual device name, but once you have found the most recent log file then use this to get the device. Just be aware this call may be expensive.
NSFileVersion *openedVersion = [NSFileVersion currentVersionOfItemAtURL:fileURL];
return openedVersion.localizedNameOfSavingComputer;
I've been having trouble getting an app submitted to the App Store. This is due to the fact that that database, which is updatable, is too large for the iCloud backup limitations. Most of the data in the db is static, but one table records the user's schedule for reviewing words (this is a vocabulary quiz).
As far as I can tell, I have two or three realistic options. The first is to put the whole database into the Library/Cache directory. This should be accepted, because it's not backed up to iCloud. However, there's no guarantee that it will be maintained during app updates, per this entry in "Make App Backups More Efficient" at this url:
http://developer.apple.com/library/IOs/#documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/PerformanceTuning/PerformanceTuning.html
Files Saved During App Updates
When a user downloads an app update, iTunes installs the update in a new app directory. It then moves the user’s data files from the old installation over to the new app directory before deleting the old installation. Files in the following directories are guaranteed to be preserved during the update process:
<Application_Home>/Documents
<Application_Home>/Library
Although files in other user directories may also be moved over, you should not rely on them being present after an update.
The second option is to put the data into the NSDocuments or NSLibrary directory, as mark it with the skipBackupFlag. However, one problem is this flag doesn't work for iOS 5.0 and previous per this entry in "How do I prevent files from being backed up to iCloud and iTunes?" at
https://developer.apple.com/library/ios/#qa/qa1719/_index.html
Important The new "do not back up" attribute will only be used by iOS 5.0.1 or later. On iOS 5.0 and earlier, applications will need to store their data in <Application_Home>/Library/Caches to avoid having it backed up. Since this attribute is ignored on older systems, you will need to insure your app complies with the iOS Data Storage Guidelines on all versions of iOS that your application supports
This means that even if I use the "skipBackupFlag", I'll still have the problem that the database is getting backed up to the cloud, I think.
So, the third option, which is pretty much of an ugly hack, is to split the database into two. Put the updatable part into the NSLibrary or NSDocuments directory, and leave the rest in application resources. This would have the small, updatable part stored on the cloud, and leave the rest in the app resources directory. The problem is that this splits the db for no good reason, and introduces possible performance issues with having two databases open at once.
So, my question is, is my interpretation of the rules correct? Am I going to have to go with option 3?
p.s. I noticed in my last post cited urls were edited to links without the url showing. How do I do this?
Have you considered using external file references as described in https://developer.apple.com/library/IOS/#releasenotes/DataManagement/RN-CoreData/_index.html . Specifically, refer to "setAllowsExternalBinaryDataStorage:" https://developer.apple.com/library/IOS/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSAttributeDescription_Class/reference.html#//apple_ref/occ/instm/NSAttributeDescription/setAllowsExternalBinaryDataStorage: . Pushing out large data into a separate file can help reduce database size .
My app was recently rejected from the app store for storing data in the Documents directory. I had moved it there because with the latest change, the db must now be writeable - it's no longer read-only.
In researching the solution, I've read that it's actually preferable to use NSLibraryDirectory. Is this so, and more importantly, will that address Apple's concerns? Their complaint is that the app lets the user download to much content (it doesn't let the user download any unless you count the db), and that it's storing "too much data" in the "incorrect location." The data is 8 mb, but could grow to about 10 or 12 mb max.
Actually its because of iCloud.
using iCloud, Application's document directory is synced to cloud and to other devices and hence Apple want developers to store only that data in document directory which they want to sync with iCloud.
I came to know this form one of my friends who work # Apple California and I'm not really sure if this material is on Apple's documentation.
I had this issue with an update to a suite of apps I develop the other week. The funny thing was that only five of the seven apps (exactly the same code base) were rejected.
I believe the issue in my case was duplicating assets from the .app bundle into ~/Documents.
My first attempt to comply with their new storage guidelines was to implement the do not backup switch on the files I was copying into ~/Documents. No deal with that so I had to change my implementation to not copy the data at all. The apps were promptly approved.
Your implementation is probably different but in my experience Apple no longer likes you copying things from the app bundle into ~/Documents, as it could be duplicating data unnecessarily (in their view).
They suggest copying into ~/Caches (or whatever it is), but this can be cleared in low storage situations and may not be right for your case either.
Hope that helps.
A product I wrote a year ago uses CoreData (with a SQLite data store). This database file is stored in ~/Library/Application Support//. This was approved by Apple without issue.
"Application Support" does not exist in ~/Library, so you will need to create it.
Documents is not a great place to store your database file for several reasons. Apple has their reasons, since they rejected your app. Another reason is that the Documents directory is accessible by the user (via iTunes), and unless the user deleting your database file is no big deal to the operation of your app, it is best to put it where they can not do anything with it directly and/or inadvertently.
You can still put those files in the Documents folder, you just have to give them an attribute that lets the file system know not to back them up to the iCloud
This is a great example of how to do so on different iOS versions