I am using Core Data with a prepopulated Store with an entity that has Binary Data with External Storage checkbox enabled to save some large images externally, and every time I run the app in the simulator and executing a fetch request, a new folder called "A Document Being Saved by AppName" inside Documents is created.
The folder contains some of the images I fetched when running the app.
The problem is this folder doesn't delete on application termination, and the app keeps increasing in size currently weighting several GB!
I have started this project using the default Xcode template with Core Data enabled.
Did I Miss something ?
Bill,
That folder is a temporary store for the externally saved BLOBs. The folder is normally emptied when the save is finished. If you crash though during the save, then the data is left behind for you to handle. I normally delete the directories on startup. Basically, if the BLOB isn't moved into the CD hidden BLOB directory, then it isn't in the DB. In my application, I've had no problem. Your mileage may vary.
Andrew
Since you're using Core Data it seems likely that you want the data that your app generates to persist, right? If so, then deleting its data store when the app terminates doesn't seem very helpful. (Also, the user really shouldn't care whether the app has actually terminated or not.) Perhaps your app should instead monitor the number of images it has stored and delete the old ones as it goes along?
Related
The Situation
I have an app that stores a core data database in the documents directory. It seems to work well for the most part, except for the fact a few users (of a very large number) are complaining their data just 'disappeared'.
It's a carefully/well coded app, no weird errors or crashes coming from Core Data.
My Suspicion
iOS sometimes shows the word 'cleaning' beneath app icons when storage space is low. This cleans some directories to free up space.
Help!
Could this be the cause? If so, how can I stop this? Any light that can be shed on this would be much appreciated.
The documents directory is the recommended place to store a core data database and iOS will never "clean up" anything stored there.
Users can manually delete files in the Documents directory, by uninstalling the app or (if you've enabled it in info.plist) browsing their phone from in iTunes.
Most users do not expect their data to be destroyed when they uninstall an app (Macs and PCs would leave the data in place for example), so this is probably what's happening.
You should consider storing a second copy of the data on your server, or on the user's iCloud account. That way it won't be destroyed by an uninstall. If it's your server, then you can justify charging money for this feature (recurring revenue is good right?).
Backups to iTunes and iCloud will both include your database, so you can instruct users to restore to a recent backup to get their data back.
Also double check your code to see how it handles an out of disk space error when attempting to save changes to the database. Depending how you're using Core Data, this could go bad.
These days Core Data in iCloud or some other cloud solution is the best approach.
I am developing an app on xCode 5, iOS 7. I have some data stored in CoreData. My requirement is to upload that data to RackSpace. Whats the best way to do this?
Where can I find .sqlite file associated with CoreData?
The SQLite file is wherever you put it. There's no magic to it, you have to tell Core Data exactly where you want the file. You do this when you call addPersistentStoreWithType:configuration:URL:options:error:. The URL argument is the location of the SQLite file.
If you try and use the file directly, make sure that:
You shut down your Core Data stack completely before doing so, to make sure that all unsaved data has been flushed to disk. That means no managed objects, managed object contexts, or persistent store coordinators in memory anywhere.
Make sure to get the SQLite journal files. If your store file were named Foo.sqlite, they will be named Foo.sqlite-wal and Foo.sqlite-shm and will be located in the same directory. If you don't get these files, most or all of your data will be missing.
However simply uploading the file is not a good solution for syncing data. To sync data, you'd have to download a copy of the data, load that, and compare every object in the file with every object that's already on the phone. It's not impossible but it's definitely making things much more difficult than necessary. There are many options that can simplify the process, including full service providers like Parse, SDKs that let you use one of a variety of back ends like Ensembles.io, and others.
So my iOS app has just been rejected because it's storing too much non-user-created data in the SQLite database, which lives in the Documents folder.
The app basically involves a relatively large library of images -- around 60-or-so megabytes of them to start, and there are also in-app purchases which each add an additional 60-or-so mb of images. Furthermore, the user can add their own images to the library.
Right now everything (images and all) is stored in an SQLite database, which is generated when the app is first launched. As the user adds more images, or purchases image packs, those images are added to the database. To the user, all the images (user-generated or not) behave the same in the app.
But Apple won't allow this: I can't have all that data stored in the SQLite database in the Documents folder unless I set it specifically NOT to back up to iCloud, as it's all recreatable data.
But if I set it to not back up, then the user-generated data won't back up either, which I definitely don't want.
Any suggestions how I might "split up" the database, such that all the user-generated stuff can be backed up, but the included-or-purchased stuff isn't?
The reason you are being rejected is not following the Data Storage Guidelines. Data created or edited by the user belongs within NSDocumentsDirectory, while application data should be stored elsewhere (i.e., the Application Support directory). These requirements are a result of how iCloud backup and disk space purging work on iOS.
For a Core Data application, this means your persistent stores must be split into two different sets of files, in two different locations. This, in turn, ends up driving much of the application architecture and data model. To have relationships between the user data and application data, for instance, you must use two different managed object model configurations and the relationship must be a fetched property.
There is more detail on how to implement this in this answer.
An alternative for your specific case would be to save the images on the file system, in the caches directory or elsewhere. User images could exist in NSDocumentsDirectory while application images could exist in NSCachesDirectory. This would remove the images from Core Data and instead your model objects would have the path to the image on the file system. This would be a short term fix to get you through submission, and would probably work.
One option will be, storing your data to a server and calling is using web service. During first launch. Or as per requirement.
I'm building an app that makes extensive use of CoreData and a lot of my models have UIImage and NSData properties (for images and videos). Since it's not a great idea to store that data directly into CoreData, I built a file manager class that writes the files into different buckets in the documents directory depends on the context in which was created and media type.
My question now is how do I manage the documents directory? Is there a way to detect how much space the app has used up out of its total allocated space? Additionally, what is the best way to go about cleaning those directories; do I check every time a file is written or only on app launch, ect ect.
Is there a way to detect how much space the app has used up out of its total allocated space?
Apps don't have a limit on total allocated space, they're limited by the amount of space on the device. You can find out how much space you're using for these files by using NSFileManager to scan the directories. There are several methods that do this in different ways-- check out enumeratorAtPath:, for example. For each file, use a method like attributesOfItemAtPath:error: to get the file size.
Better would be to track the file sizes as you create and delete files. Keep a running total, stored in user defaults. When you create a new file, increase it by the amount of new data. When you remove a file, decrease the running total.
Additionally, what is the best way to go about cleaning those directories; do I check every time a file is written or only on app launch, ect ect.
If these files are local data that's inherently part of the associated Core Data object, the sensible approach is to delete a file when its Core Data object is deleted. The managed object needs the data file, so don't delete the file if you still use the object. That means there must be some way to link the two, but I'm assuming that's already true since you say that these files are used by managed objects somehow.
If the files are something like cached data that's easily re-created or re-downloaded, you should put them in the location returned by NSTemporaryDirectory(). Then iOS can delete them when it thinks the space is needed. You can also clear out old files whenever it seems appropriate, by scanning for older files or ones that haven't been used in a while (the details depend on exactly how you use the files).
I've been having trouble getting an app submitted to the App Store. This is due to the fact that that database, which is updatable, is too large for the iCloud backup limitations. Most of the data in the db is static, but one table records the user's schedule for reviewing words (this is a vocabulary quiz).
As far as I can tell, I have two or three realistic options. The first is to put the whole database into the Library/Cache directory. This should be accepted, because it's not backed up to iCloud. However, there's no guarantee that it will be maintained during app updates, per this entry in "Make App Backups More Efficient" at this url:
http://developer.apple.com/library/IOs/#documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/PerformanceTuning/PerformanceTuning.html
Files Saved During App Updates
When a user downloads an app update, iTunes installs the update in a new app directory. It then moves the user’s data files from the old installation over to the new app directory before deleting the old installation. Files in the following directories are guaranteed to be preserved during the update process:
<Application_Home>/Documents
<Application_Home>/Library
Although files in other user directories may also be moved over, you should not rely on them being present after an update.
The second option is to put the data into the NSDocuments or NSLibrary directory, as mark it with the skipBackupFlag. However, one problem is this flag doesn't work for iOS 5.0 and previous per this entry in "How do I prevent files from being backed up to iCloud and iTunes?" at
https://developer.apple.com/library/ios/#qa/qa1719/_index.html
Important The new "do not back up" attribute will only be used by iOS 5.0.1 or later. On iOS 5.0 and earlier, applications will need to store their data in <Application_Home>/Library/Caches to avoid having it backed up. Since this attribute is ignored on older systems, you will need to insure your app complies with the iOS Data Storage Guidelines on all versions of iOS that your application supports
This means that even if I use the "skipBackupFlag", I'll still have the problem that the database is getting backed up to the cloud, I think.
So, the third option, which is pretty much of an ugly hack, is to split the database into two. Put the updatable part into the NSLibrary or NSDocuments directory, and leave the rest in application resources. This would have the small, updatable part stored on the cloud, and leave the rest in the app resources directory. The problem is that this splits the db for no good reason, and introduces possible performance issues with having two databases open at once.
So, my question is, is my interpretation of the rules correct? Am I going to have to go with option 3?
p.s. I noticed in my last post cited urls were edited to links without the url showing. How do I do this?
Have you considered using external file references as described in https://developer.apple.com/library/IOS/#releasenotes/DataManagement/RN-CoreData/_index.html . Specifically, refer to "setAllowsExternalBinaryDataStorage:" https://developer.apple.com/library/IOS/documentation/Cocoa/Reference/CoreDataFramework/Classes/NSAttributeDescription_Class/reference.html#//apple_ref/occ/instm/NSAttributeDescription/setAllowsExternalBinaryDataStorage: . Pushing out large data into a separate file can help reduce database size .