I have built a app using CoreData for persisting contents. Everything works fine. Now I would like to export the SQLite DB file of my stored data.
I checked the DB file's path by downloading container from my device. It seems to be "~\Library\Application Support\MainData.sqlite".
Screenshot of filepath:
I called shareDatabase() to share DB file using AirDrop to my Mac, and it works normally:
func shareDatabase() {
try? (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext.save()
let fileName = "MainData.sqlite"
let filePath = URL(fileURLWithPath: NSHomeDirectory()).appendingPathComponent("Library").appendingPathComponent("Application Support").appendingPathComponent(fileName)
let activityVC = UIActivityViewController(activityItems: [filePath], applicationActivities: nil)
present(activityVC, animated: true, completion: nil)
}
Strangely, the transferred MainData.sqlite is not the latest one, but the one with modified date of yesterday. I also used DB Browser opened the file. It does not contain the latest data.
Screenshot: info of AirDrop-shared file:
However, the MainData.sqlite from downloaded container is latest updated. The latest data are in this file, checking with DB Browser.
Screenshot: info of file in container:
Please help point out what is wrong with my codes.
Added contents:
Thank #user2782993 for referring the Apple answer on the topic: https://developer.apple.com/library/content/qa/qa1809/_index.html
I also read about this technical Q&A from apple library. But it is not very clear how to implement the mode changing option in XCode generated lazy var persistentContainer.
Also, this thread mentioned,
For sanity sake you should ensure you only have one connection to the
persistent store when you do this, i.e. only one persistent store
instance in a single persistent store coordinator.
I'm not sure how to close the existing connection. Any ideas? (better in Swift)
Try to use this:
https://stackoverflow.com/a/18870738/2782993
or this:
https://stackoverflow.com/a/20252663/2782993
Check this out:
http://pinkstone.co.uk/how-to-remove-wal-files-in-core-data/
Excerpt:
"Since iOS 7 and OS X 10.9 the default journalling mode in SQLite Stores is WAL. In addition to the main store file you’ll find a WAL file with the same (or larger) size as the store file, and a less important SHM file.
Prior to this implementation it was easy to save the context, extract the store file and ship it with an app as a pre-made data store. That’s no longer possible, because by default all data changes are written to the WAL file and do not sync with the main store file.
This is not a problem if you’re not shipping a pre-made store file with your app, but if you do, then this “improvement” has just ruined your way of delivering prewritten data stores.
Lucky for us we can switch this entire WAL business off by passing an option when creating our NSPersistentStoreCoordinator."
Here is the Apple answer on the topic:
https://developer.apple.com/library/content/qa/qa1809/_index.html
Please notice, synching the WAL data is called a checkpoint operation.
The important parts are in bold
Excerpt:
"A: The failure occurs because the default journaling mode for Core Data SQLite stores was changed to Write-Ahead Logging (WAL) in iOS 7 and OS X Mavericks. With the WAL mode, Core Data keeps the main store file untouched and appends transactions to a -wal file in the same location. After the Core Data context is saved, the -wal file is not deleted, and the data in that file is not merged to the store file either. Therefore, simply making copies of the store file will likely cause data loss and inconsistency.
...
To safely back up and restore a Core Data SQLite store, you can do the following:
Use the following method of NSPersistentStoreCoordinator class, rather than file system APIs, to back up and restore the Core Data store:
- (NSPersistentStore *)migratePersistentStore:(NSPersistentStore *)store toURL:(NSURL *)URL options:(NSDictionary *)options withType:(NSString *)storeType error:(NSError **)error
Note that this is the option we recommend.
Change to rollback journaling mode when adding the store to a persistent store coordinator if you have to copy the store file. Listing 1 is the code showing how to do this:
Listing 1 Use rollback journaling mode when adding a persistent store
NSDictionary *options = #{NSSQLitePragmasOption:#{#"journal_mode":#"DELETE"}};
if (! [persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
configuration:nil
URL:storeURL
options:options
error:&error]) {
// error handling.
}
For a store that was loaded with the WAL mode, if both the main store file and the corresponding -wal file exist, using rollback journaling mode to add the store to a persistent store coordinator will force Core Data to perform a checkpoint operation, which merges the data in the -wal file to the store file. This is actually the Core Data way to perform a checkpoint operation. On the other hand, if the -wal file is not present, using this approach to add the store won't cause any exceptions, but the transactions recorded in the missing -wal file will be lost.
Bundle the main store file and the -wal file into a document package and manipulate them as a single item.
For more information about the default journaling mode change, please see WWDC 2013 session 207 What's New in Core Data and iCloud.
NOTE: In iOS 6.x and Mountain Lion, the default is rollback journaling mode, in which Core Data creates a -journal file to temporarily store transactions, updates the main store file in place and deletes the -journal file after saving the context. The store file therefore contains the up-to-date database."
Related
Here's message from Apple:
2.23 Details
On launch and content download, your app stores 6.38 MB on the user's iCloud, which does not comply with the iOS Data Storage Guidelines.
Next Steps
Please verify that only the content that the user creates using your app, e.g., documents, new files, edits, etc. is backed up by iCloud as required by the iOS Data Storage Guidelines. Also, check that any temporary files used by your app are only stored in the /tmp directory; please remember to remove or delete the files stored in this location when it is determined they are no longer needed.
Data that can be recreated but must persist for proper functioning of your app - or because users expect it to be available for offline use - should be marked with the "do not back up" attribute. For NSURL objects, add the NSURLIsExcludedFromBackupKey attribute to prevent the corresponding file from being backed up. For CFURLRef objects, use the corresponding kCRUFLIsExcludedFromBackupKey attribute.
I checked my testing device iCloud backup and it is about ~0,3KB on launch.
I am using Realm.io database for storing data, but I set path of realm file to ..Library/Cache. Realm version 0.100.
change of path in code:
Realm.Configuration.defaultConfiguration.fileURL = NSURL(fileURLWithPath: NSSearchPathForDirectoriesInDomains(.CachesDirectory, .UserDomainMask, true)[0].stringByAppendingPathComponent("cache.realm"))
So I checked storage activity in debugger and also placement of file. But every option shows that files are actually saved in Cache folder and Documents folder is empty. Am I missing something ? What should I do next ?
Looks like you're writing something to the documents directory. What makes you think Realm is responsible writing to the user's documents? Are you perhaps using an image cacheing library which writes to the documents directory?
If you're certain that the documents directory is empty, why is Apple telling you otherwise?
If you access the Realm prior to overwriting Realm's default configuration, then it's possible you're actually writing to the documents directory until that time.
If you somehow confirm that Realm is writing to a different directory that you've specified, please file a bug report at https://github.com/realm/realm-cocoa/issues/new
I am trying to browse through the data written by Core Data in an iOS app I am developing.
After the app ran for a while, and I assume collected some data, I now wish to look through the data and see what was written.
I have tried getting and browsing the .sqlite file through getting the app container from the device (Xcode > Devices > myApp > Download Container...).
I got the db files, myAppDB.sqlite, myAppDB.sqlite-shm and myAppDB.sqlite-wal.
When trying to look through them, it seems like the .sqlite is an empty table (except maybe some generic CoreData/sqlite stuff), and the -wal file has all the info.
The thing is I was only able to know that the wal has useful data when opening it with TextEdit, which din't show it in a very readable way, and when I tried to use an SQLite Manager app I an alert saying the wal is encrypted and I am asked to put a password...
For what it matters, I am writing a framework which handles the db (the model file and the code for writing data is inside the framework), then I have this framework running in an app I am developing. This is the code I use to create the store from within the framework (using MagicalRecord):
NSBundle *frameworkBundle = [NSBundle bundleForClass:[self class]];
[MagicalRecord setDefaultModelNamed:#"myAppStore.momd" inBundle:frameworkBundle];
NSURL *storeURL = [[self applicationDocumentsDirectory] URLByAppendingPathComponent:#"myAppStoreDB.sqlite"];
[MagicalRecord setupCoreDataStackWithStoreAtURL:storeURL];
UPDATE: I managed to open the sqlite file with both Core-Data-Editor and CoreDataUtility but they both override and delete the contents of the .wal file, and show an empty table... It does have the model (entity names/properties etc.) but no data.
My wal file is 873KB but when I open the sqlite with one of these 2 tools it becomes 0Bytes...
tl;dr
How can I browse through the info written by Core Data of the app I am developing?
Well, for some reason I had to force not using WAL in my store (using #"journal_mode":#"DELETE" as explained here).
I then got only .sqlite file without the smh and wal files, and was able to open it and view the data using the 2 mentioned tools (Core-Data-Editor and CoreDataUtility).
My guess is that this is something to do with either the fact that I am dealing with CoreData from a framework (creating a moc, creating entities, saving etc.) and not from the application. Another guess is that it has something to do with the fact that I am using MagicalRecord.
Any insights regarding the cause would be appreciated...
I have had reports of users being unable to use an iOS app after updating via the App Store because the SQLite database in use by Core Data apparently becomes readonly. This occurs with a read/write persistent store that is kept in the Documents folder of the app bundle.
The persistent store is created from scratch for each user the first time they log into the app. Its contents are maintained with a custom progressive managed object model migration. As it happens, there was no migration to perform for the most recent release, and thus the persistent store should be ready to open as soon as the app is launched.
The error that we received from the users is the same, and it happens early in the process launch after the upgrade completes. The userInfo of the NSError object is what we captured in this case:
NSSQLiteErrorDomain = 264;
NSUnderlyingException = "error during prepareSQL for SQL string 'SELECT Z_VERSION, Z_UUID, Z_PLIST FROM Z_METADATA' : attempt to write a readonly database";
Due to some shortcomings in what we are logging and how we are managing the log files, I do not know precisely when this happens in the process of opening the persistent store. We are using UIManagedDocument but not sharing any documents via iCloud. I assume that the error occurs while opening an existing document package, but even that is a guess. (Because of this, I made improvements to the logging for future releases.)
The closest I have come to reproducing the error is to use the file permissions 0444 on the persistentStore-shm file in the document package of an iOS simulator app installation and then try the SELECT statement using the sqlite3 command line interface. The error in that case is SQLITE_CANTOPEN rather than SQLITE_READONLY, so I am probably not on the right track with a theory about file permissions. From what I can tell, UIManagedDocument automatically fixes file permissions anyway.
What I am wondering is if anyone has experienced behavior similar to this? If so, is there any way to recover so that users do not have to go through the process of recreating the local data store?
Add your own descriptor to the options to (re)enable history tracking...
Do this before loading the store as part of the container set up:
let container = NSPersistentContainer(name: "YourDataStoreIdentifier" )
let description = container.persistentStoreDescriptions.first
// This allows a 'non-iCloud' sycning
// container to keep track of changes
// as if it was an iCloud syncing container
description?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
container.loadPersistentStores(...
I ran in to this and it was rather puzzling. It turned out that the -shm file, the write-ahead log's index, had been cleared and needed to be rebuilt. This appears to happen automatically when loading the persistent store into an NSPersistentStoreCoordinator.
If you call +[NSPersistentStore metadataForPersistentStoreWithURL:error:] before doing that, which I was doing, you'll get the error you reported and the return value will be nil.
So the solution that worked for me is to attempt to retrieve the metadata and, if that fails with an error, to load the persistent store into an NSPersistentStoreCoordinator to rebuild the -shm. After doing that, calling metadataForPersistentStoreWithURL:error: will return the metadata without hitting the readonly error.
The WW2013 video on Core Data & iCloud mentioned that pre-iOS7 core data storage with iCloud can be migrated to the iOS7 way by specifying the NSPersistentStoreUbiquitousContentURLKey when setting up your persistent store coordinator.
Has anyone had any luck with this? With my persistent store, I had the SQLLite database in a .nosync folder and my log files in a different subdirectory. I've tried setting the NSPersistentStoreUbiquitousContentURLKey to point to each and I always get my entire database being over-written instead of everything migrating over.
I don't recall anything being said about migrating to iOS 7, I recall them indicating that to maintain compatibility with legacy apps where a custom path was specified for transaction logs you can continue using the NSPersistentStoreUbiquitousContentURLKey. This key should only be used to point to the log directory and nothing gets migrated when you use this, Core Data just uses the existing store and iCloud transaction logs.
To migrate it so that it uses the new defaults in iOS7 you would need to use the migratePersistentStore API to create a new store using a new file URL and only the NSPersistentStoreUbiquitousContentNameKey. Core Data will then automatically create the fallback and local (iCloud synced) store and iCloud transaction log files for you.
EDIT:
If anyone else is trying this an having problems try setting the store to use JOURNAL mode rather than the new default WAL mode. There seem to be some issues when doing certain migrations while using WAL mode. If anyone has figured out whether WAL mode has specific bugs please add a link here.
I would like to make backup copies of my app's main sqlite DB while my app is running.
1) I have read that it is safe to just copy the sqlite file if the DB has been checkpointed (at that point the wal file contains no important data). Does [managedContext save:] do that checkpointing, or is there something else I have to do? (ref -shm and -wal files in SQLite DB)
2) Is there any way, short of tearing down the whole core data stack, to be sure that core data doesn't try to write to the sqlite file while I'm copying it? My app does save frequently after any user input, and it would be nice if there was some way to force that to block for a second.
I have uploaded a sample app that provides backup and restore capabilities a number of different ways, including local backups, copy backups to and from iCloud, email backups, import from email, and file copy via iTunes. See link below for video demonstrating these capabilities and you can download the sample apps from the site.
http://ossh.com.au/design-and-technology/software-development/sample-library-style-ios-core-data-app-with-icloud-integration/sample-apps-explanations/backup-files/
EDIT
It should be safe to create a new persistentStoreCoordinator with the same fileURL and to then use migratePersistentStore API without closing the App, save the main MOC first though. I always use JOURNAL=DELETE mode to ensure I just have to use a single file to deal with. If you are using WAL mode then you would need to backup all three files used by sqlite.
Regarding 2)
When I make backup, I close all mom's, moc's and persistent store. Then it's safe to make backup or restore. Also all views are waiting for events to release all coredata resources and bring them back when database is available again. It's good to have a singleton to manage coredata.