I have had reports of users being unable to use an iOS app after updating via the App Store because the SQLite database in use by Core Data apparently becomes readonly. This occurs with a read/write persistent store that is kept in the Documents folder of the app bundle.
The persistent store is created from scratch for each user the first time they log into the app. Its contents are maintained with a custom progressive managed object model migration. As it happens, there was no migration to perform for the most recent release, and thus the persistent store should be ready to open as soon as the app is launched.
The error that we received from the users is the same, and it happens early in the process launch after the upgrade completes. The userInfo of the NSError object is what we captured in this case:
NSSQLiteErrorDomain = 264;
NSUnderlyingException = "error during prepareSQL for SQL string 'SELECT Z_VERSION, Z_UUID, Z_PLIST FROM Z_METADATA' : attempt to write a readonly database";
Due to some shortcomings in what we are logging and how we are managing the log files, I do not know precisely when this happens in the process of opening the persistent store. We are using UIManagedDocument but not sharing any documents via iCloud. I assume that the error occurs while opening an existing document package, but even that is a guess. (Because of this, I made improvements to the logging for future releases.)
The closest I have come to reproducing the error is to use the file permissions 0444 on the persistentStore-shm file in the document package of an iOS simulator app installation and then try the SELECT statement using the sqlite3 command line interface. The error in that case is SQLITE_CANTOPEN rather than SQLITE_READONLY, so I am probably not on the right track with a theory about file permissions. From what I can tell, UIManagedDocument automatically fixes file permissions anyway.
What I am wondering is if anyone has experienced behavior similar to this? If so, is there any way to recover so that users do not have to go through the process of recreating the local data store?
Add your own descriptor to the options to (re)enable history tracking...
Do this before loading the store as part of the container set up:
let container = NSPersistentContainer(name: "YourDataStoreIdentifier" )
let description = container.persistentStoreDescriptions.first
// This allows a 'non-iCloud' sycning
// container to keep track of changes
// as if it was an iCloud syncing container
description?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
container.loadPersistentStores(...
I ran in to this and it was rather puzzling. It turned out that the -shm file, the write-ahead log's index, had been cleared and needed to be rebuilt. This appears to happen automatically when loading the persistent store into an NSPersistentStoreCoordinator.
If you call +[NSPersistentStore metadataForPersistentStoreWithURL:error:] before doing that, which I was doing, you'll get the error you reported and the return value will be nil.
So the solution that worked for me is to attempt to retrieve the metadata and, if that fails with an error, to load the persistent store into an NSPersistentStoreCoordinator to rebuild the -shm. After doing that, calling metadataForPersistentStoreWithURL:error: will return the metadata without hitting the readonly error.
Related
My app under development uses core data.
When I add an optional attribute to an entity, I expected that automatic lightweight migration is done, because I use an NSPersistentCloudKitContainer, and all my NSPersistentStoreDescription have the default settings shouldMigrateStoreAutomatically = true and shouldInferMappingModelAutomatically = true.
After changing the xcdatamodeld file by adding an attribute and restarting the app with the old persistent store, no data is fetched, and no error is reported.
However when I re-install the app, everything works fine, because a new persistent store is filled with the iCloud data.
Maybe I can detect that the model has changed, delete the old persistent store, and setup a new one that is synced then with iCloud.
But it should also work without iCloud sync: I thought the old persistent store should be automatically migrated to the new model, i.e., the newly added attribute should be initialized with nil.
Is this a misunderstanding? Do I miss something? How to do the migration correctly?
PS: I have read this answer, but since my app is still under development, I thought I don't need different versions of the model, but could simply update it.
I have built a app using CoreData for persisting contents. Everything works fine. Now I would like to export the SQLite DB file of my stored data.
I checked the DB file's path by downloading container from my device. It seems to be "~\Library\Application Support\MainData.sqlite".
Screenshot of filepath:
I called shareDatabase() to share DB file using AirDrop to my Mac, and it works normally:
func shareDatabase() {
try? (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext.save()
let fileName = "MainData.sqlite"
let filePath = URL(fileURLWithPath: NSHomeDirectory()).appendingPathComponent("Library").appendingPathComponent("Application Support").appendingPathComponent(fileName)
let activityVC = UIActivityViewController(activityItems: [filePath], applicationActivities: nil)
present(activityVC, animated: true, completion: nil)
}
Strangely, the transferred MainData.sqlite is not the latest one, but the one with modified date of yesterday. I also used DB Browser opened the file. It does not contain the latest data.
Screenshot: info of AirDrop-shared file:
However, the MainData.sqlite from downloaded container is latest updated. The latest data are in this file, checking with DB Browser.
Screenshot: info of file in container:
Please help point out what is wrong with my codes.
Added contents:
Thank #user2782993 for referring the Apple answer on the topic: https://developer.apple.com/library/content/qa/qa1809/_index.html
I also read about this technical Q&A from apple library. But it is not very clear how to implement the mode changing option in XCode generated lazy var persistentContainer.
Also, this thread mentioned,
For sanity sake you should ensure you only have one connection to the
persistent store when you do this, i.e. only one persistent store
instance in a single persistent store coordinator.
I'm not sure how to close the existing connection. Any ideas? (better in Swift)
Try to use this:
https://stackoverflow.com/a/18870738/2782993
or this:
https://stackoverflow.com/a/20252663/2782993
Check this out:
http://pinkstone.co.uk/how-to-remove-wal-files-in-core-data/
Excerpt:
"Since iOS 7 and OS X 10.9 the default journalling mode in SQLite Stores is WAL. In addition to the main store file you’ll find a WAL file with the same (or larger) size as the store file, and a less important SHM file.
Prior to this implementation it was easy to save the context, extract the store file and ship it with an app as a pre-made data store. That’s no longer possible, because by default all data changes are written to the WAL file and do not sync with the main store file.
This is not a problem if you’re not shipping a pre-made store file with your app, but if you do, then this “improvement” has just ruined your way of delivering prewritten data stores.
Lucky for us we can switch this entire WAL business off by passing an option when creating our NSPersistentStoreCoordinator."
Here is the Apple answer on the topic:
https://developer.apple.com/library/content/qa/qa1809/_index.html
Please notice, synching the WAL data is called a checkpoint operation.
The important parts are in bold
Excerpt:
"A: The failure occurs because the default journaling mode for Core Data SQLite stores was changed to Write-Ahead Logging (WAL) in iOS 7 and OS X Mavericks. With the WAL mode, Core Data keeps the main store file untouched and appends transactions to a -wal file in the same location. After the Core Data context is saved, the -wal file is not deleted, and the data in that file is not merged to the store file either. Therefore, simply making copies of the store file will likely cause data loss and inconsistency.
...
To safely back up and restore a Core Data SQLite store, you can do the following:
Use the following method of NSPersistentStoreCoordinator class, rather than file system APIs, to back up and restore the Core Data store:
- (NSPersistentStore *)migratePersistentStore:(NSPersistentStore *)store toURL:(NSURL *)URL options:(NSDictionary *)options withType:(NSString *)storeType error:(NSError **)error
Note that this is the option we recommend.
Change to rollback journaling mode when adding the store to a persistent store coordinator if you have to copy the store file. Listing 1 is the code showing how to do this:
Listing 1 Use rollback journaling mode when adding a persistent store
NSDictionary *options = #{NSSQLitePragmasOption:#{#"journal_mode":#"DELETE"}};
if (! [persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
configuration:nil
URL:storeURL
options:options
error:&error]) {
// error handling.
}
For a store that was loaded with the WAL mode, if both the main store file and the corresponding -wal file exist, using rollback journaling mode to add the store to a persistent store coordinator will force Core Data to perform a checkpoint operation, which merges the data in the -wal file to the store file. This is actually the Core Data way to perform a checkpoint operation. On the other hand, if the -wal file is not present, using this approach to add the store won't cause any exceptions, but the transactions recorded in the missing -wal file will be lost.
Bundle the main store file and the -wal file into a document package and manipulate them as a single item.
For more information about the default journaling mode change, please see WWDC 2013 session 207 What's New in Core Data and iCloud.
NOTE: In iOS 6.x and Mountain Lion, the default is rollback journaling mode, in which Core Data creates a -journal file to temporarily store transactions, updates the main store file in place and deletes the -journal file after saving the context. The store file therefore contains the up-to-date database."
First post on StackOverflow, after extensively using it for a long time.
I'm building a small app (just to lear swift), and I have troubles with making some data persistent. I use NSCoding to achieve that. The problem is that when saving, the function NSKeyedArchiver.archiveRootObject() return true (so apparently it worked), but when, later, I try to retrieve these saved informations, the result of NSKeyedUnarchiver.unarchiveObjectWithFile() is nil.
Without posting all my code, I was just wondering if it were possible to explore the file in which persistent data are stored during a debug session. That would allow me to check whether I have a problem with the saving or the loading part of the process, and see if the data are indeed stored in the right file.
Thanks,
Lb
Per Apple Documentation, you get a nil with unarchiveObjectWithFile when there is no file at the mentioned path. I would advise to check your file path where you are archiving and saving your object.
As for the debugging, follow this:
Print the file path when you are archiving and saving the object. So in this NSKeyedArchiver.archiveRootObject(myObject, toFile: filePath) print filePath.
Open your terminal app and execute open <filePath> command to open the file path where data is being saved.
Check out if your data file is created in there with right archiving.
The WW2013 video on Core Data & iCloud mentioned that pre-iOS7 core data storage with iCloud can be migrated to the iOS7 way by specifying the NSPersistentStoreUbiquitousContentURLKey when setting up your persistent store coordinator.
Has anyone had any luck with this? With my persistent store, I had the SQLLite database in a .nosync folder and my log files in a different subdirectory. I've tried setting the NSPersistentStoreUbiquitousContentURLKey to point to each and I always get my entire database being over-written instead of everything migrating over.
I don't recall anything being said about migrating to iOS 7, I recall them indicating that to maintain compatibility with legacy apps where a custom path was specified for transaction logs you can continue using the NSPersistentStoreUbiquitousContentURLKey. This key should only be used to point to the log directory and nothing gets migrated when you use this, Core Data just uses the existing store and iCloud transaction logs.
To migrate it so that it uses the new defaults in iOS7 you would need to use the migratePersistentStore API to create a new store using a new file URL and only the NSPersistentStoreUbiquitousContentNameKey. Core Data will then automatically create the fallback and local (iCloud synced) store and iCloud transaction log files for you.
EDIT:
If anyone else is trying this an having problems try setting the store to use JOURNAL mode rather than the new default WAL mode. There seem to be some issues when doing certain migrations while using WAL mode. If anyone has figured out whether WAL mode has specific bugs please add a link here.
The following line fails to persist any data, although there is a sqlite file at library/Application Support/Appname/Appname.sqlite that has header information as set from the model.
[MagicalRecord setupCoreDataStackWithiCloudContainer:#"128DBLAH422.com.company.appname.ubiquitycoredata" localStoreNamed:#"Appname.sqlite"];
ICloud is also started, as the kMagicalRecordPSCDidCompleteiCloudSetupNotification notification is fired.
I loose all data on the next run, although I am able to save correctly throughout run-time ('Finished saving' on main thread) and have all (as far as I've seen) data available in-memory.
The following saves and persists properly w/ an sqlite file at the same directory path:
[MagicalRecord setupCoreDataStackWithAutoMigratingSqliteStoreNamed:#"Appname.sqlite"];
I wonder why when iCloud is enabled that I'm seeing the DB in "library/Application Support/Appname/". Is that correct?
On one device w/ 5.0.1, I get the following error on runs after first run:
Error: Can't resolve how to assign objects to stores; Coordinator does not have any stores
on iPad and simulator, no error, but no data also.
Also... After first run with setup core data with icloud (1st line in did finish launching):
[MagicalRecord setupCoreDataStackWithiCloudContainer:#"128DBLAH422.com.company.appname.ubiquitycoredata" localStoreNamed:#"Appname.sqlite"];
any future run using
[MagicalRecord setupCoreDataStackWithAutoMigratingSqliteStoreNamed:#"Appname.sqlite"];
instead of icloud setup will recover the data that wouldn't have appeared on next run otherwise. If I add data on first run with icloud, re-run with icloud and add no data, then run without icloud, the data is there.
Is there a method of loading the store on subsequent runs that is unique from when creating it for the first time?