I have an iOS app which stores it's data in a Core Data database. The basic structure has Libraries and Items, with a many-to-many relationship between them represented by LibraryItemMapping. The mapping item contains an index, so that items can be sorted differently in different containers. This works fine most of the time, but on my own device which I use for dev/test, I am running into situations where the Core Data objects are returning nil for one side or the other of those relationships. If I enumerate those objects at launch (via code in AppDelegate), the nils disappear and I'm able to retrieve objects on either side of the relationship, but when looking for the same relationships just a few seconds later they begin to return nil. This, for example, always runs OK and never errors out or finds any nils:
public func runConsistencyCheck() {
do {
let mappings = try dataContext.fetch(NSFetchRequest<LibraryItemMapping>(entityName: "LibraryItemMapping"))
for map in mappings {
if (map.item==nil || map.library==nil) {
print("Found corrupt mapping record. Deleting.")
dataContext.delete(map)
}
}
saveContext()
} catch {
print("An error occurred during consistency checking: \(error)")
}
}
But this regularly finds nil for obj.item and/or crashes on my device:
func getMapping(forItem item: Item) -> LibraryItemMapping? {
if (libraryData?.itemMappings != nil) {
for obj in libraryData!.itemMappings as! Set<LibraryItemMapping> {
if (obj.item != nil && obj.item! == item) { return obj }
else {
print("Found Core Data mapping with null item data.")
}
}
}
return nil
}
Attempting to create or modify and then save any other records results in multiple iterations of this error:
"Error Domain=NSCocoaErrorDomain Code=1570 \"item is a required value.\" UserInfo={NSValidationErrorKey=item, NSLocalizedDescription=item is a required value., NSValidationErrorObject=<LibraryItemMapping: 0x174098060> (entity: LibraryItemMapping; id: 0xd0000000001c0000 <x-coredata://ED9F1AE5-A7D0-46FD-AC94-941E9EFEF341/LibraryItemMapping/p7> ; data: {\n index = 4;\n item = nil;\n library = \"0xd000000000140004 <x-coredata://ED9F1AE5-A7D0-46FD-AC94-941E9EFEF341/LibraryData/p5>\";\n})}"
This only happens on the one device that I am aware of, so I'm tempted to put it down to a database corrupted by a bad debug session at some point, but at the very least I'd like to be able to clean up after it gracefully. If this were SQL, I would just run a consistency check and something like delete from LibraryItemMapping where item is null or library is null and move on, but I'm not sure how to do the equivalent with Core Data. How can I troubleshoot this further, and how can I gracefully recover from a seemingly corrupt database?
After pulling the sqlite files from the device and digging through the raw data, this actually turned out to be a data model bug. During app startup it goes through a "prune old records" step, and inconsistent delete rules on the relationships for the linking table allowed it to leave records in an inconsistent state. After correcting the delete rules to use Cascade or Nullify as appropriate, everything is working again.
Related
So my goal is to get rid of these bugs completely. I am in a dilemma where each decision leads to a bug.
The first thing I can do that eventually becomes an issue is use a String-interpolated collection path in all my query functions like so:
func getEventName() {
listener = db.collection("school_users/\(user?.uid)/events").order(by: "time_created", descending: true).addSnapshotListener(includeMetadataChanges: true) { (querySnapshot, error) in
if let error = error {
print("There was an error fetching the data: \(error)")
} else {
self.events = querySnapshot!.documents.map { document in
return EventName(eventName: (document.get("event_name") as! String))
}
self.tableView.reloadData()
}
}
}
The thing with this is, when I run the app on the simulator, I am restricted from pressing buttons and then sometimes I can press them and then sometimes they get restricted again. This bug is so confusing because it makes no sense where it springs from.
The other issue is I can use a Constants value in all the query functions in my collections path.
static let schoolCollectionName = "school_users/\(user?.uid)/events"
This is nested in a Firebase struct within the Constants struct. In order to keep Xcode from giving errors I create a let users = Auth.auth().currentUser variable outside the Constants struct. The issue with this value is that when I put that in all of my query functions collection paths, all the buttons are accessible and selectable all the time, but when a user logs out and I log in as a new user, the previous user's data shows up in the new user's tableview.
It would obviously make more sense to use the Constants value because you prevent typos in the future, but I can't figure out how to get rid of the bug where the old user's data shows up in the new user's tableview. Thanks in advance.
The user id should definitely not be a constant. What it sounds like is that right now, you have no reliable way to change users -- your setup probably depends on which user is logged in at app startup, since that's where your variable gets set.
I would do something more like this:
func getEventName() {
guard let user = Auth.auth().currentUser else {
//handle the fact that you don't have a user here -- don't go on to the next query
return
}
listener = db.collection("school_users/\(user.uid)/events").order(by: "time_created", descending: true).addSnapshotListener(includeMetadataChanges: true) { (querySnapshot, error) in
Note that now, user.uid in the interpolated path doesn't have the ? for optionally unwrapping it (which Xcode is giving you a warning for right now). It will also guarantee that the correct query is always made with the currently-logged-in user.
Regarding being able to press the buttons, that sounds like an unrelated issue. You could run your app in Instruments and check the Time Profiler to see if you have long-running tasks that are gumming up the main/UI thread.
Forgive me as I tend to not ask questions so I hope I'm doing this right. I have a sqlite backed CoreData stack.
Question: Is there a way to delete data from CoreData based on the size of the NSPersistentStore and to resize said store?
Background: I plan on writing lots of data to different tables. Some of these tables are in separate NSPersistentStore and in my case that means separate sqlite files.
I had written a process for checking if the sqlite file had exceeded a certain amount of megabytes so that I could delete some data from the file.
It seems that after I delete rows from the database the file stays the same size and doesn't resize. The NSSQLiteManualVacuumOption was set on the persistent store so I expected this file to resize eventually but it never does which causes my process to keep trying to delete my rows.
I even tried connecting to the file and forcing the VACUUM command using sql and that seemed to only work sometimes.
func forceCleanUp {
deleteDatabaseRows()
//Perform Cleanup
var db: OpaquePointer?
guard sqlite3_open(path, &db) == SQLITE_OK else {
print("error opening database")
sqlite3_close(db)
db = nil
return
}
//Probably not a good idea.
if sqlite3_exec(db, "VACUUM", nil, nil, nil) != SQLITE_OK {
let errmsg = String(cString: sqlite3_errmsg(db)!)
print("error running VACCUUM: \(errmsg)")
}
}
Something tells me that even if this DID work consistently it wouldn't be a good idea as it messes around with underlying settings.
In my app, I have a screen where the user can edit an object's attributes. The screen has 2 actions: save or revert.
When revert is selected, I want to revert the object's attributes to whatever is stored on the server and discard all local changes. I do this with the the fetch method:
println("1: \(localObject)")
if localObject.isDirty() {
localObject.fetchInBackgroundWithBlock { serverObject, error in
println("2: \(localObject)")
println("3: \(serverObject)")
}
}
isDirty() works correctly and only returns true if something was locally modified and not saved.
According to the docs, fetch: fetches the PFObject with the current data from the server. But what I see in the log is that all 3 print statements show the same local attributes. I would expect serverObject to be whatever is stored on the server, but it's not, serverObject also shows the locally modified attributes.
I'm not using the Local Datastore.
What is going on? Is this a bug with the Parse SDK? Or is there a caching issue?
If you call fetch without using the local datastore, Parse will give you a error in the console.
Since you're not getting an error from Parse when you call fetchInBackgroundWithBlock, it means that at some point you've called localObject.pinInBackground() and placed that object in the local datastore.
For locally stored objects, you can call revert() before fetchInBackgroundWithBlock to discard local changes before updating data from the server.
if localObject.isDirty() {
localObject.revert()
localObject.fetchInBackgroundWithBlock { serverObject, error in
println("2: \(localObject)")
println("3: \(serverObject)")
}
}
I haven't seen the docs for fetch, but I do know that you can accomplish what you're trying just by performing a query. Doing the following will query from the server (although you can adjust the cache policy if you've already used the query to look at cache or network first):
var query = PFQuery(className:"GameScore")
query.getObjectInBackgroundWithId("xWMyZEGZ") {
(gameScore: PFObject?, error: NSError?) -> Void in
if error == nil && gameScore != nil {
println(gameScore)
}else {
println(error)
}
}
Alternatively, you could just save a copy of the local object before any changes can be made, and then if it is dirty, you can just replace it with the copy. That's probably what I'd do to reduce the number of parse API calls.
I would like to have synchronized http request results with my Core Data database. For example very basic class:
class Category: NSManagedObject {
#NSManaged var id: NSNumber
#NSManaged var name: String
#NSManaged var imageUrl: String
}
And I have this method for getting results from request:
apiManager.getRequestJsonParse(Constants.Server.BackendUrl + "categories?lang=cs", completion: { (result, error) -> Void in
completion(categories: [], error: error)
})
Result is dictionary parsed from json. It's okay but now I want to parse dictionary to my class, check if exists in database, if it exists update properties and if not than add new category to database.
My approach that I was using in apps before was that I have two classes: Category and CategoryCD (CD like Core Data) and first I parse json to Category class than for all categories I check if any CategoryCD (saved in CD) has same Id and then update, or add or other things.
Now I am thinking if there is better way how can I do it. I could each time when I download new results delete my database for this class and then add all results. The problem with this way is that what if I have something for some classes that I want keep. Like if I download and save images then I would rather still have connection between class and saved image.
I was thinking about my old approach but reduce 2 almost same classes (1 for Core Data and 1 same but for parsing) to 1 Core Data class. But then there is a problem when I init this class I must always create in database right? So It could be complicated.
What are you using for this? I think it's very common problem. You have list of items and I would like to have them available offline (with all data I downloaded before) but than when I connect to internet I would like to update with new results (not download all from server, just responses which I requested).
It is a very common problem and it has been solved hundreds of ways. BTW, This is not "synchronizing" it is "caching" for offline use.
Do not create two objects as that is redundant. Create just the Core Data objects as part of the parse.
A standard JSON parse would look like this:
Convert to Objects using NSJSONSerializer.
Fetch all unique IDs from the JSON objects using KVO
Fetch all existing objects from Core Data based on uniqueIDs
Insert all objects that do not currently exist
If you are going to update objects then #4 would do both.
Search on Stackoverflow and you will find plenty of examples of how to do this.
I do something similar in one of my apps.
Here is some generic code to fetch an item and update it. If it doesn't exist, it creates it.
func insertOrUpdateManagedObject(id: Int16, name: String) -> YourManagedObject? {
var managedObject: YourManagedObject?
if let context = self.managedObjectContext {
if let fetchResult = fetchManagedObjectWithId(id) {
managedObject = fetchResult
managedObject?.name = name
}
else {
managedPhrase = NSEntityDescription.insertNewObjectForEntityForName("YourManagedObject", inManagedObjectContext: context) as? YourManagedObject
managedObject?.id = id
managedObject?.name = name
}
}
println("Created a managed object \(managedObject)")
return managedObject
}
I am trying to use a sqlite database in one of my projects.
It was working fine; but for a reason, something happened and I couldn't find that bug.
The resultSet object always quit from after the first record.
The array always has only 1 record in it. (Probably it left the while because of the error)
I created a DBManager class, and this DBManager class contains different inner Classes. I have a private global FMDatabase instance (And I initialise it somewhere before using it)
As you see, there are 2 different print error line
When I run, the second print line gives this error:
Error calling sqlite3_step (21: out of memory) rs
Error Domain=FMDatabase Code=7 "out of memory" UserInfo=0x790308d0 {NSLocalizedDescription=out of memory}
And the array which should contain over 300 records, has only 1 record in it.
(Last print line is always 1)
This part is looking simple. (I have totally similar code somewhere else, but it works fine)
private var database : FMDatabase!
class DBManager{
class Element{
class func get()->[DataElement]{
database.open()
println( database.lastError() )
var result = [DataElement]()
var resultSet: FMResultSet! = database!.executeQuery("SELECT * FROM Element WHERE working = 1", withArgumentsInArray: nil)
while resultSet.next( ) {
let data = DataElement(
id : Int(resultSet.intForColumn("id")),
name: resultSet.stringForColumn("name"),
server: resultSet.stringForColumn("server"),
working: resultSet.boolForColumn("working") )
result.append( data )
}
println( database.lastError() )
database.close()
println( result.count )
return result
}
}
}
PS: Only difference between those tables is (as I realize ) the "id" column. This Element table has an "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, but the other one does not have any id column. But both of them worked well for a long time.
The "out of memory" error is a misleading SQLite error that means that a SQLite function was called with a NULL value for the sqlite3* pointer. In FMDB this means that you closed the database but then tried to continue using the same FMDatabase instance (without calling open again).
Now, I don't see you doing that in this code sample, but this code sample is employing some practices that make that sort of error possible. Namely, rather than instantiating the FMDatabase object locally, you are using some database property and you run the risk that some other function may have used it (maybe the init method for DataElement? maybe some other function that you removed for the sake of brevity? maybe some other thread?).
Let's imagine that this function called some other function that opened the database again (which FMDB will silently let you do, basically returning immediately if the database is already open), perform some SQL, and then closed the database, then this routine, when it went to retrieve the second row of information, would of have found the database closed, resulting in the error you describe. The opening and closing of the database object is not recursive. Once the subroutine closed it, it's completely closed.
Now all of this is hypothetical, because without seeing what the rest of your code does, it's impossible for me to confirm. But its a scenario that would fit what code you have shared combined with the the symptoms you've described.
Assuming that this is really the case, there are two possible solutions here:
You could simply open the database once and leave it open until the app is terminated. This is probably the easiest approach and is probably more efficient than what you have here. SQLite is actually pretty robust in terms of committing individual SQL statements (or transactions), so people usually leave the database open.
I might even go step further, and suggest that if you might have different threads interacting with this database, that you instantiate a single FMDatabaseQueue object, and use that throughout the app, again not opening and closing all the time. Note, if you use FMDatabaseQueue, you'll want to be even more judicious about making sure that one function that is in the middle of an inDatabase or inTransaction block doesn't call another function that tries to do another inDatabase or inTransaction block (or else you'll deadlock). But like any shared resource, you want to be sensitive to where a resource is locked and when it's released, and databases are no exception to that rule.
If you're determined to open and close the database in every function like this code sample suggests (again, a practice I wouldn't advise), then do not use a class property to keep track of the database. Sure, have some function that opens the database, but return this FMDatabase object, and each function would have it's own local instance and would close that instance when it's done with it.
But you really want to avoid the unintended consequences of one function's closing the database affecting the behavior of some other function.
i also faced the same issue, as after deleting the records from the table i am closing the connection and after some time i am calling one more sql query, at that point of time i got the same issue
"FMDatabase Code=7 "out of memory" UserInfo=0x790308d0 {NSLocalizedDescription=out of memory}"
just add one line of code before closing the connection ie.,
database = nil
database.close()
Later add the method in your modal class as per below
func openDatabase() -> Bool {
if database == nil {
if !FileManager.default.fileExists(atPath: pathToDatabase) {
let documentsDirectory = (NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0] as NSString) as String
pathToDatabase = documentsDirectory.appending("/\(databaseFileName)")
print(pathToDatabase)
}
database = FMDatabase(path: pathToDatabase)
}
if !database.isOpen {
database.open()
}
return true
}
Before executing any query check this,
if openDatabase() {
//Query
}
It solved my issue.