Based on some limited testing, I see that if I
Execute a Fetch request with result type = NSDictionaryResultType
Do some manipulations on the returned values
Store back the MOC on which Fetch request was executed
the changes in step 2 are not written back to the persistent store because I am changing a dictionary and not a "managed object". Is that a correct understanding?
Most likely you are abusing the dictionary result type. Unlike in conventional database programming, you are not wasting valuable memory resources when fetching the entire objects rather than just one selected attributes, due to an under-the-hood mechanism called "faulting".
Try fetching with managed object result type (default) and you can very easily manipulate your objects and save them back to Core Data. You would not need to do an additional fetch just to get the object you want to change.
Consider dictionaries only in special situations with huge data volumes, difficult relational grouping logic, etc., which make it absolutely necessary.
(That being said, it is unlikely that it is ever absolutely necessary. I have yet to encounter a case where the necessity of dictionaries for fetches was not an indirect result of flawed data model design.)
Yes, kind of, you can't store a dictionary back into the context directly so you can't save any updates that way.
If you get a dictionary object then you need to include in it the associated managed object id (if it isn't aggregated) or do another fetch to get the object(s) to update.
Related
I'm trying to efficiently batch delete a lot of NSManagedObjects (without using an NSBatchDeleteRequest). I have been following the general procedure in this answer (adapted to Swift), by batching an operation which requests objects, deletes, saves and then resets the context. My fetch request sets includesPropertyValues to false.
However, when this runs, at the point where each object is deleted from the context, the fault is fired. Adding logging as follows:
// Fetch one object without property values
let f = NSFetchRequest<NSManagedObject>(entityName: "Entity")
f.includesPropertyValues = false
f.fetchLimit = 1
// Get the result from the fetch. This will be a fault
let firstEntity = try! context.fetch(f).first!
// Delete the object, watch whether the object is a fault before and after
print("pre-delete object is fault: \(firstEntity.isFault)")
context.delete(firstEntity)
print("post-delete object is fault: \(firstEntity.isFault)")
yields the output:
pre-delete object is fault: true
post-delete object is fault: false
This occurs even when there are no overrides of any CoreData methods (willSave(), prepareForDeletion(), validateForUpdate(), etc). I can't figure out what else could be causing these faults to fire.
Update: I've created a simple example in a Swift playground. This has a single entity with a single attribute, and no relationships. The playground deletes the managed object on the main thread, from the viewContext of an NSPersistentContainer, a demonstrates that the object property isFault changes from true to false.
I think an authoritative answer would require a look at the Core Data source code. Since that's not likely to be forthcoming, here are some reasons I can think of that this might be necessary.
For entities that have relationships, it's probably necessary to examine the relationship to handle delete rules and maintain data integrity. For example if the delete rule is "cascade", it's necessary to fire the fault to figure out what related instances should be deleted. If it's "nullify", fire the fault to figure out which related instances need to have their relationship value set to nil.
In addition to the above, entities with relationships need to have validation checks performed on related instances. For example if you delete an object with a relationship that uses the "nullify" delete rule, and the inverse relationship is not optional, you would fail the validation check on the inverse relationship. Checking this likely triggers firing the fault.
Binary attributes can have data automatically stored in external files (the "allows external storage" option). In order to clean up the external file, it's probably necessary to fire the fault, in order to know which file to delete.
I think all of these could probably be optimized away. For example, don't fire faults if the entity has no relationships and has no attributes that use external storage. However, this is looking from the outside without access to source code. There might be other reasons that require firing the fault. That seems likely. Or it could be that nobody has attempted this optimization, for whatever reason. That seems less likely but is possible.
BTW I forked your playground code to get a version that doesn't rely on an external data model file, but instead builds the model in code.
Tom Harrington has explained it best. CoreData's internal implementation apparently requires to fire fault when marking an object to be removed from the persistent store, just like it would if you were accessing a property of the object. As explained in this answer, "An NSManagedObject is always dynamically rendered. Hence, if it is deleted, Core Data faults out the data".
This seems to be the normal behaviour at least for the moment being, not really an issue.
This question is poorly phased but this can be better explained in code.
We have a Core Data Stack with private and main contexts as defined by Marcus Zarra here: http://martiancraft.com/blog/2015/03/core-data-stack/
We call a separate class to do a fetch request (main context) and return an array of NSManagedObjects:
NSArray *ourManagedObjects = [[Client sharedClient].coreDataManager fetchArrayForClass:[OurObject class] sortKey:#"name" ascending:YES];
We then do some processing and store a reference:
self.ourObjects = processedManagedObjects
Our view contains a UITableView and this data is used to populate it and that works just fine.
We change the data on our CMS, pull to refresh on the UITableView to trigger a sync (private context) and then call this same function to retrieve the updated data. However, the fetch request returns the exact same data as before even though when I check the sqlite db directly it contains the new data. To get the new values to display I have to reload the app.
I have discovered that if I don't assign the processedManagedObjects to self, the fetch request does indeed return the correct data, so it looks like holding a reference to the NSManagedObject stops it from getting new data from the main context. However I have no idea why that would be.
To clarify, we're pretty sure there's nothing wrong with our Core Data Stack, even when these managed objects are not being updated, other are being updated just fine, it's only this one where we store a local reference.
It sounds like what's going on is:
Managed objects don't automatically update themselves to reflect the latest data in the persistent store when changes are made via a different managed object context.
As a result, if you keep a reference to the objects, they keep whatever data they already had.
On the other hand if you don't keep a reference but instead re-fetch them, you get the new data because there was no managed object hanging around with its old data.
You have a few options:
You could keep the reference and have your context refresh the managed objects, using either the refresh(_, mergeChanges:) method or refreshAllObjects().
If it makes sense for your app, use an NSFetchedResultsController and use its delegate methods to be notified of changes.
Don't keep the reference.
The first is probably best-- refreshAllObjects() is probably what you want. Other options might be better based on other details of your app.
Try setting the shouldRefreshRefetchedObjects property of the fetch request to true. According to the documentation:
By default when you fetch objects, they maintain their current property values, even if the values in the persistent store have changed. Invoking this method with the parameter true means that when the fetch is executed, the property values of fetched objects are updated with the current values in the persistent store.
I have a Core Data layer with several thousand entities, constantly syncing to a server. The sync process uses fetch requests to check for deleted_at for the purposes of soft-deletion. There is a single context performing save operations in a performBlockAndWait call. The relationship mapping is handled by the RestKit library.
The CoreDataEntity class is a subclass of NSManagedObject, and it is also the superclass for all our different core data object classes. It has some attributes that are inherited by all our entities, such as deleted_at, entity_id, and all the boilerplate fetch and sync methods.
My issue is some fetch requests seem to return inconsistent results after modifications to the objects. For example after deleting an object (setting deleted_at to the current date):
[CoreDataEntity fetchEntitiesWithPredicate:[NSPredicate predicateWithFormat:#"deleted_at==nil"]];
Returns results with deleted_at == [NSDate today]
I have successfully worked around this behavior by additionally looping through the results and removing the entities with deleted_at set, however I cannot fix the converse issue:
[CoreDataEntity fetchEntitiesWithPredicate:[NSPredicate predicateWithFormat:#"deleted_at!=nil"]];
Is returning an empty array in the same conditions, preventing a server sync from succeeding.
I have confirmed deleted_at is set on the object, and the context save was successful. I just don't understand where to reset whatever cache is causing the outdated results?
Thanks for any help!
Edit: Adding a little more information, it appears that once one of these objects becomes corrupted, the only way get it to register is modifying the value again. Could this be some sort of Core Data index not updating when a value is modified?
Update: It appears to be a problem with RestKit https://github.com/RestKit/RestKit/issues/2218
You are apparently using some sintactic sugar extension to Core Data. I suppose that in your case it is a SheepData, right?
fetchEntitiesWithPredicate: there implemented as follows:
+ (NSArray*)fetchEntitiesWithPredicate:(NSPredicate*)aPredicate
{
return [self fetchEntitiesWithPredicate:aPredicate inContext:[SheepDataManager sharedInstance].managedObjectContext];
}
Are you sure that [SheepDataManager sharedInstance].managedObjectContext receives all the changes that you are making to your objects? Is it receives notifications of saves, or is it child context of your save context?
Try to replace your fetch one-liner with this:
[<your saving context> performBlockAndWait:^{
NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:#"CoreDataEntity"];
request.predicate = [NSPredicate predicateWithFormat:#"deleted_at==nil"];
NSArray *results = [<your saving context> executeFetchRequest:request error:NULL];
}];
First, after a save have you looked in the store to make sure your changes are there? Without seeing your entire Core Data stack it is difficult to get a solid understanding what might be going wrong. If you are saving and you see the changes in the store then the question comes into your contexts. How are they built and when. If you are dealing with sibling contexts that could be causing your issue.
More detail is required as to how your core data stack looks.
Yes, the changes are there. As I mentioned in the question, I can loop through my results and remove all those with deleted_at set successfully
That wasn't my question. There is a difference between looking at objects in memory and looking at them in the SQLite file on disk. The questions I have about this behavior are:
Are the changes being persisted to disk before you query for them again
Are you working with multiple contexts and potentially trying to fetch from a stale sibling.
Thus my questions about on disk changes and what your core data stack looks like.
Threading
If you are using one context, are you using more than one thread in your app? If so, are you using that context on more than one thread?
I can see a situation where if you are violating the thread confinement rules you can be corrupting data like this.
Try adding an extra attribute deleted that is a bool with a default of false. Then the attribute is always set and you can look for entities that are either true or false depending on your needs at the moment. If the value is true then you can look at deleted_at to find out when.
Alternatively try setting the deleted_at attribute to some old date (like perhaps 1 Jan 1980), then anything that isn't deleted will have a fixed date that is too old to have been set by the user.
Edit: There is likely some issue with deleted_at having never been touched on some entities that is confusing the system. It is also possible that you have set the fetch request to return results in the dictionary style in which case recent changes will not be reflected in the fetch results.
I'm trying to sync my data from a web service in a simple way. I download my data using AFNetworking, and using a unique identifier on each object, I want to either insert, delete or update that data.
The problem is that with Core Data you have to actually insert objects in the NSObjectManagedContext to instantiate NSManagedObjects. Like this:
MyModel *model = (MyModel *)[NSEntityDescription insertNewObjectForEntityForName:#"MyModel" inManagedObjectContext:moc];
model.value = [jsonDict objectForKey:#"value"];
So when I get the data from the web service, I insert them right away in Core Data. So there's no real syncing going on: I just delete everything beforehand and then insert what's being returned from my web service.
I guess there's a better way of doing this, but I don't know how. Any help?
You are running into the classic insert/update/delete paradigm.
The answer is, it depends. If you get a chunk of json data then you can use KVC to extract the unique ids from that chunk and do a fetch against your context to find out what exists already. From there it is a simple loop over the chunk of data, inserting and updating as appropriate.
If you do not get the data in a nice chunk like that then you will probably need to do a fetch for each record to determine if it is an insert or update. That is far more expensive and should be avoided. Batch fetching before hand is recommended.
Deleting is just about as expensive as fetching/updating since you need to fetch the objects to delete them anyway so you might as well handle updating properly instead.
Update
Yes there is an efficient way of building the dictionary out of the Core Data objects. Once you get your array of existing objects back from Core Data, you can turn it into a dictionary with:
NSArray *array = ...; //Results from Core Data fetch
NSDictionary *objectMap = [NSDictionary dictionaryWithObjects:array forKeys:[array valueForKey:#"identifier"]];
This assumes that you have an attribute called identifier in your Core Data entity. Change the name as appropriate.
With that one line of code you now have all of your existing objects in a NSDictionary that you can then look up against as you walk the JSON.
The easiest thing to do is to restore the Json to a entity that maps properly to it. Once you've mapped it, determine if a object matching the entities ID exists already, if so then fetch the entity and merge changes. If not, create a new entity in Core Data and restore the Json to it.
I'm building a app were I do client side syncing with Evernote. They keep a syncUpdate number on all of their objects and at the server level. So when I start my sync I check if my clients syncUpdate count is less than the servers. If so, I know I am out of sync. If my updateCount is at 400 and the server is at 410, I tell the server to provide me with all objects between updateCount 400 and 410. Then I check if I already have the objects or not and perform my update/create.
Every time a object is modified on the server, that objects updateCount is increments along with the servers.
The server also keeps a time stamp of the last update, which I can check against also.
I have a Core Data stack based on the NSInMemoryStoreType store. And I've noticed that deleting objects doesn't really remove them or make them nil, bur rather simply turns them into faults.
For example, (MyManagedObjectEntityClass as well as the <> identifier are placeholders):
MyManagedObjectEntityClass *o = [NSEntityDescription insertNewObjectForEntityForName:#"<MyManagedObjectEntityClass Entity Name>" inManagedObjectContext:self.localContext];
NSLog(#"\n%#", o);
[self.localContext deleteObject:o];
NSLog(#"\n%#", o);
Will log that the object is still there only that it's data is a fault.
And adding [self.localContext save:nil]; after the delete doesn't change this either.
I was hoping I could at some point test the o variable for nil, in which case I'd reload the object - but it seems I can't.
Just in case, yes, I know I could instead test o for -isFault. But thing is, extrapolate this test to an NSSet and I can't just rely on [[set anyObject] isFault] to conclude that all objects in that set have been removed (Ideally the set's count would be 0, but all objects are still there as faults).
So I'm wondering if it's possible at all or what alternate approach could I take to be able to test that objects have been deleted in a way transparent to the fact that they are managed objects.
This is not actually a Core Data issue. C (and by extension Objective-C) doesn't work like that.
The deleteObject: method takes one argument, a pointer to an object. It can change the object (like setting its isDeleted flag), or it can do other things related to the object (like deleting it from its managed object context). It cannot change the value of the pointer itself. So whatever it does or should do, C says that once it's done, the pointer that you pass in still points to the same location in memory. As a result it's actually impossible for that method to force that pointer to be nil in this language. If you want it to be nil, you have to change that yourself. (As an aside, it would have been possible to implement the method to take a pointer to pointer argument, which could modify your pointer. This would have no effect on other references such as those in arrays, though, so it would be kind of pointless).
This is why the isDeleted method is public, so that if you have a pointer to this object in some other location, you can check whether it has been deleted before attempting to use it.
If that's not convenient enough (and it often isn't), Core Data also provides NSManagedObjectContextObjectsDidChangeNotification and NSManagedObjectContextDidSaveNotification. You can use these anywhere in your app to get notified of changes to the context and respond in whatever way is appropriate (updating an array, for example). These notifications both try to help you out by providing lists of inserted, updated, and deleted objects. Use those when possible to check whether you actually need to update your references.