I'm dong data processing in a child moc in a background queue. I need to query the database by ID so that I can differentiate updating-existing-object from creating-new-object. I found most of the time(the total processing time is about 2s for 50 items) is consumed by executeFetchRequest:error:. The NSPredicate is of the simplest form — only to match a single ID attribute(ID attribute is already indexed), and the NSFetchRequest should return one or none(ID is unique). Is there any way to optimize this kind of NSFetchRequest?
Here is my current code:
+ (User *)userWithID:(NSNumber *)ID inManagedObjectContext:(NSManagedObjectContext *)context {
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"User"];
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"ID == %#", ID];
[fetchRequest setPredicate:predicate];
[fetchRequest setFetchBatchSize:1];
NSError *error = nil;
NSArray *users = [context executeFetchRequest:fetchRequest error:&error];
if (error) {
abort();
}
if ([users count] == 1) {
return [users objectAtIndex:0];
} else if ([users count] > 1) {
// Sanity check.
…
} else {
return nil;
}
}
As #ChrisH pointed out in comments under the question, doing a fetch for every ID is no good. So I changed my processing flow to this:
Enumerate data the first time to extract IDs.
Do a single fetch to fetch all existing users matching IDs and put them in a dictionary keyed by ID(named as existingUsers).
Enumerate data the second time to do the real processing: in each iteration, either update one existing user found in existingUsers or create a new user, add it into existingUsers if it is new.
The code is almost doubled, but so is the performance. Really good tradeoff!
To expand on my comment to the original question, it's not efficient to repeatedly perform fetch requests with Core Data when importing data.
The simplest approach, as #an0 indicated, is to perform one fetch of all the existing objects you will be checking against, and then constructing an NSDictionary containing the objects with the attribute you will be checking as keys. So sticking with the original User and userID example:
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"User"];
NSError *error = nil;
NSArray *users = [context executeFetchRequest:fetchRequest error:&error];
if (error) {
//handle appropriately
}
NSMutableDictionary *userToIdMap = [NSMutableDictionary dictionary];
for (User *user in users){
[userToIdMap setObject:user forKey:user.ID];
}
Now in your method that processes new data you can check the userToIdMap dictionary instead of making fetch requests.
A more sophisticated approach, suited to larger data sets, is outlined in the Core Data Programming Guide's Efficently Importing Data. Take a look at the section called 'Implementing Find-Or-Create Efficiently'. The approach suggested by Apple here misses out some code regarding how to walk the arrays you create, and my solution to that problem is in this SO question: Basic array comparison algorithm
Related
I have a problem with NSBatchDeleteRequest seems that is not possible to delete relationship references.
I have two entities:
News
Categories
where a category can have multiple news.
Now, when I try to delete all the objects in the core data using NSBatchDeleteRequest with the following code, then looking into the sqlite file seems that all categories are deleted, all news are deleted, but the relationship between categories and news persists, and this cause faults.
Here the delete function:
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:entityName];
NSBatchDeleteRequest *delete = [[NSBatchDeleteRequest alloc] initWithFetchRequest:fetchRequest];
[delete setResultType:NSBatchDeleteResultTypeCount];
NSError *error;
NSBatchDeleteResult *results = [deleteContext executeRequest:delete error:&error];
Any idea on how to fix this?
You can probably do [manageObjectContext reset];
Set shouldDeleteInaccessibleFaults: to YES and inaccessible/unfulfillable faults will be deleted. This solves the immediate problem.
The WWDC 2015 session What's New in Core Data talks about this a little bit. Both NSBatchDeleteRequest and NSBatchUpdateRequest modify the persistent store without the participation of the NSManagedObjectContext - which will result in the context's view of the data being inconsistent with the store.
The in-memory copy of the deleted object needs to be updated in the NSManagedObjectContext - have the batch delete request return the object IDs of the deleted objects and tell the NSManagedObjectContext to refresh those IDs.
This would look something like this:
[managedObjectContext performBlock:^{
NSBatchDeleteRequest batchDeleteRequest = [NSBatchDeleteRequest alloc] initWithFetchRequest:fetchRequest];
NSBatchDeleteResult result = nil;
result = [managedObjectContext executeRequest:batchDeleteRequest error:&error];
if ([[result result] count] > 0){
[managedObjectContext performBlock:^{
NSArray<NSManagedObjectID *> *objectIDs = (NSArray<NSManagedObjectID *>)[result result];
[objectIDs enumerateObjectsUsingBlock:^(NSManagedObjectID *objID, NSUInteger idx, BOOL *stop) {
NSError *error = nil;
NSManagedObject *obj = [managedObjectContext existingObjectWithID:objID error:&error];
if (![obj isFault] ) {
[managedObjectContext refreshObject:obj mergeChanges:YES];
}
}];
}];
}
}];
When the batch delete runs, relationships will be deleted or nullified, but a cascading set of delete rules may not be executed, and validation rules will not be executed - it is up to your application to ensure data integrity when using any of the batch change requests.
Your data model may require you to issue multiple delete requests to prevent related objects from being orphaned but still findable. For example, you may need a second batch delete to locate previously related entities that now have empty relationships. The predicate for such a request may look like:
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"toMany.#count == 0"];
Or may use a subquery, etc.
I think the best solution might be to first delete the categories in the object graph, thus nullifying all relationships.
After that you could proceed with the NSBatchDeleteRequest for the news items.
i'm inserting new objects into the database by core data. Is there any way to check if there is any duplicate in the database before i insert the values in?
AccountDetails * newEntry = [NSEntityDescription insertNewObjectForEntityForName:#"AccountDetails" inManagedObjectContext:self.managedObjectContext];
newEntry.acc_date=date;
newEntry.bank_id=bank_id1;
NSError *error;
if (![self.managedObjectContext save:&error]) {
NSLog(#"Whoops, couldn't save: %#", [error localizedDescription]);
}
[self.view endEditing:YES];
everytime i run the app , it reinsert the values again. i want to check if there is any new category in it if there isnt then i will add that new one in only.
thanks in advance..
You can fetch or you can count. Counting is much faster than fetching. Depends on what you are trying to do.
If you just want to insert new and skip duplicates then use -[NSManagedObjectContext countForFetchRequest: error:] to determine if the object exists.
You can pre-build the predicate and just replace the unique value on each loop so that even the cost of the predicate is low. This is fairly performant but not the best solution because it hits the disk on each loop.
Another option would be to change the fetch to have:
Just the unique value
A NSDictionary result
Then grab all of the unique values from your insertable array into an array of strings (for example) then do a single fetch with:
[fetchRequest setPredicate:[NSPredicate predicateWithFormat:#"myUnique in %#", uniqueIDArray]];
Then you have an array of uniques that ARE in the store already. From there as you loop over your objects you check against that array, if the unique in there you skip, otherwise you insert. That will yield the best performance for a straight insert or skip requirement.
You need to fetch from the db and check, your code will be doing something like this helper method I use frequently in my code, if the results.count is > 1, then DUPLICATE found :
- (NSManagedObject*) findOrCreateObjectByValue:(id)value
propertyName:(NSString*)propertyName
entityName:(NSString*)entityName
additionalInfo:(NSDictionary*)additionalInfo
context:(NSManagedObjectContext*)context
error:(NSError* __autoreleasing*)error
{
NSManagedObject* res = nil;
NSFetchRequest* r = [NSFetchRequest fetchRequestWithEntityName:entityName];
[r setPredicate:[NSPredicate predicateWithFormat:#"%K == %#",propertyName,value]];
NSArray* matched = [context executeFetchRequest:r
error:error];
if (matched) {
if ([matched count] < 2) {
res = [matched lastObject];
if (!res) { //No existing objects found, create one
res = [NSEntityDescription insertNewObjectForEntityForName:entityName
inManagedObjectContext:context];
[res setValue:value
forKey:propertyName];
}
} else {
if (error) {
*error = [NSError errorWithDomain:#"some_domain"
code:9999
userInfo:#{#"description" : #"duplicates found"}];
}
}
}
return res;
}
I have a POS type app that uses Core Data to store daily sales transactions using table views. I am attempting to retrieve and update certain Core Date Properties, like daily sales counts, WITHOUT using table views. Table views use row at index path to point to the correct object (row). I am using the Fetched Results controller with a predicate to retrieve the fetched object (row) Question: How do I obtain the index of the fetched row so that I can retrieve and then update the correct property values? All books and examples use table views to change properties.
Entity Product
Product *product;
______________________________
[self setupFetchedResultsController]; (This returns one object)
product = [NSFetchedResultsController objectAtIndexPath:[NSIndexPath indexPathForRow:0 inSection:0]]; (objectAtIndexPath - Errors of course)
I think you shouldn't use NSFetchedResultsController in this case. If you don't want to use it in either a UITableView or a UICollectionView, you're probably better of without it. You're probably better of using a NSFetchRequest instead, it's pretty easy to set up:
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"Entity"];
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"someValue=1"];
fetchRequest.predicate = predicate;
NSError *error = nil;
NSArray *array = [self.managedObjectContext executeFetchRequest:fetchRequest error:&error];
Now you have a NSArray with all the results, which you could use without having to deal with index paths.
If you're still using a NSFetchedResultController for a table (I'm not sure if you do), those rows will still be updated whenever you make a change.
Update: To update one of the objects returned by the fetch, could be done like this:
Entity *entity = [array firstObject];
[entity setSomeProperty:#"CoreDataIsAwesome"];
NSError *error = nil;
if ([self.managedObjectContext save:&error]) {
NSLog(#"Entity updated!");
} else {
NSLog(#"Something went wrong: %#", error);
}
You can use the method indexPathOfObject: on your fetched results controller to return the index path of the given object to then do your updates.
I'm importing data into a Core Data store using RestKit and need to check for duplicates. If the item is already in the store, I'd like to update it with the latest attributes. If it's a new item, I'd like to create it.
The import was slow so I used Instruments and saw that the longest part of importing was checking to see if the item already exists (with a fetch request)
So I'd like to know if checking to see if the item is already in the store, is it faster to:
use countForFetchRequest to see if the item already exists, then executeFetchRequest to return the item to update or
just executeFetchRequest to get the item to update
or is there a better way to do this?
I thought countForFetchRequest would be faster since the entire NSManagedObject isn't returned and only execute the fetch request if I know there's going to be a NSManagedObject.
Thanks
- (Product *)productWithId:(int)productID {
NSManagedObjectContext *context = [Model sharedInstance].managedObjectContext;
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"product_id == %d", productID];
NSFetchRequest *request = [[NSFetchRequest alloc] init];
request.entity = [NSEntityDescription entityForName:#"Product" inManagedObjectContext:context];
request.predicate = predicate;
request.fetchLimit = 1;
NSError *error = nil;
NSUInteger count = [context countForFetchRequest:request error:&error];
if (!error && count == 1) {
NSArray *results = [context executeFetchRequest:request error:&error];
if (!error && [results count]) {
return [results objectAtIndex:0];
}
return nil;
}
return nil;
}
As far I know, the best way to find and/or import objects within Core Data is described in Implementing Find-or-Create Efficiently.
The documentation describes a find or create pattern that it's based on sorting data: the data you download from the service and the data you grab form the store.
I really suggest you to read the link I provided. You will see a speed up on your performances.
Obviously you should do the work in background, preventing the main thread to freeze, using thread confinement or new iOS Core Data queue API.
Hope that helps.
I have a list of Places from a rails app that I'm trying to import in an iOS5 app. Each Place has a parent which is a Place itself.
I'm trying to import that JSON data with Core Data using a dictionary
- (void)initWithDictionary:(NSDictionary *)dictionary {
self.placeId = [dictionary valueForKey:#"id"];
id parent = [dictionary objectForKey:#"parent"];
if (parent && parent != [NSNull null]) {
NSDictionary *parentDictionary = parent;
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"placeId = %#", [parentDictionary objectForKey:#"id"]];
NSArray *matching = fetchedWithPredicate(#"Place", self.managedObjectContext, predicate, nil);
if ([matching count] > 0) {
self.parent = [matching objectAtIndex:0];
} else {
self.parent = [NSEntityDescription insertNewObjectForEntityForName:#"Place" inManagedObjectContext:self.managedObjectContext];
[self.parent initWithDictionary:parentDictionary];
}
}
}
fetchedWithPredicate is a method defined as such
NSArray* fetchedWithPredicate(NSString *entityName, NSManagedObjectContext *context, NSPredicate *predicate, NSError **error) {
NSFetchRequest *request = [[NSFetchRequest alloc] init];
[request setIncludesPendingChanges:YES];
NSEntityDescription *entity = [NSEntityDescription entityForName:entityName inManagedObjectContext:context];
[request setEntity:entity];
[request setPredicate:predicate];
NSArray *result = [context executeFetchRequest:request error:error];
return result;
}
I also have a validation method in Place.m to make sure I don't create to place with the same placeId (placeId is the id on the server side).
- (BOOL)validatePlaceId:(id *)value error:(NSError **)error {
if (*value == nil)
return YES;
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"placeId = %# AND (NOT self IN %#)", *value, [NSArray arrayWithObject:self]];
NSArray *matching = fetchedWithPredicate(#"Place", self.managedObjectContext, predicate, error);
if ([matching count] > 0) {
return NO;
}
else {
return YES;
}
}
To import the data, I fetch all places from the server, returned in JSON format.
Each Place has its own information, plus a child node with informations about the parent, which means that each parent of multiple children will appear multiple times. It looks like
{ "id": 73,
"name": "Some place",
"parent": { "id": 2,
"name": "Parent's name"}
}
I thought the above code which does kind of a "find or create", with a fetch including unsaved changes, would be alright.
But it still attempt to create multiple entries for some places (and fails to since there's a validation in place). Looking deeper, it indeed insert different core data objects for the same placeId (different pointers), but I don't know why.
Thanks
It sounds like you already have a unique index on id (which is good obviously). I think it is that you are not saving the newly inserted creations to core data prior to expecting it to be returned via fetch. The simple (if perhaps not too performant depending on having lots of rows) would be to add a saveContext call right after each object is inserted/inited.
Another way would be to do it in two passes, first entirely in memory where you create a separate dictionary where the key is the id, and the object is the value. That way you'd be able to ensure each id was only in there once. After they're all in that dictionary, you can then easily (or easier, perhaps) add them all to Core Data.
So, after a bit more investigation, it's due to the fact that I was sorting my data by name...
So if a place A had 5 children, and 3 of them had a name that was before A's name, the code would:
create those 3 children with a parent that don't have any parent itself (because my json doesn't return infos about the parent's parent)
create A
create the 2 other children with A as parent (probably because of the way it's sorted, but that doesn't change the conclusion), so a parent that does have a parent
Now we have 2 objects A, one with a parent, and one without a parent, which Core Data consider has 2 objects.
The easy way out: my tree is a nested set, so I just have to sort places by the left value, and this way I'll always create parents before children.
The "sort by name" wasn't part of my description, so I'll leave scc's answer as accepted :)