Let´s assume I have created a core data entity with:
NSDate *firstDate
NSDate *secondDate
NSData *image
When executing a fetch with executeFetchRequest, is the request then fetching the whole image with it or just the pointer to the image?
In other terms: Does it make a big difference performance and memory wise if I fetch all the entities or just an NSDictionary with firstDate/secondDate when I do not need the image directly?
Thanks for the help.
You can alter your NSFetchRequest behavior. Read Fetching Managed Objects document from Apple.
Related
I'm currently using coredata for my project. But when the api returns 54000 objects that the app need to update, the user has to wait almost 2 hours.
It's the major problem for the current project and I am thinking to use sqlite and not using coredata anymore to update thousands of objects.
Is it a right decision to use Sqlite or is there any suggestion for CoreData? I can't decide. Any help will be great. Thank you.
Here is what I am doing:
NSManagedObjectContext *privateObjectContext = [AppDelegate appDelegate].privateManagedObjectContext;
[privateObjectContext performBlock:^{
int i = 1;
for (NSDictionary *item in itemlist) {
i++;
[fetchRequest setPredicate:[NSPredicate predicateWithFormat:
#"itemID == %#",[item objectForKey:#"item_id"]
]];
NSError *error;
NSMutableArray *inventories = [[NSMutableArray alloc]initWithArray:
[privateObjectContext executeFetchRequest:fetchRequest
error:&error]];
ItemManagedObject *itemMO;
if(inventories.count){
itemMO = inventories.firstObject;
}else{
itemMO = [NSEntityDescription insertNewObjectForEntityForName:#"ItemObject"
inManagedObjectContext:privateObjectContext];
}
[itemMO prepareWithDictionary:item];
}
NSError *error;
if (![privateObjectContext save:&error]) {
completionHandler(NO);
}
}
Core Data provides NSBatchUpdateRequest which allows you to make updates directly on the persistent store without involving instantiating and processing managed objects in memory.
You should run this code using the core data performance instrument as well. If itemList contains 54,000 objects then you are performing 54,000 fetches to the persistent store to check a single ID each time. It would be far faster to fetch all of the IDs up front and then check the results in memory than to perform repeated fetch requests - that code will be almost as slow in raw SQL as it is in Core Data.
This code also looks wrong:
ItemManagedObject *itemMO;
if(itemMO.count){
It's never going to pass that if test, unless you've missed a line somewhere.
2 hours is very long. That's weird.
Yet you can massage your code by having core data do less work. Much less work.
Perform a single fetch request instead of 54K fetch requests
Don't call a managed object property setter when a property value does not change, so that no object is unnecessarily flagged as dirty, and Core Data does not have to perform a costly but useless update of the object when the "save" method is invoked.
This will dramatically reduce the amount of work performed by Core Data, and the performance of your application.
The second point is easy, but very verbose: compare each individual property values with dictionary values before calling setters.
The first point requires an algorithm change:
Perform a single fetch request, sorted by id (with [NSFetchRequest setSortDescriptors:])
Sort dictionaries by id (with [NSArray sortedArray...])
Synchronize the two sorted lists (it is paramount that both lists are sorted):
NSEnumerator *itemMOEnum = [itemMOs objectEnumerator];
NSEnumerator *dicEnum = [dictionaries objectEnumerator];
ItemManagedObject *itemMO = [itemMOEnum nextObject];
NSDictionary *itemDic = [dicEnum nextObject];
while (itemDic) {
NSComparisonResult comparison = itemMO ? [itemDic[#"item_id"] compare:itemMO.itemID] : NSOrderedAscending;
switch (comparison) {
case NSOrderedSame:
// id present in both lists: update
[itemMO prepareWithDictionary:itemDic];
itemMO = [itemMOEnum nextObject];
itemDic = [dicEnum nextObject];
break;
case NSOrderedAscending: {
// id present only in dictionaries: create
itemMO = [NSEntityDescription insertNewObjectForEntityForName:#"ItemObject"
inManagedObjectContext:privateObjectContext];
[itemMO prepareWithDictionary:itemDic];
itemDic = [dicEnum nextObject];
} break;
case NSOrderedDescending:
// id present only in managed object: delete or do nothing
itemMO = [itemMOEnum nextObject];
break;
}
}
while (itemMO) {
// id present only in managed object: delete or do nothing
itemMO = [itemMOEnum nextObject];
}
And save.
Finally, maybe SQLite will be faster (see https://github.com/groue/GRDB.swift/wiki/Performance for an attempt at comparing the performance of Core Data with SQLite libraries).
But SQLite won't turn a slow algorithm into a fast one.
I've never redone a core data project in sqlite or visa versa. So I cannot tell you whether there is a performance difference or not/
However the 54k = 2 hours thing sounds very strange. You talk about an API which makes me suspect a server is involved, your question is about databases. Certainly 2 hours sounds way too long and makes me wonder whether you have issues with the core design of your database. For example, lack of indexes. Depending on your queries and database, a single update could be triggering all sorts of heavy duty processing.
Another though is why are you processing this column of data on a device. It's a lot to handle and I wonder if there are ways to reduce the volume down, selectively do updates or perhaps even better - move it to a server.
I think you need to rethink your question. Provide more context about the database, exactly what you are doing with it and why.
CoreData is not a database manager but a object graph and persistent manager. CoreData can store its objects in a sqlite database but also in XML files or binary file (the developer chooses the option best suited to its needs).
The main difference between CoreData and a database manager is that to access an object with CoreData, CoreData need to instantiate the objective-C/Swift corresponding object.
Sqlite can access part of data without having to extract the full record containing the data.
And then, CoreData need to maintain the relational graph between objects (the relationships between 2 CoreData classes, and in general, in both ways).
So, when updating 54k objects, you ask CoreData to instantiate 54k objects (in memory) and to eventually update their relationships.
That is very heavy work for CoreData on mobile.
Perhaps your CoreData model is not correctly optimized.
Perhaps you should save the CoreData context regularly and flush CoreData scratchpad (the part of memory containing actually read or updated objects).
But in my experience, CoreData is not suited to heavy data work.
Re-implementing your needs with sqlite can be quite some work if you want to be able to re-instantiate your classe objects from sqlite records and manage quite automatic relationship, but it is doable. I did it on some projects. This add the benefit to have a model object that is more shareable with other platform as Android for instance, as sqlite is available on many platforms.
One more thing: sqlite is more suited to be used from multiple threads. CoreData is more touchy about this, and need one context by thread, and eventually, some contexts synchronization.
I am starting to develop core data database and i do have a few questions that I can,t understand. Can any one please explain in brief and please keep it simple.
1)
NSManagedObject *employee=[NSEntityDescription insertNewObjectForEntityForName:#"Employee" inManagedObjectContext:_managedObjectContext];
[employee setValue:self.empnametextfield.text forKey:#"empname"];
[employee setValue:self.empidtextfield.text forKey:#"empid"];
while saving the data into database into the database i use this code. But why I am creating instances of NSManagedObject & NsentityDescription?
2)
_fetchrequest=[[NSFetchRequest alloc]init];
NSEntityDescription *entity=[NSEntityDescription entityForName:#"Employee" inManagedObjectContext:_managedObjectContext];
[_fetchrequest setEntity:entity];
NSError *error;
_fetchedobjects=[_managedObjectContext executeFetchRequest:_fetchrequest error:&error];
And when i fetch data i use this coding. So my question is why do i use the instance of NSEntity description here?
3)What is the difference between the purposes for which we use "NSEntityDescription" while in saving & fetching data?
Please answer the above 3 questions of mine as I am quite stuck in it?
Thanks in advance.
When you build your core data stack you load a model which describes the data structure objects and relationships. This is built in terms of entity descriptions. They hold the format for the data, the names, types, multiplicities and rules associated. Without this you have no structure, you might as well just have a generic NSSet.
So, when you're doing operations on the data structure, creating new entity instances or querying, you need to get the description of the entity you're working with so the system knows the rules to work with.
I'm trying to sync my data from a web service in a simple way. I download my data using AFNetworking, and using a unique identifier on each object, I want to either insert, delete or update that data.
The problem is that with Core Data you have to actually insert objects in the NSObjectManagedContext to instantiate NSManagedObjects. Like this:
MyModel *model = (MyModel *)[NSEntityDescription insertNewObjectForEntityForName:#"MyModel" inManagedObjectContext:moc];
model.value = [jsonDict objectForKey:#"value"];
So when I get the data from the web service, I insert them right away in Core Data. So there's no real syncing going on: I just delete everything beforehand and then insert what's being returned from my web service.
I guess there's a better way of doing this, but I don't know how. Any help?
You are running into the classic insert/update/delete paradigm.
The answer is, it depends. If you get a chunk of json data then you can use KVC to extract the unique ids from that chunk and do a fetch against your context to find out what exists already. From there it is a simple loop over the chunk of data, inserting and updating as appropriate.
If you do not get the data in a nice chunk like that then you will probably need to do a fetch for each record to determine if it is an insert or update. That is far more expensive and should be avoided. Batch fetching before hand is recommended.
Deleting is just about as expensive as fetching/updating since you need to fetch the objects to delete them anyway so you might as well handle updating properly instead.
Update
Yes there is an efficient way of building the dictionary out of the Core Data objects. Once you get your array of existing objects back from Core Data, you can turn it into a dictionary with:
NSArray *array = ...; //Results from Core Data fetch
NSDictionary *objectMap = [NSDictionary dictionaryWithObjects:array forKeys:[array valueForKey:#"identifier"]];
This assumes that you have an attribute called identifier in your Core Data entity. Change the name as appropriate.
With that one line of code you now have all of your existing objects in a NSDictionary that you can then look up against as you walk the JSON.
The easiest thing to do is to restore the Json to a entity that maps properly to it. Once you've mapped it, determine if a object matching the entities ID exists already, if so then fetch the entity and merge changes. If not, create a new entity in Core Data and restore the Json to it.
I'm building a app were I do client side syncing with Evernote. They keep a syncUpdate number on all of their objects and at the server level. So when I start my sync I check if my clients syncUpdate count is less than the servers. If so, I know I am out of sync. If my updateCount is at 400 and the server is at 410, I tell the server to provide me with all objects between updateCount 400 and 410. Then I check if I already have the objects or not and perform my update/create.
Every time a object is modified on the server, that objects updateCount is increments along with the servers.
The server also keeps a time stamp of the last update, which I can check against also.
Purpose: I have to create entities from files.
So entities represent my data model in CoreData and files have all information for this entities.
All files I get from Internet. For this I use AFNetworking framework.
How I get files (algorithm):
Request plist file. Plist file has values to other urls that I have to download.
When plist was downloaded to my Documents directory on device. I parse it.
When I parse plist I grab url from each item from NSDictionary that represent this plist.
Then I request zip files from this urls.
After zip files were downloaded I unzip them and go to the next step.
Parse unzipped files and create data model.
It is my problem. I have version of file that stored locally and that stored on the server and when version on the server changed I need to reload my data model with actual data. The bad way it is load all data from server again next delete all entities in storage and make new entities from new data. But it is not professional way at first and second it is an additional burden on the traffic, because if I have just one entity that I need to reload why I have to reload other entities that are in the actual state. So maybe someone knows best practice with this question. Of course I can create my solution and it will work, but I want to see how people solve this problem and figure out with the differences in my solution and in the other solutions also.
This is trivial. You simply keep an attribute with the time stamp of the last update and just request the changed and new entities from your server. Then you insert, update or delete as appropriate.
It sounds like you are talking about find-or-create. Depending on the size of the data set and your performance requirements you can do this a couple of ways:
The first way is to fetch your existing Core Data objects and store them in an a dictionary with a unique attribute of the entity serving as the key. Then when you download the new data you can take this key for each parsed object and check your dictionary to find (and optionally update) any existing object, or create a new one:
NSArray *fetchedObjects = //Fetch the existing objects;
NSMutableDictionary *existingObjects = [NSMutableDictionary dictionary];
for (MyClass *object in fetchedObjects)
[existingObjects setObject:object forKey:object.uniqueKey];
//Now iterate through the new data (using JSON as example here)
NSDictionary *downloadedItems = //Download data and serialise into JSON
for (NSDictionary *item in downloadedItems){
NSString *uniqueValue = [item valueForKey:#"uniqueKey"];
MyClass *object = [existingObjects objectForKey:uniqueValue];
if (object==nil){
object = //Insert new MyClass entity into MOC
object.uniqueKey = uniqueValue;
//Set any other values you only need to set once
}
//Set any values you may need to update every time
//This would be where to check a modified date attribute
}
The second way is more sophisticated, and involves less memory overhead. It's described in the Efficiently Importing Data section of the Core Data Programming Guide.
The guide gives a good start but doesn't offer a complete solution; I attempted my own in an answer here: Basic array comparison algorithm
I am trying to save a pic taken from the iphone camera to core data.
I have the UIImage taken from the camera. AFter that, I don't know what I'm supposed to do.
Each pic is associated with a question entity. Now a question entity does not have an image field. It has a relationship to an Image entity. This is so that each question can have many UIImages associated with it.
So I have created a question entity. Now how do I save the UIImage into core data into an Image entity, and how do I join it with a question entity? Do I need to create an Image entity? I have created an ImageToDataTransformer, but I do not know who calls that--core data or me? In my xcdatamodel, I have an Image entity. I put in ImageToDataTransformer there.
I am aware that I should be storing urls to the UIImage, but I want to understand how to store images. It seems kind of confusing to me.
Thanks
Question *question = [NSEntityDescription insertNewObjectForEntityForName:#"Question" inManagedObjectContext:managedObjectContext];
question.date = [NSDate date];
ImageToDataTransformer *transformer = [[ImageToDataTransformer alloc]init];
NSData *imageData = [transformer transformedValue:mm];//mm is an id, but it is the UIImage saved from the camera.
question.image = imageData;
I would suggest you make a QuestionImage entity (or something named similar) and give it a single string attribute named path. This path will store a location on disk where you've previously written out your image. Placing the image in the documents folder will ensure it does not get cleared with any cache dumps. In the implementation of your QuestionImage entity (or whatever you name it) you can implement prepareForDeletion and remove the image from the disk to make sure you don't have any un-referenced images hanging around.
Storing binary data in Core Data can be troublesome, and should be avoided where possible.