I use a collection view to display images stored in CoreData, only a few (around 15) but when I scroll down it lags, It seams to be because I am not fetching my data asyncrhronously, is it possible with only 15 UIImages?
So here is my problem I cannot find a descent tutorial on asynchronous fetching in swift anywhere ,3 days I am looking. Sorry if I didn't search well.
This is what arrived to do with some bits from tutorials
let entity = NSEntityDescription.entityForName("Dreams", inManagedObjectContext: contxt)
let fetchRequest = NSFetchRequest()
fetchRequest.entity = entity
let asyncFetch = NSAsynchronousFetchRequest(fetchRequest: fetchRequest) {
(result:NSAsynchronousFetchResult!) -> Void in
if result.finalResult!.count > 0 {
self.dreams = result.finalResult! as [Dream]
}
NSLog("Done.")
}
Dreams is an array of type [Dream] but when I display the collection view nothing appears
This is how I did it previously
let fetchRequest = NSFetchRequest(entityName: "Dreams")
if let fetchResults = contxt.executeFetchRequest(fetchRequest, error: nil) as? [Dream] {
dreams = fetchResults
}
It worked very well,
Thank you very much for your help
Unfortunately, I don't have time to offer specific code in this instance. I did want to present a basic strategy, however. Fetching on the main thread is exceedingly unlikely to be your problem here, but fetching everything at once definitely could be, particularly with large binary data involved. This should be sped up significantly if:
You start using an NSFetchedResultsController with a good batch size rather than a raw fetch request. NSFetchedResultsController is a little finicky about updates with collection views, in ways which are beyond the specific scope of this question, but they work very well once set up correctly.
You store the images as files instead of in Core Data itself, and store the path to the file instead. Then that image data won't get loaded into memory every time you fault in the object, and you can simply load and display the images as the cells come on screen using that file URL.
People who insist on storing binary data in Core Data usually find that storing the data as the only attribute in a separate "Image" entity with a relationship to the entity you are displaying helps. The image object (along with its binary image data) can then be kept a fault, and thus not in memory, even when the object you fetch for the collection view is in memory. I, however, prefer not to store them in Core Data, and simply handle the resulting atomicity gotchas and the possibility the file Core Data points to got deleted somehow.
Related
TL;DR: Is there a way to programmatically read/recall (NOT write!) an instance of a Core Data entity using the p-numbered "serial number" that's tacked on to the instance's x-coredata:// identifier? Is this a good/bad idea?
I'm using a method similar to the following to retrieve the instances of an Entity called from a Core Data data store:
var managedContext: NSManagedObjectContext!
let fetchRequest : NSFetchRequest<TrackInfo> = TrackInfo.fetchRequest()
fetchResults = try! managedContext.fetch(fetchRequest)
for (i, _) in Global.Vars.numberOfTrackButtons! {
let workingTrackInfo = fetchResults.randomElement()!
print("current track is: \(workingTrackInfo)")
The list of tracks comes back in fetchResults as an array, and I can select one of them at random (fetchResults.randomElement()). From there, I can examine the details of that one item by coercing it to a string and displaying it in the console (the print statement). I don't list the code below, but using workingTrackInfo I am able to see that instance, read its properties into other variables, etc.
In the console, iOS/Xcode lists the selected item as follows:
current track is: <MyProjectName.TrackInfo: 0x60000374c2d0> (entity:
TrackInfo; id: 0xa7dc809ab862d89d
<x-coredata://2B5DDCDB-0F2C-4CDF-A7B9-D4C43785FDE7/TrackInfo/p22>;
data: <fault>)
The line beginning with x-coredata: got my attention. It's formatted like a URL, consisting of what I assume is a UUID for the specific Core Data store associated with the current build of the app (i.e. not a stable address that you could hardcode; you'd need to programmatically look up the Core Data store, similar to the functions we use for programmatically locating the Documents Folder, App Bundle, etc.) The third item is the name of the Entity in my Core Data model -- easy enough.
But that last number is what I'm curious about. From examining the SQLite database associated with this data store, it appears to be a sort of "instance serial number" associated with the Z_PK field in the data model.
I AM NOT interested in trying to circumvent Core Data's normal mechanisms to modify the contents of a managed object. Apple is very clear about that being a bad idea.
What I AM interested in is whether it's possible to address a particular Core Data instance using this "serial number".**
In my application, where I'm randomly selecting one track out of what might be hundreds or even thousands of tracks, I'd be interested in, among other things, the ability to select a single track on the basis of that p-number serial, where I simply ask for an individual instance by generating a random p-number, tack it on to a x-coredata:// statement formatted like the one listed above, and loading the result (on a read-only basis!) into a variable for further use elsewhere in the app.
For testing purposes, I've tried simply hardcoding x-coredata://2B5DDCDB-0F2C-4CDF-A7B9-D4C43785FDE7/TrackInfo/p22 as a URL, but XCode doesn't seem to like it. Is there some other data Type (e.g. an NSManagedObject?) that allows you to set an x-coredata:// "URL" as its contents?
QUESTIONS: Has anyone done anything like this; are there any memory/threading considerations why grabbing instance names in this manner is a bad idea (I'm an iOS/Core Data noob, so I don't know what I don't know; please humor me!); what would the syntax/method for these types of statements be?
Thanks!
You are quite close.
x-coredata://2B5DDCDB-0F2C-4CDF-A7B9-D4C43785FDE7/TrackInfo/p22
is the uriRepresentation() of the NSManagedObjectID of the record.
You get this URL from an NSManagedObject with
let workingTrackInfo = fetchResults.randomElement()!
let objectIDURL = workingTrackInfo.objectID.uriRepresentation()
With this URL you can get the managed Object ID from the NSPersistentStoreCoordinator and the coordinator from the managed object context.
Then call object(with: on the context to get the object.
let persistentStoreCoordinator = managedContext.persistentStoreCoordinator!
if let objectID = persistentStoreCoordinator.managedObjectID(forURIRepresentation: objectIDURL) {
let object = managedContext.object(with: objectID) as! TrackInfo
print(object)
}
I have a UITableView that gets data from an NSFetchedResultsController. To this end, i basically copied Apples example implementation from https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/CoreData/nsfetchedresultscontroller.html.
Now at WWDC16 Apple announced the UITableViewDataSourcePrefetching protocol, which provides callbacks that let you prefetch data so it's already loaded when it's needed to be displayed by the tableview. I'm looking for an example on how to integrate this with the NSFetchedResultsController, because i can't figure out if i'm doing this correctly.
Should i simply create a dictionary, as an in-memory cache, to hold prefetched data and be used in cellForRowAtIndexPath instead of querying fetchedResultsController.object(at: indexPath) directly?
As i understand it, CoreData already caches fetched data automatically, so maybe i just need to call fetchedResultsController.object(at: indexPath) in the prefetch callbacks, to ensure the data gets cached?
Or shouldn't i prefetch data at all when using a fetched results controller, because i would work around the magical integration provided by Apple?
Or something else entirely?
*edit*: I found a slide from the Core Data talk at WWDC16 that supposedly explains this, but i understand it at all.
An async fetch request? I thought they don't work with NSFetchedResultsController. I guess that's why it's performed on the managedObjectContext directly?
The async fetch request is created from the results of calling .performFetch() on the NSFetchedResultsController. Nothings showing up until i call that. But since all the results are there after calling it, i don't get why i would need to prefetch them again.
It will improve performance if your model has faults. Whether your models have faults or not depends on iOS optimization, and how you create your schema and relationships.
So to be safe your could add the prefetch code. I have the code for Swift 4.
func tableView(_ tableView: UITableView, prefetchRowsAt indexPaths: [IndexPath]) {
let fetchRequest: NSFetchRequest<MyModel> = MyModel.fetchRequest()
fetchRequest.returnsObjectsAsFaults = false
let items = indexPaths.map { fetchedResultsController.object(at: $0) }
fetchRequest.predicate = NSPredicate(format: "SELF IN %#", items)
let asyncFetchRequest = NSAsynchronousFetchRequest(fetchRequest: fetchRequest)
do {
try fetchedResultsController.managedObjectContext.execute(asyncFetchRequest)
} catch { }
}
What it does is pretty self explanatory. It runs an async fetch request on the managed object context, therefore resolving faults, if any.
In short, you are creating a custom predicate to "warm" the predicted specific objects in the MOC. The idea, I think, is that the table view is telling you more targeted scroll target info (based on velocity etc.) about where it thinks the table view will end up. This allows you to execute an async request to pre-fetch these objects in the MOC so that by the time you are actually asking for properties on it, they're ready to go.
This is why you also have a nil completion on the async request. You aren't directly using the results because you may not actually be displaying any of these index paths yet. It also means it's unnecessary to do additional tracking (like in a Dictionary, etc.) of results when you get pre-fetched objects.
The real question is, if you are using an FRC with batching, pre-fetched properties, etc., is this really buying you anything to side-step it? I'm not really sure. I haven't noticed a huge difference between a pretty highly tuned FRC+UITableView with vs without pre-fetching on a table of around 1000 results. It's also possible by merely touching those objects that the FRC will see that (because you are sharing the same MOC) and then will run through it's property/relationship pre-fetching, etc. I haven't seen much documentation on this mechanism, but if Apple is suggesting it, especially using an FRC, I have to think it helps.
I'm pretty new to swift and core data. So if this logic doesn't make sense, please let me know.
I'm trying to save user's images to core data and be able to delete selected one.
I have Entity "Media", attribute "photo" type "binary data"
Here I convert UIImage to NSData
let data = UIImagePNGRepresentation(addedImage!) as NSData?
And save it.
let newPhoto = NSEntityDescription.insertNewObject(forEntityName: "Media", into: context)
newPhoto.setValue(imageData, forKey: "photo")
Here I try to retrieve that one selected image from core data using predicate
func deleteImage(imageData: NSData) {
let context = appDelegate.persistentContainer.viewContext
let request = NSFetchRequest<NSFetchRequestResult>(entityName: "Media")
request.returnsObjectsAsFaults = false
request.predicate = NSPredicate(format: "photo =%#", imageData)
do {
let results = try context.fetch(request)
...
And I'm getting this error message.
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[OS_dispatch_data _bytesPtrForStore]: unrecognized selector sent to instance
Saving and getting all images from entity "Media" is working perfectly. I just can't get the specific one.
Any thoughts/suggestions?
Thanks.
Saving images like this, with external storage enabled for the property, can be OK. But there are some very unusual things going on here.
The common approach would be to have the image be part of an entity with other information, but not as the primary value of the entity. For example an entity type that represents a person and has name, address, etc, and a picture. Storing just a picture in an entity is kind of weird. You're not getting much of anything from using Core Data over something simpler. If nothing else I'd guess you had a photo ID, or title, or some other data in the entity.
The second thing relates to that-- it's extremely unusual to try to fetch using a predicate based on a binary data property. I guess it might work but only if you can ensure that the predicate value is identical to one in the persistent store all the way through-- not a single bit changed in the binary blob. That might be true but I wouldn't rely on UIKit not doing something along the way that makes it subtly different. I've never seen that particular error message but I'm not surprised that this isn't working.
Finally, it's also unusual (though not quite so strange as the above) to delete an object based on a fetch request. You would have fetched these objects before to get the images. Why fetch the same object again just to delete it? Delete the one you already fetched.
A more reliable approach would be to at least include a unique ID in the entity, which might be an integer or a UUID or whatever else works for you. Fetch this entity and display images using the photo property. When you want to delete one, take the managed object from your fetch and tell the context to delete that one. Or if fetching is necessary for some reason, use the unique ID in the predicate instead of the binary data.
You better stop doing it. Saving heavy data (image, audio, video, ...) to core data will greatly decrease fetching speed when you have more records. Instead, save image to application folder, and then only save its url to core data.
I am using Core Data to see whether messages in a table view have been seen before by the user. The way I do this is to save the message Id to Core Data the first time it is seen, and then I run a fetch request when I update the table view to see if there is an entry in the persistent memory with the same Id.
Now what I want to know is how I should most effectively implement my fetch request, based on how time consuming it is. Should I either run a request that returns all saved message Ids as an array when the view is loaded, and then in cellForRowAtIndexPathcheck if that array contains that cell's message Id, or run the fetch request with a predicate in cellForRowAtIndexPath? The latter would be my preferred method, but If i have 100 or so cells I wondered if this would be poor etiquette.
Any help would be very much appreciated.
This is my fetch Request :
func persistQuery(predicateValueString: String!) -> Bool! {
let fetchRequest = NSFetchRequest(entityName: "LogItem")
let predicate = NSPredicate(format: "itemText == %#", predicateValueString)
fetchRequest.predicate = predicate
var didFindResult = true
if let fetchResults = managedObjectContext!.executeFetchRequest(fetchRequest, error: nil) as? [LogItem] {
if fetchResults.count == 0 {
didFindResult=false
}
}
return didFindResult
}
The best way is to use a NSFetchedResultsController. It will optimize the fetching and the memory footprint as well. It is specifically designed for table views.
To get started, take a look at the Xcode template (Master/Detail, check Core Data). It is really quite simple.
Make sure you also implement the delegate methods - they will automatically be called when your managed objects change, so there is only minimal code that is executed to update the UI (an only if the object is actually on screen).
Presumably each of your table view cells represent a LogItem (the NSManagedObject subclass) with a property to indicate the read status. Once you change that, the delegate method will try to update it based on the index path.
That's all there is to it. With the fetched results controller you get a lot of optimization for free, so I would strongly recommend using it whenever you populate a table view with Core Data entities.
This has been asked before, but no solution described that is fast enough for my app needs.
In the communications protocol we have set up, the server sends down a new set of all customers every time a sync is performed. Earlier, we had been storing as a plist. Now want to use Core Data.
There can be thousands of entries. Deleting each one individually takes a long time. Is there a way to delete all rows in a particular table in Core Data?
delete from customer
This call in sqlite happens instantly. Going through each one individually in Core Data can take 30 seconds on an iPad1.
Is it reasonable to shut down Core Data, i.e. drop the persistence store and all managed object contexts, then drop into sqlite and perform the delete command against the table? No other activity is going on during this process so I don't need access to other parts of the database.
Dave DeLong is an expert at, well, just about everything, and so I feel like I'm telling Jesus how to walk on water. Granted, his post is from 2009, which was a LONG time ago.
However, the approach in the link posted by Bot is not necessarily the best way to handle large deletes.
Basically, that post suggests to fetch the object IDs, and then iterate through them, calling delete on each object.
The problem is that when you delete a single object, it has to go handle all the associated relationships as well, which could cause further fetching.
So, if you must do large scale deletes like this, I suggest adjusting your overall database so that you can isolate tables in specific core data stores. That way you can just delete the entire store, and possibly reconstruct the small bits that you want to remain. That will probably be the fastest approach.
However, if you want to delete the objects themselves, you should follow this pattern...
Do your deletes in batches, inside an autorelease pool, and be sure to pre-fetch any cascaded relationships. All these, together, will minimize the number of times you have to actually go to the database, and will, thus, decrease the amount of time it takes to perform your delete.
In the suggested approach, which comes down to...
Fetch ObjectIds of all objects to be deleted
Iterate through the list, and delete each object
If you have cascade relationships, you you will encounter a lot of extra trips to the database, and IO is really slow. You want to minimize the number of times you have to visit the database.
While it may initially sound counterintuitive, you want to fetch more data than you think you want to delete. The reason is that all that data can be fetched from the database in a few IO operations.
So, on your fetch request, you want to set...
[fetchRequest setRelationshipKeyPathsForPrefetching:#[#"relationship1", #"relationship2", .... , #"relationship3"]];
where those relationships represent all the relationships that may have a cascade delete rule.
Now, when your fetch is complete, you have all the objects that are going to be deleted, plus the objects that will be deleted as a result of those objects being deleted.
If you have a complex hierarchy, you want to prefetch as much as possible ahead of time. Otherwise, when you delete an object, Core Data is going to have to go fetch each relationship individually for each object so that it can managed the cascade delete.
This will waste a TON of time, because you will do many more IO operations as a result.
Now, after your fetch has completed, then you loop through the objects, and delete them. For large deletes you can see an order of magnitude speed up.
In addition, if you have a lot of objects, break it up into multiple batches, and do it inside an auto release pool.
Finally, do this in a separate background thread, so your UI does not pend. You can use a separate MOC, connected to a persistent store coordinator, and have the main MOC handle DidSave notifications to remove the objects from its context.
WHile this looks like code, treat it as pseudo-code...
NSManagedObjectContext *deleteContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateConcurrencyType];
// Get a new PSC for the same store
deleteContext.persistentStoreCoordinator = getInstanceOfPersistentStoreCoordinator();
// Each call to performBlock executes in its own autoreleasepool, so we don't
// need to explicitly use one if each chunk is done in a separate performBlock
__block void (^block)(void) = ^{
NSFetchRequest *fetchRequest = //
// Only fetch the number of objects to delete this iteration
fetchRequest.fetchLimit = NUM_ENTITIES_TO_DELETE_AT_ONCE;
// Prefetch all the relationships
fetchRequest.relationshipKeyPathsForPrefetching = prefetchRelationships;
// Don't need all the properties
fetchRequest.includesPropertyValues = NO;
NSArray *results = [deleteContext executeFetchRequest:fetchRequest error:&error];
if (results.count == 0) {
// Didn't get any objects for this fetch
if (nil == results) {
// Handle error
}
return;
}
for (MyEntity *entity in results) {
[deleteContext deleteObject:entity];
}
[deleteContext save:&error];
[deleteContext reset];
// Keep deleting objects until they are all gone
[deleteContext performBlock:block];
};
[deleteContext preformBlock:block];
Of course, you need to do appropriate error handling, but that's the basic idea.
Fetch in batches if you have so much data to delete that it will cripple memory.
Don't fetch all the properties.
Prefetch relationships to minimize IO operations.
Use autoreleasepool to keep memory from growing.
Prune the context.
Perform the task on a background thread.
If you have a really complex graph, make sure you prefetch all the cascaded relationships for all entities in your entire object graph.
Note, your main context will have to handle DidSave notifications to keep its context in step with the deletions.
EDIT
Thanks. Lots of good points. All well explained except, why create the
separate MOC? Any thoughts on not deleting the entire database, but
using sqlite to delete all rows from a particular table? – David
You use a separate MOC so the UI is not blocked while the long delete operation is happening. Note, that when the actual commit to the database happens, only one thread can be accessing the database, so any other access (like fetching) will block behind any updates. This is another reason to break the large delete operation into chunks. Small pieces of work will provide some chance for other MOC(s) to access the store without having to wait for the whole operation to complete.
If this causes problems, you can also implement priority queues (via dispatch_set_target_queue), but that is beyond the scope of this question.
As for using sqlite commands on the Core Data database, Apple has repeatedly said this is a bad idea, and you should not run direct SQL commands on a Core Data database file.
Finally, let me note this. In my experience, I have found that when I have a serious performance problem, it is usually a result of either poor design or improper implementation. Revisit your problem, and see if you can redesign your system somewhat to better accommodate this use case.
If you must send down all the data, perhaps query the database in a background thread and filter the new data so you break your data into three sets: objects that need modification, objects that need deletion, and objects that need to be inserted.
This way, you are only changing the database where it needs to be changed.
If the data is almost brand new every time, consider restructuring your database where these entities have their own database (I assume your database already contains multiple entities). That way you can just delete the file, and start over with a fresh database. That's fast. Now, reinserting several thousand objects is not going to be fast.
You have to manage any relationships manually, across stores. It's not difficult, but it's not automatic like relationships within the same store.
If I did this, I would first create the new database, then tear down the existing one, replace it with the new one, and then delete the old one.
If you are only manipulating your database via this batch mechanism, and you do not need object graph management, then maybe you want to consider using sqlite instead of Core Data.
iOS 9 and later
Use NSBatchDeleteRequest. I tested this in the simulator on a Core Data entity with more than 400,000 instances and the delete was almost instantaneous.
// fetch all items in entity and request to delete them
let fetchRequest = NSFetchRequest(entityName: "MyEntity")
let deleteRequest = NSBatchDeleteRequest(fetchRequest: fetchRequest)
// delegate objects
let myManagedObjectContext = (UIApplication.sharedApplication().delegate as! AppDelegate).managedObjectContext
let myPersistentStoreCoordinator = (UIApplication.sharedApplication().delegate as! AppDelegate).persistentStoreCoordinator
// perform the delete
do {
try myPersistentStoreCoordinator.executeRequest(deleteRequest, withContext: myManagedObjectContext)
} catch let error as NSError {
print(error)
}
Note that the answer that #Bot linked to and that #JodyHagins mentioned has also been updated to this method.
Really your only option is to remove them individually. I do this method with a ton of objects and it is pretty fast. Here is a way someone does it by only loading the managed object ID so it prevents any unnecessary overhead and makes it faster.
Core Data: Quickest way to delete all instances of an entity
Yes, it's reasonable to delete the persistent store and start from scratch. This happen fairly quick. What you can do is remove the persistent store (with the persistent store URL) from the persistent store coordinator, and then use the url of the persistent store to delete the database file from your directory folder. I did it using NSFileManager's removeItemAtURL.
Edit: one thing to consider: Make sure to disable/release the current NSManagedObjectContext instance, and to stop any other thread which might be doing something with a NSManagedObjectContext which is using the same persistent store. Your application will crash if a context tries to access the persistent store.