I realize this question has been asked before but I didn't find any answers that actually resolved it, so...asking again.
I'm using ARC. My app takes photos from an AVCaptureSession at periodic intervals and saves them to core data. I get the NSData object by calling UIImagePNGRepresentation(). As this is happening, memory steadily climbs and the app eventually quits due to memory pressure...but there are no leaks. Instruments shows that for each photo, 500k is being malloced and it never gets released.
The answer I keep coming across is to wrap UIImagePNGRepresentation, or indeed the whole method body, in an autoreleasepool, but this did not help.
I am reasonably certain that the UIImagePNGRepresentation call is the culprit, because when I comment it out, no more memory problems (or images to save).
Would appreciate any help here...is there another way to grab NSData from a UIImage? Is this just another SDK bug we have to live with?
-(void)photoTimerFired:(NSTimer*)timer
{
...
ManagedDataPoint *lastPoint = [_currentSession.dataPoints lastObject];
_lastImage = [_imageCapturer singleImage];
Photo *newPhoto = [NSEntityDescription insertNewObjectForEntityForName:#"Photo"
inManagedObjectContext:self.managedObjectContext];
// Line below is the culprit.
newPhoto.photoData = UIImagePNGRepresentation(_lastImage);
newPhoto.managedDataPoint = lastPoint;
}
I think this page of the CoreData Programming Guide is worth reading in your case, talking to things you need to take care to minimize overheads when using BLOBs (binary data) like images in CoreData.
Especially the part when they talk about creating a dedicated entity to only hold your binary attribute / image, separating it from other attributes of your primary entity and allowing it to "fault", so that it will only be loaded from the database into memory when the attribute is actually referenced/used.
Another thing to try is to check the "Use External Storage" checkbox on the binary attribute of your entity that holds the image. This way in practice the image won't actually be saved as a BLOB directly in your sqlite database, but will be saved in an external file instead, the attribute holding only a reference (path) to this external file, limiting the growth of your database (and corruption risks as the base grows in size).
(Hopefully it will also reduce your memory footprint by avoiding to keep the image in memory when the NSManagedObject is around and not faulting...?)
Note: all of this "External Storage" stuff is working totally transparently for you as for the usage of this attribute in your code: you still access it as if the attribute directly contained the binary data.
Related
I'm building a tab bar application for iPhone and i'm using Core Data with two UIManagedDocuments. In the first tab, i write the data to database and in the second i read them into UITableView with UIFetchedResultsController.
At the start of application, if i write data first, and after then i read results, it works fine. Results appear in the second tab immediately. However, if i read some data first and after then if i write something to database, results appear in second tab with considerable delay (almost 1 minutes). If is there any synchronization problem between two UIManagedObjectContexts or two UIManagedDocuments, how does it works in the first condition? And, is there any solution for this delay?
The way that you can ensure that your UIManagedDocument is up to date is to make sure you're saving your changes properly. Given the information you've shown above, I'm not really sure about how you're managing your documents or your managedObjectContexts. There are just too many factors that could be affecting this to be able to give you a 100% concrete answer.
So without knowing what your code looks like and without knowing how you're managing your context, the only thing I can do is give you what I use in my own projects. This may or may not help you, but it helped me - more times than one - when it comes to handing core data by UIManagedDocument.
When it comes to Context:
I use a singleton to manage UIManagedDocument. I do this because I don't want to have to deal with what you're talking about above - having more than one managedObjectContext. When you start dealing with multiple contexts, you have the issue where the data will not be consistent unless you manage all of your contexts properly. If you save on one but don't update the other - then your data can become out of sync. You also have to make sure that each context is working on the property thread - the Apple Docs is a great resource for understanding the whys ad hows this even matters.
The point is, though, that this is one of the biggest problems with working with UIManagedDocument that isn't as bad as when you're working with pure core data and using a SQL persistent store. The main reason that I've found is because of how UIManagedDocument actually saves to its UIDocument store. It is very unpredictable about when it wants to save. This makes knowing when your UIManagedDocument will actually persist and have your data a freaking shot in the dark. You end having to do all kinds of stupid stuff just to make sure that it is always readily available.
Considering I have a belief (that many, maybe rightfully so, believe is an ignorant belief) that working with core data is hard, and UIManagedDocument makes it easier than it would be if you didn't work with it at all. That being said, I don't like it when working with something as simple as UIManagedDocument begins to get complicated - so I use the one thing that has always kept it simple, and that is a singleton, shared-instance of a single UIManagedDocument so that I only have 1 managedObjectContext, ever, to have to work with.
When it comes to saving:
Whenever I make any significant change to a model ( Create, Update, Delete, Edit ), I always make sure to call [document updateChangeCount:UIDocumentChangeDone]; I do this because I do not use the "undo manager" (NSUndoManager) when working with UIManagedDocument. Simply because I haven't needed to yet, plus because I hate all the "workaround" garbage you have to do with it.
Working only on the Main Thread:
Whenever I do anything with my UIManagedDocument or Core Data, I always make sure its on the main thread. I think I've already said it once, but I'll say it again: working in threads is helpful when you need it, but also when you actually understand threading in general. I like working in threads, but it comes at a cost of complexity that makes me not want to work in them when it comes to core data. With that being the case, I tend to stay strictly on the main thread as this keeps things simple and easy (for me).
Saving the Document
When I absolutely need to make sure that the UIManagedDocument is "saved" ( written to disk ), I have 2 methods that I wrote and use that are always readily available for me to call: saveDocument and forceSaveDocument.
The first one ( saveDocument ) merely checks the context for changes. If it has any, it then checks to see if we have any newly inserted objects. When insertedObjects are found, it obtains the perm ids for these items. You can think of this one as a good way to ensure that your core data model is up to date, and that your managed context is in a safe state, so that when your document is actually saved, that you get everything saved in the state that it needs to be in (your ids are realized, your contexts are clean, and what you are about to save represents a state of your model once all work has been complete on it).
Its big brother, forceSaveDocument, actually calls saveDocument first. Again, to make sure that your actual model/context is saved and proper. If it returns successful ( YES ), then it will actually do the real saving and write the document to disk by means of saveToUrl.
Some Code (hopefully it helps):
Here are those 2 methods in case it helps:
-(BOOL)saveDocument {
NSManagedObjectContext *context = self.document.managedObjectContext;
if(!context.hasChanges) return YES;
NSSet *inserts = [context insertedObjects];
if([inserts count] > 0) {
NSError *error = nil;
if(![context obtainPermanentIDsForObjects:[inserts allObjects] error:&error] && error) {
[self reportError:error];
return NO;
}
}
return YES;
}
-(void)forceSaveDocument {
if( [self saveDocument] ) {
[self.document saveToURL:self.document.fileURL forSaveOperation:UIDocumentSaveForOverwriting completionHandler:self.onSaveBlock ? self.onSaveBlock : nil];
}
}
General Rules/Guidelines
Overall, these are my guidelines that I follow ( and have worked for me for about 3 years now ) when working with UIManagedDocument and Core Data. I'm sure there are better out there from guys/gals much smarter than me, but these have what I use. The benefit I get out of them is that it makes me have to worry less about managing my data and gives me more freedom to work with everything else:
Use a singleton to manage my UIManagedDocument until the need of multiple threads is absolutely necessary - then migrate over to start using multiple contexts ( i've never needed to do this yet - but then again I try to keep things simple )
Always call updateChangeCount:UIDocumentChangeDone when I make any change to a model. It is very light weight and has little impact. If anything, it will help ensure your document stays up to date and never gets too out of sync with your data.
Don't use undo manager unless you actually need it ( I have yet to need it )
Use save/ForceSave sparingly, and only when absolutely necessary (deletes are a good reason to use it. Or if you create a new item on one view controller and need it on the next one, but can't wait for core data and the document to sync up - its kind of like kicking it in the ass and saying "I object to you saving whenever you want - save now lol.. )
Final Thoughts
All of the above is my own belief and understandings. These come from a lot of research, reading, and being a pain the ass when it comes to wanting to do things right, all while keeping it simple. Anyone can write a complex solution - but I think the fundamental question is always: do you really need the complexity, or do you just need it to work so you can focus on more complex issues?
I'm sure the above is way more than you probably wanted, and may even add more questions than you have. If you need some links and resources, let me know and I'll try to throw a few together.
Either way, hope that helps.
I have a database object and some photos objects.
The database object contains an integer property and a mutable dictionary property.
The integer keeps track of the next free number to use as a mutable dictionary key when I create a new photo object and the mutable dictionary contains pointers to photo objects.
Photo objects contain an image, an image description and the date the image was taken.
I've been using NSKeyedUnarchiver and NSKeyedArchiver to read and write these objects in and out when my applicationDidBecomeActive and applicationWillResignActive methods trigger.
And, it's all been working well. When applicationWillResignActive triggers, it calls NSKeyedArchiver and points it to the database object as the root. The coder then writes out the integer and then when it encounters the mutable dictionary, it descends into it and each photo object is called to save its properties which are the image, the description and the date.
As I said, it's all been working well. But, it has seemed slower and slower as the number of photos has increased so I did some timings.
I found that reading the archive in is roughly 25 times faster than writing it out.
So, I conceived the idea of only writing out the photos which are new or changed as a method of speeding up the write side. After all, most of the photos are from past sessions and I might have 30 or 50 of them from before and I might only shoot two or three new ones this time.
I created some flags that indicate if a photo is new or old. When applicationWillResignActive triggers and I find myself down in the photos object handling each encodeWithCoder call, I save the image, description and date if the photo is new and I skip saving if it is old.
Well, I did not get the result I'd hoped for :-)
When applicationWillResignActive triggers, all the photos I skip writing out end up getting written out as empty photo objects which overwrites the previous photo objects out there with the same key. Then, when I load them back in, I've got bupkis, nada, zip.
First question, I guess, is can I write out only part of my object tree and have the parts I don't write out still remain out there intact from an earlier full write? I'm beginning to wonder if that might be a naive idea.
Gallymon
Using archives of some array of objects is going to be increasingly inefficient as the archive grows. This is exacerbated by the fact that your objects include images. Two thoughts:
You should consider using Core Data for storing this data. (You could use SQLite, directly, too, but Core Data is the preferred object persistence technology in iOS.) This way, when you want to save a new object, you just add that new object, and don't archive the whole collection of objects every time.
Furthermore, if speed is of primary interest and if your images are large, SQLite (the database that Core Data employs by default) is inefficient when handling large BLOBs (i.e. things like images). So, as inelegant as it sounds, you probably want to save images to your Documents folder, and only store a image file URL or path in CoreData.
why don't you save the image only on the init method ? if you create the object chances are you want to save it. so place the flag hasChanges == off in the init method and you check that flag for saving...
I'm using a plist file which contains all app my data. The file is quite big and currently I'm loading all the stuff into Arrays and Dictionaries at first launch and save them into UserDefaults so that I don't have to touch the plist again. As this takes about 10 secs (iP4) I wonder if there is an even faster (better) way to process the plist. I checked the whole startup with Instruments and going through the hundreds of entries is actually the fastest part. It takes very long to save these processed stuff into NSUserDefaults.
You might benefit from saving the plist to your own file. That way you control the reading/writing, don't have any overhead associated with NSUserDefaults, and, most importantly, can ensure the format. That is, if reading/writing is producing the slow down, then you'll have to minimize the plist file size. Likely using a plist format of NSPropertyListBinaryFormat_v1_0 will do that:
See:
+ (NSInteger) writePropertyList: (id) plist
toStream: (NSOutputStream *) stream
format: (NSPropertyListFormat)format
options: (NSPropertyListWriteOptions) opt
error: (NSError **) error
From Apple's Property List Programming Guide:
The first approach [using NSDictionary or NSArray writeToFile] is
simpler—it requires only one method invocation instead of two—but the
second approach [as above] has its advantages. It allows you to convert the
runtime property list to binary format as well as an XML property
list. When you convert a static representation of a property list
back into a graph of objects, it also lets you specify with more
flexibility whether those objects are mutable or immutable.
Several points.
NSUserDefaults is probably just a big plist, so why use it? Stick your entries into a singleton that holds the in-memory structure.
If you're doing this on first load because you want it to be mutable, put the defaults into your resource folder. When you want to load it, check if you have it in the documents folder, and if you don't ( first load), copy it from the resource bundle to the documents.
If you're using NSUserDefaults for persistence, just write out your data to your plist in applicationShouldResignActive, and at any other times where you make important changes.
Write it in a background thread, but you probably need to do some locking here.
Best practise when load and save times become to big is probably move to core data, but 1-4 should give you some more mileage before you need to do that.
I am planning on creating my app in a 'Model-View-Controller'(MVC)-style, and in the end, for me at least, this means that all data is stored in the controller-class. Let's say I have a class Player, and the player has several objects of class Weapons or Equipment or whatever. The initialization of Controller* stores the player(s), so if I can store/save only the Controller-object over time, even if the app or the device restarts, that would be nice. I did this in Java one, I put in Serialization = 100L;(or something like it) in the top of the file of every object that would be included when saving the Controller-object, and it worked perfectly. Is this possible in ios/cocoa-touch/objective-c?
I have read and used Core Data (not very much), but that is just a database-table, sql?, which would have me extract every piece of information of every object?
For instance, if the object Player* has a member NSString *name;, I would have to save the actual string in Core Data, instead of saving the object of the player? Like, varchar.
If there is any way to store an entire custom object on the device for further use, I would very much like to know what it's called, and where I can read about it/tutorials.
Read up on the NSCoding protocol. You can make your object complient to it, then serialized it and save it to a file. Later you can restore it to the same state by using a decoder. For sure some other posts that cover this topic on SO.
I'm optimising my first iOS app before it hits the store, and noting methods which take seemingly larger amounts of time. I have a fairly simple master-detail app where entities from the Core Data SQLite are shown in a UITableView, then tapping one brings up a detail view where the user can mark it as a favorite (setting a BOOL flag in that object to YES. As soon as they hit their Favorite button, I call [NSManagedObjectContext save] to ensure their changes are reflected immediately, and in case of an unscheduled terminate, etc.
This save operation is currently taking around 205ms when testing on my iPhone 4S. There are around 4,500 entries in the database, each with a few strings, and a few boolean values (wrapped in NSNumbers).
First question: should it take 200ms to make this change? I'm only setting one boolean value, then saving the context, but I've never used Core Data before so I don't know if this is about normal.
Second question: the code I'm using is below – am I doing something wrong in the code to make the method take this long to execute?
- (IBAction) makeFavorite: (id) sender
{
[self.delegate detailViewControllerDidMakeFavorite];
[_selectedLine setIsLiked: [NSNumber numberWithBool: YES]];
[_selectedLine setIsDisliked: [NSNumber numberWithBool: NO]];
NSError *error;
if (![[[CDManager sharedManager] managedObjectContext] save:&error]) NSLog(#"Saving changes failed: %#, %#", error, [error userInfo]);
}
Perhaps I'm worrying over nothing (I am still a relatively new programmer), but on a wider note, 200ms is enough for me to at least try to address this issue, right? :)
Consider UIManagedDocument. It automatically handles saving in a background context. I especially recommend it if you are on iOS 6. If you are not passing object IDs around, or merging with other contexts, then you should be able to use it fairly easily and reliably.
Your simple use case seems tailor made for it.
1) Should the save of one change of boolean value take 200 ms?
Yes, it might take this long. You are performing an IO operation and according to the documentation:
When Core Data saves a SQLite store, SQLite updates just part of the store file. Loss of that partial update would be catastrophic, so you may want to ensure that the file is written correctly before your application continues. Unfortunately, doing so means that in some situations saving even a small set of changes to an SQLite store can take considerably longer than saving to, say, an XML store.
-
2) am I doing something wrong in the code to make the method take this long to execute?
No. you are making the save to the store (under the assumption you have no parent context).
-
3) Are 200ms enough for me to at least try to address this issue?
Yes. 200ms are a noticeable time for a human, and will be felt. you could try and perform the save in the background, but this is unsafe according to the documentation. or, move it to the end of the entire object editing.
My advise would be to read and see if you could make some compromises in your context architecture (your CoreData stack structure).
From my experience, saving in the background is not that bad.