I understand how Automatic Reference Counting works:
At compile time, it is determined the possible types of relationships between objects and thus where releases may occur, then at run time, the number of strong pointer references to each object is tracked and it is released when that number reaches 0. I have yet to encounter problems with this in concept or in practice, at least when dealing with the main thread.
I noticed that when I start a new background thread, it does not release any produced objects until the thread ends. I ran an example:
Essentially, an automatic '#autoreleasepool' is placed around the thread call, so it should be obvious that placing my own will not fix this issue. In fact, I have tested that with the same results. If I am correct in that, then the existence of that enforced pool is exactly causing my issue, but I assume that is the best that ARC can perform on a multithreaded application. The memory usage slowly and consistently inclines. If I leave this thread too long, the app eventually runs out of memory. This is a problem since the thread needs to be able to run indefinitely at worst.
I removed some of the main allocing in the thread already. I believe that I have determined that some of the remaining memory allocing are NSNumbers being released from an NSMutableArray because I am overwriting to it.
So I suppose I will have to do one of the following:
Remove consistent allocing in the thread altogether.
Change app to non-ARC to manually release memory in background thread.
Detect when memory is high, save the state of the thread, sync with main thread to release objects, then resume the algorithm.
Find out if there exists some way to notify the main thread or ARC that I want an object synchronized so that it may be released.
Realize that Apple actually has a way to dispatch ARC to properly handle another threads asynchronously and never said anything about it on the main reference page.
Disable ARC in the files with the alloced objects such as the dictionary. How can I disable ARC for a single file in a project?
None of those look like pretty solutions to my problem, although I may try 1 or 6. Does anyone have advice?
Update:
I ran the same algorithm but with the following autorelease blocks added to the code, at first thinking I was going to disprove rmaddy and Aaron Brager's responses.
-(void)setInt: (int)value For: (NSString *)variableName {
#autoreleasepool {
[self.intDictionary setValue:#(value) forKey:variableName];
}
}
-(void)setBool: (bool)value For: (NSString *)variableName {
#autoreleasepool {
[self.boolDictionary setValue:#(value) forKey:variableName];
}
}
Here is the resulting memory allocation graph:
They were correct. I am glad that I was wrong in this case. It means my coding is going to be a lot easier than I was starting to imagine.
Autoreleased objects are deallocated when their autorelease pool drains. This typically occurs at the end of a thread's run loop.
This behavior is the same whether you're talking about the main thread or a background thread. And of course, it only applies to objects that no longer have any strong references.
This analysis of yours is not entirely correct:
Essentially, an automatic '#autoreleasepool' is placed around the thread call, so it should be obvious that placing my own will not fix this issue.
Consider this code:
#autoreleasepool {
for (int i = 0; i < 100000; i++) {
// create an expensive autoreleased object
// do something with it
}
}
In this case, none of the autoreleased objects will be deallocated until the end of the run loop when the autorelease pool drains.
However, if you add your own autorelease pool:
#autoreleasepool {
for (int i = 0; i < 100000; i++) {
#autoreleasepool {
// create an expensive autoreleased object
// do something with it
}
}
}
The objects will be deallocated when the inner pool drains, for each iteration of the for loop.
If adding your own autorelease pool as shown above didn't resolve your issue, then the remaining possibilities are:
You are using ARC and the objects are not getting deallocated because they still have a strong reference (perhaps you have a strong reference cycle)
Your increase in memory usage is not from Objective-C objects (for example, you created a CGImageRef and never called CGImageRelease)
You are mistakenly not using ARC for this class (and you're not calling release)
Your six proposed solutions could also resolve the problem, but it's probably easier to fix than that.
Related
Are there any conditions in Objective-C (Objective-C++) where the compiler can detect that a variable capture in a block is never used and thus decide to not capture the variable in the first place?
For example, assume you have an NSArray that contains a large number of items which might take a long time to deallocate. You need to access the NSArray on the main thread, but once you're done with it, you're willing to deallocate it on a background queue. The background block only needs to capture the array and then immediately deallocate. It doesn't actually have to do anything with it. Can the compiler detect this and, "erroneously", skip the block capture altogether?
Example:
// On the main thread...
NSArray *outgoingRecords = self.records;
self.records = incomingRecords;
dispatch_async(background_queue, ^{
(void)outgoingRecords;
// After this do-nothing block exits, then outgoingRecords
// should be deallocated on this background_queue.
});
Am I guaranteed that outgoingRecords will always be captured in that block and that it will always be deallocated on the background_queue?
Edit #1
I'll add a bit more context to better illustrate my issue:
I have an Objective-C++ class that contains a very large std::vector of immutable records. This could easily be 1+ million records. They are basic structs in a vector and accessed on the main thread to populate a table view. On a background thread, a different set of database records might be read into a separate vector, which could also be quite large.
Once the background read has occurred, I jump over to the main thread to swap Objective-C objects and repopulate the table.
At that point, I don't care at all about the contents of the older vector or its parent Objective-C class. There's no fancy destructors or object-graph to teardown, but deallocating hundreds of megabytes, maybe even gigabytes of memory is not instantaneous. So I'm willing to punt it off to a background_queue and have the memory deallocation occur there. In my tests, that appears to work fine and gives me a little bit more time on the main thread to do other stuff before 16ms elapses.
I'm trying to understand if I can get away with simply capturing the object in an "empty" block or if I should do some sort of no-op operation (like call count) so that the compiler cannot optimize it away somehow.
Edit #2
(I originally tried to keep the question as simple as possible, but it seems like it's more nuanced then that. Based on Ken's answer below, I'll add another scenario.)
Here's another scenario that doesn't use dispatch_queues but still uses blocks, which is the part I'm really interested in.
id<MTLCommandBuffer> commandBuffer = ...
// A custom class that manages an MTLTexture that is backed by an IOSurface.
__block MyTextureWrapper *wrapper = ...
// Issue some Metal calls that use the texture inside the wrapper.
// Wait for the buffer to complete, then release the wrapper.
[commandBuffer addCompletedHandler:^(id<MTLCommandBuffer> cb) {
wrapper = nil;
}];
In this scenario, the order of execution is guaranteed by Metal. Unlike the example above, in this scenario performance is not the issue. Rather, the IOSurface that is backing the MTLTexture is being recycled into a CVPixelBufferPool. The IOSurface is being shared between processes and, from what I can tell, MTLTexture does not appear to increase the useCount on the surface. My wrapper class does. When my wrapper class is deallocated, the useCount is decremented and the bufferPool is then free to recycling the IOSurface.
This is all working as expected but I end up with silly code like above just out of uncertainty whether I need to "use" the wrapper instance in the block to ensure it's captured or not. If the wrapper is deallocated before the completion handler runs, then the IOSurface will be recycled and the texture will get overwritten.
Edit to address question edits:
From the Clang Language Specification for Blocks:
Local automatic (stack) variables referenced within the compound
statement of a Block are imported and captured by the Block as const
copies. The capture (binding) is performed at the time of the Block
literal expression evaluation.
The compiler is not required to capture a variable if it can prove
that no references to the variable will actually be evaluated.
Programmers can force a variable to be captured by referencing it in a
statement at the beginning of the Block, like so:
(void) foo;
This matters when capturing the variable has side-effects, as it can
in Objective-C or C++.
(Emphasis added.)
Note that using this technique guarantees that the referenced object lives at least as long as the block, but does not guarantee it will be released with the block, nor by which thread.
There's no guarantee that the block submitted to the background queue will be the last code to hold a strong reference to the array (even ignoring the question of whether the block captures the variable).
First, the block may in fact run before the context which submitted it returns and releases its strong reference. That is, the code which called dispatch_async() could be swapped off the CPU and the block could run first.
But even if the block runs somewhat later than that, a reference to the array may be in an autorelease pool somewhere and not released for some time. Or there may be a strong reference someplace else that will eventually be cleared but not under you explicit control.
I am working on an app where I am presenting 100 sentences using AVAudioplayer. Rather than have 100 AVAudioplayer objects I wanted to just have one property and change the object associated with it on the fly. My code boils down to the following (though the lines arent immediately following each other in the actual code):
self.thePlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:url1 error:&error];
self.thePlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:url2 error:&error];
Does the object initialized with url1 get released when thePlayer is allocated and initialized a second time with url2, or are both objects only released when the view is dismissed? As I am dealing with 100 sound files I don't want them all hanging around in memory. I'm using ARC
Thanks in advance
In your specific case, guessing at what your code likely includes, the objects will probably be deallocated when you want them to be. That's a lot of "guessing," "likely," and "probably." You really need to understand how the memory management works in order to reason about it.
If the AVAudioPlayer is strongly referenced by anything else, then it won't be released until those strong references are gone. In other words, setting thePlayer won't deallocate the player if something else has a strong reference to it. (That "something" may be some part of the system frameworks, or even itself in some rare cases. It doesn't have to be your code.)
If the AVAudioPlayer has pending autorelease calls on it, then it won't be released until the autorelease pool drains (usually at the end of event loop, which basically means "when your method that UIKit called returns.") For example, if you create a large number of objects in a loop and immediately throw them away, they may or may not be deallocated until the autorelease pool drains. Again, autoreleases may be injected by system frameworks. In practice, this means that the object will usually be deallocated "soon" (in a small fraction of a second), but not necessarily immediately. You can clean up autoreleased objects sooner by using #autoreleasepool blocks, which is sometimes necessary if you create many temporary objects in a loop. This is not needed very often.
But to a first-order approximation, in many of the most common cases, yes, replacing the property will automatically and immediately deallocate the previous object.
It would be useful to show how you declared thePlayer. If they are synthesized properly the memory management would be handled automatically. It appears that you are using "self" to access thePlayer and if so you'd be setting the value through a setter/getter and that would handle the memory management for you. But I also notice that "Self" is capitalized and should not be in order to properly use the setter/getter. For more info on synthesized variables check out: What exactly does #synthesize do?. Note there are some places where you should NOT use self and this link discusses that: How does an underscore in front of a variable in a cocoa objective-c class work?.
As I was walking through some line of codes I stumbled upon this problem a couple of days ago,
- (void)dealloc {
...
[self.postOfficeService deregister:self];
...
}
Where the de-registration from the Post Office Service is an asynchronous operation, even if it's not self evident from the interface as there's no block or function passed to the postOfficeService.
The internal implementation of postOfficeService's -deregister method is something like that
// -deregister:(id)formerSubscriber implementation
//some trivial checks here
// deregister former subscriber
dispatch_asynch(_serialQueue, ^{
[self.subcribers removeObject:formerSubscriber];
});
...
The container, self.subscribers, does perfectly its job and contains only weak references. I.e. it is a NSHashTable.
As long as the deregistration method got called within the dealloc method, I keep on getting a crash while postOfficeService is trying to remove the former subscribers from its list inside that asynch block, which is used for thread safety purposes I guess.
Adding a breakpoint on [self.subscribers removeObject:formerSubscriber], it's possible to notice that the formerSubscriber object is always a NSZombieObject. That's the reason for crashes.
I know that it's possible to get thread safety for deregister method without incurring in this problem - I figure it should be enough use the dispatch_synch in lieu of the dispatch_asynch version
I think this is one of the reason why asynchronous methods shouldn't be called within dealloc methods.
But the question is how's possible to constantly get NSZombie objects even if we are in an ARC environment and the container objects is a NSHashTable (so it should be working I guess)?
The rule is: When dealloc is called, the object will be gone once dealloc returns to its caller (whoever called release when the reference count was 0), and nothing is going to prevent this.
Before ARC, you might have tried to retain an object inside dealloc - doesn't help; once dealloc is called the object will go (and dealloc will be called only once for it, in case you do a retain / release inside dealloc). ARC does the same, just automatically.
Using ARC doesn't means all your memory problem magically disappeared.
What happened is
[obj release] called by ARC
[obj dealloc]
[obj.postOfficeService deregister:obj]
[obj retain] - sorry you can't cancel the deallocation process
dispatch_async
free(obj) - from this point, obj is a zombie
GCD scheduling tasks
dispatch_async execute task
use obj - crash
The correct solution is use dispatch_sync to make sure you not trying to use object after it is deallocated. (be careful about dead lock)
Don't call asynchronous cleanup methods from dealloc. It's just not a good idea. Your -deregister should be synchronous.
NSHashTable stores pointers - it's the equivalent of __unsafe_unretained or assign - UNLESS it was created using +weakObjectsHashTable or the equivalent set of options (NSHashTableZeroingWeakMemory and NSPointerFunctionsObjectPersonality). If it was not created that way, it is quite likely you will have values pointing to zombie objects.
The question of "why am I getting zombies" is best answered by profiling your application with the Zombies template in Instruments and stimulating the required behavior.
I agree with the others that you should probably avoid asynchronous cleanup in your -dealloc method. However, it may be possible to fix this by making the parameter to -deregister: __unsafe_unretained. That method would then have to treat the pointer purely as a opaque value. It must not dereference it or message it. Unfortunately, you don't control the implementation of NSHashTable and can't guarantee that. Even if NSHashTable could be relied upon, the interface of -removeObject: takes an implicitly strong object pointer, so ARC might retain the pointer when it's copied from the unsafe unretained pointer.
You might use the C function API for hash tables (e.g. NSHashRemove()) as suggested in the overview for the NSHashTable class.
I am new to IOS development and I have started to learn objective c to program towards IOS 7. and as I know, it is way easier to code now than it has been before because of the Automatic reference counting.
there are a couple of things I do not understand . in MAIN method we have the autoreleasepool block, so my first question is that in order to enable ARC , the code has to be inside this block? if no, then what is the difference between the code that is inside autoreleasepool and the rest those aren't?
my second question is that when I am writing my IPHONE programs , I have bunch of classes and non of those codes are inside "autoreleasepool" , only the code inside the MAIN method.
int main(int argc, char * argv[])
{
#autoreleasepool {
return UIApplicationMain(argc, argv, nil,
NSStringFromClass([HomepwnerAppDelegate class]));
}
}
so , does this mean that this block somehow magically gets applied to all lines of code inside any other classes of the same program?
My last question is that whether with ARC or without it, if we had a declared pointer variable inside a method, does the object gets released/destroyed when the method returns/exit?
assume we have a method like this :
- (void)doSomething {
NSMutableArray *allItems = [[NSMutableArray alloc] init];
NSString *myString = #"sample string";
[allItems addObject:myString]
}
then when we call this method and it exits, what would happen to those local variables defined inside the method ? is there any difference in the outcome if we are using ARC or not ? (Object are still in the memory or not)
Autorelease pools predate ARC by about 15 years. Cocoa uses a reference-counting memory management scheme, where (conceptually, at least) objects are created with a reference count of 1, retain increases it by 1 and release decreases the count by 1, and the object is destroyed when the count gets to 0.
A problem with this scheme is that it makes returning an object kind of awkward, because you can't release the object before you return it — if you did, it might be destroyed before the other method got to use it — but you don't want to require the other method to release the object either. This is where autorelease pools come in. An autorelease pool lets you hand an object to it, and it promises to release the object for you later. Under manual retain/release (the way we used to do things before ARC), you did this by sending autorelease to an object.
OK, so then ARC comes into the picture. The only thing that really changes with ARC is that you aren't the one writing retain, release and autorelease anymore — the compiler inserts them for you. But you still need an autorelease pool for autoreleased object to go into.
As for your second question:
My last question is that whether with ARC or without it, if we had a declared pointer variable inside a method, does the object gets released/destroyed when the method returns/exit?
assume we have a method like this :
- (void)doSomething {
NSMutableArray *allItems = [[NSMutableArray alloc] init];
NSString *myString = #"sample string";
[allItems addObject:myString]
}
then when we call this method and it exits, what would happen to those local variables defined inside the method ? is there any difference in the outcome if we are using ARC or not ?
If you're using ARC, the compiler will release any objects referenced by local variables. If you're not using ARC, you'd need write [allItems release] yourself, because the variable going out of scope does not magically cause the object it references to be released.
new to IOS development
Best not to worry, Automatic means that you mostly concentrate on other things ^)
does this mean that this block somehow magically gets applied to all lines of code inside any other classes of the same program
Yes. You're in main function, so all the code that is executed has to be inside this function - your app will terminate once it ends. Unless you create a separate thread, but it's hard to do that by accident.
the code has to be inside this block
As said above, all of your code on main thread will execute within this block.
what would happen to those local variables defined inside the method
You're guaranteed that they will be destroyed before returning.
in MAIN method we have the autoreleasepool block, so my first question is that in order to enable ARC, the code has to be inside this block? if no, then what is the difference between the code that is inside autoreleasepool and the rest those aren't?
ARC is enabled by corresponding Objective-C compiler setting. If you create a new project in the latest version of Xcode it will be enabled by default.
The #autorelease keyword places code inside the curly brackets into autorelease pool scope. Autorelease pools are used both with ARC and manual memory management.
my second question is that when I am writing my IPHONE programs , I have bunch of classes and non of those codes are inside "autoreleasepool" , only the code inside the MAIN method.
iOS applications are event based. Main thread starts event loop when you call UIApplicationMain function processing touch events, notifications etc. This event loop has its own autorelease pool that autoreleases objects at the end of the loop iteration. This autorelease pool has nothing to do with the autorelease pool you see in main function.
My last question is that whether with ARC or without it, if we had a declared pointer variable inside a method, does the object gets released/destroyed when the method returns/exit?
If you use ARC the objects will be released (unless you return a reference to an object from the method). In MMR you would need to manually send release message to destroy the objects.
I read this:
If you ever create a secondary thread in your application, you need to provide it with its own autorelease pool. Autorelease pools and the objects they contain are discussed further in
in the iOS 5 Developer cookbook.
I'm compiling with ARC. I have been creating many background threads, and it seems that I am doing fine. None of my background threads are long-running. Will all those objects ever be released by say, the main thread's autorelease pool? Or what?
This is what I do to call background thread:
+(void)doBackground:(void (^)())block
{
//DISPATCH_QUEUE_PRIORITY_HIGH
//dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND,0), ^{
dispatch_async(dispatch_get_global_queue(-2,0), ^{
block();
});
}
Should I change that to
+(void)doBackground:(void (^)())block
{
//DISPATCH_QUEUE_PRIORITY_HIGH
//dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND,0), ^{
dispatch_async(dispatch_get_global_queue(-2,0), ^{
#autoreleasepool{
block();
}
});
}
Consider it at minimum a programmer error if you do not create an autorelease pool for your new thread. Whether that's fatal to your program is defined by your program's implementation. The classic problem is leaked objects and consequently objects' dealloc which is never executed (could be fatal).
The modern way to create an autorelease pool under ARC is:
void MONThreadsEntry() { // << entry is e.g. a function or method
#autoreleasepool {
...do your work here...
}
}
In more detail, autorelease pools behave as thread-local stacks -- you can push and pop, but there should always be one in place before anything on that thread is autoreleased. Autorelease messages are not transferred from one thread to another.
You may not see issues (e.g. in the console or leaks) if your idea of "creating a thread" is using a higher level asynchronous mechanism, such as using an NSOperationQueue, or if the underlying implementation creates a secondary thread and its own autorelease pool.
Anyways, rather than guessing when to create autorelease pools, just learn where you need to create them and when you should create them. It's all well-defined -- there is no need for guesswork, and there is no need to fear creating them.
Similarly, you will never need to create an autorelease pool for your thread if you are using lower level abstractions, and never autorelease objects on that thread. For example, pthreads and pure C implementations won't need to bother with autorelease pools (unless some API you use assumes they are in place).
Even the main thread in a Cocoa app needs an autorelease pool -- it's just typically not something you write because it exists in the project templates.
Update -- Dispatch Queues
In response to updated question: Yes, you should still create autorelease pools for your programs which run under dispatch queues -- note that with a dispatch queue, you're not creating threads so this is quite a different question from the original question. The reason: Although dispatch queues do manage autorelease pools, no guarantee is made regarding the time/point they are emptied. That is to say, your objects would be released (at some point), but you should also create autorelease pools in this context because the implementation could (in theory) drain the pool every 10,000 blocks it runs, or approximately every day. So in this context, it's really only fatal in scenarios such as when you end up consuming too much memory, or when your programs expects that its objects will be destroyed in some determined fashion -- for example, you could be loading or processing images in the background and wind up consuming a ton of memory if the life of those images is extended unexpectedly because of the autorelease pools. The other example is shared resources or global objects, where you could respond to notifications or introduce race conditions because your "block local" objects may live a lot longer than you expect. Also remember that the implementation/frequency is free to change as its implementors see fit.
Seems now autorelease pool is created for new threads automatically. Don't know when this has changed and why documentation states opposite but that's it.