I'm trying to simulate the behaviour of a component under heavy memory load.
The main idea is to allocate a bunch of memory, keep it resident, then work with my component, however no matter what I do, the memory I allocate on purpose seem do disappear somehow.
After a first test using a strong NSArray of NSData chunks (1Mb each), I tried with a rougher but lower level approach (to prevent some ARC magic to happen):
// somewhere after the includes
static char *foo[MAX_CHUNKS];
[...]
// this function is actually called, I checked :)
- (void) allocateMemoryChunks:(int) chunks ofSize:(long) bytes {
for(int i = 0; i < chunks; ++i) {
foo[i] = (char *) malloc(bytes);
}
}
No matter what I set for chunks and bytes, as long as the memory can be allocated, it 'disappears' from the memory count in Xcode and leaves me with the usual meager 2Mb consumed by the application (I'm talking of allocations in the range of 400 chunks of 1024 * 1024 bytes, or 1 chunk of 400 * 1024 * 1024 bytes, for what's worth...).
The memory is clearly outside the reach of ARC (I'm using plain C mallocs!) yet, I can't see it (note that the same happened when I used NSData and NSArray to hold the data).
Where does that memory go? What am I forgetting?
Thank you!
Addendum
For the records, the ARC version of the allocateMemoryChunks is:
- (void) allocateMemoryChunks2:(int) nchunks ofSize:(long) bytes {
self.allocated = [NSMutableArray arrayWithCapacity:nchunks];
for(int i = 0; i < nchunks; ++i) {
char *tmpraw = (char *) malloc(bytes);
NSData *tmp = [NSData dataWithBytes:tmpraw length:bytes];
[self.allocated addObject:tmp];
free(tmpraw);
}
}
with allocated declared as:
#property (nonatomic, strong) NSMutableArray *allocated;
Related
I have started out with Objective-C recently and trying to understand the memory management. I came across a peculiar problem yesterday which was causing a memory leak of over 150MB! I traced it down to a piece of code that was creating NSArray literal of NSNumbers. I managed to solve the issue using NSMutableArray (using addObject), but I haven't been able to grasp the concept as to why one method works and the other doesn't. I would love for someone with a better understanding of the memory management to explain the concept behind it so that I can avoid such mistakes in the future.
To better illustrate my question, let me also provide the code snippets. So for instance, I have a function that creates a NSArray of few NSNumbers and returns it and it gets called many times:
-(NSArray *)getNSNumberArray
{
float num1 = 23.56;
float num2 = 75.34;
float num3 = 223.56;
NSArray *numArray = #[[NSNumber numberWithFloat:num1], [NSNumber numberWithFloat:num2], [NSNumber numberWithFloat:num3]];
return numArray;
}
-(void)causeMemoryLeak
{
for (int i = 0; i < 2000000; i++)
{
NSArray *recieverArray = [self getNSNumberArray];
}
}
This results in memory being occupied to be more than 200MB on an iphone6. However if I change my function to be:
-(NSArray *)getNSNumberArray
{
float num1 = 23.56;
float num2 = 75.34;
float num3 = 223.56;
NSMutableArray *numMutableArray = [[NSMutableArray alloc] init];
NSNumber *nsNumber1 = [NSNumber numberWithFloat:num1];
NSNumber *nsNumber2 = [NSNumber numberWithFloat:num2];
NSNumber *nsNumber3 = [NSNumber numberWithFloat:num3];
[numMutableArray addObject:nsNumber1];
[numMutableArray addObject:nsNumber2];
[numMutableArray addObject:nsNumber3];
return numMutableArray;
}
-(void)causeMemoryLeak
{
for (int i = 0; i < 2000000; i++)
{
NSArray *recieverArray = [self getNSNumberArray];
}
}
This does not cause any memory issues.
Maybe I am missing something very obvious, but still cant figure out the reason behind this. Would really appreciate the help on this. There could also be better ways of doing it, and such answers are welcome but basically I am looking for an explanation behind it.
Many thanks in advance!
Attaching links to screenshots showing the memory allocation on device (iphone 6) (I can not attach images here as of now, so have to provide links)
Approach 1 Memory Allocation (memory is retained and not freed up): https://drive.google.com/open?id=0B-a9WJSBuIL4bTR5RTVuaWpqYkE&authuser=0
Approach 2 Memory Allocation (memory is freed up and there is no surge in memory allocation as well): https://drive.google.com/open?id=0B-a9WJSBuIL4QzQzbGYyQzZDdW8&authuser=0
I tried your code because I really didn't found any reason for memory leak and there is no memory leak.
When you use the regular array, the device runs all the code, and releases the memory only at the end. This causing the heap to alloc 250 MB, and release them at the end.
If you use the mutable array, the device releases the previous array before creating the new one, so there is no point with to many allocation.
I guess that the different may be caused by run time complexity of the called function in the loop.
As you are saying, only one approach is causing surge in memory, it does not seem so. According to the documentation, NSMutableArray uses slightly more memory because it can change size, it can't store the contents inside the object and must store a pointer to out of line storage as well as the extra malloc node for the storage.
In both the approaches, memory is freed up instantly, there is no issue with me.
Update 1 :
I have noticed that if you use the array, for example if you log the array, the memory is freed up instantly in your first approach. It is happening as you do not use that array. It solves your problem.Try this code :
-(NSArray *)getNSNumberArray
{
float num1 = 23.56;
float num2 = 75.34;
float num3 = 223.56;
NSArray *numArray = #[[NSNumber numberWithFloat:num1], [NSNumber numberWithFloat:num2], [NSNumber numberWithFloat:num3]];
return numArray;
}
-(void)causeMemoryLeak
{
for (int i = 0; i < 2000000; i++)
{
NSArray *recieverArray = [self getNSNumberArray];
NSLog(#"%#", recieverArray);
}
}
First, you need to read up on "autorelease" and "autorelease pools". Objects often are kept inside an autorelease pool and only disappear when the autorelease pool disappears. So a loop iterating two million times without using an autorelease pool is a bad, bad idea. You don't actually have a memory leak, the objects will all be released some time after the method returns.
Most methods returning objects return autoreleased objects, but the result of [[xxx alloc] init] is not autoreleased, so there you have the difference. But also ARC is quite clever, and if a method returns an autoreleased object and the caller immediately retains the object, the object will not actually be added to the autorelease pool. That's what happened when the object was passed to NSLog; in order to do that, it had to be retained first. No autorelease pool. In your code, the object was never used, so that optimisation didn't happened.
Of course your code is really useless, so you were punished for writing useless code.
My original project was leaking so I searched for the leak. When I found it I created a simple new project.
The project uses ARC and the only code I added is the following.
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
int elements = 10000000;
//memory usage 5,2 MB
NSMutableArray *array = [NSMutableArray arrayWithCapacity:elements];
//memory usage 81,7 MB
for (int i = 0; i < elements; i++) {
[array addObject:[NSObject new]];
}
//memory usage 234,3 MB
[array removeAllObjects];
//memory usage 234,3 MB
array = nil;
//memory usage 159,5 MB
}
After calling [array removeAllObjects] all NSObjects in the array should be deallocated and the memory usage should be 81,7 MB again.
What am I doing wrong?
Here
NSMutableArray *array = [NSMutableArray arrayWithCapacity:elements];
you are creating autoreleased object (autorelease pool).
Many programs create temporary objects that are autoreleased. These
objects add to the program’s memory footprint until the end of the
block. In many situations, allowing temporary objects to accumulate
until the end of the current event-loop iteration does not result in
excessive overhead; in some situations, however, you may create a
large number of temporary objects that add substantially to memory
footprint and that you want to dispose of more quickly. In these
latter cases, you can create your own autorelease pool block. At the
end of the block, the temporary objects are released, which typically
results in their deallocation thereby reducing the program’s memory
footprint
Wrap with #autoreleasepool {} method [NSMutableArray arrayWithCapacity:elements]:
NSMutableArray *array;
#autoreleasepool {
array = [NSMutableArray arrayWithCapacity:elements];
// [NSMutableArray arrayWithCapacity:] creates object with retainCount == 1
// and pushes it to autorelease pool
// array = some_object; usually (and in this case also) is transformed by ARC to
// [array release]; [some_object retain]; array = some_object;
// so here array will have retainCount == 2 and 1 reference in autorelease pool
} // here autorelease pool will call `release` for its objects.
// here array will have retainCount == 1
or change it to
NSMutableArray *array = [[NSMutableArray alloc] initWithCapacity:elements];
You've been bitten by the dreaded autorelease pool. Essentially to make MRC (manual reference counting) manageable by people instead of releasing an object immediately it can be handed to the autorelease pool (an instance of NSAutoreleasePool, it's documentation gives more details) which will retain the object until the pool is later drained. ARC (automatic reference counting) could be designed so the autorelease machinery is not needed, but to maintain compatibility with MRC is remains.
The pool automatically drained at the end of a run loop cycle - i.e. when the application has finished processing an event. However if an application creates a lot of temporary objects and then discards them is some localised part of the program then a using a local autorelease pool can drastically reduce the maximum memory use. It is not that such temporary objects will not be release, just that they will live far longer than needed. A local pool can be create with the #autoreleasepool { ... } construct.
You can see the effect in your example by wrapping the whole of the body of applicationDidFinishLaunching:
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
#autoreleasepool
{
...
}
}
and stepping through with the debugger.
In your real code you need to work back from the point which is producing lots of temporary objects to locate a suitable point to add an autorelease pool.
HTH.
Addendum
It is not the objects in your array that are not getting released when you think they should , you can test this by using a simple class which counts initialisations and deallocations, e.g.:
#interface TestObject : NSObject
+ (void) showCounts;
#end
#implementation TestObject
static uint64_t initCount = 0, deallocCount = 0;
- (id) init
{
self = [super init];
if(self) initCount++;
return self;
}
- (void) dealloc
{
deallocCount++;
}
+ (void) showCounts
{
NSLog(#"init: %llu | dealloc: %llu", initCount, deallocCount);
initCount = deallocCount = 0;
}
#end
Use this instead of NSObject and call showCounts after you are done with your test - try with/without autorelease, etc.
Your memory is always getting released, it is just the time at which it is release that is the issue. Some objects end up in an autorelease pool, either the default one which is emptied once per event, or a local one.
Unless you create a lot of temporary objects in response to a single event you normally won't see an issue. Consider whether you are chasing a real issue for your application here. If you are among the things to try to alleviate the problem are:
Avoid uses of convenience constructors of the form <name>WithX... which are shorthands for [[[C alloc] initWithX...] autorelease]. On many, but not all, occasions the compiler can remove such objects from the autorelease pool just after the convenience constructor returns (and your case appears to be one in which is can fail). The better route is to use alloc/init, new (shorthand for alloc/init) or, if provided, newWithX... (shorthand for alloc/initWithX...). Try these options with your example and see the differences in when (not if) the memory is released.
Well placed #autoreleasepool blocks.
HTH
I'm trying to figure out if there is a general way to essentially wipe or encrypt the memory associated with NSObjects. I don't really care if it's a bit cumbersome, I just want to make sure it really can't be read.
For example if I have an
NSString* str = [[NSString alloc] initWithFormat:#"TESTING"];
it's relatively simple to do
unsigned char* strPtr = (unsigned char *) CFStringGetCStringPtr
((CFStringRef) str, CFStringGetSystemEncoding());
memset(strPtr, 0, [str length]);
And I can do similar things for NSData. But I would really like to have something more general.
I have looked into, with no luck:
Zones, which seem like they are no longer used.
Creating a parent class of NSObject and casting everything to that and keeping track of the memory regions. This has just been a complete pain though.
Encrypting the actual contents of the data and forcing a decrypt on access. This gets back to the problem of knowing something about every single type.
Encrypting then decrypting memory regions, can't find a way to reliably know where in memory a general object is.
Any hints or ideas would be greatly appreciated.
You can add a category to NSObject that uses reflection via the runtime API. This can be used to clear ivars/properties, even properties declared as readonly.
One drawback is that it does not clear certain properties, such as the frame of a UIView, and I'm not sure why it misses that.
NSObject+Scrub.h:
#interface NSObject (Scrub)
- (void) scrub;
#end
NSObject+Scrub.m:
#import "NSObject+Scrub.h"
#import <objc/runtime.h>
#implementation NSObject (Scrub)
- (void) scrub
{
Class myClass = [self class];
unsigned int count;
//Scrub the Ivars
Ivar *ivars = class_copyIvarList(myClass, &count);
for (int i = 0; i < count ; i++) {
Ivar ivar = ivars[i];
object_setIvar(self, ivar, nil);
}
free(ivars);
}
#end
Good luck!
I am still new to objective-c and I need some insight about variable declaration.
The point of my program is to read a message delivered in several NSData parts. Whenever the program receive a NSData it adds it to the already partially complete message as unsigned chars. This means that the message buffer must be define as a property of the viewController:
#property unsigned char* receiveBuf;
#property int bufSize;
At that point the program has a NSData object named data, and treat it like this:
int len = [data length];
unsigned char* tempBytebuf = (unsigned char*)[data bytes];
for(int i = 0; i < len; i++){ //read all bytes
if (tempBytebuf[i] == 0x7E) { //this char means start of package
_bufSize = 0;
}
else if (tempBytebuf[i] == 0x7D) { //this char means end of package
[self readMsgByte:_receiveBuf :_bufSize];
}
else { //any other char is to be put in the message
_receiveBuf[_bufSize++] = tempBytebuf[i]; //Error happens here
}
}
if I proceed like this it will result in an error EXC_BAD_ACCESS.
If I could tell the program to reserve space for the buffer it would solve the problem:
#property unsigned char receiveBuf[256];
but it seems I can't do this with #properties.
Is there a way of assigning that space in the ViewDidLoad() for example or somewhere else?
Thanks for your support!
EDIT 1:
It seems I just found a solution in some of my previous codes, I should have declare my char table in the implementation instead.
#implementation ViewController
{
unsigned char msgBuf[256];
}
Still if someone could tell me the real difference between the #property and implementation space for variable declaration that would prevent me from doing other mistakes like this.
Thanks a lot.
If you needed to declare that as unsigned char*, then malloc/calloc/realloc/free are your friends.
In reality, making that ivar NSMutableData and using APIs like -appendBytes:length: should be all that's necessary.
Use NSMutableData:
#property (strong) NSMutableData *recievedData;
It provides a nice wrapper around an arbitrary piece of memory who's contents and length can change at any time - and most importantly it fits in perfectly with the objective-c runtime and memory management.
Just create an empty (0 length) data object like this:
self.receivedData = [[NSMutableData alloc] init];
And then at any time you can do:
[self.recivedData appendBytes:bytes length:length];
Or:
[self.recievedBytes appendData:anotherNSDataObj];
From there you can read it with the getBytes: method.
There are other ways to do it, by declaring lower level instance variables, but #property is the modern approach. The older style of instance variable is able to what you tried, but we are moving away from them and it will probably be deprecated soon.
I have a problem with an existing application for iPad. It is quite sophisticated application written partly native (downloading, local caching and so on), partly using Sencha Touch for UI (without PhoneGap).
Application runs fine on both iPad 1 and iPad 2 under iOS4. But with public release of iOS 5 aproblem appears for iPad 1 users. Application crashes after several seconds of work. Some guys write that instead of 78 there are only 30 megs of memory available for iPad 1 under iOS 5. And the other blackbox is the UIWebView. Nobody knows its internal limitations.
I have a few ideas how to optimize JavaScript and HTML aspect. For example I've optimized structure and reduce memory allocations in scripts as I could. I've also replaced IMG tags with DIV + background-image. But crashes remains.
Below is the insane solution to the problem.
Solution
There are memory warnings but I can release neither something native nor anything inside JavaScript, I thought. So I decided to do ridiculous thing - to allocate the dummy array of several megabytes and release it on memory warning. This is nuts but it works for me: I could allocate up to 100 megs of RAM on start with simple malloc()
// Member variable
void* dummy;
// - (void)init
dummy = malloc(100000000L);
and free it with free() when system asks.
- (void)didReceiveMemoryWarning {
[super didReceiveMemoryWarning];
if (dummy) {
free(dummy);
dummy = nil;
}
}
This time app works longer. So far so good... Next step - make possible repeated memory freeing.
// Member variable
NSMutableArray* _memoryWorkaround;
// - (void)init
int chunkCount = 100;
int chunkSize = 1L * 1024L * 1024L; // 1 megabyte
_memoryWorkaround = [[NSMutableArray alloc] initWithCapacity:chunkCount];
for (int i = 0; i < chunkCount; i++)
[_memoryWorkaround addObject:[NSValue valueWithPointer:malloc(chunkSize)]];
Here it is - 100 chunks of memory by 1 megabyte allocated. This parameters is subject to revise.
Now we can free up as much memory as needed:
- (void)didReceiveMemoryWarning {
[super didReceiveMemoryWarning];
if ([_memoryWorkaround count]) {
NSValue* val = [_memoryWorkaround objectAtIndex:0];
free([val pointerValue]);
[_memoryWorkaround removeObject:val];
}
}
On exit free remaining:
- (void)dealloc {
if ([_memoryWorkaround count]) {
for (NSValue* val in _memoryWorkaround)
free([val pointerValue]);
[_memoryWorkaround removeAllObjects];
}
[_memoryWorkaround release];
[super dealloc];
}
The last thing to do - is to fill buffer back to chunkCount with NSTimer one by one block.
Seems crazy I know. Is better solution exists?