What is the difference between
+ (instancetype)dataWithBytes:(const void *)bytes length:(NSUInteger)length;
and
+ (instancetype)dataWithBytesNoCopy:(void *)bytes length:(NSUInteger)length;
Also,
+ (instancetype)dataWithBytesNoCopy:(void *)bytes length:(NSUInteger)length freeWhenDone:(BOOL)b;
if b == YES, will it free the bytes automatically after converted to data?
I am working on an app and almost finished it. But the last problem is it crashes with memory error when it runs on device. It only crashes when on device, but in simulator it is perfect.
"malloc: * error for object 0x17415d0c0: Invalid pointer dequeued from free list * set a breakpoint in malloc_error_break to debug";
I have been working on this issue for several days:
iOS - My app crashes with memory Error if only runs on Device
But finally I found the problem, inside my Encryption and Decryption function, I have this:
Byte *buffer = (Byte*)malloc(asciiDataLength);
After I process with buffer, I convert it to NSData:
NSData *plainData = [NSData dataWithBytesNoCopy:buffer length:asciiDataLength freeWhenDone:YES];
This code caused my app to crash continuously, I changed it to
NSData *plainData = [NSData dataWithBytes:buffer length:asciiDataLength];
free(buffer);
Then my app never crash again.
So, I have to free the Byte by myself, ARC will not free it for me.
+ dataWithBytes:length::
Creates and returns a data object containing a given number of bytes copied from a given buffer.
+ dataWithBytesNoCopy:length::
Creates and returns a data object that holds length bytes from the buffer bytes.
dataWithBytes makes a copy of the buffer for the data, while the NoCopy version does not.
Important note: in the discussion section of dataWithBytesNoCopy:length::
The returned object takes ownership of the bytes pointer and frees it on deallocation. Therefore, bytes must point to a memory block allocated with malloc.
This means that initialising with this method essentially hands ownership of the memory to the NSData object, which will release it with free once it is done. If you try to initialise it with memory that you didn't allocate with malloc, your app will crash when the data object is deallocated.
dataWithBytesNoCopy is useful for when you get the bytes in a buffer from somewhere else, and are ready to hand them over to the NSData object, and won't use them yourself again outside of that.
If you want to initialise the data with memory you manage yourself, use + dataWithBytesNoCopy:length:freeWhenDone:. This is useful if the buffer will be stored somewhere persistently, and not changed or released.
However, if you are not sure how to correctly manage this memory manually, it is better to use dataWithBytes. The other methods are present for performance reasons, as avoiding copying large chunks of data can save a lot of time, but if you aren't sure how to use them, it's probably best not to — an app that doesn't crash is preferable to an app that crashes quickly.
[[NSData alloc] initWithBytes:buffer length:buflength] create a data object containing buflength bytes copied from the buffer bytes.
[NSData dataWithBytesNoCopy:buffer length:buflength] creates a data object that holds buflength bytes from the buffer bytes. The returned object takes ownership of the buffer pointer and frees it on deallocation. Therefore, buffer must point to a memory block allocated with malloc.
Related
I'm working on a NSData extension that encrypts data with a key, as shown below. I'm not too savvy with Objective-C, but would like to use it for this Cordova plugin, instead of requiring another plugin to bridge Swift files.
I'm curious if I need to do any work to ensure that all clean-up in my methods is happening, so that each time this method is called, there are no leaks.
When extending an NS object, does one need to wrap their methods in #autoreleasepool {}?
This is the method that encrypts data (NSData+AES256Encrypt.m):
- (NSData *)AES256EncryptWithKey:(NSString *)key {
// 'key' should be 32 bytes for AES256, will be null-padded otherwise
char keyPtr[kCCKeySizeAES256 + 1]; // room for terminator (unused)
bzero(keyPtr, sizeof(keyPtr)); // fill with zeros for padding
// Get key data
[key getCString:keyPtr maxLength:sizeof(keyPtr) encoding:NSUTF8StringEncoding];
NSUInteger dataLength = [self length];
size_t bufferSize = dataLength + kCCBlockSizeAES128;
void *buffer = malloc(bufferSize);
size_t numBytesEncrypted = 0;
CCCryptorStatus cryptStatus = CCCrypt(kCCEncrypt, kCCAlgorithmAES128,
kCCOptionPKCS7Padding, keyPtr,
kCCKeySizeAES256, NULL, [self bytes],
dataLength, buffer, bufferSize, &numBytesEncrypted);
if(cryptStatus == kCCSuccess) {
return [NSData dataWithBytesNoCopy:buffer length:numBytesEncrypted];
}
free(buffer);
return nil;
}
It is used in conjunction with NSString+AES256Encrypt.m:
- (NSString *)AES256EncryptWithKey:(NSString *)key {
NSData *plainData = [self dataUsingEncoding:NSUTF8StringEncoding];
NSData *encryptedData = [plainData AES256EncryptWithKey:key];
NSString *encryptedString = [encryptedData base64Encoding];
return encryptedString;
}
The line that concerns me is free(buffer) used in the first method posted above, called if cryptStatus is false (meaning it failed to encrypt). I also notice that method dataWithBytesNoCopy has a parameter freeWhenDone which:
If YES, the returned object takes ownership of the bytes pointer and frees it on deallocation.
But I'm not sure if it applies to my situation.
Thanks for any and all help.
I don't see any issues.
Autoreleasing is only for Objective-C objects. It basically delays the actual -release call a little bit by putting the object in an autorelease pool, which gets flushed after every iteration of the main run loop, calling -release on every object in the pool. Objects returned from methods are typically autoreleased, though ARC has a mechanism which can often avoid the overhead of the actual pool by figuring out that the value is only needed by the caller, and can keep track of the reference and just call -release there. In ARC mode, the compiler figures out for you when autorelease vs release is needed, and does not let you call those methods yourself.
Most of the time, you don't need your own autorelease pools, though if you are doing something in a loop where every iteration can create lots of temporary objects, you may want an autorelease_pool for every loop iteration so that memory does not get built up but rather gets cleaned up each time so the next iteration can re-use that memory. If you are writing a command-line program or some other tool which does not have its own Objective-C support, then yes you need an autorelease pool around the entry point at least.
C heap memory using malloc/free is outside the autorelease concept (which only pertains to the retain/release mechanism of NSObject). For every malloc(), you need to eventually call free() on that memory once it is no longer needed, otherwise it's a leak. The code above is correct -- there is a malloc(), and then either free() is called when returning nil, or the initWithBytesNoCopy: method is called (which is a special method which uses the passed-in bytes as the actual NSData storage, avoiding the overhead of a memory copy and further internal malloc, and will then call free() when the object itself is dealloced).
initWithBytesNoCopy:length: just calls -initWithBytesNoCopy:length:freeWhenDone: with a YES parameter for freeWhenDone, basically, per its documentation. You can call the longer method explicitly if you think it makes it more readable (as it does more clearly indicate you were aware of the free behavior), but it will work the same either way.
The NSData encrypt method really doesn't create any objects other than the one you are returning -- all of its code is more straight C code. So an autorelease pool would not help anything. The NSString encrypt method does create a few temporary objects, so if the amount of memory being encrypted is substantial, an autorelease pool around that may help if you have subsequent significant work (but be sure to have a strong reference to the object being returned outside the pool scope). And really, ARC will most likely figure out the temporary nature of most of those objects, and more efficiently deal with them than an autorelease pool anyways.
If you can profile your code using Instruments, you can see the memory usage of your program as it runs, and it would only be places with significant spikes that you would consider using a local autorelease pool.
what is the expected memory usage of NSKeyedArchiver during the encoding process ?
in encoding ~4MB of an object graph into an NSData object, Xcode reveals the process uses almost 4-5x the memory just in the process of encoding (which is eventually causing a crash with larger object graphs), but the final, encoded NSData object remains around ~4MB. A picture is attached below, where the massive spike occurs during the encoding process. Is this expected behavior?
I have some event from C++ written library which works in background thread:
virtual void OnData(const char* data)
{
NSLog(#"Here 'data' string is present %s", data);
#autoreleasepool {
NSString* sData= [NSString stringWithCString:data encoding:NSUTF8StringEncoding];
dispatch_async(dispatch_get_main_queue(), ^{
NSLog(#"Here _sometimes_ 'data'(%s) is nil (\0). But sData is always present %#", data, sData);
[callback OnData:sData];
});
};
}
And sometimes I have NULL(I suspect its garbage actually) in dispatch_async block in argument variable. But local NSString variable is always here. Why?
P.S. Do I actually must use #autoreleasepool in this situation?
You have no assurances about the lifespan of the buffer that const char *data was pointing to by the time the async block is performed. The data could be dangling pointer by that point (and should be assumed to be so). It's very dangerous to use C-style pointers in any asynchronous references or outside the context they were originally created.
You should either use memory managed objects (e.g. NSData, NSString, etc.) or, if you insist on using C-style pointers and need to reference this pointer in the asynchronous block, copy the data to your own buffer, use that buffer, and then free it when you're done using that buffer in your asynchronous routine. In this case, you have your sData, so just don't refer to data after that point, and you'll be fine.
P.S. You later ask whether you must use #autoreleasepool in this situation.
In short, in most cases, no additional autorelease pool is needed. Notably, when using Grand Central Dispatch (e.g. dispatch_async), it has its own autorelease pools, so you don't have to create one. And, when your main thread yield back to its run loop, again, it's pool is drained. In short, you only need manually created autorelease pools when instantiating your own NSThread objects.
Having said that, sometimes you will introduce autorelease pools if doing significant memory intensive operations prior to yielding back to the run loop. In that case, you'll add autorelease pools in order to reduce the peak memory usage of the app. But this would not appear to be one of those cases.
If you had something like this:
void CallOnData()
{
char *test = malloc(5 * sizeof(char));
strcpy(test, "test");
OnData(test);
free(test);
}
You should expect data to be "NULL" in the block.
And autorelease is not needed, assuming you're using ARC, which you should be.
I am getting NSData on my socket receiving function and I'm trying to copy that data in a tempbuffer of my audio class, I am using external type global variable to do so.
This is my code:
memcpy([recorder tempBuffer].mdata,(__bridger const void *)data,data.length);
Here recorder is my extern type global variable of audio class.
When control reaches this line of code an exception is thrown, what possibly be the mistake.
There are really three possibilities here:
[recorder tempBuffer].mdata is not a valid pointer. (What type is it, for instance? If it's a NSMutableData, you should be accessing its mutableBytes property.)
[recorder tempBuffer].mdata is not a valid pointer of sufficient size (data.length).
(__bridger const void *)data is not a valid pointer of sufficient size.
Of the three, I can guarantee that #3 needs addressing. A NSData is not itself the data you want, but an object wrapping the data you want. Instead of using a bridge here, you should be using data.bytes.
The other two, I can't help you with. I don't know what type mdata is or where it was allocated.
If the destination buffer is really a buffer you allocated with malloc or a uint8_t (or equivalent) buffer, you should:
Check to make sure that the destination buffer is big enough to hold the entire data contents.
Don't try to cast the NSData to a (void *) pointer, but rather use:
memcpy(destination, data.bytes, data.length);
If the NSData is not in a contiguous block (which in iOS 7 and later, it might not be), data.bytes will copy it to a contiguous buffer, which you can then use with memcpy.
Or better, you can avoid this redundant copy, by removing memcpy altogether:
[data getBytes:destination length:data.length];
This will, if the NSData is not in a contiguous block, avoid having data.bytes copy it to a contiguous buffer which you would then copy again with the memcpy.
Bottom line, NSData has a rich interface that should eliminate the need to use low-level memcpy calls yourself.
From the question, it's not clear what [recorder tempBuffer].mdata is and how you allocated it, so perhaps you can clarify. Hopefully that's not another NSData object that you're trying to copy into.
I'm parsing a JSON file on an iPad which has about 53 MB. The parsing is working fine, I'm using Yajlparser which is a SAX parser and have set it up like this:
NSData *data = [NSData dataWithContentsOfFile:path options:NSDataReadingMappedAlways|NSDataReadingUncached error:&parseError];
YAJLParser *parser = [[YAJLParser alloc] init];
parser.delegate = self;
[parser parse:data];
Everything worked fine until now, but the JSON-file became bigger and now I'm suddenly experiencing memory warnings on the iPad 2. It receives 4 Memory Warnings and then just crashes. On the iPad 3 it works flawlessly without any mem warnings.
I have started profiling it with Instruments and found a lot of CFNumber allocations (I have stopped Instruments after a couple of minutes, I had it run before until the crash and the CFNumber thing was at about 60 mb or more).
After opening the CFNumber detail, it showed up a huge list of allocations. One of them showed me the following:
and another one here:
So what am I doing wrong? And what does that number (e.g. 72.8% in the last image) stand for? I'm using ARC so I'm not doing any Release or Retain or whatever.
Thanks for your help.
Cheers
EDIT: I have already asked the question about how to parse such huge files here: iPad - Parsing an extremely huge json - File (between 50 and 100 mb)
So the parsing itself seems to be fine.
See Apple's Core Data documentation on Efficiently Importing Data, particularly "Reducing Peak Memory Footprint".
You will need to make sure you don't have too many new entities in memory at once, which involves saving and resetting your context at regular intervals while you parse the data, as well as using autorelease pools well.
The general sudo code would be something like this:
while (there is new data) {
#autoreleasepool {
importAnItem();
if (we have imported more than 100 items) {
[context save:...];
[context reset];
}
}
}
So basically, put an autorelease pool around your main loop or parsing code. Count how many NSManagedObject instances you have created, and periodically save and reset the managed object context to flush these out of memory. This should keep your memory footprint down. The number 100 is arbitrary and you might want to experiment with different values.
Because you are saving the context for each batch, you may want to import into a temporary copy of your store in case something goes wrong and leaves you with a partial import. When everything is finished you can overwrite the original store.
Try to use [self.managedObjectContext refreshObject:obj refreshChanges:NO] after certain amount of insert operations. This will turn NSManagedObjects into faults and free up some memory.
Apple Docs on provided methods