I am facing a strange crash where an instance of NSNumber seems to be deallocated although it persists in array. I have created a system to download multiple files from remote server and have a block to indicate progress (an average progress really). And the computation of the progress produces a crash. The crash is not consistent and happens "usually" at can occur at any point. [_NSProgressFractionTuple floatValue]: unrecognized selector sent to instance 0x17042ab80 leads me to believe the NSNumber is somehow deallocated and I fail to see how this is possible.
To give the full method code:
- (void)downloadFilesFromURLs:(NSArray<NSString *> *)urlPaths withProgress:(void (^)(float progress))progressBlock completion:(void (^)(NSError *error))completionBlock {
NSMutableArray *toLoad = [[NSMutableArray alloc] init];
for(NSString *path in urlPaths) {
if([self fileForURL:path] == nil) {
[toLoad addObject:path];
}
}
NSInteger itemsToLoad = toLoad.count;
if(itemsToLoad <= 0) {
if(completionBlock) {
completionBlock(nil);
}
return;
}
// Set progresses to zero
__block NSMutableArray *progresses = [[NSMutableArray alloc] init];
for(int i=0; i<itemsToLoad; i++) [progresses addObject:[[NSNumber alloc] initWithFloat:.0f]];
__block NSInteger requestsOut = itemsToLoad;
__block NSError *latestError = nil;
for(int i=0; i<itemsToLoad; i++) {
NSInteger index = i;
[self downloadFileFromURL:toLoad[index] named:nil withProgress:^(float progress) {
progresses[index] = [[NSNumber alloc] initWithFloat:progress];
if(progressBlock) {
float overallProgress = .0f;
for(NSNumber *number in [progresses copy]) {
overallProgress += number.floatValue;
}
progressBlock(overallProgress/itemsToLoad);
}
} completion:^(NSString *filePath, NSError *error) {
if(error) latestError = error;
requestsOut -= 1;
if(requestsOut <= 0) {
if(completionBlock) {
completionBlock(latestError);
}
}
}];
}
}
Code explanation:
So this method accepts an array of URLs. It then checks if some of the files were already downloaded and creates a new array which only contains URLs that need to be downloaded. If all files exist or no URLs are provided then the completion is called and the operation breaks.
Next I create a mutable array and fill it with NSNumber instances all having a zero value. I remember how many requests will be made and I create a placeholder for an error. I iterate through all the URLs and initialize requests where each will report a progress and completion and both of these are on a main thread.
So in progress block I access the array of progresses to assign the new values through indexing. I then compute an average progress and report overall progress to an input block.
The request completion decreases the number of requests counter and when that one falls to zero the input completion is called.
The situation:
It all works as expected, the whole procedure is correct. The given values are all valid and all the files are there on the server and are accessible. When the app does not crash it all works as it should.
But when it crashes it crashes in
for(NSNumber *number in [progresses copy]) {
overallProgress += number.floatValue;
}
and the crash is random but in any case the number.floatValue seems to be accessing a memory that it shouldn't.
I now have a solution where I replaced the progresses array with pure C pointer float *progresses = malloc(sizeof(float)*itemsToLoad); which is freed on completion. It seems to work but still, what am I missing here? What could be the cause of array with NSNumbers not working here?
Some additional info:
Memory is OK, this is writing directly into files and even if it didn't the overall file size is relatively small
Disk space is OK
I was using #(progress) syntax but changed it to explicit allocation in hopes of removing the issue
progresses does not need __block, I added it just in case
Completion does not get called before all the progresses get called and even if it did I see no reason to crash the app
Thank you!
NSMutableArray is not thread safe. So even though there is no explicit memory management issue, if NSMutableArray is accessed at the same time by two different thread bad things can happen. I believe that dispatching the withProgress block in a serial queue would solve the issue.
Related
I have an array of PFFiles I retrieve from parse. The PFFiles must be converted to images and I am trying to convert them in a loop, however;
The array of converted images must be in the same order of the array containing the PFFiles.
The problem is, is the loop runs and causes all of the blocks to trigger, then they are all loading at the same time thus the imageArray will result in a different order to the original array of PFFiles, because if one object finishes loading before the previous object, it will be added to the array before the previous one, making it go out of order.
Instead, I would like a loop where it loops through every object in the array, however it doesn't loop onto the next object until the getDataInBackgroundBlock has finished loading and the current object has been added to the new array.
-(void)organizePhotos {
for (PFFile *picture in pictureArray) {
//This block loads, after the next index in the loop is called, causing the array to be out of order.
[picture getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
[imageArray addObject:[UIImage imageWithData:data]];
[self savePhotos];
}];
}
}
Above is my code. Is there anyway I can make the loop wait until the getDatanBackgroundWithBlock finishes loading?
The problem is, is the loop runs and causes all of the blocks to trigger, then they are all loading at the same time
That is not a problem, it is good - the reason why the call is asynchronous is it can take an arbitrarily long time to complete. If you've got multiple downloads to do then doing then concurrently can be a big win.
thus the imageArray will result in a different order to the original array of PFFiles, because if one object finishes loading before the previous object, it will be added to the array before the previous one, making it go out of order.
This is the problem and can be addressed in a number of ways.
As you are using arrays, here is a simple array based solution: first create yourself an array of the right size and fill it with nulls to indicate the image hasn't yet arrived:
(all code typed directly into answer, treat as pseudo-code and expect some errors)
NSUInteger numberOfImages = pictureArray.length;
NSMutableArray *downloadedImages = [NSMutableArray arrayWithCapacity:numberOfImages];
// You cannot set a specific element if it is past the end of an array
// so pre-fill the array with nulls
NSUInteger count = numberOfImages;
while (count-- > 0)
[downloadedImages addObject:[NSNull null]];
Now you have your pre-filled array just modify your existing loop to write the downloaded image into the correct index:
for (NSUInteger ix = 0; ix < numberOfImages; ix++)
{
PFFile *picture = pictureArray[ix];
[picture getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
dispatch_async(dispatch_get_main_queue(),
^{ [imageArray replaceObjectAtIndex:ix
withObject:[UIImage imageWithData:data]
});
[self savePhotos];
}];
}
The use of dispatch_async here is to ensure there are not concurrent updates to the array.
If you wish to know when all images have been downloaded you can check for that within the dispatch_async block, e.g. it can increment a counter safely as it is running on the main thread and call a method/issue a notification/invoke a block when all the images have downloaded.
You are also possibly making things harder on yourself by using arrays, and trying to keep items in different arrays related by position. Dictionaries could save you some of the hassle, for example each of your PFFile objects presumably relates to a different URL, and a URL is a perfectly valid key for a dictionary. So you could do the following:
NSMutableDictionary *imageDict = [NSMutableDictionary new];
for (PFFile *picture in pictureArray) {
[picture getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
dispatch_async(dispatch_get_main_queue(),
^{ [imageDict setObject:[UIImage imageWithData:data] forKey:picture.URL];
};
[self savePhotos];
}];
}
And your can locate the image for a PFFile instance by looking up its URL in the dictionary - which will return nil if the image is not yet loaded.
There are other solutions, and you might want to look into making your code more asynchronous. Whatever you do trying to call asynchronous code synchronously is not a good idea and will impact the user experience.
HTH
Ideally, you should use a synchronous method; however the behavior you are asking for can be achieved using Grand Central Dispatch.
First, create a dispatch_group_t using dispatch_group_create(). Let's call it asyncGroup.
Then, before calling the async method, call dispatch_group_enter(asyncGroup). This increments the counter of the number of calls in the group.
Then, at the end of the async block, call dispatch_group_leave(asyncGroup) to decrement the counter of the number of calls in the group.
Finally, after calling the async method, call dispatch_group_wait(asyncGroup, timeout) to pause thread execution until the group counter reaches zero. Since you increment, make the call, and then wait, the loop will only continue when the async block has been run. Just ensure the timeout is longer than the operation will take.
You can make the code synchronous:
-(void)organizePhotos {
for (PFFile *picture in pictureArray) {
[imageArray addObject:[UIImage imageWithData:[picture getData]]];
[self savePhotos];
}
}
and run it in the background:
[self performSelectorInBackground:#selector(organizePhotos) withObject:nil];
You can use GCD approach which is to use dispatch_group. So, before you start an asynchronous task, call dispatch_group_enter, and then when the asynchronous task finishes, call dispatch_group_leave, and you can then create a dispatch_group_notify which will be called when the asynchronous tasks finish. You can marry this with a completion block pattern (which is a good idea for asynchronous methods, anyway):
Here is similar Question. Perhaps it may also be helpful:
How to wait for method that has completion block (all on main thread)?
You could do that using dispatch_async way, using semaphory, groups, but for your need better practice is using block to get response and do what you need.
Doing that:
-(void)organizePhotos:(void(^)(BOOL responseStatus))status {
for (PFFile *picture in pictureArray) {
//This block loads, after the next index in the loop is called, causing the array to be out of order.
[picture getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
[imageArray addObject:[UIImage imageWithData:data]];
status(YES);
}];
}
}
So you next at your call you could do that:
__weak typeof(self) weakSelf = self;
[self organizePhotos:^(BOOL responseStatus) {
if(responseStatus){
[weakSelf savePhotos];
}
}];
Or if you dont wan't create extra parameter response to your method you could do like this:
-(void)organizePhotos {
__weak typeof(self) weakSelf = self;
void (^ResponseBlock)(void) = ^{
[weakSelf savePhotos];
};
for (PFFile *picture in pictureArray) {
//This block loads, after the next index in the loop is called, causing the array to be out of order.
[picture getDataInBackgroundWithBlock:^(NSData *data, NSError *error) {
[imageArray addObject:[UIImage imageWithData:data]];
ResponseBlock();
}];
}
}
And another point that you are making wrong when calling "self" inside block, this could lead you to retain cycle and its a bad practice, look at my code how you should do.
I have the following requirement:
Given a hierarchical tree-like structure, I am performing a breadth-first-search to walk through the WHOLE dataset. The data is being provided by an API with a method : (makes a request to a server using AFNetworking, saves the result to Core Data and calls back the completion block on success with the stored entries)
-(void) getChildrenForNodeId:(NSNumber*)nodeId
completion:(void (^)(NSArray *nodes))completionBlock;
The method which a controller executes to fetch data:
-(void)getAllNodesWithCompletion:(void (^)(NSArray *nodes))completionBlock{
NSNumber *rootId = ...
[MyNetworkManager getChildrenForNodeId:rootId completion:^(NSArray *nodes){
for(Node *node in nodes){
[self iterateOverNode:node.nodeId];
}
//execute completionBlock with nodes fetched from database that contain all their children until the very last leaf
}];
}
Here is the problem:
-(void)iterateOverNode:(NSNumber*)nodeId {
NSMutableArray *elements = [NSMutableArray array];
[elements addObject:nodeId];
while ([elements count]) {
NSNumber *current = [elements objectAtIndex:0];
[MyNetworkManager getChildrenForNodeWithId:current completion:^(NSArray *nodes) {
/**
In order to continue with the loop the elements array must be updated. This can only happen once we have retrieved the children of the current node.
However since this is in a loop, all of these requests will be sent off at the same time, thus unable to properly keep looping.
*/
for(Node *node in nodes){
[elements addObject:node.nodeId];
}
[elements removeObjectAtIndex:0];
}];
}
}
Basically I need the result of the callback to control the flow of the while loop but I am not sure how to achieve it. My understanding is that the request to getChildrenForNodeWithId:completion: from within the while-loop should happen in a new thread in a SERIAL order so that another should commence after the first one has completed. I am not sure how to achieve this neither with NSOperation nor with GCD. Any help will be greatly appreciated.
What you need here is some recursion. This problem is tricky as we also need a way to track and detect the point at which we have explored every branch to a leaf node.
I'm not a expert with tree search algorithms, so some folks could probably improve on my answer here. Kick this off by calling it with the root node id. self.trackingArray is an NSMutableArray property with __block qualifier. Each time we start a request for a Node, we add it's nodeId into this array, and when it returns, we remove it's nodeId, and add the nodeIds of it's children. We can then know that when count of the tracking array reaches 0, every request made has returned, and has not added child ids to the array. Once you detect we are finished, you could call a saved block or a delegate method.
This solution does not include any error handling. If any request fails, it won't be retried, and all child nodes will be ignored.
- (void)getNodesRecursively:(NSNumber *)nodeId {
// Only do this once
if (self.trackingArray == nil) {
self.trackingArray = [NSMutableArray new];
[self.trackingArray addObject:nodeId];
}
__weak typeof(self) weakSelf = self;
[MyNetworkManager getChildrenForNodeWithId:nodeId completion:^(NSArray *nodes) {
[self.trackingArray removeObject:nodeId];
for (Node *node in nodes) {
[weakSelf.trackingArray addObject:node.nodeId];
[weakSelf getNodesRecursively:node.nodeId];
}
if (weakSelf.trackingArray.count == 0) {
// We are done.
// Reset the state of the trackingArray
self.trackingArray = nil;
// call a method here to notify who ever needs to know.
[weakSelf allNodesComplete];
}
}];
}
Your other methods would look something like this
-(void)getAllNodesWithCompletion:(void (^)(NSArray *nodes))completionBlock{
// Save the block to a property
self.completion = completionBlock;
// Call -getNodesRecursively with root id
[self getNodesRecursively:rootNodeId];
}
Then you could have a second method
- (void)allNodesComplete {
// Call saved block
// Completion should load nodes from core data as needed
self.completion();
}
I haven't tested the code, but does that approach seem sensible? I'm assuming we don't need to capture the here nodes, as they can be loaded from core data as required.
Currently I am trying to do some async and concurrent tasks, and I am using Azures blob to upload all the images, however the concern is that, for every blob I need to get a SASURL and then upload the images. Also the another side towards it is that I want to have all the operations of the images completed to be uploaded, and hence send a final upload to the database. Although I can send the operation to the database earlier, without having the confirmation of the images completed, but I just wanted to make sure, that the operation does gets completed.
Below is the code for the SASURL block.
- (void)storageServiceBlob:(NSArray*)images
{
StorageService *storageService = [StorageService getInstance];
NSLog(#"%#",[storageService containers]);
NSLog(#"%#",[storageService blobs]);
for (int i = 0; i < [images count]; i++) {
NSString *file_name = [images objectAtIndex:i];
NSString *result = [self imageName:file_name];
NSLog(#"Final: %#", result);
[storageService getSasUrlForNewBlob:result forContainer:#"misccontainer" withCompletion:^(NSString *sasUrl) {
NSLog(#"%#",sasUrl);
[self postBlobWithUrl:sasUrl Image:[images objectAtIndex:i]];
}];
}
}
I want to use gcd in group somehow to determine that after all the completion blocks is called in a group, it executes Post method. Is there anyway to do this in gcd?
There are many ways you could do this. Here's one:
- (void)storageServiceBlob:(NSArray *)imageFilenames
{
StorageService *storageService = [StorageService getInstance];
__block NSMutableSet *remainingImageFilenames = [NSMutableSet setWithArray:imageFilenames];
for (NSString *imageFilename in imageFilenames) {
NSString *imageName = [self imageNameForImageFilename:imageFilename];
[storageService getSasUrlForNewBlob:imageName forContainer:#"misccontainer" withCompletion:^(NSString *sasUrl) {
[self postBlobWithUrl:sasUrl imageFilename:imageFileName];
[remainingImageFilenames removeObject:imageFilename];
if ([remainingImageFilenames count] == 0) {
// you're done, do your thing
}
}];
}
}
A few tips:
Be careful with your naming. There seems to be some ambiguity there.
Generally, idiomatic method name parameters start with a lower-case letter, e.g. myMethodWithThis:andThat:, not MyMethodWithThis:AndThat:.
Fast enumeration, e.g. for (id obj in array) is your friend. Learn and use it.
You can shortcut [array objectAtIndex:1] as array[1].
If you have access to the queue that the requests are going in then you can issue a barrier block.
When you have an async queue a barrier block will sit and wait to be executed until all of the blocks issued before it have run.
If you don't have access to the queue then your best bet is to keep a count.
I've been having a memory leak in a upload speedtest function that has been recently converted to ARC. I believe I've adhered to the memory management guidelines for ARC.
The issue seems to be with the chuck of random data I create for the upload test. Its memory doesn't seem to get freed.
Here is where I create the upload data and ASIHTTPRequest object:
ASIHTTPRequest *request0 = [ASIHTTPRequest requestWithURL:uploadTestURL];
__weak ASIHTTPRequest *request = request0;
NSData *uploadData ;
if ([speedTier isEqualToString:#"Wifi"]) {
uploadData = [self createRandomNSDataOfSize:1000000];
}else
{
uploadData = [self createRandomNSDataOfSize:4000000];
}
[request appendPostData:uploadData];
The function that actually creates the data is:
NSMutableData* theData = [NSMutableData dataWithCapacity:size];
for( unsigned int i = 0 ; i < size/4 ; ++i )
{
u_int32_t randomBits = arc4random();
[theData appendBytes:(void*)&randomBits length:4];
}
return theData;
I then proceed to set up the block for setBytesSentBlock, where I manage the graphics for the upload and moment of termination of upload. Some of the code is below:
[request0 setBytesSentBlock:^(unsigned long long size, unsigned long long total) {
double timeDiffereceFromStart = [[NSDate date] timeIntervalSinceDate:start];
if (totalUploadSize == 0)
{
start=[NSDate date];
totalUploadSize = [request.postBody length];
return;
}
if(startPosition == 0 && timeDiffereceFromStart >= 1)//[request totalBytesSent] > 20000)
{
startPosition = [request totalBytesSent];
start=[NSDate date];
return;
}
I've just posted some of the code, but wanted to show where I used the variable 'request' within the block. I'm pretty sure I've fixed the circular retain issue here, but I wanted to make sure there wasn't some other problem.
On other thing I should note - I've put a break point within the ASIHTTPRequest dealloc function. All of the objects of this type that I create hit the dealloc breakpoint. So they are all being freed properly. But I don't understand why the memory usage keeps going up when it hits the upload function.
Thanks!
I've figured out the issue, and it was a retain cycle which involved the parent class of the class from which I posted the code. Because this part of the system isn't wasn't written by me, I missed it. I ended up fixing the warnings that point out retain cycles when using blocks, and the memory leaks were gone.
I'm basically implementing a fancier NSURLConnection class that downloads data from a server parses it into a dictionary, and returns an NSDictionary of the data. I'm trying add a completion block option (in addition to a delegate option), but it crashes anytime I try to store that data in another class.
[dataFetcher_ fetchDataWithURL:testURL completionHandler:^(NSDictionary *data, NSInteger error) {
contentDictionary_ = data;
}];
I can NSLog that data just fine, and basically do whatever I want with it, but as soon as I try to save it into another variable it crashes with a really obscure message.
EDIT: the crash message is EXC_BAD_ACCESS, but the stack trace is 0x00000000 error: address doesn't contain a section that points to a section in a object file.
I'm calling this function in the init method of a singleton. It DOES let me save the data if I set this in the completion block.
[SingletonClass sharedInstance].contentDictionary = data
But then the app gets stuck forever because sharedInstance hasn't returned yet, so the singleton object is still nil, so sharedInstance in the completion block calls init again, over and over.
EDIT 2: The singleton code looks like this:
+ (SingletonClass*)sharedInstance {
static SingletonClass *instance;
if (!instance) {
instance = [[SingletonClass alloc] init];
}
return instance;
}
- (id)init {
self = [super init];
if (self) {
dataFetcher_ = [[DataFetcher alloc] init];
NSString *testURL = #"..."
[dataFetcher_ fetchDataWithURL:testURL completionHandler:^(NSDictionary *data, NSInteger error) {
[SingletonClass sharedInstance].contentDictionary = data;
}];
}
return self;
}
Like I said, this works fine but repeats the initialize code over and over until the app crashes. This only happens the first time I run the app on a device, because I cache the data returned and it doesn't crash once I have the data cached. I would like to be able to just say self.contentDictionary = data, but that crashes.
Specify a variable to be used in the block with the __block directive outside of the block:
__block NSDictionary *contentDictionary_;
[dataFetcher_ fetchDataWithURL:testURL completionHandler:^(NSDictionary *data, NSInteger error) {
contentDictionary_ = data;
}];
You're invoking recursion before ever setting the "instance". (which I now see you understand from OP).
In your block, you can use the ivar or an accessor instead of
[SingletonClass sharedInstance].contentDictionary
use:
_contentDictionary = [data copy]; or self.contentDictionary=data;
assuming that the ivar backing the contentDictionary property is _contentDictionary.
It sounds like you tried self.contentDictionary and it failed? I got it to work in a test, with ARC turned, so there may be something about your dataFetcher that is affecting this. In my test dataFetcher just returns a dictionary with a single element.
Turns out the issue was with a bunch of different parts. My URL was empty sometimes, and my data fetcher would just fail immediately and call the completion block. In my completion block I hadn't included any error handling, so if the singleton class hadn't initialized, it would repeat forever. With a real URL this doesn't happen.
I still would like to figure out why it crashes when I try to assign the data to an ivar, though.