Memory issue generated in iPad because data receiving very lengthy - ios

Question

I am struck in a big problem. What I am trying to do is getting data from the server but the server in a single hit give all the data which is very large in quantity and I am doing all my process on the main thread, So there are around 400-500 images in the form of URL which I am saving in document directory in the form of NSData. So in the dubug navigator when the memory consumption reached around 80-90 mb then my application crashed and showing the following error:-
mach_vm_map(size=135168) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
2015-01-23 17:10:03.946 ArchaioMobileViewer[853:148470] *** Terminating app due to uncaught exception 'NSMallocException', reason: 'Attempt to allocate 262144 bytes for NS/CFData failed'
I am Using ARC but still I am getting the memory problem. This is my code `-
(void)downloadDocumentsFromServer:(NSDictionary *)documentsList IsUpdate:(BOOL)isUpdate;
{
//Main Target(22)
BusinessLayer* bLL = [[BusinessLayer alloc]init];
FileManager* downloadImages = [FileManager alloc];
for(NSDictionary* inspDocumentResult in documentsList)
{
FloorDocument* floorDocument = [[FloorDocument alloc]init];
floorDocument.docID = [inspDocumentResult objectForKey:#"docID"];
floorDocument.buildingID = selectedBuildingID;
floorDocument.clientID = clientID;
NSDictionary* documentArray = [inspDocumentResult objectForKey:#"Document"];
floorDocument.docType = [[documentArray objectForKey:#"Type"] stringByTrimmingCharactersInSet:
[NSCharacterSet whitespaceAndNewlineCharacterSet]];
floorDocument.docScale = [documentArray objectForKey:#"Scale"];
floorDocument.docDescription = [documentArray objectForKey:#"DocDesc"];
//floorDocument.floor = [bLL getFloorNameByDocIDAndBuildingID:selectedBuildingID DocID:floorDocument.docID];
floorDocument.floor = [inspDocumentResult objectForKey:#"Floor"];
NSLog(#"%#",[inspDocumentResult objectForKey:#"hiResImage"]);
[downloadImages downloadInspectionDocuments:[inspDocumentResult objectForKey:#"hiResImage"] ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID];
NSLog(#"Floor %# - High Res Image copied for %#",floorDocument.floor,floorDocument.docID);
//Download the Low Res Image
NSString* lowResImage = [inspDocumentResult objectForKey:#"lowResImage"];
[downloadImages downloadInspectionDocumentsLowRes:lowResImage ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID LowResName:#"lowResImage"];
//Copy the Quarter Size File
lowResImage = [lowResImage stringByReplacingOccurrencesOfString:#"LowRes" withString:#"LowRes4"];
[downloadImages downloadInspectionDocumentsLowRes:lowResImage ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID LowResName:#"lowResImage4"];
NSLog(#"Floor %# - Low Res Images copied for %#",floorDocument.floor,floorDocument.docID);
//Download the tiles
NSArray* tiles = [inspDocumentResult objectForKey:#"lsUrls"];
for(NSString* tile in tiles)
{
#autoreleasepool {
NSArray* tileNameArray = [tile componentsSeparatedByString:#"/"];
if(tileNameArray.count > 0)
{
NSString* destTile = [tileNameArray objectAtIndex:tileNameArray.count-1];
destTile = [destTile stringByReplacingOccurrencesOfString:[NSString stringWithFormat:#".%#",floorDocument.docType] withString:#""];
NSLog(#"TileName:%#",destTile);
[downloadImages downloadInspectionDocumentsTiles:tile ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID TileName:destTile];
}
}
}
NSLog(#"Floor %# - Tiles Image copied for %#",floorDocument.floor,floorDocument.docID);
NSLog(#"Downloading Documents Tiles For %# Completed at %#",floorDocument.docID,[bLL getCurrentDate]);
[bLL saveFloorDocuments:floorDocument IsUpdate:isUpdate];
// downloadImages=nil;
}
bLL = nil;
}
please help me out in this problem.`
This is the code which I am using inside the DownloadInspectionDocuments:-
-(void)downloadInspectionDocuments:(NSString *)url ImageName:(NSString *)imageName FileType:(NSString*)fileType Folder:(NSString*)folder
{
#autoreleasepool
{
NSString* source =[FileManager getInspectionDocumentsFolder];
//Lets get the destination folder
NSString *destination = [NSString stringWithFormat:#"%#/%#/%#",source,folder,imageName];
[self createFolder:destination CreateSubFolders:true];
NSString *filePath = [NSString stringWithFormat:#"%#/%#.%#",destination,imageName,fileType];
NSFileManager* fm = [[NSFileManager alloc]init];
if(![fm fileExistsAtPath:filePath])
{
NSData *data1 = [NSData dataWithContentsOfURL:[NSURL URLWithString:url]];
[data1 writeToFile:filePath atomically:YES];
}
}
// return [NSString stringWithFormat:#"%#.%#",imageName,fileType];
}

Answer 1

ARC isn't garbage collection: it will insert the memory management code (retain/release) for you but you still need to make sure you are not using up too many resources (in the same way you would in non-ARC code).
You are running this large loop on the main thread, so any memory being consumed is not going to be freed until the next run loop.
You need to break this function down into smaller steps that can be carried out in stages.
For now, if there isn't too much memory consumed for a single iteration of the outer-loop of the function you can add an autorelease pool at that level (I see you have on on the inner loop)
for(NSDictionary* inspDocumentResult in documentsList)
{
#autoreleasepool {
.... remaining code goes here
}
}
and it will at least drain what it can each iteration.
Given that you are downloading a large number of files and will be relying on network connectivity I would recommend performing the downloads asynchronously though. If you haven't already, check out AFNetworking to simplify this. This will give you much more control over your resources than you are getting now with a resource-intensive blocking call on the main thread.

Answer 2

You can save yourself a lot of work by following davbryn's and Andrea's suggestions to use AFNetworking and stream the file. Basically, don't put the whole file in memory and then write it to disk, write to disk as you get the bytes from the network. This should reduce the pressure on memory. For example:
- (void)downloadFile:(NSString *)urlString {
NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:urlString]];
NSString *destinationPath = [NSDocumentDirectory() stringByAppendingPathComponent:#"some-file-name"];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setOutputStream:[NSOutputStream outputStreamToFileAtPath:destinationPath append:NO]];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
NSLog(#"Super duper awesome!");
// Maybe start another download here?
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Error downloading file: %#", error);
}];
[operation start];
}
So then all you need to do is generate the list of things to download and in your success block start downloading another file.