I have a UITableView with cells to display images and text from a largish (5000 items) JSON file. I want to stream the JSON in and start updating the UITableView immediately, but can't seem to work out the plumbing for this.
- (NSArray *)parseJSONIntoImageObjectsFromData:(NSData *)rawJSONData {
NSError *error;
NSMutableArray *arrayOfImageObjects = [[NSMutableArray alloc] init];
NSURL *myURL = [[NSURL alloc] initWithString:self.urlString];
NSData *objects = [NSData dataWithContentsOfURL:myURL];
NSInputStream *stream = [[NSInputStream alloc] initWithData:objects];
[stream open];
NSMutableArray *arrayFromStream = [NSJSONSerialization JSONObjectWithStream:stream options:NSJSONReadingAllowFragments error:&error];
for (NSDictionary *JSonDictionary in arrayFromStream) {
NSLog(#"Count is %lu", (unsigned long)arrayOfImageObjects.count);
NSInteger imgID = (NSInteger)JSonDictionary[#"id"];
ImageObject *newImageObject = [[ImageObject alloc] initWithID:imgID andTitle:JSonDictionary[#"title"] andThumbnailURL:JSonDictionary[#"thumbnailUrl"]];
[arrayOfImageObjects addObject:newImageObject];
}
return arrayOfImageObjects;
}
This definitely gets them as a stream, as the NSLog reveals in the debug window. But since it waits for the return it has to complete. I'm a little puzzled at going about this and can't find a good code sample. Do I perhaps return a stream?
EDIT: I am not terribly concerned about the brief delay I am encountering and I am sure the delay is more on the retrieval than in the parsing, I just want to learn to retrieve the data as a stream and update the UITableView incrementally as a way to do this better. I enjoy working on data retrieval and manipulation and am trying to improve my skills by knowing more.
Also, the images are retrieved asynchronously at display time using an NSOperationQueue and don't really matter for this task.
If you benchmark this, I think you'll find that the parsing time of the JSON is inconsequential. The slow parts are going to be the download of the original JSON (and possibly the creation of the ImageObject objects). You should benchmark this in Instruments (use the "time profiler" tool) and use the "record waiting threads" option. See WWDC video Building Concurrent User Interfaces on iOS for a demonstration on how to use Instruments to diagnose these sorts of issues.
I would first retire the dataWithContentsOfURL, as that runs synchronously. I would advise using an asynchronous technique such as NSURLSession method dataWithURL (or if you need support for pre-iOS 7, NSURLConnection method sendAsynchronousRequest).
Usually in these cases, the JSON is small enough, that the biggest delay stems from the network latency in making the initial request. I mention that so that you don't bother embarking on some major refactoring of the code for paging/streaming approaches without confirming that this will solve the problem.
Also, you haven't shared this ImageObject logic, but if that is synchronously loading images, that's a likely candidate for refactoring for asynchronous retrieval, too. Without knowing more about that class, it's hard to advise you further on that point.
Define NSMutableArray *arrayOfImageObjectsas a property or variable outside this method and then in your for loop, call [self.tableView reloadData] after maybe every 100 objects.
That's assuming that your numberOfRowsInSection is keying off of arrayOfImageObjects as well and cellForRowAtIndexPath is using it to populate the table data.
But also consider 'paging' your data, so as to only load 50 objects or so, at once (assuming your API supports this like 'http://example.com/imagedata?page=1'). Then if the user flicks or scrolls the tableview you can do another api call, increasing the page number and adding that new set of data to your current set and calling reloadData.
EDIT: also I'm assuming your "parseJSONIntoImageObjectsFromData" is running asynchronously. If not then use something like AFNetworking (or sendAsynchronousRequest:queue:completionHandler: in NSURLConnection) and in the completion block you can start adding to your array.
Related
I am retrieving a tremendous amount of data from a server and I have to quickly initialize and deallocate a NSMutableDictionary.
I would use core data but requirements say I have to use one allocated object for fast storage of incoming JSON data, save it to NSUserDefaults and completely remove it. This will be happening multiple times in a background process over a period of time.
Tests were on a live iPhone 6S (not a simulator)
I tested a completely bare "Single View Application" that resulted with:
As you can see a bare bones "Single View Application" consumes 9MB of memory.
I then ran a test on a NSMutableDictionary. My test was to initialize with data, remove all objects, and restore the RAM.
The dictionary stored 10,000 tiny values (in viewDidLoad - for testing purposes):
_dictionary = [[NSMutableDictionary alloc] initWithDictionary:#{
#"Item00000" : [NSNumber numberWithInt:0],
...
#"Item10000" : [NSNumber numberWithInt:10000]}];
After initializing - I waited two seconds and removed all objects and attempted to remove the NSMutableDictionary.
dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(2.0 * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
[_largeDictionary removeAllObjects];
_largeDictionary = nil;
});
But the memory consumption did not show the NSMutableDictionary getting removed:
Does anyone have a solution to this problem? I thought this was taken care of when Apple depreciated the autorelease method. My live environment is quickly initializing relatively large chunks of data per key-value pair - then deallocating and preparing for the next chunk. The next chunk may be stored in a different NSMutableDictionary. That's why it is necessary for me to completely remove the first NSMutableDictionary from memory. I have seen many answers pertaining to this question but none more recent than 2013.
Update: this #property is: (nonatomic, strong)
Thank you in advance.
I'm parsing a JSON file on an iPad which has about 53 MB. The parsing is working fine, I'm using Yajlparser which is a SAX parser and have set it up like this:
NSData *data = [NSData dataWithContentsOfFile:path options:NSDataReadingMappedAlways|NSDataReadingUncached error:&parseError];
YAJLParser *parser = [[YAJLParser alloc] init];
parser.delegate = self;
[parser parse:data];
Everything worked fine until now, but the JSON-file became bigger and now I'm suddenly experiencing memory warnings on the iPad 2. It receives 4 Memory Warnings and then just crashes. On the iPad 3 it works flawlessly without any mem warnings.
I have started profiling it with Instruments and found a lot of CFNumber allocations (I have stopped Instruments after a couple of minutes, I had it run before until the crash and the CFNumber thing was at about 60 mb or more).
After opening the CFNumber detail, it showed up a huge list of allocations. One of them showed me the following:
and another one here:
So what am I doing wrong? And what does that number (e.g. 72.8% in the last image) stand for? I'm using ARC so I'm not doing any Release or Retain or whatever.
Thanks for your help.
Cheers
EDIT: I have already asked the question about how to parse such huge files here: iPad - Parsing an extremely huge json - File (between 50 and 100 mb)
So the parsing itself seems to be fine.
See Apple's Core Data documentation on Efficiently Importing Data, particularly "Reducing Peak Memory Footprint".
You will need to make sure you don't have too many new entities in memory at once, which involves saving and resetting your context at regular intervals while you parse the data, as well as using autorelease pools well.
The general sudo code would be something like this:
while (there is new data) {
#autoreleasepool {
importAnItem();
if (we have imported more than 100 items) {
[context save:...];
[context reset];
}
}
}
So basically, put an autorelease pool around your main loop or parsing code. Count how many NSManagedObject instances you have created, and periodically save and reset the managed object context to flush these out of memory. This should keep your memory footprint down. The number 100 is arbitrary and you might want to experiment with different values.
Because you are saving the context for each batch, you may want to import into a temporary copy of your store in case something goes wrong and leaves you with a partial import. When everything is finished you can overwrite the original store.
Try to use [self.managedObjectContext refreshObject:obj refreshChanges:NO] after certain amount of insert operations. This will turn NSManagedObjects into faults and free up some memory.
Apple Docs on provided methods
I am fetching data from a server in JSON format. It's only about 150 records and I was not using GCD initially but every now and again when I hit the button in the app to view the table with data it would delay for about a couple of seconds before switching to the table view and displaying the data. So I implemented GCD and now when I hit the button it switches to the tableview immediately but then there is a few seconds delay in loading the data, which seems longer than the pre-GCD implementation. So I'm not sure if I am using GCD correctly, or if it's my server causing the delay (which I think is the culprit). Here is the implementation of GCD in a method called retrieveData which I call in viewDidLoad as [self retrieveData]:
- (void)retrieveData
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0),^{
NSURL *url = [NSURL URLWithString:#"http://MY_URL/JSON/document.json"];
NSData * data = [NSData dataWithContentsOfURL:url];
dispatch_async(dispatch_get_main_queue(), ^{
json = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:nil];
//Set up our exhibitors array
exhibitorsArray = [[NSMutableArray alloc] init];
for (int i = 0; i < json.count; i++) {
//create exhibitors object
NSString * blabel = [[json objectAtIndex:i] objectForKey:#"BoothLabel"];
NSString * bName = [[json objectAtIndex:i] objectForKey:#"Name"];
NSString * bURL = [[json objectAtIndex:i] objectForKey:#"HyperLnkFldVal"];
exhibitors * myExhibitors = [[exhibitors alloc] initWithBoothName: bName andboothLabel: blabel andBoothURL: bURL];
//Add our exhibitors object to our exhibitorsArray
[exhibitorsArray addObject:myExhibitors];
//Sort by name
NSSortDescriptor *sort = [NSSortDescriptor sortDescriptorWithKey:#"name" ascending:YES];
[exhibitorsArray sortUsingDescriptors:[NSMutableArray arrayWithObject:sort]];
}
[self.myTableView reloadData];
});
});
}
This is basically correct. Dispatch the data retrieval to the background queue, and then dispatch the model and UI update back to the main queue. Well done.
In terms of its being slower, I don't see anything there that would account for that. GCD introduces some overhead, but generally not observable. It may be a bit of a "watched kettle never boils" issue.
A couple of unrelated thoughts, though:
I might suggest moving the sort to outside of the for loop, but before the reloadData. You're sorting it 150 times. If doing an insertion sort, you could do it within the loop, but I don't think that's happening here. I'd move the sort to the end of the loop. I'm not sure if the performance gain will be observable, but there should be some modest improvement.
You might want to make sure data is not nil (e.g. no network, or some other network issue), because if it is, JSONObjectWithData will crash.
Your json object is an external variable. It should probably be a local variable of your retrieveData method. There's no need to make it an instance variable. It's cleaner to make it a local variable if appropriate.
You probably should adopt the naming convention whereby class names start with uppercase letters (e.g. Exhibitor instead of exhibitors).
Very minor point, but your blabel variable should probably be bLabel. Even better, I might rename these three variables boothLabel, boothName, and boothUrlString.
You're using instance variable for exhibitorsArray. I presume you're doing this elsewhere, too. You might want to consider using declared properties instead.
You might want to turn on the network activity indicator before dispatching your code to the background, and turning it back off when you perform reloadData.
[[UIApplication sharedApplication] setNetworkActivityIndicatorVisible:YES];
If you wanted to get really sophisticated, you might reconsider whether you want to use GCD's global queues (because if you refresh 10 times quickly, all ten will run, whereas you probably only want the last one to run). This is a more complicated topic, so I won't cover that here, but if you're interested, you might want to refer to the discussion of operation queues the Concurrency Programming Guide, in which you can create operations that are cancelable (and thus, when initiating a new operation, cancel the prior one(s)).
You might also want to refer to the Building Concurrent User Interfaces on iOS WWDC 2012 video.
But this is all tangential to your original question: Yes, you have tackled this appropriately.
Trying to implement an app which sends offline data stored on local db to web server when connected to internet. I use the code shown below. As far I have tested it works fine, not sure it will work fine for huge number of records. I would like to know whether any tweaking on this code may increase the performance???
NOTE
I know this would be a worst code for offline sync purpose, so trying
to tweak it better.
Its a single way synchronization, from app to server.
-(void)FormatAnswersInJSON {
DMInternetReachability *checkInternet = [[DMInternetReachability alloc] init];
if ([checkInternet isInternetReachable]) {
if ([checkInternet isHostReachable:#"www.apple.com"]) {//Change to domain
responseArray = [[NSMutableArray alloc] init];
dispatch_async(backgroundQueue, ^(void) {
NSArray *auditIDArray = [[NSArray alloc] initWithArray: [self getUnuploadedIDs]];
for (int temp = 0; temp < [auditIDArray count]; temp ++) {
// Code to post JSON to server
NSURLResponse *response;
NSData *urlData=[NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
if (!error) {
NSString *responseID = [[NSString alloc]initWithData:urlData encoding:NSUTF8StringEncoding];
if ([responseID isEqualToString:#"ERROR"]) {
//Error uploading records
} else {
[responseArray addObject:responseID];
}
} else {
//Error
return;
}
}
dispatch_async( backgroundQueue, ^{
/* Based on return code update local DB */
for (int temp = 0; temp < [responseArray count]; temp ++) {
[self updateRecordsForID:[auditIDArray objectAtIndex:temp] withID:[responseArray objectAtIndex:temp]];
}
});
});
}
}
}
- (void)upload { //Called when internet connection available
if(backgroundQueue){
dispatch_suspend(backgroundQueue);
dispatch_release(backgroundQueue);
backgroundQueue = nil;
}
backgroundQueue = dispatch_queue_create("com.XXXX.TestApp.bgqueue", NULL);
dispatch_async(backgroundQueue, ^(void) {
[self FormatAnswersInJSON];
});
}
If this code were sitting in front of me, my approach would be:
Look at the use cases and define 'huge number of records': Will 50 record updates at a time occur regularly? Or will it be in 1s and 2s? Do my users have wifi connections or is it over the paid network?, etc.
If possible, test in the wild. If my user base was small enough, gather real data and let that guide my decisions, or only release the feature to a subset of users/beta tests and measure.
If the data tells you to, then optimize this code to be more efficient.
My avenue of optimization would be doing group processing. The rough algorithm would be something like:
for records in groups of X
collect
post to server {
on return:
gather records that updated successfully
update locally
}
This assumes you can modify the server code. You could do groups of 10, 20, 50, etc. all depends on the type of data being sent, and the size.
A group algorithm means a bit more pre-processing client side, but has the pro of reducing HTTP requests. If you're only ever going to get a small number of updates, this is YAGNI and pre-mature optimization.
Don't let this decision keep you from shipping!
Your code has a couple of issues. One convention is to always check the return value before you test the error parameter. The error parameter might be set - even though the method succeeded.
When using NSURLConnection for anything else than a quick sample or test, you should also always use the asynchronous style with handling the delegate methods. Since using NSURLConnection properly may become quickly cumbersome and error prone, I would suggest to utilize a third party framework which encapsulates a NSURLConnection object and all connection related state info as a subclass of NSOperation. You can find one example implementation in the Apple samples: QHTTPOperation. Another appropriate third party framework would be AFNetworking (on GitHub).
When you use either the async style with delegates or a third party subclass, you can cancel the connection, retrieve detailed error or progress information, perform authentication and much more - which you can't with the synchronous API.
I think, once you have accomplished this and your approach works correctly, you may test whether the performance is acceptable. But unless you have large data - say >2 MByte - I wouldn't worry too much.
If your data becomes really large, say >10 MByte you need to consider to improve your approach. For example, you could provide the POST data as file stream instead a NSData object (see NSURLRequest's property HTTPBodyStream). Using a stream avoids to load all the POST data into RAM which helps alleviate the limited RAM problem.
If you have instead smaller POST data, but possibly many of them, you might consider to use a NSOperationQueue where you put your NSOperation connection subclass. Set the maximum number of concurrent operations to 2. This then may leverage HTTP pipelining - if the server supports this, which in effect reduces latency.
Of course, there might be other parts in your app, for example you create or retrieve the data which you have to send, which may affect the overall performance. However, if your code is sound and utilizes dispatch queues or NSOperations which let things perform in paralel there aren't many more options to improve the performance of the connection.
This is my problem, I'm building an iOS with some JSON returs from my server, here, no problems, all works fine.
The problem is that when I run the program, it take a long long time to parse the result into a NSMutableArray: this is the log
2013-01-10 12:03:48.627 project[5917:907] <- time begin parser
2013-01-10 12:03:48.755 project[5917:907] <- time finish parser
2013-01-10 12:03:48.756 project[5917:907] <- time begin implement arrays
2013-01-10 12:03:58.522 project[5917:907] <- time finish implement array
As you can see, implementing the arrays is really long.
I know that I have to use queueing and grand central dispatch to make my UI responsive, but I don't know how to, could you please help me to do that ?
This is my viewDidLoad Method
- (void)viewDidLoad
{
[super viewDidLoad];
if debug
NSLog(#"<- time begin parser");
endif
NSString *URLStr = #"http://myJsonFile.json";
NSDictionary *myDictwithReturn = [TOCJSONParser awesomeParserWithStringOfYourJSONFile:URLStr]; //really cool parser, i can put it on gitHub if you want
NSArray *speakersArray = [myDictwithReturn objectForKey:#"speakers"];
myArray = [[NSMutableArray alloc]init];
NSLog(#"<- time finish parser");
NSLog(#"<- time begin implement arrays");
for (NSDictionary *myDict in speakersArray) {
_nextSpeaker = [[TOCSpk alloc]init];
[_nextSpeaker setName:[myDict objectForKey:#"name"]];
[_nextSpeaker setBusiness:[myDict objectForKey:#"business"]];
[_nextSpeaker setDesc:[myDict objectForKey:#"desc"]];
[_nextSpeaker setTwitter:[NSURL URLWithString:[myDict objectForKey:#"twitter"]]];
[_nextSpeaker setPicture:[_nextSpeaker retrieveImageFromServer:[myDict objectForKey:#"picture"]]];
[myArray addObject:_nextSpeaker];
}
NSLog(#"<- time finish implement array");
}
I suspect that the slowness comes from calling retrieveImageFromServer, which lets me think that you are accessing the network. If that access is synchronous, as it seems from the fact that you are assigning the image in the same statement, than this is bound to be slow.
You should review your code and make it run on a separate thread or use asynchronous network access.
EDIT:
After your comment about using, dataWithContentsOfURL, my above hypothesis is confirmed.
You can read this S.O. post about a way to download images asynchronously, or you might use any of various networking frameworks available out there.
Possibly, the easiest path forward is using SDWebImage, which is a class that offers async download for images, so you don´t have to bother yourself with thread management or NSURLConnection:
Just #import the UIImageView+WebCache.h header, and call the setImageWithURL:placeholderImage: method from the tableView:cellForRowAtIndexPath: UITableViewDataSource method. Everything will be handled for you, from async downloads to caching management.