Handle large JSON files in TableView - ios

I'm trying to parse a large JSON file (approx: 1000 rows containg a tuple with 8 strings) and display this in a UITableView. What I already have is working but I am looking for a more efficient way of displaying them.
At the moment my code looks likes this:
public func GET(request: String, callback: (result: JSON?, response: NSHTTPURLResponse?, error: NSError?) -> Void) {
let session = NSURLSession.sharedSession()
let url = NSURL(string : "SOMEURL")
let task = session.dataTaskWithURL(url!){
(data, response, error) -> Void in
if error != nil {
callback(result: nil, response: response as? NSHTTPURLResponse, error: error!)
} else {
callback(result: JSON(data : data!), response: response as? NSHTTPURLResponse, error: nil)
}
}
task.resume()
}
This does parse the data using SwiftJSON (see JSON(data : data!)), then when it comes to actually filling an array i use a class containing two attributes (one for the Main text in table and one for detail text)
class SomeClass {
let MainText : String
let DetailText : String
init(MainText : String, DetailText : String) {
self.MainText = MainText
self.DetailText = Detailtext
}
}
Now in the UITableView i have a .swift file and in the
override func ViewDidLoad() {
//code
}
I use a loop to get the data from the result callback in the GET method to append to an array of
var rows : [SomeClass] = []
This is very CPU intensive but I did not find another way to deal with this problem. I tried only displaying 50 rows in the table and only creating 50 class items for the rows. But none of that matters, what I fear is that the SwiftyJSON way of dealing with this problem is not the right one but i thought that maybe I am overlooking something.

If I understood your problem, you are worried about CPU / Energy Efficiency.
What you should consider, if it's not how your app already works, is implementing the parsing process in the background thread, make your [SomeClass] array observable and update the table when it changes (aka when the background parsing added an new value to it).
So first make your parsing function run in background (for instance with the Async GCD wrapper) :
func callback(JSON?, response: NSHTTPURLResponse, error: NSError?) {
Async.background {
//Do your JSON parsing stuff here, XXX is a SomeClass object
rows <- rows + [XXX]
}
}
You might have noticed the unusual syntax for the array appending method. That's because making your array "observable" is part of the solution. I advise you to get the Observable-Swift library to make it easier to observe.
Once added to your project, change your array declaration :
var rows = Observable([SomeClass]())
Now implement the method that will be called when your callback parsed a new item (for instance in your viewDidLoad:)
rows.afterChange += { self.table.reloadData() }
where table is your table view
If you want to implement a power-friendly runtime, you might want to update the table every time 50 or 100 objects are added to the array. This can be done so (if you want to do so do not implement the method right above):
rows.afterChange += { if $1.count / 100 = 1 { self.table.reloadData() }}
where 100 is the value of new object required to be added in order to update the table. With Observable-Swift, $0 represents the array before it was updated and $1 the array after its update.
One last thing : the rows array is no longer of type [SomeClass] but Observable<SomeClass>. If you want to access the [SomeClass] value, just replace rows by rows.value
Hope I didn't misunderstood your question. Anyway if I did, I think that can still help providing a better implementation of JSON parsing.

You should not be worried about how much of data you have to display in TableView.
TableView class handles everything for you as long as you pass the json object properly as a Tablesource.

It's actually a pretty good concern about how you use the resources. Normally, we will go with pagination if you don't want to query back whole amount of data from a request. Then, you will implement some proper logic based on the skip and limit in order to get further data.
As for the UITableView, there is nothing to worry about. Because, it's developed in an efficient way. The total number of cell in memory is the total number of cell visible. The UITableView will help populating the data via delegation methods. It's not like: you have 500 rows of data, then it has 500 UITableViewCell. It's reusability.

Related

Tableview reload not accurate

I have a controller, that allows the user to type in a TextField.
Every time the user types a character, the string in that textfield is compared to an array of strings. If there is a match, the resulting array is displayed in a uitableview.
Here's the code:
func searchAutocompleteEntriesWithSubstring(substring:String){
let SUBSSTRING = substring.uppercased()
autocompleteStrings.removeAll()
for thisSchool in schoolArray{
if(thisSchool.name?.uppercased() .contains(SUBSSTRING))!{
autocompleteStrings.append(thisSchool)
}
}
autocompleteTableView.reloadData()
}
Basically, this works fine. BUT!
If the user types rather fast, the autocompleteTableView displays one or more (empty) rows than there actually are strings in the autocompleteStrings array.
I tried encapsulating the above code in DispatchQueue.main.async {}, but that made things even worse.
I guess it has something to do with NeedsLayout or NeedsDisplay, but I've never really understood the mechanism behind it, and how/where to apply these.
I hope you can advise me
Try this code
func searchAutocompleteEntriesWithSubstring(substring: String) {
let filtered = schoolArray.filter() { ($0.name ?? "").uppercased().range(of: substring.uppercased()) != nil }
autocompleteStrings = filtered.map() { $0.name! }
autocompleteTableView.reloadData()
}
Maybe you need a lock?
1.in a async queue.
2.lock locks.
3.matching and array appending
4.lock unlocks.
5.reload in mainqueue

Use Realm with Collection View Data Source Best Practise

I'll make it short as possible.
I have an API request that I fetch data from (i.e. Parse).
When I'm getting the results I'm writing it to Realm and then adding them to a UICollectionView's data source.
There are requests that take a bit more time, which run asynchronous. I'm getting the needed results after the data source and collection view was already reloaded.
I'm writing the needed update from the results to my Realm database.
I have read that it's possible to use Realm's Results. But I honestly didn't understood it. I guess there is a dynamic and safe way working with collection views and Realm. Here is my approach for now.
This is how I populate the collection view's data source at the moment:
Declaration
var dataSource = [Realm_item]()
where Realm_item is a Realm Object type.
Looping and Writing
override func viewDidLoad() {
super.viewDidLoad()
for nowResult in FetchedResultsFromAPI
{
let item = Realm_item()
item.item_Title = nowResult["Title"] as! String
item.item_Price = nowResult["Price"] as! String
// Example - Will write it later after the collectionView Done - Async request
GetFileFromImageAndThanWriteRealm(x.image)
// Example - Will write it later after the collectionView Done - Async request
dataSource.append(item)
}
//After finish running over the results *Before writing the image data*
try! self.realm.write {
self.realm.add(self.dataSource)
}
myCollectionView.reloadData()
}
After I write the image to Realm to an already created "object". Will the same Realm Object (with the same primary key) automatically update over in the data source?
What is the right way to update the object from the data source after I wrote the update to same object from the Realm DB?
Update
Model class
class Realm_item: Object {
dynamic var item_ID : String!
dynamic var item_Title : String!
dynamic var item_Price : String!
dynamic var imgPath : String?
override class func primaryKey() -> String {
return "item_ID"
}
}
First I'm checking whether the "object id" exists in the Realm. If it does, I fetch the object from Realm and append it to the data source. If it doesn't exist, I create a new Realm object, write it and than appending it.
Fetching the data from Parse
This happens in the viewDidLoad method and prepares the data source:
var query = PFQuery(className:"Realm_item")
query.limit = 100
query.findObjectsInBackgroundWithBlock { (respond, error) -> Void in
if error == nil
{
for x in respond!
{
if let FetchedItem = self.realm.objectForPrimaryKey(Realm_item.self, key: x.objectId!)
{
self.dataSource.append(FetchedItem)
}
else
{
let item = Realm_item()
item.item_ID = x.objectId
item.item_Title = x["Title"] as! String
item.item_Price = x["Price"] as! String
let file = x["Images"] as! PFFile
RealmHelper().getAndSaveImageFromPFFile(file, named: x.objectId!)
self.dataSource.append(item)
}
}
try! self.realm.write {
self.realm.add(self.dataSource)
}
self.myCollectionView.reloadData()
print(respond?.count)
}
}
Thank you!
You seem to have a few questions and problems here, so I'll do my best.
I suggest you use the Results type as your data source, something like:
var dataSource: Results<Realm_item>?
Then, in your viewDidLoad():
dataSource = realm.objects(Realm_item).
Be sure to use the relevant error checking before using dataSource. We use an optional Results<Realm_item> because the Realm object you're using it from needs to be initialised first. I.e., you'll get something like "Instance member * cannot be used on type *" if you try declaring the results like let dataSource = realm.objects(Realm_item).
The Realm documentation (a very well-written and useful reference to have when you're using Realm as beginner like myself), has this to say about Results...
Results are live, auto-updating views into the underlying data, which means results never have to be re-fetched. Modifying objects that affect the query will be reflected in the results immediately.
Your mileage may vary depending on how you have everything set up. You could try posting your Realm models and Parse-related code for review and comment.
Your last question:
What is the right way to update the "object" from the Data Source after i wrote the update to same object from the Realm DB?
I gather you're asking the best way to update your UI (CollectionView) when the underlying data has been updated? If so...
You can subscribe to Realm notifications to know when Realm data is updated, indicating when your app’s UI should be refreshed for example, without having to re-fetch your Results.

Capturing closure values in Swift

My question is very similar to several others here but I just can't get it to work. I'm making an API call via a helper class that I wrote.
First I tried a standard function with a return value and the result was as expected. The background task completed after I tired to assign the result.
Now I'm using a closure and I can get the value back into my view controller but its still stuck in the closure, I have the same problem. I know I need to use GCD to get the assignment to happen in the main queue.
this is what I have in my view controller
var artists = [String]()
let api = APIController()
api.getArtistList("foo fighters") { (thelist) -> Void in
if let names = thelist {
dispatch_async(dispatch_get_main_queue()) {
artists = names
print("in the closure: \(artists)")
}
}
}
print ("method 1 results: \(artists)")
as the results are:
method 1 results: []
in the closure: [Foo Fighters & Brian May, UK Foo Fighters, John Fogerty with Foo Fighters, Foo Fighters, Foo Fighters feat. Norah Jones, Foo Fighters feat. Brian May, Foo Fighters vs. Beastie Boys]
I know why this is happening, I just don't know how to fix it :( The API calls need to be async, so what is the best practice for capturing these results? Based on what the user selects in the table view I'll be making subsequent api calls so its not like I can handle everything inside the closure
I completely agree with the #Craig proposal of the use of the GCD, but as your question involves the request of the API call every time you select a row, you can do the following:
Let's suppose you use the tableView:didSelectRowAtIndexPath: method to handle the selection, then you can do the following inside it:
func tableView(tableView: UITableView, didSelectRowAtIndexPath indexPath: NSIndexPath) {
// it is just a form to get the item
let selectedItem = items.objectAtIndex(indexPath.row) as String
api.getArtistList(selectedItem) { (thelist) -> Void in
if let names = thelist {
dispatch_async(dispatch_get_main_queue()) {
artists = names
}
}
}
}
And then you can observe the property and handle do you want inside it :
var artists: [String] = [] {
didSet {
self.tableView.reloadData() // or anything you need to handle.
}
}
It just another way to see it. I hope this help you.
The easy solution is to do whatever you're doing at your print(), inside the closure.
Since you're already dispatch_asyncing to the main queue (the main/GUI thread), you can complete any processing there. Push a new view controller, present some modal data, update your current view controller, etc.
Just make sure that you don't have multiple threads modifying/accessing your local/cached data that is being displayed. Especially if it's being used by UITableViewDelegate / UITableViewDataSource implementations, which will throw fits if you start getting wishy-washy or inconsistent with your return values.
As long as you can retrieve the data in the background, and the only processing that needs to occur on the main thread is an instance variable reassignment, or some kind of array appending, just do that on the main thread, using the data you retrieved on the back end. It's not heavy. If it is heavy, then you're going to need more sophisticated synchronization methods to protect your data.
Normally the pattern looks like:
dispatch_async(getBackgroundQueue(), {
var theData = getTheDataFromNetwork();
dispatch_async(dispatch_get_main_queue() {
self.data = theData // Update the instance variable of your ViewController
self.tableView.reloadData() // Or some other 'reload' method
});
})
So where you'd normally refresh a table view or notify your ViewController that the operation has completed (or that local data has been updated), you should continue your main-thread processing.

UITableViewController with Section and Parse as BackEnd server

I don't find any good and update answers about loading data from Parse to an UITableViewController with sections.
I know that using PFQueryTableViewController is only possible for 1 section.
I have a class Recipes with the following columns in Parse:
Section; Name; Calories
Hence my database looks like this
Morning; Eggs; 120
Morning, Bacon; 250
Lunch; Meat; 340
....
I compute a function to query the data from Parse like this:
func queryData(){
var query = PFQuery(className: self.recipesClass as String)
//query.cachePolicy = .CacheElseNetwork
query.orderByDescending("createdAt")
query.findObjectsInBackgroundWithBlock {
(objects: [AnyObject]?, error: NSError?) -> Void in
if error == nil {
// Results were successfully found, looking first on the
// network and then on disk.
// Do something with the found objects
if let objects = objects as? [PFObject] {
for object in objects {
self.tableSections.addObject(object)
}
} else {
// The network was inaccessible and we have no cached data for
// this query.
}
}
}
Where tableSections is a NSMutableArray.
From here, I'm a bit lost on how to proceed to achieve the required results.
Please help,
Thank you in advance
You will want to create different sections based on the 'Section' property of each returned object. This is a bit tricky for someone new to UITableView, but it can be accomplished in around an hour with Sensible TableViews (http://sensiblecocoa.com/).
Sensible TableViews will let you take an array of objects and separate it into several sections. It even interfaces directly with Parse.com if you so desire. I use it in almost all of my apps for its simple, clean approach to tables and cloud data.
Start with their online guide here:
http://sensiblecocoa.com/usermanual/latest/
You can also skip right to Parse.com Integration here: http://sensiblecocoa.com/usermanual/latest/#ExploringParseComBinding

Error creating a separate NSManagedObjectContext

Before getting into my issue, please have a look at this image.
Here is the actual data model:
I retrieve a set of Records from a web API, create objects out of them, save them in core data and display them in the Today view. By default these records are returned for the current date.
The user can tap on Past button to go to a separate view where he can choose a past or future date from a date picker view and view Records for that selected date. This means I have to call the API again passing the selected date, retrieve the data and save that data in core data and display them. When the user leaves this view, this data should be discarded.
This is the important part. Even though I get a new set of data, the old original data for the current date in the Today view must not go away. So if/when the user returns to the Today view, that data should be readily available as he left it without the app having to call the API and get the data for the current date again.
I thought of creating a separate NSManagedObjectContext to hold these temporary data.
I have a separate class called DatabaseManager to handle core data related tasks. This class initializes with an instance of `NSManagedObjectContext. It creates the managed object classes in the given context.
import CoreData
import Foundation
import MagicalRecord
import SwiftyJSON
public class DatabaseManager {
private let context: NSManagedObjectContext!
init(context: NSManagedObjectContext) {
self.context = context
}
public func insertRecords(data: AnyObject, success: () -> Void, failure: (error: NSError?) -> Void) {
let json = JSON(data)
if let records = json.array {
for recordObj in records {
let record = Record.MR_createInContext(context) as Record
record.id = recordObj["Id"].int
record.name = recordObj["Name"].string!
record.date = NSDate(string: recordObj["Date"].string!)
}
context.MR_saveToPersistentStoreAndWait()
success()
}
}
}
So in the Today view I pass NSManagedObjectContext.MR_defaultContext() to insertRecords() method. I also have a method to fetch Records from the given context.
func fetchRecords(context: NSManagedObjectContext) -> [Record]? {
return Record.MR_findAllSortedBy("name", ascending: true, inContext: context) as? [Record]
}
The data is retrieved from the API, saved in core data and gets displayed successfully. All good so far.
In the Past View, I have to do basically the same thing. But since I don't want the original data to change. I tried to do this a few ways which MagicalRecord provides.
Attempt #1 - NSManagedObjectContext.MR_context()
I create a new context with NSManagedObjectContext.MR_context(). I change the date in Past view, the data for that selected date gets retrieved and saved in the database successfully. But here's the issue. When I fetch the objects from core data, I get that old data as well. For example, each day has only 10 records. In Today view I display 10 records. When the fetch objects in the Past view, I get 20 objects! I assume it's the old 10 objects plus the new ones. Also when I try to display them in the tableview, it crashes with a EXC_BAD_ACCESS error in the cellForRowAtIndexPath method.
override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCellWithIdentifier("Cell", forIndexPath: indexPath) as UITableViewCell
let record = records[indexPath.row]
cell.textLabel?.text = record.name // EXC_BAD_ACCESS
cell.detailTextLabel?.text = record.date.toString()
return cell
}
Attempt #2 - NSManagedObjectContext.MR_newMainQueueContext()
The app crashes when I change the date with the following error.
'+entityForName: nil is not a legal NSPersistentStoreCoordinator for searching for entity name 'Record''
Attempt #3 - NSManagedObjectContext.MR_contextWithParent(NSManagedObjectContext.MR_defaultContext())
Same result as Attempt #1.
Attempt #4 - From Hal's Answer I learned that even though I create two MOCs, they both refer to the same NSPersistentStore. So I created another new store to hold the temporary data in my AppDelegate.
MagicalRecord.setupCoreDataStackWithStoreNamed("Records")
MagicalRecord.setupCoreDataStackWithStoreNamed("Records-Temp")
Then when I change the date to get the new data, I set that temporary store as the default store like this.
func getDate(date: NSDate) {
let url = NSPersistentStore.MR_urlForStoreName("Records-Temp")
let store = NSPersistentStore(persistentStoreCoordinator: NSPersistentStoreCoordinator.MR_defaultStoreCoordinator(), configurationName: nil, URL: url, options: nil)
NSPersistentStore.MR_setDefaultPersistentStore(store)
let context = NSManagedObjectContext.MR_defaultContext()
viewModel.populateDatabase(date, context: context)
}
Note that I'm using the default context. I get the data but it's the same result as Attempt 1 and 3. I get 20 records. They include data from both the old date and the new date. If I use NSManagedObjectContext.MR_context(), it would simply crash like in Attempt 1.
I also discovered something else. After creating the stores in App Delegate, I printed out the default store name println(MagicalRecord.defaultStoreName()) in the Today's view. Strangely it didn't print the name I gave the store which is Records. Instead it showed Reports.sqlite. Reports being the project's name. Weird.
Why do I get the old data as well? Am I doing something with when initializing a new context?
Sorry if my question is a little confusing so I uploaded a demo project to my Dropbox. Hopefully that will help.
Any help is appreciated.
Thank you.
Thread Safety
First of all I want to mention the Golden Rule of Core Data. NSManagedObject's are not thread safe, hence, "Thou shalt not cross the streams" (WWDC). What this means is that you should always access a Managed Object in its context and never pass it outside of its context. This is why your importer class worries me, you are inserting a bunch of objects into a context without guaranteeing that you are running the insert inside the Context.
One simple code change would fix this:
public func insertRecords(data: AnyObject, success: () -> Void, failure: (error: NSError?) -> Void) {
let json = JSON(data)
context.performBlock { () -> Void in
//now we are thread safe :)
if let records = json.array {
for recordObj in records {
let record = Record.MR_createInContext(context) as Record
record.id = recordObj["Id"].int
record.name = recordObj["Name"].string!
record.date = NSDate(string: recordObj["Date"].string!)
}
context.MR_saveToPersistentStoreAndWait()
success()
}
}
}
The only time you don't need to worry about this is when you are using the Main Queue Context and accessing objects on the main thread, like in tableview's etc.
Don't forget that MagicalRecord also has convenient save utilities that create context's ripe for saving :
MagicalRecord.saveWithBlock { (context) -> Void in
//save me baby
}
Displaying Old Records
Now to your problem, the following paragraph in your post concerns me:
The user can tap on Past button to go to a separate view where he can
choose a past or future date from a date picker view and view Records
for that selected date. This means I have to call the API again
passing the selected date, retrieve the data and save that data in
core data and display them. When the user leaves this view, this data
should be discarded.
I don't like the idea that you are discarding the information the user has requested once they leave that view. As a user I would expect to be able to navigate back to the old list and see the results I just queried without another unecessary network request. It might make more sense to maybe have a deletion utility that prunes your old objects on startup rather than while the user is accessing them.
Anyways, I cannot illustrate how important it is that you familiarize yourself with NSFetchedResultsController
This class is intended to efficiently manage the results returned from
a Core Data fetch request.
You configure an instance of this class using a fetch request that
specifies the entity, optionally a filter predicate, and an array
containing at least one sort ordering. When you execute the fetch, the
instance efficiently collects information about the results without
the need to bring all the result objects into memory at the same time.
As you access the results, objects are automatically faulted into
memory in batches to match likely access patterns, and objects from
previous accessed disposed of. This behavior further serves to keep
memory requirements low, so even if you traverse a collection
containing tens of thousands of objects, you should never have more
than tens of them in memory at the same time.
Taken from Apple
It literally does everything for you and should be your go-to for any list that shows objects from Core Data.
When I fetch the objects from core data, I get that old data as well
Thats to be expected, you haven't specified anywhere that your fetch should include the reports in a certain date range. Here's a sample fetch:
let fetch = Record.MR_createFetchRequest()
let maxDateForThisController = NSDate()//get your date
fetch.predicate = NSPredicate(format: "date < %#", argumentArray: [maxDateForThisController])
fetch.fetchBatchSize = 10// or an arbitrary number
let dateSortDescriptor = NSSortDescriptor(key: "date", ascending: false)
let nameSortDescriptor = NSSortDescriptor(key: "name", ascending: true)
fetch.sortDescriptors = [dateSortDescriptor,nameSortDescriptor]//the order in which they are placed in the array matters
let controller = NSFetchedResultsController(fetchRequest: fetch,
managedObjectContext: NSManagedObjectContext.MR_defaultContext(),
sectionNameKeyPath: nil, cacheName: nil)
Importing Discardable Records
Finally, you say that you want to see old reports and use a separate context that won't save to the persistent store. Thats also simple, your importer takes a context so all you would need to do is make sure that your importer can support imports without saving to the persistent store. That way you can discard the context and the objects will go with it. So your method signature could look like this:
public func insertRecords(data: AnyObject, canSaveToPersistentStore: Bool = true,success: () -> Void, failure: (error: NSError?) -> Void) {
/**
Import some stuff
*/
if canSaveToPersistentStore {
context.MR_saveToPersistentStoreWithCompletion({ (complete, error) -> Void in
if complete {
success()
} else {
error
}
})
} else {
success()
}
}
The old data that was in your persistent store, and addressed with the original MOC, is still there, and will be retrieved when the second MOC does a fetch. They're both looking at the same persistent store. It's just that the second MOC also has new data fetched from your API.
A synchronous network operation saving to Core Data will hang your app, and (for a large enough set of records) cause the system to kill your app, appearing to the user as a crash. Your client is wrong on that point, and needs to be educated.
Break apart your logic for fetching, saving, and viewing. Your view that shows a particular date's records should just do that--which it can do, if it accepts a date and uses a predicate.
Your 'cellForRowAtIndexPath' crash smells like a problem with a missing or misspelled identifier. What happens if you hard code a string instead of using 'record.name'?

Resources