Convert asynchronously large no of GPS data to city/country - ios

Background: I have a list of contacts, which are retrieved on an asynchronous queue from a cloud based database. Once done, I dispatch back to the main queue and show these contacts in a TableView.
Besides names and other details, each contact object has GPS coordinate properties (latitude and longitude). I want to use these GPS coordinates to retrieve the name of the city and country of each contact and update the TableView showing that information in the local language of the device the user has.
Problem: The problem I am trying to overcome is that I have a few hundred contacts. Initially, I used concurrent queues to get the city/country strings. But I paused the app in XCode and realised the disaster of 300+ threads created. So I changed the code running each lookup of the city/country on the same serial queue.
The issue now is that only for about 50 of the contacts the data is updated. I do not know why and not sure even if the queue is serial. Debugging shows there are still 100+ “serial queues” created. I expected one at the time. What am I doing wrong? Thanks
The code for the class, in which I have my TableView is as follows:
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.rawValue), 0)) {
// get or update the contactsList
activeUser.contactsList.getAllContacts()
// once we have the contacts we go back to the main queue
dispatch_async(dispatch_get_main_queue()) {
// and refresh tableview to show the contacts
self.tableView.reloadData()
// now we refresh the locations of the contacts
for index in 0...activeUser.contactsList.listOfContacts.count
{
// set local variables for lat and long
let lat = activeUser.contactsList.listOfContacts[index].latitude
let long = activeUser.contactsList.listOfContacts[index].longitude
// call location service method with completion handler
MyLocationServices().updateLocationToLocalLanguage(lat, longitude: long, completionHandler:
{ (city, country) -> () in
// update contact's details
activeUser.contactsList.listOfContacts[index].city = city
activeUser.contactsList.listOfContacts[index].country = country
// refresh each time the table to show updated contact data
self.tableView.reloadData()
})
}
}
The method within the MyLocationServices class looks as follows:
func updateLocationToLocalLanguageDispatched(latitude: String, longitude: String, completionHandler: (city: String, country: String) -> ())
{
let serialQ = dispatch_queue_create("AddressUpdateQ", DISPATCH_QUEUE_SERIAL)
dispatch_async(serialQ)
{
var cityString = "NA"
var countryString = "NA"
let group = dispatch_group_create()
dispatch_group_enter(group)
let location = CLLocation(latitude: Double(latitude)!, longitude: Double(longitude)!)
self.geocoder.reverseGeocodeLocation(location)
{
(placemarks, error) -> Void in
if let placemarks = (placemarks as [CLPlacemark]!) where placemarks.count > 0
{
let placemark = placemarks[0]
if ((placemark.addressDictionary!["City"]) as? String != nil) { cityString = ((placemark.addressDictionary!["City"]) as? String)! }
if ((placemark.addressDictionary!["Country"]) as? String != nil) { countryString = ((placemark.addressDictionary!["Country"]) as? String)! }
}
dispatch_group_leave(group)
}
dispatch_group_wait(group, DISPATCH_TIME_FOREVER)
dispatch_async(dispatch_get_main_queue())
{
completionHandler(city: cityString, country: countryString)
}
}
}

I have solved the issue and documenting hoping it is helpful in case others face similar problems.
(1) Serial Queue: I have realised serial queue initiation should not happen within the method, but enclose the for-loop. Thus the following code should be removed from the method and put around the for-loop. But this does not solve the fact that I only get about 50 results and nothing for the remaining GPS data of the contacts.
let serialQ = dispatch_queue_create("AddressUpdateQ", DISPATCH_QUEUE_SERIAL);
dispatch_async(serialQ)
{ [for index....] }
(2) Location: The second issue is related to Apple's Geocoder, which happens online. Officially, the number of requests are limited. The documentation states that
one "should not send more than one geocoding request per minute".
Applications should be conscious of how they use geocoding. Geocoding
requests are rate-limited for each app, so making too many requests in
a short period of time may cause some of the requests to fail.
Checking the error code I get, it is Error Domain=kCLErrorDomain Code=2. It seems the service is denied and I cannot make further requests.
(3) Solutions: There are two solutions in my view. One is to update 30 or so users, wait some time, and then dispatch after that another queue to get again some data. The second solution is to live with having all data in English, make a string that each user saves in the cloud. So for other users retrieving contacts, it's just a matter of fetching a string from the database rather than checking GPS data "on the fly".

Related

Which to use, addSnapshotListener() or getDocuments()

I'm a little bit confused about addSnapshotListener and getDocuments. As I read in the firebase docs, getDocuments() is retrieving data once and addSnapshotListener is retrieving in real-time.
What I want to ask.
If I'm using getDocuments, and im changing some documents in the Firestore , it will not make the change in the app ? But if im using addSnapshotListener it will ?
I'm making an delivery app, which is the best to use to store pictures of food , descriptions etc.
This is what im using to retrieve labels and pictures from my app :
db.collection("labels").getDocuments { (snapshot, error) in
if let error = error {
print(error)
return
} else {
for document in snapshot!.documents {
let data = document.data()
let newEntry = Labels(
firstLabel: data["firstLabel"] as! String,
secondLabel: data["secondLabel"] as! String,
photoKey: data["photoKey"] as! String
)
self.labels
.append(newEntry)
}
}
DispatchQueue.main.async {
self.tableViewTest.reloadData()
}
getDocuments will return results one time, with the current Firestore data.
addSnapshotListener will return an initial result set (same as getDocuments) and get called any time that data changes.
If your data is modified in Firestore and you've used getDocuments, your app will not be notified of those changes. For example, in your delivery app, perhaps the item goes out-of-stock while the user is using it. Or, the price gets changed, the user is logged in from another device, etc -- many possibilities for why the data might change. By using a snapshot listener, you'd get notified if any of these changes happen.
However, if you're relatively confident you don't need updates to the data (like getting a user's address from the database, for example), you could opt to just use getDocuments.

Firestore in Swift - Listener producing needless reads

I have an app that uses a snapshot listener to listen to data in a particular document. However, when a field in the document is updated, the data is read 7-10x over. Never read once, and never read the number of fields that are in my document, it always seems to be an arbitrary number. Also, when the read data prints out, it seems like every printout is the same except for a couple of fields that I'm not setting (like an array prints out "<__NSArrayM 0x282d9f240>" but the number changes on each print). As a result, minimal usage of my app is causing 5-10k reads. I'm trying to reduce the number of reads and I don't know exactly how, but the app has to read as data is updated, but my two questions are:
when I print the data from the listener, does each data print out signify a separate read operation? and
is there any way for the listener to be alerted of the update but wait to actually perform the read until the data is updated, then perform one read instead of multiple reads every time any field is updated? Or another strategy to reduce reads when multiple writes occur?
Not sure if this is helpful, but here is the code I'm using to perform the read...its pretty much the standard code from the firestore sdk:
env.db.collection(env.currentSessionCode!).document(K.FStore.docName).addSnapshotListener { [self] documentSnapshot, error in
guard let document = documentSnapshot else {
print("Error fetching snapshot: \(error!)")
return
}
guard let data = document.data() else {
print("Document data was empty.")
return
}
self.env.data1 = data[K.FStore.data1] as? String ?? "????"
self.env.data2 = data[K.FStore.data2] as? String ?? "????"
self.env.data3 = data[K.FStore.data3] as? [String] ?? ["????"]
self.env.data4 = data[K.FStore.data4] as? [String] ?? ["????"]
self.env.data5 = data[K.FStore.data5] as? Double ?? 0
self.env.data6 = data[K.FStore.data6] as? Double ?? 0
self.env.data7 = data[K.FStore.data7] as! Bool
self.env.data8 = data[K.FStore.data8] as! Bool
print("Current data: \(data)")
Update - For clarification, the way I have been updating my data to firebase is with a environment object, and using "didSet" when the new data is changed/updated in the environment to update it on firebase...I think this might be the root of the problem, as the function called on didSet runs 4-5 times each time it is called...
relevant code:
#Published var data1: String {
didSet {
postValuesToFB(fb: K.FStore.data1, string: data1)
}
}
func postValuesToFB(fb: String, string: String) {
guard let code = currentSessionCode else {
fatalError("Error - Connection Check - no value for current session code in Global Env")
}
let docRef = db.collection(code).document(K.FStore.docName)
docRef.getDocument { document, _ in
guard let document = document else {
return
}
if document.exists {
let session = self.db.collection(code).document(K.FStore.docName)
session.updateData([
fb: string,
K.FStore.dateLastAccessed: FieldValue.serverTimestamp(),
])
return
}
}
}
Based on your comments, it sounds as if you've written no code to remove a listener after it's been added. Based on this, it's relatively safe to assume that your code could be adding many listeners over time, and each one is getting called for each change.
You should take a moment to think about the architecture of your app and figure out when is the appropriate time to remove listeners when they're no longer needed. Usually this corresponds with the lifecycle of whatever component is responsible for display of the data from the query. Review the documentation for getting realtime updates, especially the section on detaching a listener. It's up to you to determine the right time to remove your listener, but you definitely don't want to "leak" a listener as you are now.
A common source of unexpected read charges for developers who are new to Firestore is the Firebase console itself. When that console displays Firestore content, you are charged for those read too. To ensure you measure the impact of your code correctly, test it with the Firebase console closed.
when I print the data from the listener, does each data print out signify a separate read operation?
Not really. You get charged for a document read, when the document is read on your behalf on the server. You are not charted for printing the same DocumentSnapshot multiple times.
is there any way for the listener to be alerted of the update but wait to actually perform the read until the data is updated
Nope. To know the document has changed, the server needs to read it. So that requires a charged read operation.

Firebase observe slow for large database

I am working on an app displaying places (downloaded from firebase) based on user location.
I have currently 5k entries and they are displayed in about 10seconds.
I plan to have 80k entries and I don't want users to wait that long.
What I did :
I created a Place class, I do 'observe'(.value) on my firebase ref and on each child I put each element in an attribute of the Place class.
Then the place:Place = Place(attributes) id added to an array:Place until all places have been downloaded.
self.ref.queryOrderedByKey().observe(.value, with: {(snapshot) in
if snapshot.childrenCount > 0 {
for place in snapshot.children.allObjects as! [DataSnapshot] {
When all places are in the array I compare places locations with the user location and sort the array to display them by distance in a tableview.
What I tried:
I also tried to use GeoFire but it is slower.
How the db looks like (80k elements) :
{
"users": {
"DFkjdhfgYG": {
"id":"DFkjdhfgYG"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
},
"BVvfdTRZ": {
"id":"BVvfdTRZ"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
}
}
}
Now I don't know what to do and I absolutely need to user Firebase.
Can you help me to improve the way I download firebase db elements, or to show me another way to do it, to make the whole process faster ?
Thanks !
You're using a for loop in a function that is being called the same number of times as there are children in your database path, making the for loop completely useless and overkill, which can add extra time to the whole process.
Another thing that you can do is have this be called on a different thread and making it the highest priority over the rest of your code. Here's how to do both of those:
func handleFirebase() {
DispatchQueue.global(qos: .userInteractive).async {
self.ref.queryOrderedByKey().observe(.value, with: { (snapshot) in
guard let value = snapshot.value as? String else { return }
let key = snapshot.key
print("KEY: \(key), VALUE: \(value)")
}, withCancel: nil)
}
}

Error acessing list of Realm's objects: Realm accessed from incorrect thread

I have a list of points of interest. This points were loaded from a Realm database. Each point should present its distance to the user's position.
Each time I get a new location, I calculate the distance to all points. To avoid a frozen screen, I was doing the math in a background thread, after i display the list in a table in the main thread.
func updatedLocation(currentLocation: CLLocation) {
let qualityOfServiceClass = QOS_CLASS_BACKGROUND
let backgroundQueue = dispatch_get_global_queue(qualityOfServiceClass, 0)
dispatch_async(backgroundQueue, {
for point in self.points{
let stringDistance = self.distanceToPoint(currentLocation, destination: point.coordinate)
point.stringDistance = stringDistance
}
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.tableView?.reloadData()
})
})
}
However I get this error:
libc++abi.dylib: terminating with uncaught exception of type realm::IncorrectThreadException: Realm accessed from incorrect thread.
I know i am getting this error because I'm accessing the realm objects in a background thread, however, they are already loaded into an array and I never make a new query to the database.
In addition, the var i'm updating his not saved into the database.
Any idea how to solve this? I wanted to avoid doing the math in the main thread.
thanks in advance
I assume you wrap Realm Results objects into Array like the following:
let results = realm.objects(Point)
self.points = Array(results)
However, that is not enough. Because each element in the array is still tied with Realm, that cannot be access another thread.
A recommended way is re-create Realm and re-fetch the Results each threads.
dispatch_async(backgroundQueue, {
let realm = try! Realm()
let points = realm.objects(...)
try! realm.write {
for point in points{
let stringDistance = self.distanceToPoint(currentLocation, destination: point.coordinate)
point.stringDistance = stringDistance
}
}
dispatch_async(dispatch_get_main_queue(), { () -> Void in
...
})
})
Realm objects have live-update feature. When committed changed to Realm objects on sub-thread, those changes reflect to the objects in other thread immediately. So you do not need to re-fetch the query in the main thread. What you should do is just reload the table view.
If you'd like to wrap array and pass it to other thread directly, you should wrap all elements of resutls as follows:
let results = realm.objects(Point)
self.points = results.map { (point) -> Point in
return Point(value: point)
}

Asynchronous function not updating variable

var endLat: Double = 0.0
var endLong: Double = 0.0
func forwardGeocodingStarting(address: String) {
CLGeocoder().geocodeAddressString(address, completionHandler: { (placemarks, error) in
if error != nil {
print(error)
return
}
let placemark = placemarks?[0]
let location = placemark?.location
let coordinate = location?.coordinate
dispatch_async(dispatch_get_main_queue()) {
startLat = (coordinate?.latitude)!
startLong = (coordinate?.longitude)!
print("\(startLat) is lat and \(startLong) is long")
}
})
}
Hello, here is a geocoding function that I created, that simply takes an address and returns the address's coordinates. My problem is, that at the end of the code section, when I do print(endLat) it prints out 0, however when I do it in the dispatch_async in the function, it comes out to the proper latitude.
I realize this is an asynchronous function, which is why I tried to use the dispatch_async to update the variable instantly.
My question is, how can I have these variables update instantly? Thanks in advance.
You've got a little confused with what you mean by "Asynchronous function not updating variable". forwardGeocodingStarting(_:) is NOT an Async function. geocodeAddressString is an async function designed by the API to allow you to use the following values you get after the operation is done within that function only. Hence you keep getting 0 as the it doesn't wait till the operation is done completely to display the value. dispatch_async(dispatch_get_main_queue) is used only to update UI on the Main thread. You've to use a completion handler in your function like so:
typealias theDouble = (Double, Double) -> ()
func forwardGeocodingStarting(address: String, completion: theDouble){
//your code
let aVar: Double!
let bVar: Double!
let placemark = placemarks?[0]
let location = placemark?.location
let coordinate = location?.coordinate
aVar = (coordinate?.latitude)!
bVar = (coordinate?.longitude)!
completion(aVar, bVar)
}
Now when you call your function, update the variables startLat and startLong like so:
forwardGeocodingStarting(<your string address>) { firstVar, secondVar in
startLat = firstVar
startLong = secondVar
print("\(startLat) is lat and \(startLong) is long") //You will get correct values
}
You cannot have these variables update instantaneously, as you are calling an asynchronous function. Thats an obvious contradiction :-)
That you dispatch back to the main queue is fine, but that doesn't change the asynchronous behaviour. In fact your are even issuing yet another dispatch_async ... (meaning the body of that code will get run at some later point in time)
You could wait for the results to be available, e.g. using a dispatch group, but quite likely that is not what you want here.
I assume you want to use the values in some view controller? In this case you probably fill in some instance variables as the results arrive and update the UI state once both values are available.

Resources