I am working on an app displaying places (downloaded from firebase) based on user location.
I have currently 5k entries and they are displayed in about 10seconds.
I plan to have 80k entries and I don't want users to wait that long.
What I did :
I created a Place class, I do 'observe'(.value) on my firebase ref and on each child I put each element in an attribute of the Place class.
Then the place:Place = Place(attributes) id added to an array:Place until all places have been downloaded.
self.ref.queryOrderedByKey().observe(.value, with: {(snapshot) in
if snapshot.childrenCount > 0 {
for place in snapshot.children.allObjects as! [DataSnapshot] {
When all places are in the array I compare places locations with the user location and sort the array to display them by distance in a tableview.
What I tried:
I also tried to use GeoFire but it is slower.
How the db looks like (80k elements) :
{
"users": {
"DFkjdhfgYG": {
"id":"DFkjdhfgYG"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
},
"BVvfdTRZ": {
"id":"BVvfdTRZ"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
}
}
}
Now I don't know what to do and I absolutely need to user Firebase.
Can you help me to improve the way I download firebase db elements, or to show me another way to do it, to make the whole process faster ?
Thanks !
You're using a for loop in a function that is being called the same number of times as there are children in your database path, making the for loop completely useless and overkill, which can add extra time to the whole process.
Another thing that you can do is have this be called on a different thread and making it the highest priority over the rest of your code. Here's how to do both of those:
func handleFirebase() {
DispatchQueue.global(qos: .userInteractive).async {
self.ref.queryOrderedByKey().observe(.value, with: { (snapshot) in
guard let value = snapshot.value as? String else { return }
let key = snapshot.key
print("KEY: \(key), VALUE: \(value)")
}, withCancel: nil)
}
}
Related
I have a video based app. Someone suggested that I use a video recommendation system similar to any other video app that shows videos based on what a user watches. If a user scrolls through 50 videos, and 20 of them are pet related (dogs, cats etc), then as the user scrolls further they would see more animal related videos rather than non animal related videos.
When I pull posts, I paginate based on the snapshot key from newest to oldest. Each video has an attribute like -category: "pets" but that has nothing to do with the snapshot key for paging. After determining that user prefers pet video what I was thinking was as the user pages, if the post doesn't contain that attribute, skip it, but there is an issue.
The problem with using an attribute is if the first 50 videos it is determined that the user prefers pet related videos, as they scroll the next 200 videos might not have a pet attribute, which means either skip all 200 videos, or show them all. But if they have to scroll through all 200 then there isn't any point to the recommendation system.
If 10 of the next 50 videos after the 200 do have the pets attribute, using the pagination system below, there is no way to skip all 200 videos and get to the next 10 of the following 50 (basically skipping straight to keys 201 - 250).
Is there anyway to skip over batches of posts that do not contain a specific attribute?
var snapKey: String?
func paginate() {
let postsRef = Database.database().reference().child("posts")
if snapKey == nil {
postsRef
.queryOrderedByKey
.queryLimited(toLast: 20)
.obsereSingleEvent(of: value, with: (snapshot) {
guard let firstChild = snapshot.children.allObjects.first as? DataSnapshot else { return }
for child in snapshot.children.allObjects as! [DataSnapshot] {
// each child is a post, get it, fetch and attach user model, append to datasource, reloadData
}
self.snapKey = firstChild.key
})
} else {
postsRef
.queryOrderedByKey
.queryEnding(atValue: snapKey!)
.queryLimited(toLast: 21)
.obsereSingleEvent(of: value, with: (snapshot) {
guard let firstChild = snapshot.children.allObjects.first as? DataSnapshot else { return }
for child in snapshot.children.allObjects as! [DataSnapshot] {
// each child is a post, get it, fetch and attach user model, append to datasource, reloadData
}
self.snapKey = firstChild.key
})
}
}
Is there anyway to skip over batches of posts that do not contain a specific attribute?
Firebase Realtime Database doesn't have an NOT EQUAL operation. The closest equivalent is to do two queries: one for the matches before the value, the other for the matches after the value.
So something along the lines of:
let query1 = postsRef.queryOrdered(byChild: "category").queryEnding(atValue: "o~")
let query2 = postsRef.queryOrdered(byChild: "category").queryStarting(atValue: "q")
Where the ~ is just the last printable ASCII character, so that your first query ends just before the first item whose category starts with a p.
I have a query running to check .childAdded at a location in my database.
It works well when data is found, however, if there is not data at the location, it can't fire the query and therefore this does not allow me to use snapshot.exists because it doesn't even run the query.
This is my current code
let favouriteRef = self.databaseRef.child("users").child(userID!).child("Favourites")
// Code doesn't run past this line when no data at location
favouriteRef.queryOrderedByKey().observe(.childAdded, with: { (snapshot) in
let favouriteID = "\(snapshot.value!)"
let usersRef = self.databaseRef.child("users")
usersRef.observeSingleEvent(of: .value, with: { (users) in
for user in users.children {
let favCleaner = UserClass(snapshot: user as! DataSnapshot)
if favouriteID == favCleaner.uid {
tempFav.append(favCleaner)
}
}
self.hasFavourites = true
self.tableView.reloadData()
self.usersArray = tempFav
self.collectionView.reloadData()
})
})
I would like to find a way to receive a callback if the query doesn't run (no data at location)
Thanks.
If there is absolutely no data at the location, then obviously this event trigger is not sufficient for you to get the data because this event only gets triggered when something gets added.
So you have two options
Add another trigger of type .observeSingleEvent(of: .value, with: { (snapshot) in
// Get the values and write the relevant actions
}
Or you can update this event to type .observe(.value, with: {snapshot in // write the relevant code
}
Both approaches have their advantage and disadvantage.
First approach will need you to write more code while minimising the number of triggers from your database to your UI. In second approach, there will be more triggers to the UI but it can be fairly easy to code.
From what I can see, you are first trying to establish whether data exists in the favorite node of your database and then comparing it to another snapshot. So if the number of delete or update events are relatively small, my suggestion is to go for approach two.
override func viewDidLoad() {
super.viewDidLoad()
getImageUrl()
}
func getImageUrl(){
ref.child("posts").queryOrderedByKey().observeSingleEvent(of: .value) { (snapchot) in
let postsss = snapchot.value as! [String : AnyObject]
for (_,posst) in postsss {
if let uid = posst["userID"] as? String{
if uid == Auth.auth().currentUser?.uid{
if let myPostURL = posst["pathToImage"] as? String{
self.imageURLs.append(myPostURL)
}
}
}
}
}
}
I want my code to go through all the posts on Firebase and then check if their userID matches the currentusers uid, if they matched which means they are my images. then send the URL in pathToImage to an array in my code called imageURLs()[ "" ].but I don't know how to to that??. I am using SDWebImage to display my images to the collectionView cell. i have tested it and it works fine if i copy and paste a random URL in the array called imageURLs[ "URL here" ]
I am very new to Swift and Firebase, so any help would be greatly appreciated!! :)
HERE IS AN IMAGE OF MY FIREBASE FILES.
https://ibb.co/bXLMcb
Psst! posts/pathToImage holds the URL so that's the one i want to retrieve.
users/urlToImage is just a profile picture. i don't really need it right now
To locate specific data within a node a Query is used.
While iterating over the nodes works, if there's a million nodes it's going to take a long time and tie up the UI while waiting for that to happen.
The bottom line is; if you know what you are looking for - in this case a uid - then a query is a super simple solution.
let postsRef = self.ref.child("posts")
let query = postsRef.queryOrdered(byChild: "userID").queryEqual(toValue: "uid_0")
query.observeSingleEvent(of: .value) { (snapshot) in
if snapshot.exists() {
for child in snapshot.children { //.value can return more than 1 match
let snap = child as! DataSnapshot
let dict = snap.value as! [String: Any]
let myPostURL = dict["urlToImage"] as! String
self.imageURLs.append(myPostURL)
}
} else {
print("no user posts found")
}
}
With the above code, we first create a reference to our posts node. Then a query that will only return this users (uid_0) posts.
We then iterate over the snapshot of posts and add them to the array.
This is much more efficient that loading and evaluating every node in the database.
See Sorting And Filtering data for more information.
You should think about storing an index of all your users posts somewhere in your database. That way you don't need to observe all posts every time. This is called denormalization. Firebase has an article in their docs about organizing your database.
Here's some information regarding firebase filtering in swift.
The best solution is to add a separate node that has a list of "post IDs" organized by user. Then you could observe that node, and only download each post specifically by the returned ID. Here's a link about flattening data structures in firebase. It would look something like this.
"posts":{
"somePostID":{
"timestamp": "0200231023",
"postContent": "here's my post content",
"authorUID" : "0239480238402934"
} ...
},
"postsByGivenUID":{
"someAuthorID":{
"somePostID": "true",
"somePostID": "true",
},
"someOtherAuthorID":{
"somePostID": "true",
"somePostID": "true",
"somePostID": "true"
}
}
This is actually a much bigger problem than just changing how you structure your code. For scalability sake, you're going to have to reevaluate how you structure the data in firebase altogether.
Then, you can nest your firebase query, sort of like this:
ref.child("postByGivenUID").child("Auth.auth().currentUser?.uid").observe(.childAdded) { (snapshot) in
ref.child("posts").child(snapshot.value).observeSingleEventOfType(of: .value) { (snap) in
// your actual post data will be here
// that way you won't be downloadin ALL posts EVERY time
}
}
My code is:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
})
the key is equal to a post key(-Kef2J6vJBCjHGYjSlkI). I want to retrieve only one element at a time.
There is only 1 dictionary under that child:
I need to create a dictionary out of the contents of this. However, when I implement regular algorithm:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] {
for snap in snapshot {
print("Zhenya: here is the snap: \(snap)")
if let postDict = snap.value as? Dictionary<String, AnyObject> {
let key = snap.key
let post = Post(postKey: key, postData: postDict)
self.posts.append(post)
} else {
print("Zhenya: failed to convert")
}
}
}
})
The 'snap' prints elements of the dictionary itself, not the whole dictionary. So it always fails the conversion on the following line.
Bonus for ninjas:
If you can suggest a better architecture, I would appreciate that.
It seems that retrieving posts from FB one by one will slow down performance considerably.
The reason it is 'one by one' is this one post is the result of user's GeoFire specification. Geofire returns keys to posts that are nearby.
I want to filter Firebase's main branch of posts based on those returned keys, but there is no way to do that - only load one by one (.orderByChild - doesn't returns posts, but instead, orders all the posts. If you you know how it can help me, please advice)
This is the code provided in your question:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] {
for snap in snapshot {
print("Zhenya: here is the snap: \(snap)")
if let postDict = snap.value as? Dictionary<String, AnyObject> {
let key = snap.key
let post = Post(postKey: key, postData: postDict)
self.posts.append(post)
} else {
print("Zhenya: failed to convert")
}
}
}
})
Let's take a look at what's happening here.
The first line is observing posts/$post-id in Firebase using the .value method. The .value method returns the reference you provide in a FIRDataSnapshot, where snapshot.key is the name of the child and snapshot.value is a dictionary containing its children.
So, snapshot in DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in ... }) should look like this:
$post-id
imageUrl: http://...
likes: 0
userKey: ...
But on the very next line you redefine snapshot using a local definition. That is why you are getting the children of the post, because you explicitly define it that way:
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] { ... }
If you remove this, you should see the behavior you want.
So what should you really be doing? Well I don't like using .value because it has a few drawbacks:
It downloads the entire node requested. If you mess up your reference for a large node (i.e, ref/users), it's easy for a .value operation to take several seconds before you see any data.
It is only called once. This is fine if your database is static, but whose database is static?
This is why I prefer using .childAdded for things like posts, messages, etc. It returns a single node at a time, which means you can populate a list progressively as new data becomes available. Using a few clever UI functions (such as tableView.insertRows(...) you can fairly smoothly display large lists of data without waiting for results. I believe it also supports paging, where you get them in batches of 10, for example. Here is a quick primer on how you can begin using .childAdded to make the most of large lists of data.
My realtime database looks like this:
It contains only 1 child as you can see. I had 4 more children in RunningGames a few minutes before. I deleted them in the browser. When now calling this:
private lazy var runningGamesRef: FIRDatabaseReference = FIRDatabase.database().reference().child("RunningGames")
self.runningGamesRef.observeSingleEvent(of: .value, with: { (snapshot) -> Void in
for gameSnap in snapshot.children {
let id = (gameSnap as! FIRDataSnapshot).key
print(id)
}
})
It still prints those games I deleted in the browser. Calling runningGameRef!.removeValues() in my app does deletes it in the browser and on the iPhone (the print(id) is fixed). I have this error on multiple observeSingleEvent functions on different children, not only children of RunningGames. What would cause this annoying error?
Some children in RunningGames also have children, but they do auto remove themselves in the app. However, these values are also still visible when calling observeSingleEvent.
It's probably your local cache that's still holding the outdated info. This often happens when you're manipulating data from multiple sources.
I would try using observe instead of observeSingleEvent. I know it's a bit odd (and not really what you want if you only want to load the data once) but that should keep your info up to date.
Maybe by doing this you could fetch the info just once.
var handle: UInt = 0
handle = ref.observe(.value, with: { snapshot in
for gameSnap in snapshot.children {
let id = (gameSnap as! FIRDataSnapshot).key
print(id)
}
ref.removeObserver(withHandle: handle)
})
Source of the code (Frank van Puffelen)