I have a video based app. Someone suggested that I use a video recommendation system similar to any other video app that shows videos based on what a user watches. If a user scrolls through 50 videos, and 20 of them are pet related (dogs, cats etc), then as the user scrolls further they would see more animal related videos rather than non animal related videos.
When I pull posts, I paginate based on the snapshot key from newest to oldest. Each video has an attribute like -category: "pets" but that has nothing to do with the snapshot key for paging. After determining that user prefers pet video what I was thinking was as the user pages, if the post doesn't contain that attribute, skip it, but there is an issue.
The problem with using an attribute is if the first 50 videos it is determined that the user prefers pet related videos, as they scroll the next 200 videos might not have a pet attribute, which means either skip all 200 videos, or show them all. But if they have to scroll through all 200 then there isn't any point to the recommendation system.
If 10 of the next 50 videos after the 200 do have the pets attribute, using the pagination system below, there is no way to skip all 200 videos and get to the next 10 of the following 50 (basically skipping straight to keys 201 - 250).
Is there anyway to skip over batches of posts that do not contain a specific attribute?
var snapKey: String?
func paginate() {
let postsRef = Database.database().reference().child("posts")
if snapKey == nil {
postsRef
.queryOrderedByKey
.queryLimited(toLast: 20)
.obsereSingleEvent(of: value, with: (snapshot) {
guard let firstChild = snapshot.children.allObjects.first as? DataSnapshot else { return }
for child in snapshot.children.allObjects as! [DataSnapshot] {
// each child is a post, get it, fetch and attach user model, append to datasource, reloadData
}
self.snapKey = firstChild.key
})
} else {
postsRef
.queryOrderedByKey
.queryEnding(atValue: snapKey!)
.queryLimited(toLast: 21)
.obsereSingleEvent(of: value, with: (snapshot) {
guard let firstChild = snapshot.children.allObjects.first as? DataSnapshot else { return }
for child in snapshot.children.allObjects as! [DataSnapshot] {
// each child is a post, get it, fetch and attach user model, append to datasource, reloadData
}
self.snapKey = firstChild.key
})
}
}
Is there anyway to skip over batches of posts that do not contain a specific attribute?
Firebase Realtime Database doesn't have an NOT EQUAL operation. The closest equivalent is to do two queries: one for the matches before the value, the other for the matches after the value.
So something along the lines of:
let query1 = postsRef.queryOrdered(byChild: "category").queryEnding(atValue: "o~")
let query2 = postsRef.queryOrdered(byChild: "category").queryStarting(atValue: "q")
Where the ~ is just the last printable ASCII character, so that your first query ends just before the first item whose category starts with a p.
Related
Background
I am currently building a task manager feature in my iOS app. Users can add/complete tasks for each day of the week.
When a user goes to add a task under a certain day, a reference is made to that specific day, and the task is listed under that day in Firebase RTD.
let ref = Database.database().reference(withPath: "users").child("profile").child(uid).child("todos").child(String(self.getDate(self.dayNum)))
(The getDate function is what I use to get the current day in terms of numbers. Ex Feb 18 would be "18")
Adding and storing the tasks works completely fine. The tasks are always stored under the correct users profile, todo list path, and day.
Problem
The issue comes when a user goes to add a task to a day with no previous tasks stored in the database.
When you toggle between different days, such as going from the "26" to the "27", if there are previous tasks stored in the database then it will load each days tasks fine and update the table accordingly.
But this is only the case if the days have previous tasks that are already stored in Firebase RTD.
If you have no tasks for a certain day stored in Firebase, the UiTableView simply presents the tasks stored for the previous day you had selected because it cannot find the path in RTD.
For example, if you had clicked on "27", and it had two tasks stored for that day, then moved to day "28", which had no tasks stored for that day, it would present the tasks from day 27 on day 28.
I understand the reason for this is that the reference cannot find the path in Firebase RTD when going to load the tasks, since there are no tasks to load and thus the path does not exist.
I attempted to comment next to my code to explain everything that is happening
func loadTodos(n: Int){
self.todos.removeAll() //removes all tasks from the array "todos"
guard let uid = Auth.auth().currentUser?.uid else { return } //gets current user uid
let ref = Database.database().reference(withPath: "users").child("profile").child(uid).child("todos").child(String(getDate(n)) //creates path to Firebase for specific day
ref.observeSingleEvent(of: .value) { (snapshot) in
for child in snapshot.children.allObjects as! [DataSnapshot]{
let todoName = child.key
let todoRef = ref.child(todoName)
todoRef.observeSingleEvent(of: .value) { (todoSnapshot) in
let value = todoSnapshot.value as? NSDictionary
let isChecked = value!["isChecked"] as? Bool
self.todos.append(Todo(isChecked: isChecked!, todoName: todoName)) //adds new todo to array "todos"
self.todoTV.reloadData() //reloads UITableView "todoTV"
}
}//end of for
}//end of .observeSingleEvent
}//end of func
Question
I am looking to see if there is a way you can detect that the path does not exist, and present some type of message instead of the wrong days tasks.
I think it could be something as easy as an if/else statement to detect if the path exists, and if not present a message. I am just not sure how to do it.
I am open to any other solution really.
My goal is that I would ideally like to place a message under the days with no tasks, such as "Add a task", that would disappear once a task is added.
Whats Happening
You're only reloading your tableView in the for loop.
That means if the for loop has zero items, your table never gets reloaded.
How to Fix It
Here's where you're currently reloading your tableView.
todoRef.observeSingleEvent(of: .value) { (todoSnapshot) in
let value = todoSnapshot.value as? NSDictionary
let isChecked = value!["isChecked"] as? Bool
self.todos.append(Todo(isChecked: isChecked!, todoName: todoName)) //adds new todo to array "todos"
self.todoTV.reloadData() //reloads UITableView "todoTV"
}
Instead of having it only in the second callback. Add it to the first one too. Like this:
func loadTodos(n: Int){
self.todos.removeAll() //removes all tasks from the array "todos"
guard let uid = Auth.auth().currentUser?.uid else { return } //gets current user uid
let ref = Database.database().reference(withPath: "users").child("profile").child(uid).child("todos").child(String(getDate(n)) //creates path to Firebase for specific day
ref.observeSingleEvent(of: .value) { (snapshot) in
for child in snapshot.children.allObjects as! [DataSnapshot]{
let todoName = child.key
let todoRef = ref.child(todoName)
todoRef.observeSingleEvent(of: .value) { (todoSnapshot) in
let value = todoSnapshot.value as? NSDictionary
let isChecked = value!["isChecked"] as? Bool
self.todos.append(Todo(isChecked: isChecked!, todoName: todoName)) //adds new todo to array "todos"
self.todoTV.reloadData() //reloads UITableView "todoTV"
}
}//end of for
// ----- ----- ----- ----- ----- ----- ----- -----
self.todoTV.reloadData() // ---- Reload Data Here Too ----
// ----- ----- ----- ----- ----- ----- ----- -----
}//end of .observeSingleEvent
}//end of func
This way if the loop has no items, the table will still reload with the empty data.
I am working on an app displaying places (downloaded from firebase) based on user location.
I have currently 5k entries and they are displayed in about 10seconds.
I plan to have 80k entries and I don't want users to wait that long.
What I did :
I created a Place class, I do 'observe'(.value) on my firebase ref and on each child I put each element in an attribute of the Place class.
Then the place:Place = Place(attributes) id added to an array:Place until all places have been downloaded.
self.ref.queryOrderedByKey().observe(.value, with: {(snapshot) in
if snapshot.childrenCount > 0 {
for place in snapshot.children.allObjects as! [DataSnapshot] {
When all places are in the array I compare places locations with the user location and sort the array to display them by distance in a tableview.
What I tried:
I also tried to use GeoFire but it is slower.
How the db looks like (80k elements) :
{
"users": {
"DFkjdhfgYG": {
"id":"DFkjdhfgYG"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
},
"BVvfdTRZ": {
"id":"BVvfdTRZ"
,"key2":"value"
,"key3":"value"
,"key4":"value"
,"key5":"value"
,"key6":"value"
,"key7":"value"
,"key8":"value"
,"key9":"value"
,"key10":"value"
,"key11":"value"
,"key12":value
,"key13":value
,"key14":"value"
,"key15":"value"
,"key16":"value"
,"key17":"value"
,"key18":"value"
,"key19":"value"
,"key20":"value"
,"key21":value
,"key22":value
,"key23":value
,"key24":value
,"key25":value
,"key26":"value"
,"key27":value
,"key28":value
,"key29":"value"
}
}
}
Now I don't know what to do and I absolutely need to user Firebase.
Can you help me to improve the way I download firebase db elements, or to show me another way to do it, to make the whole process faster ?
Thanks !
You're using a for loop in a function that is being called the same number of times as there are children in your database path, making the for loop completely useless and overkill, which can add extra time to the whole process.
Another thing that you can do is have this be called on a different thread and making it the highest priority over the rest of your code. Here's how to do both of those:
func handleFirebase() {
DispatchQueue.global(qos: .userInteractive).async {
self.ref.queryOrderedByKey().observe(.value, with: { (snapshot) in
guard let value = snapshot.value as? String else { return }
let key = snapshot.key
print("KEY: \(key), VALUE: \(value)")
}, withCancel: nil)
}
}
override func viewDidLoad() {
super.viewDidLoad()
getImageUrl()
}
func getImageUrl(){
ref.child("posts").queryOrderedByKey().observeSingleEvent(of: .value) { (snapchot) in
let postsss = snapchot.value as! [String : AnyObject]
for (_,posst) in postsss {
if let uid = posst["userID"] as? String{
if uid == Auth.auth().currentUser?.uid{
if let myPostURL = posst["pathToImage"] as? String{
self.imageURLs.append(myPostURL)
}
}
}
}
}
}
I want my code to go through all the posts on Firebase and then check if their userID matches the currentusers uid, if they matched which means they are my images. then send the URL in pathToImage to an array in my code called imageURLs()[ "" ].but I don't know how to to that??. I am using SDWebImage to display my images to the collectionView cell. i have tested it and it works fine if i copy and paste a random URL in the array called imageURLs[ "URL here" ]
I am very new to Swift and Firebase, so any help would be greatly appreciated!! :)
HERE IS AN IMAGE OF MY FIREBASE FILES.
https://ibb.co/bXLMcb
Psst! posts/pathToImage holds the URL so that's the one i want to retrieve.
users/urlToImage is just a profile picture. i don't really need it right now
To locate specific data within a node a Query is used.
While iterating over the nodes works, if there's a million nodes it's going to take a long time and tie up the UI while waiting for that to happen.
The bottom line is; if you know what you are looking for - in this case a uid - then a query is a super simple solution.
let postsRef = self.ref.child("posts")
let query = postsRef.queryOrdered(byChild: "userID").queryEqual(toValue: "uid_0")
query.observeSingleEvent(of: .value) { (snapshot) in
if snapshot.exists() {
for child in snapshot.children { //.value can return more than 1 match
let snap = child as! DataSnapshot
let dict = snap.value as! [String: Any]
let myPostURL = dict["urlToImage"] as! String
self.imageURLs.append(myPostURL)
}
} else {
print("no user posts found")
}
}
With the above code, we first create a reference to our posts node. Then a query that will only return this users (uid_0) posts.
We then iterate over the snapshot of posts and add them to the array.
This is much more efficient that loading and evaluating every node in the database.
See Sorting And Filtering data for more information.
You should think about storing an index of all your users posts somewhere in your database. That way you don't need to observe all posts every time. This is called denormalization. Firebase has an article in their docs about organizing your database.
Here's some information regarding firebase filtering in swift.
The best solution is to add a separate node that has a list of "post IDs" organized by user. Then you could observe that node, and only download each post specifically by the returned ID. Here's a link about flattening data structures in firebase. It would look something like this.
"posts":{
"somePostID":{
"timestamp": "0200231023",
"postContent": "here's my post content",
"authorUID" : "0239480238402934"
} ...
},
"postsByGivenUID":{
"someAuthorID":{
"somePostID": "true",
"somePostID": "true",
},
"someOtherAuthorID":{
"somePostID": "true",
"somePostID": "true",
"somePostID": "true"
}
}
This is actually a much bigger problem than just changing how you structure your code. For scalability sake, you're going to have to reevaluate how you structure the data in firebase altogether.
Then, you can nest your firebase query, sort of like this:
ref.child("postByGivenUID").child("Auth.auth().currentUser?.uid").observe(.childAdded) { (snapshot) in
ref.child("posts").child(snapshot.value).observeSingleEventOfType(of: .value) { (snap) in
// your actual post data will be here
// that way you won't be downloadin ALL posts EVERY time
}
}
My code is:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
})
the key is equal to a post key(-Kef2J6vJBCjHGYjSlkI). I want to retrieve only one element at a time.
There is only 1 dictionary under that child:
I need to create a dictionary out of the contents of this. However, when I implement regular algorithm:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] {
for snap in snapshot {
print("Zhenya: here is the snap: \(snap)")
if let postDict = snap.value as? Dictionary<String, AnyObject> {
let key = snap.key
let post = Post(postKey: key, postData: postDict)
self.posts.append(post)
} else {
print("Zhenya: failed to convert")
}
}
}
})
The 'snap' prints elements of the dictionary itself, not the whole dictionary. So it always fails the conversion on the following line.
Bonus for ninjas:
If you can suggest a better architecture, I would appreciate that.
It seems that retrieving posts from FB one by one will slow down performance considerably.
The reason it is 'one by one' is this one post is the result of user's GeoFire specification. Geofire returns keys to posts that are nearby.
I want to filter Firebase's main branch of posts based on those returned keys, but there is no way to do that - only load one by one (.orderByChild - doesn't returns posts, but instead, orders all the posts. If you you know how it can help me, please advice)
This is the code provided in your question:
_ = DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] {
for snap in snapshot {
print("Zhenya: here is the snap: \(snap)")
if let postDict = snap.value as? Dictionary<String, AnyObject> {
let key = snap.key
let post = Post(postKey: key, postData: postDict)
self.posts.append(post)
} else {
print("Zhenya: failed to convert")
}
}
}
})
Let's take a look at what's happening here.
The first line is observing posts/$post-id in Firebase using the .value method. The .value method returns the reference you provide in a FIRDataSnapshot, where snapshot.key is the name of the child and snapshot.value is a dictionary containing its children.
So, snapshot in DataService.ds.REF_POSTS.child(key!).observe(.value, with: { (snapshot) in ... }) should look like this:
$post-id
imageUrl: http://...
likes: 0
userKey: ...
But on the very next line you redefine snapshot using a local definition. That is why you are getting the children of the post, because you explicitly define it that way:
if let snapshot = snapshot.children.allObjects as? [FIRDataSnapshot] { ... }
If you remove this, you should see the behavior you want.
So what should you really be doing? Well I don't like using .value because it has a few drawbacks:
It downloads the entire node requested. If you mess up your reference for a large node (i.e, ref/users), it's easy for a .value operation to take several seconds before you see any data.
It is only called once. This is fine if your database is static, but whose database is static?
This is why I prefer using .childAdded for things like posts, messages, etc. It returns a single node at a time, which means you can populate a list progressively as new data becomes available. Using a few clever UI functions (such as tableView.insertRows(...) you can fairly smoothly display large lists of data without waiting for results. I believe it also supports paging, where you get them in batches of 10, for example. Here is a quick primer on how you can begin using .childAdded to make the most of large lists of data.
My realtime database looks like this:
It contains only 1 child as you can see. I had 4 more children in RunningGames a few minutes before. I deleted them in the browser. When now calling this:
private lazy var runningGamesRef: FIRDatabaseReference = FIRDatabase.database().reference().child("RunningGames")
self.runningGamesRef.observeSingleEvent(of: .value, with: { (snapshot) -> Void in
for gameSnap in snapshot.children {
let id = (gameSnap as! FIRDataSnapshot).key
print(id)
}
})
It still prints those games I deleted in the browser. Calling runningGameRef!.removeValues() in my app does deletes it in the browser and on the iPhone (the print(id) is fixed). I have this error on multiple observeSingleEvent functions on different children, not only children of RunningGames. What would cause this annoying error?
Some children in RunningGames also have children, but they do auto remove themselves in the app. However, these values are also still visible when calling observeSingleEvent.
It's probably your local cache that's still holding the outdated info. This often happens when you're manipulating data from multiple sources.
I would try using observe instead of observeSingleEvent. I know it's a bit odd (and not really what you want if you only want to load the data once) but that should keep your info up to date.
Maybe by doing this you could fetch the info just once.
var handle: UInt = 0
handle = ref.observe(.value, with: { snapshot in
for gameSnap in snapshot.children {
let id = (gameSnap as! FIRDataSnapshot).key
print(id)
}
ref.removeObserver(withHandle: handle)
})
Source of the code (Frank van Puffelen)