Updating tableview without reloading the view (Swift, Firebase) - ios

Question

So I am trying to get all of the favourites from a database to show up in a tableview. However, when I favourite a user, it only displays one.
The structure of my database looks like:
Favourites:
UserID:
key: User 1
key: User 2
and my code:
var usersArray[UserClass]()
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(true)
fetchFavourites()
}
func fetchFavourites () {
let userID = Auth.auth().currentUser?.uid
let favouriteRef = self.databaseRef.child("favourites").child(userID!)
favouriteRef.queryOrderedByKey().observe(.childAdded, with: { (snapshot) in
let favouriteID = "\(snapshot.value!)"
let usersRef = self.databaseRef.child("users")
usersRef.observe(.value, with: { (users) in
self.usersArray.removeAll()
for user in users.children {
let user = UserClass(snapshot: user as! DataSnapshot)
if favouriteID == user.uid {
self.usersArray.append(user)
}
self.tableView.reloadData()
}
})
})
}
I have tried creating another array within the function and then making the main array = to that, however it doesn't update the tableview when switching between screens. This code only works when the page view is loaded for the first time. However, I am using a back button that doesn't up date the screen.
Thanks, let me know if you want me to provide anything else.
Edited code (new problem, see below):
func fetchFavourites () {
DispatchQueue.main.async( execute: {
let userID = Auth.auth().currentUser?.uid
var tempFav = [UserClass]()
let favouriteRef = self.databaseRef.child("favourites").child(userID!)
favouriteRef.queryOrderedByKey().observe(.childAdded, with: { (snapshot) in
let favouriteID = "\(snapshot.value!)"
let usersRef = self.databaseRef.child("users")
tempFav.removeAll()
usersRef.observe(.value, with: { (users) in
self.usersArray.removeAll()
for user in users.children {
let user = UserClass(snapshot: user as! DataSnapshot)
if favouriteID == user.uid {
tempFav.append(user)
}
self.usersArray = tempFav
self.tableView.reloadData()
}
})
})
})
}
The problem has been fixed where it populates the tableview with all of the liked users and gets rid of them if I unlike them etc. However, if I unlike all of the users. The array duplicates a user twice even though it should be empty.