Pull firebase data that doesn't have key [duplicate] - ios

I want to display only children of location i.e Sindh and Punjab (not their children). Is it possible, and if so how can I do it?

From Best practices for data structure in the docs:
Avoid nesting data
Because the Firebase Realtime Database allows nesting data up to 32
levels deep, you might be tempted to think that this should be the
default structure. However, when you fetch data at a location in your
database, you also retrieve all of its child nodes. In addition, when
you grant someone read or write access at a node in your database, you
also grant them access to all data under that node. Therefore, in
practice, it's best to keep your data structure as flat as possible.
That is how Firebase works: If you get an item, you get its children as well. If you don't want this, you should restructure the database.

Avoid Nesting Data
Because the Firebase Realtime Database allows nesting data up to 32 levels deep, you might be tempted to think that this should be the default structure. However, when you fetch data at a location in your database, you also retrieve all of its child nodes. In addition, when you grant someone read or write access at a node in your database, you also grant them access to all data under that node. Therefore, in practice, it's best to keep your data structure as flat as possible.
Below is a structure of yours on how to implement flat database as well as how to retrieve the location key.
{
"location": {
"punjab": {
"lahore": true,
"multan": true
},
"sindh": {
"karachi": true
}
},
"place": {
"lahore": {
"location": "punjab",
"details": {
"0": "model",
"1": "steel town"
}
},
"multan": {
"location": "punjab",
"contacts": {
"0": "myarea"
}
},
"karachi": {
"location": "sindh",
"contacts": {
"0": "hadeed",
"1": "Steel town"
}
}
}
}
Below is the code that you can use to retrieve the location key.
private DatabaseReference mDatabase;
// ...
mDatabase = FirebaseDatabase.getInstance().getReference();
mLocation = mDatabase.child("location");
mPlace = mDatabase.child("place");
ValueEventListener placeListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
// Get Post object and use the values to update the UI
Place place = dataSnapshot.getValue(Place.class);
String location = place.location;
System.out.println(location);
}
};
mPlace.addValueEventListener(placeListener);
For more information on Firebase:
Firebase Security & Rules

DatabaseReference databaseReference = FirebaseDatabase.getInstance().getReference().child("location");
final ArrayList<String> statesArrayList= new ArrayList<>();
databaseReference.addChildEventListener(new ChildEventListener() {
#Override
public void onChildAdded(DataSnapshot dataSnapshot, String s) {
statesArrayList.add(dataSnapshot.getKey());
}
#Override
public void onChildChanged(DataSnapshot dataSnapshot, String s) {
}
#Override
void onChildRemoved(DataSnapshot dataSnapshot) {
}
#Override
public void onChildMoved(DataSnapshot dataSnapshot, String s) {
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});

Yes, you can
ValueEventListener getValueListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
// Get data from firebase
Log.d("The name of child that you need:", dataSnapshot.getKey());
// ...
}
};
databaseReference.addValueEventListener(getValueListener );
Read more: https://firebase.google.com/docs/database/android/read-and-write#listen_for_value_events

Though nesting data is not recommended in NoSQL databases like Firebase, to get the name of child nodes your code will look like this
DatabaseReference mainRef = FirebaseDatabase.getInstance().getReference();
DatabaseReference locationref = mainRef.child("location");
final ArrayList<String> locationNames = new ArrayList<>();
locationref.addChildEventListener(new ChildEventListener() {
#Override
public void onChildAdded(DataSnapshot dataSnapshot, String s) {
locationNames.add(dataSnapshot.getKey());
}
#Override
public void onChildChanged(DataSnapshot dataSnapshot, String s) {
}
#Override
void onChildRemoved(DataSnapshot dataSnapshot) {
}
#Override
public void onChildMoved(DataSnapshot dataSnapshot, String s) {
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});
You can now use locationNames to do what you want.

Firebase can as quickly look up a node at level 1 as it can at level 32. Depth is not a factor that affects speed on a technical level but years of experience with Firebase has shown that deeply nested data often goes hand in hand with performance problems. As an example, i recomand you reading the offical documentation, Best practices for data structure in Firebase and Structuring your Firebase Data correctly for a Complex App.
If you don't want to change your actual database structure and assuming that location node is a direct child of your Firebase root, please see the code below:
DatabaseReference rootRef = FirebaseDatabase.getInstance().getReference();
DatabaseReference locationRef = rootRef.child("location");
ValueEventListener eventListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
for(DataSnapshot ds : dataSnapshot.getChildren()) {
String key = ds.getKey();
Log.d("TAG", key);
}
}
#Override
public void onCancelled(DatabaseError databaseError) {}
};
locationRef.addListenerForSingleValueEvent(eventListener);
Your output will be:
punjab
sindh

I know this question was asked a while ago and for android, but I was looking for a solution to this and came upon this question. I couldn't use the answers provided, so here is the solution I found. This will only get the children and nothing else.
This is in javascript, but you can use whichever method is the alternative in java to make a request from a REST endpoint.
const databaseURL = "https://example.firebaseio.com" // replace value with yours
const endpoint = "location"
const user = firebase.auth().currentUser
user.getIdToken()
.then(token => {
return fetch(`${databaseURL}/${endpoint}.json?auth=${token}&shallow=true`)
})
.then(response => {
return response.json()
})
.then(json => {
console.log({json}) // contains only the children
})
.catch(error => {
console.log(error.message)
})
The important bit here is the &shallow=true otherwise you get all the data in the children.
Using curl this would be
curl 'https://example.firebaseio.com/location.json?auth=INSERT_TOKEN&shallow=true'
This assumes location is a child of the root.
More details can be had by looking at the docs
Before calling this, you have to make sure you have the currentUser available in firebase.
Also, I would heed the warnings mentioned here about proper firebase realtime database structure. However, if you are already in this spot, then this can help.

This is how u can get child names
HashMap<String, Object> allData = (HashMap<String, Object>) dataSnapshot.getValue();
String[] yourChildArray = allData.keySet().toArray(new String[0]);

Related

How to create a Grid without POJO (dynamic columns)?

The question was asked here: https://vaadin.com/forum/thread/18095407/how-to-create-a-grid-without-binder
However the vaadin's forum closed so i want to continue it here.
On Vaadin 14, Any recommendation on the best way to implement grid with dynamic varying number of columns. Using column Index (1,2,3...) is not a good choice for me. Let say I have a simple Json file (only 1 level: key-value) to map to a grid and this Json has an unknown list of properties.
which approach is better in term of performance ?:
[Option 1]
class Data {
private Map<String, Object> values = new HashMap<>();
public void set(String key, Object val) {
values.put(key, val);
}
public Object get(String key) {
return values.get(key);
}
}
Grid<Data> myGrid = new Grid<>();
[Option 2]
public class GridDynamicValueProvider implements ValueProvider<GridDynamicRow, Object> {
private int columnIndex;
public GridDynamicValueProvider(int columnIndex) {
this.columnIndex = columnIndex;
}
#Override
public Object apply(GridDynamicRow dynamicRow) {
return dynamicRow.getValue(columnIndex);
}
}
public class GridDynamicRow {
private List<Object> values = new ArrayList<>();
public void addValue(String value) {
values.add(value);
}
public Object getValue(int columnIndex) {
return values.get(columnIndex);
}
}
The SerializablePredicate of Vaadin accepts both function references and Lambdas, thus it is possible to use non-POJO data types with Grid and Binder in Vaadin, although that is a bit unconventional. The key ingredients are:
Grid<Map<String, Integer>> grid = new Grid<>();
...
grid.addColumn(map -> map.get("column")).setHeader("Column");
You can also wrap the Map in custom class if you have need to protect the internals.
class Data {
private Map<String, Object> values = new HashMap<>();
public void set(String key, Object val) {
values.put(key, val);
}
public Object get(String key) {
return values.get(key);
}
}
Grid<Data> myGrid = new Grid<>();
As for the performance, essentially, you're comparing between using a List where you fetch by index versus a HashMap where you fetch by key. Here's a related question: ArrayList .get faster than HashMap .get?
You can use also ArrayList as Grid's type if you can index the columns with a number.
The both approaches allow generating Grid's with varying dynamic number of columns, for example if you read the data directly from a file or have raw data backend queries.

View How much to download and how much time left

Is there any option to get how much data downloaded and how much time left When using firebase realtime database
ValueEventListener valueEventListener=new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
};
No, there are no progress or status notifications for Realtime Database listeners.

How to Fetch Firebase Database for Only Selected List of Keys in Swift?

I am using firebase database for storing data of my iOS application.
Here is my firebase database high level structure:
In this database I am saving data of a menu item in menu tab. This is list of all menus. This is list menus where all menu item has a unique key:
There is one more node "Users" and it has one child "Consumer"
I am saving 'liked' and 'disliked' menu item in different children of "Consumer" node.
I am working on a screen where I have to show all details of menuItems which are liked by user. As you can see I am saving only menuItem keys in "Like_Deal_Dish" node.
This is my code by which I am fetching all keys which have added in "Like_Deal_Deish" child:
Database.database().reference().child("Users").child("Consumer").child((Auth.auth().currentUser?.uid)!).child("Like_Deals_Dish").observeSingleEvent(of: .value) { datasnapshot in
if datasnapshot.exists() {
print("Like Deals - \(datasnapshot)")
}
else{
print("Liked data is not available")
}
}
Console log where I am getting list of all liked ,menuItem:
What should be the query to fetch only those menuItems whose keys are added in the "Like_Deal_Dish" ?
You fetched menu id from Like deals. so now you can use that value to query again the menus child
Here is example (not tested on xcode so forgive me for syntax error if any)
Database.database().reference().child("Users").child("Consumer")
.child((Auth.auth().currentUser?.uid)!).child("Like_Deals_Dish")
.observeSingleEvent(of: .value) { datasnapshot in
if datasnapshot.exists() {
print("Like Deals - \(datasnapshot)")
// Loop here for menu items
for rest in datasnapshot.children.allObjects as [FIRDataSnapshot] {
// Fetch Menu Info here with rest.value
Database.database().reference()
.child("menus").child(rest.value)
.observeSingleEvent(of: .value) { menu in
if datasnapshot.exists() {
print("MENU - \(menu)")
}
}
}
} else {
print("Liked data is not available")
}
}
Hope it is helpful
You can do this without using multiple queries as suggested by Prashant. If you have a large dataset, using a query to obtain each key would be painfully slow. You could get each key like this:
Database.database().reference().child("Users").child("Consumer").child((Auth.auth().currentUser?.uid)!).child("Like_Deals_Dish").observeSingleEvent(of: .value) { datasnapshot in
if datasnapshot.exists() {
var keyArray = [String]()
for snap in datasnapshot.children.allObjects {
if let snap = snap as? DataSnapshot {
let key = snap.key
keyArray.append(key)
}
}
//use keyArray
}
else{
print("Liked data is not available")
}
}
//Create a class
public class FirebaseDataHelper { //...}
//create a interface in it with method for whatever you want to return
public interface OnDataFetchedListener{
void onDataFetched(List<User> data);
}
//Add below Methods as your concern
//This method helps you to perform mysql 'IN' operation on firebase
Here my concern to get List of users store on Node NODE_USERS on firebase for given list of keys in a Syncronized way
public void fetchUserDataForSelectedKeys(List<String> keyList,OnDataFetchedListener dataFetchedListener){
List<User> data = new ArrayList<>();
FirebaseDatabase.getInstance().getReference().child(CommonConstants.NODE_USERS).get().addOnCompleteListener(new OnCompleteListener<DataSnapshot>() {
#Override
public void onComplete(#NonNull Task<DataSnapshot> task) {
if(task.isSuccessful()){
DataSnapshot snapshot = task.getResult();
for (String key:
keyList) {
if(snapshot.hasChild(key)){
data.add(snapshot.child(key).getValue(User.class));
}
}
dataFetchedListener.onDataFetched(data);
}
}
});
}
//This method used when using concept of pk in andorid and you can modify it according to your requirement
//I stored ids of users in node NODE_FREINDS under current userID and i want to get complete user for each id stored in that node(Actual user info stored in NODE_USER) and return list when completely fetched
public void fetchUserDataForPath(String path,OnDataFetchedListener dataFetchedListener){
List<User> data = new ArrayList<>();
String[] children = path.split("/");
DatabaseReference fromReference = FirebaseDatabase.getInstance().getReference();
for (String child:
children) {
fromReference=fromReference.child(child);
}
DatabaseReference finalFromReference = fromReference;
FirebaseDatabase.getInstance().getReference()
.child(CommonConstant.NODE_USER)
.get()
.addOnCompleteListener(new OnCompleteListener<DataSnapshot>() {
#Override
public void onComplete(#NonNull Task<DataSnapshot> task) {
if(task.isSuccessful()){
DataSnapshot snapshot = task.getResult();
finalFromReference.get().addOnCompleteListener(new OnCompleteListener<DataSnapshot>() {
#Override
public void onComplete(#NonNull Task<DataSnapshot> task) {
if(task.isSuccessful()){
DataSnapshot snapshot1 = task.getResult();
for (DataSnapshot sn:
snapshot1.getChildren()) {
data.add(snapshot
.child(Objects.requireNonNull(sn.getKey()))
.getValue(User.class));
}
dataFetchedListener.onDataFetched(data);
}
}
});
}
}
});
}
}

How can I do batch deletes millions on entities using DatastoreIO and Dataflow

I'm trying to use Dataflow to delete many millions of Datastore entities and the pace is extremely slow (5 entities/s). I am hoping you can explain to me the pattern I should follow to allow that to scale up to a reasonable pace. Just adding more workers did not help.
The Datastore Admin console has the ability to delete all entities of a specific kind but it fails a lot and takes me a week or more to delete 40 million entities. Dataflow ought to be able to help me delete millions of entities that match only certain query parameters as well.
I'm guessing that some type of batching strategy should be employed (where I create a mutation with 1000 deletes in it for example) but its not obvious to me how I would go about that. DatastoreIO gives me just one entity at a time to work with. Pointers would be greatly appreciated.
Below is my current slow solution.
Pipeline p = Pipeline.create(options);
DatastoreIO.Source source = DatastoreIO.source()
.withDataset(options.getDataset())
.withQuery(getInstrumentQuery(options))
.withNamespace(options.getNamespace());
p.apply("ReadLeafDataFromDatastore", Read.from(source))
.apply("DeleteRecords", ParDo.of(new DeleteInstrument(options.getDataset())));
p.run();
static class DeleteInstrument extends DoFn<Entity, Integer> {
String dataset;
DeleteInstrument(String dataset) {
this.dataset = dataset;
}
#Override
public void processElement(ProcessContext c) {
DatastoreV1.Mutation.Builder mutation = DatastoreV1.Mutation.newBuilder();
mutation.addDelete(c.element().getKey());
final DatastoreV1.CommitRequest.Builder request = DatastoreV1.CommitRequest.newBuilder();
request.setMutation(mutation);
request.setMode(DatastoreV1.CommitRequest.Mode.NON_TRANSACTIONAL);
try {
DatastoreOptions.Builder dbo = new DatastoreOptions.Builder();
dbo.dataset(dataset);
dbo.credential(getCredential());
Datastore db = DatastoreFactory.get().create(dbo.build());
db.commit(request.build());
c.output(1);
count++;
if(count%100 == 0) {
LOG.info(count+"");
}
} catch (Exception e) {
c.output(0);
e.printStackTrace();
}
}
}
There is no direct way of deleting entities using the current version of DatastoreIO. This version of DatastoreIO is going to be deprecated in favor of a new version (v1beta3) in the next Dataflow release. We think there is a good use case for providing a delete utility (either through an example or PTransform), but still work in progress.
For now you can batch your deletes, instead of deleting one at a time:
public static class DeleteEntityFn extends DoFn<Entity, Void> {
// Datastore max batch limit
private static final int DATASTORE_BATCH_UPDATE_LIMIT = 500;
private Datastore db;
private List<Key> keyList = new ArrayList<>();
#Override
public void startBundle(Context c) throws Exception {
// Initialize Datastore Client
// db = ...
}
#Override
public void processElement(ProcessContext c) throws Exception {
keyList.add(c.element().getKey());
if (keyList.size() >= DATASTORE_BATCH_UPDATE_LIMIT) {
flush();
}
}
#Override
public void finishBundle(Context c) throws Exception {
if (keyList.size() > 0) {
flush();
}
}
private void flush() throws Exception {
// Make one delete request instead of one for each element.
CommitRequest request =
CommitRequest.newBuilder()
.setMode(CommitRequest.Mode.NON_TRANSACTIONAL)
.setMutation(Mutation.newBuilder().addAllDelete(keyList).build())
.build();
db.commit(request);
keyList.clear();
}
}

ReactiveUI Google Maps

Question: I have a IReactiveDerivedList<SensorViewModel> in my ViewModel and want to subscribe to changes on it including what's already in the list before I observe it. This then gets fed into GoogleMaps for Xamarin Android.
This seems to work when I add something :
public void OnMapReady(GoogleMap map)
{
_itemsAdded = this.ViewModel.Sensors.ItemsAdded
.ObserveOn(RxApp.MainThreadScheduler)
.Select(s => new CircleOptions()
.InvokeCenter(new LatLng(s.Latitude, s.Longitude))
.InvokeRadius(1000)
.InvokeFillColor(Color.Blue.ToArgb())
.InvokeStrokeColor(Color.Red.ToArgb()))
.Subscribe(
Observer.Create<CircleOptions>(options => map.AddCircle(options)));
}
But I also need to keep track of the Circle returned from map.AddCircle so I can remove it from the map when the object goes away. What's the reactive way to handle a case like this?
I don't know if this is the most reactive way to do it but I figured something out that seems to work:
private IDisposable _itemsAdded;
private IDisposable _itemsRemoved;
private readonly Dictionary<string, Circle> _circleMap = new Dictionary<string, Circle>();
public void OnMapReady(GoogleMap map)
{
_circleMap.Clear();
_itemsAdded = this.ViewModel.Sensors.ItemsAdded
.StartWith(this.ViewModel.Sensors)
.Subscribe(s =>
{
var options = new CircleOptions()
.InvokeCenter(new LatLng(s.Latitude, s.Longitude))
.InvokeRadius(1000)
.InvokeFillColor(Color.Blue.ToArgb())
.InvokeStrokeColor(Color.Red.ToArgb());
var circle = map.AddCircle(options);
this._circleMap.Add(s.Id, circle);
});
_itemsRemoved = this.ViewModel.Sensors.ItemsRemoved
.Subscribe(s =>
{
Circle circle = null;
if (this._circleMap.TryGetValue(s.Id, out circle))
{
this._circleMap.Remove(s.Id);
circle.Remove();
}
});
}

Resources