Related
From the docs:
You can also chain multiple where() methods to create more specific queries (logical AND).
How can I perform an OR query?
Example:
Give me all documents where the field status is open OR upcoming
Give me all documents where the field status == open OR createdAt <= <somedatetime>
OR isn't supported as it's hard for the server to scale it (requires keeping state to dedup). The work around is to issue 2 queries, one for each condition, and dedup on the client.
Edit (Nov 2019):
Cloud Firestore now supports IN queries which are a limited type of OR query.
For the example above you could do:
// Get all documents in 'foo' where status is open or upcmoming
db.collection('foo').where('status','in',['open','upcoming']).get()
However it's still not possible to do a general OR condition involving multiple fields.
With the recent addition of IN queries, Firestore supports "up to 10 equality clauses on the same field with a logical OR"
A possible solution to (1) would be:
documents.where('status', 'in', ['open', 'upcoming']);
See Firebase Guides: Query Operators | in and array-contains-any
suggest to give value for status as well.
ex.
{ name: "a", statusValue = 10, status = 'open' }
{ name: "b", statusValue = 20, status = 'upcoming'}
{ name: "c", statusValue = 30, status = 'close'}
you can query by ref.where('statusValue', '<=', 20) then both 'a' and 'b' will found.
this can save your query cost and performance.
btw, it is not fix all case.
I would have no "status" field, but status related fields, updating them to true or false based on request, like
{ name: "a", status_open: true, status_upcoming: false, status_closed: false}
However, check Firebase Cloud Functions. You could have a function listening status changes, updating status related properties like
{ name: "a", status: "open", status_open: true, status_upcoming: false, status_closed: false}
one or the other, your query could be just
...where('status_open','==',true)...
Hope it helps.
This doesn't solve all cases, but for "enum" fields, you can emulate an "OR" query by making a separate boolean field for each enum-value, then adding a where("enum_<value>", "==", false) for every value that isn't part of the "OR" clause you want.
For example, consider your first desired query:
Give me all documents where the field status is open OR upcoming
You can accomplish this by splitting the status: string field into multiple boolean fields, one for each enum-value:
status_open: bool
status_upcoming: bool
status_suspended: bool
status_closed: bool
To perform your "where status is open or upcoming" query, you then do this:
where("status_suspended", "==", false).where("status_closed", "==", false)
How does this work? Well, because it's an enum, you know one of the values must have true assigned. So if you can determine that all of the other values don't match for a given entry, then by deduction it must match one of the values you originally were looking for.
See also
in/not-in/array-contains-in: https://firebase.google.com/docs/firestore/query-data/queries#in_and_array-contains-any
!=: https://firebase.googleblog.com/2020/09/cloud-firestore-not-equal-queries.html
I don't like everyone saying it's not possible.
it is if you create another "hacky" field in the model to build a composite...
for instance, create an array for each document that has all logical or elements
then query for .where("field", arrayContains: [...]
you can bind two Observables using the rxjs merge operator.
Here you have an example.
import { Observable } from 'rxjs/Observable';
import 'rxjs/add/observable/merge';
...
getCombinatedStatus(): Observable<any> {
return Observable.merge(this.db.collection('foo', ref => ref.where('status','==','open')).valueChanges(),
this.db.collection('foo', ref => ref.where('status','==','upcoming')).valueChanges());
}
Then you can subscribe to the new Observable updates using the above method:
getCombinatedStatus.subscribe(results => console.log(results);
I hope this can help you, greetings from Chile!!
We have the same problem just now, luckily the only possible values for ours are A,B,C,D (4) so we have to query for things like A||B, A||C, A||B||C, D, etc
As of like a few months ago firebase supports a new query array-contains so what we do is make an array and we pre-process the OR values to the array
if (a) {
array addObject:#"a"
}
if (b) {
array addObject:#"b"
}
if (a||b) {
array addObject:#"a||b"
}
etc
And we do this for all 4! values or however many combos there are.
THEN we can simply check the query [document arrayContains:#"a||c"] or whatever type of condition we need.
So if something only qualified for conditional A of our 4 conditionals (A,B,C,D) then its array would contain the following literal strings: #["A", "A||B", "A||C", "A||D", "A||B||C", "A||B||D", "A||C||D", "A||B||C||D"]
Then for any of those OR combinations we can just search array-contains on whatever we may want (e.g. "A||C")
Note: This is only a reasonable approach if you have a few number of possible values to compare OR with.
More info on Array-contains here, since it's newish to firebase docs
If you have a limited number of fields, definitely create new fields with true and false like in the example above. However, if you don't know what the fields are until runtime, you have to just combine queries.
Here is a tags OR example...
// the ids of students in class
const students = [studentID1, studentID2,...];
// get all docs where student.studentID1 = true
const results = this.afs.collection('classes',
ref => ref.where(`students.${students[0]}`, '==', true)
).valueChanges({ idField: 'id' }).pipe(
switchMap((r: any) => {
// get all docs where student.studentID2...studentIDX = true
const docs = students.slice(1).map(
(student: any) => this.afs.collection('classes',
ref => ref.where(`students.${student}`, '==', true)
).valueChanges({ idField: 'id' })
);
return combineLatest(docs).pipe(
// combine results by reducing array
map((a: any[]) => {
const g: [] = a.reduce(
(acc: any[], cur: any) => acc.concat(cur)
).concat(r);
// filter out duplicates by 'id' field
return g.filter(
(b: any, n: number, a: any[]) => a.findIndex(
(v: any) => v.id === b.id) === n
);
}),
);
})
);
Unfortunately there is no other way to combine more than 10 items (use array-contains-any if < 10 items).
There is also no other way to avoid duplicate reads, as you don't know the ID fields that will be matched by the search. Luckily, Firebase has good caching.
For those of you that like promises...
const p = await results.pipe(take(1)).toPromise();
For more info on this, see this article I wrote.
J
OR isn't supported
But if you need that you can do It in your code
Ex : if i want query products where (Size Equal Xl OR XXL : AND Gender is Male)
productsCollectionRef
//1* first get query where can firestore handle it
.whereEqualTo("gender", "Male")
.addSnapshotListener((queryDocumentSnapshots, e) -> {
if (queryDocumentSnapshots == null)
return;
List<Product> productList = new ArrayList<>();
for (DocumentSnapshot snapshot : queryDocumentSnapshots.getDocuments()) {
Product product = snapshot.toObject(Product.class);
//2* then check your query OR Condition because firestore just support AND Condition
if (product.getSize().equals("XL") || product.getSize().equals("XXL"))
productList.add(product);
}
liveData.setValue(productList);
});
For Flutter dart language use this:
db.collection("projects").where("status", whereIn: ["public", "unlisted", "secret"]);
actually I found #Dan McGrath answer working here is a rewriting of his answer:
private void query() {
FirebaseFirestore db = FirebaseFirestore.getInstance();
db.collection("STATUS")
.whereIn("status", Arrays.asList("open", "upcoming")) // you can add up to 10 different values like : Arrays.asList("open", "upcoming", "Pending", "In Progress", ...)
.addSnapshotListener(new EventListener<QuerySnapshot>() {
#Override
public void onEvent(#Nullable QuerySnapshot queryDocumentSnapshots, #Nullable FirebaseFirestoreException e) {
for (DocumentSnapshot documentSnapshot : queryDocumentSnapshots) {
// I assume you have a model class called MyStatus
MyStatus status= documentSnapshot.toObject(MyStatus.class);
if (status!= null) {
//do somthing...!
}
}
}
});
}
I have a domain class
class Url {
UUID id
String url
static hasMany = [
indications:UrlIndication
]
...
}
And
class UrlIndication {
UUID id
String name
static belongsTo = Url
...
}
I want to choose urls so that it has all the necessary UrlIndication elements in a given list indicationsId.
For that I use an association and criteria like this one:
indications {
and {
indicationsId.each{
indication->
eq ('id',UUID.fromString(indication as String))
}
}
}
However, all I got is an empty result. Can you suggest any modifications/ other methods so that I can do this? Thanks in advance
Your query returned an empty list because it's the equivalent of the expression (pseudo-code): if 1 = 1 and 1 = 2 and 1 = 3
Such an expression would always be false. in or inList would not work for the reason #innovatism described.
In theory, Criteria's eqAll() or HQL's = ALL would work. But, I don't know for sure because I could not get either one to work.
What will work is to use inList to return a subset of Urls: those which contain at least one of the UrlIndication IDs. Then use Groovy's containsAll() to finish the job.
def ids = indicationsId.collect { UUID.fromString(it as String) }
Url.createCriteria()
.buildCriteria {
indications {
inList 'id', ids
}
}
.setResultTransformer(org.hibernate.Criteria.DISTINCT_ROOT_ENTITY)
.list()
.findAll {
it.indications.id.containsAll(ids)
}
Since the query has the potential to return duplicate Url instances, the ResultTransformer is set to return a unique list.
Finally, findAll() is used along with containsAll() to filter the list further.
Using eqAll (maybe)
Something like the following might work. Something funky is going on with Grails' HibernateCriteriaBuilder that causes the eqAll method to look up properties in the root entity; completely ignoring the sub criteria. So the following uses Hibernate directly. It didn't work for me, but it's as close as I could get. And it gave me a head-ache!
Url.createCriteria().buildCriteria {}
.createCriteria('indications', 'i')
.add(org.hibernate.criterion.Property.forName('i.id').eqAll(org.hibernate.criterion.DetachedCriteria.forClass(UrlIndication)
.add(org.hibernate.criterion.Restrictions.in('id', ids))
.setProjection(org.hibernate.criterion.Property.forName('id'))
))
.setResultTransformer(org.hibernate.Criteria.DISTINCT_ROOT_ENTITY)
.list()
The problem I had is I could not get Restrictions.in to work. Restrictions.eq works fine.
the in clause should do:
indications {
'in' 'id', indicationsId.collect{ UUID.fromString indication.toString() }
}
I have an entity, Student, defined in Student.groovy as:
#EqualsAndHashCode(includes = ['id'])
class Student {
Long id
String name
String type
University university
static mapping = {
university column : 'UNIVERSITY_ID'
}
}
and a University entity, defined in University.groovy as:
class University {
Long id
String name
static mapping = {
id column : 'id', generator : 'assigned'
}
}
I've been trying to switch from calling
Student.list(sort: ..., order: ...)
to calling:
Student.findAll("from Student s where type = :type ", [type : 'T'], [ sort : 'name' ])
This fails to order correctly by the name field. The previous version, using list worked fine.
I've also tried calling something like
Student.findAll(sort : 'name') { type == "T" }
which worked fine like this, but when trying to sort by the university.name
Student.findAll(sort : 'university.name') { type == 'T" }
it raised an error regarding the university.name field not being found.
Anybody have any idea on how to do this properly?
Thank you.
Use executeQuery instead of findAll - they should function the same, but I've found that executeQuery is for some reason a more direct caller of the HQL, and findAll fails or returns unexpected results in some cases.
So that first query would be
Student.executeQuery(
'select s from Student s where s.type = :type order by s.name',
[type : 'T'])
and ordering by university name would be
Student.executeQuery(
'select s from Student s where s.type = :type order by s.university.name',
[type : 'T'])
I like HQL and tend to use it a lot, but it couples you to Hibernate and relational databases - if you want to switch to a NoSQL database these queries will fail. Criteria queries, "where" queries and finders all use criteria queries internally, and those are converted to native query API calls by the GORM implementation.
The equivalent criteria queries would be
Student.withCriteria {
eq 'type', 'T'
order 'name', 'asc'
}
and
Student.withCriteria {
eq 'type', 'T'
university {
order 'name', 'desc'
}
}
Some unrelated notes:
You shouldn't use id in equals or hashCode calculations; if you have a persistent Student and a new non-persistent instance with the same name, type, and University, they should be considered equal, but since the non-persistent instance's id will be null they'll be considered different.
You don't need to specify the id property - Grails adds it and the version field to the bytecode via an AST transformation during compilation.
There's no need to map the column name of the university property to 'UNIVERSITY_ID' - that's what it would be anyway.
You can omit the redundant column setting in the id mapping.
Here's the Student class with cruft removed:
#EqualsAndHashCode(includes = ['name', 'type', 'university'])
class Student {
String name
String type
University university
}
and University:
class University {
String name
static mapping = {
id generator: 'assigned'
}
}
Some pseudo-code of the model I'm working with:
User { int Id, string Username }
Activity { int Id, string Name }
Place { int Id, string Name }
Basically I have a bunch of Users and they belong to certain places (many to many relationship in RDBMS world). What I'd like to do now that I've created all of the nodes already is create the relationship between them. To do that I believe I need to get references to each node and then simply create the relationship between them.
Note: So far no relationships exist. It does look like in some of the examples they have added the User nodes with a relationship that points to the RootNode but I have no idea why. I'm not sure if I need to do that or not.
More pseudo-code:
var userRef = _graphClient...GetUserNodeWhereIdEquals(user.Id);
// or something like _graphClient.OutV<User>("[[id={0}]]", user.Id)
// or even just _graphClient.V<User>(id == user.Id)
var placeRef = _graphClient...GetPlaceNodeWhereIdEquals(place.Id);
_graphClient...CreateRelationshipBetween(userRef, placeRef, "belongs_to");
Unfortunately the documentation starts off pretty great then goes south when you get to relationships.
Update 3/29/12
Here's the code I have so far:
foreach (var a in _activityTasks.GetAll())
{
_graphClient.Create(a, new ActivityBelongsTo(_graphClient.RootNode));
}
foreach (var p in _placeTasks.GetAll().Take(1))
{
var placeNode = _graphClient.Create(p, new PlaceBelongsTo(_graphClient.RootNode));
foreach (var activity in p.Activities)
{
Activity activity1 = activity;
var activityNode = _graphClient.RootNode.In<Activity>(ActivityBelongsTo.TypeKey, a => a.Id == activity1.Id).SingleOrDefault();
_graphClient.CreateRelationship(placeNode, new PlaceHasActivity(activityNode.Reference));
}
}
The activity nodes are created fine. The place node is created fine. An error is now being thrown when trying to get the activityNode. It's a rather large stack trace so I'll try to paraphrase here:
Received an exception when executing the request.
The query was: g.v(p0).in(p1).filter{ it[p2] == p3
}.drop(p4).take(p5)._()
The exception was: Value cannot be null. Parameter name: key
System.ArgumentNullException: Value cannot be null.Parameter name: key
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue
value, Boolean add) ... The raw response body was: [ {
"outgoing_relationships" :
"http://localhost:7474/db/data/node/2/relationships/out", "data" : {
"Name" : "Aerobics", "Id" : 2 }, "all_typed_relationships" :
"http://localhost:7474/db/data/node/2/relationships/all/{-list|&|types}",
"traverse" :
"http://localhost:7474/db/data/node/2/traverse/{returnType}", "self"
: "http://localhost:7474/db/data/node/2", "property" :
"http://localhost:7474/db/data/node/2/properties/{key}",
"outgoing_typed_relationships" :
"http://localhost:7474/db/data/node/2/relationships/out/{-list|&|types}",
"properties" : "http://localhost:7474/db/data/node/2/properties",
"incoming_relationships" :
"http://localhost:7474/db/data/node/2/relationships/in", "extensions"
: { }, "create_relationship" :
"http://localhost:7474/db/data/node/2/relationships",
"paged_traverse" :
"http://localhost:7474/db/data/node/2/paged/traverse/{returnType}{?pageSize,leaseTime}",
"all_relationships" :
"http://localhost:7474/db/data/node/2/relationships/all",
"incoming_typed_relationships" :
"http://localhost:7474/db/data/node/2/relationships/in/{-list|&|types}"
} ]
Something to do when adding a item to a Dictionary when the key is null. Problem is, I don't see any nulls when I debug on my end, activity1 is there, RootNode is there, TypeKey is a const string.
I'm almost wondering if I should just keep the created nodes within a array or Dictionary myself and then just working with the NodeReference. That's what I'm going to try next.
Later that morning
This seems to load everything into the graph database fine:
var activityNodes = _activityTasks.GetAll().ToDictionary(a => a.Id, a => _graphClient.Create(a, new ActivityBelongsTo(_graphClient.RootNode)));
foreach (var p in _placeTasks.GetAll())
{
var placeNode = _graphClient.Create(p, new PlaceBelongsTo(_graphClient.RootNode));
foreach (var activity in p.Activities)
{
_graphClient.CreateRelationship(placeNode, new PlaceHasActivity(activityNodes[activity.Id]));
}
}
foreach (var u in _userTasks.GetAllUserGraph())
{
var userNode = _graphClient.Create(u, new UserBelongsTo(_graphClient.RootNode));
foreach(var activity in u.Activities)
{
_graphClient.CreateRelationship(userNode, new UserParticipatesIn(activityNodes[activity.Id]));
}
}
Now the problem is similar to what I had before. Now I want to get an activity that has a relationship to the RootNode:
Node<Activity> activity = _graphClient
.RootNode
.In<Activity>(ActivityBelongsTo.TypeKey, a => a.Id == 1)
.SingleOrDefault();
Throwing the key value can't be null exception again. I think I need to investigate the gremlin syntax more. I'm guessing the problem is there.
This afternoon
Started to experiment with Gremlin queries:
g.v(0).inE.filter{it.label=="ACTIVITY_BELONGS_TO"}.outV.filter{it.Id==1}.Name
works fine. I tried to replicate that using neo4jClient syntax:
_graphClient.RootNode.InE(ActivityBelongsTo.TypeKey).OutV(b => b.Id == 1).SingleOrDefault();
Same null exception, it spits out:
g.v(p0).inE.filter{ it[p1].equals(p2) }.outV.filter{ it[p3] == p4 }.drop(p5).take(p6)._()
which looks right to me, except for the end. Ran this though:
g.v(0).inE.filter{it.label=="ACTIVITY_BELONGS_TO"}.outV.filter{it.Id==1}.drop(0).take(1)._()
And that works fine. Something stinks here...maybe I should try the other library although I liked the de/serialization support. Sigh...
Thought maybe a raw query would work. Nope! This method no longer accepts a string and the required GremlinQuery I have no idea how to you. Grooooooooooooooooan.
var users = graphClient.ExecuteGetAllNodesGremlin<IsCustomer>("g.v(0).out('IsCustomer'){it.'Name' == 'BobTheBuilder'}");
Update 3/30/12
Created a new project, everything below works fine. Super confused why it will work here... :( Maybe version differences, I have no idea.
var client = new GraphClient(new Uri("http://localhost:7474/db/data"));
client.Connect();
client.Create(new User { Id = 1, Username = "joe" }, new UserBelongsTo(client.RootNode));
client.Create(new User { Id = 2, Username = "cloe" }, new UserBelongsTo(client.RootNode));
client.Create(new Activity { Id = 1, Name = "Bocce Ball" }, new ActivityBelongsTo(client.RootNode));
client.Create(new Activity { Id = 2, Name = "Programming" }, new ActivityBelongsTo(client.RootNode));
var user = client.RootNode.In<User>(UserBelongsTo.TypeKey, u=>u.Id == 1).SingleOrDefault();
var activity = client.RootNode.In<Activity>(ActivityBelongsTo.TypeKey, a=>a.Id == 1).SingleOrDefault();
client.CreateRelationship(user.Reference, new Plays(activity.Reference));
user = client.RootNode.In<User>(UserBelongsTo.TypeKey, u => u.Id == 1).SingleOrDefault();
activity = client.RootNode.In<Activity>(ActivityBelongsTo.TypeKey, a => a.Id == 1).SingleOrDefault();
I'm just getting started too. I would suggest you check out this blog:
http://romikoderbynew.com/2011/07/30/neo4jclient-primer/
Also, check http://frictionfree.org and its source code (in the about section) for more examples.
Creating relationships on existing - as I understand, this is possible. However, it appears to be easier to associate nodes as you create them. From the blog:
You can also create relationships between existing nodes.
graphClient.CreateRelationship(customerNodeReference, new
Speaks(languageNode.Reference));
RootNode - I believe you need to start a query from a node, I don't think you can do a
SELECT * FROM ... WHERE
Therefore, it would make sense that you need to attach nodes to the root node. This is an example from the FrictionFreeApp:
var node = graphClient.Create(
user,
new UserBelongsTo(rootNode));
I am using Nimble and Shiro for my security frameworks and I've just come accross a GORM bug. Indeed :
User.createCriteria().list {
maxResults 10
}
returns 10 users whereas User.list(max: 10) returns 9 users !
After further investigations, I found out that createCriteria returns twice the same user (admin) because admin has 2 roles!!! (I am not joking).
It appears that any user with more than 1 role will be returned twice in the createCriteria call and User.list will return max-1 instances (i.e 9 users instead of 10 users)
What workaround can I use in order to have 10 unique users returned ?
This is a very annoying because I have no way to use pagination correctly.
My domain classes are:
class UserBase {
String username
static belongsTo = [Role, Group]
static hasMany = [roles: Role, groups: Group]
static fetchMode = [roles: 'eager', groups: 'eager']
static mapping = {
roles cache: true,
cascade: 'none',
cache usage: 'read-write', include: 'all'
}
}
class User extends UserBase {
static mapping = {cache: 'read-write'}
}
class Role {
static hasMany = [users: UserBase, groups: Group]
static belongsTo = [Group]
static mapping = { cache usage: 'read-write', include: 'all'
users cache: true
groups cache: true
}
}
Less concise and clear, but using an HQL query seems a way to solve this problem. As described in the Grails documentation (executeQuery section) the paginate parameters can be added as extra parameters to executeQuery.
User.executeQuery("select distinct user from User user", [max: 2, offset: 2])
this way you can still use criteria and pass in list/pagination paramaters
User.createCriteria().listDistinct {
maxResults(params.max as int)
firstResult(params.offset as int)
order(params.order, "asc")
}
EDIT: Found a way to get both! Totally going to use it now
http://www.intelligrape.com/blog/tag/pagedresultlist/
If you call createCriteria().list() like this
def result=SampleDomain.createCriteria().list(max:params.max, offset:params.offset){
// multiple/complex restrictions
maxResults(params.max)
firstResult(params.offset)
} // Return type is PagedResultList
println result
println result.totalCount
You will have all the information you need in a nice PagedResultList format!
/EDIT
Unfortunately I do not know how to get a combination of full results AND max/offset pagination subset in the same call. (Anyone who can enlighten on that?)
I can, however, speak to one way I've used with success to get pagination working in general in grails.
def numResults = YourDomain.withCriteria() {
like(searchField, searchValue)
order(sort, order)
projections {
rowCount()
}
}
def resultList = YourDomain.withCriteria() {
like(searchField, searchValue)
order(sort, order)
maxResults max as int
firstResult offset as int
}
That's an example of something I'm using to get pagination up and running. As KoK said above, I'm still at a loss for a single atomic statement that gives both results. I realize that my answer is more or less the same as KoK now, sorry, but I think it's worth pointing out that rowCount() in projections is slightly more clear to read, and I don't have comment privileges yet :/
Lastly: This is the holy grail (no pun intended) of grails hibernate criteria usage references; bookmark it ;)
http://www.grails.org/doc/1.3.x/ref/Domain%20Classes/createCriteria.html
Both solutions offered here by Ruben and Aaron still don't "fully" work for pagination
because the returned object (from executeQuery() and listDistinct) is an ArrayList
(with up to max objects in it), and not PagedResultList with the totalCount property
populated as I would expect for "fully" support pagination.
Let's say the example is a little more complicated in that :
a. assume Role has an additional rolename attribute AND
b. we only want to return distinct User objects with Role.rolename containing a string "a"
(keeping in mind that a User might have multiple Roles with rolename containing a string "a")
To get this done with 2 queries I would have to do something like this :
// First get the *unique* ids of Users (as list returns duplicates by
// default) matching the Role.rolename containing a string "a" criteria
def idList = User.createCriteria().list {
roles {
ilike( "rolename", "%a%" )
}
projections {
distinct ( "id" )
}
}
if( idList ){
// Then get the PagedResultList for all of those unique ids
PagedResultList resultList =
User.createCriteria().list( offset:"5", max:"5" ){
or {
idList.each {
idEq( it )
}
}
order ("username", "asc")
}
}
This seems grossly inefficient.
Question : is there a way to accomplish both of the above with one GORM/HQL statement ?
You can use
User.createCriteria().listDistinct {
maxResults 10
}
Thanks for sharing your issue and Kok for answering it. I didn't have a chance to rewrite it to HQL. Here is my solution (workaround): http://ondrej-kvasnovsky.blogspot.com/2012/01/grails-listdistinct-and-pagination.html
Please tell me if that is useful (at least for someone).