Getting the last record is trivial in SQL, e.g. (for MySQL)
class ExchangeRate {
Currency from
Currency to
BigDecimal rate // e.g. 10
Date dateCreated
}
class Currency {
String iso
etc..
}
SQL to get the latest is trivial:
Select max(id), rate
from exchange_rate
where from_id = 1
and to_id = 3
or
select rate
from exchange_rate
where from_id = 2
order by id desc
limit 1
The question is, how does one do this efficiently in Grails? I only want a single result.
This obviously wont work:
def query = ExchangeRate.where {
from == from && to == to && id == max(id)
}
ExchangeRate exchangeRate = query.find()
There have been several posts on this, but not with an actual answer which I could figure out how to apply (I am a SQL guy, and don't know hibernate language and would prefer a solution which did not involve it if there was one)
If there was an easy way to run SQL directly without having to hand manage result sets that would work (as we will never use another DB other than MySQL)
I am sure it could be done with sort and limit, but a) haven't found an example I could copy, and b) would assume this be inefficient, because it appears that the sorting and limiting is done in code, not SQL?
This example is in the documentation:
Book.findAll("from Book as b where b.author=:author",
[author: 'Dan Brown'], [max: 10, offset: 5])
could lead to this:
def exchangeRates = ExchangeRate.findAll("from ExchangeRate as e where e.from = :from order by id desc", [from: from], [max: 1])
if (exchangeRates.size() == 1) {
return exchangeRates.first().rate
}
return null
is there a better way (e.g. one which doesnt use hibernate low level language, or one which uses SQL instead, or one which is more efficient?)
Try using a subquery according to the documentation.
def query = ExchangeRate.where {
id = max(id).of { from == fromValue } && to == toValue
}
query.find()
Related
I am using grails-2.5.6 version. I am using spring-security-core plugin. I have a criteria query on UserRole table. Where I want to find all distinct users by a role. It is working properly.
But the problem is the pagination effect. When I am counting on the list it is counting on UserRole list object. But I need the count on distinct projection items. Here is my attempt below:
def list(Integer max) {
def userInstanceList = UserRole.createCriteria().list(params) {
createAlias('user', 'au')
createAlias('role', 'ar')
projections {
distinct("user")
}
if (params.roleId) {
eq('ar.id', params.getLong("roleId"))
}
}
def totalCount = userInstanceList.totalCount
[userInstanceList: userInstanceList, totalCount: totalCount]
}
Here, totalCount is the number of UserRole list. But I want the distinct projection count.
I would tackle this slightly differently, you want to analyse the users, not the userroles.
So I'd do something like:
List<User> usersWithRole = UserRole.createCriteria().list(params) {
role {
eq('id', params.getLong("roleId"))
}
}*.user
int count = usersWithRole.size()
Unless of course there's hundreds or thousands of users, in which case I wouldn't want to load all of them each time and would revert to SQL.
Is this a custom version of spring security you're using? I've never seen Roles with a 'long' based ID, usually, the key is a String representing the Authority name.
Usually the DBAs see the use of distinct keyword as a code-smell.
In your case I would rather use the User as the main domain object to run the query against and a group by clause:
long id = params.getLong "roleId"
def list = User.createCriteria().list( params ) {
projections{
groupProperty 'userRole.role.id'
}
if( id )
userRole{
role{
eq 'id', id
}
}
}
I have nodes- named "options". "Users" choose these options. I need a chpher query that works like this:
retrieve users who had chosen all the options those are given as a list.
MATCH (option:Option)<-[:CHOSE]-(user:User) WHERE option.Key IN ['1','2','2'] Return user
This query gives me users who chose option(1), option(2) and option(3) and also gives me the user who only chose option(2).
What I need is only the users who chose all of them -option(1), option(2) and option(3).
For an all cypher solution (don't know if it's better than Chris' answer, you'll have to test and compare) you can collect the option.Key for each user and filter out those who don't have a option.Key for each value in your list
MATCH (u:User)-[:CHOSE]->(opt:Option)
WITH u, collect(opt.Key) as optKeys
WHERE ALL (v IN {values} WHERE v IN optKeys)
RETURN u
or match all the options whose keys are in your list and the users that chose them, collect those options per user and compare the size of the option collection to the size of your list (if you don't give duplicates in your list the user with an option collection of equal size has chosen all the options)
MATCH (u:User)-[:CHOSE]->(opt:Option)
WHERE opt.Key IN {values}
WITH u, collect(opt) as opts
WHERE length(opts) = length({values}) // assuming {values} don't have duplicates
RETURN u
Either should limit results to users connected with all the options whose key values are specified in {values} and you can vary the length of the collection parameter without changing the query.
If the number of options is limited, you could do:
MATCH
(user:User)-[:Chose]->(option1:Option),
(user)-[:Chose]->(option2:Option),
(user)-[:Chose]->(option3:Option)
WHERE
option1.Key = '1'
AND option2.Key = '2'
AND option3.Key = '3'
RETURN
user.Id
Which will only return the user with all 3 options.
It's a bit rubbishy as obviously you end up with 3 lines where you have 1, but I don't know how to do what you want using the IN keyword.
If you're coding against it, it's pretty simple to generate the WHERE and MATCH clause, but still - not ideal. :(
EDIT - Example
Turns out there is some string manipulation going on here (!), but you can always cache bits. Importantly - it's using Params which would allow neo4j to cache the queries and supply faster responses with each call.
public static IEnumerable<User> GetUser(IGraphClient gc)
{
var query = GenerateCypher(gc, new[] {"1", "2", "3"});
return query.Return(user => user.As<User>()).Results;
}
public static ICypherFluentQuery GenerateCypher(IGraphClient gc, string[] options)
{
ICypherFluentQuery query = new CypherFluentQuery(gc);
for(int i = 0; i < options.Length; i++)
query = query.Match(string.Format("(user:User)-[:CHOSE]->(option{0}:Option)", i));
for (int i = 0; i < options.Length; i++)
{
string paramName = string.Format("option{0}param", i);
string whereString = string.Format("option{0}.Key = {{{1}}}", i, paramName);
query = i == 0 ? query.Where(whereString) : query.AndWhere(whereString);
query = query.WithParam(paramName, options[i]);
}
return query;
}
MATCH (user:User)-[:CHOSE]->(option:Option)
WHERE option.key IN ['1', '2', '3']
WITH user, COUNT(*) AS num_options_chosen
WHERE num_options_chosen = LENGTH(['1', '2', '3'])
RETURN user.name
This will only return users that have relationships with all the Options with the given keys in the array. This assumes there are not multiple [:CHOSE] relationships between users and options. If it is possible for a user to have multiple [:CHOSE] relationships with a single option, you'll have to add some conditionals as necessary.
I tested the above query with the below dataset:
CREATE (User1:User {name:'User 1'}),
(User2:User {name:'User 2'}),
(User3:User {name:'User 3'}),
(Option1:Option {key:'1'}),
(Option2:Option {key:'2'}),
(Option3:Option {key:'3'}),
(Option4:Option {key:'4'}),
(User1)-[:CHOSE]->(Option1),
(User1)-[:CHOSE]->(Option4),
(User2)-[:CHOSE]->(Option2),
(User2)-[:CHOSE]->(Option3),
(User3)-[:CHOSE]->(Option1),
(User3)-[:CHOSE]->(Option2),
(User3)-[:CHOSE]->(Option3),
(User3)-[:CHOSE]->(Option4)
And I get only 'User 3' as the output.
For shorter lists, you can use path predicates in your WHERE clause:
MATCH (user:User)
WHERE (user)-[:CHOSE]->(:Option { Key: '1' })
AND (user)-[:CHOSE]->(:Option { Key: '2' })
AND (user)-[:CHOSE]->(:Option { Key: '3' })
RETURN user
Advantages:
Clear to read
Easy to generate for dynamic length lists
Disadvantages:
For each different length, you will have a different query that has to be parsed and cached by Cypher. Too many dynamic queries will watch your cache hit rate go through the floor, query compilation work go up, and query performance go down.
so I have a form with several fields which are criteria for searching in a database.
I want to formulate a query using LINQ like so:
var Coll = (from obj in table where value1 = criteria1 && value2 = criteria2...)
and so on.
My problem is, I don't want to write it using If statements to check if every field has been filled in, nor do I want to make separate methods for the various search cases (criteria 1 and criteria 5 input; criteria 2 and criteria 3 input ... etc.)
So my question is: How can I achieve this without writing an excessive amount of code? If I just write in the query with comparison, will it screw up the return values if the user inputs only SOME values?
Thanks for your help.
Yes, it will screw up.
I would go with the ifs, I don't see what's wrong with them:
var query = table;
if(criteria1 != null)
query = query.Where(x => x.Value1 == criteria1);
if(criteria2 != null)
query = query.Where(x => x.Value2 == criteria2);
If you have a lot of criteria you could use expressions, a dictionary and a loop to cut down on the repetitive code.
In an ASP.NET MVC app, chances are your user input is coming from a form which is being POSTed to your server. In that case, you can make use of strongly-typed views, using a viewmodel with [Required] on the criteria that MUST be provided. Then you wrap your method in if (ModelState.IsValid) { ... } and you've excluded all the cases where the user hasn't given you something they need.
Beyond that, if you can collect your criteria into a list, you can filter it. So, you could do something like this:
filterBy = userValues.Where(v => v != null);
var Coll = (from obj in table where filterBy.Contains(value1) select obj);
You can make this more complex by having a Dictionary (or Lookup for non-unique keys) that contains a user-entered value along with some label (an enum, perhaps) that tells you which field they're filtering by, and then you can group them by that label to separate out the filters for each field, and then filter as above. You could even have a custom SearchFilter object that contains other info, so you can have filters with AND, NOT and OR conditions...
Failing that, you can remember that until you trigger evaluation of an IQueryable, it doesn't hit the database, so you can just do this:
var Coll = (from obj in table where value1 == requiredCriteria select obj);
if(criteria1 != null)
{
query = query.Where(x => x.Value1 == criteria1);
}
//etc...
if(criteria5 != null)
{
query = query.Where(x => x.Value5 == criteria5);
}
return query.ToList();
That first line applies any criteria that MUST be there; if there aren't any mandatory ones then it could just be var Coll = table;.
That will add any criteria that are provided will be applied, any that aren't will be ignored, you catch all the possible combinations, and only one query is made at the end when you .ToList() it.
As I understand of your question you want to centralize multiple if for the sake of readability; if I were right the following would be one of some possible solutions
Func<object, object, bool> CheckValueWithAnd = (x, y) => x == null ? true : x==y;
var query = from obj in table
where CheckValue(obj.value1, criteria1) &&
CheckValue(obj.value2, criteria2) &&
...
select obj;
It ls flexible because in different situations or scenarios you can change the function in the way that fulfill your expectation and you do not need to have multiple if.
If you want to use OR operand in your expression you need to have second function
Func<object, object, bool> CheckValueWithOr = (x, y) => x == null ? false : x==y;
I am using telerik mvc grid in my mvc project , my table have around 1 Million records. My grid taking too much time to load.
This is my Query
//
var bib = (from a in db.Bibs
join inf in db.InfoTypes
on a.InfoTypeId equals inf.Id
where a.Status == "A"
select new BibViewModel
{
Id = a.Id,
Type = inf.Type,
InfoType = inf.Description,
Title = (from asd in db.BibContents where asd.BibId == a.Id && asd.TagNo == "245" && asd.Sfld == "a" select asd.Value).FirstOrDefault(),
Author = (from asd in db.BibContents where asd.BibId == a.Id && asd.TagNo == "100" && asd.Sfld == "a" select asd.Value).FirstOrDefault(),
CatalogueDate = a.CatalogDate,
Contents = "",
CreatedOn = a.CreatedOn,
ItemRelation = db.Items.Any(item => item.BibId == a.Id),
IssueRelation = db.Issues.Any(item => item.BibId == a.Id),
});
return View(new GridModel(bib.OrderByDescending(x => x.CreatedOn).Tolist()));
ToList() actually invokes the query, so if calling ToList() is taking too long, that means the issue is with the query.
In LINQ, you can use paging like in the following post; the idea is to use Skip and Take to skip X records, and only take Y records, as in:
var results = (from .. select ..).Skip(X).Take(Y)
With 1M records, I would highly suggest replacing it with a stored procedure, which will be much, much faster for what you are trying to do. Consider a custom pagination approach, which works very well for me with large result sets:
http://www.neiland.net/blog/article/dynamic-paging-in-sql-server-stored-procedures/
http://www.beansoftware.com/ASP.NET-Tutorials/Paging-Stored-Procedures.aspx
http://www.sqlpointers.com/2009/10/custom-sorting-and-paging-in-sql-server.html
T-SQL stored procedure with sorting and paging enabled not working properly
If you can't use stored procedures, reading this will help understand what needs to be accomplished with pagination. With LINQ, you'll want to examine the SQL being generated to see where you also can fine-tune the query, either using SQL profiler, or LINQPad.
I am using Nimble and Shiro for my security frameworks and I've just come accross a GORM bug. Indeed :
User.createCriteria().list {
maxResults 10
}
returns 10 users whereas User.list(max: 10) returns 9 users !
After further investigations, I found out that createCriteria returns twice the same user (admin) because admin has 2 roles!!! (I am not joking).
It appears that any user with more than 1 role will be returned twice in the createCriteria call and User.list will return max-1 instances (i.e 9 users instead of 10 users)
What workaround can I use in order to have 10 unique users returned ?
This is a very annoying because I have no way to use pagination correctly.
My domain classes are:
class UserBase {
String username
static belongsTo = [Role, Group]
static hasMany = [roles: Role, groups: Group]
static fetchMode = [roles: 'eager', groups: 'eager']
static mapping = {
roles cache: true,
cascade: 'none',
cache usage: 'read-write', include: 'all'
}
}
class User extends UserBase {
static mapping = {cache: 'read-write'}
}
class Role {
static hasMany = [users: UserBase, groups: Group]
static belongsTo = [Group]
static mapping = { cache usage: 'read-write', include: 'all'
users cache: true
groups cache: true
}
}
Less concise and clear, but using an HQL query seems a way to solve this problem. As described in the Grails documentation (executeQuery section) the paginate parameters can be added as extra parameters to executeQuery.
User.executeQuery("select distinct user from User user", [max: 2, offset: 2])
this way you can still use criteria and pass in list/pagination paramaters
User.createCriteria().listDistinct {
maxResults(params.max as int)
firstResult(params.offset as int)
order(params.order, "asc")
}
EDIT: Found a way to get both! Totally going to use it now
http://www.intelligrape.com/blog/tag/pagedresultlist/
If you call createCriteria().list() like this
def result=SampleDomain.createCriteria().list(max:params.max, offset:params.offset){
// multiple/complex restrictions
maxResults(params.max)
firstResult(params.offset)
} // Return type is PagedResultList
println result
println result.totalCount
You will have all the information you need in a nice PagedResultList format!
/EDIT
Unfortunately I do not know how to get a combination of full results AND max/offset pagination subset in the same call. (Anyone who can enlighten on that?)
I can, however, speak to one way I've used with success to get pagination working in general in grails.
def numResults = YourDomain.withCriteria() {
like(searchField, searchValue)
order(sort, order)
projections {
rowCount()
}
}
def resultList = YourDomain.withCriteria() {
like(searchField, searchValue)
order(sort, order)
maxResults max as int
firstResult offset as int
}
That's an example of something I'm using to get pagination up and running. As KoK said above, I'm still at a loss for a single atomic statement that gives both results. I realize that my answer is more or less the same as KoK now, sorry, but I think it's worth pointing out that rowCount() in projections is slightly more clear to read, and I don't have comment privileges yet :/
Lastly: This is the holy grail (no pun intended) of grails hibernate criteria usage references; bookmark it ;)
http://www.grails.org/doc/1.3.x/ref/Domain%20Classes/createCriteria.html
Both solutions offered here by Ruben and Aaron still don't "fully" work for pagination
because the returned object (from executeQuery() and listDistinct) is an ArrayList
(with up to max objects in it), and not PagedResultList with the totalCount property
populated as I would expect for "fully" support pagination.
Let's say the example is a little more complicated in that :
a. assume Role has an additional rolename attribute AND
b. we only want to return distinct User objects with Role.rolename containing a string "a"
(keeping in mind that a User might have multiple Roles with rolename containing a string "a")
To get this done with 2 queries I would have to do something like this :
// First get the *unique* ids of Users (as list returns duplicates by
// default) matching the Role.rolename containing a string "a" criteria
def idList = User.createCriteria().list {
roles {
ilike( "rolename", "%a%" )
}
projections {
distinct ( "id" )
}
}
if( idList ){
// Then get the PagedResultList for all of those unique ids
PagedResultList resultList =
User.createCriteria().list( offset:"5", max:"5" ){
or {
idList.each {
idEq( it )
}
}
order ("username", "asc")
}
}
This seems grossly inefficient.
Question : is there a way to accomplish both of the above with one GORM/HQL statement ?
You can use
User.createCriteria().listDistinct {
maxResults 10
}
Thanks for sharing your issue and Kok for answering it. I didn't have a chance to rewrite it to HQL. Here is my solution (workaround): http://ondrej-kvasnovsky.blogspot.com/2012/01/grails-listdistinct-and-pagination.html
Please tell me if that is useful (at least for someone).