Is there any way to get a PagedResultList in grails without using criteria? I would like to avoid criteria as they are slightly more complex and make unit testing rather annoying. Code Below
def pagedResultList = MyDomainClass.createCriteria().list(max:10, offset:0)
{ order("id", "asc") }
//Below does not return pagedResultList
def aList = MyDomainClass.list(sort:"id", order:"asc", max:10, offset: 0)
PagedResultList is just used to wrap the results of Criteria-based queries (you can see its use in the source here). If you really want to use it, you could always just invoke the constructor directly, since it will handle any list. Of course the totalCount property (which is probably what you are interested in) will then be unset.
If the idea is to get both the paged results list and the total number of results, I'm not aware of any magic that can get both in one query (even the use of PagedResultList in the source linked above issues two queries).
This seems to break in Grails 2.x because there is no costructor that just takes a list. Compare http://grails.org/doc/2.1.0/api/grails/orm/PagedResultList.html and http://docs.huihoo.com/grails/1.3.7/api/grails/orm/PagedResultList.html
Related
I have the following code
PagedResultList res = myService.getPage(paginateParams, ...)
println res.size() // returns 2
println res.getTotalCount() // returns 1
getPage looks like:
def criteria = MyDomain.createCriteria()
criteria.list(max: paginateParams.max, offset: paginateParams.offset) { // max is 10, offset is 0, sortBy is updatedAt and sortOrder is desc
eq('org', org)
order(paginateParams.sortBy, paginateParams.sortOrder)
}
why do the two method return different values? The documentation doesn't explain the difference, but does mention that getTotalCount is for number of records
currently on grails 2.4.5
edits:
println on res prints out:
res: [
com.<hidden>.MyDomain: 41679f98-a7c5-4193-bba8-601725007c1a,
com.<hidden>.MyDomain: 41679f98-a7c5-4193-bba8-601725007c1a]
Yes, res has a SINGLE object twice - that's the bug I'm trying to fix. How do I know that? I have an primary key on MyDomain's ID, and when I inspect the database, it's also showing one record for this particular org (see my criteria)
edit 2: I found this comment (http://docs.grails.org/2.4.5/ref/Domain%20Classes/createCriteria.html)
listDistinct If subqueries or associations are used, one may end up
with the same row multiple times in the result set. In Hibernate one
would do a "CriteriaSpecification.DISTINCT_ROOT_ENTITY". In Grails one
can do it by just using this method.
Which, if I understand correctly, is their way of saying "list" method doesn't work in this scenario, use listDistinct instead but then they go on to warn:
The listDistinct() method does not work well with the pagination
options maxResult and firstResult. If you need distinct results with
pagination, we currently recommend that you use HQL. You can find out
more information from this blog post.
However, the blog post is a dead link.
Related: GORM createCriteria and list do not return the same results : what can I do?
Not related to actual problem after question edited but this quote seems useful
Generally PagedResultList .size() perform size() on resultList property (in-memory object represent database record), while .getTotalCount() do count query against database. If this two value didn't match your list may contain duplicate.
After viewing related issues (GORM createCriteria and list do not return the same results : what can I do?) I determined that there were several approaches:
Use grails projection groupBy('id') - doesn't work b/c i need the entire object
USe HSQL - Domain.executeQuery - actually this didn't work for my scenario very well because this returns a list, whereas criteria.list returns a PagedResultList from which I previously got totalCount. This solution had me learning HSQL and also made me break up my existing logic into two components - one that returned PagedResultList and one that didn't
Simply keep a set of IDs as I process my PagedResultList and make sure that I didn't have any duplicates.
I ended up going with option 3 because it was quick, didn't require me to learn a new language (HSQL) and I felt that I could easily write the code to do it and I'm not limited by the CPU to do such a unique ID check.
Problem:
We're querying our database with a lot of entries. The overall application performance is okay. But I think it could be better if our bottleneck, querying a special table, could be done parallel.
We are using for that dynamic finders like:
Object.findAllByObjectCollectionAndParentIsNullAndDeletedByUserIsNull(objectCollection, [sort: 'attr1.number', fetch: [objectType: 'eager']])
I think about this solution: Dividing the query into 2 steps.
The first step loads only the Ids in a sorted way.
The second step (could be done in parallel threads) loads the objects itself by the id and inserts it into the resultset.
I already googled for some hints about that, but find nothing.
Is it possible, that it makes no sense?
Or is there already a adequate solution in grails standard/extensions?
If it makes sense and there is no solution: Can someone give me a hint implementing it? But, we need the ids in that sorted manner, explained in the example.
We're using grails 2.3.11, hibernate:3.6.10.13 with jdk 1.7 under it.
I found a solution!
ArrayList<Long> objectIds = Attribute.executeQuery("select o.id from Object o, ObjectType ot where o.objectCollection = :oc and o.parent is null and o.deletedByUser is null and oc.typeUsage = ot order by ot.nr asc", [oc: objectCollection])
Object[] tmp = new Object[objectIds.size()]
withPool(10, {
objectIds.eachWithIndexParallel {
Long objectId, i ->
Object o = Object.findById(objectId, [cache: true])
tmp[i] = a
}
})
for (Object a : tmp)
a.merge()
Its round about 3 times slower!
I think the reason is, that
we have to create the threads and databaseconnections too.
After loading the objects, they need to be merged into our local thread.
But in the end - we cant use it, beause the lazy loaded fields couldn't be loaded on demand later. So this could be only a solution for read-only Objects
I have been using parameters to query node indexes as such (using the rest api in java)-
final QueryResult<Map<String,Object>> result = engine.query("start nd=node:name_index(name={src}) return nd.age as age", MapUtil.map("src", "Susan");
However I haven't been able to get this to work for a collection of nodes/names. I have been trying something along the lines of-
final QueryResult<Map<String,Object>> result = engine.query("start nd=node:name_index(name={src}) return nd.age as age", MapUtil.map("src", Arrays.asList("Susan","Brian", "Ian"));
But it refuses to compile. I as wondering if there is something wrong in my syntax or that parameters are not designed to work in this context.
The name= syntax in the start is meant to do an index lookup on a property. It won't do an IN lookup. The way you can do this sort of lookup is like this (note it depends on Apache's StringUtils):
List<String> names = Arrays.asList("Susan","Brian", "Ian");
String luceneQuery = "name:("+StringUtils.join(names, ",")+")";
engine.query("start nd=node:name_index({luceneQuery}) return nd.age as age", MapUtil.map("luceneQuery", luceneQuery));
Just a note, this is the "legacy" index way of doing things. In 2.0 they've introduced label-based indexes, which work entirely differently.
Thanks a lot; though it would still only return a non empty answer when I added a space after the comma in line 2. I used-
String luceneQuery = "name:("+StringUtils.join(names, ", ")+")";
and it returned the age of one person. When I tried this:
String luceneQuery = "fs:(fs:"+ StringUtils.join(names, " OR fs:")+")";
it gave me all three ages. However, I am still unsure about whether this query will be able to leverage the usual advantages of parameters , i.e. will the engine be able to reuse the query and execution path the next time around (this time we may want to query for 4 names instead of 3)
can I, and if I can, how, write LINQ statement like this:
public IQueryable<Advert> SearchSimilarAdverst(string query)
{
Levenshtein compute = new Levenshtein();
return _db.Adverts.Where(a => a.IsActive &&
(compute.FindSimilarity(a.Name, query) <= 2));
}
Thanks
EDIT
I've tired solution Jeffery suggested and it worked but when I've tried this line of code I got EntityCommandExecutionException, does anybody know why ?
adverts.Where(a => a.WhoAmILookingForTags.Any
(t => compute.FindSimilarity(t.Name,query) <= 2));
Tags and Adverts are connected with many to many relation, and WhoAmILookingForTags is list of tags
EF will not be able to translate compute.FindSimilarity(a.Name, query) to SQL, so you're going to have to take the performance hit and do the following.
public IEnumerable<Advert> SearchSimilarAdverst(string query)
{
Levenshtein compute = new Levenshtein();
var adverts = _db.Adverts.Where(a => a.IsActive).ToList();
return adverts.Where(a => compute.FindSimilarity(a.Name, query) <= 2));
}
Note the return type of the method also needed to be changed to reflect the return type
As commented below, the query should be forced to run before filtering using FindSimilarity because of delayed execution. query.ToList() is one of many options.
In short, no - because EF needs to be able to convert your Linq predicate into T-SQL (or whatever dialect of SQL your RDBMS uses). Only a subset of .NET BCL functions are supported (such as String.Contains and custom user code is right-out).
For complicated predicates I recommend writing your own SQL by hand - you'll also get considerably better performance, EF can be slow at generating SQL.
I hope I'm wrong on this, but I do not believe that you would be allowed to that in a Linq-to-Entities query. In those kinds of LINQ queries it has to translate everything in the Where function into a SQL statement to be run on the underlying database. Since your method is not something that is on the SQL side of things, it will cause an exception when you try this.
For a regular LINQ query (i.e. after everything has already been put into memory), there should be no problem with accomplishing this.
You could also try it and see if it does or does not work...
I'm attempting to pull up all entities that have a name that doesn't partially match a given string.
MyEntity.findAllByNameNotLike('bad%')
This gives me the following error:
No such property: nameNot for class: MyEntity
Possible solutions: name" type="groovy.lang.MissingPropertyException">
I had a quick look at the criteria style but I can't seem to get that going either,
def results = MyEntity.withCritieria {
not(like('name', 'bad%'))
}
No signature of method: MyEntity.withCritieria() is applicable for argument types: (MyService$_doSomething_closure1)
Ideally I would like to be able to apply this restriction at the finder level as the database contains a large number of entities that I don't want to load up and then exclude for performance reasons.
[grails 1.3.1]
I've worked out how to do this using withCriteria, the not should have been a closure of its own.
def results = MyEntity.withCritieria {
not {
like('name', 'bad%'))
}
}
The problem I initially had using withCriteria was that I was trying to test this as a unit test, which works fine with the dynamic finders, but not with the criteria API (as far as I can tell).
(I'll leave this unanswered for a day to see if anyone has a better solution, otherwise I'll accept my answer)