I've got a complicated query which I can solve using SQL but I can't find a proper GORM way to do the same.
I have Story objects. Each Story object refers to many Tag objects. It's a unidirectional OneToMany relation:
class Story {
...
Collection tags
static hasMany = [
tags: Tag
]
...
}
Several Tags are considered as being "aliased" (technically using a separate "AliasGroup" table, which is not the problem here).
Now, I want to search for multiple tags. Returned stories have to be tagged with ALL of these tags ("Match All tags"). But also, for each of those tags, an 'aliased' tag should also be accepted.
In SQL I solve that by using a dynmically composed sequence of joins. For example, if there are two tags given, both having several "aliases", the resulting SQL statement looks like this (using the actual SQL row names):
select s.id from story
join story_tag st1 on s.id = st1.story_id
join story_tag st2 on s.id = st2.story_id
where
st1.tag_id in (<list of ids for tag1 and all its aliases>) and
st2.tag_id in (<list of ids for tag2 and all its aliases>)
It's important to understand that this cannot be written as a single join, because then the SAME tag would have to be in both groups of tag lists, which is not what I want.
So, this statement is working fine. But I want to achieve the same thing using grails directly. The GORM documentation on joins is rather terse, but I figured that the feature using an association query and 'and' them together would give the correct result. So I tried:
def c = Story.createCriteria()
def stories = c.list {
and {
srchTags.each { tag ->
def ids = []
tag.aliasGroup.aliases.each { alias ->
ids << alias.id
}
tags {
inList('id', ids)
}
}
}
}
This works fine if I just use one single tag as input. I.e. the list of aliased tags ids is properly resolved and the statement is working in principle.
But if I use two or more tags, it doesn't work correctly. The code runs, but GORM seems to just create the wrong SQL query, it seems that it again tries to match each tag with all of the match-lists which cannot work.
I realize that this problem is not easy to understand, and it's even more difficult to describe properly.
Does anyone have a solution how to create a proper Grails query for this problem?
In a project at work now I had several sql queries that I could not translate to gorm(criteria, where). With the need to move forward I used native sql queries in grails and everything goes well.
Please have a look at the article Using Hibernate Native SQL Queries for an explanation of the implementation method.
I know you can not always decide which method to use but having this facility at hand is always useful.
I hope it's useful for you. regards
Related
I have an HABTM association between two models, User and Conversation.
I want to be able to query the logged-in user's conversations (current_user.conversations), passing it another user's id, and have it return a conversation that is shared by both users, if one exists.
(an additional perk would be to have the query create one if one doesn't exist.)
The associations are working fine, so I can access the associated objects through instance variables like #user.conversations and #conversation.users, so I could go the long way and loop through each conversation object and search the user associations in each of them... But there must be an efficient way to construct this query.
What I would like to be able to do is something like current_user.conversations.where(conversation.users.exists?(id: #user_id)) or something like Conversation.find(users: {id: #user_id AND current_user.id}).
I imagine there is an obvious answer to this, and I've been searching around here for similar questions but haven't found any. After looking through the Rails API docs, I imagine that the solution I'm looking for involves .includes() in some way, but I can't get it working.
Any advice? Thanks.
Suppose you have two users #sender and #receiver, and there is one (or perhaps more) conversation between them. One possible way to query their shared conversations is by using the merge function in ActiveRecord. merge will select the intersection of conversations between between them:
#shared_conversations = #sender.conversations.merge(#receiver.conversations)
Note that the result will be a collection, not an individual conversation. In SQL this would translate to an INNER JOIN.
If you wanted to amend this query to create the conversation if it didn't exist, then you can use first_or_create. Like its name implies, first_or_create returns the first item in the collection. If the collection is empty, it will create a new object. You can pass it a block for further control over the newly created object:
#shared_conversations = #sender.conversations.merge(#receiver.conversations).first_or_create do |conversation|
conversation.title = "#{#sender.first_name}'s conversation"
end
You can find more details on merge in the Rails docs: http://apidock.com/rails/ActiveRecord/SpawnMethods/merge.
Try
current_user.conversations.includes(:users).where(user.id:current_user.id)
One of our contractors implemented a repository pattern with code first approach. We use Service Locator as DI pattern. what we do when we retrieve data from DB, we pass interface to GetQueryable function and get the data. However, I see serious performance issues on our application. I implemented MiniProfiler and MiniProfiler.EF to see where the bottleneck is.
We have a case table which has quite a few fields(around 25) and some of those fields are associated to other tables as one to one and one to many(only one field has many relation to other table). when I try to see the case detail, it runs around 400 SQL queries and SQL takes around 40 percent of the load time as far as the miniprofiler concerned. Here our GetQueryable and Find methods
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
Type type = _impls.Value[typeof (T).Name].GetType();
DbSet dbSet = Db.Set(type);
foreach (var include in includes)
{
dbSet.Include(include);
}
return ((IQueryable<T>) dbSet);
}
I added included to this method to attach other related tables, but it did not make any difference. and here is the Find Method
public T Find<T>(long? id)
{
Type type = _impls.Value[typeof(T).Name].GetType();
return (T) Db.Set(type).Find(id);
}
I pretty much tried to apply all the performance improvements, but the number of the SQL queries has not gone down. I tried to disable lazy loading, but it caused many problems in other parts of the application.
Just some additional information, in case table, there are 70000 rows and in out dialogs table, there are 500000 rows. Case and Dialog are associates as one-to-many. and each case has 20-40 dialog entries.
My questions are;
Why does include not make any difference when I use?
Is there any other way to crop number of the queries run?
Do you think the implementation is the problem?
Thanks
Include returns a new IQueryable and does not modify the source query. In addition you can use the generic version of Set which simplifies the code a bit:
public IQueryable<T> GetQueryable<T>(params string[] includes)
{
IQueryable<T> query = Db.Set<T>();
foreach (var include in includes)
{
query = query.Include(include);
}
return query;
}
Step 1: Fire your contractor. Seriously. Like right now. That is some awful code. Not only did they miss something as simple and basic as using the generic version of Set, but they've successfully only made working with Entity Framework more complex, because all the repository does is proxy Entity Framework methods with its own unique and bastardized API.
That said, there's really not enough here to diagnose what your problem is. The use of Include may give you larger queries, but it should actually serve to reduce the overall number of queries issued. It's possible, you're just not using includes where you should be.
Now, the fact that you "tried to disable lazy loading, but it caused many problems in other parts of the application", means that you're relying too heavily on lazy-loading. Basically, you're loading in stuff you don't even know about, which is the antithesis of optimization. Ironically, you'd actually be best served by going ahead and disabling lazy-loading, and then tracking down where your code fails because of that. If you want to actually lazy-load that thing, you can use .Load (see: Explicit Loading). But, if you want to eager-load to reduce queries, then you know what includes you need to add.
I'm trying to override the queryset() of a ModelAdmin class so that the list of objects shown in admin would be sorted by two levels.
I've tried the following code, but it does not work, i.e. the table is not sorted as expected
class ProductAdmin(admin.ModelAdmin):
def queryset(self, request):
qs = super(ProductAdmin, self).queryset(request)
return qs.order_by('category','market')
list_display = ('category', 'market', 'name', 'quantity')
admin.site.register(Product, ProductAdmin)
btw, you can't use ordering = ('category','market') as django specifically states that only the first item in the ordering tuple takes effect (see note in the documentation here)
get_queryset works in Django 1.8.
I had this exactly problem. Here's what I did:
I subclassed ChangeList and overrode ChangeList.get_query_set to redo the correct order_by that was previously changed by ChangeList.get_ordering:
This is what I did in my case (I'm using django-easytree, but the same applies for django-mptt):
class MyChangeList(ChangeList):
def get_query_set(self):
qs = super(MyChangeList, self).get_query_set()
return qs.order_by('tree_id', 'lft')
Also, check these tickets:
"NFA: Don't override order_by if no default ordering is specified"
"Admin ChangeList doesn't apply 'order_by' clause specified by ModelAdmin.queryset"
The release notes for Django 1.4 say that Django now supports Multiple sort in admin interface:
The admin change list now supports sorting on multiple columns. It
respects all elements of the ordering attribute, and sorting on
multiple columns by clicking on headers is designed to mimic the
behavior of desktop GUIs.
And from ModelAdmin ordering:
Set ordering to specify how lists of objects should be ordered in the
Django admin views. This should be a list or tuple in the same format
as a model's ordering parameter. [...] Django honors all elements in the list/tuple; before 1.4, only the first was respected.
On a semi-related note - if you do override queryset to provide a custom sort order, it seems that the Changelist will override that sort order. It applies any sorting found in the ordering parameter, and if there isn't one, it applies a default sort by pk, thus negating any sorting you did in queryset.
I think it's supposed to work - at least this Django Ticket says fixed. But I was just trying to apply a custom sort using queryset a few days ago, and it didn't work at all for me. Even sorting on a single field, seemed to be overridden in the final view. So either I did something wrong, or it's not all that fixed. :)
Note that it is possible to do a custom sort via code, but you have to subclass Changelist, and override its get_query_set() method, as per this snippet. (Although this is overkill, if you only need multiple fields sorted, since Django 1.4 now supports multiple fields).
I have about the following:
class Object_1 {
static hasMany = [tags:Tag]
Set tags;
...
}
Now I have a set of tags and want to find all Object_1-instances with intersecting (!= matching) tags. I was thinking of something like
Object_1.findAllByTagsInList(tags);
But that does not work at all - I get a "nested exception is org.hibernate.exception.SQLGrammarException: could not execute query". I have the feeling I am missing something important. Help highly appreciated.
I actually found an elegant way to solve the problem. I redesigned the relationship to be many-to-many which allows for simply iterating over the tags list finding all the relevant objects.
... of course now I have to take care of that relationship a couple of times - but I am happy to have this working with few locs.
"in list" is not one of the operators recognized in dynamic finder methods - that won't work.
Instead, you'll have to use HQL or the criteria builder to formulate your query (and perhaps put it in a static finder method).
I'm very new to grails (day 2).
First, I find it hard to find easily browsable resources (the documentation is very raw, and the tutorials are spread out and only show 'hello world' types of examples ).
I've set up my domain class with a relationship to other data types.
class ShopCategoryPage{
//some stuff
ProductProcedure procedure;
ProductCategory category;
//some stuff
}
In my controller, I am getting a category id and a procedure id as parameters, and I am trying to get the ShopCategoryPage associated with those parameters.
How do I "find" them? I tried passing the ids as procedureId or procedure_id, I tried passing a ProductProcedure object generated by findById ...
I'm not sure how to find by a property that is not of native type.
First, I find it hard to find easily browsable resources (the documentation is very raw,
and the tutorials are spread out and only show 'hello world' types of examples ).
In my opinion the documentation is great, perhaps we're using different docs. I use:
Grails Reference Documentation
Grails JavaDoc
GDK, i.e. methods Groovy adds to Java classes
If that still isn't satisfactory, I highly recommend the book "The Definitive Guide to Grails". I believe "Grails in Action" is also very good, but haven't read it. For learning Groovy, "Programming Groovy" is a great book (albeit a little out of date).
In my controller, I am getting a category id and a procedure id as parameters, and I am
trying to get the ShopCategoryPage associated with those parameters.
The easiest way (though not the most efficient) is to use the dynamic finders.
// First of all load the ProductProcedure and ProductCategory
// I'm assuming here the request params are named 'procedureId' and 'categoryId'
ProductProcedure productProcedure = ProductProcedure.get(params.procedureId.toLong())
ProductCategory productCategory = ProductCategory .get(params.categoryId.toLong())
// Now get the ShopCategoryPage associated with these. Replace 'find' with 'findAll'
// if there could be multiple associated ShopCategoryPages
ShopCategoryPage shopCategoryPage = ShopCategoryPage.findByProcedureAndCategory(productProcedure, productCategory)
A shortcoming of this approach is that it will cause 3 SELECT statements to be executed. If you're only interested in the shopCategoryPage returned by the last query, you could load this in "one shot" using HQL or a criteria query instead.
You should never use findById since it'll bypass the id-based 2nd-level cache and only use the query cache, which is a lot more volatile. Use get() instead.
Having said that, there's two ways to do this. One is to use get():
def shopCategoryPage = ShopCategoryPage.findByProcedureAndCategory(
ProductProcedure.get(params.procedureId),
ProductCategory.get(params.categoryId))
(use the appropriate param name for the two ids)
or using an HQL query (or a criteria query) to fetch the instance in one query instead of 3:
def shopCategoryPage = ShopCategoryPage.executeQuery(
'from ShopCategoryPage p where p.procedure.id=:procedureId and p.category=:categoryId',
[procedureId: params.procedureId.toLong(), categoryId: params.categoryId.toLong()])[0]
I agree, the easiest way is to use an dynamic finder.
In addition to 'The Definitive Guide to Grails' by Rocher and Brown, I suggest the IBM developerWorks track 'Mastering Grails' by Scott Davis.
http://www.ibm.com/developerworks/views/java/libraryview.jsp?site_id=1&contentarea_by=Java&sort_by=Date&sort_order=1&start=1&end=19&topic_by=&product_by=&type_by=All%20Types&show_abstract=true&search_by=mastering%20grails