N+1 queries detected what does that mean what should i do to make it work.
I am using Bullet gem for showing N+1 queries
user: kalyan
N+1 Query detected
Emp => [:functions]
Add to your finder: :include => [:functions]
N+1 Query method call stack
/home/Documents/app/views/tics/show.html.haml:20:in `_app_views_tics_show_html_haml___2301551533476719406_237290860'
This is the message from bullet gem.
_app_views_tics_show.html.haml
- if (#tic.assigned_to == current_user.emp) or (current_user.emp_functions.map{|x| x.id}.include?(1) if current_user.emp_functions.present? )
= best_in_place #tic, :subject
- else
= #tic.subject
help me to reduce n+1 query problem
emp.rb
has_many :emp_functions, inverse_of: :emp,dependent: :restrict_with_exception
belongs_to :user, inverse_of: :emp
emp_functions.rb
belongs_to :emp , inverse_of: :emp_functions
belongs_to :function
function.rb
has_many :emp_functions, dependent: :restrict_with_exception
has_many :emp, through: :emp_functions
user.rb
has_one :emp, inverse_of: :user, dependent: :restrict_with_exception
Imho, the most efficient way to get rid of this N+1 query is to alter the current_user method, adding includes(:emp_functions) to the User.find call. The way you do it depends on the way you handle your authentication. If you're using some kind of gem (Devise or Sorcery for example) you'll need to reopen those classes and alter the method, not forgetting to use super. If youэму written your own authentication, you'd be able to do that more easily.
Second thing I noticed is that you don't actually need to use map on your user.emp_functions in the view, provided that emp_functions is a has_many association on the User model. You can just flatten it to current_user.emp_function_ids.include?(1). This will help you get rid of the N+1 query problem, but only in this particular case. If you do tackle the emp_functions of the current user often in many places, I'd recomend using the first path I described.
Third third thing, not directly relevant to the question is that I really don't think that code belongs in the view. You should move it to a helper or a decorator, keeping the view clean and readable.
If emp_functions is a has_many association on your user model (and it seems like it is), You should be doing:
current_user.emp_function_ids.include?(1)
This is a very specific solution to your issue and I advise against using it in a view
change current_user.emp_functions.map{|x| x.id}.include?(1) to
current_user.emp_functions.where(id: 1).exists?
This will result in 1 single query to the db everytime you hit the page containing it.
You can also remove that if statement to remove another db query.
Related
Is it possible to use eager loading as part of a .map() method?
For example, I have the following:
tag_images = tags.map(&:images)
And elsewhere, each of those tag_images has its parent_image accessed with image.parent_image.
The bullet gem is advising I use .includes(:parent_image), but I'm not sure how to apply eager loading in this situation.
Here is how the applicable parts of the models are set up...
Image.rb
has_many :image_tags
has_many :tags, through: :image_tags
has_many :crops, class_name: "Image", foreign_key: "parent_image_id"
belongs_to :parent_image, class_name: "Image"
Tag.rb
has_many :image_tags
has_many :images, through: :image_tags
What's going on is that it is going through a series of one or more tags and "collecting" all of the images associated with them (thus the use of tags.map(&:images)). And later on, the parent_image for each of those images is referenced with image.parent_image.
So ideally, while it's going through to "collect" the tag_images, is there a way it could 'get' the associated parent_images while it's already in there during the initial .map method (or perhaps some other query that is more suitable)?
Any ideas?
Assuming you have a variable tags that contains an enumerble of Tags (like an ActiveRecord_Relation), I think you might want something more like:
tag_images = Image.joins(:tags).where(tags: {id: tags}).distinct.includes(:parent_image)
You may or may not need that .distinct - you'll just have to mess with it to see.
BTW, you can check the Specifying Conditions on the Joined Tables docs to learn more about that .joins(:tags).where(tags: {id: tags}) bit.
If you write it as what Bullet suggested tag_images = tags.includes(:parent_image), that will eager load the included associations and add them in memory. If you call parent_image on any tag instances, it will not fire off any additional queries.
Hi can anyone tell me how can i increase performance if association returns large no. of records. for example in my app :-
class Restaurant < ActiveRecord::Base
has_many :inventory_items
end
class InventoryItem < ActiveRecord::Base
belongs_to :vendor
end
i am trying to find the vendors of my restuarant as follow :-
current_restaurant.inventory_items.includes(:vendor).uniq
current_restaurant.inventory_items returns large no. of records which takes maximum time. so how can i reduce this time please help me.
There are a number of solutions that you can use depending on how your application is configured and what it needs to do -
Only select the columns that you want, for example, if you are only looking for the IDs, you can use the pluck or select methods.
As Chetan suggested in his answer, you can also add scopes, and in addition to that also add indexes for the columns in the scope depending on what kind of columns they are.
If you are looking at calculated values, consider caching them on the Restaurant table.
You can add a scope to your model and add a condition for the records you wanna fetch. Like
scope :your_scope_name, -> { includes(:vendor).where(*some more conditions*) }
This will help query to not to go through all data
Use pagination, loading all records is never recommended..
see will_paginate OR kaminari gems
Update:
class Restaurant < ActiveRecord::Base
has_many :inventory_items
has_many :vendors, through: :inventory_items
end
Then,
current_restaurant.vendors.uniq
It depends on the size of the tables and how they are indexed, one sub query might be faster than a huge join:
Vendor.where(id: current_restaurant.inventory_items.select(:vendor_id).distinct)
I have a polymorphic model that can relate to itself:
class Comment < ActiveRecord::Base
belongs_to :commentable, polymorphic: true
has_many :comments, as: :commentable
end
These relationships work perfectly, except when I'm trying to call the full tree of child/parent comments through an include statement:
Post.find(1).include(:comments)
This only includes the comments directly tied to the post. I could perhaps retrieve a second level with:
Post.find(1).include(comments: :comments)
But what if I wanted to get all comments descending from the post, no matter how deeply nested? Is this possible?
It seems that you want to retrieve an adjacency list. Rails has no immediate support for it, but if you are using postgresql, you can use the "WITH RECURSIVE" operator.
That plugin takes care of it : https://github.com/chrisroberts/acts_as_sane_tree
Otherwise, you can pretty easily create your own postgresql function (declare it in a migration), and then use it in your queries. Have a look at : http://wiki.postgresql.org/wiki/Getting_list_of_all_children_from_adjacency_tree
WITH RECURSIVE is not currently implemented in mysql or sqlite3.
I have a polymorphic association like this -
class Image < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Page < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Site < ActiveRecord::Base
has_one :approval, :as => :approvable
end
class Approval < ActiveRecord::Base
belongs_to :approvable, :polymorphic => true
end
I need to find approvals where approval.apporvable.deleted = false
I have tried something like this -
#approvals = Approval.find(:all,
:include => [:approvable],
:conditions => [":approvable.deleted = ?", false ])
This gives "Can not eagerly load the polymorphic association :approvable" error
How can the condition be given correctly so that I get a result set with approvals who's approvable item is not deleted ?
Thanks for any help in advance
This is not possible, since all "approvables" reside in different tables. Instead you will have to fetch all approvals, and then use the normal array methods.
#approvals = Approval.all.select { |approval| !approval.approvable.deleted? }
What your asking, in terms of SQL, is projecting data from different tables for different rows in the resultset. It is not possible to my knowledge.
So you'll have to be content with:
#approvals = Approval.all.reject{|a| a.approvable.deleted? }
# I assume you have a deleted? method in all the approvables
I would recommend either of the answers already presented here (they are the same thing) but I would also recommend putting that deleted flag into the Approval model if you really care to do it all in a single query.
With a polymorphic relationship rails can use eager fetching on the polys, but you can't join to them because yet again, the relationships are not known so the query is actually multiple queried intersected.
So in the end if you REALLY need to, drop into sql and intersect all the possible joins you can do to all the types of approvables in a single query, but you will have to do lots of joining manually. (manually meaning not using rails' built-in mechanisms...)
thanks for your answers
I was pretty sure that this couldn't be done. I wanted some more confirmation
besides that I was hoping for some other soln than looping thru the result set
to avoid performance related issues later
Although for the time being both reject/select are fine but in the long run I
will have to do those sql joins manually.
Thanks again for your help!!
M
The business logic is this: Users are in a Boat through a join table, I guess let's call that model a Ticket. But when a User instance wants to check who else is on the boat, there's a condition that asks if that user has permission see everyone on the Boat, or just certain people on the Boat. If a User can see everyone, the normal deal is fine: some_user.boats.first.users returns all users with a ticket for that boat. But for some users, the only people that are on the boat (as far as they're concerned) are people in, let's say the dining room. So if User's ticket is "tagged" (using an acts_as_taggable style system) with "Dining Room", the only Users returned from some_user.boats.first.users should be Users with tickets tagged "Dining Room".
Just for the record, I'm not trying to design something to be insane from the getgo - I'm trying to wedge this arbitrary grouping into a (mostly) existent system.
So we've got:
class User
has_many :tickets
has_many :boats, :through => :tickets
end
class Ticket
belongs_to :user
belongs_to :boat
end
class Boat
has_many :tickets
has_many :users, :through => :tickets
end
Initially, I thought that I could conditionally modify the virtual class like:
singleton = class << a_user_instance ; self ; end
singleton.class_eval(<<-code
has_many :tickets, :include => :tags, :conditions => ['tags.id in (?)', [#{tag_ids.to_s(:db)}]]
code
)
That gets all the way down to generating the SQL, but when generated, it generates SQL ending in:
LEFT OUTER JOIN "tags" ON ("tags"."id" = "taggings"."tag_id") WHERE ("tickets"._id = 1069416589 AND (tags.id in (5001,4502)))
I've tried digging around the ActiveRecord code, but I can't find anywhere that would prefix that 'id' in the SQL above with an underscore. I know that associations are loaded when an ActiveRecord class is loaded, and I'd assume the same with a singleton class. shrug.
I also used an alias_method_chain like:
singleton = class << a_user_instance ; self ; end
singleton.class_eval(<<-code
def tickets_with_tag_filtering
tags = Tag.find(etc, etc)
tickets_without_tag_filtering.scoped(:include => :tags, :conditions => {:'tags.id' => tags})
end
alias_method_chain :tickets, :tag_filtering
code
)
But while that approach produces the desired Tickets, any joins on those tickets use the conditions in the class, not the virtual class. some_user.boats.first.users returns all users.
Any type of comment will be appreciated, especially if I'm barking up the wrong tree with this approach. Thanks!
So a wild guess about your underscore issue is that Rails is generating the assocation code based on the context at the time of evaluation. Being in a singleton class could mess this up, like so:
"#{owner.table_name}.#{association.class.name}_id = #{association.id}"
You could get in there and define a class name property on your singleton class and see if that fixes the issue.
On the whole I don't recommend this. It creates behavior that is agonizing to track down and impossible to extend effectively. It creates a landmine in the codebase that will wound you or someone you love at a later time.
Instead, consider using a named_scope declaration:
class User
has_many :taggings, :through => :tickets
named_scope :visible_to, lambda { |looking_user|
{ :include => [ :tickets, :taggings ],
:conditions => [ "tickets.boat_id in (?) and taggings.ticket_id = tickets.id and taggings.tag_id in (?)", looking_user.boat_ids, looking_user.tag_ids ]
}
}
end
While you may have to go back and edit some code, this is much more flexible in the ways it can be used:
Boat.last.users.visible_to( current_user )
It's clear that a restriction is being placed on the find, and what the purpose of that restriction is. Because the conditions are dynamically calculated at runtime, you can deal with the next weird modification your client hits you with. Say some of their users have xray vision and clairvoyance:
class User
named_scope :visible_to, lambda { |looking_user|
if looking_user.superhuman?
{}
else
{ :include => [ :tickets, :taggings ],
:conditions => [ "tickets.boat_id in (?) and taggings.ticket_id = tickets.id and taggings.tag_id in (?)", looking_user.boat_ids, looking_user.tag_ids ]
}
end
}
end
By returning an empty hash, you can effectively nullify the effect of the scope.
Why not just grab all users on the boat and include their tags.
Then run a quick filter to include & return only the users with the same tag as the inquiring user.
What version of Rails are you using? Have you tried upgrading to see if the underscore issue is fixed? It's like it can't find the foreign key to put in as "tag_id" or somethin'.
My ruby-fu is limited, so I'm not sure how to dynamically include the correct method options at run-time.
Just to help you clarify, you have to worry about this two places. You want to filter a user's viewable users so they only see users with the same tags. Your structure is:
user <--> tickets <--> boats <--> tickets <--> users
... right?
So, you need to filter both sets of tickets down to the ones with the current_user's tags.
Maybe you just need a current_user.viewable_users() method and then filter everything through that? I'm not sure what existing functionality you've got to preserve.
Blech, I don't feel like I'm helping you at all. Sorry.
Your approach is the problem. I know it seems expedient at the moment to hack something in where you don't have to refactor the existing call sites, but I believe given time this will come back to haunt you as the source of bugs and complexity.
Sleeping dogs that lie come back to bite you hard, in my experience. Typically in the form of a future developer who doesn't know your association is "magic" and uses it assuming it's just pail ole rails. He/she likely won't even have a reason to write a test case that would expose the behavior either, which raises the odds you'll only find out about the bug when it's live in production and the client is unhappy. Is it really worth the time you're saving now?
Austinfrombostin is pointing the way. Different semantics? Different names. Rule number one is always to write code that says what it does as clearly as possible. Anything else is the path of madness.