I'm looking for help with Ruby optimization regarding loading of associations on demand.
This is simplified example. I have 3 models: Post, Comment, User. References are: Post has many comments and Comment has reference to User (:author). Now when I go to the post page, I expect to see post body + all comments (and their respective authors names). This requires following 2 queries:
select * from Post -- to get post data (1 row)
select * from Comment inner join User -- to get comment + usernames (N rows)
In the code I have:
Post.find(params[:id], :include => { :comments => [:author] }
But it doesn't work as expected: as I see in the back end, there're still N+1 hits (some of them are cached though). How can I optimize that?
UPD
After some investigation, it looks like code was correct, but it doesn't work as expected in case I have named belongs_to in a Comment model. Once I changed from :author to :user, it worked as expected.
In my project I have a similar relationship to your Post, Comment, and User models. I only see three actual sql queries.
Post.find(1, :include => { :comments => [:author] })
From the debug log it shows these three queries
SELECT * FROM `posts` WHERE (`posts`.`id` = 1)
SELECT `comments`.* FROM `comments` WHERE (`comments`.`post_id` = 1)
SELECT * FROM `authors` WHERE (`authors`.`id` IN (4,8,15,16,23,42))
If you are happy with 2/3 queries, you can try:
#post = Post.find params[:id]
#comments = Comments.find_by_post_id(params[:id], :include => [:author])
or
#comments = #post.comments(:include => [:author])
Edit: Have you tried with:
Post.find(params[:id], :include => { :comments => :author }
Related
I have below table structure. This is just an example
UserPost => user_id, post_id, Post, Comment
So, If I try to fetch all user_posts using the below query and do where on the comments table then it fires query for the comments table
user_posts = UserPost.includes(post: :comments)
user_posts.each do |up|
post = up.post # No Query
comments = up.comments # No query
comments_with_condition = up.comments.where(visibility: true).order(position: :asc).first.data # Fires query for .where and .order as well.
end
So, is this the expected behavior or I am doing something wrong?
How to prevent the query for each user_post
What you can do is add another has_many to your model with a filter of what you want.
# You can name this anything you want but a descriptive name helps
has_many :special_comments, -> { where(visibility: true).order(..) }, class_name: 'Comment'
...and eager load that in your query which will eager load both type of comments. This will inevitably lead to one extra query, but it is not N+1.
user_post = UserPost.includes(post: [:comments, :special_comments])
user_posts.each do |user_post|
post = user_post.post
comments = user_post.comments
comments_with_condition = user_post.special_comments
end
I have Page and Paragraph models with a has_and_belongs_to_many relation. Given a paragraph_id, I'd like to get all matching pages. e.g.:
pages = Paragraph.find(paragraph_id).pages.all
However, this takes two queries. It can be done in one query:
SELECT "pages".* FROM "pages"
INNER JOIN "pages_paragraphs" ON "pages_paragraphs"."page_id" = "pages"."id"
WHERE "pages_paragraphs"."paragraph_id" = 123
But can this be done without
using find_by_sql
without modifications to the page_paragraphs table (e.g. adding an id).
Update:
My page model looks like this:
class Page < ActiveRecord::Base
has_and_belongs_to_many :paragraphs, uniq: true
end
With a has_many :through relationship, you could use this:
pages = Page.joins(:pages_paragraphs).where(:pages_paragraphs => {:paragraph_id => 1})
Take a look at Specifying Conditions on the Joined Tables here: http://guides.rubyonrails.org/active_record_querying.html
If you want the pages and paragraphs together:
pages = Page.joins(:pages_paragraphs => :paragraph).includes(:pages_paragraphs => :paragraph).where(:pages_paragraphs => {:paragraph_id => 1})
With a has_and_belongs_to_many:
pages = Page.joins("join pages_paragraphs on pages.id = pages_paragraphs.page_id").where(["pages_paragraphs.paragraph_id = ?", paragraph_id])
You can use eager_load for this
pages = Paragraph.eager_load(:pages).find(paragraph_id).pages
I found an article all about this sort of thing:
3 ways to do eager loading (preloading) in Rails 3 & 4 by Robert Pankowecki
You can use includes for this :
pages = Paragraph.includes(:pages).find(paragraph_id).pages
I have a database model set up such that a post has many votes, a user has many votes and a post belongs to both a user and a post. I'm using will paginate and I'm trying to create a filter such that the user can sort a post by either the date or the number of votes a post has. The date option is simple and looks like this:
#posts = Post.paginate :order => "date DESC"
However, I can't quite figure how to do the ordering for the votes. If this were SQL, I would simply use GROUP BY on the votes user_id column, along with the count function and then I would join the result with the posts table.
What's the correct way to do with with ActiveRecord?
1) Use the counter cache mechanism to store the vote count in Post model.
# add a column called votes_count
class Post
has_many :votes
end
class Vote
belongs_to :post, :counter_cache => true
end
Now you can sort the Post model by vote count as follows:
Post.order(:votes_count)
2) Use group by.
Post.select("posts.*, COUNT(votes.post_id) votes_count").
join(:votes).group("votes.post_id").order(:votes_count)
If you want to include the posts without votes in the result-set then:
Post.select("posts.*, COUNT(votes.post_id) votes_count").
join("LEFT OUTER JOIN votes ON votes.post_id=posts.id").
group("votes.post_id").order(:votes_count)
I prefer approach 1 as it is efficient and the cost of vote count calculation is front loaded (i.e. during vote casting).
Just do all the normal SQL stuff as part of the query with options.
#posts = Post.paginate :order => "date DESC", :join => " inner join votes on post.id..." , :group => " votes.user_id"
http://apidock.com/rails/ActiveRecord/Base/find/class
So I don't know much about your models, but you seem to know somethings about SQL so
named scopes: you basically just put the query into a class method:
named_scope :index , :order => 'date DESC', :join => .....
but they can take parameters
named_scope :blah, {|param| #base query on param }
for you, esp if you are more familiar with SQL you can write your own query,
#posts = Post.find_by_sql( <<-SQL )
SELECT posts.*
....
SQL
I'm having trouble with Rails 3 using conditions on an associated table while eager loading. It appears that Rails is applying the condition when it loads the original model data, so it won't load the parent model unless a non-zero number of the child/associated models match the condition. This is easier to explain in code (simplified for example):
#post = Post.includes(:comments).where(:comments => { :approved => true }).find(1)
This would generate a SQL query similar to:
SELECT DISTINCT `posts`.id FROM `posts`
LEFT OUTER JOIN `comments` ON `comments`.`post_id` = `posts`.`id`
WHERE (`comments`.`approved` = 1) AND (`posts`.`id` = '1')
LIMIT 1
In the case that there aren't any comments that meet the approved = 1 condition, no rows are returned, and thus the Post never gets loaded at all.
What is the right way to load a post and the associated comments eagerly with a condition on the comments?
Update
I'd stil love to hear a better way of doing this, but for now I'm using the following to work around it (works with deeply nested eager loading):
#post = Post.find(1)
#comments = #post.comments.where(:approved => true).all
# allows deeper/more complex nesting without getting into SQL:
#post = Post.includes(:author => [ :websites, :photo ]).find(1)
#comments = #post.comments.includes(:editor).where(:approved => true).all
I guess what you are looking for is joins method, it will let you put your condition within join definition, not outside of it. For example:
#post = Post.joins("LEFT JOIN comments on posts.id = comments.post_id AND comments.approved = 1").first
Not sure about the correctness of the condition itself but you get my point.
Unfortunately you have to use that ugly string as joins is using INNER JOIN if you pass array/hash.
There's more about joins at rails guides
Update: There might be some nugget of wisdom in this post on includes vs eager_load vs preload.
I'd still love to hear a better way of doing this, but for now I'm using the following to work around it (works with deeply nested eager loading, unlike using joins):
#post = Post.find(1)
#comments = #post.comments.where(:approved => true).all
# allows deeper/more complex nesting without getting into SQL:
#post = Post.includes(:author => [ :websites, :photo ]).find(1)
#comments = #post.comments.includes(:editor).where(:approved => true).all
I need to delete a lot of records at once and I need to do so based on a condition in another model that is related by a "belongs_to" relationship. I know I can loop through each checking for the condition, but this takes forever with my large record set because for each "belongs_to" it makes a separate query.
Here is an example. I have a "Product" model that "belongs_to" an "Artist" and lets say that artist has a property "is_disabled".
If I want to delete all products that belong to disabled artists, I would like to be able to do something like:
Product.delete_all(:joins => :artist, :conditions => ["artists.is_disabled = ?", true])
Is this possible? I have done this directly in SQL before, but not sure if it is possible to do through rails.
The problem is that delete_all discards all the join information (and rightly so). What you want to do is capture that as an inner select.
If you're using Rails 3 you can create a scope that will give you what you want:
class Product < ActiveRecord::Base
scope :with_disabled_artist, lambda {
where("product_id IN (#{select("product_id").joins(:artist).where("artist.is_disabled = TRUE").to_sql})")
}
end
You query call then becomes
Product.with_disabled_artist.delete_all
You can also use the same query inline but that's not very elegant (or self-documenting):
Product.where("product_id IN (#{Product.select("product_id").joins(:artist).where("artist.is_disabled = TRUE").to_sql})").delete_all
In Rails 4 (I tested on 4.2) you can almost do how OP originally wanted
Application.joins(:vacancy).where(vacancies: {status: 'draft'}).delete_all
will give
DELETE FROM `applications` WHERE `applications`.`id` IN (SELECT id FROM (SELECT `applications`.`id` FROM `applications` INNER JOIN `vacancies` ON `vacancies`.`id` = `applications`.`vacancy_id` WHERE `vacancies`.`status` = 'draft') __active_record_temp)
If you are using Rails 2 you can't do the above. An alternative is to use a joins clause in a find method and call delete on each item.
TellerLocationWidget.find(:all, :joins => [:widget, :teller_location],
:conditions => {:widgets => {:alt_id => params['alt_id']},
:retailer_locations => {:id => #teller_location.id}}).each do |loc|
loc.delete
end