For example I have a blog object, and that blog has many posts. I want to do eager loading of say the first blog object and include say the first 10 posts of it. Currently I would do #blogs = Blog.limit(4) and then in the view use #blogs.posts.limit(10). I am pretty sure there is a better way to do this via an include such as Blog.include(:posts).limit(:posts=>10). Is it just not possible to limit the number of included objects, or am I missing something basic here?
Looks like you can't apply a limit to :has_many when eager loading associations for multiple records.
Example:
class Blog < ActiveRecord::Base
has_many :posts, :limit => 5
end
class Post < ActiveRecord::Base
belongs_to :blog
end
This works fine for limiting the number of posts for a single blog:
ruby-1.9.2-p290 :010 > Blog.first.posts
Blog Load (0.5ms) SELECT `blogs`.* FROM `blogs` LIMIT 1
Post Load (0.6ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`blog_id` = 1 LIMIT 5
However, if you try to load all blogs and eager load the posts with them:
ruby-1.9.2-p290 :011 > Blog.includes(:posts)
Blog Load (0.5ms) SELECT `blogs`.* FROM `blogs`
Post Load (1.1ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`blog_id` IN (1, 2)
Note that there's no limit on the second query, and there couldn't be - it would limit the number of posts returned to 5 across all blogs, which is not at all what you want.
EDIT:
A look at the Rails docs confirms this. You always find these things the minute you've figured them out :)
If you eager load an association with a specified :limit option, it
will be ignored, returning all the associated objects
You need to limit the number of posts in your blog model like this:
class Blog < ActiveRecord::Base
has_many :included_posts, :class_name => 'Post', :limit => 10
has_many :posts
end
So then you can do:
$ Blog.first.included_posts.count
=> 10
$ Blog.first.posts.count
=> 999
Related
So let's say I have the following models in Rails.
class Post < ApplicationRecord
belongs_to :user
end
class User < ApplicationRecord
has_many :posts
end
When I put the Post instance in a variable and call user on it, the following sql query runs once, after that the result it is saved/cached.
post.user
User Load (0.9ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
#<User id: 1 ...>
post.user
#<User id: 1 ...>
etc.
However, when I go the other way around with user.posts, it always runs the query.
user.posts
Post Load (1.0ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`user_id` = 1 LIMIT 11
user.posts
Post Load (1.0ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`user_id` = 1 LIMIT 11
etc.
Unless I convert it to an array, in which case it does get saved.
user.posts.to_a
Post Load (1.0ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`user_id` = 1 LIMIT 11
user.posts
# No query here.
But user.posts still produces a ActiveRecord::Associations::CollectionProxy object on which I can call other activerecord methods. So no real downside here.
Why does this work like this? It seems like an unnecessary impact on sql optimization. The obvious reason I can think of is that user.posts updates correctly when a Post is created after user is set. But in reality that doesn't happen so much imo, since variables are mostly set and reset in consecutively ran Controller actions. Now I know there is a caching system in place that shows a CACHE Post sql in the server logs, but I can't really rely on that when I'm working with activerecord objects between models or with more complex queries.
Am I missing some best practice here? Or an obvious setting that fixes exactly this?
You're examining this in irb and that's why you're seeing the query always running.
In an actual block of code in a controller or other class if you were to write
#users_posts = user.posts
The query is NOT executed... not until you iterate through the collection, or request #count
irb, to be helpful, always runs queries immediately
This is a follow up question to watching a video at codeschool on Scopes, as well as watching a video by Chris Oliver on the merge method.
What I am trying to do is find only those authors which have at least one book that is available. Then after it filters for those authors, I want to eager load all of the books for those selected authors because I do not want to query the database each time I pull data out about those books. I tried a number of different scopes but none of them are giving me exactly what I need:
#app/models/book.rb
class Book < ActiveRecord::Base
belongs_to :author
scope :available, ->{where(availability: true)}
scope :unavailable, ->{where(availability: false)}
end
#app/models/author.rb
class Author < ActiveRecord::Base
has_many :books, dependent: :destroy
scope :with_available_books, ->{joins(:books).merge(Book.available)}
scope :with_available_books_uniq, ->{uniq.joins(:books).merge(Book.available)}
scope :with_available_books_includes, ->{joins(:books).merge(Book.available).includes(:books)}
scope :with_available_books_uniq_includes, ->{uniq.joins(:books).merge(Book.available).includes(:books)}
def to_s
self.name
end
end
Here is what is a snapshot of what is in my databases
I have three authors:
Neil, and he has 10 associated books total, ALL are available
John, and he has 10 associated books total, ALL are unavailable
Mixture Author, and he has 10 books total, 5 are available, 5 are unavailable
I ran all the queries and outputted the results in HTML. Here is what I am getting:
# Duplicating the authors AND N + 1 problem with associated books
Author.with_available_books.size: 15
Books for author[0]: 10
Books for author[1]: 10
# Fixed the duplication but still N + 1 problem with the associated books
Author.with_available_books_uniq.size: 2
Books for author[0]: 10
Books for author[1]: 10
# Fixed the N + 1 problem but duplicating authors
Author.with_available_books_includes.size: 15
# Fixed the duplication and fixed the N + 1 problem
# BUT now it is filtering out the unavailable books!
# But I want all the Books for these authors!
Author.with_available_books_uniq_includes.size: 2
Books for author[0]: 10
Books for author[1]: 5
How do I grab ALL the books for the unduplicated authors? I want to filter the authors by their associated object's attribute (the available attribute on the books), and I want to eager load those books.
Thanks TREMENDOUSLY to Chris Oliver himself for responding to my email query about this situation.
First grab the authors:
#uniq_authors_with_available_books = Author.with_available_books_uniq
This properly grabs Neil and Mixture Author who both have available books:
2.2.1 :004 > #authors_with_available_books.size
=> 2
However, as can be seen below, the N + 1 problem is still present if we want to grab any info about those two authors' books:
2.2.1 :005 > #authors_with_available_books[0].books.size
(0.2ms) SELECT COUNT(*) FROM "books" WHERE "books"."author_id" = ? [["author_id", 1]]
=> 10
2.2.1 :006 > #authors_with_available_books[1].books.size
(0.2ms) SELECT COUNT(*) FROM "books" WHERE "books"."author_id" = ? [["author_id", 3]]
=> 10
So thanks to Chris's advice. What we have to do is a separate query on books with a subquery using the id's from the #authors_with_available_books:
2.2.1 :007 > #books_by_those_authors = Book.where(author_id: #authors_with_available_books.map(&:id))
Book Load (0.4ms) SELECT "books".* FROM "books" WHERE "books"."author_id" IN (1, 3)
From the sql query we can see that it is grabbing only those books where the id is equal to 1 or 3. It is only grabbing those authors because the first query told us that only those authors are the ones with available books.
Now I can do this:
#books.size
=> 20
Which makes sense because Neil has ten total books and Mixture Author has ten total books, yielding a combined total of 20.
Is there a way to avoid the n+1 problem when eager loading and also applying a limit to the subquery?
I want to avoid lots of sql queries like this:
Category.all.each do |category|
category.posts.limit(10)
end
But I also want to only get 10 posts per category, so the standard eager loading, which gets all the posts, does not suffice:
Category.includes(:posts).all
What is the best way to solve this problem? Is N+1 the only way to limit the amount of posts per category?
From the Rails docs
If you eager load an association with a specified :limit option, it will be ignored, returning all the associated objects
So given the following model definition
class Category < ActiveRecord::Base
has_many :posts
has_many :included_posts, -> { limit 10 }, class_name: "Post"
end
Calling Category.find(1).included_posts would work as expected and apply the limit of 10 in the query. However, if you try to do Category.includes(:included_posts).all the limit option will be ignored. You can see why this is the case if you look at the SQL generated by an eager load
Category.includes(:posts).all
Category Load (0.2ms) SELECT "categories".* FROM "categories"
Post Load (0.4ms) SELECT "posts".* FROM "posts" WHERE "posts"."category_id" IN (1, 2, 3)
If you added the LIMIT clause to the posts query, it would return a total of 10 posts and not 10 posts per category as you might expect.
Getting back to your problem, I would eager load all posts and then limit the loaded collection using first(10)
categories = Category.includes(:posts).all
categories.first.posts.first(10)
Although you're loading more models into memory, this is bound to be more performant since you're only making 2 calls against the database vs. n+1. Cheers.
In a very simple forum made from Rails app, I get 30 topics from the database in the index action like this
def index
#topics = Topic.all.page(params[:page]).per_page(30)
end
However, when I list them in the views/topics/index.html.erb, I also want to have access to the first post in each topic to display in a tooltip, so that when users scroll over, they can read the first post without having to click on the link. Therefore, in the link to each post in the index, I add the following to a data attribute
topic.posts.first.body
each of the links looks like this
<%= link_to simple_format(topic.name), posts_path(
:topic_id => topic), :data => { :toggle => 'tooltip', :placement => 'top', :'original-title' => "#{ topic.posts.first.body }"}, :class => 'tool' %>
While this works fine, I'm worried that it's an n+1 query, namely that if there's 30 topics, it's doing this 30 times
User Load (0.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = 1 ORDER BY "users"."id" ASC LIMIT 1
Post Load (0.4ms) SELECT "posts".* FROM "posts" WHERE "posts"."topic_id" = $1 ORDER BY "posts"."id" ASC LIMIT 1 [["topic_id", 7]]
I've noticed that Rails does automatic caching on some of these, but I think there might be a way to write the index action differently to avoid some of this n+1 problem but I can figure out how. I found out that I can
include(:posts)
to eager load the posts, like this
#topics = Topic.all.page(params[:page]).per_page(30).includes(:posts)
However, if I know that I only want the first post for each topic, is there a way to specify that? if a topic had 30 posts, I don't want to eager load all of them.
I tried to do
.includes(:posts).first
but it broke the code
This appears to work for me, so give this a shot and see if it works for you:
Topic.includes(:posts).where("posts.id = (select id from posts where posts.topic_id = topics.id limit 1)").references(:posts)
This will create a dependent subquery in which the posts topic_id in the subquery is matched up with the topics id in the parent query. With the limit 1 clause in the subquery, the result is that each Topic row will contain only 1 matching Post row, eager loaded thanks to the includes(:post).
Note that when passing an SQL string to .where, that references an eager loaded relation, the references method should be appended to inform ActiveRecord that we're referencing an association, so that it knows to perform appropriate joins in the subsequent query. Apparently it technically works without that method, but you get a deprecation warning, so you might as well throw it in lest you encounter problems in future Rails updates.
To my knowledge you can't. Custom association is often used to allow conditions on includes except limit.
If you eager load an association with a specified :limit option, it will be ignored, returning all the associated objects. http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html
class Picture < ActiveRecord::Base
has_many :most_recent_comments, -> { order('id DESC').limit(10) },
class_name: 'Comment'
end
Picture.includes(:most_recent_comments).first.most_recent_comments
# => returns all associated comments.
There're a few issues when trying to solve this "natively" via Rails which are detailed in this question.
We solved it with an SQL scope, for your case something like:
class Topic < ApplicationRecord
has_one :first_post, class_name: "Post", primary_key: :first_post_id, foreign_key: :id
scope :with_first_post, lambda {
select(
"topics.*,
(
SELECT id as first_post_id
FROM posts
WHERE topic_id = topics.id
ORDER BY id asc
LIMIT 1
)"
)
}
end
Topic.with_first_post.includes(:first_post)
I have two simple models, ProjectSetting and ProjectSettingQuestions. ProjectSettingQuestions belongs to ProjectSetting. I want to load ProjectSettingQuestions data when I query ProjectSetting. Can't seem to possibly do that with this query:
ProjectSetting.includes(:project_setting_questions).where(:project_id=>params[:project_id])
params is not the issue. This line gets ProjectSetting data but not the questions. Thanks!
Logs show the following:
ProjectSetting Load (0.3ms) SELECT "project_settings".* FROM "project_settings" WHERE "project_settings"."project_id" = 31 LIMIT 1
ProjectSettingQuestion Load (0.4ms) SELECT "project_setting_questions".* FROM "project_setting_questions" WHERE "project_setting_questions"."project_setting_id" IN (2)
Assuming you are having has_many and belongs_to relationship, you can do it like this:
project_setting = ProjectSetting.first
project_setting.project_setting_questions
Or you can also do it this way, since ProjectSettingQuestions belongs_to ProjectSetting, it contains the foreign_key of ProjectSetting
project_setting = ProjectSetting.first
ProjectSettingQuestion.where(:project_setting_id => project_setting.id)
You should take a look at this guide if you haven't already:
http://guides.rubyonrails.org/association_basics.html
Hope it helps.