Let's say I have a simple model association, where a blog Post has many Comments on it.
class Post < ActiveRecord::Base
has_may :comments
end
If I wanted to avoid "N + 1" queries and eager load all the associations beforehand, I could do -
Post.includes(:comments).where(title: "foo")
Which runs two queries. The first one looks up the Post using the where condition and the second looks up all the associated comments at once.
But what if I already have a Post object? Can I "add" an includes to it after the initial result set to run the 2nd bulk query that looks up all the associations?
It seems counter intuitive to do a delayed eager load, but I assume looking up all the associations at once would still save me from having to look them up individually as I loop through them.
e.g.
p = Post.where(title: "foo")
# Will the below work?
p.includes(:comments)
p.comments.each do |comment|
puts comment.text
end
Let's break down your code.
posts = Post.where(title: 'foo')
That searches for all posts with a title of foo. posts is an ActiveRecord::Relation object and you can chain more ActiveRecord commands like select, limit, includes, etc.
So doing posts.includes(:comments) is valid and should eager load the comments. Just don't forget to assign the result to posts again (or another variable).
What will not work is posts.comments because comments is a method that works on an instance of Post. Changing to the following code will work
posts = Post.where(title: 'foo')
posts = posts.includes(:comments)
posts.each do |post|
post.comments.each do |comment|
puts comment.text
end
end
Related
I don't understand the Rails includes method as well as I'd like, and I ran into an issue that I'm hoping to clarify. I have a Board model that has_many :members and has_many :lists (and a list has_many :cards). In the following boards controller, the show method looks as follows:
def show
#board = Board.includes(:members, lists: :cards).find(params[:id])
...
end
Why is the includes method needed here? Why couldn't we just use #board = Board.find(params[:id]) and then access the members and lists via #board.members and #board.lists? I guess I'm not really seeing why we would need to prefetch. It'd be awesome if someone could detail why this is more effective in terms of SQL queries. Thanks!
Per the Rails docs:
Eager loading is the mechanism for loading the associated records of
the objects returned by Model.find using as few queries as possible.
When you simply load a record and later query its different relationships, you have to run a query each time. Take this example, also from the Rails docs:
clients = Client.limit(10)
clients.each do |client|
puts client.address.postcode
end
This code executes a database call 11 times to do something pretty trivial, because it has to do each lookup, one at a time.
Compare the above code to this example:
clients = Client.includes(:address).limit(10)
clients.each do |client|
puts client.address.postcode
end
This code executes a database call 2 times, because all of the necessary associations are included at the onset.
Here's a link to the pertinent section of the Rails docs.
Extra
A point to note as of recent: if you do a more complex query with an associated model like so:
Board.includes(:members, lists: :cards).where('members.color = ?', 'foo').references(:members)
You need to make sure to include the appended references(:used_eager_loaded_class) to complete the query.
I have an Event model which has many DateEntry because an event can have multiple datetimes.
I have a view where I see all the events with date entries for a specific day. For which I've made this class method in models/event.rb:
def self.by_date(date)
includes(:date_entries)
.where(date_entries: { begins_at: date.beginning_of_day..date.end_of_day })
end
And it works fine.
Now on the view I loop over the events I grabbed with that query:
#events.each do |event|
# whatever
I am looking for a way to use the date from the selected DateEntry in the loop. I think I have to use ActiveRecords joins method, but I have tried in many ways and when I am not getting an error the output is still the same.
For clarifying, I want to do in the loop something like event.selected_date_entry_by_the_by_date_method.begins_at or something like that.
First, in your example the date for all events will be the same. You could easily set #date in your controller and then use that in the views.
Second, It seems like you should be doing things the other way around. Instead of finding the Events you should be finding DateEntries and iterating over those.
Your method Event.by_date used includes so it eager loads the date_entries for the events you need. It automatically does a join. In the loop just do
#events.each do |event|
data_entries = event.date_entries
# do something with the dates...
end
Your Event model should have has_many :date_entries.
The date_entries call will return all the dates for that event and won't result in another query.
Run this in the rails console and you will see the queries being run.
You can do it this way, but i suggest to do it as followng, this class method should be a scope or even class method but in DateEntry model:
class DateEntry < ActiveRecord::Base
scope :by_date,->(date){ where(...)}
end
#events.each do |event|
event.date_entries.by_date(date).each do |date_entry|
# .....
end
# .....
end
I have a rails 4 that has a Post model related to a Tag model by an habtm relation.
Tags have a name field and a category field. Multiple tags can have the same category.
I need a view that displays only posts that have at least one tag belonging to the "foo" category. Foo is static and will always remain "foo".
I've been able to make it work using this code in my posts controller:
def myview
ids = []
Tag.where(category: 'foo').each do |tag|
tag.posts.each do |post|
ids << post.id
end
end
#posts = Post.where(id: ids).all
end
Despite working my code looks really ugly to read.
I'm sure rails provides a way like "#posts = Post.where tag categories include 'foo'.all" but I cannot figure out a way. I'm sure I'm missing something very obvoius.
Post.joins(:tags).where('tags.category = ?', "foo").all
The answer in the comment is where, for example, you didn't want to query by the tag but maybe wanted to include some information relating to the tag in your view. By using includes, you will avoid the N+1 problem.
I have a users object that I'm exposing through the simple call:
#users = User.all
I also have some more expensive queries I'm doing where I'm using custom SQL to generate the results.
#comment_counts = User.received_comment_counts
This call and others like it will return objects with one element per user in my list ordered by id.
In my view I'd like to loop through the users and the comment_counts object at the same time. Eg:
for user, comment_count in #users, #comment_counts do
end
I can't figure out how to do this. I also can't seem to figure out how to get an individual record from the #comment_counts result without running it through #comment_counts.each
Isn't there some way to build an iterator for each of these lits objects, dump them into a while loop and step the iterator individually on each pass of the loop? Also, if you can answer this where can I go to learn about stepping through lists in rails.
Something like this should work for you:
#users.zip(#comment_counts).each do |user, comments_count|
# Do something
end
I've never come across manual manipulation of iterators in Ruby, not to say that it can't be done. Code like that would likely be messy and so I would likely favor more elegant solutions.
Assuming the arrays have equal values:
In your controller:
#users = User.all
#comments = Comment.all
In your view.
<% #users.each_with_index do |user, i |%>
<% debug user %>
<% debug #comments[i] %>
<% end %>
Then you can just check if the values exists on #comments if you don't know if the arrays have the same number of objects.
It's not obvious because your data requires a bit of restructuring. comment_count clearly belongs to instances of User and this should be reflected so that only loop on one collection (of Users) is necessary. I can think of two possible ways to achieve this:
Eager loading. Only works for associated models, not the case here probably. Say, each of your users has comments, so writing User.includes(:comments).all would return all users with their comments and only do 2 queries to fetch the data.
For counting each user's comments you could just use counter_cache (clickable). Then you'll have one more field in the users table for caching how many comments a user has. Again, works for associated Comments only.
I'm using Mongoid to work with MongoDB in Rails.
What I'm looking for is something like active record include. Currently I failed to find such method in mongoid orm.
Anybody know how to solve this problem in mongoid or perhaps in mongomapper, which is known as another good alternative.
Now that some time has passed, Mongoid has indeed added support for this. See the "Eager Loading" section here:
http://docs.mongodb.org/ecosystem/tutorial/ruby-mongoid-tutorial/#eager-loading
Band.includes(:albums).each do |band|
p band.albums.first.name # Does not hit the database again.
end
I'd like to point out:
Rails' :include does not do a join
SQL and Mongo both need eager loading.
The N+1 problem happens in this type of scenario (query generated inside of loop):
.
<% #posts.each do |post| %>
<% post.comments.each do |comment| %>
<%= comment.title %>
<% end %>
<% end %>
Looks like the link that #amrnt posted was merged into Mongoid.
Update: it's been two years since I posted this answer and things have changed. See tybro0103's answer for details.
Old Answer
Based on the documentation of both drivers, neither of them supports what you're looking for. Probably because it wouldn't solve anything.
The :include functionality of ActiveRecord solves the N+1 problem for SQL databases. By telling ActiveRecord which related tables to include, it can build a single SQL query, by using JOIN statements. This will result in a single database call, regardless of the amount of tables you want to query.
MongoDB only allows you to query a single collection at a time. It doesn't support anything like a JOIN. So even if you could tell Mongoid which other collections it has to include, it would still have to perform a separate query for each additional collection.
Although the other answers are correct, in current versions of Mongoid the includes method is the best way to achieve the desired results. In previous versions where includes was not available I have found a way to get rid of the n+1 issue and thought it was worth mentioning.
In my case it was an n+2 issue.
class Judge
include Mongoid::Document
belongs_to :user
belongs_to :photo
def as_json(options={})
{
id: _id,
photo: photo,
user: user
}
end
end
class User
include Mongoid::Document
has_one :judge
end
class Photo
include Mongoid::Document
has_one :judge
end
controller action:
def index
#judges = Judge.where(:user_id.exists => true)
respond_with #judges
end
This as_json response results in an n+2 query issue from the Judge record. in my case giving the dev server a response time of:
Completed 200 OK in 816ms (Views: 785.2ms)
The key to solving this issue is to load the Users and the Photos in a single query instead of 1 by 1 per Judge.
You can do this utilizing Mongoids IdentityMap Mongoid 2 and Mongoid 3 support this feature.
First turn on the identity map in the mongoid.yml configuration file:
development:
host: localhost
database: awesome_app
identity_map_enabled: true
Now change the controller action to manually load the users and photos. Note: The Mongoid::Relation record will lazily evaluate the query so you must call to_a to actually query the records and have them stored in the IdentityMap.
def index
#judges ||= Awards::Api::Judge.where(:user_id.exists => true)
#users = User.where(:_id.in => #judges.map(&:user_id)).to_a
#photos = Awards::Api::Judges::Photo.where(:_id.in => #judges.map(&:photo_id)).to_a
respond_with #judges
end
This results in only 3 queries total. 1 for the Judges, 1 for the Users and 1 for the Photos.
Completed 200 OK in 559ms (Views: 87.7ms)
How does this work? What's an IdentityMap?
An IdentityMap helps to keep track of what objects or records have already been loaded. So if you fetch the first User record the IdentityMap will store it. Then if you attempt to fetch the same User again Mongoid queries the IdentityMap for the User before it queries the Database again. This will save 1 query on the database.
So by loading all of the Users and Photos we know we are going to want for the Judges json in manual queries we pre-load the data into the IdentityMap all at once. Then when the Judge requires it's User and Photo it checks the IdentityMap and does not need to query the database.
ActiveRecord :include typically doesn't do a full join to populate Ruby objects. It does two calls. First to get the parent object (say a Post) then a second call to pull the related objects (comments that belong to the Post).
Mongoid works essentially the same way for referenced associations.
def Post
references_many :comments
end
def Comment
referenced_in :post
end
In the controller you get the post:
#post = Post.find(params[:id])
In your view you iterate over the comments:
<%- #post.comments.each do |comment| -%>
VIEW CODE
<%- end -%>
Mongoid will find the post in the collection. When you hit the comments iterator it does a single query to get the comments. Mongoid wraps the query in a cursor so it is a true iterator and doesn't overload the memory.
Mongoid lazy loads all queries to allow this behavior by default. The :include tag is unnecessary.
This could help https://github.com/flyerhzm/mongoid-eager-loading
You need update your schema to avoid this N+1 there are no solution in MongoDB to do some jointure.
Embed the detail records/documents in the master record/document.
In my case I didn't have the whole collection but an object of it that caused n+1 (bullet says that).
So rather than writing below which causes n+1
quote.providers.officialname
I wrote
Quote.includes(:provider).find(quote._id).provider.officialname
That didn't cause a problem but left me thinking if I repeated myself or checking n+1 is unnecessary for mongoid.