In RoR, it is pretty common mistake for new people to load a class and assiocations like this# the solution to eager load
# The bellow generates an insane amount of queries
# post has many comments
# If you have 10 posts with 5 comments each
# this will run 11 queries
posts = Post.find(:all)
posts.each do |post|
post.comments
end
The solution is pretty simple to eager load
# should be 2 queries
# no matter how many posts you have
posts = Post.find(:all, :include => :comments) # runs a query to get all the comments for all the posts
posts.each do |post|
post.comments # runs a query to get the comments for that post
end
But what if you don't have access to the class methods, and only have access to a collection of instance methods.
Then you are stuck with the query intensive lazy loading.
Is there a way to minimize queries to get all the comments for the collection of posts, from the collection of instances?
Addition for Answer (also added to the code above)
So to eager load from what I can see in the rdoc for rails is a class method on any extension of ActiveRecord::Associations, the problem is say you you don't have the ability to use a class method, so you need to use some sort of instance method
a code example of what I think it would look like would be is something like
post = Posts.find(:all)
posts.get_all(:comments) # runs the query to build comments into each post without the class method.
In Rails 3.0 and earlier you can do:
Post.send :preload_associations, posts, :comments
You can pass arrays or hashes of association names like you can to include:
Post.send :preload_associations, posts, :comments => :users
In Rails 3.1 this has been moved and you use the Preloader like this:
ActiveRecord::Associations::Preloader.new(posts, :comments).run()
And since Rails 4 its invocation has changed to:
ActiveRecord::Associations::Preloader.new.preload(posts, :comments)
I think I get what you're asking.
However, I don't think you have to worry about what methods you have access to. The foreign key relationship (and the ActiveRecord associations, such as has_many, belongs_to, etc.) will take care of figuring out how to load the associated records.
If you can provide a specific example of what you think should happen, and actual code that isn't working, it would be easier to see what you're getting at.
How are you obtaining your collection of model instances, and what version of Rails are you using?
Are you saying that you have absolutely no access to either controllers or models themselves?
giving you the best answer depends on knowing those things.
Related
Imagine we have an Article and a Comment model. We setup our routes like:
# routes.rb
resources :articles do
resources :comments
end
Now, we can destroy a comment via the CommentController, but there are a number of approaches that I've seen been implemented.
# method 1
def destroy
Comment.where(article_id: params[:article_id]).find(params[:id]).destroy
end
# method 2
def destroy
Comment.find(params[:id]).destroy
end
# method 3
def destroy
article = Article.find(params[:article_id])
comment = article.comments.find(params[:id])
comment.destroy
end
Which is better and why?
I've seen in old Railscasts episodes and blogs that we should do the former for "security" reasons or because it is better to ensure that comments can only be found within their respective article, but why is that better? I haven't been able to find anything that goes too deep into the answer.
When you are working with nested data in this way, it's better to scope your model lookup under the parent to avoid people iterating through ids in a simplistic way. You generally want to discourage that, and it will protect you against more serious security problems if it's a habit.
For instance, say you have some sort of visibility permission on Article. With method 2, it is possible to use an article_id that you are allowed to see to access Comments that you aren't.
Methods 1 & 3 are ostensibly doing the same thing, but I would prefer 1 because it uses fewer trips to the DB.
I don't understand the Rails includes method as well as I'd like, and I ran into an issue that I'm hoping to clarify. I have a Board model that has_many :members and has_many :lists (and a list has_many :cards). In the following boards controller, the show method looks as follows:
def show
#board = Board.includes(:members, lists: :cards).find(params[:id])
...
end
Why is the includes method needed here? Why couldn't we just use #board = Board.find(params[:id]) and then access the members and lists via #board.members and #board.lists? I guess I'm not really seeing why we would need to prefetch. It'd be awesome if someone could detail why this is more effective in terms of SQL queries. Thanks!
Per the Rails docs:
Eager loading is the mechanism for loading the associated records of
the objects returned by Model.find using as few queries as possible.
When you simply load a record and later query its different relationships, you have to run a query each time. Take this example, also from the Rails docs:
clients = Client.limit(10)
clients.each do |client|
puts client.address.postcode
end
This code executes a database call 11 times to do something pretty trivial, because it has to do each lookup, one at a time.
Compare the above code to this example:
clients = Client.includes(:address).limit(10)
clients.each do |client|
puts client.address.postcode
end
This code executes a database call 2 times, because all of the necessary associations are included at the onset.
Here's a link to the pertinent section of the Rails docs.
Extra
A point to note as of recent: if you do a more complex query with an associated model like so:
Board.includes(:members, lists: :cards).where('members.color = ?', 'foo').references(:members)
You need to make sure to include the appended references(:used_eager_loaded_class) to complete the query.
Here are the relevant facts:
Each topic has_many comments.
The Comment model has a scope called very_popular, which we'll pretend involves comparing a several of its columns.
def self.very_popular
# lots of cool stuff
end
The Topic model has a scope called exciting, which includes all topics with very_popular comments.
Number 3 is where I'm stuck. The following results in a missing method exception, and as pitiful as it sounds, I don't know what else to try!
def self.exciting
join(:comments).very_popular
end
How can I re-use the very_popular scope from the Comment model in the Topic model's scope?
You can't use the scope from another model directly. What you can do is merge the queries.
Topic.joins(:comments).merge(Comment.very_popular)
Ryan explains it beautifully here: http://railscasts.com/episodes/215-advanced-queries-in-rails-3
Using ActiveResource, Ruby on Rails, Is there a clean way to do the following:
I have house ActiveResource model and chair ActiveResource model, and both can have comments. I want to reuse the comment ActiveResource model for both.
# ActiveResource wraps HTTP POST Requests to the following
# And then parsess the responses, and then creates instances of my ActiveResource models
POST http://3rd.party.restful.api.com/admin/houses/1/comments
POST http://3rd.party.restful.api.com/admin/houses/1/chairs/3/comments
I can only think of the following:
class Comment < ActiveResource::Base
self.site = "http://3rd.party.restful.api.com"
self.prefix = "/admin/:prefix_path/"
end
And then doing the following:
comment = Comment.new(:text => "some text", :prefix_path => "houses/1/chairs/3")
Please stop me from doing this.
I think you're asking if you can define the routes for each while only using one model, right?
There are two options:
First, the simplest way: just define the route twice.
resources :houses do
resources :comments
resources :chairs do
resources :comments
end
end
The routes file won't care that you're telling it the comments model can be reached from two places, and it will work basically as you expect -- views will just live in the 'comments' folder.
A second, more complex way to do it, define a namespace that you want to nest under. Then you'll end up with two controllers: CommentsController and Chairs::CommentsContoller. You might also create two sets of views, but you don't have to (the second controller can simply explicitly render the first controller's views).
For a good explanation of how namespacing would work, you can see the answer where I originally learned about it.
For what it's worth, this second approach is nice, because you can make some minor tweaks in how the model is presented depending on how it's accessed, but you've still only got one model in the DB.
Good luck! I'll be happy to try and answer questions in the comments!
I'm using Mongoid to work with MongoDB in Rails.
What I'm looking for is something like active record include. Currently I failed to find such method in mongoid orm.
Anybody know how to solve this problem in mongoid or perhaps in mongomapper, which is known as another good alternative.
Now that some time has passed, Mongoid has indeed added support for this. See the "Eager Loading" section here:
http://docs.mongodb.org/ecosystem/tutorial/ruby-mongoid-tutorial/#eager-loading
Band.includes(:albums).each do |band|
p band.albums.first.name # Does not hit the database again.
end
I'd like to point out:
Rails' :include does not do a join
SQL and Mongo both need eager loading.
The N+1 problem happens in this type of scenario (query generated inside of loop):
.
<% #posts.each do |post| %>
<% post.comments.each do |comment| %>
<%= comment.title %>
<% end %>
<% end %>
Looks like the link that #amrnt posted was merged into Mongoid.
Update: it's been two years since I posted this answer and things have changed. See tybro0103's answer for details.
Old Answer
Based on the documentation of both drivers, neither of them supports what you're looking for. Probably because it wouldn't solve anything.
The :include functionality of ActiveRecord solves the N+1 problem for SQL databases. By telling ActiveRecord which related tables to include, it can build a single SQL query, by using JOIN statements. This will result in a single database call, regardless of the amount of tables you want to query.
MongoDB only allows you to query a single collection at a time. It doesn't support anything like a JOIN. So even if you could tell Mongoid which other collections it has to include, it would still have to perform a separate query for each additional collection.
Although the other answers are correct, in current versions of Mongoid the includes method is the best way to achieve the desired results. In previous versions where includes was not available I have found a way to get rid of the n+1 issue and thought it was worth mentioning.
In my case it was an n+2 issue.
class Judge
include Mongoid::Document
belongs_to :user
belongs_to :photo
def as_json(options={})
{
id: _id,
photo: photo,
user: user
}
end
end
class User
include Mongoid::Document
has_one :judge
end
class Photo
include Mongoid::Document
has_one :judge
end
controller action:
def index
#judges = Judge.where(:user_id.exists => true)
respond_with #judges
end
This as_json response results in an n+2 query issue from the Judge record. in my case giving the dev server a response time of:
Completed 200 OK in 816ms (Views: 785.2ms)
The key to solving this issue is to load the Users and the Photos in a single query instead of 1 by 1 per Judge.
You can do this utilizing Mongoids IdentityMap Mongoid 2 and Mongoid 3 support this feature.
First turn on the identity map in the mongoid.yml configuration file:
development:
host: localhost
database: awesome_app
identity_map_enabled: true
Now change the controller action to manually load the users and photos. Note: The Mongoid::Relation record will lazily evaluate the query so you must call to_a to actually query the records and have them stored in the IdentityMap.
def index
#judges ||= Awards::Api::Judge.where(:user_id.exists => true)
#users = User.where(:_id.in => #judges.map(&:user_id)).to_a
#photos = Awards::Api::Judges::Photo.where(:_id.in => #judges.map(&:photo_id)).to_a
respond_with #judges
end
This results in only 3 queries total. 1 for the Judges, 1 for the Users and 1 for the Photos.
Completed 200 OK in 559ms (Views: 87.7ms)
How does this work? What's an IdentityMap?
An IdentityMap helps to keep track of what objects or records have already been loaded. So if you fetch the first User record the IdentityMap will store it. Then if you attempt to fetch the same User again Mongoid queries the IdentityMap for the User before it queries the Database again. This will save 1 query on the database.
So by loading all of the Users and Photos we know we are going to want for the Judges json in manual queries we pre-load the data into the IdentityMap all at once. Then when the Judge requires it's User and Photo it checks the IdentityMap and does not need to query the database.
ActiveRecord :include typically doesn't do a full join to populate Ruby objects. It does two calls. First to get the parent object (say a Post) then a second call to pull the related objects (comments that belong to the Post).
Mongoid works essentially the same way for referenced associations.
def Post
references_many :comments
end
def Comment
referenced_in :post
end
In the controller you get the post:
#post = Post.find(params[:id])
In your view you iterate over the comments:
<%- #post.comments.each do |comment| -%>
VIEW CODE
<%- end -%>
Mongoid will find the post in the collection. When you hit the comments iterator it does a single query to get the comments. Mongoid wraps the query in a cursor so it is a true iterator and doesn't overload the memory.
Mongoid lazy loads all queries to allow this behavior by default. The :include tag is unnecessary.
This could help https://github.com/flyerhzm/mongoid-eager-loading
You need update your schema to avoid this N+1 there are no solution in MongoDB to do some jointure.
Embed the detail records/documents in the master record/document.
In my case I didn't have the whole collection but an object of it that caused n+1 (bullet says that).
So rather than writing below which causes n+1
quote.providers.officialname
I wrote
Quote.includes(:provider).find(quote._id).provider.officialname
That didn't cause a problem but left me thinking if I repeated myself or checking n+1 is unnecessary for mongoid.