I get the benefits of using eager loading to avoid N+1 when fetching an array of objects with their associated records, but is it important to use eager loading when fetching a single record?
In my case
user has_many :addresses
user has_many :skills
user has_many :devices
user has_many :authentications
In the show action, I am trying to see with rack mini profiler if it is interesting to use eager loading
User.includes(:skills, :addresses, :devices, :authentications).find(params[:id])
But I seem to have the same number of sql requests..
Any advice on using eager loading for such case or not?
is it important to use eager loading when fetching a single record?
For associations one level deep, no.
If you have nested associations, yes.
# class User
has_many :skills
# class Skill
belongs_to :category
# this code will fire N + 1 queries for skill->category
user.skills.each do |skill|
puts skill.category
end
In this case, it is better to eager load skills: :category
User.includes(skills: :category).find(id)
Edit
Rails provide two ways to avoid N+1 queries, which it refers to as preloading and eager_loading.
Preload fires individual SQL queries for each collection.
Eager load attempts to construct one massive left-joined SELECT to retrieve all collections in 1 query.
The short version is that includes lets Rails pick which one to use. But you can force one way or the other.
User.eager_load(:skills, :addresses, :devices, :authentications).find(params[:id])
Should retrieve all records in 1 query.
Further reading:
What's the difference between “includes” and “preload” in an ActiveRecord query?
http://blog.bigbinary.com/2013/07/01/preload-vs-eager-load-vs-joins-vs-includes.html
http://blog.arkency.com/2013/12/rails4-preloading/
Try using the Bullet gem for detecting unused or missing eager loading. It's designed to tell you if there are wasted include statements, or inefficient N+1 queries, where includes would help.
If there's a problem, it can be configured to output to the Rails logger to let you know. Or you can have it show you a notification in the browser on pages that need optimising.
Related
I have a self-referential association like this:
class User
belongs_to :parent_user, class_name: "User"
has_many :child_users, class_name: "User", foreign_key: :parent_user_id
end
When I get a list of users, I would like to be able to eager-load the child_users association, but I only need the id column from it. I need to be able to do something like this without causing n+1 queries, and preferably without having to load all of the other data in this model:
users = User.preload(:child_users)
users.map(&:child_user_ids)
The above works to limit the query to two queries instead of n+1, but I'm loading a lot of full objects into memory that I would prefer to avoid when I only care about the id itself on those child assocations.
You don't want eager loading which is for loading models. You want to select an aggregate. This isn't something the ActiveRecord query interface handles so you'll need to use SQL strings or Arel and use the correct function for your DB.
For example:
# This is for Postgres, use JSON_ARRAYAGG on MySQL
User.left_joins(:child_users)
.select(
"users.*",
'json_agg("child_users_users".id) AS child_ids'
)
.group(:id)
child_users_users is the wonky alias that .left_joins(:child_users) creates.
At least on Postgres the driver (the PG gem) will automatically cast the result to a an array but YMMV on the other dbs.
Here are my models:
class Team < ApplicationRecord
has_many :team_permissions
end
class TeamPermission < ApplicationRecord
belongs_to :team
belongs_to :permissible, polymorphic: true
end
class User < ApplicationRecord
has_many :team_permissions, as: :permissible
end
I understand you can solve your N+1 problem with includes like so:
Team.includes(team_permissions: :permissible)
Now, I want to only join the permissions under a condition. For example, if they do not belong to a group of ids, so I would expect this to work, but it throws an error.
ActiveRecord:
Team.includes(team_permissions: :permissible).where.not(team_permissions: { id: team_permission_ids })
Error:
ActionView::Template::Error (Cannot eagerly load the polymorphic association :permissible):
Playing around with it further, I found the following worked the way I want it to, but it does not solve the N+1 issue.
Team.includes(:team_permissions).where.not(team_permissions: { id: team_permission_ids })
How could I include eager loading for the .includes with a condition?
Unfortunately Active Record isn't smart enough (nor, to be honest, trusting enough) to work out that it needs to join the first table to apply your condition, but not the second.
You should be able to help it out by being slightly more explicit:
Team.
includes(:team_permissions). # or eager_load(:team_permissions).
preload(team_permissions: :permissible).
where.not(team_permissions: { id: team_permission_ids }
When there are no conditions referencing includes tables, the default behaviour is to use preload, which handles the N+1 by doing a single additional query, and is compatible with polymorphic associations. When such a condition is found, however, all the includes are converted to eager_load, which does a LEFT JOIN in the main query (and is consequently incompatible: can't write a query that joins to tables we don't even know about yet).
Above, I've separated the part we definitely want loaded via preload, so it should do the right thing.
class Category
has_many :images
has_many :articles
end
class Image
belongs_to :category
end
class Article
belongs_to :category
end
I'm trying to understand what solutions there are in Rails for children of different models to be queried by the same parent?
E.g. I'd like to get all images and articles that belong to the same category and sort them all by created_at.
You can try 'includes' in rails
Article.includes(:Category)
As I said it seems to me you can use eager loading multiple associations. In your case it could be something like this:
Category.where(id: 2).includes(:images, :articles).sort_by(&:created_at)
Basically you pass your desired Category ID and get :images, :articles which belongs_to Category with particular ID. sort_byprobably should do the sorting thing.
This blog post on eager loading could help you as well.
You can't simply force Active Record to bring all their dependences in a single query (afaik), regardless if is lazy/eager loading. I think your best bet is:
class Category
has_many :images, -> { order(:created_at) }
has_many :articles, -> { order(:created_at) }
end
categories = Category.includes(:images, :articles)
As long as you iterate categories and get their images and articles, this will make three queries, one for each table categories, images and articles, which is a good tradeoff for the ease of use of an ORM.
Now, if you insist to bring all that info in just one query, for sure it must be a way using Arel, but think twice if it worths. The last choice I see is the good old SQL with:
query = <<-SQL
SELECT *, images.*, articles.*
FROM categories
-- and so on with joins, orders, etc...
SQL
result = ActiveRecord::Base.connection.execute(query)
I really discourage this option as it will bring A LOT of duplicated info as you will joining three tables and it really would be a pain to sort them for your use.
I have the following model (simplified):
class Job < ActiveRecord::Base
belongs_to :customer
has_one :priority
end
I am trying to load all jobs at once into a table that displays both the customer and priority, so I am eager loading both associations to avoid N+1 queries:
Job.includes(:customer, :priority)
So far so good. The problem comes when I want to order by customer name. Following Rails 4 instructions, I do this:
Job.includes(:customer, :priority).order("customers.name ASC")
ActiveRecord stops eager loading priorities! Rendering the table results in a separate query for each priority.
As a side note, AR is doing the join automatically without ".references(:customers)". This seems contrary to the documentation.
In a Rails 2.3.5 application I've got something like the following models:
class Foo < ActiveRecord::Base
has_many :bars
end
class Bar < ActiveRecord::Base
belongs_to :foo
end
And when I'm calling
Foo.all(:include => :bars)
I see the following queries in console:
SELECT * FROM "foos"
SELECT "bars".* FROM "bars" WHERE ("bars".foo_id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21))
with all the foo's ids in the where clause.
I guess this is not an optimal query while the number of ids may be large, and I need to preload all the 'bars'. Also, actually I've got not two models, but a chain of them.
Is there a way to make the eager loading query be like
SELECT "bars".* FROM "bars"
when I'm using find all?
That's actually an optimization, in fact Rails changes the querying strategy if the number of id's goes high.
You could also use :join instead of using :include