In a Rails 2.3.5 application I've got something like the following models:
class Foo < ActiveRecord::Base
has_many :bars
end
class Bar < ActiveRecord::Base
belongs_to :foo
end
And when I'm calling
Foo.all(:include => :bars)
I see the following queries in console:
SELECT * FROM "foos"
SELECT "bars".* FROM "bars" WHERE ("bars".foo_id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21))
with all the foo's ids in the where clause.
I guess this is not an optimal query while the number of ids may be large, and I need to preload all the 'bars'. Also, actually I've got not two models, but a chain of them.
Is there a way to make the eager loading query be like
SELECT "bars".* FROM "bars"
when I'm using find all?
That's actually an optimization, in fact Rails changes the querying strategy if the number of id's goes high.
You could also use :join instead of using :include
Related
Here are my models:
class Team < ApplicationRecord
has_many :team_permissions
end
class TeamPermission < ApplicationRecord
belongs_to :team
belongs_to :permissible, polymorphic: true
end
class User < ApplicationRecord
has_many :team_permissions, as: :permissible
end
I understand you can solve your N+1 problem with includes like so:
Team.includes(team_permissions: :permissible)
Now, I want to only join the permissions under a condition. For example, if they do not belong to a group of ids, so I would expect this to work, but it throws an error.
ActiveRecord:
Team.includes(team_permissions: :permissible).where.not(team_permissions: { id: team_permission_ids })
Error:
ActionView::Template::Error (Cannot eagerly load the polymorphic association :permissible):
Playing around with it further, I found the following worked the way I want it to, but it does not solve the N+1 issue.
Team.includes(:team_permissions).where.not(team_permissions: { id: team_permission_ids })
How could I include eager loading for the .includes with a condition?
Unfortunately Active Record isn't smart enough (nor, to be honest, trusting enough) to work out that it needs to join the first table to apply your condition, but not the second.
You should be able to help it out by being slightly more explicit:
Team.
includes(:team_permissions). # or eager_load(:team_permissions).
preload(team_permissions: :permissible).
where.not(team_permissions: { id: team_permission_ids }
When there are no conditions referencing includes tables, the default behaviour is to use preload, which handles the N+1 by doing a single additional query, and is compatible with polymorphic associations. When such a condition is found, however, all the includes are converted to eager_load, which does a LEFT JOIN in the main query (and is consequently incompatible: can't write a query that joins to tables we don't even know about yet).
Above, I've separated the part we definitely want loaded via preload, so it should do the right thing.
I've been trying to figure out some odd behavior when combining a has_one association and includes.
class Post < ApplicationRecord
has_many :comments
has_one :latest_comment, -> { order('comments.id DESC').limit(1) }, class_name: 'Comment'
end
class Comment < ApplicationRecord
belongs_to :post
end
To test this I created two posts with two comments each. Here are some rails console commands that show the odd behavior. When we use includes then it ignores the order of the latest_comment association.
posts = Post.includes(:latest_comment).references(:latest_comment)
posts.map {|p| p.latest_comment.id}
=> [1, 3]
posts.map {|p| p.comments.last.id}
=> [2, 4]
I would expect these commands to have the same output. posts.map {|p| p.latest_comment.id} should return [2, 4]. I can't use the second command because of n+1 query problems.
If you call the latest comment individually (similar to comments.last above) then things work as expected.
[Post.first.latest_comment.id, Post.last.latest_comment.id]
=> [2, 4]
If you have another way of achieving this behavior I'd welcome the input. This one is baffling me.
I think the cleanest way to make this work with PostgreSQL is to use a database view to back your has_one :latest_comment association. A database view is, more or less, a named query that acts like a read-only table.
There are three broad choices here:
Use lots of queries: one to get the posts and then one for each post to get its latest comment.
Denormalize the latest comment into the post or its own table.
Use a window function to peel off the latest comments from the comments table.
(1) is what we're trying to avoid. (2) tends to lead to a cascade of over-complications and bugs. (3) is nice because it lets the database do what it does well (manage and query data) but ActiveRecord has a limited understanding of SQL so a little extra machinery is needed to make it behave.
We can use the row_number window function to find the latest comment per-post:
select *
from (
select comments.*,
row_number() over (partition by post_id order by created_at desc) as rn
from comments
) dt
where dt.rn = 1
Play with the inner query in psql and you should see what row_number() is doing.
If we wrap that query in a latest_comments view and stick a LatestComment model in front of it, you can has_one :latest_comment and things will work. Of course, it isn't quite that easy:
ActiveRecord doesn't understand views in migrations so you can try to use something like scenic or switch from schema.rb to structure.sql.
Create the view:
class CreateLatestComments < ActiveRecord::Migration[5.2]
def up
connection.execute(%q(
create view latest_comments (id, post_id, created_at, ...) as
select id, post_id, created_at, ...
from (
select id, post_id, created_at, ...,
row_number() over (partition by post_id order by created_at desc) as rn
from comments
) dt
where dt. rn = 1
))
end
def down
connection.execute('drop view latest_comments')
end
end
That will look more like a normal Rails migration if you're using scenic. I don't know the structure of your comments table, hence all the ...s in there; you can use select * if you prefer and don't mind the stray rn column in your LatestComment. You might want to review your indexes on comments to make this query more efficient but you'd be doing that sooner or later anyway.
Create the model and don't forget to manually set the primary key or includes and references won't preload anything (but preload will):
class LatestComment < ApplicationRecord
self.primary_key = :id
belongs_to :post
end
Simplify your existing has_one to just:
has_one :latest_comment
Maybe add a quick test to your test suite to make sure that Comment and LatestComment have the same columns. The view won't automatically update itself as the comments table changes but a simple test will serve as a reminder.
When someone complains about "logic in the database", tell them to take their dogma elsewhere as you have work to do.
Just so it doesn't get lost in the comments, your main problem is that you're abusing the scope argument in the has_one association. When you say something like this:
Post.includes(:latest_comment).references(:latest_comment)
the scope argument to has_one ends up in the join condition of the LEFT JOIN that includes and references add to the query. ORDER BY doesn't make sense in a join condition so ActiveRecord doesn't include it and your association falls apart. You can't make the scope instance-dependent (i.e. ->(post) { some_query_with_post_in_a_where... }) to get a WHERE clause into the join condition, then ActiveRecord will give you an ArgumentError because ActiveRecord doesn't know how to use an instance-dependent scope with includes and references.
I am using this query to get my data
user = User.includes(:skills).order(user: :id)
it is working fine. but when i try to display skills by alphabetical order like below
user.skills.order(name: :asc)
It shows in logs that it goes in the database as order() is an activerecord method. It seems like eager loading is failing here because what's the point to use eager loading if it has to go in the database anyway.
Can anyone guide me what is a good way to do this.
When you eager load associated records using .includes, you should access the association as it is. Otherwise, if you add more query conditions to the association, that will cause a new DB query.
There are a few ways how you can order the associated eager loaded records.
1. Add order condition to the main scope.
user = User.includes(:skills).order("users.id, skills.name ASC")
In this case, it won't work like include method works by default, making two queries. One query will be performed using 'LEFT OUTER JOIN' to fetch the associated records. This is equivalent to using the eager_load method instead of includes
user = User.eager_load(:skills).order("users.id, skills.name ASC")
2. Add order condition to association when you define it.
In this case whenever you access the association, the associated records will always be ordered by name.
class User < ActiveRecord::Base
has_many :skills, -> { order(:name) }
end
3. Create another association with required order for using only in this particular case.
This allows you to avoid unnecessary conditions on the main association which is skills.
class User < ActiveRecord::Base
has_many :skills_ordered_by_name, -> { order(:name) }, class_name: "Skill"
end
# usage
users = User.includes(:skills_ordered_by_name)
users.each do |user|
# access eager loaded association
user.skills_ordered_by_name
end
4. Set default order for the association model.
This will cause the condition to be applied to every association and query related to the associated model.
class Skill < ActiveRecord::Base
default_scope { order(:name) }
end
5. Sort eager loaded records using Ruby code (not ActiveRecord query methods)
This approach is appropriate when there are not many records to sort.
users = User.includes(:skills)
users.each do |user|
# sorting with Ruby's 'sort_by' method
user.skills.sort_by(&:name)
# or something like
user.skills.sort { |one, another| one.name <=> another.name }
end
You can achieve flexibility by using built-in method ActiveRecord::Associations::Preloader#preload.
It accept three arguments:
preload(records, associations, preload_scope = nil)
The first argument accepts ActiveRecord::Base record or array of records.
Second is one or more associations that you want preload to records specified in the first argument.
Last is Hash or Relation which merged with associations.
Use third argument to get sorted and preloaded associations:
users = User.order(user: :id)
ActiveRecord::Associations::Preloader.new.preload(
users,
:skills,
{ order: :name } # or Skill.order(:name)
)
You can use this:
user = User.includes(:skills).order(user: :id, name: :desc)
I have the following model (simplified):
class Job < ActiveRecord::Base
belongs_to :customer
has_one :priority
end
I am trying to load all jobs at once into a table that displays both the customer and priority, so I am eager loading both associations to avoid N+1 queries:
Job.includes(:customer, :priority)
So far so good. The problem comes when I want to order by customer name. Following Rails 4 instructions, I do this:
Job.includes(:customer, :priority).order("customers.name ASC")
ActiveRecord stops eager loading priorities! Rendering the table results in a separate query for each priority.
As a side note, AR is doing the join automatically without ".references(:customers)". This seems contrary to the documentation.
Suppose I have the following models:
class Foo < ActiveRecord::Base
belongs_to :bar
end
class Bar < ActiveRecord::Base
belongs_to :a
belongs_to :b
end
I want to find all the Foos, including Bar and grouping by a_id and b_id.
I'm pretty sure the following query will work:
Foo.joins(:bar).group('bar.a_id, bar.b_id').all
I'm wondering if there's a way of doing it without writing the SQL in the group statement?
Sub question
What is this query style called and where can I read the full documentation of it? The rails query guide has a mix of several styles and doesn't go into great detail about any of them.
The mix of styles is because styles using hashes cannot currently describe all possible SQL queries. Thus, there is always the fallback of using strings.
The query you gave works, and there is no reason why it shouldn't be used, since the string is very standard SQL, and shouldn't fail if a different database is used.
It is however possible to write it without strings.
Do note that your current query as written will throw an error since by default all the fields of Foo are selected - however with group, you can only select aggregate functions or the group by columns.
The query would be something like:
Foo.select("COUNT(*) as count").joins(:bar).group([:bar => :a_id, :bar => :b_id])
I just added the select clause so that only an aggregate function is selected.