EDIT - Using 'includes' generates a SQL 'IN' clause. When using Oracle this has a 1,000 item limit. It will not work for my company. Are there any other solutions out there?
Is it possible to eager load an association on an association?
For example, let's say I have an Academy class, and an academy has many students. Each student belongs_to student_level
class Academy < ActiveRecord::Base
has_many :students
end
class Student < ActiveRecord::Base
belongs_to :academy
belongs_to :student_level
end
class StudentLevel < ActiveRecord::Base
has_many :students
end
Is it possible to tailor the association in Academy so that when I load the students, I ALWAYS load the student_level with the student?
In other words, I would like the following section of code to produce one or two queries total, not one query for every student:
#academy.students.each do |student|
puts "#{student.name} - #{student.student_level.level_name}"
end
I know I can do this if I change students from an association to a method, but I don't want to do that as I won't be able to reference students as an association in my other queries. I also know that I can do this in SQL in the following manner, but I want to know if there's a way to do this without finder_sql on my association, because now I need to update my finder_sql anytime my default scope changes, and this won't preload the association:
SELECT students.*, student_levels.* FROM students
LEFT JOIN student_levels ON students.student_level_id = student_levels.id
WHERE students.academy_id = ACADEMY_ID_HERE
Have you tried using includes to eager load the data?
class Academy < ActiveRecord::Base
has_many :students
# you can probably come up with better method name
def students_with_levels
# not sure if includes works off associations, see alternate below if it does not
self.students.includes(:student_level)
end
def alternate
Student.where("academy_id = ?", self.id).includes(:student_level)
end
end
see also: http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations
should result in 3 queries
the initial find on Academy
the query for a collection of Student objects
the query for all of those Students StudentLevel objects
Additions:
# the same can be done outside of methods
#academy.students.includes(:student_level).each do |student|
puts "#{student.name} - #{student.student_level.level_name}"
end
Student.where("academy_id = ?", #academy.id).includes(:student_level).each do |student|
puts "#{student.name} - #{student.student_level.level_name}"
end
ActiveRelation queries are also chainable
#academy.students_with_levels.where("name ILIKE ?", "Smi%").each do # ...
Sort of related a nice article on encapsulation of ActiveRecord queries (methods) - http://ablogaboutcode.com/2012/03/22/respect-the-active-record/
Related
I'm having trouble combining two has_many relations. Here are my associations currently:
def Note
belongs_to :user
belongs_to :robot
end
def User
has_many :notes
belongs_to :group
end
def Robot
has_many :notes
belongs_to :group
end
def Group
has_many :users
has_many :robots
has_many :user_notes, class_name: 'Note', through: :users, source: :notes
has_many :robot_notes, class_name: 'Note', through: :robots, source: :notes
end
I'd like to be able to get all notes, both from the user and the robots, at the same time. The way I currently do that is:
def notes
Note.where(id: (user_notes.ids + robot_notes.ids))
end
This works, but I don't know a clever way of getting all notes for a given collection of groups (without calling #collect for efficiency purposes).
I would like the following to return all user/robot notes for each group in the collection
Group.all.notes
Is there a way to do this in a single query without looping through each group?
Refer Active record Joins and Eager Loading documentation for detailed and efficient ways.
For example, You could avoid n+1 query problem here in this case as follows,
class Group
# Add a scope to eager load user & robot notes
scope :load_notes, -> { includes(:user_notes, :robot_notes) }
def notes
user_notes & robot_notes
end
end
# Load notes for group collections
Group.load_notes.all.notes
You can always handover the querying to the db which is built for such purposes. For example, your earlier query for returning all the notes associated with users and robots can be achieved by:
Notes.find_by_sql ["SELECT * FROM notes WHERE user_id IN (SELECT id FROM users) UNION SELECT * FROM notes WHERE robot_id IN (SELECT id FROM robots)"]
If you want to return the notes from users and robots associated with a given group with ID gid(say), you'll have to modify the nested sql query:
Notes.find_by_sql ["SELECT * FROM notes WHERE user_id IN (SELECT id FROM users WHERE group_id = ?) UNION SELECT * FROM notes WHERE robot_id IN (SELECT id FROM robots WHERE group_id = ?)", gid, gid]
Note:
If you want your application to scale then you may want as many DB transactions executed within a given period as possible, which means you run shorter multiple queries. But if you want to run as little queries as possible from ActiveRecord using the above mentioned method, then it will effect the performance of you DB due to larger queries.
I have a "Loan" model in Rails I'm trying to build. There's a corresponding "Payment" model. The balance on the loan is the original amount of the loan minus the sum of all the payments. Calculating the balance is easy, but I'm trying to calculate the balance on lots of loans while avoiding an N+1 query, while at the same time making the "balance" a property of the "Loan" model.
When I call the index method of the Loans controller, I can run a custom select query, which allows me to return a "balance" property through a straight SQL query.
class LoansController < ApplicationController
def index
#loans = Loan
.joins("LEFT JOIN payments on payments.loan_id = loan.id")
.group("loans.id")
.select("loans.*, loans.amount - SUM(payments.amount) as balance")
end
def index_002
#loans = Loan.includes(:payments)
end
def index_003
#loans = Loan.includes(:payments)
end
end
class Loan < ActiveRecord::Base
has_many :payments
def balance=(value)
# I'd like balance to load automatically in the Loan model.
raise NotImplementedError.new("Balance of a loan cannot be set directly.")
end
def balance_002
# No N+1 query, but iterating through each payment in Ruby
# is grossly inefficient as well
amount - payments.map(:amount).inject(0, :+)
end
def balance_003
# Even with the "includes" in the controller, this is N+1
amount - (payments.sum(:amount) || 0)
end
end
Now my question is how to do this all the time with my Loan model. Normally ActiveRecord loads one or more models using the following query:
SELECT * FROM loans
--where clause optional
WHERE id IN (?)
Is there any way to override the Loan model so that it loads the following query:
SELECT
loans.*, loans.amount - SUM(payments.amount) as balance
FROM
loans
LEFT JOIN
payments ON payments.loan_id = loans.id
GROUP BY
loans.id
This way "balance" is a property of the model and only has to be declared in one place, but we also avoid the inefficiencies of N+1 queries.
I like to use database views for this, so that rails thinks it's talking to a regular database table (ensuring things like eager loading work normally) when in fact there are aggregations or complex joins going on. In your case, I might define a second loan_balances view:
create view loan_balances as (
select loans.id as loan_id, loans.amount - sum(payments.amount) as balance
from loans
left outer join payments on payments.loan_id = loans.id
group by 1
)
Then just do regular rails association stuff:
class LoanBalance < ActiveRecord::Base
belongs_to :loan, inverse_of: :loan_balance
end
class Loan < ActiveRecord::Base
has_one :loan_balance, inverse_of: :loan
delegate :balance, to: :loan_balance, prefix: false
end
This way in actions where you want the balance you can eager load it with includes(:loan_balance), but you won't get into thorny issues from violating rails conventions in all the standard CRUD stuff surrounding Loan itself.
It looks like I finally answered my own question. Here it is. I overrode the default scope.
class Loan < ActiveRecord::Base
validates :funded_amount, presence: true, numericality: {greater_than: 0}
has_many :payments, dependent: :destroy, inverse_of: :loan
default_scope {
joins("LEFT JOIN payments as p ON p.loan_id = loans.id")
.group("loans.id").select("loans.*, sum(p.amount) as paid")
}
def balance
funded_amount - (paid || 0)
end
end
I have a many2many relationship with a has_many through association:
class User < ActiveRecord::Base
has_many :trips_users
has_many :trips, through: :trips_users
end
class Trip < ActiveRecord::Base
has_many :trips_users
has_many :users, through: :trips_users
end
class TripsUser < ActiveRecord::Base
belongs_to :user
belongs_to :trip
end
The joining table trips_user contains a column named 'pending' which id like get when I ask for a list of trips of a user.
So in my controller I need to get all trips a user has, but also adding the 'pending' column.
I was trying
current_user.trips.includes(:trips_users)
that will be done by this select statement:
SELECT trips.* FROM trips INNER JOIN trips_users ON trips.id
= trips_users.trip_id WHERE trips_users.user_id = 3
which is missing the information in the trips_users table that I want.
The desired sql would be:
SELECT trips.*, trips_users.* FROM trips INNER JOIN trips_usersON trips.id =
trips_users.trip_id WHERE trips_users.user_id = 3
This finally worked:
current_user.trips.select('trips_users.*, trips.*')
Overriding the select part of the SQL.
Not very pretty in my opinion thou, I shouldn't be messing with tables and queries but models, specially in such a common case of a m2m association with extra data in the middle.
You'll want to use joins rather than includes for this... See the following Rails Guide:
http://guides.rubyonrails.org/active_record_querying.html#joining-tables
Essentially you'd do something like this:
current_user.trips.joins(:trips_users)
The includes method is used for eager loading, while joins actually performs the table join.
You could also try:
trips_users = current_user.trips_users.includes(:trip)
trips_users.first.pending?
trips_users.first.trip
Which should give you the trips_users records for that user but also eager loading the trips so that accessing them wouldn't hit the database again.
I have the following models
class Forum < ActiveRecord::Base
has_many :topics
end
class Topic < ActiveRecord::Base
belongs_to :forum
has_many :posts
end
class Posts < ActiveRecord::Base
belongs_to :topic
belongs_to :user
end
I would like to join these associations (with some conditions on both topics and posts), so I did Topic.joins(:forum, :posts).where(some condition), but I would like to access the attributes in all three tables. How can I achieve this?
You can do a multiple join like that. All three tables are available, you just have to specify in the wheres.
Topic.joins(:forum, :posts).where('topics.foo' => 'bar').where('posts.whizz' => 'bang)
The two caveats are that that is a multiple inner join so the topic will need to have both forum and post to show up at all and that the where's are anded together. For outer joins or 'or' logic in the where's you will have to get fancier. But you can write sql for both .joins and .where to make that happen. Using .includes instead of .joins is another way to get outer join behavior--but of course you are then maybe loading a lot of records.
To clarify the comment below, and show the improved (more rails-y) syntax where syntax suggested by another user:
topics = Topic.joins(:forum, :posts).where(topic: {foo: 'bar'}).where(posts: {whizz: 'bang'})
topics.each do |t|
puts "Topic #{t.name} has these posts:"
t.posts.each do |p|
puts p.text
end
end
I think I need something akin to a rails eager loaded query with a limit on it but I am having trouble finding a solution for that.
For the sake of simplicity, let us say that there will never be more than 30 Persons in the system (so Person.all is a small dataset) but each person will have upwards of 2000 comments (so Person.include(:comments) would be a large data set).
Parent association
class Person < ActiveRecord::Base
has_many :comments
end
Child association
class Comment < ActiveRecord::Base
belongs_to :person
end
I need to query for a list of Persons and include their comments, but I only need 5 of them.
I would like to do something like this:
Limited parent association
class Person < ActiveRecord::Base
has_many :comments
has_many :sample_of_comments, \
:class_name => 'Comment', :limit => 5
end
Controller
class PersonController < ApplicationController
def index
#persons = Person.include(:sample_of_comments)
end
end
Unfortunately, this article states: "If you eager load an association with a specified :limit option, it will be ignored, returning all the associated objects"
Is there any good way around this? Or am I doomed to chose between eager loading 1000s of unneeded ActiveRecord objects and an N+1 query? Also note that this is a simplified example. In the real world, I will have other associations with Person, in the same index action with the same issue as comments. (photos, articles, etc).
Regardless of what "that article" said, the issue is in SQL you can't narrow down the second sql query (of eager loading) the way you want in this scenario, purely by using a standard LIMIT
You can, however, add a new column and perform a WHERE clause instead
Change your second association to Person has_many :sample_of_comments, conditions: { is_sample: true }
Add a is_sample column to comments table
Add a Comment#before_create hook that assigns is_sample = person.sample_of_comments.count < 5