I know how and what eager loading is, and I've searched around Google and StackOverflow for other eager loading questions. Some of them were enlightening, but none solved my particular question so I'm going to ask one myself.
Here is my schema:
class Organization < ActiveRecord::Base
has_many :activities
has_many :users_logged, source: :user, through: :activities
end
class Activity < ActiveRecord::Base
belongs_to :user
belongs_to :organization
end
class User < ActiveRecord::Base
has_many :activities
end
This project was inherited and I can't do any major schema redesigns so if the question is not possible with the schema then so be it. I just want to try and speed up generation of these reports which involves loading all the Organizations, something like:
Organizations.includes(:users_logged).joins(:activities)
Now, I realize here that the includes(:users_logged) will load all the activities but the joins(:activities) is still necessary for the fields that are ordered/grouped by which I've excluded as they are not important to the question.
The question now is, I would like to eager load all the Activities for all users that have been selected by the includes(:users_logged) directive as I will not only need the Organization details, it's associated activities, and all users that have logged activities for the organization. In addition to that I need to load all the activities for the user (ideally associated with the organization but I can work that out once I figure out how to eager load).
My current implementation:
#orgs = Organization.includes(:users_logged).joins(:activities).all
#orgs.each do |org|
org.users_logged.each do |user|
# Just work with user.activities
end
end
This results in a query per user that averages 0.3 ms with the current amount of test data which adds up rather quickly. Is there a way to eager load all activities for a collection of users?
I'm posting this an answer to make sure that it's clear it's been resolved. Once I'm capable I will accept this as the correct answer.
Baldrick's comment got me to thinking about how exactly I was filtering out activities for the user so that I only had the activities logged to the active organization. I was performing:
user.activities.where("organization_id = ?", org.id)
Which would force a query every time regardless of what I've done. So I changed it to:
user.activities.select { |o| o.id == org.id }
Which now performs what I've wanted (after #Baldrick's recommendation as well).
Related
Here are my models:
class Team < ApplicationRecord
has_many :team_permissions
end
class TeamPermission < ApplicationRecord
belongs_to :team
belongs_to :permissible, polymorphic: true
end
class User < ApplicationRecord
has_many :team_permissions, as: :permissible
end
I understand you can solve your N+1 problem with includes like so:
Team.includes(team_permissions: :permissible)
Now, I want to only join the permissions under a condition. For example, if they do not belong to a group of ids, so I would expect this to work, but it throws an error.
ActiveRecord:
Team.includes(team_permissions: :permissible).where.not(team_permissions: { id: team_permission_ids })
Error:
ActionView::Template::Error (Cannot eagerly load the polymorphic association :permissible):
Playing around with it further, I found the following worked the way I want it to, but it does not solve the N+1 issue.
Team.includes(:team_permissions).where.not(team_permissions: { id: team_permission_ids })
How could I include eager loading for the .includes with a condition?
Unfortunately Active Record isn't smart enough (nor, to be honest, trusting enough) to work out that it needs to join the first table to apply your condition, but not the second.
You should be able to help it out by being slightly more explicit:
Team.
includes(:team_permissions). # or eager_load(:team_permissions).
preload(team_permissions: :permissible).
where.not(team_permissions: { id: team_permission_ids }
When there are no conditions referencing includes tables, the default behaviour is to use preload, which handles the N+1 by doing a single additional query, and is compatible with polymorphic associations. When such a condition is found, however, all the includes are converted to eager_load, which does a LEFT JOIN in the main query (and is consequently incompatible: can't write a query that joins to tables we don't even know about yet).
Above, I've separated the part we definitely want loaded via preload, so it should do the right thing.
I've got kind of a strange case, in which I have a DayReport that has many Reports, which belong to an Account. I set-up my DayReport module using a has_many through for the accounts:
class DayReport < ActiveRecord::Base
has_many :reports
has_many :accounts, -> { order(:last_name) }, through: :reports
end
What I would like to do now, is get all the accounts and display all the reports for that account, but only if they are related to the DayReport. I can't use account.reports, because that also contains other reports.
One approach I took is to create an instance method that uses a where clause to fetch the appropriate reports:
def reports_for_account account
reports.where(account: account)
end
Problem is that this will trigger a query for each of the accounts, which I think is unnecessary. I'm only having trouble figuring out the correct approach.
I hope i understand it correctly.
You can decrease the numbers of quires by using eager loadet association with: .includes, because includes will load (1. query) all records of the parent and (2. query) all of the records referenced in the includes method (where).
In .where you can look if the day_report id exists or is a certain id (i'm not quite sure for what you're asking here)
For example (with id):
Account.includes(:reports).where(reports: {day_report_id: specified_id})
Or if they have a relationship at all:
Account.includes(:reports).where.not(reports: {day_report_id: nil})
That would decrease the number of queries to two.
Hope it helps!
Maybe joins helps you:
Account.includes(:reports).joins(:day_reports)
It fetches all accounts which have records in DayReport and preloaded reports.
I need to grab all users that have an application. User is part of my core engine, which is used by many other engines. I'd like to keep User unaware of what is using it, which is why I don't want to add has_many :training_applications in my User model.
Here are the classes
module Account
class User < ActiveRecord::Base
end
end
module Training
class TrainingApplication < ActiveRecord::Base
belongs_to :user, class: Account::User
end
end
The following obviously won't work because User has no concept of TrainingApplication:
Account::User.joins(:training_application).distinct
Is there an elegant way to return a distinct collection of User objects that are associated with a TrainingApplication?
What I landed on as a quick solution is
Account::User.where(id: Training::TrainingApplication.all.pluck(:user_id))
but I'm thinking that there's a better solution.
In case there is no way you can add a has_many :training_applications association to the User, the following should be suitable solutions:
You could type up a joins string yourself:
t1 = Account::User.table_name
t2 = Training::TrainingApplication.table_name
Account::User.
joins("INNER JOINS #{t2} ON #{t2}.user_id = #{t1}.id").
group("#{t1}.id")
For the sake of variety, let me cover the subquery method as well:
Account::User.where("id IN (SELECT user_id FROM #{t2})")
I would go with the joins method but I believe both solutions will be faster than your current implementation.
I'm seeking brainstorming input for a Rails design issue I've run across.
I have simple Book reviews feature. There's a Book class, a User class, and a UserBook class (a.k.a., reviews and ratings).
class User < ActiveRecord::Base
has_many :user_books
end
# (book_id, user_id, review data...)
class UserBook < ActiveRecord::Base
belongs_to :user
belongs_to :book
end
In the corresponding book controller for the "show" book action, I need to load the book data along with the set of book reviews. I also need to find out whether the current user (if there is one) has contributed to those reviews.
I'm currently running two queries, Book.where(...) and UserBook.where(...), and placing the results into two separate objects passed on to the view. Now, while I could run a third query to find whether the user is among those reviews (on UserBook), I'd prefer to pull that from the #reviews result set. But do I do that in the controller, or in the view?
Also worth noting is that in the view I have to draw Add vs Update review buttons accordingly, with their corresponding ajax URLs. So I'd prefer to know it before I start looping through a result set.
If I detect this in the controller though, I'll need three instance variables passed in, which I understand is considered distasteful in Rails land. Not sure how to avoid this.
Suggestions appreciated.
This smells like a case for has_many through, which is designed for cases where you want to access the data of a third table through an intermediate table (in this case, UserBook)
Great explanation of has_many :through here
Might look something like this:
class User < ActiveRecord::Base
has_many :user_books
has_many :users, through: :books
end
Then you can simply call
#user = User.find(x)
#user.user_books` # (perhaps aliased as `User.find(x).reviews`)
and
#user.books
to get a list of all books associated with the User.
This way, you can gain access to all of the information you need for a particular user with a single #user instance variable.
PS - You'll want to take a look at the concept of Eager loading, which will prevent you from making extraneous database calls while fetching all of this information.
I have a Rails app that keeps count of User, Comments for many Movie_Categories. The models look like this:
class User < ActiveRecord::Base
has_many :comments
end
class Movie < ActiveRecord::Base
belongs_to :movie_category
has_many :comments
end
class MovieCategory < ActiveRecord::Base
has_many :movies
end
class Comment < ActiveRecord::Base
belongs_to :movie
belongs_to :user
end
I can of course find User's Comments count by MovieCategory doing something like this for each MovieCategory:
#user.comment.where("movie_category_id =?", movie_category_id)
However, it puts a lot of load on the server, if I need to make this call too frequently, so I was thinking about doing the calculation once per hour (in a background job) for all Users for all Movie_Categories and then storing the counts in the User table, each Movie Category with it's own column. That way, I don't have to run the calculations for each user as frequently and can just access the counts from the User table.
The thing I'm wondering is if there is a more dry way to do this since I'm not sure when my Movie_Categories will stop growing (and with each time comes a new table field). I also thought about caching the User show views (where these counts appear), but even so, if I don't have these columns in the User table then it seems like each time a new User page is loaded (or cached expired) it will have to run through calculating all of this for the User comment counts again.
Is there a better approach for the issue that I'm facing with not putting too much burden on the server?
Given your comments about being in development at the moment, I would say don't worry about it until you have to worry about it! However, if you want to plan ahead my suggestions would be to go with fragment caching and indexes on the foreign keys.
If your site grows to the size you're talking about, running migrations to add additional fields to your users table could take significant time.
I note you've referred to caching in your question so assume you're broadly familiar with it but given a view such as:
<ul>
<li>Action Movies: 23 Comments</li>
<li>Comedy Movies: 14 Comments</li>
</ul>
You would wrap this in a cache block:
<% cache "user-#{user.id}-comments", #user.comments.last.created_at.to_i do %>
...
<% end %>
This will cache the fragment displaying the counts and auto expire it each time a new comment is posted by that user. You could really get into granular detail by caching each <li> and expiring only when a comment is posted in that category but it might be overkill at an early stage.
For the index on the foreign keys, you can add this in a migration using the syntax:
add_index :comments, :movie_category_id
I don't think the query you run is that bad but you never know until you hit production and scale quite what effect it will have.