My rails 4 application has many Measures which belong to a Station.
I tried to eager load the measures in my controller:
#station = Station.includes(:measures).friendly.find(params[:id])
#measures = #station.measures
However when I added a method to the measure model which access a property of the station it causes an additional query per measure:
SELECT "stations".* FROM "stations" WHERE "stations"."id" = ? ORDER BY "stations"."id" ASC LIMIT 1
This means about 50 queries per page load which totally tanks performance.
How do I properly eager load the relationship to avoid this n+1 query? Am I going about it wrong?
github: /app/controllers/stations_controller.rb
class StationsController < ApplicationController
...
# GET /stations/1
# GET /stations/1.json
def show
#station = Station.includes(:measures).friendly.find(params[:id])
#measures = #station.measures
end
...
end
github: /app/models/measure.rb
class Measure < ActiveRecord::Base
belongs_to :station, dependent: :destroy, inverse_of: :measures
after_save :calibrate!
after_initialize :calibrate_on_load
...
def calibrate!
# this causes the n+1 query
unless self.calibrated
unless self.station.speed_calibration.nil?
self.speed = (self.speed * self.station.speed_calibration).round(1)
self.min_wind_speed = (self.min_wind_speed * self.station.speed_calibration).round(1)
self.max_wind_speed = (self.max_wind_speed * self.station.speed_calibration).round(1)
self.calibrated = true
end
end
end
def calibrate_on_load
unless self.new_record?
self.calibrate!
end
end
def measure_cannot_be_calibrated
if self.calibrated
errors.add(:speed_calbration, "Calibrated measures cannot be saved!")
end
end
end
github: /app/models/stations.rb
class Station < ActiveRecord::Base
# relations
belongs_to :user, inverse_of: :stations
has_many :measures, inverse_of: :station, counter_cache: true
# slugging
extend FriendlyId
friendly_id :name, :use => [:slugged, :history]
...
end
ADDITION
It interesting to note that this does not cause a n+1 query. But I would rather not duplicate it across my controllers.
class Measure < ActiveRecord::Base
...
# after_initialize :calibrate_on_load
...
end
#station = Station.includes(:measures).friendly.find(params[:id])
#measures = #station.measures
#measures.each do |m|
m.calibrate!
end
Try to include station to measure as well.
def show
#station = Station.includes(:measures).friendly.find(params[:id])
#measures = #station.measures.includes(:station)
end
The :includes option will usually trigger loads in separate queries as you are seeing, because in many cases it is more performant. If you want to ensure it's done in one query, try using :joins instead, but beware that the resulting records returned will be read-only.
after_initialize occurs before relations are eager-loaded by joins or include. See https://github.com/rails/rails/issues/13156
I decided to after some good advice use a dfferent approach and came up with this:
class Station < ActiveRecord::Base
...
alias_method :measures_orig, :measures
def measures
measures_orig.map do |m|
m.calibrate!
end
end
end
Related
In my app controller looks pretty simple:
class ProductsController
before_action :set_company
def index
#products = company.products.includes(:shipment)
end
private
def set_company
#company = Company.find(params[:company_id])
end
end
But what I worry about is the #product inside Index action was properly declared? What if I'll have millions of products? is there more efficient solution?
Model relations below:
class Products < ApplicationRecord
belongs_to :company
has_many :shipment
end
class Company < ApplicationRecord
has_many :products
end
class Shipment < ApplicationRecord
belongs_to :products
end
There is definitely a problem if you have million of records because your query is going to be too big, in this case you can add pagination in any flavor or use any other strategy that reduce the number of records queried each time.
To do pagination in Rails you can use https://github.com/kaminari/kaminari but this is not the only strategy available to do this.
Models:
class Audio < ActiveRecord::Base
has_many :tests, as: :item
end
class Video < ActiveRecord::Base
has_many :tests, as: :item
end
class Test < ActiveRecord::Base
belongs_to :user
belongs_to :item, polymorphic: true
end
class User < ActiveRecord::Base
has_many :tests
def score_for(item)
return 0 unless tests.where(item: item).any?
tests.where(item: item).last.score
end
end
Serializers:
class VideoSerializer < ActiveModel::Serializer
attributes :id, :name
attribute(:score) { user.score_for(object) }
def user
instance_options[:user]
end
end
I try serialise lot of Video objects like this, but N+1 coming:
options = { each_serializer: VideoSerializer, user: User.last }
videos = ActiveModelSerializers::SerializableResource.new(Video.all, options).serializable_hash
If I try this, empty array returned(looks like videos not has tests for this user):
options = { each_serializer: VideoSerializer, user: User.last }
videos = ActiveModelSerializers::SerializableResource.new(Video.includes(:tests).where(tests: {user: User.last}), options).serializable_hash
How I can organise serialisation w/o N+1 queries problem.
You cannot avoid an N+1 query if you are using a method that triggers another SQL query (in this case where).
The method score_for does another query (or 2, which would definitely need refactoring) when you invoke the relation with where.
One way you could change this method would be not to use relation methods but array methods over already loaded relations. This is very inefficient for memory but much less heavy on DB.
def score_for(item)
tests.sort_by&:created_at).reverse.find { |test| test.user_id == id }&.score.to_f
end
You would need to load the video with its tests and the user.
I read this interesting article about Using Polymorphism to Make a Better Activity Feed in Rails.
We end up with something like
class Activity < ActiveRecord::Base
belongs_to :subject, polymorphic: true
end
Now, if two of those subjects are for example:
class Event < ActiveRecord::Base
has_many :guests
after_create :create_activities
has_one :activity, as: :subject, dependent: :destroy
end
class Image < ActiveRecord::Base
has_many :tags
after_create :create_activities
has_one :activity, as: :subject, dependent: :destroy
end
With create_activities defined as
def create_activities
Activity.create(subject: self)
end
And with guests and tags defined as:
class Guest < ActiveRecord::Base
belongs_to :event
end
class Tag < ActiveRecord::Base
belongs_to :image
end
If we query the last 20 activities logged, we can do:
Activity.order(created_at: :desc).limit(20)
We have a first N+1 query issue that we can solve with:
Activity.includes(:subject).order(created_at: :desc).limit(20)
But then, when we call guests or tags, we have another N+1 query problem.
What's the proper way to solve that in order to be able to use pagination ?
Edit 2: I'm now using rails 4.2 and eager loading polymorphism is now a feature :)
Edit: This seemed to work in the console, but for some reason, my suggestion of use with the partials below still generates N+1 Query Stack warnings with the bullet gem. I need to investigate...
Ok, I found the solution ([edit] or did I ?), but it assumes that you know all subjects types.
class Activity < ActiveRecord::Base
belongs_to :subject, polymorphic: true
belongs_to :event, -> { includes(:activities).where(activities: { subject_type: 'Event' }) }, foreign_key: :subject_id
belongs_to :image, -> { includes(:activities).where(activities: { subject_type: 'Image' }) }, foreign_key: :subject_id
end
And now you can do
Activity.includes(:part, event: :guests, image: :tags).order(created_at: :desc).limit(10)
But for eager loading to work, you must use for example
activity.event.guests.first
and not
activity.part.guests.first
So you can probably define a method to use instead of subject
def eager_loaded_subject
public_send(subject.class.to_s.underscore)
end
So now you can have a view with
render partial: :subject, collection: activity
A partial with
# _activity.html.erb
render :partial => 'activities/' + activity.subject_type.underscore, object: activity.eager_loaded_subject
And two (dummy) partials
# _event.html.erb
<p><%= event.guests.map(&:name).join(', ') %></p>
# _image.html.erb
<p><%= image.tags.first.map(&:name).join(', ') %></p>
This will hopefully be fixed in rails 5.0. There is already an issue and a pull request for it.
https://github.com/rails/rails/pull/17479
https://github.com/rails/rails/issues/8005
I have forked rails and applied the patch to 4.2-stable and it works for me. Feel free to use my fork, even though I cannot guarantee to sync with upstream on a regular basis.
https://github.com/ttosch/rails/tree/4-2-stable
You can use ActiveRecord::Associations::Preloader to preload guests and tags linked, respectively, to each of the event and image objects that are associated as a subject with the collection of activities.
class ActivitiesController < ApplicationController
def index
activities = current_user.activities.page(:page)
#activities = Activities::PreloadForIndex.new(activities).run
end
end
class Activities::PreloadForIndex
def initialize(activities)
#activities = activities
end
def run
preload_for event(activities), subject: :guests
preload_for image(activities), subject: :tags
activities
end
private
def preload_for(activities, associations)
ActiveRecord::Associations::Preloader.new.preload(activities, associations)
end
def event(activities)
activities.select &:event?
end
def image(activities)
activities.select &:image?
end
end
image_activities = Activity.where(:subject_type => 'Image').includes(:subject => :tags).order(created_at: :desc).limit(20)
event_activities = Activity.where(:subject_type => 'Event').includes(:subject => :guests).order(created_at: :desc).limit(20)
activities = (image_activities + event_activities).sort_by(&:created_at).reverse.first(20)
I would suggest adding the polymorphic association to your Event and Guest models.
polymorphic doc
class Event < ActiveRecord::Base
has_many :guests
has_many :subjects
after_create :create_activities
end
class Image < ActiveRecord::Base
has_many :tags
has_many :subjects
after_create :create_activities
end
and then try doing
Activity.includes(:subject => [:event, :guest]).order(created_at: :desc).limit(20)
Does this generate a valid SQL query or does it fail because events can't be JOINed with tags and images can't be JOINed with guests?
class Activity < ActiveRecord::Base
self.per_page = 10
def self.feed
includes(subject: [:guests, :tags]).order(created_at: :desc)
end
end
# in the controller
Activity.feed.paginate(page: params[:page])
This would use will_paginate.
Using Rails 3.2. Let's say I want 2 options:
Get all trip photos.
Get the first trip photo.
I have the following code:
# trip.rb
class Trip < ActiveRecord::Base
has_many :trip_days
def trip_photos
if (photos = trip_days.map(&:spots).flatten.map(&:photos).flatten.map)
photos.each do |photo|
photo.url(:picture_preview)
end
end
end
def trip_photo
trip_photos.first
end
end
# trip_day.rb
class TripDay < ActiveRecord::Base
belongs_to :trip
has_many :trip_day_spots
has_many :spots, :through => :trip_day_spots
end
# trip_day_spot.rb
class TripDaySpot < ActiveRecord::Base
belongs_to :trip_day
belongs_to :spot
end
#spot.rb
class Spot < ActiveRecord::Base
end
# trips_controller.rb
class TripsController < ApplicationController
def index
#trips = Trip.public.paginate(:page => params[:page], :per_page => 25)
end
end
As expected, the trip_photos method generates lots of SQL query. I wonder if there is any better way to do it?
It is because of N+1 queries. In this cases, we need to eager load all the associations of base object, so that when ever you call its associated object, it wont fire any queries for fetching them, simply it will get them from its cached object.
Hope this will work, but not tested. I assumed and wrote the following query.
def trip_photos
user_trip_days = trip_days.includes(:spots => :photos)
photos = user_trip_days.collect {|trip_day| trip_day.spots.map(&:photos).flatten}.flatten
photos.each do |photo|
photo.url(:picture_preview)
end if photos
end
Let me know if you get any errors.
For more info on eager loading associated objects in ActiveRecord, go through
Guides for Rails and Rails cast and Rails Tips
This might not be the most rails-y way, but if you truly wanted to get all the spots in one hit you could do something like:
def spots
Spot.joins("join trip_days_spots on spots.id = trip_days_spots.spot_id join trip_days on trip_days.id = trip_days_spots.trip_day_id join trips on trips.id = trip_days.trip_id").where("trips.id = ?", self.id)
end
then change your loop to:
def trip_photos
spots.map(&:photos).flatten.each do |photo|
photo.url(:picture_preview)
end
end
The code works fine, but to eager load, just add :include:
# trips_controller.rb
class TripsController < ApplicationController
def index
#trips = Trip.public.paginate(:include => [:trip_days => [:spots => :photos]], :page => params[:page], :per_page => 25)
end
end
I am trying to calculate the average (mean) rating for all entries within a category based on the following model associations ...
class Entry < ActiveRecord::Base
acts_as_rateable
belongs_to :category
...
end
class Category < ActiveRecord::Base
has_many :entry
...
end
class Rating < ActiveRecord::Base
belongs_to :rateable, :polymorphic => true
...
end
The rating model is handled by the acts as rateable plugin, so the rateable model looks like this ...
module Rateable #:nodoc:
...
module ClassMethods
def acts_as_rateable
has_many :ratings, :as => :rateable, :dependent => :destroy
...
end
end
...
end
How can I perform the average calculation? Can this be accomplished through the rails model associations or do I have to resort to a SQL query?
The average method is probably what you're looking for. Here's how to use it in your situation:
#category.entries.average('ratings.rating', :joins => :ratings)
Could you use a named_scope or custom method on the model. Either way it would still require some SQL since, if I understand the question, your are calculating a value.
In a traditional database application this would be a view on the data tables.
So in this context you might do something like... (note not tested or sure it is 100% complete)
class Category
has_many :entry do
def avg_rating()
#entries = find :all
#entres.each do |en|
#value += en.rating
end
return #value / entries.count
end
end
Edit - Check out EmFi's revised answer.
I make no promises but try this
class Category
def average_rating
Rating.average :rating,
:conditions => [ "type = ? AND entries.category_id = ?", "Entry", id ],
:join => "JOIN entries ON rateable_id = entries.id"
end
end