Rails 6.1 eager load ActiveStorage::VariantRecord - ruby-on-rails

I have been eager loading ActiveStorage attachments as follows:
Journey.includes(created_by_user: [profile_picture_attachment: :blob])
We have been using variants and every since we upgraded to Rails 6.1 and enabled tracking Active Storage variants in database, we notice n+1 queries in the logs because of a loop as follows:
Journey.includes(created_by_user: [profile_picture_attachment: :blob]).each do |j|
j.created_by_user.profile_picture.variant(resize_to_fill: [32, 32]).processed
end
Journey Load (0.8ms) SELECT "journeys".* FROM "journeys"
User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 607]]
ActiveStorage::Attachment Load (0.6ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "profile_picture"], ["record_id", 607]]
ActiveStorage::Blob Load (0.6ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 144]]
ActiveStorage::VariantRecord Load (0.5ms) SELECT "active_storage_variant_records".* FROM "active_storage_variant_records" WHERE "active_storage_variant_records"."blob_id" = $1 AND "active_storage_variant_records"."variation_digest" = $2 LIMIT $3 [["blob_id", 144], ["variation_digest", "k9S9jJS87DbFgXD1sW9j5XkOr1c="], ["LIMIT", 1]]
ActiveStorage::VariantRecord Load (0.5ms) SELECT "active_storage_variant_records".* FROM "active_storage_variant_records" WHERE "active_storage_variant_records"."blob_id" = $1 AND "active_storage_variant_records"."variation_digest" = $2 LIMIT $3 [["blob_id", 144], ["variation_digest", "k9S9jJS87DbFgXD1sW9j5XkOr1c="], ["LIMIT", 1]]
ActiveStorage::VariantRecord Load (0.5ms) SELECT "active_storage_variant_records".* FROM "active_storage_variant_records" WHERE "active_storage_variant_records"."blob_id" = $1 AND "active_storage_variant_records"."variation_digest" = $2 LIMIT $3 [["blob_id", 144], ["variation_digest", "k9S9jJS87DbFgXD1sW9j5XkOr1c="], ["LIMIT", 1]]
ActiveStorage::VariantRecord Load (0.5ms) SELECT "active_storage_variant_records".* FROM "active_storage_variant_records" WHERE "active_storage_variant_records"."blob_id" = $1 AND "active_storage_variant_records"."variation_digest" = $2 LIMIT $3 [["blob_id", 144], ["variation_digest", "k9S9jJS87DbFgXD1sW9j5XkOr1c="], ["LIMIT", 1]]
ActiveStorage::VariantRecord Load (0.5ms) SELECT "active_storage_variant_records".* FROM "active_storage_variant_records" WHERE "active_storage_variant_records"."blob_id" = $1 AND "active_storage_variant_records"."variation_digest" = $2 LIMIT $3 [["blob_id", 144], ["variation_digest", "k9S9jJS87DbFgXD1sW9j5XkOr1c="], ["LIMIT", 1]]
I tried to eager load with the following but it doesn't seem to work:
Journey.includes(created_by_user: [profile_picture_attachment: { blob: :variant_records }])
Has anyone tried eager loading the tracked variant records to share your ideas?


Turns out there is a pull request addressing exactly this. For anyone looking for a solution, please follow the merge request.
https://github.com/rails/rails/pull/37901
It eventually winds its way to a secondary PR, which allows eager loading the stored variants. It's now merged into future versions of Rails, here was the code they used to eager load variants:
https://github.com/rails/rails/pull/40842/files

Related

Rails ActiveStorage - has_on_attached/has_many_attached associations is load by default

I got a problem with ActiveStorage, Currently I have has_on_attachedand has_many_attached relation on my Model
Every time I call my model it loads the relation as below:
ActiveStorage::Attachment Load (4.2ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_id" = $1 AND "active_storage_attachments"."record_type" = $2 AND "active_storage_attachments"."name" = $3 LIMIT $4 [["record_id", 4934], ["record_type", "User"], ["name", "profile_picture"], ["LIMIT", 1]]
ActiveStorage::Attachment Exists (0.9ms) SELECT 1 AS one FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_id" = $1 AND "active_storage_attachments"."record_type" = $2 AND "active_storage_attachments"."name" = $3 LIMIT $4 [["record_id", 4934], ["record_type", "User"], ["name", "home_pictures"], ["LIMIT", 1]]
How can I disable this behaviour by default ?

You can use Model.with_attached_images.find(:id) to avoid the N+1
https://github.com/rails/rails/tree/master/activestorage#examples
I'm not sure if you can disable eager loading the attachments, but the above should help clean up a little bit.

Stop rails loading associated models when checkin if record is valid

I have an object called message which belongs to carrier, company, country
I allowing for a bulk insert of users via a CSV - what i want to do before is make sure every row is valid before i import it (so that i can inform the user prior to the beginning the import)
So i have created a method that loops through all new data and does Message.new(PARAMS_IN_HERE) and then call .valid? on it, which is fine and achieves the desired results.
However, when i look in the logs i see loads of queries like this
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.3ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Company Load (0.2ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Country Load (0.2ms) SELECT "countries".* FROM "countries" WHERE "countries"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Carrier Load (0.2ms) SELECT "carriers".* FROM "carriers" WHERE "carriers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Which is obviously quite wasteful as one its doing the same query multiple times. Is there a way to make rails cache the value/preload when it needs to look for/ or just stop it happening?
This is what my message class look like
class Message < ApplicationRecord
belongs_to :company
belongs_to :carrier
belongs_to :country
before_validation :set_default_details, on: :create
private
def set_default_details
if self.user.present?
self.carrier_id = self.user.company.tariff.carrier_id
self.country_id = self.user.country_id
self.company_id = self.user.company_id
end
end
end

Your validation references self.user but that's not defined in the model snippet you posted.
If your validations refer to to association objects, like self.company, give those objects to the constructor: Message.new(company: company) rather than Message.new(company_id: company.id). If necessary look up all dependent objects ahead of time - this can be done with a single query - and store id -> object mapping in a Hash.
Similarly you can have validations reference fields like company_id but it's probably better to use association objects everywhere.

Prevent multiple DB requests for the Rails Admin custom field

I have the custom boolean field in rails admin with nested resource usage
class Sector
has_many :holds, dependent: :destroy, inverse_of: :sector
def holden
holds.find_by_end_date(nil).present?
end
end
When the system renders Sectors list I see the multiple DB requests to get the holden field value for the each Sector.
Hold Load (2.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 119], ["LIMIT", 1]]
Hold Load (2.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 118], ["LIMIT", 1]]
Hold Load (4.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 117], ["LIMIT", 1]]
Hold Load (3.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 116], ["LIMIT", 1]]
Hold Load (2.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 115], ["LIMIT", 1]]
Hold Load (3.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 114], ["LIMIT", 1]]
Hold Load (3.0ms) SELECT "holds".* FROM "holds" WHERE "holds"."sector_id" = $1 AND "holds"."end_date" IS NULL LIMIT $2 [["sector_id", 102], ["LIMIT", 1]]
...
How to increase performance for this feature?

Double everything in my db logs Rails

I have double everything in my logs. Not sure how to get rid of it. Any suggestions?
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 7]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 7]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 7]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 7]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 8]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 8]]
Size Load (0.3ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 8]]
Size Load (0.3ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 8]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 9]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 9]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 9]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 9]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 10]]
Bid Load (0.2ms) SELECT "bids".* FROM "bids" WHERE "bids"."order_id" = $1 [["order_id", 10]]
CACHE (0.0ms) SELECT "printers".* FROM "printers" WHERE "printers"."id" = $1 LIMIT 1 [["id", 1]]
CACHE (0.0ms) SELECT "printers".* FROM "printers" WHERE "printers"."id" = $1 LIMIT 1 [["id", 1]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 10]]
Size Load (0.2ms) SELECT "sizes".* FROM "sizes" WHERE "sizes"."order_id" = $1 LIMIT 1 [["order_id", 10]]
Rails Version Rails 4.2.5
Ruby Version 2.3.0

you can add the following to application.rb
if Rails.env.development?
# Don't log to STDOUT, by default rails s will handle it
config.logger = Logger.new('/dev/null')
else
# Don't log to file, sending everything to unicorn file.
config.logger = Logger.new(STDOUT)
end
Basically rails server and rails logger both send their messages to stdout which results in duplication in the logs.

Rails 4 ActiveRecord multiple queries

I use rails 4.2.5.
I have some sort of N+1 problem in this code
seats=SeatItem.where(:b => hall.id).all
seats.each do |seat|
arr << Ticket.new(:a => seat.id)
end
Ticket.import arr
Problem of this code is that i have this in log
MIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE "seat_item_types"."id" = $1 LIMIT 1 [["id", 13]]
CACHE (0.0ms) SELECT "seat_item_types".* FROM "seat_item_types" WHERE
Why :all method does not load all in memory and how fix it?
Thank you for your input!

I don't think :all is the problem. Notice that the repeated SQL query is identical and that it's fetching rows from seat_item_types not seat_items. My guess is that you have a before/after hook or some other code that is being triggered by Ticket.new or Ticket.import that is accessing seat_item_type.

You should use eager_load for solve N + 1 query problem
use includes method like this
clients = Client.includes(:address).limit(10)
clients.each do |client|
puts client.address.postcode
end
http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations
But first of all, I think you define associations on SeatItem and Ticket
http://guides.rubyonrails.org/association_basics.html

Resources