Does Rails actually cache the query result? The documentation says that same query will be never executed twice on the same request:
1.7 SQL Caching
The second time the same query is run against the database, it's not actually going to hit the database. The first time the result is returned from the query it is stored in the query cache (in memory) and the second time it's pulled from memory.
I did an experiment to proof that Rails actually cache the query:
def test
data = ""
User.find(1).update(first_name: 'Suwir Suwirr')
data << User.find(1).first_name
data << "\n"
User.find(1).update(first_name: 'Pengguna')
data << User.find(1).first_name
data << "\n"
render plain: data
end
If the result is cached, i would get the same result for each User.find(1). However, the result was Rails does not actually cache the query; i was expecting the update does not reflected on the result since it was "cached":
Suwir Suwirr
Pengguna
But the console says that it was cached: (Please highlight the CACHE word)
Started GET "/diag/test" for 10.0.2.2 at 2017-02-21 10:30:16 +0700
Processing by DiagController#test as HTML
User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 4], ["LIMIT", 1]]
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
(0.1ms) BEGIN
SQL (0.4ms) UPDATE "users" SET "first_name" = $1, "updated_at" = $2 WHERE "users"."id" = $3 [["first_name", "Suwir Suwirr"], ["updated_at", 2017-02-21 03:30:16 UTC], ["id", 1]]
(16.5ms) COMMIT
User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
(0.1ms) BEGIN
SQL (0.3ms) UPDATE "users" SET "first_name" = $1, "updated_at" = $2 WHERE "users"."id" = $3 [["first_name", "Pengguna"], ["updated_at", 2017-02-21 03:30:16 UTC], ["id", 1]]
(0.9ms) COMMIT
User Load (0.5ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendering text template
Rendered text template (0.0ms)
Completed 200 OK in 380ms (Views: 3.5ms | ActiveRecord: 21.9ms)
So my question, does Rails actually cache the query result? Or, only several query result on some request?
Update: Using Batch #update_all
I made another experiment to "fool" the query logic. Now Rails does not "cache" the query. Why this behaviour can happen?
# Controller
def test
data = ""
User.where(id: 1).update_all(first_name: 'Suwir Suwirr')
data << User.find(1).first_name
data << "\n"
User.where(id: 1).update_all(first_name: 'Pengguna')
data << User.find(1).first_name
data << "\n"
logger.info 'hi'
render plain: data
end
# Console
Started GET "/diag/test" for 10.0.2.2 at 2017-02-21 10:45:43 +0700
Processing by DiagController#test as HTML
User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 4], ["LIMIT", 1]]
SQL (13.8ms) UPDATE "users" SET "first_name" = 'Suwir Suwirr' WHERE "users"."id" = $1 [["id", 1]]
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
SQL (2.9ms) UPDATE "users" SET "first_name" = 'Pengguna' WHERE "users"."id" = $1 [["id", 1]]
User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
hi
Rendering text template
Rendered text template (0.0ms)
Completed 200 OK in 28ms (Views: 0.8ms | ActiveRecord: 17.8ms)
# Browser result
Suwir Suwirr
Pengguna
I was stupid.
Yes, Rails does actually cache the query, but update and destroy will invalidate its query cache. update_all is basically iterating each record with update.
I tried the experiment by really "fooling" the ActiveRecord query mechanism. And yes, it works.
# Controller
def test
data = ""
ActiveRecord::Base.connection.execute('UPDATE "users" SET "first_name" = \'Suwir Suwirr\' WHERE "users"."id" = 1')
data << User.find(1).first_name
data << "\n"
ActiveRecord::Base.connection.execute('UPDATE "users" SET "first_name" = \'Pengguna\' WHERE "users"."id" = 1')
data << User.find(1).first_name
data << "\n"
render plain: data
end
# Browser
Suwir Suwirr
Suwir Suwirr
Related
In my app I have overwritten current_user devise method a bit. The idea is that if certain cookie is present method check the organization by the id inside that cookie and returns owner of this organization instead of regular user:
def current_user
user = warden.authenticate(scope: :user)
return nil if user.nil?
if user.admin? && cookies.key?('mock_admin_login')
organization = Organization.includes(:creator).find(cookies.encrypted[:mock_admin_login])
return organization.creator
end
user
end
Everything works correct but when I take a look at my console I noticed that Organization query is performed multiple times:
CACHE Organization Load (0.5ms) SELECT "organizations".* FROM
"organizations" WHERE "organizations"."id" = $1 LIMIT $2 [["id", 9],
["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (0.9ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (0.7ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (0.3ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (0.4ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.5ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (2.0ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (4.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (0.4ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (42.8ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (0.9ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user' CACHE Organization Load (4.5ms) SELECT
"organizations".* FROM "organizations" WHERE "organizations"."id" = $1
LIMIT $2 [["id", 9], ["LIMIT", 1]] ↳
app/controllers/concerns/current_methods_overwritten.rb:11:in
current_user' CACHE User Load (1.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 [["id", 10]] ↳ app/controllers/concerns/current_methods_overwritten.rb:11:in current_user'
Although It might seem like a not big deal but server spends additional 30-40ms to perform this action every time when current_user method is called. Why this query is called so many times instead of one and how can I fix it?
You need to memoize the result so that its not reevaluated every time you call current_user.
If you look at the helper that devise generates you can see that it does just that:
def current_#{mapping}
#current_#{mapping} ||= warden.authenticate(scope: :#{mapping})
end
If you want to fix your existing method you want to make sure to memoize the DB calls:
def current_user
#current_user ||= warden.authenticate(scope: :#{mapping})
if #current_user&.admin? && cookies.key?('mock_admin_login')
#current_org || = Organization.includes(:creator)
.find(cookies.encrypted[:mock_admin_login])
#current_user = #current_org.creator
end
#current_user
end
But you really should implement this as a custom Warden strategy instead.
In my Rails app I have the N+1 problem where I'm making extra call(s) to the database to get associated data over and over again, specially when logging impressions per model.
For example:
Started GET "/" for 127.0.0.1 at 2018-12-02 16:21:05 -0500
Processing by JobsController#index as HTML
Job Load (4.4ms) SELECT "jobs".* FROM "jobs" WHERE "jobs"."published_at" IS NOT NULL ORDER BY "jobs"."published_at" DESC LIMIT $1 OFFSET $2 [["LIMIT", 30], ["OFFSET", 0]]
↳ app/controllers/jobs_controller.rb:22
User Load (1.1ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 13], ["LIMIT", 1]]
↳ app/controllers/jobs_controller.rb:24
Impression Exists (1.3ms) SELECT 1 AS one FROM "impressions" WHERE "impressions"."impressionable_id" = $1 AND "impressions"."impressionable_type" = $2 AND "impressions"."session_hash" = $3 LIMIT $4 [["impressionable_id", 705], ["impressionable_type", "Job"], ["session_hash", "d80d52dd401011a626d600167140e49f"], ["LIMIT", 1]]
↳ app/controllers/jobs_controller.rb:24
Impression Exists (0.6ms) SELECT 1 AS one FROM "impressions" WHERE "impressions"."impressionable_id" = $1 AND "impressions"."impressionable_type" = $2 AND "impressions"."session_hash" = $3 LIMIT $4 [["impressionable_id", 704], ["impressionable_type", "Job"], ["session_hash", "d80d52dd401011a626d600167140e49f"], ["LIMIT", 1]]
↳ app/controllers/jobs_controller.rb:24
Impression Exists (0.4ms) SELECT 1 AS one FROM "impressions" WHERE "impressions"."impressionable_id" = $1 AND "impressions"."impressionable_type" = $2 AND "impressions"."session_hash" = $3 LIMIT $4 [["impressionable_id", 703], ["impressionable_type", "Job"], ["session_hash", "d80d52dd401011a626d600167140e49f"], ["LIMIT", 1]]
↳ app/controllers/jobs_controller.rb:24
I'm using the impressionist gem and using the impressionist method directly, taking all of the Jobs on the page and logging an impression. The problem is because I'm only recording unique impressions, for records that already have an impression, these additional calls are redundant. I tried to use #jobs.includes(:impressions).each to preload the associated data hoping Rails was smart enough to figure out which records existed, but Rails still outputs numerous Impression Exists queries.
#jobs.each{|job| impressionist(job,'', :unique => [:session_hash])}
I'm allowing a user to enter IP addresses in an input field which may be of different types delimited by a comma, such as (in no particular order):
192.168.1.1,192.168.2.1-25,10.10.10.0/24,192.168.1.2
This 'string' would get saved in my DB under device.ips_to_scan.
I want to validates_format_of on these, but am finding it a little difficult to write a regex that seems to work in rails, while it does work on regex101 (https://regex101.com/r/nf2bnM/1):
validates_format_of :ips_scan, with: /\A([0-9]{1,3}\.){3}[0-9]{1,3}(\/([1-2][0-9]|[0-9]|3[0-2]))?(-([0-9]{1,3}))?,?\Z/i, on: :update
This one is expected to fail:
Started PUT "/devices/2" for 127.0.0.1 at 2018-02-19 22:03:15 -0500
Processing by DevicesController#update as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"EQCFG6/xoJHtP6Nd3oqaYRW6mypfEoCMrnio1yj6loP+KtvjgLZ9Gmhb0oTwCjD0RGH+qQuctZFVIvF5HBJcGw==", "device"=>{"ips_scan"=>"192.168.1.1,192.168.2.1-25,a.b.c.d", "ips_exclude"=>"10.10.10.1"}, "commit"=>"Save", "id"=>"2"}
User Load (0.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 2], ["LIMIT", 1]]
Device Load (1.6ms) SELECT "devices".* FROM "devices" WHERE "devices"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
(0.5ms) BEGIN
User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
(0.5ms) ROLLBACK
Redirected to http://localhost:3000/devices/2/edit
Completed 302 Found in 47ms (ActiveRecord: 12.1ms)
...But this one should have worked:
Processing by DevicesController#update as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"JJfmT/0l5MEDc+gUH/WHHp3bbgyzjGa0xTzaXM3E/WHLvbi30mI5SoYXmc0xdS2LzAALj+cCU6k+ZoPy+Sw3+Q==", "device"=>{"ips_scan"=>"192.168.1.1,192.168.2.1-25,192.168.1.2", "ips_exclude"=>"10.10.10.1"}, "commit"=>"Save", "id"=>"2"}
User Load (0.9ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 2], ["LIMIT", 1]]
Device Load (0.7ms) SELECT "devices".* FROM "devices" WHERE "devices"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
(0.6ms) BEGIN
User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
(0.6ms) ROLLBACK
Redirected to http://localhost:3000/devices/2/edit
Completed 302 Found in 17ms (ActiveRecord: 3.5ms)
Last thing I can think of, is that I do have strong parameters, but I'm permitting ips_scan, so that this shouldn't be an issue:
def update
if #device.update(device_params)
flash[:notice] = 'Successful update'
respond_with :edit, :device
else
flash[:warning] = 'Unable to update'
respond_with :edit, :device
end
end
private def device_params
params.require(:device).permit(:token, :ips_scan, :ips_exclude)
end
I'm hoping you rubyist's out there have a eloquent solution. The first thought that comes to mind is that I have to split the string, and check each element sequentially to ensure it matches instead.
While I'm still open to a nice eloquent one-liner within the Model itself, I was able to get this working through creating a concern:
models/concerns/ip_validator.rb
class IpValidator < ActiveModel::Validator
def validate(record)
ips = record.ips_scan.split(',')
ips.each do |ip|
/([0-9]{1,3}\.){3}[0-9]{1,3}(\/([1-2][0-9]|[0-9]|3[0-2]))?(-([0-9]{1,3}))?/ =~ ip
record.errors.add(:ips_scan, ' is not valid') unless $LAST_MATCH_INFO
end
end
end
The call in my model now looks like:
validates :ips_scan, :ips_exclude, ip: true, on: :update
You can use this method in your custom validator to check an IP address
require 'ipaddr'
def valid_ip_addr?(ip_addr)
IPAddr.new(ip_addr)
true
rescue IPAddr::InvalidAddressError => _error
false
end
I'm using the public_activity gem and in the output, I'm checking if the trackable owner is the same as the current user:
= a.owner == current_user ? 'You' : a.owner.name
did this activity
I get a bunch of cache calls in the log:
User Load (1.8ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/post/_create.html.haml (1.4ms)
Rendered public_activity/_snippet.html.haml (11.4ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/post/_create.html.haml (13.9ms)
Rendered public_activity/_snippet.html.haml (18.9ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/comment/_comment.html.haml (0.9ms)
Rendered public_activity/_snippet.html.haml (12.1ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/comment/_comment.html.haml (2.7ms)
Rendered public_activity/_snippet.html.haml (56.3ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/comment/_comment.html.haml (0.6ms)
Rendered public_activity/_snippet.html.haml (4.5ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Rendered public_activity/content/_comment.html.haml (2.1ms)
Rendered public_activity/_snippet.html.haml (9.5ms)
Is there any way to eager load the conditional?
#jverban is correct that you can compare the record IDs to avoid needless record loading. To answer your question about eager loading though, yes you can eager load using the includes method in the ActiveRecord query chain. For example:
Activity.includes(:owner).latest
That will tell Rails you intend to reference the owner relation and so they should be loaded as well.
I highly recommend adding the bullet gem to your project (only in development and test environments) to detect N+1 queries and warn you when you've got an N+1 query situation like this happening.
You shouldn't need to load the user record, just compare id attributes
= a.owner_id == current_user.id ? 'You' : a.owner.name
The cache calls will likely still happen if multiple activity owners are not the current user (to get the owner name).
Here is my problem:
I'm using Devise's guest_user, that contains a logging_in method to transfer guest_user parameters to the registered user when he logs in. So in my case, the user has_many periods, dependent: :destroy, so here is the logging_in method:
def logging_in
guest_periods = guest_user.periods.all
guest_periods.each do |p|
p.user_id = current_user.id
p.save!
end
current_user.latest_entry = guest_user.latest_entry
current_user.is_in_zone = guest_user.is_in_zone
current_user.save
end
However, when a guest_user logs in, his periods gets destroyed instead of being transfered. Here is the log:
Started GET "/" for ::1 at 2015-05-11 00:18:03 +0300
Processing by WelcomeController#index as HTML
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT 1 [["id", 24]]
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 LIMIT 1 [["id", 23]]
Period Load (0.3ms) SELECT "periods".* FROM "periods" WHERE "periods"."user_id" = $1 [["user_id", 23]]
(0.2ms) BEGIN
CACHE (0.0ms) SELECT "periods".* FROM "periods" WHERE "periods"."user_id" = $1 [["user_id", 23]]
SQL (0.8ms) UPDATE "periods" SET "user_id" = $1, "updated_at" = $2 WHERE "periods"."id" = $3 [["user_id", 24], ["updated_at", "2015-05-10 21:18:03.863162"], ["id", 170]]
(0.9ms) COMMIT
(0.2ms) BEGIN
SQL (2.1ms) UPDATE "users" SET "is_in_zone" = $1, "latest_entry" = $2, "updated_at" = $3 WHERE "users"."id" = $4 [["is_in_zone", "t"], ["latest_entry", "2015-05-04"], ["updated_at", "2015-05-10 21:18:03.875572"], ["id", 24]]
(15.8ms) COMMIT
(0.5ms) BEGIN
SQL (0.3ms) DELETE FROM "periods" WHERE "periods"."id" = $1 [["id", 170]]
SQL (0.7ms) DELETE FROM "users" WHERE "users"."id" = $1 [["id", 23]]
(1.2ms) COMMIT
So we can see that the transfer is done, but then in the end, the periods are destroyed anyway. They should not be, as they are not belonging to the user to be destroyed any more.
Why is it happening?
Even though Period#user_id has changed, guest_user.periods is still loaded in memory and is what gets destroyed when you destroy the guest user. If you guest_user.reload, its associations will clear out and it becomes safe to destroy. You could also guest_user.periods(true) to force reload of just the periods.
Another option is:
guest_user.periods.update_all(user_id: current_user.id)
This executes a single query to perform the update, which will be nice if there are a lot of periods, and also doesn't load the guest_user.periods association, so it will load fresh during the destroy and find the correct empty set.