Caching with instance variable is degrading performance - ruby-on-rails

All examples below would have called both pending_notifications? and reviewable_notifications
I have the following set of instance methods for User:
def pending_notifications?
return true if reviewable_notifications.size > 0
end
def reviewable_notifications
#reviewable_notifications ||= self.employee.notifications.where(read: [nil, false])
end
They are used by the view in the following manner:
<% if current_user.pending_notifications? %>
<li><%= link_to fa_icon("envelope") + " #{current_user.reviewable_notifications.count} Notification(s)", user_notifications_path(id: current_user.id) %></li>
<% else %>
<li><%= link_to fa_icon("inbox") + " Notification Center", user_notifications_path(id: current_user.id) %></li>
<% end %>
When I analyze the load, one query is being called:
SELECT COUNT(*) FROM "notifications" WHERE "notifications"."employee_id" = ? AND (("notifications"."read" = 'f' OR "notifications"."read" IS NULL))
This is fine, but before my refactor, I was not using the recommended technique of caching with an instance variable, but I got the exact same query in my analysis. Furthermore, it was consistently running at about 20ms less. Below is how the code was originally written. Why wasn't Rails calling the same query twice? Why is the performance better with the code written this way?
def pending_notifications?
return true if self.employee.notifications.where(read: [nil, false]).size > 0
end
def reviewable_notifications
self.employee.notifications.where(read: [nil, false])
end

Memoizing a value like this is only useful if there's an expensive calculation made in Ruby and you use that value in multiple places.
This particular calculation happens in SQL and Rails already caches DB queries by default, so that's why you didn't see any change.

The difference is not based on the query that is being generated...
It is that if you use
`#reviewable_notifications ||= self.employee.notifications.where(read: [nil, false])`
you will only hit the database as long as #reviewable_notifications is nil.
In the moment it takes a value, it will be used.
As an easy example you can write this in the console:
2.1.2 :001 > 5.times { User.first } # no caching, hit 5 times your DB
User Load (0.2ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
User Load (0.1ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
User Load (0.1ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
User Load (0.1ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
User Load (0.1ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
=> 5
2.1.2 :002 > 5.times { #user ||= User.first } # caching, only 1 hit
User Load (0.1ms) SELECT `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1
=> 5
Of course, mysql hast its own query cache, so if the same query hits the database, it might be possible that the result is given back from the database query cache (you can see in the example above that the last queries takes less time than the first one, which probably is because mysq serves the results from its cache)

Related

Handling a massive query in Rails

What's the best way to handle a large result set with Rails and Postgres? I didn't have a problem until today, but now I'm trying to return a 124,000 record object of #network_hosts, which has effectively DoS'd my development server.
My activerecord orm isn't the prettiest, but I'm pretty sure cleaning it up isn't going to help in relation to performance.
#network_hosts = []
#host_count = 0
#company.locations.each do |l|
if l.grace_enabled == nil || l.grace_enabled == false
l.network_hosts.each do |h|
#host_count += 1
#network_hosts.push(h)
#network_hosts.sort! { |x,y| x.ip_address <=> y.ip_address }
#network_hosts = #network_hosts.first(5)
end
end
end
In the end, I need to be able to return #network_hosts to the controller for processing into the view.
Is this something that Sidekiq would be able to help with, or is it going to be just as long? If Sidekiq is the path to take, how do I handle not having the #network_hosts object upon page load since the job is running asyncronously?
I believe you want to (1) get rid of all that looping (you've got a lot of queries going on) and (2) do your sorting with your AR query instead of in the array.
Perhaps something like:
NetworkHost.
where(location: Location.where.not(grace_enabed: true).where(company: #company)).
order(ip_address: :asc).
tap do |network_hosts|
#network_hosts = network_hosts.limit(5)
#host_count = network_hosts.count
end
Something like that ought to do it in a single DB query.
I had to make some assumptions about how your associations are set up and that you're looking for locations where grace_enabled isn't true (nil or false).
I haven't tested this, so it may well be buggy. But, I think the direction is correct.
Something to remember, Rails won't execute any SQL queries until the result of the query is actually needed. (I'll be using User instead of NetworkHost so I can show you the console output as I go)
#users = User.where(first_name: 'Random');nil # No query run
=> nil
#users # query is now run because the results are needed (they are being output to the IRB window)
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 LIMIT $2 [["first_name", "Random"], ["LIMIT", 11]]
# => #<ActiveRecord::Relation [...]>
#users = User.where(first_name: 'Random') # query will be run because the results are needed for the output into the IRB window
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 LIMIT $2 [["first_name", "Random"], ["LIMIT", 11]]
# => #<ActiveRecord::Relation [...]>
Why is this important? It allows you to store the query you want to run in the instance variable and not execute it until you get to a view where you can use some of the nice methods of ActiveRecord::Batches. In particular, if you have some view (or export function, etc.) where you are iterating the #network_hosts, you can use find_each.
# Controller
#users = User.where(first_name: 'Random') # No query run
# view
#users.find_each(batch_size: 1) do |user|
puts "User's ID is #{user.id}"
end
# User Load (0.5ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# User's ID is 1
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 AND ("users"."id" > 1) ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# User's ID is 2
# User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 AND ("users"."id" > 2) ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# => nil
Your query is not executed until the view, where it will now load only 1,000 records (configurable) into memory at a time. Once it reaches the end of those 1,000 records, it will automatically run another query to fetch the next 1,000 records. So your memory is much more sane, at the cost of extra database queries (which are usually pretty quick)

Self-referring association: N+1 issue

Let say I have following Model:
class Strategy < ActiveRecord::Base
belongs_to :user
has_many :snapshots, :class_name => 'Strategy', :foreign_key => 'master_id'
belongs_to :master, :class_name => 'Strategy', :counter_cache => :snapshots_count
scope :master_only, -> { where(:master_id => nil) }
end
So any user can create a snapshot of a master-instance of Strategy
In controller I am getting all "master" instances of Strategy that belongs to current_user
#strategies = current_user.strategies.master_only.includes(:user,:snapshots)
Rails correctly loads strategies and snapshots in two queries but fetches user for each snapshot in separate query thus introducing N+1 issue (N+2 in this particular case):
Strategy Load (0.3ms) SELECT `strategies`.* FROM `strategies` WHERE strategies`.`user_id` = 2 AND `strategies`.`master_id` IS NULL LIMIT 10 OFFSET 0
User Load (0.7ms) SELECT `users`.* FROM `users` WHERE `users`.`id` IN (2)
Strategy Load (0.8ms) SELECT `strategies`.* FROM `strategies` WHERE `strategies`.`master_id` IN (56, 8, 55, 1, 58, 57, 24, 22)
User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 2 LIMIT 1
....
User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 5 LIMIT 1
Is there a way to load the snapshot user more effectively?
You can make a slight change to the way you include :user and :snapshots to only generate one query to load all the users that belong to the :snapshots:
#strategies = current_user.strategies.master_only.includes(snapshots: :user)
The generated SQL queries will be something like:
SELECT `strategies`.* FROM `strategies` WHERE strategies`.`user_id` = 2 AND `strategies`.`master_id` IS NULL LIMIT 10 OFFSET 0
SELECT `users`.* FROM `users` WHERE `users`.`id` IN (2)
SELECT `strategies`.* FROM `strategies` WHERE `strategies`.`master_id` IN (56, 8, 55, 1, 58, 57, 24, 22)
SELECT `users`.* FROM `users` WHERE `users`.`id` IN (2,5)
UPDATE
According to the OP, the following resolved the N+1 issue:
#strategies = current_user.strategies.master_only.includes(:user, snapshots: :user)

Active Record "Where" Query Failing to find Record

So have a token model that has keys which are strings. I am doing a search for a token that exists and when i do a direct comparrison of the string with the record it should find i get an empty active record relation object.
Anyone know what is going on here? Am i stepping on a models toes by using the Model name 'Token'. I didn't find anything in the googles about it.
I've stored the key as 'a' below and Token.last.key is the database entry that has the matching key.
irb(main):023:0> a
=> "279684d7488254c41bb4039ad0962007"
irb(main):024:0> Token.last.key
Token Load (0.4ms) SELECT "tokens".* FROM "tokens" ORDER BY "tokens"."id" DESC LIMIT 1
=> "279684d7488254c41bb4039ad0962007"
irb(main):025:0> a == Token.last.key
Token Load (0.1ms) SELECT "tokens".* FROM "tokens" ORDER BY "tokens"."id" DESC LIMIT 1
=> true
irb(main):026:0> Token.where(key: a)
Token Load (0.2ms) SELECT "tokens".* FROM "tokens" WHERE "tokens"."key" = '279684d7488254c41bb4039ad0962007'
=> #<ActiveRecord::Relation []>
irb(main):027:0> Token.where(key: a.to_s)
Token Load (0.2ms) SELECT "tokens".* FROM "tokens" WHERE "tokens"."key" = '279684d7488254c41bb4039ad0962007'
=> #<ActiveRecord::Relation []>
Oddly enough, when i do a search through all the records i do return the correct user
#DOES WORK
def current_user(key)
Token.all.first {|t| return t.key == key }
end
What i have will work but what i would like it to do this in the database with something like this
#DOES NOT WORK
def current_user(key)
Token.where(key: key).try(:user)
end
Doing search based on suggestion below returning nil for find_by
irb(main):004:0> a = Token.last.key
Token Load (0.2ms) SELECT "tokens".* FROM "tokens" ORDER BY "tokens"."id" DESC LIMIT 1
=> "279684d7488254c41bb4039ad0962007"
irb(main):005:0> Token.find_by(key: a)
Token Load (0.2ms) SELECT "tokens".* FROM "tokens" WHERE "tokens"."key" = '279684d7488254c41bb4039ad0962007' LIMIT 1
=> nil
Solution was to do this
def find_user_by_api(key)
tokens = Token.arel_table
Token.where(tokens[:key].matches(key)).try(:first).try(:user)
end
Wish it helps
Token.find_by(key: a)

Spree/Rails user password reset

I'm trying to set user's (admin) password from Rails console:
bundle exec rails console
> Spree::User.first.email
=> "admin#mysite.com"
> Spree::User.first.encrypted_password
Spree::User Load (1.1ms) SELECT "spree_users".* FROM "spree_users" LIMIT 1
=> "4ec556............................................."
> Spree::User.first.password='spree123'
Spree::User Load (1.0ms) SELECT "spree_users".* FROM "spree_users" LIMIT 1
=> "spree123"
> Spree::User.first.password_confirmation='spree123'
Spree::User Load (1.0ms) SELECT "spree_users".* FROM "spree_users" LIMIT 1
=> "spree123"
> Spree::User.first.save!
Spree::User Load (1.0ms) SELECT "spree_users".* FROM "spree_users" LIMIT 1
(0.3ms) BEGIN
(1.3ms) SELECT COUNT(DISTINCT "spree_users"."id") FROM "spree_users" LEFT OUTER JOIN "spree_roles_users" ON "spree_roles_users"."user_id" = "spree_users"."id" LEFT OUTER JOIN "spree_roles" ON "spree_roles"."id" = "spree_roles_users"."role_id" WHERE "spree_roles"."name" = 'admin'
(0.3ms) COMMIT
=> true
> Spree::User.first.encrypted_password
Spree::User Load (1.0ms) SELECT "spree_users".* FROM "spree_users" LIMIT 1
=> "1bc15d.............................................."
So far so good. It looks like the new password for the user has been changed and commited to the database. However when I try to log in later with a web client and using the new password, it fails with invalid identity/password message.
I even tried to update password with Spree::User.first.reset_password!('spree123', 'spree123') but sill cann't sign in.
Rails 3.2.12
Spree 1.3.2
Any idea what am I doing wrong ? How to properly set a new password ?
Thanks.
The problem is that every time you're doing Spree::User.first it's reloading the record from the database. This means you are setting the value on one instance of the record, reloading it, and then saving the reloaded model that hasn't actually changed. An easy way around this is to create a local instance variable containing the record and update that instead:
user = Spree::User.first
user.password='spree123'
user.password_confirmation='spree123'
user.save!
Spree::User.first.update_attributes(password: 'password')

Proper way to find records for a certain foreign key

I have two Models called User and Membership.
User has_many :memberships
Membership belongs_to :user
What is the proper way to modify MembershipsController's index method to set #memberships to all the memberships there are for the user "session[:user_id]"?
I tried something like:
#memberships = Membership.find(:all, :conditions => ["user_id = ?", session[:user_id]] )
but then Rails is Selecting from users instead of memberships:
Rendering memberships/index
←[4;35;1mUser Columns (3.0ms)←[0m ←[0mSHOW FIELDS FROM `users`←[0m
←[4;36;1mUser Load (1.0ms)←[0m ←[0;1mSELECT * FROM `users` WHERE (`users`.`id` = 1) LIMIT
←[4;35;1mCACHE (0.0ms)←[0m ←[0mSELECT * FROM `users` WHERE (`users`.`id` = 1) LIMIT 1←[0m
user_id wasn't being set correctly.
script/console:
>> Membership.find(:first, :conditions => "user_id = 1")
=> nil
Logs:
Completed in 19ms (View: 9, DB: 6) | 200 OK [http://localhost/memberships]
[4;36;1mSQL (0.0ms)[0m [0;1mSET NAMES 'utf8'[0m
[4;35;1mSQL (0.0ms)[0m [0mSET SQL_AUTO_IS_NULL=0[0m
[4;36;1mUser Columns (2.0ms)[0m [0;1mSHOW FIELDS FROM `users`[0m
[4;35;1mSQL (0.0ms)[0m [0mSHOW TABLES[0m
[4;36;1mUser Load (0.0ms)[0m [0;1mSELECT * FROM `users` WHERE (`users`.`id` = 1) [0m
[4;35;1mMembership Load (1.0ms)[0m [0mSELECT * FROM `memberships` WHERE (user_id = 1) LIMIT 1[0m
[4;36;1mMembership Load (0.0ms)[0m [0;1mSELECT * FROM `memberships` WHERE (user_id = 1) LIMIT 1[0m
What does session[:user_id] represent? Are you trying to roll your own authentication system?
There are a number of authentication solutions that can handle the ins and outs of that for you.
To answer your question though, you probably want to be making use of the associations you've set up between the two models:
def index
#user = User.find(session[:user_id])
#memberships = #user.memberships if #user
end
Using an application-wide authentication system (whether of your own-making or using a library) this would most likely be simplified to:
def index
#memberships = current_user.memberships
end
where current_user is a method defined in ApplicationController that returns the currently logged in user and the index action has a before_filter that ensures that the user is logged in.

Resources