I have a few counters getting stored in REDIS that get updated upon state changes in customer.rb. The things I need to store are:
1) count of customers associated with a user (user has_many customers)
2) count of customers that have a state of (using aasm_state) 'open' or 'claimed'
3) count of customers that have a state of (using aasm_state) 'open
Whenever a customer's state changes, I increment/decrement the redis counters accordingly. However, no matter what i've tried, the counts seems to always be off after a certain time period.
I'm using Sidekiq but I don't think it's a concurrency issue since REDIS shouldn't be subject to problems of concurrency, right?
Here's my count updater method:
def reset_stats
if aasm_state_was == 'open' && aasm_state == 'claimed' # open => assigned
# update company and user
user.redis_increment_my_customers_length
company.redis_decrement_open_customers_length
elsif user_id_changed? && aasm_state_was == 'claimed' && aasm_state == 'claimed' # assigned => assigned
# update users (assigner and assignee)
user_was = User.find(user_id_was)
user.redis_increment_my_customers_length
user_was.redis_decrement_my_customers_length
elsif aasm_state_was == 'claimed' && aasm_state == 'closed' # assigned => closed
# update company and user
user_was = User.find(user_id_was)
user_was.redis_decrement_my_customers_length
company.redis_decrement_all_customers_length
elsif aasm_state_was == 'closed' && aasm_state == 'claimed' # closed => assigned
# update company and user
user.redis_increment_my_customers_length
company.redis_increment_all_customers_length
elsif aasm_state_was == 'closed' && aasm_state == 'open' # closed => open
# update company
company.redis_increment_all_customers_length
company.redis_increment_open_customers_length
elsif aasm_state_was == 'open' && aasm_state == 'closed' # open => closed
# update company
company.redis_decrement_all_customers_length
company.redis_decrement_open_customers_length
end
and in user.rb:
def redis_length_key
"my_customers_length_for_#{id}"
end
def set_my_customers_length(l)
RED.set(redis_length_key, l)
l.to_i
end
def redis_increment_my_customers_length
RED.get(redis_length_key) ? RED.incr(redis_length_key) : my_customers_length
end
def redis_decrement_my_customers_length
RED.get(redis_length_key) ? RED.decr(redis_length_key) : my_customers_length
end
def my_customers_length
if l = RED.get(redis_length_key)
l.to_i
else
set_my_customers_length(my_customers.length)
end
end
and in company.rb:
def open_customers
customers.open
end
def redis_open_length_key
"open_customers_length_for_#{id}"
end
def set_open_customers_length(l)
RED.set(redis_open_length_key, l)
l.to_i
end
def redis_increment_open_customers_length
RED.get(redis_open_length_key) ? RED.incr(redis_open_length_key) : open_customers_length
end
def redis_decrement_open_customers_length
RED.get(redis_open_length_key) ? RED.decr(redis_open_length_key) : open_customers_length
end
def open_customers_length
if l = RED.get(redis_open_length_key)
return l.to_i
else
set_open_customers_length(open_customers.length)
end
end
def redis_all_length_key
"all_customers_length_for_#{id}"
end
def set_all_customers_length(l)
RED.set(redis_all_length_key, l)
l
end
def redis_increment_all_customers_length
RED.get(redis_all_length_key) ? RED.incr(redis_all_length_key) : all_customers_length
end
def redis_decrement_all_customers_length
RED.get(redis_all_length_key) ? RED.decr(redis_all_length_key) : all_customers_length
end
def all_customers_length
if l = RED.get(redis_all_length_key)
l.to_i
else
set_all_customers_length(open_or_claimed_customers.length)
end
end
def open_or_claimed_customers
customers.open_or_claimed
end
Is there a better pattern for what I'm trying to accomplish? This has been extremely frustrating because the counts always seem to become incorrect after a while. Please help!
You have a race condition between the time you call set_my_customers_length(my_customers_length + 1) and the time you call RED.set(redis_open_length_key, l).
Two processes start.
my_customers_length is 5 when that first call is made for both processes.
First process makes the second call and sets Redis to 6.
Second process makes the second call and sets Redis to 6 again.
Redis value should in fact be 7.
Consider using Redis' INCR and DECR functions to atomically update the values.
http://redis.io/commands/incr
http://redis.io/commands/decr
http://redis.io/commands/incrby
http://redis.io/commands/decrby
You have a race condition here:
RED.get(redis_all_length_key) ? RED.incr(redis_all_length_key) : all_customers_length
You cannot do any logic between reading from and writing to Redis.
Related
I am implementing a program in rails where there is a form and after submitting the form it will check if there is any record with duplicate value for a specific field in database. My database table is students. So my corresponding model name is Student. I am writing this code (what I have just discussed) in my controller.
But I am facing the following error. I am using some arrays for internal operations. When I wrote that particular function in ruby only(not rails) then it was working fine. Moreover I am also facing error due to the use of "length".
My error is:
NoMethodError in StudentsController#create
undefined method `[]' for nil:NilClass
My controller code is:
class StudentsController < ApplicationController
def new
#student=Student.new
#students=Student.all
end
def create
#student=Student.new(u_params)
ret_val=string_check
if ret_val==1
#student.save
redirect_to new_student_path , :notice => "Inserted!!!"
else
redirect_to new_student_path , :notice => "Match,Not inserted!!!"
end
end
def u_params
params.require(:student).permit(:id ,:firstname, :lastname)
end
def u_params_second
params.require(:student).permit(:firstname)
end
def string_check
count =0;
#temp1=:firstname
temp1=params[:firstname]
supplied_val=temp1
puts "Entered in string_check method"
for i in 46..100
temp2=Student.find_by(id:i)
table_val=temp2.firstname
size1=supplied_val.to_s.length
size2=table_val.to_s.length
arr1=Array.new
arr2=Array.new
# arr1[i] ||= []
# arr2[i] ||= []
for i in 0..size1
arr1.push(supplied_val[i])
end
for i in 0..size2
arr2.push(table_val[i])
end
for i in 0..size1
if arr1[i]=="#" || arr1[i]=="#" || arr1[i]=="{" || arr1[i]=="}" || arr1[i]=="(" || arr1[i]==")" || arr1[i]=="[" || arr1[i]=="]" || arr1[i]=="." || arr1[i]==";" || arr1[i]=="," || arr1[i]=="%" || arr1[i]=="&" || arr1[i]=="*" || arr1[i]=="!" || arr1[i]=="?" || arr1[i]=="$" || arr1[i]=="^" || arr1[i]==":" || arr1[i]=="-" || arr1[i]=="/"
count=count+1
# puts count
arr1[i]=""
end
end
# puts arr1
puts arr1.join
final1=arr1.join
for i in 0..size2
if arr2[i]=="#" || arr2[i]=="#" || arr2[i]=="{" || arr2[i]=="}" || arr2[i]=="(" || arr2[i]==")" || arr2[i]=="[" || arr2[i]=="]" || arr2[i]=="." || arr2[i]==";" || arr2[i]=="," || arr2[i]=="%" || arr2[i]=="&" || arr2[i]=="*" || arr2[i]=="!" || arr2[i]=="?" || arr2[i]=="$" || arr2[i]=="^" || arr2[i]==":" || arr2[i]=="-" || arr2[i]=="/"
count=count+1
# puts count
arr2[i]=""
end
end
# puts arr2
puts arr2.join
final2=arr2.join
if final1==final2
flag=0
else
flag=1
end
return flag
end
end
end
The routes.rb file is:
Rails.application.routes.draw do
resources :students
end
My error is: NoMethodError in StudentsController#create
undefined method `[]' for nil:NilClass
It simply means that you are trying to access something as an array that is actually a nil object, and not an array.
To get rid of this error, you can a technique called short-circuit in Ruby.
Let's say your following piece of code is producing the said error:
arr1[i]
You can use an if condition like this:
if arr1
arr1[i]
end
Or use short-circuit technique like this:
arr1 && arr1[i]
If you sure that the relevant code snippet was working for ruby and it's not for rails, the problem is most likely due to variable i used at inner and outer loops both. In any case, this needs to be fixed first or else it will result in unexpected behaviour only.
Outer Loops:
for i in 46..100
Inner Loops:
for i in 0..size1
for i in 0..size2
...
Keep i for outer loop and change the inner loop iterator to j
Hope it Helps : )
Adding to the answers of #harish and #arslan, there may be a case where, temp2=Student.find_by(id:i) may fail because there may not be a student with that id.
So, temp2 may return nil at that time.
for i in 0..size2
arr2.push(table_val[i]) // this may get failed
end
Then arr2.push will not work because table_val[i] is nil, so there are chances of undefined method [] for nil class.
This is a long question, so apologies in advance. We're using Redis to track customer states for each company. The customer states are tracked because we have customers separated by states in tabs on our website.
Whenever a customer instance is saved (after_save), we look to see what has changed and increment/decrement the relevant counts. For some reason, the counts are off pretty frequently and we can't figure out why. We wrote tests and verified all logic looks correct.
One thing to note is that we use sidekiq for background processing quite often, not sure if that will affect it or not.
We have 3 counts that we're calculating: my_customers_length (customers that have a user_id, which means they are 'assigned' to that user), all_customers_length (customers that are either 'open' or 'assigned', open means aasm_state = 'open' and they don't have a user_id), and open_customers_length (customers that aasm_state = 'open' and don't have a user_id).
Here's the logic that increments/decrements the Redis counters (after_save in customer.rb):
def reset_stats
if user_id_changed?
if user_id_was == nil # open => assigned
# update company and user
user.redis_increment_my_customers_length
company.redis_decrement_open_customers_length
elsif user_id_was != nil && user_id != nil # assigned => assigned
# update users (assigner and assignee)
user_was = User.find(user_id_was)
user.redis_increment_my_customers_length
user_was.redis_decrement_my_customers_length
elsif user_id_was != nil && user_id == nil # assigned => closed
# update company and user
user_was = User.find(user_id_was)
user_was.redis_decrement_my_customers_length
company.redis_decrement_all_customers_length
end
else
if aasm_state_was == 'closed' && aasm_state == 'open' # closed => open
# update company
company.redis_increment_all_customers_length
company.redis_increment_open_customers_length
elsif aasm_state_was == 'open' && aasm_state == 'closed' # open => closed
# update company
company.redis_decrement_all_customers_length
company.redis_decrement_open_customers_length
end
end
end
and here are the redis functions:
user.rb:
def redis_length_key
"my_customers_length_for_#{id}"
end
def set_my_customers_length(l)
RED.set(redis_length_key, l)
l
end
def redis_increment_my_customers_length
set_my_customers_length(my_customers_length.to_i + 1)
end
def redis_decrement_my_customers_length
set_my_customers_length(my_customers_length.to_i - 1)
end
and company.rb:
def open_customers
customers.open
end
def redis_open_length_key
"open_customers_length_for_#{id}"
end
def set_open_customers_length(l)
RED.set(redis_open_length_key, l)
l
end
def redis_increment_open_customers_length
set_open_customers_length(open_customers_length.to_i + 1)
end
def redis_decrement_open_customers_length
set_open_customers_length(open_customers_length.to_i - 1)
end
def open_customers_length(reset = false)
l = RED.get(redis_open_length_key)
if l.present? && reset == false && l.to_i >=0
l
else
set_open_customers_length(open_customers.length)
end
end
def redis_all_length_key
"all_customers_length_for_#{id}"
end
def set_all_customers_length(l)
RED.set(redis_all_length_key, l)
l
end
def all_customers_length
RED.get(redis_all_length_key)
end
def redis_increment_all_customers_length
set_all_customers_length(all_customers_length.to_i + 1)
end
def redis_decrement_all_customers_length
set_all_customers_length(open_or_claimed_customers_length.to_i - 1)
end
def open_or_claimed_customers_length(reset = false)
l = RED.get(redis_all_length_key)
if l.present? && reset == false && l.to_i >=0
l
else
set_all_customers_length(open_or_claimed_customers.length)
end
end
def open_or_claimed_customers
customers.open_or_claimed
end
We noticed that the "open" customers count in Redis is always less than the actual count by hitting the database. For "my customers count" and "all customers count," Redis is usually lower but not always.
Is our logic wrong? Are we missing something? Could this be a Redis problem? Sidekiq problem?
Your Redis logic is not atomic, it's got a giant race condition in redis_increment_open_customers_length. Use the INCR command instead of GET + SET.
http://redis.io/commands/incr
I have the following code on a model, which works, but I'd love to find a way to refactor it to be something a little smaller and neater. I'd also love to only run it under the condition that the state field actually changes. Can someone point me in the right direction. This is rails 3 BTW.
What it does is look at the "state" field on a model called subscription, and if that changes from one condition to another, then makes an update in salesforce. I'm looking for a way to optimise this code.
after_save :update_salesforce
def update_salesforce
if self.state_changed? && salesforce_client
salesforce_client.materialize("Opportunity")
o = Opportunity.find_by_breatheHR_id__c(self.subscriber_id)
if o
old_state = self.state_was
new_state = self.state
#update salesforce when accounts go active
if (old_state =='trial' || old_state == 'suspended') && new_state =='active'
o.update_attribute(:Stage__c, "Active Account")
end
#update salesforce when accounts go active
if old_state =='trial' && new_state =='suspended'
o.update_attribute(:Stage__c, "Trial Suspended")
end
#update salesforce when accounts go active
if old_state =='active' && new_state =='suspended'
o.update_attribute(:Stage__c, "Account Suspended")
end
#update salesforce when accounts go active
if old_state =='active' && new_state =='inactive'
o.update_attribute(:Stage__c, "Account Cancelled")
end
#update salesforce when accounts go active
if old_state =='trial' && new_state =='inactive'
o.update_attribute(:Stage__c, "Trial Cancelled")
end
end_of_day
end
end
In terms of only running it, I tried
after_save :update_salesforce, :if => self.state_changed?
but it doesn't recognise "self" at this point.
Here are several refactorings that you can apply:
Remove unnecessary self
You don't need to write self every time, only when setting values, because otherwise Ruby will create a local variable instead of calling the setter method. In your case, all occurrences of self are superfluous and just add noise.
Flatten nested conditionals
You need to get rid of unnecessary nesting, it makes the code much easier to read and understand. Generally speaking, you can return early when preconditions fail. For example:
def update_salesforce
if state_changed? && salesforce_client
o = Opportunity.find_by_breatheHR_id__c(subscriber_id)
if o
# perform hard work
end
end
end
# becomes
def update_salesforce
return unless state_changed? && salesforce_client
o = Opportunity.find_by_breatheHR_id__c(subscriber_id)
return unless o
# perform hard work
end
Use ActiveRecord's :if option
In the special case of the check whether the state has changed, you can use the :if option on the after_save callback. You were very close, but you either need to supply the method name as symbol, or use a proc. For a simple situation like this one, you can use the symbol form:
after_save :update_salesforce, :if => :state_changed?
Use statement modifiers
The other case is also special in that you can tack the return at the end of the line with a statement modifier or. This is actually the intended use of or, whereas you should always use || for boolean operations. The transformation is
x = do_something
return unless x
# becomes
x = do_something or return
Replace repetitive conditionals with hash access
When you look at the method, most lines repeat over and over again with only slight variation. What you can do about this is to extract the state transition logic into a separate method. In this method you can pull off a trick and put the varying states into a hash, from which you select the appropriate value. The keys are arrays representing the transition, and the values are the new values for the Stage__c field. When nil is returned, Stage__c must not be updated. For example:
def new_stage_from_state_transition(old_state, new_state)
{
['suspended', 'active' ] => 'Active Account',
['trial' , 'active' ] => 'Active Account',
['trial' , 'suspended'] => 'Trial Suspended',
['active' , 'suspended'] => 'Account Suspended',
['active' , 'inactive' ] => 'Account Cancelled',
['trial' , 'inactive' ] => 'Trial Cancelled'
}[[old_state, new_state]]
end
Put everything together
All of this results in much cleaner and less repetitive code:
after_save :update_salesforce, :if => :state_changed?
def update_salesforce
return unless salesforce_client
o = Opportunity.find_by_breatheHR_id__c(subscriber_id) or return
new_stage = new_stage_from_state_transition(state_was, state) or return
o.update_attribute(:Stage__c, new_stage)
end
def new_stage_from_state_transition(old_state, new_state)
{
['suspended', 'active' ] => 'Active Account',
['trial' , 'active' ] => 'Active Account',
['trial' , 'suspended'] => 'Trial Suspended',
['active' , 'suspended'] => 'Account Suspended',
['active' , 'inactive' ] => 'Account Cancelled',
['trial' , 'inactive' ] => 'Trial Cancelled'
}[[old_state, new_state]]
end
#update salesforce when accounts go active
if new_state =='active'
o.update_attribute(:Stage__c, "Active Account")
else
what = old_state == 'active' ? "Account" : "Trial"
change = new_state == 'inactive' ? "Canceled" : "Suspended"
o.update_attribute(:Stage__c, "#{what} #{change}" )
end
So I have some code here I need to modify regarding a Rails Sweeper:
class UserTrackingSweeper < ActionController::Caching::Sweeper
observe User
def after_update(user)
return if user.nil? || user.created_at.nil? #fix weird bug complaining about to_date on nil class
return if user.created_at.to_date < Date.today || user.email.blank?
user.send_welcome_email if user.email_was.blank?
end
#use sweeper as a way to ingest metadata from the user access to the site automatically
def after_create(user)
begin
if !cookies[:user_tracking_meta].nil?
full_traffic_source = cookies[:user_tracking_meta]
else
if !session.empty? && !session[:session_id].blank?
user_tracking_meta = Rails.cache.read("user_meta_data#{session[:session_id]}")
full_traffic_source = CGI::unescape(user_tracking_meta[:traffic_source])
end
end
traffic_source = URI::parse(full_traffic_source).host || "direct"
rescue Exception => e
Rails.logger.info "ERROR tracking ref link. #{e.message}"
traffic_source = "unknown"
full_traffic_source = "unknown"
end
# if I am registered from already, than use that for now (false or null use site)
registered_from = user.registered_from || "site"
if params && params[:controller]
registered_from = "quiz" if params[:controller].match(/quiz/i)
# registered_from = "api" if params[:controller].match(/api/i)
end
meta = {
:traffic_source => user.traffic_source || traffic_source,
:full_traffic_source => full_traffic_source,
:registered_from => registered_from,
:id_hash => user.get_id_hash
}
user.update_attributes(meta)
end
end
The problem is I've noticed that it dosen't seem possible to access the cookies and parameters hash within a sweeper yet it appears fine in some of our company's integration environments. It does not work in my local machine though. So my questions are:
How is it possible to access params / cookies within a Sweeper?
If it's not possible, what would you do instead?
Thanks
I'm sure you can use session variables in a Cache Sweeper so if anything put whatever you need there and you're set
I have an activity model which has_many participants and I'd like to ensure that a participant always exists when updating an activity and its participants. I have the following method in my activity model which does the trick:
def must_have_participant
if self.participants.size == 0 || self.participants.size == self.participants.to_ary.find_all{ |p| p.marked_for_destruction? }.count
self.errors[:base] << I18n.t(:msg_activity_must_have_participant)
end
end
The problem is that the participants are lazy loaded if I'm simply updating the activity on its own which I'd like to avoid. I've tried the following alternative, however, loaded? returns false when removing all participants using the :_destroy flag.
def must_have_participant
if self.new_record? || self.participants.loaded?
if self.participants.size == 0 || self.participants.size == self.participants.to_ary.find_all{ |p| p.marked_for_destruction? }.count
self.errors[:base] << I18n.t(:msg_activity_must_have_participant)
end
end
end
Is there an alternative to loaded? that I can use to know whether the participants are going to be updated?
I did something like this in a recent validation that I created. I searched for the original record and checked the original value against the new value. No guarantees my code will work for you but here is my code for your application:
orig_rec = self.find(id)
if participant_ids.size != orig_rec.participant_ids.size
Note that I checked the size of participant_ids instead of fetching all the participant records and checking the size of them. That should be more efficient.
I don't know if there is some kind of built in function to do this or not in ruby, I'll be curious to see what someone who is more rails specific may suggest.
For reference I've amended the method like so:
def must_have_participant
if self.new_record? || self.association(:participants).loaded?
if self.participants.size == 0 || self.participants.size == self.participants.select{ |p| p.marked_for_destruction? }.size
self.errors[:base] << I18n.t(:msg_must_have_participant)
end
end
end