Reducing Rails Queries (N+1?) - ruby-on-rails

So in my past application, I was somewhat familiar with using .includes in Rails, but for some reason I'm having a bit of a difficult time in my current scenario.
Here's what I'm working with:
# If non-existent, create. Otherwise, update.
existing_data = Page.all
updated_data = {}
new_records = []
#latest_page_data.each do |key, value|
existing_record = existing_data.find_by(symbol: key)
if existing_record != nil
updated_data[existing_record.id] = value
else
new_records << Page.new(value)
end
end
if !new_records.empty?
Page.import new_reocrds
end
if !updated_data.empty?
Page.update(updated_data.keys, updated_data.values)
end
end
The problem that I'm having is that the .find_by portion of the code results in a query every single iteration of #latest_page_data. I guess I would think that existing_data would hold all of the data it needs in memory, but obviously it doesn't work that way.
So next, I tried something like this:
# If non-existent, create. Otherwise, update.
existing_data = Page.includes(:id, :symbol)
updated_data = {}
new_records = []
#latest_currency_data.each do |key, value|
existing_record = existing_data.find_by(symbol: key)
but then rails throws an error, stating:
ActiveRecord::AssociationNotFoundError (Association named 'id' was not
found on Page; perhaps you misspelled it?):
so I can't use this example to find the id and symbol attributes.
I tried to take out :id in the Page.includes method, but I need to be able to get to the ID attribute in order to update the respective record later down in the code.
I've also saw some other posts pertaining to this topic, but I think the problem I may be running into is I'm not dealing with associations (and I believe that's what .includes is for? If this is the case, is there any other way that I can reduce all of the queries that I'm submitting here?

The includes method is used to preload associated models. I think what you are looking for is a select. Modifying your code to use select, do this :
existing_data = Page.select(:id, :symbol).load
updated_data = {}
new_records = []
#latest_currency_data.each do |key, value|
existing_record = existing_data.find_by(symbol: key)
if existing_record
updated_data[existing_record.id] = value
else
new_records << Page.new(value)
end
end
The drawbacks of using select over pluck is that since Rails constructs an object for you, so it is slower than a pluck. Benchmark: pluck vs select

Rather than trying to figure out a way to do it in Rails (since I'm not familiar with the 100% correct/accurate Rails way), I just decided to use .pluck and convert it into a hash to get the data that I'm looking for:
existing_data = Page.pluck(:id, :symbol)
existing_data = Hash[*existing_data.flatten]
updated_data = {}
new_records = []
#latest_currency_data.each do |key, value|
if existing_data.values.include? key
id = existing_data.find{|k,v| v.include? key}[0]
updated_data[id] = value
else
new_records << Page.new(value)
end
end
If anyone has a better way, it'd be gladly appreciated. Thanks!

Related

Optimize 'if else' conditions in rails

I am making an application, part of whose code requires many if .. else conditions:
if #model_name == "Style"
if row.include? ('colors')
colors = row['colors'].split(';')
model.style_colors.concat Color.where('code IN (?)', colors).map {|i| i.id.to_s }
row.delete('colors')
end
if row.include? ('gender') and row.include? ('garments')
#garments = row['garments']
#gender = row['gender']
row.delete('garments')
row.delete('gender')
end
if row.include? ('sports')
#sports = row['sports']
row.delete('sports')
end
if row.include?('decoration_packages')
#decorations_packages = row['decoration_packages']
row.delete('decoration_packages')
end
model.attributes = row.to_hash.merge!(active: FALSE)
else
model.attributes = row.to_hash
end
I need to make objects of row hash to access subclasses, and then delete them from row so it can be saved to a model.
Any idea how I can minimize the use of conditions or optimize it?
There's a few optimisations here...
row.include? ('gender') and row.include? ('garments')
could be implemented as
['gender', 'garments'].all?{|x| row.include?(x)}
#garments = row['garments']
row.delete('garments')
could be implemented as
#garments = row.delete('garments')
You could actually squash a lot of these onto one line:
if row.include? ('sports')
#sports = row['sports']
row.delete('sports')
end
could be
#sports = row.delete('sports') if row.include? ('sports')
Also worth considering:
Do you need to delete the values from 'row'? Could you just retrieve the value?
What are you trying to do here? It looks like you're pulling a hash into instance variables... Which is what ActiveRecord does, basically. Could you just create a model with these attributes and then call it in this style?
Style.new(row)
if #model_name == "Style"
if row.include?('colors')
model.style_colors.concat(
Color.where(code: row.delete('colors').split(';')).pluck(:id).map(&:to_s)
)
end
if row.include?('gender') and row.include?('garments')
#garments = row.delete('garments')
#gender = row.delete('gender')
end
if row.include?('sports')
#sports = row.delete('sports')
end
if row.include?('decoration_packages')
#decorations_packages = row.delete('decoration_packages')
end
model.attributes = row.to_hash.merge!(active: false)
else
model.attributes = row.to_hash
end
I would do something like this with your current code:
if #model_name == "Style"
row_key_set = row.keys.to_set
if row.include? 'colors'
colors = row['colors'].split(';')
color_ids = Color.where(code: colors).pluck(:id)
model.style_colors.concat(color_ids.map(&:to_s))
end
if row_key_set >= Set['gender', 'garments']
#garments = row.delete('garments')
#gender = row.delete('gender')
end
#sports = row.delete('sports')
#decorations_packages = row.delete('decoration_packages')
model.attributes = row.to_hash.merge(active: false)
else
model.attributes = row.to_hash
end
Instead of using Color.where('code IN (?)', colors) you can just use Color.where(code: colors).
Instead of using .map {|i| i.id.to_s } you can use pluck (.pluck(:id)) to get an array of color ids. This also makes for a quicker query since only the ids get fetched from the database instead of the whole records.
I personally like to use sets to check if multiple values are present in another set. For this reason I create the row_key_set variable row.keys.to_set. Now you can easily check certain keys are present on your hash by just checking if the key set is greater or equal than another set (thus being a superset). row_key_set >= Set['gender', 'garments'] With just one check you could leave this out, but if you have multiple checks this might be worth the trouble. I also find code written this way also more readable, but that's just personal peference.
You don't need to check if a key is present on a Hash, the documentation tells us the following:
Deletes the key-value pair and returns the value from hsh whose key is equal to key. If the key is not found, it returns nil.
This means you can leave out the include? check and write the result from the delete directly to the instance variable. If the key is not present nil will be set for the instance variable.
Lastly I would leave out the explanation mark in row.to_hash.merge!(active: false). The version without explanation mark doesn't alter the original array and reduces the chance on accidental side effects. You're saving the variable to model.attributes anyway and toss away the generated array from the to_hash method. It's normally better to use non-altering versions of methods, unless you explicitly want a certain effect to happen.

Rails model results to hash - variable as a key issue

I've tried to put results from my vote model in a hash for further usage, but I don't know how to create a hash key from a variable in Ruby. See example below:
def create_hash_array(campaign_votes)
target_hash = Hash.new
campaign_votes.each_with_index do |cv,i|
target_hash[cv.content_id] = {} if i == 0
if target_hash[cv.content_id].member?(cv.vote_button_id)
target_hash[cv.content_id][cv.vote_button_id] = (target_hash[cv.content_id][cv.vote_button_id]).to_i + 1
else
target_hash[cv.content_id] = {cv.vote_button_id => nil}
end
end
target_hash
end
Usually I got an error:
undefined method `member?' for nil:NilClass
but it comes from unrecognized target_hash[cv.content_id], how can I make does variable to be recognized target_hash[cv.content_id] ??
I think your code can be boiled down to this:
def create_hash_array(campaign_votes)
target_hash = Hash.new { |h,k| h[k] = Hash.new(0) }
campaign_votes.each do |cv|
target_hash[cv.content_id][cv.vote_button_id] += 1
end
target_hash
end
There's multiple problems here, many to do with getting all tangled up in the process. You initialize the element of the target_hash structure only on the 0 index position, yet each campaign_vote could have different content_id values, meaning you're missing out on those.
This approach creates a single auto-generating Hash that will populate keys with counter hashes, that is hashes defaulting to 0. That means you can always navigate them and += 1 will work because of the default.
This approach is pretty common in Ruby, especially Hash.new(0), which is super handy for doing simple counters of arbitrary objects.

Pass a Rails variable into a case statement

I'm using Rails and have an array that I'm looking to control the output of.
In short, if a project with an ID exists in #projects, then I don't want to output the corresponding part of the array, if that exists. So if a project with ID of 1 exists, then I don't want to output #array[1], even if that's present.
The following hardcoded case statement works successfully, to avoid outputting #array[1], #array[2] and #array[3]:
#array.each do |key, value|
case key when '1','2','3' then
next # Skip this array key
end
# Otherwise, do something else
end
But instead of hardcoding '1','2','3', I actually want these to be passed in dynamically, via ActiveRecord. i tried to build a string:
#projects_string = #projects.map {|element|
"'#{element.id}'"
}.join(',')
This outputs successfully as '1','2','3'. But I can't figure out a way to pass this into the case statement. This fails to match:
#array.each do |key, value|
case key when "#{#projects_string}" then
next # Skip this array key
end
# Otherwise, do something else
end
Is there a way to achieve this using this method, or maybe a better way to achieve this using some of Rails' capabilities?
I did look through this answer, which seemed most relevant, but couldn't see how to implement it.
You can do it in this way as well.
#project_arr = #projects.map{|p| p.id.to_s}
#array.each do |key, value|
unless #project_arr.include?(key.to_s)
#YOUR CODE GOES HERE
else
# SKIP IT
end
end
You should not use string but use array and do like
#array = Array.new
#array = #projects.map{|arr| arr.id}
#array.each do |key, value|
if #array.include?(key)
#code to skip if project is included
else
#your code
end
end
Array can be used easily for comparisons than string
I feel like this depends on the type of collection you are dealing with: Array or ActiveRelation.
If they are just arrays and you want elements of #array that aren't in #parent, you can just do:
#array - #parent
If they are both ActiveRecord objects, why not just filter out the #array when it's created?
#array = Project.where.not(id: #projects.pluck(:id))
It seems inefficient (n^2) to have to scan a lookup array (#projects) for each element in the collection you are iterating over.
You can try the following:
#array = [1,2,3,4]
#projects = [2,3]
#array.each do |key|
case
when true === #projects.include?(key)
next
end
puts "Doing something else for #{key}"
end
Outputs:
Doing something else for 1
Doing something else for 4

Setting many key/value pairs

I'm working on a rake task which imports from a JSON feed into an ActiveRecord called Person.
Person has quite a few attributes and rather than write lines of code for setting each attribute I'm trying different methods.
The closest I've got is shown below. This works nicely as far as outputing to screen but when I check the values have actually been set on the ActiveRecord itself it's always nil.
So it looks like I can't use .to_sym to solve my problem?
Any suggestions?
I should also mention that I'm just starting out with Ruby, have been doing quite a bit of Objective-c and now need to embrace the Interwebs :)
http = Net::HTTP.new(url.host, url.port)
http.read_timeout = 30
json = http.get(url.to_s).body
parsed = JSON.parse(json)
if parsed.has_key? 'code'
updatePerson = Person.find_or_initialize_by_code(parsed['code'])
puts updatePerson.code
parsed.each do |key, value|
puts "#{key} is #{value}"
symkey = key.to_sym
updatePerson[:symkey] = value.to_s
updatePerson.save
puts "#{key}....." # shows the current key
puts updatePerson[:symkey] # shows the correct value
puts updatePerson.first_name # a sample key, it's returning nil
end
You're probably looking for update_attributes():
if parsed.has_key?('code')
code = parsed.delete('code')
person = Person.find_or_initialize_by_code(code)
if person.update_attributes(parsed)
puts "#{person.first_name} successfully saved"
else
puts "Failed to save #{person.first_name}"
end
end
Your code can not assign any attribute, because you are always assigning to the single attribute named "symkey":
symkey = key.to_sym
updatePerson[:symkey] = value.to_s # assigns to attribute "symkey", not to the attribute with the name stored in variable symkey
If you want to make key into a symbol (which is probably not even necessary) and then use that as an index to access the attribute in updatePerson, you can write:
updatePerson[key.to_sym] = value.to_s
updatePerson.save
But this - more or less - is the same as
updatePerson.updateAttribute(key.to_sym, value.to_s) # update and save
except that no validation is triggered, so use with care.
And performancewise it might not be such a good idea to save the person after each assignment, so maybe you want to defer the .save() call until after you have assigned all attributes.
Nevertheless, updateAttributes(...) is something you might want to be looking into - if you do, do not forget to inform yourself on attr_protected or attr_accessible, as they protect attributes from "bulk assignment"
You can use write_attribute:
parsed.each do |key, value|
updatePerson.write_attribute(key, value)
end
updatePerson.save

Rails, using time_select on a non active record model

I am trying to use a time_select to input a time into a model that will then perform some calculations.
the time_select helper prepares the params that is return so that it can be used in a multi-parameter assignment to an Active Record object.
Something like the following
Parameters: {"commit"=>"Calculate", "authenticity_token"=>"eQ/wixLHfrboPd/Ol5IkhQ4lENpt9vc4j0PcIw0Iy/M=", "calculator"=>{"time(2i)"=>"6", "time(3i)"=>"10", "time(4i)"=>"17", "time(5i)"=>"15", "time(1i)"=>"2009"}}
My question is, what is the best way to use this format in a non-active record model. Also on a side note. What is the meaning of the (5i), (4i) etc.? (Other than the obvious reason to distinguish the different time values, basically why it was named this way)
Thank you
You can create a method in the non active record model as follows
# This will return a Time object from provided hash
def parse_calculator_time(hash)
Time.parse("#{hash['time1i']}-#{hash['time2i']}-#{hash['time3i']} #{hash['time4i']}:#{hash['time5i']}")
end
You can then call the method from the controller action as follows
time_object = YourModel.parse_calculator_time(params[:calculator])
It may not be the best solution, but it is simple to use.
Cheers :)
The letter after the number stands for the type to which you wish it to be cast. In this case, integer. It could also be f for float or s for string.
I just did this myself and the easiest way that I could find was to basically copy/paste the Rails code into my base module (or abstract object).
I copied the following functions verbatim from ActiveRecord::Base
assign_multiparameter_attributes(pairs)
extract_callstack_for_multiparameter_attributes(pairs)
type_cast_attribute_value(multiparameter_name, value)
find_parameter_position(multiparameter_name)
I also have the following methods which call/use them:
def setup_parameters(params = {})
new_params = {}
multi_parameter_attributes = []
params.each do |k,v|
if k.to_s.include?("(")
multi_parameter_attributes << [ k.to_s, v ]
else
new_params[k.to_s] = v
end
end
new_params.merge(assign_multiparameter_attributes(multi_parameter_attributes))
end
# Very simplified version of the ActiveRecord::Base method that handles only dates/times
def execute_callstack_for_multiparameter_attributes(callstack)
attributes = {}
callstack.each do |name, values|
if values.empty?
send(name + '=', nil)
else
value = case values.size
when 2 then t = Time.new; Time.local(t.year, t.month, t.day, values[0], values[min], 0, 0)
when 5 then t = Time.time_with_datetime_fallback(:local, *values)
when 3 then Date.new(*values)
else nil
end
attributes[name.to_s] = value
end
end
attributes
end
If you find a better solution, please let me know :-)

Resources