I have the following method:
def self.get_rec(product, type)
if type == "A"
db = Pwvav
elsif type == "B"
db = Pwbab
elsif type == "C"
db = Pwvub
else type == "D"
db = Tic
db.find_by_id(product.id).recommendation.split(",").each do |rec|
r = Recommendation.find_by_id(rec)
pr = Model.new(:rating_set => rating_set.id, :recommendation_id => r.id, :product_id => product.id)
pr.save
end
end
end
When I run the method, the db.find is not working as I expect. If replace db.find_by_id(product.id).recommendation.split(",").each do |rec| with Pwvav.find_by_id(product.id).recommendation.split(",").each do |rec| for example, it works. How do I chose which model to call based on what the type equals?
your error, if I'm right, is with your structure. the find_by_id code is inside the else so it doesn't create other records when type is not 'D'. Try the following code which I think is more readable
def self.get_rec(product, type)
db = case type
when 'A' then Pwvav
when 'B' then Pwbab
when 'C' then Pwvub
when 'D' then Tic
end
db.find_by_id(product.id).recommendation.split(",").each do |rec|
r = Recommendation.find_by_id(rec)
pr = Model.new(rating_set: rating_set.id, recommendation_id: r.id, product_id => product.id)
pr.save
end
end
You haven't close your "if - elsif - else condition, then the last block with the find, find_by_id and your new is only executed when type == 'D'.
Insert an end after
db = Tic
Related
What I'm thinking of is something where I can say:
e = Foo.new
e.bar = "hello"
e.save
e.reload
e.bar.nil!
e.reload
e.bar.nil? => true
Kind of #touch but sets nil and saves.
EDIT
Super sorry guys. I mean this:
e = Foo.new
e.bar = "hello"
e.save
e.reload
e.bar.nil!
e.reload
e.bar.nil? => true
Maybe something like:
module ActiveRecord
class Base
def nil!(*names)
unless persisted?
raise ActiveRecordError, <<-MSG.squish
cannot nil on a new or destroyed record object. Consider using
persisted?, new_record?, or destroyed? before nilling
MSG
end
unless names.empty?
changes = {}
names.each do |column|
column = column.to_s
changes[column] = write_attribute(column, nil)
end
primary_key = self.class.primary_key
scope = self.class.unscoped.where(primary_key => _read_attribute(primary_key))
if locking_enabled?
locking_column = self.class.locking_column
scope = scope.where(locking_column => _read_attribute(locking_column))
changes[locking_column] = increment_lock
end
clear_attribute_changes(changes.keys)
result = scope.update_all(changes) == 1
if !result && locking_enabled?
raise ActiveRecord::StaleObjectError.new(self, "nil")
end
#_trigger_update_callback = result
result
else
true
end
end
end
end
Put that in an initializer and it'll let you null out the title of a comment with Comment.last.nil!(:title).
You can't save a nil to the database, and furthermore, once an object has been created as a particular class you can never change that. It can only be converted by creating a new object, something an in-place modifier like this hypothetical nil! does.
The closest thing you can get is:
e = Foo.new
e.bar = "hello"
e.save
e.reload
e.delete!
e.reload
e.destroyed? # => true
f = Foo.find_by(id: e.id)
f.nil? # => true
I am trying to write a ruby on rails function that will create a new object for any model. Here is what I have so far
def create_populated_object(model)
test_object = model.new
model.columns.each do |column|
attr_name = column.name
attr_type = column.type
#test_object.assign_attributes(attr_name+':'+ "9")
puts "#{':'+attr_name} => #{attr_type}"
if attr_type.to_s == 'integer'
b = '{:' + attr_name.to_s + '=>' + 9.to_s + '}'
puts b
test_object.assign_attributes(b)
puts "it worked"
elsif attr_type.to_s == 'string'
puts "string"
elsif attr_type.to_s == 'datetime'
puts "date time"
elsif attr_type.to_s == 'boolean'
puts "boolean"
elsif attr_type.to_s == 'text'
puts "text"
else
puts "UNKNOWN ATTRIBUTE TYPE"
end
end
puts test_object
end
In my example, id is the first attribute of the model. I try to assign the value 9 to it, but I keep getting this error:
NoMethodError: undefined method `stringify_keys' for "{:id=>9}":String
Anyone know how to fix this?
You need to send a Hash object instead of a string to the method:
b = { attr_name => 9 }
test_object.assign_attributes(b)
assign_attributes expects a hash of attributes to be passed to it. You are passing it a string. Would it be problematic to simply say b = {attr_name.to_sym => 9}?
In our Rails 3.2.13 app (Ruby 2.0.0 + Postgres on Heroku), we are often retreiving a large amount of Order data from an API, and then we need to update or create each order in our database, as well as the associations. A single order creates/updates itself plus approx. 10-15 associcated objects, and we are importing up to 500 orders at a time.
The below code works, but the problem is it's not at all efficient in terms of speed. Creating/updating 500 records takes approx. 1 minute and generates 6500+ db queries!
def add_details(shop, shopify_orders)
shopify_orders.each do |shopify_order|
order = Order.where(:order_id => shopify_order.id.to_s, :shop_id => shop.id).first_or_create
order.update_details(order,shopify_order,shop) #This calls update_attributes for the Order
ShippingLine.add_details(order, shopify_order.shipping_lines)
LineItem.add_details(order, shopify_order.line_items)
Taxline.add_details(order, shopify_order.tax_lines)
Fulfillment.add_details(order, shopify_order.fulfillments)
Note.add_details(order, shopify_order.note_attributes)
Discount.add_details(order, shopify_order.discount_codes)
billing_address = shopify_order.billing_address rescue nil
if !billing_address.blank?
BillingAddress.add_details(order, billing_address)
end
shipping_address = shopify_order.shipping_address rescue nil
if !shipping_address.blank?
ShippingAddress.add_details(order, shipping_address)
end
payment_details = shopify_order.payment_details rescue nil
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end
end
def update_details(order,shopify_order,shop)
order.update_attributes(
:order_name => shopify_order.name,
:order_created_at => shopify_order.created_at,
:order_updated_at => shopify_order.updated_at,
:status => Order.get_status(shopify_order),
:payment_status => shopify_order.financial_status,
:fulfillment_status => Order.get_fulfillment_status(shopify_order),
:payment_method => shopify_order.processing_method,
:gateway => shopify_order.gateway,
:currency => shopify_order.currency,
:subtotal_price => shopify_order.subtotal_price,
:subtotal_tax => shopify_order.total_tax,
:total_discounts => shopify_order.total_discounts,
:total_line_items_price => shopify_order.total_line_items_price,
:total_price => shopify_order.total_price,
:total_tax => shopify_order.total_tax,
:total_weight => shopify_order.total_weight,
:taxes_included => shopify_order.taxes_included,
:shop_id => shop.id,
:email => shopify_order.email,
:order_note => shopify_order.note
)
end
So as you can see, we are looping through each order, finding out if it exists or not (then either loading the existing Order or creating the new Order), and then calling update_attributes to pass in the details for the Order. After that we create or update each of the associations. Each associated model looks very similar to this:
class << self
def add_details(order, tax_lines)
tax_lines.each do |shopify_tax_line|
taxline = Taxline.find_or_create_by_order_id(:order_id => order.id)
taxline.update_details(shopify_tax_line)
end
end
end
def update_details(tax_line)
self.update_attributes(:price => tax_line.price, :rate => tax_line.rate, :title => tax_line.title)
end
I've looked into the activerecord-import gem but unfortunately it seems to be more geared towards creation of records in bulk and not update as we also require.
What is the best way that this can be improved for performance?
Many many thanks in advance.
UPDATE:
I came up with this slight improvement, which essentialy removes the call to update the newly created Orders (one query less per order).
def add_details(shop, shopify_orders)
shopify_orders.each do |shopify_order|
values = {:order_id => shopify_order.id.to_s, :shop_id => shop.id,
:order_name => shopify_order.name,
:order_created_at => shopify_order.created_at,
:order_updated_at => shopify_order.updated_at,
:status => Order.get_status(shopify_order),
:payment_status => shopify_order.financial_status,
:fulfillment_status => Order.get_fulfillment_status(shopify_order),
:payment_method => shopify_order.processing_method,
:gateway => shopify_order.gateway,
:currency => shopify_order.currency,
:subtotal_price => shopify_order.subtotal_price,
:subtotal_tax => shopify_order.total_tax,
:total_discounts => shopify_order.total_discounts,
:total_line_items_price => shopify_order.total_line_items_price,
:total_price => shopify_order.total_price,
:total_tax => shopify_order.total_tax,
:total_weight => shopify_order.total_weight,
:taxes_included => shopify_order.taxes_included,
:email => shopify_order.email,
:order_note => shopify_order.note}
get_order = Order.where(:order_id => shopify_order.id.to_s, :shop_id => shop.id)
if get_order.blank?
order = Order.create(values)
else
order = get_order.first
order.update_attributes(values)
end
ShippingLine.add_details(order, shopify_order.shipping_lines)
LineItem.add_details(order, shopify_order.line_items)
Taxline.add_details(order, shopify_order.tax_lines)
Fulfillment.add_details(order, shopify_order.fulfillments)
Note.add_details(order, shopify_order.note_attributes)
Discount.add_details(order, shopify_order.discount_codes)
billing_address = shopify_order.billing_address rescue nil
if !billing_address.blank?
BillingAddress.add_details(order, billing_address)
end
shipping_address = shopify_order.shipping_address rescue nil
if !shipping_address.blank?
ShippingAddress.add_details(order, shipping_address)
end
payment_details = shopify_order.payment_details rescue nil
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end
end
and for the associated objects:
class << self
def add_details(order, tax_lines)
tax_lines.each do |shopify_tax_line|
values = {:order_id => order.id,
:price => tax_line.price,
:rate => tax_line.rate,
:title => tax_line.title}
get_taxline = Taxline.where(:order_id => order.id)
if get_taxline.blank?
taxline = Taxline.create(values)
else
taxline = get_taxline.first
taxline.update_attributes(values)
end
end
end
end
Any better suggestions?
Try wrapping your entire code into a single database transaction. Since you're on Heroku it'll be a Postgres bottom-end. With that many update statements, you can probably benefit greatly by transacting them all at once, so your code executes quicker and basically just leaves a "queue" of 6500 statements to run on Postgres side as the server is able to dequeue them. Depending on the bottom end, you might have to transact into smaller chunks - but even transacting 100 at a time (and then close and re-open the transaction) would greatly improve throughput into Pg.
http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html
http://www.postgresql.org/docs/9.2/static/sql-set-transaction.html
So before line 2 you'd add something like:
def add_details(shop, shopify_orders)
Order.transaction do
shopify_orders.each do |shopify_order|
And then at the very end of your method add another end:
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end //shopify_orders.each..
end //Order.transaction..
end //method
You can monkey-patch ActiveRecord like this:
class ActiveRecord::Base
#http://stackoverflow.com/questions/15317837/bulk-insert-records-into-active-record-table?lq=1
#https://gist.github.com/jackrg/76ade1724bd816292e4e
# "UPDATE THIS SET <list_of_column_assignments> FROM <table_name> THIS JOIN (VALUES (<csv1>, <csv2>,...) VALS ( <column_names> ) ON <list_of_primary_keys_comparison>"
def self.bulk_update(record_list)
pk = self.primary_key
raise "primary_key not found" unless pk.present?
raise "record_list not an Array of Hashes" unless record_list.is_a?(Array) && record_list.all? {|rec| rec.is_a? Hash }
return nil if record_list.empty?
result = nil
#test if every hash has primary keys, so we can JOIN
record_list.each { |r| raise "Primary Keys '#{self.primary_key.to_s}' not found on record: #{r}" unless hasAllPKs?(r) }
#list of primary keys comparison
pk_comparison_array = []
if (pk).is_a?(Array)
pk.each {|thiskey| pk_comparison_array << "THIS.#{thiskey} = VALS.#{thiskey}" }
else
pk_comparison_array << "THIS.#{pk} = VALS.#{pk}"
end
pk_comparison = pk_comparison_array.join(' AND ')
#SQL
(1..record_list.count).step(1000).each do |start|
key_list, value_list = convert_record_list(record_list[start-1..start+999])
#csv values
csv_vals = value_list.map {|v| "(#{v.join(", ")})" }.join(", ")
#column names
column_names = key_list.join(", ")
#list of columns assignments
columns_assign_array = []
key_list.each {|col|
unless inPK?(col)
columns_assign_array << "THIS.#{col} = VALS.#{col}"
end }
columns_assign = columns_assign_array.join(', ')
sql = "UPDATE THIS SET #{columns_assign} FROM #{self.table_name} THIS JOIN ( VALUES #{csv_vals} ) VALS ( #{column_names} ) ON ( #{pk_comparison} )"
result = self.connection.execute(sql)
return result if result<0
end
return result
end
def self.inPK?(str)
pk = self.primary_key
test = str.to_s
if pk.is_a?(Array)
(pk.include?(test))
else
(pk==test)
end
end
#test if given hash has primary keys included as hash keys and those keys are not empty
def self.hasAllPKs?(hash)
h = hash.stringify_keys
pk = self.primary_key
if pk.is_a?(Array)
(pk.all? {|k| h.key?(k) and h[k].present? })
else
h.key?(pk) and h[pk].present?
end
end
def self.convert_record_list(record_list)
# Build the list of keys
key_list = record_list.map(&:keys).flatten.map(&:to_s).uniq.sort
value_list = record_list.map do |rec|
list = []
key_list.each {|key| list << ActiveRecord::Base.connection.quote(rec[key] || rec[key.to_sym]) }
list
end
# If table has standard timestamps and they're not in the record list then add them to the record list
time = ActiveRecord::Base.connection.quote(Time.now)
for field_name in %w(created_at updated_at)
if self.column_names.include?(field_name) && !(key_list.include?(field_name))
key_list << field_name
value_list.each {|rec| rec << time }
end
end
return [key_list, value_list]
end
end
Then, you can generate a array of hashes containing your models attributes (including theirs primary keys) and do something like:
ActiveRecord::Base.transaction do
Model.bulk_update [ {attr1: val1, attr2: val2,...}, {attr1: val1, attr2: val2,...}, ... ]
end
It will be a single SQL command without Rails callbacks and validations.
For PostgreSQL, there are several issues that the above approach does not address:
You must specify an actual table, not just an alias, in the update target table.
You cannot repeat the target table in the FROM phrase. Since you are joining the target table to a VALUES table (hence there is only one table in the FROM phrase, you won't be able to use JOIN, you must instead use "WHERE ".
You don't get the same "free" casts in a VALUES table that you do in a simple "UPDATE" command, so you must cast date/timestamp values as such (#val_cast does this).
class ActiveRecord::Base
def self.update!(record_list)
raise ArgumentError "record_list not an Array of Hashes" unless record_list.is_a?(Array) && record_list.all? {|rec| rec.is_a? Hash }
return record_list if record_list.empty?
(1..record_list.count).step(1000).each do |start|
field_list, value_list = convert_record_list(record_list[start-1..start+999])
key_field = self.primary_key
non_key_fields = field_list - [%Q["#{self.primary_key}"], %Q["created_at"]]
columns_assign = non_key_fields.map {|field| "#{field} = #{val_cast(field)}"}.join(",")
value_table = value_list.map {|row| "(#{row.join(", ")})" }.join(", ")
sql = "UPDATE #{table_name} AS this SET #{columns_assign} FROM (VALUES #{value_table}) vals (#{field_list.join(", ")}) WHERE this.#{key_field} = vals.#{key_field}"
self.connection.update_sql(sql)
end
return record_list
end
def self.val_cast(field)
field = field.gsub('"', '')
if (column = columns.find{|c| c.name == field }).sql_type =~ /time|date/
"cast (vals.#{field} as #{column.sql_type})"
else
"vals.#{field}"
end
end
def self.convert_record_list(record_list)
# Build the list of fields
field_list = record_list.map(&:keys).flatten.map(&:to_s).uniq.sort
value_list = record_list.map do |rec|
list = []
field_list.each {|field| list << ActiveRecord::Base.connection.quote(rec[field] || rec[field.to_sym]) }
list
end
# If table has standard timestamps and they're not in the record list then add them to the record list
time = ActiveRecord::Base.connection.quote(Time.now)
for field_name in %w(created_at updated_at)
if self.column_names.include?(field_name) && !(field_list.include?(field_name))
field_list << field_name
value_list.each {|rec| rec << time }
end
end
field_list.map! {|field| %Q["#{field}"] }
return [field_list, value_list]
end
end
I am a Ruby novice. But due to some problem I have to handle the code as our ruby developer is not available. We are using cassandra database to get values from a Ruby (Sinatra) web service and put it into the Cassandra keyspace. But due to some problem , the data is failing to insert.
In the following code partners_daily , partner_monthly etc are column family (tables) in the stats keyspace(database).
if params and !partner_id.nil? and !activity_type.nil?
{
:partners_daily => "#{partner_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}_#{time.day}",
:partners_monthly => "#{partner_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}",
:partners_alltime => "#{partner_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}",
:channels_daily => "#{channel_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}_#{time.day}",
:channels_monthly => "#{channel_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}",
:channels_alltime => "#{channel_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}",
:countries_daily => "#{country}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}_#{time.day}",
:countries_monthly => "#{country}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}",
:countries_alltime => "#{country}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}"
}.each do |k,v|
stats.add(k, v, 1, 'count')
end
return "Activity stored in stats"
end
else
return "Error: client headers missing"
end
end
def count(table, key)
require 'cassandra-cql' # requiring this at the top was leading to error: unconfigured columnfamily
cqldb = CassandraCQL::Database.new('127.0.0.1:9160', {:keyspace => 'plystats'})
query = "update partners_daily set count = ? where key = ?"#"update #{table} set count = count+1 where key = ?;"
#return cqldb.execute(query, 0, 'sonia').inspect
return query
end
I want to know how the data inserting logic in it is being performed, and where ? Is it in stats.add(k, v, 1, 'count') ?
and is there any error in the inserting part because its failing.
I want to know how the data inserting logic in it is being performed, and where ? Is it in stats.add(k, v, 1, 'count') ?
Yes, that's where it should be happening. Between the {} are dictionary/hash values:
{
:partners_daily => # …
}.each do |k,v|
A loop is started with the each method, and each entry is decomposed and put into k and v, the key in k and the value in v. For example, the first record in the hash is:
:partners_daily => "#{partner_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}_#{time.day}",
This would then decompose within the each loop to:
k = :partners_daily
v = # The result of
"#{partner_id}_#{activity_type}_#{success == 1 ? 'sucess' : "failure:#{failure_code}"}_#{time.year}_#{time.month}_#{time.day}",
I don't now what the values are for partner_id etc, but making some up it'd look something like "123_sales_sucess_2013_6_01"
Notice there's a typo for the word success in there.
It's a bit confusing due to the multiple double quotes and braces, so I'd change this to:
[partner_id, activity_type, (success == 1 ? 'success' : "failure:#{failure_code}"), time.year, time.month, time.day].join("_")
But notice that there's a lot of repetition in there, so I'd change the whole hash to (at least):
success_string = success == 1 ?
'success' :
"failure:#{failure_code}"
data = {
:partners_daily => [partner_id, activity_type,success_string,time.year,time.month,time.day].join("_"),
:partners_monthly => [partner_id,activity_type,success_string,time.year,time.month].join("_"),
:partners_alltime => [partner_id,activity_type,success_string].join("_"),
:channels_daily => [channel_id,activity_type,success_string,time.year,time.month,time.day].join("_"),
:channels_monthly => [channel_id,activity_type,success_string,time.year,time.month].join("_"),
:channels_alltime => [channel_id,activity_type,success_string].join("_"),
:countries_daily => [country,activity_type,success_string,time.year,time.month,time.day].join("_"),
:countries_monthly => [country,activity_type,success_string,time.year,time.month].join("_"),
:countries_alltime => [country,activity_type,success_string].join("_")
}
data.each do |k,v|
# more code…
It starts to be easier to read and see the logic. Also, by putting the hash into the data variable instead of working on it immediately, it allows you to inspect it more easily, e.g.
warn "data = #{data.inspect}"
would output a representation of the data to the console, so at least you could get an idea of what the script is attempting to put in. At the top of this code, you could also add warn "script = #{script.inspect}" to check what the script object looks like.
If the script object is a Cassandra instance i.e. there's something like script = Cassandra.new "blah", "blahblah" that sets it up, then the add method is this one
The signature given is add(column_family, key, value, *columns_and_options) but that doesn't seem to match the call you have:
stats.add(k, v, 1, 'count')
should (probably) be:
stats.add('count', k, v, 1)
In fact, I'm not even sure that the concatenation in the data hash should happen and maybe all of that should just be passed to add, but it's your data so I can't be sure.
Feel free to comment below and I'll update this.
Trying it in IRB to check it for syntax errors:
success = 1
# => 1
partner_id = 123
# => 123
activity_type = "something"
# => "something"
time = Time.now
# => 2013-06-05 11:17:50 0100
channel_id = 456
# => 456
country = "UK"
# => "UK"
success_string = success == 1 ?
'success' :
"failure:#{failure_code}"
# => "success"
data = {
:partners_daily => [partner_id, activity_type,success_string,time.year,time.month,time.day].join("_"),
:partners_monthly => [partner_id,activity_type,success_string,time.year,time.month].join("_"),
:partners_alltime => [partner_id,activity_type,success_string].join("_"),
:channels_daily => [channel_id,activity_type,success_string,time.year,time.month,time.day].join("_"),
:channels_monthly => [channel_id,activity_type,success_string,time.year,time.month].join("_"),
:channels_alltime => [channel_id,activity_type,success_string].join("_"),
:countries_daily => [country,activity_type,success_string,time.year,time.month,time.day].join("_"),
:countries_monthly => [country,activity_type,success_string,time.year,time.month].join("_"),
:countries_alltime => [country,activity_type,success_string].join("_")
}
# => {:partners_daily=>"123_something_success_2013_6_5", :partners_monthly=>"123_something_success_2013_6", :partners_alltime=>"123_something_success", :channels_daily=>"456_something_success_2013_6_5", :channels_monthly=>"456_something_success_2013_6", :channels_alltime=>"456_something_success", :countries_daily=>"UK_something_success_2013_6_5", :countries_monthly=>"UK_something_success_2013_6", :countries_alltime=>"UK_something_success"}
I am writing a loop with switch case , instead of querying database three time , I am getting all lookup in one query and looping with conditions to assign to different instance variable ( which I will use in simple form as collections) . I am not sure how to append active record relations to instance variable, pls help me.
class QuestionsController < ApplicationController
def index
#question = Question.new
#question_lookups = Lookup.where({:look_up_for => "question"})
#question_lookups.each do |lk|
case lk.look_up_type
when 'mode'
#question_mode = lk #How can i do this here..?
when 'status'
#question_status = lk
else
#question_type = lk
end
end
session[:lk] = #question_mode
# #question_mode = Lookup.where({:look_up_for => "question", :look_up_type => "mode"})
# #question_status = Lookup.where({:look_up_for => "question", :look_up_type => "status"})
# #question_type = Lookup.where({:look_up_for => "question", :look_up_type => "type"})
end
end
Convert question_mode and so on into arrays and push lk to it
#question_mode, #question_status, #question_type = []
#question_lookups.each do |lk|
case lk.look_up_type
when 'mode'
#question_mode << lk #How can i do this here..?
when 'status'
#question_status << lk
else
#question_type << lk
end
end