To avoid making the last question full of edits, I am spinning off a new question on debugging this. Original question was this Neo4j gem - Plucking multiple nodes/relationships
This is sorta where I ended up. There is some flaw with the day detection but as a query it does work for now. #collection returns a slew of things as written.
Event.rb
def self.reminder
one_day = 1.day.to_i
time = Time.zone.now.to_i
#collection = Event.as(:e).where( "( ( e.date_start - {current_time} ) / {one_day_p} ) < {one_p} " ).users(:u, :rel).where(setting_reminder: true).rel_where(reminded: false ).params(one_day_p: one_day, current_time: time, one_p: 1).pluck(:e, 'COLLECT(u)', 'COLLECT(rel)')
#collection.each do |event, users, rels|
users.each_with_index do |user, i|
UserMailer.reminder(event,user[i]).deliver
end
rels.each_with_index do |rels, i|
rels[i].reminded = true
end
end
end
I originally had it as the last answer did, but I think I need to track both the indexes of the user and the rel nested within the each block.
Running in rails c, when I run Event.reminder I get
TypeError: 0 is not a symbol
What's wrong with my nested loop?
#collection is one big array that contains other arrays. You can't do #collection.each do |event, users, rels|, you need to return each array within it and then loop through those. Two ways:
#collection = Event.as(:e).where( "( ( e.date_start - {current_time} ) / {one_day_p} ) < {one_p} " ).users(:u, :rel).where(setting_reminder: true).rel_where(reminded: false ).params(one_day_p: one_day, current_time: time, one_p: 1).pluck(:e, 'COLLECT(u)', 'COLLECT(rel)')
#collection.each do |row|
event = row[0]
users = row[1]
rels = row[2]
users.each_with_index do |user, i|
UserMailer.reminder(event, user).deliver
rels[i].reminded = true
rels[i].save
end
end
# OR
events = collection.map { |row| row[0] }
users = collection.map { |row| row[1] }
rels = collection.map { |row| row[2] }
events.each_with_index do |event, i|
UserMailer.reminder(event, users[i]).deliver
rels[i].reminded = true
rels[i].save
end
I think you just need to do a normal each:
users.each do |user|
UserMailer.reminder(event,user).deliver
end
Related
I need some help at my recursion function to get it run.
I had a database table with following attributes:
id
title
parent_id
Some example entries would be:
1,title1, 0
2,title2, 1
3,title3, 1
4,tilte4, 0
5,title5, 3
with these entries I wants to create the following nested list
title1
title2
tille3
title5
title4
I write the following functions to generate these but there is a failure in there which overfolows my stack always
def topic_nested_list(topics_list)
get_nested_list(topics_list, topics_list.first)
end
def get_nested_list(topics, parent)
ul_contents = ""
ul_contents << "<ul>"
childs = get_topic_childs(topics,parent.id)
if childs.blank?
ul_contents << "<li>" << parent.title << "</li>"
else
for child in topics
ul_contents << get_nested_list(topics, child)
end
end
ul_contents << "</ul>"
end
def get_topic_childs(topic_list, id)
childs = []
topic_list.each do |topic|
if topic.parent_id == id
childs.push(topic)
end
end
return childs
end
I am not too sure everything that your code is doing, but it seems you want something like this
def process_topics topics_list
topics_list.each do |t|
# do something
process_topics children_of_topic(topic_list, t)
# or do something
end
end
def children_of_topic(topic_list, topic)
topic_list.select(|t| t.id == topic.parent_id)
end
It might be good to simplify your example down to something like this then add stuff in as you go.
Your very first call
get_nested_list(topics_list, topics_list.first)
Is not making sense. You want to cycle through each topic in your list and get the children of that topic and then recurse on that.
In our Rails 3.2.13 app (Ruby 2.0.0 + Postgres on Heroku), we are often retreiving a large amount of Order data from an API, and then we need to update or create each order in our database, as well as the associations. A single order creates/updates itself plus approx. 10-15 associcated objects, and we are importing up to 500 orders at a time.
The below code works, but the problem is it's not at all efficient in terms of speed. Creating/updating 500 records takes approx. 1 minute and generates 6500+ db queries!
def add_details(shop, shopify_orders)
shopify_orders.each do |shopify_order|
order = Order.where(:order_id => shopify_order.id.to_s, :shop_id => shop.id).first_or_create
order.update_details(order,shopify_order,shop) #This calls update_attributes for the Order
ShippingLine.add_details(order, shopify_order.shipping_lines)
LineItem.add_details(order, shopify_order.line_items)
Taxline.add_details(order, shopify_order.tax_lines)
Fulfillment.add_details(order, shopify_order.fulfillments)
Note.add_details(order, shopify_order.note_attributes)
Discount.add_details(order, shopify_order.discount_codes)
billing_address = shopify_order.billing_address rescue nil
if !billing_address.blank?
BillingAddress.add_details(order, billing_address)
end
shipping_address = shopify_order.shipping_address rescue nil
if !shipping_address.blank?
ShippingAddress.add_details(order, shipping_address)
end
payment_details = shopify_order.payment_details rescue nil
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end
end
def update_details(order,shopify_order,shop)
order.update_attributes(
:order_name => shopify_order.name,
:order_created_at => shopify_order.created_at,
:order_updated_at => shopify_order.updated_at,
:status => Order.get_status(shopify_order),
:payment_status => shopify_order.financial_status,
:fulfillment_status => Order.get_fulfillment_status(shopify_order),
:payment_method => shopify_order.processing_method,
:gateway => shopify_order.gateway,
:currency => shopify_order.currency,
:subtotal_price => shopify_order.subtotal_price,
:subtotal_tax => shopify_order.total_tax,
:total_discounts => shopify_order.total_discounts,
:total_line_items_price => shopify_order.total_line_items_price,
:total_price => shopify_order.total_price,
:total_tax => shopify_order.total_tax,
:total_weight => shopify_order.total_weight,
:taxes_included => shopify_order.taxes_included,
:shop_id => shop.id,
:email => shopify_order.email,
:order_note => shopify_order.note
)
end
So as you can see, we are looping through each order, finding out if it exists or not (then either loading the existing Order or creating the new Order), and then calling update_attributes to pass in the details for the Order. After that we create or update each of the associations. Each associated model looks very similar to this:
class << self
def add_details(order, tax_lines)
tax_lines.each do |shopify_tax_line|
taxline = Taxline.find_or_create_by_order_id(:order_id => order.id)
taxline.update_details(shopify_tax_line)
end
end
end
def update_details(tax_line)
self.update_attributes(:price => tax_line.price, :rate => tax_line.rate, :title => tax_line.title)
end
I've looked into the activerecord-import gem but unfortunately it seems to be more geared towards creation of records in bulk and not update as we also require.
What is the best way that this can be improved for performance?
Many many thanks in advance.
UPDATE:
I came up with this slight improvement, which essentialy removes the call to update the newly created Orders (one query less per order).
def add_details(shop, shopify_orders)
shopify_orders.each do |shopify_order|
values = {:order_id => shopify_order.id.to_s, :shop_id => shop.id,
:order_name => shopify_order.name,
:order_created_at => shopify_order.created_at,
:order_updated_at => shopify_order.updated_at,
:status => Order.get_status(shopify_order),
:payment_status => shopify_order.financial_status,
:fulfillment_status => Order.get_fulfillment_status(shopify_order),
:payment_method => shopify_order.processing_method,
:gateway => shopify_order.gateway,
:currency => shopify_order.currency,
:subtotal_price => shopify_order.subtotal_price,
:subtotal_tax => shopify_order.total_tax,
:total_discounts => shopify_order.total_discounts,
:total_line_items_price => shopify_order.total_line_items_price,
:total_price => shopify_order.total_price,
:total_tax => shopify_order.total_tax,
:total_weight => shopify_order.total_weight,
:taxes_included => shopify_order.taxes_included,
:email => shopify_order.email,
:order_note => shopify_order.note}
get_order = Order.where(:order_id => shopify_order.id.to_s, :shop_id => shop.id)
if get_order.blank?
order = Order.create(values)
else
order = get_order.first
order.update_attributes(values)
end
ShippingLine.add_details(order, shopify_order.shipping_lines)
LineItem.add_details(order, shopify_order.line_items)
Taxline.add_details(order, shopify_order.tax_lines)
Fulfillment.add_details(order, shopify_order.fulfillments)
Note.add_details(order, shopify_order.note_attributes)
Discount.add_details(order, shopify_order.discount_codes)
billing_address = shopify_order.billing_address rescue nil
if !billing_address.blank?
BillingAddress.add_details(order, billing_address)
end
shipping_address = shopify_order.shipping_address rescue nil
if !shipping_address.blank?
ShippingAddress.add_details(order, shipping_address)
end
payment_details = shopify_order.payment_details rescue nil
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end
end
and for the associated objects:
class << self
def add_details(order, tax_lines)
tax_lines.each do |shopify_tax_line|
values = {:order_id => order.id,
:price => tax_line.price,
:rate => tax_line.rate,
:title => tax_line.title}
get_taxline = Taxline.where(:order_id => order.id)
if get_taxline.blank?
taxline = Taxline.create(values)
else
taxline = get_taxline.first
taxline.update_attributes(values)
end
end
end
end
Any better suggestions?
Try wrapping your entire code into a single database transaction. Since you're on Heroku it'll be a Postgres bottom-end. With that many update statements, you can probably benefit greatly by transacting them all at once, so your code executes quicker and basically just leaves a "queue" of 6500 statements to run on Postgres side as the server is able to dequeue them. Depending on the bottom end, you might have to transact into smaller chunks - but even transacting 100 at a time (and then close and re-open the transaction) would greatly improve throughput into Pg.
http://api.rubyonrails.org/classes/ActiveRecord/Transactions/ClassMethods.html
http://www.postgresql.org/docs/9.2/static/sql-set-transaction.html
So before line 2 you'd add something like:
def add_details(shop, shopify_orders)
Order.transaction do
shopify_orders.each do |shopify_order|
And then at the very end of your method add another end:
if !payment_details.blank?
PaymentDetail.add_details(order, payment_details)
end
end //shopify_orders.each..
end //Order.transaction..
end //method
You can monkey-patch ActiveRecord like this:
class ActiveRecord::Base
#http://stackoverflow.com/questions/15317837/bulk-insert-records-into-active-record-table?lq=1
#https://gist.github.com/jackrg/76ade1724bd816292e4e
# "UPDATE THIS SET <list_of_column_assignments> FROM <table_name> THIS JOIN (VALUES (<csv1>, <csv2>,...) VALS ( <column_names> ) ON <list_of_primary_keys_comparison>"
def self.bulk_update(record_list)
pk = self.primary_key
raise "primary_key not found" unless pk.present?
raise "record_list not an Array of Hashes" unless record_list.is_a?(Array) && record_list.all? {|rec| rec.is_a? Hash }
return nil if record_list.empty?
result = nil
#test if every hash has primary keys, so we can JOIN
record_list.each { |r| raise "Primary Keys '#{self.primary_key.to_s}' not found on record: #{r}" unless hasAllPKs?(r) }
#list of primary keys comparison
pk_comparison_array = []
if (pk).is_a?(Array)
pk.each {|thiskey| pk_comparison_array << "THIS.#{thiskey} = VALS.#{thiskey}" }
else
pk_comparison_array << "THIS.#{pk} = VALS.#{pk}"
end
pk_comparison = pk_comparison_array.join(' AND ')
#SQL
(1..record_list.count).step(1000).each do |start|
key_list, value_list = convert_record_list(record_list[start-1..start+999])
#csv values
csv_vals = value_list.map {|v| "(#{v.join(", ")})" }.join(", ")
#column names
column_names = key_list.join(", ")
#list of columns assignments
columns_assign_array = []
key_list.each {|col|
unless inPK?(col)
columns_assign_array << "THIS.#{col} = VALS.#{col}"
end }
columns_assign = columns_assign_array.join(', ')
sql = "UPDATE THIS SET #{columns_assign} FROM #{self.table_name} THIS JOIN ( VALUES #{csv_vals} ) VALS ( #{column_names} ) ON ( #{pk_comparison} )"
result = self.connection.execute(sql)
return result if result<0
end
return result
end
def self.inPK?(str)
pk = self.primary_key
test = str.to_s
if pk.is_a?(Array)
(pk.include?(test))
else
(pk==test)
end
end
#test if given hash has primary keys included as hash keys and those keys are not empty
def self.hasAllPKs?(hash)
h = hash.stringify_keys
pk = self.primary_key
if pk.is_a?(Array)
(pk.all? {|k| h.key?(k) and h[k].present? })
else
h.key?(pk) and h[pk].present?
end
end
def self.convert_record_list(record_list)
# Build the list of keys
key_list = record_list.map(&:keys).flatten.map(&:to_s).uniq.sort
value_list = record_list.map do |rec|
list = []
key_list.each {|key| list << ActiveRecord::Base.connection.quote(rec[key] || rec[key.to_sym]) }
list
end
# If table has standard timestamps and they're not in the record list then add them to the record list
time = ActiveRecord::Base.connection.quote(Time.now)
for field_name in %w(created_at updated_at)
if self.column_names.include?(field_name) && !(key_list.include?(field_name))
key_list << field_name
value_list.each {|rec| rec << time }
end
end
return [key_list, value_list]
end
end
Then, you can generate a array of hashes containing your models attributes (including theirs primary keys) and do something like:
ActiveRecord::Base.transaction do
Model.bulk_update [ {attr1: val1, attr2: val2,...}, {attr1: val1, attr2: val2,...}, ... ]
end
It will be a single SQL command without Rails callbacks and validations.
For PostgreSQL, there are several issues that the above approach does not address:
You must specify an actual table, not just an alias, in the update target table.
You cannot repeat the target table in the FROM phrase. Since you are joining the target table to a VALUES table (hence there is only one table in the FROM phrase, you won't be able to use JOIN, you must instead use "WHERE ".
You don't get the same "free" casts in a VALUES table that you do in a simple "UPDATE" command, so you must cast date/timestamp values as such (#val_cast does this).
class ActiveRecord::Base
def self.update!(record_list)
raise ArgumentError "record_list not an Array of Hashes" unless record_list.is_a?(Array) && record_list.all? {|rec| rec.is_a? Hash }
return record_list if record_list.empty?
(1..record_list.count).step(1000).each do |start|
field_list, value_list = convert_record_list(record_list[start-1..start+999])
key_field = self.primary_key
non_key_fields = field_list - [%Q["#{self.primary_key}"], %Q["created_at"]]
columns_assign = non_key_fields.map {|field| "#{field} = #{val_cast(field)}"}.join(",")
value_table = value_list.map {|row| "(#{row.join(", ")})" }.join(", ")
sql = "UPDATE #{table_name} AS this SET #{columns_assign} FROM (VALUES #{value_table}) vals (#{field_list.join(", ")}) WHERE this.#{key_field} = vals.#{key_field}"
self.connection.update_sql(sql)
end
return record_list
end
def self.val_cast(field)
field = field.gsub('"', '')
if (column = columns.find{|c| c.name == field }).sql_type =~ /time|date/
"cast (vals.#{field} as #{column.sql_type})"
else
"vals.#{field}"
end
end
def self.convert_record_list(record_list)
# Build the list of fields
field_list = record_list.map(&:keys).flatten.map(&:to_s).uniq.sort
value_list = record_list.map do |rec|
list = []
field_list.each {|field| list << ActiveRecord::Base.connection.quote(rec[field] || rec[field.to_sym]) }
list
end
# If table has standard timestamps and they're not in the record list then add them to the record list
time = ActiveRecord::Base.connection.quote(Time.now)
for field_name in %w(created_at updated_at)
if self.column_names.include?(field_name) && !(field_list.include?(field_name))
field_list << field_name
value_list.each {|rec| rec << time }
end
end
field_list.map! {|field| %Q["#{field}"] }
return [field_list, value_list]
end
end
I have such code:
def accum_search
if params[:akbcap].present?
akbcap_array = [12,18,19,20,25,30,35,36,38,40,41,42,44,45,46,47,50,52,53,54,55,56,58,60,61,62,63,64,65,66,68,69,70,71,72,74,75,77,80,85,88,90,91,92,95,98,100,102,110,115,120,125,130,135,140,170,180,185,190,192,200,210,220,225]
min, max = params[:akbcap].split('-').map {|s| s.to_i }
logger.warn("!#!!!!!!!!!!!! AAA !!!!!!!!!!")
logger.warn(min)
logger.warn(max)
caprange = min...max
sa = akbcap_array.select {|n| caprange.include? n }
##cross = OtherProductsCrossList.find(:all, :conditions => {:cross_value => 1})
cap = "*"+params[:akbcap]+"*"
sa.each do |s|
logger.warn(s)
#accums = Accumulator.by_capacity(s).by_size(params[:akbsize]).by_brand(params[:akbbrand])
end
else
#accums = Accumulator.by_capacity(50).by_size(params[:akbsize]).by_brand(params[:akbbrand])
end
end
As you see i have such part:
sa.each do |s|
logger.warn(s)
#accums = Accumulator.by_capacity(s).by_size(params[:akbsize]).by_brand(params[:akbbrand])
end
but could i add on every iteration in #accums data from search? now it has last value( I could done it via arrays... but how to do via class-variable?
Yes, initiate it before the loop and use the << operator to append. End with flatten to make it a single dimension array.
#accums = []
# ...
sa.each do |s|
#accums << Accumulator.several_method_calls......
end
#accums.flatten!
or for compactness:
result = sa.map{|s| Accumulator.several_method_calls...... }.flatten
I'm using has_scope gem and I want to create filtering with two params — it may be one param or two same time.
Mymodel (Product):
scope :brand, proc { |brand| joins(:product_values).where('product_values.value_id' => brand) }
scope :zamena, proc { |zamena| joins(:product_values).where('product_values.value_id' => zamena) }
Index action of controller:
#products = apply_scopes(Product).all
It works, but only by one :(
/products?brand=12 - Ok
/products?zamena=24 - Ok
/products?brand=12&zamena=24 - Fail (sorted only by 'zamena', not by both params)
2nd. variant (not works too)
In my controller:
query = Product.scoped
query = query.brand(params[:brand]) if params[:brand]
query = query.zamena(params[:zamena]) if params[:zamena]
#products = query.all
Works by one, but not both (0 results).
My answer. Maybe not elegant, but works nice.
fcount = 0
fcount += 1 if params[:brand]
fcount += 1 if params[:zamena]
prods = []
if params[:brand]
Product.all.each do |p|
prods << p if p.product_values.where(value_id: params[:brand]).count > 0
end
end
if params[:zamena]
Product.all.each do |p|
prods << p if p.product_values.where(value_id: params[:zamena]).count > 0
end
end
#products = prods.select{|item| prods.count(item) == fcount}.uniq
No scopes needed. You can use a lot of filters using this way.
I am working on an events application where i want to filter events depending on the 3 parameters location or starts_at or ends_at in the query string. There can be any one, two or all the parameters in the query string. In i use if-else statement i need to make 6 cases which will make my code clumsy. Rather i am thinking to implement something this way:
class EventsController < ApplicationController
def index
unless params.empty?
unless params[:location].nil?
#events = Event.where("location = ?", params[:location])
end
unless params[:starts_at].nil?
unless #events.empty?
#events = #events.where("start_date = ?", params[:start_date])
else
#events = Event.where("Date(starts_at) = Date(?)", params[:starts_at])
end
end
unless params[:ends_at].nil?
unless #events.empty?
#events = #events.where("end_date = ?", params[:end_date])
else
#events = Event.where("Date(ends_at) = Date(?)", params[:ends_at])
end
end
end
end
end
But this code doesnt work since where query doen not work on an array. Can someone suggest me some solution for this..
You should be able to pass your params hash directly to where, and it will form the correct SQL based on the keys and values of that hash:
Event.where(params)
An example in the console:
1.9.3p194 :001 > puts Example.where(:location => 'here', :started_at => '2012-08-13').to_sql
SELECT "examples".* FROM "examples" WHERE "examples"."location" = 'here' AND "examples"."started_at" = '2012-08-13'
Try Following
def index
unless params.empty?
where_array, arr = [], []
if params[:location]
where_array << "location = ?"
arr << params[:location]
end
if params[:starts_at]
where_array << "start_date = ?"
arr << params[:starts_at]
end
if params[:ends_at]
where_array << "end_date = ?"
arr << params[:ends_at]
end
#events = arr.blank? ? [] : Event.where([where_array.join(" AND "), *arr])
end
end