Is there a way to make this method more efficient? I am querying a large number of transactions and it seems inefficient to perform a separate activerecord query for each period.
Should I make a single query for the whole day and sort through the results, grouping them by period? If so, what's the most efficient way to do this? Look forward to your thoughts.
def self.transactions(period)
today = Date.today
time1 = Time.utc(today.year, today.month, today.day, 9, 30, 0)
time2 = Time.utc(today.year, today.month, today.day, 16, 0, 0)
transactions_by_period = {}
while time1 < time2
transactions = self.where("created_at >= ? and created_at <= ?", time1, time2)
transactions_by_period[time1] = transactions
time1 += period
end
return transactions_by_period
end
#todays_transactions_by_hour = Stock.transactions(1.hour)
First off, you can use ranges for nicer queries. I'd also rename time1 and time2 to start and finish
transactions = self.where(:created_at => start..finish)
Then you can reduce this to get them by periods.
transactions.reduce(Hash.new{|h, k| h[k] = []) do |periods, transaction|
# Calculate how many periods have gone by for this transaction
offset = (transaction.created_at - start).to_i / period
# Find the actual period
transaction_period = start + offset * period
periods[transaction_period] << transaction
periods
end
Edit
I haven't tested the code, it is shown more for the logic
Related
I am trying to created a list of year-week (equivalent to mySQL's YEARWEEK(date,1)) falling between two date values in Rails. List is generating perfectly if start-date and end-date are falling in same year. Here is my code:
campaign_start_date = "2013-08-02 06:59:00"
campaing_end_date = "2013-09-01 06:59:00"
start_year = DateTime.parse(campaign_start_date).cwyear
start_week = "%04d%02d" % [start_year, DateTime.parse(campaign_start_date).cweek]
end_year = DateTime.parse(campaing_end_date).cwyear
end_week = "%04d%02d" % [end_year, DateTime.parse(campaing_end_date).cweek]
if start_year == end_year
(start_week..end_week).each{ |i| result << i }
else
# need to build a suitable logic here. to handle the case when duration spans over multiple years. for example started in 01-Nov-14 and ended in 01-May-15
end
return result
there will be no problem with above date values, it will fall to if case and the result I will get is:
[
"201331",
"201332",
"201332",
"201333",
"201334",
"201335"
]
which is also what I exactly want. BUT if my start-date and end-date values are these for example:
campaign_start_date = "2014-07-23 06:59:00"
campaing_end_date = "2015-03-01 06:59:00"
means falling in different years, then it need different logic that the one I have in if condition because for these date values (start_week=201430 and end_week=201509) the if condition is not suitable here because it would generate 80 values, which is wrong because number of weeks between these dates are not 80. Need help to develop the logic for else case. May be its easy but right now I am just tired to dig it any deeper.
Special attention: the solutions should care about commercial year and commercial week (refer .cwyear and .cweek functions of rails) For example yearweek of 2016-01-01 would be 201553 not 201601
any help in this regard would be much appreciated.
Thank you to those who replied t. I've finally solved the problem like this:
campaign_weeks = []
campaign_start_date = "2014-07-23 06:59:00" # or any date
campaing_end_date = "2015-03-01 06:59:00" # or any date
start_year = DateTime.parse(campaign_start_date).cwyear
start_cweek_of_the_campaign = "%04d%02d" % [start_year, DateTime.parse(campaign_start_date).cweek]
end_year = DateTime.parse(campaing_end_date).cwyear
end_cweek_of_the_campaign = "%04d%02d" % [end_year, DateTime.parse(campaing_end_date).cweek]
if start_year == end_year
(start_cweek_of_the_campaign..end_cweek_of_the_campaign).each do |w|
campaign_weeks << ("%04d%02d" % [start_year, w])
end
else
(start_year..end_year).each do |y|
first_cweek_number_of_the_year = (y == start_year) ? start_cweek_of_the_campaign : 1
last_cweek_number_of_the_year = (y == end_year) ? end_cweek_of_the_campaign : DateTime.new(y, 12, 28).cweek
(first_cweek_number_of_the_year .. last_cweek_number_of_the_year).each do |w|
campaign_weeks << ("%04d%02d" % [y, w])
end
end
end
return campaign_weeks
Notes: 28th Dec always fall in the last cweek/iso-week of the year. last ISO week of the year is either 52 or 53.
Reference: http://en.wikipedia.org/wiki/ISO_week_date#Last_week
Got some hint from this answer: Calculating the number of weeks in a year with Ruby
The bottleneck is (start_week..end_week) range. It apparently goes through hundred (since we are on decimals):
2014xx ⇒ 201452 ⇒ 201453 ⇒ ... ⇒ 201499 ⇒ 201500 ⇒ ...
You should probably filter your range, like:
r = (start_week..end_week)
r.to_a.reject { |e| e[-2..-1].to_i > 52 }
Depending on how you count weeks (=-based, or 1-based,) the 201500 should be likely filtered as well:/
r.to_a.select { |e| e[-2..-1].to_i.between? 1, 52 }
Try this out; it will work for any set of dates irrespective of whether the years are the same or not:
campaign_start_date = "2014-07-23 06:59:00"
campaign_end_date = "2015-03-01 06:59:00"
start_date = DateTime.parse(campaign_start_date)
end_date = DateTime.parse(campaign_end_date)
while start_date < end_date
puts "%04d%02d" % [start_date.cw_year, start_date.cweek]
start_date = start_date + 7.days
end
A bit late in the discussion but here is what I used to get the number of commercial weeks between two dates:
def cweek_diff(start_date, end_date)
return if end_date < start_date
cweek_diff = (end_date.cweek - start_date.cweek) + 1
cwyear_diff = end_date.cwyear - start_date.cwyear
cyear_diff * 53 + cweek_diff - cwyear_diff
end
It worked perfectly in my case. Hope it helps ;)
In my Rails application I have this loop:
dates = []
loop do
date = start_date += increment
break if date > Date.today
dates << date
end
How can I limit this loop to a certain number of repetitions?
The start_date is dynamic in my application and a user could easily bring my app to its knees by inserting e.g. 0000-00-00 there which will cause millions of repetitions.
Thanks a lot for any help.
You could keep a counter of the repetitions and stop the loop as soon as a threshold is reached.
For example to limit the loop to 100 iterations:
dates = []
counter = 0
while counter < 100 # threshold
date = start_date += increment
break if date > Date.today
dates << date
counter += 1
end
Otherwise, you could only enter the loop if start_date is inside a certain timeframe:
dates = []
loop do
date = start_date += increment
break if date > Date.today
dates << date
end if start_date >= Date.today - 3.months.ago
Having pulled donations from the past two years, I'm trying to derive the sum of those donations per month, storing the keys (each month) and the values (the sum of donations for each month) in an array of hashes. I would like the keys to be numbers 1 to 24 (1 being two years ago and 24 being this month) and if there are no donations for a given month, the value would be zero for that month. How would I do this as an array of hashes in Ruby/Rails?
This is my variable with the donations already in it.
donations = Gift.where(:date => (Date.today - 2.years)..Date.today)
the following gives you a hash, with keys '2013/09" , etc...
monthly_donations = {}
date = Time.now
while date > 2.years.ago do
range = date.beginning_of_month..date.end_of_month
monthly_donations[ "{#date.year}/#{date.month}" ] = Giftl.sum(:column, :conditions => {created_at >= range})
date -= 30.days
end
To select the records in that time-span, this should be enough:
donations = Gift.where("date >= #{2.years.ago}")
you can also do this:
donations = Gift.where("date >= :start_date AND date <= :end_date",
{start_date: 2.years.ago, end_date: Time.now} )
See also: 2.2.1 "Placeholder Conditions"
http://guides.rubyonrails.org/active_record_querying.html
To sum-up a column in the database record, you can then do this:
sum = Gift.sum(:column , :conditions => {created_at >= 2.years.ago})
First, we need a function to find the difference in months from the current time.
def month_diff(date)
(Date.current.year * 12 + Date.current.month) - (date.year * 12 + date.month)
end
Then we iterate through #donation, assuming that :amount is used to store the value of each donation:
q = {}
#donations.each do |donation|
date = month_diff(donation.date)
if q[date].nil?
q[date] = donation.amount
else
q[date] += donation.amount
end
end
I found a good solution that covered all the bases--#user1185563's solution didn't bring in months without donations and #Tilo's called the database 24 times, but I very much appreciated the ideas! I'm sure this could be done more efficiently, but I created the hash with 24 elements (key: beginning of each month, value: 0) and then iterated through the donations and added their amounts to the hash in the appropriate position.
def monthly_hash
monthly_hash = {}
date = 2.years.ago
i = 0
while date < Time.now do
monthly_hash["#{date.beginning_of_month}"] = 0
date += 1.month
i += 1
end
return monthly_hash
end
#monthly_hash = monthly_hash
#donations.each do |donation|
#monthly_hash["#{donation.date.beginning_of_month}"] += donation.amount
end
I have question is there maybe a fine simple solution to this task:
I have first_date = "2011-02-02" , last_date = "2013-01-20" and period = 90 (days).
I need to get arrays with two elements for example:
[first_date, first_date + period] ... [some_date, last_date].
I will make it with some kind of a loop but maybe there is some nice fancy way to do this :D.
Date has a step method:
require 'date'
first_date = Date.parse("2011-02-02")
last_date = Date.parse("2013-02-20")
period = 90
p first_date.step(last_date-period, period).map{|d| [d, d+period]}
#or
p first_date.step(last_date, period).map.each_cons(2).to_a
require 'pp'
require 'date'
first_date=Date.parse "2011-02-02"
last_date=Date.parse "2013-01-20"
period = 90
periods = []
current = first_date
last = current + period
while(last < last_date ) do
periods << [current, last]
current = last
last = current + period
end
if periods[-1][1] != last_date
periods << [periods[-1][1], last_date]
end
p periods
I am assuming that the last period must end on last_date regardless of its length, as your question implies.
I've got a Session model that has a :created_at date and a :start_time date, both stored in the database as :time. I'm currently spitting out a bunch of results on an enormous table and allowing users to filter results by a single date and an optional range of time using scopes, like so:
class Session < ActiveRecord::Base
...
scope :filter_by_date, lambda { |date|
date = date.split(",")[0]
where(:created_at =>
DateTime.strptime(date, '%m/%d/%Y')..DateTime.strptime(date, '%m/%d/%Y').end_of_day
)
}
scope :filter_by_time, lambda { |date, time|
to = time[:to]
from = time[:from]
where(:start_time =>
DateTime.strptime("#{date} #{from[:digits]} #{from[:meridian]}", '%m/%d/%Y %r')..
DateTime.strptime("#{date} #{to[:digits]} #{to[:meridian]}", '%m/%d/%Y %r')
)
}
end
The controller looks more or less like this:
class SessionController < ApplicationController
def index
if params.include?(:date) ||
params.include?(:time) &&
( params[:time][:from][:digits].present? && params[:time][:to][:digits].present? )
i = Session.scoped
i = i.filter_by_date(params[:date]) unless params[:date].blank?
i = i.filter_by_time(params[:date], params[:time]) unless params[:time].blank? || params[:time][:from][:digits].blank? || params[:time][:to][:digits].blank?
#items = i
#items.sort_by! ¶ms[:sort].to_sym if params[:sort].present?
else
#items = Session.find(:all, :order => :created_at)
end
end
end
I need to allow users to filter results using multiple dates. I'm receiving the params as a comma-separated list in string format, e.g. "07/12/2012,07/13/2012,07/17/2012", and need to be able to query the database for several different date ranges, and time ranges within those date ranges, and merge those results, so for example all of the sessions on 7/12, 7/13 and 7/17 between 6:30 pm and 7:30 pm.
I have been looking everywhere and have tried several different things but I can't figure out how to actually do this. Is this possible using scopes? If not what's the best way to do this?
My closest guess looks like this but it's not returning anything so I know it's wrong.
scope :filter_by_date, lambda { |date|
date = date.split(",")
date.each do |i|
where(:created_at =>
DateTime.strptime(i, '%m/%d/%Y')..DateTime.strptime(i, '%m/%d/%Y').end_of_day
)
end
}
scope :filter_by_time, lambda { |date, time|
date = date.split(",")
to = time[:to]
from = time[:from]
date.each do |i|
where(:start_time =>
DateTime.strptime("#{i} #{from[:digits]} #{from[:meridian]}", '%m/%d/%Y %r')..
DateTime.strptime("#{i} #{to[:digits]} #{to[:meridian]}", '%m/%d/%Y %r')
)
end
}
Another complication is that the start times are all stored as DateTime objects so they already include a fixed date, so if I want to return all sessions started between 6:30 pm and 7:30 pm on any date I need to figure something else out too. A third party is responsible for the data so I can't change how it's structured or stored, I just need to figure out how to do all these complex queries. Please help!
EDIT:
Here's the solution I've come up with by combining the advice of Kenichi and Chuck Vose below:
scope :filter_by_date, lambda { |dates|
clauses = []
args = []
dates.split(',').each do |date|
m, d, y = date.split '/'
b = "#{y}-#{m}-#{d} 00:00:00"
e = "#{y}-#{m}-#{d} 23:59:59"
clauses << '(created_at >= ? AND created_at <= ?)'
args.push b, e
end
where clauses.join(' OR '), *args
}
scope :filter_by_time, lambda { |times|
args = []
[times[:from], times[:to]].each do |time|
h, m, s = time[:digits].split(':')
h = (h.to_i + 12).to_s if time[:meridian] == 'pm'
h = '0' + h if h.length == 1
s = '00' if s.nil?
args.push "#{h}:#{m}:#{s}"
end
where("CAST(start_time AS TIME) >= ? AND
CAST(start_time AS TIME) <= ?", *args)
}
This solution allows me to return sessions from multiple non-consecutive dates OR return any sessions within a range of time without relying on dates at all, OR combine the two scopes to filter by non-consecutive dates and times within those dates. Yay!
An important point I overlooked is that the where statement must come last -- keeping it inside of an each loop returns nothing. Thanks to both of you for all your help! I feel smarter now.
something like:
scope :filter_by_date, lambda { |dates|
clauses = []
args = []
dates.split(',').each do |date|
m, d, y = date.split '/'
b = "#{y}-#{m}-#{d} 00:00:00"
e = "#{y}-#{m}-#{d} 23:59:59"
clauses << '(start_time >= ? AND start_time <= ?)'
args.push b, e
end
where clauses.join(' OR '), *args
}
and
scope :filter_by_time, lambda { |dates, time|
clauses = []
args = []
dates.split(',').each do |date|
m, d, y = date.split '/'
f = time[:from] # convert to '%H:%M:%S'
t = time[:to] # again, same
b = "#{y}-#{m}-#{d} #{f}"
e = "#{y}-#{m}-#{d} #{t}"
clauses << '(start_time >= ? AND start_time <= ?)'
args.push b, e
end
where clauses.join(' OR '), *args
}
So, the easy part of the question is what to do about datetimes. The nice thing about DateTimes is that they can be cast to times really easily with this:
CAST(datetime_col AS TIME)
So you can do things like:
i.where("CAST(start_time AS TIME) IN(?)", times.join(", "))
Now, the harder part, why aren't you getting any results. The first thing to try is to use i.to_sql to decide whether the scoped query looks reasonable. My guess is that when you print it out you'll find that all those where are chaining together with AND. So you're asking for objects with a date that is on 7/12, 7/13, and 7/21.
The last part here is that you've got a couple things that are concerning: sql injections and some overeager strptimes.
When you do a where you should never use #{} in the query. Even if you know where that input is coming from your coworkers may not. So make sure you're using ? like in the where I did above.
Secondly, strptime is extremely expensive in every language. You shouldn't know this, but it is. If at all possible avoid parsing dates, in this case you can probably just gsub / into - in that date and everything will be happy. MySQL expects dates in m/d/y form anyways. If you're still having trouble with it though and you really need a DateTime object you can just as easily do: Date.new(2001,2,3) without eating your cpu.