I have this scope that pulls orders for a rolling 14-day analytics graph. Problem is it pulls standard UTC, so sales show up as coming from tomorrow if a sale happens after 5pm PDT time (7 hour dif between UTC and PDT).
Current scope:
scope :recent, complete.joins(:line_items)
.group("DATE_TRUNC('day', processed_at)")
.select("DATE_TRUNC('day', processed_at) as day, sum(line_items.discounted_price) as count")
.order('day DESC').limit(14)
How can I make it so it only pulls orders 'processed_at' within the PDT time zone? Not sure how to do this syntactically, but basically I want to add 'in_time_zone' to the 'processed_at' timestamp.
When scope is called:
Order Load (2.7ms) SELECT DATE_TRUNC('day', processed_at) as day, sum(line_items.discounted_price) as count FROM "orders" INNER JOIN "line_items" ON "line_items"."order_id" = "orders"."id" WHERE "orders"."status" = 'complete' GROUP BY DATE_TRUNC('day', processed_at) ORDER BY day DESC LIMIT 14
I run into similar situations all the time. The solution depends on which specific operation you're working with, but in all cases I prioritize writing fanatically thorough unit tests of the querying behavior (using Timecop or similar) to ensure that it's doing what I want it to be doing.
If you're doing this in a WHERE clause, it's easier because you can adjust the timestamp in Ruby. That might look something like this:
tz_adjusted_start_time = (Date.today - 14.days - 5.hours)
#result = Thingy.where("processed_at >= ?", tz_adjusted_start_time)
The above produces SQL that will look for records created after May 2nd 19:00:00 UTC, or whatever.
If you're trying to GROUP by date in a different time zone, then you need to do the adjustment in raw SQL and it can be a bit hairier to think your way through, but the same principle applies (and the same testing method works just as well). I've done this before and it was messy but I've run into no trouble with it so far. The SQL might look something like this:
...
GROUP BY (`processed_at` - INTERVAL 5 HOUR)
...
I seem to remember using this simple minus sign syntax, but a quick Google search tells me it's more common to use DATE_SUB (or find another way around the timezone issue altogether), so do your homework before considering this an end solution.
Good luck!
Related
What is the best way to query InfluxDB for specific hours every day, for example, I have a Series that have checkin/checkout activities, and I need to see them between hour 2PM - 3PM every day for last month, am aware that there's no direct way to do this on the query language -current version 1.2- Not sure if there is a work around or something ?
I have been searching for the same and found your question. As you say, the syntax does not seem to allow to do it.
My closest attempt was trying to use a regular expression for a time WHERE clausule, which is not currently supported by InfluxDB.
So that should probably be the answer, and I would not post an answer to just say that.
However, working on a different problem, I have found a way that may or may not help you in your specific case. It is a workaround that is not very nice, but it seems to work in the case that you can formulate an aggregation/selection of what you want to see in that given hour so that you end up with having one value per hour. For example, (mean/max/count number of checkin-checkouts in that hour for a given person, which may be what you are looking for, or that you may use to identify the days that you would like to them individually query to see what happened there).
For example, I want to obtain the measurement of electricity consumption daily from 00:00 to 06:00 a.m. I make a first subquery that divides the measurements grouping by 6 hours starting at 00:00 of a given date. Then in the main query, I group by 24 hours and I select the first value. Like this
SELECT first("mean") FROM (SELECT mean("value") FROM "Energy" WHERE "devicename" = 'Electricity' AND "deviceid" = '0_5' AND time > '2017-01-01' GROUP BY time(6h) ) WHERE time > '2017-01-01' GROUP BY time(24h)
If you want 2-4 pm, so 14:00-16:00, you need to first group by 2 hours in the subquery, then offseting the set by 14h so that it starts at 14:00.
SELECT first("mean") FROM ( SELECT mean("value") FROM "Energy" WHERE "devicename" = 'Electricity' AND "deviceid" = '0_5' AND time > '2017-01-01T14:00:00Z' GROUP BY time(2h) ) WHERE time > '2017-01-01T14:00:00Z' GROUP BY time(24h,14h)
Just for checking it. In my 1.2 InfluxDB this is the final result:
Energy
time first
2017-01-01T14:00:00Z 86.41747572815534
2017-01-02T14:00:00Z 43.49722222222222
2017-01-03T14:00:00Z 81.05416666666666
The subquery returns:
Energy
time mean
2017-01-01T14:00:00Z 86.41747572815534
2017-01-01T16:00:00Z 91.46879334257974
2017-01-01T18:00:00Z 89.14027777777778
2017-01-01T20:00:00Z 94.47434119278779
2017-01-01T22:00:00Z 89.94305555555556
2017-01-02T00:00:00Z 86.29542302357837
2017-01-02T02:00:00Z 92.2625
2017-01-02T04:00:00Z 89.93619972260748
2017-01-02T06:00:00Z 87.78888888888889
2017-01-02T08:00:00Z 50.790277777777774
2017-01-02T10:00:00Z 0.6597222222222222
2017-01-02T12:00:00Z 0.10957004160887657
2017-01-02T14:00:00Z 43.49722222222222
2017-01-02T16:00:00Z 86.0610263522885
2017-01-02T18:00:00Z 86.59778085991678
2017-01-02T20:00:00Z 91.56527777777778
2017-01-02T22:00:00Z 90.52565880721221
2017-01-03T00:00:00Z 86.79166666666667
2017-01-03T02:00:00Z 87.15533980582525
2017-01-03T04:00:00Z 89.47988904299584
2017-01-03T06:00:00Z 91.58888888888889
2017-01-03T08:00:00Z 41.67732962447844
2017-01-03T10:00:00Z 16.216366158113733
2017-01-03T12:00:00Z 25.27739251040222
2017-01-03T14:00:00Z 81.05416666666666
If you would need 13:00-15:00, you need to offset the subquery in the previous example by 1h.
For 14:00-15:00:
SELECT first("mean") FROM ( SELECT mean("value") FROM "Energy" WHERE "devicename" = 'Electricity' AND "deviceid" = '0_5' AND time > '2017-01-01T14:00:00Z' GROUP BY time(1h) ) WHERE time > '2017-01-01T14:00:00Z' GROUP BY time(24h,14h)
Hope this helps :)
Say I have an Event model with a date_time field representing the date time the event is held, and I want to see all Events that are held, say, 'after 10pm', or 'before 7am' across multiple dates. How could I do this?
My first thought was something like this:
scope :after_time ->(time){ where("events.date_time::time between ?::time and '23:59'::time", time) }
But this doesn't work because dates are stored in UTC and converted to the app's timezone by ActiveRecord.
So let's say I'm searching for Events after 5pm, from my local Adelaide time. The eventual query is this:
WHERE (events.date_time::time between '2016-10-09 06:30:00.000000'::time and '23:59'::time)
That is, because my timezone is +10:30 (Adelaide time), it's now trying to calculate between 6:30am and midnight, where it really needs to be finding ones created between 6:30am and 1:30pm utc.
Now, for this example in particular I could probably hack something together to work out what the 'midnight' time needs to be given the time zone difference. But the between <given time> and <midnight in Adelaide> calculation isn't going to work if that period spans midnight utc. So that solution is bust.
UPDATE:
I think I've managed to get the result I want by trial and error, but I'm not sure I understand exactly what's going on.
scope :after_time, ->(time) {
time = time.strftime('%H:%M:%S')
where_clause = <<-SQL
(events.date_time at time zone 'UTC' at time zone 'ACDT')::time
between ? and '23:59:59'
SQL
joins(:performances).where(where_clause, time)
}
It's basically turning everything into the one time zone so the query for each row ends up looking something like WHERE '20:30:00' between '17:00:00' and '23:59:59', so I'm not having to worry about times spanning over midnight.
Even still, I feel like there's probably a proper way to do this, so I'm open to suggestions.
Check if this works for you,
s = DateTime.now.change(hour: 6, min: 30).utc
e = Date.today.end_of_day.utc
Event.where("date_time::time between ?::time and ?::time", s, e)
this may help you and then you need not to convert every date of DB, instead you can convert the parameterized timestamp into UTC time:
scope :after, ->(start_time) { where('created_at::time > :time', time: start_time.utc.strftime('%H:%M:%S')) }
Now,
for e.g. I do have 3 events for following timestamps(all in UTC):
2013-04-11 11:43:43
2013-04-11 15:10:40
2013-04-12 07:39:26
and then you can call:
start_time = Time.zone.parse('2016-01-01 20:00:00')
# => Fri, 01 Jan 2016 20:00:00 ACDT +10:30
Event.after(start_time) # this will return 2 events(1, 2)
query will be:
SELECT "events".* FROM "events" WHERE (created_at::time > '09:30:00')
Note: This will raise an error ActiveRecord::StatementInvalid: PG::AmbiguousColumn: ERROR: column reference "created_at" is ambiguous if you will use this query with any another model that will have created_at column
I have a scenario in which i get a timestamp and i need to search for all bookings for that date in that timestamp. The timestamp is in users respective timezone and all the records in the database are stored in UTC. so naturally i need to convert that timestamp back to UTC and then search.
Here's something that i'm doing:
Booking.where("date_time >= '#{DateTime.parse(timestamp).in_time_zone('UTC').beginning_of_day}' and date_time <= '#{DateTime.parse(timestamp).in_time_zone('UTC').end_of_day}'")
which basically means to fetch all bookings from the beginning of day till the end
However, when i use the following query it gives me a different result:
Booking.where("date_time >= '#{DateTime.parse(timestamp).beginning_of_day.in_time_zone('UTC')}' and date_time <= '#{DateTime.parse(timestamp).end_of_day.in_time_zone('UTC')}'")
I'm wondering which one is actually the correct statement to use in my use case and i would appreciate some input here.
I wouldn't use either one.
This one:
DateTime.parse(timestamp).in_time_zone('UTC').beginning_of_day
gives you the beginning of the UTC day, not the beginning of the local-time-zone-day offset to UTC. In short, it is incorrect and won't give you what you're looking for.
This one:
DateTime.parse(timestamp).beginning_of_day.in_time_zone('UTC')
is correct as it changes the time to the beginning of the day in the local time zone and then converts the timestamp to UTC.
If you let ActiveRecord deal with the quoting using a placeholder, then it will apply the UTC adjustment itself.
I'd also use < t.tomorrow.beginning_of_day rather than <= t.end_of_day to avoid timestamp truncation and precision issues; the end of the day is considered to be at 23:59:59.999... and that could leave a little tiny window for errors to creep in. I'm being pretty pedantic here, you might not care about this.
I'd probably do it more like this:
t = DateTime.parse(timestamp)
Booking.where('date_time >= :start and date_time < :end',
:start => t.beginning_of_day,
:end => t.tomorrow.beginning_of_day
)
Sorry if that question sounds strange, but I'm diving into Rails and I'm still learning the jargon. Basically, I'm trying to create a single-pass query that uses the value of one of the model's attributes in a calculation in the query (assuming that's even possible).
I have a Tournament model that has a start_date attribute that is a DateTime object. I'm trying to create a query that returns all the Tournaments that have a start_date no older than 1 hour + the length of the tournament, or put another way, all tournaments that haven't yet started or have started, but haven't ended longer than an hour ago. My current query, which doesn't work, looks like this...
validTourneys = Tournament.where("start_date > (? - duration_in_mins)", (DateTime.now.utc - 1.hour))
where duration_in_mins is an integer attribute of the Tournament model, but this query doesn't work and it seems to be returning all the Tournaments all the time. I'd like to include duration_in_mins in the (DateTime.now.utc - 1.hour) part of the calculation, but I don't know how to reference it, which is why I included it in the string part of the query, hoping that would work. Am I at least on the right track?
I should mention I'm using SQLite for development and PostgreSQL for production.
Thanks for your wisdom!
The problem is that if you subtract minutes from a DateTime object, you are not subtracting minutes but days.
# This works as expected
dt = DateTime.now # Thu, 28 Apr 2011 09:55:14 +0900
an_hour_ago = dt - 1.hour # Thu, 28 Apr 2011 08:55:14 +0900
# But, this does not...
two_hours_in_minutes = 120
two_hours_ago = dt - two_hours_in_minutes # Wed, 29 Dec 2010 09:55:14 +0900
In the last example 120 days are subtracted instead of minutes. This is probably also happening in your query. You have to convert duration_in_minutes to days and then subtract.
I don't know enough about SQL to answer your question directly (I think this will probably also depend on what database you're using, so you might want to mention that).
Have you considered, though, having start_date and end_date as DateTime columns instead of start_date and duration_in_mins? If this is going to be a common query, that would certainly make it more performant, as well as making your code easier to read and understand.
This query will only work if your database is smart enough to know how to add (what I am assuming) is a DateTime and and integer. And I can't think of a database that will do that correctly the way you have it coded. No database will assume minutes. Some might do ticks, seconds, or days.
This part of the calculation
(? - duration_in_mins)
is going to happen on the database, not in Ruby-land.
I need to grab the records for same day of the week for the preceeding X days of the week. There must be a better way to do it than this:
Transaction.find_by_sql "select * from transactions where EXTRACT(DOW from date) = 1 and organisation_id = 4 order by date desc limit 7"
It gets me what I need but is Postgres specific and not very "Rails-y". Date is a timestamp.
Anyone got suggestions?
How many days do you want to go back?
I have written a gem called by_star that has a dynamic finder suited for finding up to a certain number of days in the past. If the number of days was always a number you could use this finder:
Transaction.as_of_3_days_ago
If it was dynamic then I would recommend using something such as future or between, depending on if you have transactions in the future (i.e. time travel):
Transaction.future(params[:start_date].to_time)
Transaction.between(params[:start_date].to_time, Time.now)
AFAIK Rails has no any methods to do this by other way. So best, and faster, solution - build DOW index on date column and use your query.