Consider the following case. I have a Voucher model with a datetime activation_due_date field and user model that has up-to-date information about his location (timezone, UTC offset).
I want to check if he requests voucher activation before due date in any of available time zones. For instance, If a due date is set to 28.08.2018 23:59 UTC
I want my scope before_activation_due to check if he requests something before 28.08.2018 - 23:59 in his current time zone so my due date is not something fixed - it depends on users location - In one place it can be after due date and in the other before.
I have tried the following approach.
models/voucher.rb
scope :before_activation_due, lambda { |user|
where('activation_due_date > ? ', Time.current.to_utc + user.utc_offset)
}
My questions are:
Is this a right approach? If not, what is the proper way for dealing with such cases?
How to test such a scope? The current timestamp is probably taken from a database server when comparing datetimes during query execution so I am not sure how to mock it in my specs.
Thanks in advance.
You can store just the timezone of the user, not the offset, then do:
where('activation_due_date > ? ', Time.now.utc.in_time_zone(user.timezone))
where timezone is any valid timezone shown in
rake time:zones
That'd be the more rails-y way to do things at least. But I don't think storing an offset then manually adding it to the time is a bad approach.
To test this, you can manually insert any date you want in to your database. Then you can use a gem like https://github.com/travisjeffery/timecop to travel in time to that point, and test your scope:
Voucher.create(activation_due_date: '2018-01-02 00:00:00')
format = '%Y-%m-%d %H:%M:%S %Z'
time = DateTime.strptime("2018-01-02 00:00:00 Central Time (US & Canada)",format)
Timecop.travel(time)
Voucher.before_activation_due.all ...
One approach is to convert the activation_due_date into the timezone of the user. As you say "my due date is not something fixed - it depends on users location".
To do this as a scope the easiest thing would be to use your databases timezone functions. This depends on which database you are using, but in PostgreSQL it will be something like:-
where('activation_due_date AT TIME ZONE ? > NOW() ', user.timezone)
An even simpler way would be to do a string comparison
where('to_char(activation_due_date, 'YYYYMMDDHH24MISS') > ?', Time.current.in_time_zone(user.timezone).strftime('%Y%m%d%H%M%S');
In this case we are saying what is the time on the wall for the user, and is it less than time in the database (which is "stored" in UTC).
In general, best practice when dealing with dates is to store them in UTC and convert back to whatever the user expects within the application layer.
That doesn't necessarily work with future dates, particularly where the date/time is specific to the user in their timezone. A schedule based on local time requires storing a local time.
In my instance,there’s one attribute that’s a timestamp containing the start_time of a future event, compared to everything else that's now or in the past (including the created_at and updated_at timestamps).
Why
This particular field is the timestamp of a future event where the user selects the time.
For future events, it seems best practice is not to store UTC.
Instead of saving the time in UTC along with the time zone, developers can save what the user expects us to save: the wall time.
When the user chooses 10am, it needs to stay 10am even when the user’s offset from UTC changes between creation and the event date due to daylight savings.
So, in June 2016, if a user creates an event for 1st Jan 2017 at midnight in Sydney, that timestamp will be stored in the database as 2017-01-01 00:00. The offset at time of creation would be +10:00, but at the time of the event, it’d be +11:00.. unless government decides to change that in the meantime.
Like wise, I’d expect a separate event that I create for 1 Jan 2016 at midnight in Brisbane to also be stored as 2017-01-01 00:00. I store the timezone i.e. Australia/Brisbane in a separate field.
What’s a best practice way to do this in Rails?
I’ve tried lots of options with no success:
1. Skip conversion
Problem, this only skips conversion on read, not writing.
self.skip_time_zone_conversion_for_attributes = [:start_time]
2. Change the whole app configuration to use config.default_timestamp :local
To do this, I set:
config/application.rb
config.active_record.default_timezone = :local
config.time_zone = 'UTC'
app/model/event.rb
...
self.skip_time_zone_conversion_for_attributes = [:start_time]
before_save :set_timezone_to_location
after_save :set_timezone_to_default
def set_timezone_to_location
Time.zone = location.timezone
end
def set_timezone_to_default
Time.zone = 'UTC'
end
...
To be frank, I’m not sure what this is doing.. but not what I want.
I thought it was working as my Brisbane event was stored as 2017-01-01 00:00 but when I created a new event for Sydney, it was stored as 2017-01-01 01:00even though it displays as midnight correctly in the view.
That being the case, I’m concerned that still have the same problem with the Sydney event that I’m trying to avoid.
3. Override the getter and setter for the model to store as integer
I’ve tried to also store the event start_time as an integer in the database.
I tried doing this by monkey patching the Time class and adding a before_validates callback to do the conversion.
config/initializers/time.rb
class Time
def time_to_i
self.strftime('%Y%m%d%H%M').to_i
end
end
app/model/event.rb
before_validation :change_start_time_to_integer
def change_start_time_to_integer
start_time = start_time.to_time if start_time.is_a? String
start_time = start_time.time_to_i
end
# read value from DB
# TODO: this freaks out with an error currently
def start_time
#take integer YYYYMMDDHHMM and convert it to timestamp
st = self[:start_time]
Time.new(
st / 100000000,
st / 1000000 % 100,
st / 10000 % 100,
st / 100 % 100,
st % 100,
0,
offset(true)
)
end
Ideal Solution
I’d like to be able to store a timestamp in its natural datatype in the database so queries don’t get messy in my controllers, but I can’t figure out how to store “wall time” that doesn’t convert.
Second best, I’d settle for the integer option if I have to.
How do others deal with this? What am I missing? Particularly with the "integer conversion" option above, I'm making things far more complicated than they need to be.
I propose that you still use the first option but with a little hack: in essence, you can switch off the time zone conversion for the desired attribute and use a custom setter to overcome the conversion during attribute writes.
The trick saves the time as a fake UTC time. Although technically it has an UTC zone (as all the times are saved in db in UTC) but by definition it shall be interpreted as local time, regardless of the current time zone.
class Model < ActiveRecord::Base
self.skip_time_zone_conversion_for_attributes = [:start_time]
def start_time=(time)
write_attribute(:start_time, time ? time + time.utc_offset : nil)
end
end
Let's test this in rails console:
$ rails c
>> future_time = Time.local(2020,03,30,11,55,00)
=> 2020-03-30 11:55:00 +0200
>> Model.create(start_time: future_time)
D, [2016-03-15T00:01:09.112887 #28379] DEBUG -- : (0.1ms) BEGIN
D, [2016-03-15T00:01:09.114785 #28379] DEBUG -- : SQL (1.4ms) INSERT INTO `models` (`start_time`) VALUES ('2020-03-30 11:55:00')
D, [2016-03-15T00:01:09.117749 #28379] DEBUG -- : (2.7ms) COMMIT
=> #<Model id: 6, start_time: "2020-03-30 13:55:00">
Note that Rails saved the time as a 11:55, in a "fake" UTC zone.
Also note that the time in the object returned from create is wrong because the zone is converted from the "UTC" in this case. You would have to count with that and reload the object every time after setting the start_time attribute, so that the zone conversion skipping can take place:
>> m = Model.create(start_time: future_time).reload
D, [2016-03-15T00:08:54.129926 #28589] DEBUG -- : (0.2ms) BEGIN
D, [2016-03-15T00:08:54.131189 #28589] DEBUG -- : SQL (0.7ms) INSERT INTO `models` (`start_time`) VALUES ('2020-03-30 11:55:00')
D, [2016-03-15T00:08:54.134002 #28589] DEBUG -- : (2.5ms) COMMIT
D, [2016-03-15T00:08:54.141720 #28589] DEBUG -- : Model Load (0.3ms) SELECT `models`.* FROM `models` WHERE `models`.`id` = 10 LIMIT 1
=> #<Model id: 10, start_time: "2020-03-30 11:55:00">
>> m.start_time
=> 2020-03-30 11:55:00 UTC
After loading the object, the start_time attribute is correct and can be manually interpreted as local time regardless of the actual time zone.
I really don't get it why Rails behaves the way it does regarding the skip_time_zone_conversion_for_attributes configuration option...
Update: adding a reader
We can also add a reader so that we automatically interpret the saved "fake" UTC time in local time, without shifting the time due to timezone change:
class Model < ActiveRecord::Base
# interprets time stored in UTC as local time without shifting time
# due to time zone change
def start_time
t = read_attribute(:start_time)
t ? Time.local(t.year, t.month, t.day, t.hour, t.min, t.sec) : nil
end
end
Test in rails console:
>> m = Model.create(start_time: future_time).reload
D, [2016-03-15T08:10:54.889871 #28589] DEBUG -- : (0.1ms) BEGIN
D, [2016-03-15T08:10:54.890848 #28589] DEBUG -- : SQL (0.4ms) INSERT INTO `models` (`start_time`) VALUES ('2020-03-30 11:55:00')
D, [2016-03-15T08:10:54.894413 #28589] DEBUG -- : (3.1ms) COMMIT
D, [2016-03-15T08:10:54.895531 #28589] DEBUG -- : Model Load (0.3ms) SELECT `models`.* FROM `models` WHERE `models`.`id` = 12 LIMIT 1
=> #<Model id: 12, start_time: "2020-03-30 11:55:00">
>> m.start_time
=> 2020-03-30 11:55:00 +0200
I.e. the start_time is correctly interpreted in local time, even though it was stored as the same hour and minute, but in UTC.
This may sound a bit out there, but I have dealt with similar issues with a recent application I was tasked with - but on the opposite side - when I run an ETL to load data for the application, dates from the source are stored in EST. Rails believes that it is UTC when serving the data, so for that, I converted the dates back to UTC using P/SQL. I did not want these dates to be different than the other date fields within the app.
Option A
In this case, could you capture the user timezone at creation, and send that back as a hidden field in the form? I am still learning RoR, so am not sure on the "proper" way to do this, but right now I would do something like this:
Example (I tested this, and it will submit the offset (minutes) in a hidden field):
<div class="field">
<%= f.hidden_field :offset %>
</div>
<script>
utcDiff = new Date().getTimezoneOffset();
dst_field = document.getElementById('timepage_offset');
dst_field.value = utcDiff;
</script>
If you then send utcDiff along with the user selected date, you could calculate the UTC date before storing. I suppose you could add that to the model as well if that data is necessary to know at a later date.
I think that no matter how this is done, there will always be slight area for confusion, unless the user is capable of providing the proper information, which leads me to...
Option B:
You could, instead of a hidden field, provide a select list (and to be friendly, default it to the users' local offset), to allow them to provide the zone for which their date is specified in.
Update - TimeZone select
I've done some research, and it looks like there is already a form helper for a time zone select box.
I vote the simplest route and that's saving to UTC. Create a column for country of origin, set up a google alert for "daylight savings time", and if Chile decides to stop using daylight savings or alter it in some crazy way, you can adapt by querying your database for Chilean users and adjusting their dates accordingly with a script. Then, you can use Time.parse with the date, time and timezone. To illustrate, here are the results the day before daylight savings and after daylight savings on 3/13/2016:
Time.parse("2016-03-12 12:00:00 Pacific Time (US & Canada)").utc
=> 2016-03-12 20:00:00 UTC
Time.parse("2016-03-14 12:00:00 Pacific Time (US & Canada)").utc
=> 2016-03-14 19:00:00 UTC
This will get you a list of the accepted time zone names:
timezones = ActiveSupport::TimeZone.zones_map.values.collect{|tz| tz.name}
I want to use MorrisJs-Rails to build up a graphical statistic of user registered within the last 4 weeks.
For that I iterate through each day of the last month and check for users registered at this specific date. Here's the problem though.
admin_helper.rb:
def users_chart_date
(4.weeks.ago.to_date..Date.today).map do |date|
{
created_at: date,
count: User.where("created_at = ? ", date).count
}
end
end
Both dates are stored like so:
puts User.first.created_at # => Tue, 10 Mar 2015 20:08:18 UTC +00:00
puts date # => Thu, 23 Jul 2015
You can see there's quite the difference between those two stored dates thus I can not possibly get an actual count of users registered per day.
Is there a way to "cast" my SQL query somehow? I feel the solution to my problem is easy peasy, but I can't seem to get it.
Thanks for your help,
let me know if you need something.
As you've noticed, the created_at timestamp is of type DateTime (or Time), so you're correct in that its not directly comparable with a Date type. You could compare it with a range of times for each day:
(4.weeks.ago.to_date..Date.today).map do |date|
{
created_at: date,
count: User.where(created_at: date.to_time.all_day).count
}
end
(Note that this will use the times for your computer's timezone.)
However, this method can become very slow especially if the time range becomes longer (e.g., over the entire life of your app), in which case a SQL GROUP BY statement work much faster. Again, you need to deal with a comparison by dates on a time data field, which is database-dependent. With MySQL or PostgreSQL you can:
User.group("DATE(created_at)").count
If you want to restrict the time periods you get data for:
User.where("created_at >= ?", 4.weeks.ago.beginning_of_day).group("DATE(created_at)").count
These queries will return a hash of Date objects to the count of users with created_at times on that day.
I have a scenario in which i get a timestamp and i need to search for all bookings for that date in that timestamp. The timestamp is in users respective timezone and all the records in the database are stored in UTC. so naturally i need to convert that timestamp back to UTC and then search.
Here's something that i'm doing:
Booking.where("date_time >= '#{DateTime.parse(timestamp).in_time_zone('UTC').beginning_of_day}' and date_time <= '#{DateTime.parse(timestamp).in_time_zone('UTC').end_of_day}'")
which basically means to fetch all bookings from the beginning of day till the end
However, when i use the following query it gives me a different result:
Booking.where("date_time >= '#{DateTime.parse(timestamp).beginning_of_day.in_time_zone('UTC')}' and date_time <= '#{DateTime.parse(timestamp).end_of_day.in_time_zone('UTC')}'")
I'm wondering which one is actually the correct statement to use in my use case and i would appreciate some input here.
I wouldn't use either one.
This one:
DateTime.parse(timestamp).in_time_zone('UTC').beginning_of_day
gives you the beginning of the UTC day, not the beginning of the local-time-zone-day offset to UTC. In short, it is incorrect and won't give you what you're looking for.
This one:
DateTime.parse(timestamp).beginning_of_day.in_time_zone('UTC')
is correct as it changes the time to the beginning of the day in the local time zone and then converts the timestamp to UTC.
If you let ActiveRecord deal with the quoting using a placeholder, then it will apply the UTC adjustment itself.
I'd also use < t.tomorrow.beginning_of_day rather than <= t.end_of_day to avoid timestamp truncation and precision issues; the end of the day is considered to be at 23:59:59.999... and that could leave a little tiny window for errors to creep in. I'm being pretty pedantic here, you might not care about this.
I'd probably do it more like this:
t = DateTime.parse(timestamp)
Booking.where('date_time >= :start and date_time < :end',
:start => t.beginning_of_day,
:end => t.tomorrow.beginning_of_day
)
I have a time field in table "timestamp without time zone". When record is saved to database, the utc time might be a different day compared to the local time. However, I need to group the records by date. Hence, I am doing something like this:
result = transmissions.joins(:report).where('reports.time::timestamp::date = ?', record.time.to_date)
The problem is if the utc date is on a different date than local time, then that record is not included in result. Any ideas how to get the right result?
And apparently I cannot change the "without time zone" either:
Rails database-specific data type
It says:
"concluded that the default ActiveRecord datetime and timestamp column types in schema migrations cannot be modified to force PostgreSQL to use timestamp with time zone."
So I have no idea how to group by date, as obviously something like this is wrong:
Unit.where(id: 1100).first.reports.order("DATE(time)").group("DATE(time)").count
=> {"2013-12-14"=>19, "2013-12-15"=>5}
That return value is completely wrong. All 25 records should be on 2013-12-14 and 0 records on 2013-12-15.
Assuming your records are timestamped with a particular UTC offset, you can try passing in the start and end times of the date in question in UTC format to your query:
result = transmissions.joins(:report).where('reports.time >= ? AND reports.time < ?', record.time.midnight.utc, (record.time.midnight + 1.day).utc)
Explanation:
midnight is a Rails method on an instance of Time that returns the Time object that represents midnight on the date of the original Time object. Similarly, record.time.midnight + 1.day returns the Time object representing midnight of the following day. Then, converting both Time objects – which are presumably timestamped in a standard UTC offset – to UTC creates a time period representing midnight-to-midnight for the system timezone in UTC format (not midnight in UTC time), which is precisely what you're seeking to query.
How about something like result = transmissions.joins(:report).where('reports.time >= ? AND reports.time <= ?', record.time.beginning_of_day.utc, record.time.end_of_day.utc)
The .utc part may not be necessary.