I'm on Rails 7.0.4 and Postgres 15. I have a trigger that checks if a timestamp(6) with time zone is within a certain date range:
CREATE TABLE public.contracts (
id bigint NOT NULL,
starts_on date NOT NULL,
ends_on date
);
CREATE TABLE public.time_entries (
id bigint NOT NULL,
contract_id bigint,
"from" timestamp(6) with time zone,
"to" timestamp(6) with time zone
);
And this is the trigger:
CREATE FUNCTION time_entries_contract_trigger() RETURNS trigger AS $time_entries_contract_trigger$
BEGIN
IF EXISTS (SELECT 1 FROM contracts WHERE contracts.id = NEW."contract_id" AND NEW."from"::date < starts_on) THEN
RAISE EXCEPTION 'from % is before contracts.starts_on', NEW."from";
END IF;
IF EXISTS (SELECT 1 FROM contracts WHERE contracts.id = NEW."contract_id" AND NEW."to"::date > ends_on) THEN
RAISE EXCEPTION 'to % is after contracts.ends_on', NEW."to";
END IF;
RETURN NEW;
END;
$time_entries_contract_trigger$ LANGUAGE plpgsql;
CREATE TRIGGER time_entries_contract_trigger BEFORE INSERT OR UPDATE ON time_entries
FOR EACH ROW EXECUTE PROCEDURE time_entries_contract_trigger();
And I set the default timezone like this:
module App
class Application < Rails::Application
# ...
config.time_zone = "Europe/Berlin"
# ...
end
The problem now is, that whenever I try to store a TimeEntry, Rails converts it to UTC and I get an error like this:
> contract = Contract.create(:contract, starts_on: Date.parse("2021-12-19"))
> contract.time_entries << build(:time_entry, from: Time.zone.parse("2021-12-19T00:00"))
ActiveRecord::StatementInvalid:
PG::RaiseException: ERROR: from 2021-12-18 23:00:00+00 is before contracts.starts_on
CONTEXT: PL/pgSQL function time_entries_contract_trigger() line 4 at RAISE
It makes sense insofar, that Rails converts Time.zone.parse("2021-12-19T00:00") to UTC and my trigger just casts it to a date with NEW."from"::date, just returning the date part (which now is 2021-12-18 and not 2021-12-19).
My questions now are:
How could I force Rails to store the columns with the timezone information in it? My Postgres columns are already timezone aware, so I guess converting it to UTC doesn't make sense any more? Or are there any drawbacks that I'm missing?
Related
I have a PostGres 9.4 database. I want to change the default column type of a DATETIME column to be the time when the record was created. I thought this was the right way, in as far as this is my rails migration
class ChangeDefaultValueForStratumWorkerSubmissions < ActiveRecord::Migration[5.1]
def change
change_column_default(:stratum_worker_submissions, :created_at, 'NOW')
end
end
but when I look at my database, the default timestamp shows as the time when I ran the migration, instead of the expression I want. How do I write a migration that will do what I want?
Column | Type | Modifiers
-------------------+-----------------------------+----------------------------------------------------------------------------
id | integer | not null default nextval('stratum_worker_submissions_id_seq'::regclass)
stratum_worker_id | integer |
created_at | timestamp without time zone | not null default '2018-04-04 19:46:22.781613'::timestamp without time zone
It isn't well documented but you can supply a lambda as the default value in a migration and that will do The Right Thing. If you say this:
def change
change_column_default :stratum_worker_submissions, :created_at, -> { 'now()' }
end
then the column's default value will be set to now() and the database function now() won't be called until a default value is needed for the column. Then if you \d stratum_worker_submissions in psql you'll see:
created_at | timestamp without time zone | not null default now()
as desired. Any other default will be evaluated when the migration runs and you'll end up with a fixed timestamp as the default.
Alternatively, you can always do it by hand using SQL:
def up
connection.execute(%q(
alter table stratum_worker_submissions
alter column created_at
set default now()
))
end
def down
connection.execute(%q(
alter table stratum_worker_submissions
alter column created_at
drop default
))
end
Note that if you start manually changing the schema with SQL you might start doing things that won't appear in db/schema.rb as you can quickly get into SQL that ActiveRecord doesn't understand. If that happens then you can change from db/schema.rb to db/structure.sql by changing config/application.rb:
config.active_record.schema_format = :sql
and then replacing db/schema.rb with db/structure.sql in revision control and using the db:structure rake tasks in place of the usual db:schema tasks.
I have a PostGres 9.4 database. I want to change the default column type of a DATETIME column to be the time when the record was created. I thought this was the right way, in as far as this is my rails migration
class ChangeDefaultValueForStratumWorkerSubmissions < ActiveRecord::Migration[5.1]
def change
change_column_default(:stratum_worker_submissions, :created_at, 'NOW')
end
end
but when I look at my database, the default timestamp shows as the time when I ran the migration, instead of the expression I want. How do I write a migration that will do what I want?
Column | Type | Modifiers
-------------------+-----------------------------+----------------------------------------------------------------------------
id | integer | not null default nextval('stratum_worker_submissions_id_seq'::regclass)
stratum_worker_id | integer |
created_at | timestamp without time zone | not null default '2018-04-04 19:46:22.781613'::timestamp without time zone
It isn't well documented but you can supply a lambda as the default value in a migration and that will do The Right Thing. If you say this:
def change
change_column_default :stratum_worker_submissions, :created_at, -> { 'now()' }
end
then the column's default value will be set to now() and the database function now() won't be called until a default value is needed for the column. Then if you \d stratum_worker_submissions in psql you'll see:
created_at | timestamp without time zone | not null default now()
as desired. Any other default will be evaluated when the migration runs and you'll end up with a fixed timestamp as the default.
Alternatively, you can always do it by hand using SQL:
def up
connection.execute(%q(
alter table stratum_worker_submissions
alter column created_at
set default now()
))
end
def down
connection.execute(%q(
alter table stratum_worker_submissions
alter column created_at
drop default
))
end
Note that if you start manually changing the schema with SQL you might start doing things that won't appear in db/schema.rb as you can quickly get into SQL that ActiveRecord doesn't understand. If that happens then you can change from db/schema.rb to db/structure.sql by changing config/application.rb:
config.active_record.schema_format = :sql
and then replacing db/schema.rb with db/structure.sql in revision control and using the db:structure rake tasks in place of the usual db:schema tasks.
date_start = Time.parse('11/08/2015').beginning_of_day
date_end = Time.parse('11/08/2015').end_of_day
created_at_day_tz = "date(created_at AT TIME ZONE \'UTC\'
AT TIME ZONE \'#{Time.zone.tzinfo.identifier}\')"
users = User.where("users.created_at BETWEEN ? AND ?", date_start, date_end)
Grouping by created_at as created_at_day (date only, new name for the groupped attribute)
grouped_with_timezone_day = users.group(created_at_day_tz).
order(created_at_day_tz).
select("#{created_at_day_tz} as created_at_day, count(*) as count")
# grouped_with_timezone_day.map {|u| [u.created_at_day, u.count] }
# => [[Tue, 11 Aug 2015, 186]]
Grouping by created_at as created_at (date only, same name for the groupped attribute)
grouped_with_timezone = users.group(created_at_day_tz).
order(created_at_day_tz).
select("#{created_at_day_tz} as created_at, count(*) as count")
# grouped_with_timezone.map {|u| [u.created_at, u.count] }
# => [[Mon, 10 Aug 2015 21:00:00 BRT -03:00, 186]]
Why the results differ if the records are the same? Why one result comes with timezone, as DateTime, and the other comes as Date only?
Is activerecord 'casting' to DateTime with Timezone because created_at is defined that way (btw, this makes the dates incorrect in this case)?
The timestamp isn't incorrect - that is, it's 2015-08-11 at midnight UTC - it's just displaying in your local time.
Rails has a bit of special behavior for created_at and updated_at:
The timestamps macro adds two columns, created_at and updated_at. These special columns are automatically managed by Active Record if they exist.
It always treats created_at coming back from a query as a timestamp. Your query returns just the date 2015-08-11, which is interpreted as midnight. When printed, the timestamp is displayed in your locale's timezone (which I presume must be -03:00), leading to 3 hours before midnight on the 11th.
When you name the result created_at_day, you avoid Rails converting it to a timestamp and get just the date you expect.
How do I use the AT TIME ZONE method while using a joins table?
scope :not_reserved_between, ->(start_at, end_at) { ids = Reservation.joins(:ride).where(":e >= (reservations.start_at AT TIME ZONE rides.time_zone_name)::timestamp AND :s <= (reservations.end_at AT TIME ZONE rides.time_zone_name)::timestamp", s: start_at.utc, e: end_at.utc).pluck(:ride_id); where('rides.id not in (?)', ids) }
In this query I'm taking the reservations.start_at and end_at times and I want to convert them to a different timezone (the time_zone_name in the joined rides table)
Every reservation has a ride and will thus have a time_zone_name. I want to apply that ride's time_zone_name to the reservation's start_at and end_at stamps. If I do:
scope :not_reserved_between, ->(start_at, end_at) { ids = Reservation.joins(:ride).where(":e >= (reservations.start_at AT TIME ZONE rides.time_zone_name)::timestamp AND :s <= (reservations.end_at)::timestamp", s: start_at.utc, e: end_at.utc).pluck(:ride_id); where('rides.id not in (?)', ids) }
This will work for some reason. The only difference between this second query and the first original query is that the second query does not have AT TIME ZONE rides.time_zone_name for reservations.ends_at.
Here is the table schema:
# == Schema Information
#
# Table name: reservations
#
# id :integer not null, primary key
# ride_id :integer
# start_at :datetime
# end_at :datetime
# created_at :datetime
# updated_at :datetime
#
# == Schema Information
#
# Table name: rides
#
# id :integer not null, primary key
# user_id :integer
# latitude :float
# longitude :float
# created_at :datetime
# updated_at :datetime
# utc_offset :integer
# time_zone_name :string(255)
#
All datetimes are stored in UTC zero in PG (PostgreSQL 9.3.5). My goal is to pass 'start_at' and 'end_at' timestamps to my scope and have the correct rides returned. Rides can be stored in different time zones (they have a column for utc_offset and time_zone_name, time_zone_name being 'America/Los_Angeles', etc) and the Reservations that belong to them (ride has_many reservations) have their start and end times stored in utc zero as well.
My goal is to pass a datetime (without timezone) as my 'start_at' and 'end_at' params for the scope. Here is an example: I want to find all rides that do not have reservations at 8am-9am relative to the ride's (the reservation's ride) timezone. This means I can't just search the DB for conflicts with two specific start and end timestamp because that will only find collisions with that exact time. Also this method can give me "false collisions": if I search for two specific start and end timestamps (say 7/10/2015 8am PST - 7/10/2015 9am PST) I may end up having collisions with Reservations that occur at the same time however they occur at a different relative time (the reservation may occur at the same time stamp as 7/10/2015 8am PST - 7/10/2015 9am PST however because the ride is located in EST the Reservation should not be counted as a collision as we are searching for rides available between 8-9am in the local time of the ride). To put it bluntly I am storing everything in UTC zero as a standard however I will be treating some timestamps as "datetimes without timezones" because they need to be converted from a 'relative time'.
scope :not_reserved_between, ->(start_at, end_at) { ids = Reservation.joins(:ride).where(":e >= (reservations.start_at AT TIME ZONE rides.time_zone_name)::timestamp AND :s <= (reservations.end_at AT TIME ZONE rides.time_zone_name)::timestamp", s: start_at.utc, e: end_at.utc).pluck(:ride_id); where('rides.id not in (?)', ids) }
^My thought process from the above code is that I am comparing the scope params with reservation start_at and end_at times. I should be able to take all of the reservation start and end times (stored in UTC zero), convert them to their local time in the ride's time zone (by using 'AT TIME ZONE rides.time_zone_name') and then '::timestamp' the value to receive just the datetime without the time zone. This local time can be compared to the 'relative' UTC zero start/end times I am supplying to the scope.
Rails ActiveRecord converts all datetimes and timestamps – which are synonymous in Rails – to UTC to avoid performing any time zone-dependent logic in the database. This applies to storing as well as querying values. As such, you should be able to define the scope without any time zone logic:
self.not_reserved_between(start_at, end_at)
includes(:reservations).where.not("tsrange(reservations.start_at, reservations.end_at, '[]') && tsrange(?, ?, '[]')", start_at, end_at)
end
Note that I've made use of PostgreSQL's tsrange function for ranges of non-zoned times, and that the '[]' in the last argument indicates that they are inclusive ranges, i.e. that the endpoints are a part of the range. For exclusive ranges, use '()'.
Model
scope :completed_at, select("(
SELECT goals.created_at
FROM goals
WHERE goals.user_id = users.id
ORDER BY goals.created_at
LIMIT 1
) as completed_at
")
Controller
#users = User.select("email, first, last, created_at").completed_at.order("created_at ASC").limit(1).all
Resulting SQL Query
SELECT email, first, last, created_at, (
SELECT goals.created_at
FROM goals
WHERE goals.user_id = users.id
ORDER BY goals.created_at
LIMIT 1
) as completed_at
FROM "users" ORDER BY created_at ASC LIMIT 5
Sample JSON Output
"user": {
"completed_at": "2011-06-07 15:04:56",
"created_at": "2010-01-01T06:00:00Z",
"email": "user#user.com",
"first": "Test",
"last": "User"
}
Note how created_at is coming out as UTC, but "completed_at" is coming out local time? Any idea why or how to get them to be the same format?
Jon, I'm not sure exactly how you're generating the json, but I'd fire up the console and see if I could figure out what type of datetime object you're getting for each column.
Also, are you setting Time.zone?
rails c
> Time.zone = 'Central Time (US & Canada)'
> u = User.select("email, first, last, created_at").completed_at.first
> u.created_at.class
=> ActiveSupport::TimeWithZone
> u.completed_at.class
=> Time # (just a guess)
The module in AR that handles time zone conversion is here.
One thing to note is that AR will not TZ convert any columns not in the columns_hash, i.e. your 'completed_at' column will NOT get TZ converted. I know, this seems backwards from what you're experiencing, but it may give you a clue.