I have run with a problem which i believe is Active Records fault. I am parsing an XML file which contains jobs. This xml file contains nodes which indicate walltime in the time format 00:00:00. I also have a model which will accept these jobs. However, when the time is larger than an actual 24H time, Active record inserts it as NULL. Examples below:
INSERT INTO `jobs` (`jobid`, `walltime`) VALUES('71413', 'NULL')
INSERT INTO `jobs` (`jobid`, `walltime`) VALUES('71413', '15:24:10')
Any ideas? Thank you!
The standard SQL time and datetime data types aren't intended to store a duration. Probably in agreement with those standards, ActiveRecord's time attribute assignment logic uses the time parsing rules of the native Ruby Time class to reject invalid time of day.
The way to store durations, as you intend, is either:
Store the duration as an integer (e.g. "number of seconds"), or
Store two (date)times, a start and an end, and use date arithmetic on them.
class Thing < ActiveRecord::Base
...
def duration
return start - end
end
def duration=(length)
start = Time.now
end = start + length
end
...
end
Related
I have Order model in which I have datetime column start and int columns arriving_dur, drop_off_dur, etc.. which are durations in seconds from start
Then in my model I have
class Order < ApplicationRecord
def finish_time
self.start + self.arriving_duration + self.drop_off_duration
end
# other def something_time ... end
end
I want to be able to do this:
Order.where(finish_time: Time.now..(Time.now+2.hours) )
But of course I can't, because there's no such column finish_time. How can I achieve such result?
I've read 4 possible solutions on SA:
eager load all orders and select it with filter - that would not work well if there were more orders
have parametrized scope for each time I need but that means soo much code duplication
have sql function for each time and bind it to model with select() - it's just pain
somehow use http://api.rubyonrails.org/classes/ActiveRecord/Attributes/ClassMethods.html#method-i-attribute ? But I have no idea how to use it for my case or whether it even solves the problem I have.
Do you have any idea or some 'best practice' how to solve this?
Thanks!
You have different options to implement this behaviour.
Add an additional finish_time column and update it whenever you update/create your time values. This could be done in rails (with either before_validation or after_save callbacks) or as psql triggers.
class Order < ApplicationRecord
before_validation :update_finish_time
private
def update_finish_time
self.finish_time = start_time + arriving_duration.seconds + drop_off_duration.seconds
end
end
This is especially useful when you need finish_time in many places throughout your app. It has the downside that you need to manage that column with extra code and it stores data you actually already have. The upside is that you can easily create an index on that column should you ever have many orders and need to search on it.
An option could be to implement the finish-time update as a postgresql trigger instead of in rails. This has the benefit of being independent from your rails application (e.g. when other sources/scripts access your db too) but has the downside of splitting your business logic into many places (ruby code, postgres code).
Your second option is adding a virtual column just for your query.
def orders_within_the_next_2_hours
finishing_orders = Order.select("*, (start_time + (arriving_duration + drop_off_duration) * interval '1 second') AS finish_time")
Order.from("(#{finishing_orders.to_sql}) AS orders").where(finish_time: Time.now..(Time.now+2.hours) )
end
The code above creates the SQL query for finishing_order which is the order table with the additional finish_time column. In the second line we use that finishing_orders SQL as the FROM clause ("cleverly" aliased to orders so rails is happy). This way we can query finish_time as if it was a normal column.
The SQL is written for relatively old postgresql versions (I guess it works for 9.3+). If you use make_interval instead of multiplying with interval '1 second' the SQL might be a little more readable (but needs newer postgresql version, 9.4+ I think).
Does Rails really not properly support PostgreSQL's interval data type?
I had to use this Stack Overflow answer from 2013 to create an interval column, and now it looks like I'll need to use this piece of code from 2013 to get ActiveRecord to treat the interval as something other than a string.
Is that how it is? Am I better off just using an integer data type to represent the number of minutes instead?
From Rails 5.1, you can use postgres 'Interval' Data Type, so you can do things like this in a migration:
add_column :your_table, :new_column, :interval, default: "2 weeks"
Although ActiveRecord only treat interval as string, but if you set the IntervalStyle to iso_8601 in your postgresql database, it will display the interval in iso8601 style: 2 weeks => P14D
execute "ALTER DATABASE your_database SET IntervalStyle = 'iso_8601'"
You can then directly parse the column to a ActiveSupport::Duration
In your model.rb
def new_column
ActiveSupport::Duration.parse self[:new_column]
end
More infomation of ISO8601 intervals can be find at https://en.wikipedia.org/wiki/ISO_8601#Time_intervals
I had a similar issue and went with defining reader method for the particular column on the ActiveRecord model. Like this:
class DivingResults < ActiveRecord::Base
# This overrides the same method for db column, generated by rails
def underwater_duration
interval_from_db = super
time_parts = interval_from_db.split(':')
if time_parts.size > 1 # Handle formats like 17:04:41.478432
units = %i(hours minutes seconds)
in_seconds = time_parts
.map.with_index { |t,i| t.to_i.public_send(units[i]) }
.reduce(&:+) # Turn each part to seconds and then sum
ActiveSupport::Duration.build in_seconds
else # Handle formats in seconds
ActiveSupport::Duration.build(interval_from_db.to_i)
end
end
end
This allows to use ActiveSupport::Duration instance elsewhere. Hopefully Rails will start handling the PostgreSQL interval data type automatically in near future.
A more complete and integrated solution is available in Rails 6.1
The current answers suggest overriding readers and writers in the models. I took the alter database suggestion and built a gem for ISO8601 intervals, ar_interval.
It provides a simple ActiveRecord::Type that deals with the serialization and casting of ISO8601 strings for you!
The tests include examples for how to use it.
If there is interest, the additional formats Sam Soffes demonstrates could be included in the tests
Similar to Madis' solution, this one handles fractions of a second and ISO8601 durations:
def duration
return nil unless (value = super)
# Handle ISO8601 duration
return ActiveSupport::Duration.parse(value) if value.start_with?('P')
time_parts = value.split(':')
if time_parts.size > 1
# Handle formats like 17:04:41.478432
units = %i[hours minutes seconds]
in_seconds = time_parts.map.with_index { |t, i| t.to_f.public_send(units[i]) }.reduce(&:+)
ActiveSupport::Duration.build in_seconds
else
# Handle formats in seconds
ActiveSupport::Duration.build(value)
end
end
def duration=(value)
unless value.is_a?(String)
value = ActiveSupport::Duration.build(value).iso8601
end
self[:duration] = value
end
This assumes you setup your database like Leo mentions in his answer. No idea why sometimes they come back from Postgres in the PT42S format and sometimes in the 00:00:42.000 format :/
One problem I often run into in Rails is this:
Let's say I have an invoices table with a date and a days column.
How can I retrieve all invoices which are due?
class Invoice < ActiveRecord::Base
def self.due
where("due_date > ?", Date.today) # this doesn't work because there is no database column "due_date"
end
private
def due_date
date + days
end
end
Can anybody tell me how to do this without having to add a database column due_date to my invoices table?
Thanks for any help.
In PostgreSQL, adding an integer to a date adds that many days:
date '2001-09-28' + integer '7' = date '2001-10-05'
so you can simply say:
where('due_date + days > :today', :today => Date.today)
However, SQLite doesn't really have a date type at all, it stores dates as ISO 8601 strings. That means that adding a number to a date will end up concatenating the strings and that's sort of useless. SQLite does have a date function though:
date(timestring, modifier, modifier, ...)
[...]
All five date and time functions take a time string as an argument. The time string is followed by zero or more modifiers.
so you can say things like date('2014-01-22', '+ 11 days') to do your date arithmetic. That leaves you with this:
where("date(due_date, '+' || days || ' days') > :today", :today => Date.today)
Thankfully, ISO 8601 date strings compare properly as strings so > still works.
Now you're stuck with two versions of the same simple query. You could check what sort of thing self.connection is to differentiate between dev/SQLite and production/PostgreSQL or you could look at Rails.env.production?. This of course leaves a hole in your test suite.
I think you should stop developing on top of SQLite if you intend on deploying on top of PostgreSQL and you should do that right now to minimize the pain and suffering. The truth is that any non-trivial application will be wedded to the database you use in production or you will have to expend significant effort (including running your test suite against all the different databases you use) to maintain database portability. Database independence is a nice idea in theory but wholly impractical unless someone is prepared to cover the non-trivial costs (in time and treasure) that such independence requires. ORMs won't protect you from the differences between databases unless your application is yet another "15 minute blog" toy.
You could do something like:
class Invoice < ActiveRecord::Base
def self.due
Invoice.all.select { |invoice| invoice.due_date > Date.today }
end
private
def due_date
date + days
end
end
I'm having an issue with Rails 4's support for Postgresql's ts_range data type. Here is the code that I am trying to persist:
before_validation :set_appointment
attr_accessor :starting_tsrange, :ending_tsrange
def set_appointment
self.appointment = convert_to_utc(starting_tsrange)...convert_to_utc(ending_tsrange)
end
def convert_to_utc
ActiveSupport::TimeZone.new("America/New_York").parse(time_string).utc
end
Basically I set an instance variable for the beginning and end of the appointment ts_range with two strings representing date_times. Before validation it converts them to utc and saves those values to the appointment attribute which should then be persisted. It sets things correctly but when I try to retrieve the record, the appointment attribute is now nil. Why is this code not working as expected?
Figured out the subtle bug in the code. The issue here is with the triple dot range operator. If we get two values that are the exact same time. The triple dot will say include everything from time a up until time b if they are the same exact time, then nothing will be included and the result will be nil. This can be visualized with the code below
(1...1).to_a # []
(1..1).to_a # [1]
So the way to fix this is to not use the triple dot notation when using ranges that can have the same value for a time. Use the double dot notation instead.
I have the below code snippet:
class Foo
include DataMapper::Resource
property :id, Serial
property :timestamp, DateTime
end
I just want to convert the current time to ms:
class Time
def to_ms
(self.to_f * 1000.0).to_i
end
end
def current_time
time = Time.now
return time.to_ms
end
time = current_time # => 1352633569151
but when I am going to save the Foo with above timestamp, then it can't be saved to the database and I'm not getting any error message.
foo = Foo.new
foo.timestamp = time
foo.save
Any idea?
are you using a correct format for your :datetime property?
should be like:
DateTime.now.to_s
=> "2012-11-11T14:04:02+02:00"
or a "native" DateTime object, without any conversions.
DataMapper will carry to convert it to according values based on adapter used.
also, to have exceptions raised when saving items:
DataMapper::Model.raise_on_save_failure = true
that's a global setting, i.e. all models will raise exceptions.
to make only some model to behave like this:
YourModel.raise_on_save_failure = true
http://datamapper.org/docs/create_and_destroy.html
See "Raising an exception when save fails" chapter
btw, to see what's wrong with your item before saving it, use and item.valid? and item.errors
foo = Foo.new
foo.timestamp = time
if foo.valid?
foo.save
else
p foo.errors
end
I replicated your code and got following error:
#errors={:timestamp=>["Timestamp must be of type DateTime"]}
See live demo here
The PostgreSQL data types would be timestamp or timestamp with time zone. But that contradicts what you are doing. You take the epoch value and multiply by 1000. You'd have to save that as integer or some numeric type.
More about the handling of timestamps in Postgres in this related answer.
I would save the value as timestamp with time zone as is (no multiplication). You can always extract ms out of it if need should be.
If you need to translate the Unix epoch value back to a timestamp, use:
SELECT to_timestamp(1352633569.151);
--> timestamptz 2012-11-11 12:32:49.151+01
Just save "now"
If you actually want to save "now", i.e. the current point in time, then let Postgres do it for you. Just make sure the database server has a reliable local time - install ntp. This is generally more reliable, accurate and simple.
Set the DEFAULT of the timestamp column to now() or CURRENT_TIMESTAMP.
If you want timestamp instead of timestamptz you can still use now(), which is translated to "local" time according to the servers timezone setting. Or, to get the time for a given time zone:
now() AT ZIME ZONE 'Europe/Vienna' -- your time zone here
Or, in your particular case, since you seem to want only three fractional digits: now()::timestamp(3) or CURRENT_TIMESTAMP(3) or CURRENT_TIMESTAMP(3) AT ZIME ZONE 'Europe/Vienna'.
Or, if you define the type of the column as timestamp(3), all timestamp values are coerced to the type and rounded to 3 fractional decimal digits automatically.
So this would be all you need:
CREATE TABLE tbl (
-- other columns
,ts_column timestamp(3) DEFAULT now()
);
The value is set automatically on INSERT, you don't even have to mention the column.
If you want to update it ON UPDATE, add a TRIGGER like this:
Trigger function:
CREATE OR REPLACE FUNCTION trg_up_ts()
RETURNS trigger AS
$BODY$
BEGIN
NEW.ts_column := now();
RETURN NEW;
END
$BODY$ LANGUAGE plpgsql VOLATILE
Trigger:
CREATE TRIGGER log_up_ts
BEFORE UPDATE ON tbl
FOR EACH ROW EXECUTE PROCEDURE trg_up_ts();
Now, everything works automatically.
If that's not what you are after, #slivu's answer seems to cover the Ruby side just nicely.
I'm not familiar with PostgreSQL, but why are you assigning a Fixnum (time) to timestamp (which is a DateTime)? Your model must be failing to convert time to a proper DateTime value before generating the SQL.
Try foo.save!. I'm pretty sure you'll see an error, either reported from PostgreSQL, saying 1352633569151 is not a valid value for the table column, or your model will say it can't parse 1352633569151 to a valid DateTime.
foo.timestamp = Time.now or foo.timestamp = '2012-11-11 00:00:00' is something that'll work.