I'm trying to find a database agnostic way of comparing dates with active record queries. I've the following query:
UserRole.where("(effective_end_date - effective_start_date) > ?", 900.seconds)
This works fine on MySQL but produces an error on PG as the sql it generates doesn't contain the 'interval' syntax. From the console:
←[1m←[36mUserRole Load (2.0ms)←[0m ←[1mSELECT "user_roles".* FROM "user_roles" WHERE "user_roles"."effective_end_date" IS NULL AND ((effective_end_d
ate - effective_start_date) > '--- 900
...
')←[0m
ActiveRecord::StatementInvalid: PG::Error: ERROR: invalid input syntax for type interval: "--- 900
When I run this with the to_sql I option I get:
irb(main):001:0> UserRole.where("effective_end_date - effective_start_date) > ?", 900.seconds).to_sql
=> "SELECT \"user_roles\".* FROM \"user_roles\" WHERE \"user_roles\".\"effective_end_date\" IS NULL AND (effective_end_date - effective_start_date) >
'--- 900\n...\n')"
All help appreciated.
If your effective_end_date and effective_start_date columns really are dates then your query is pointless because dates have a minimum resolution of one day and 900s is quite a bit smaller than 86400s (AKA 25*60*60 or 1 day). So I'll assume that your "date" columns are actually datetime (AKA timestamp) columns; if this is true then you might want to rename the columns to avoid confusion during maintenance, effectively_starts_at and effectively_ends_at would probably be good matches for the usual Rails conventions. If this assumption is invalid then you should change your column types or stop using 900s.
Back to the real problem. ActiveRecord converts Ruby values to SQL values using the ActiveRecord::ConnectionAdapters::Quoting#quote method:
def quote(value, column = nil)
# records are quoted as their primary key
return value.quoted_id if value.respond_to?(:quoted_id)
case value
#...
else
"'#{quote_string(YAML.dump(value))}'"
end
end
So if you try to use something as a value for a placeholder and there isn't any specific handling built in for that type, then you get YAML (a bizarre choice of defaults IMO). Also, 900.seconds is an ActiveSupport::Duration object (despite what 900.seconds.class says) and the case value has no branch for ActiveSupport::Duration so 900.seconds will get YAMLified.
The PostgreSQL adapter provides its own quote in ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#quote but that doesn't know about ActiveSupport::Duration either. The MySQL adapter's quote is also ignorant of ActiveSupport::Duration. You could monkey patch some sense into these quote methods. Something like this in an initializer:
class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter
# Grab an alias for the standard quote method
alias :std_quote :quote
# Bludgeon some sense into things
def quote(value, column = nil)
return "interval '#{value.to_i} seconds'" if(value.is_a?(ActiveSupport::Duration))
std_quote(value, column)
end
end
With that patch in place, you get intervals that PostgreSQL understands when you use an ActiveSupport::Duration:
> Model.where('a - b > ?', 900.seconds).to_sql
=> "SELECT \"models\".* FROM \"models\" WHERE (a - b > interval '900 seconds')"
> Model.where('a - b > ?', 11.days).to_sql
=> "SELECT \"models\".* FROM \"models\" WHERE (a - b > interval '950400 seconds')"
If you add a similar patch to the MySQL adapter's quote (which is left as an exercise for the reader), then things like:
UserRole.where("(effective_end_date - effective_start_date) > ?", 900.seconds)
will do The Right Thing in both PostgreSQL and MySQL and your code won't have to worry about it.
That said, developing and deploying on different databases is a really bad idea that will make Santa Claus cry and go looking for some coal (possibly laced with arsenic, possibly radioactive) for your stocking. So don't do that.
If on the other hand you're trying to build database-agnostic software, then you're in for some happy fun times! Database portability is largely a myth and database-agnostic software always means writing your own portability layer on top of the ORM and database interfaces that your platform provides. You will have to exhaustively test everything on each database you plan to support, everyone pays lip service to the SQL Standard but no one seems to fully support it and everyone has their own extensions and quirks to worry about. You will end up writing your own portability layer that will consist of a mixture of utility methods and monkey patches.
Related
I have Order model in which I have datetime column start and int columns arriving_dur, drop_off_dur, etc.. which are durations in seconds from start
Then in my model I have
class Order < ApplicationRecord
def finish_time
self.start + self.arriving_duration + self.drop_off_duration
end
# other def something_time ... end
end
I want to be able to do this:
Order.where(finish_time: Time.now..(Time.now+2.hours) )
But of course I can't, because there's no such column finish_time. How can I achieve such result?
I've read 4 possible solutions on SA:
eager load all orders and select it with filter - that would not work well if there were more orders
have parametrized scope for each time I need but that means soo much code duplication
have sql function for each time and bind it to model with select() - it's just pain
somehow use http://api.rubyonrails.org/classes/ActiveRecord/Attributes/ClassMethods.html#method-i-attribute ? But I have no idea how to use it for my case or whether it even solves the problem I have.
Do you have any idea or some 'best practice' how to solve this?
Thanks!
You have different options to implement this behaviour.
Add an additional finish_time column and update it whenever you update/create your time values. This could be done in rails (with either before_validation or after_save callbacks) or as psql triggers.
class Order < ApplicationRecord
before_validation :update_finish_time
private
def update_finish_time
self.finish_time = start_time + arriving_duration.seconds + drop_off_duration.seconds
end
end
This is especially useful when you need finish_time in many places throughout your app. It has the downside that you need to manage that column with extra code and it stores data you actually already have. The upside is that you can easily create an index on that column should you ever have many orders and need to search on it.
An option could be to implement the finish-time update as a postgresql trigger instead of in rails. This has the benefit of being independent from your rails application (e.g. when other sources/scripts access your db too) but has the downside of splitting your business logic into many places (ruby code, postgres code).
Your second option is adding a virtual column just for your query.
def orders_within_the_next_2_hours
finishing_orders = Order.select("*, (start_time + (arriving_duration + drop_off_duration) * interval '1 second') AS finish_time")
Order.from("(#{finishing_orders.to_sql}) AS orders").where(finish_time: Time.now..(Time.now+2.hours) )
end
The code above creates the SQL query for finishing_order which is the order table with the additional finish_time column. In the second line we use that finishing_orders SQL as the FROM clause ("cleverly" aliased to orders so rails is happy). This way we can query finish_time as if it was a normal column.
The SQL is written for relatively old postgresql versions (I guess it works for 9.3+). If you use make_interval instead of multiplying with interval '1 second' the SQL might be a little more readable (but needs newer postgresql version, 9.4+ I think).
I do a lot of time without date querying in my app, and I would like to abstract some of the queries away.
So say I have a model with a DateTime starts_at field:
Shift.where('starts_at::time > ?', '20:31:00.00')
-> SELECT "shifts".* FROM "shifts" WHERE (starts_at::time > '20:31:00.00')
This correctly returns all of the 'starts_at' values greater than the time 20:31.
I want to dynamically pass in the column name into the query, so I can do something like:
Shift.where('? > ?', "#{column_name}::time", '20:31:00.00').
-> SELECT "shifts".* FROM "shifts" WHERE ('starts_at::time' > '20:31:00.00')
In this example, this does not work as the search executes starts_at::time as a string, not as a column with the time cast.
How can I safely pass in column_name into a query with the ::time cast? While this will not accept user input, I would still like to ensure SQL injection is accounted for.
This is more complicated than you might think at first because identifiers (column names, table names, ...) and values ('pancakes', 6, ...) are very different things in SQL that have different quoting rules and even quote characters (single quotes for strings, double quotes for identifiers in standard SQL, backticks for identifiers in MySQL, brackets for identifiers in SQL-Server, ...). If you think of identifiers like Ruby variable names and values like, well, literal Ruby values then you can start to see the difference.
When you say this:
where('? > ?', ...)
both placeholders will be treated as values (not identifiers) and quoted as such. Why is this? ActiveRecord has no way of knowing which ? should be an identifier (such as the created_at column name) and which should be a value (such as 20:31:00.00).
The database connection does have a method specifically for quoting column names though:
> puts ActiveRecord::Base.connection.quote_column_name('pancakes')
"pancakes"
=> nil
so you can say things like:
quoted_column = Shift.connection.quote_column_name(column_name)
Shift.where("#{quoted_name}::time > ?", '20:31:00.00')
This is a little unpleasant because we recoil (or at least we should) at using string interpolation to build SQL. However, quote_column_name will take care of anything dodgy or unsafe in column_name so this isn't actually dangerous.
You could also say:
quoted_column = "#{Shift.connection.quote_column_name(column_name)}::time"
Shift.where("#{quoted_name} > ?", '20:31:00.00')
if you didn't always need to cast the column name to a time. Or even:
clause = "#{Shift.connection.quote_column_name(column_name)}::time > ?"
Shift.where(clause, '20:31:00.00')
You could also use extract or one of the other date/time functions instead of a typecast but you'd still be left with the quoting problem and the somewhat cringeworthy quote_column_name call.
Another option would be to whitelist column_name so that only specific valid values would be allowed. Then you could throw the safe column_name right into the query:
if(!in_the_whitelist(column_name))
# Throw a tantrum, hissy fit, or complain in your preferred fashion
end
Shift.where("#{column_name} > ?", '20:31:00.00')
This should be fine as long as you don't have any funky column names like "gotta have some breakfast" or similar things that always need to be properly quoted. You could even use Shift.column_names or Shift.columns to build your whitelist.
Using both a whitelist and then quote_column_name would probably be the safest but the quote_column_name method should be sufficient.
I decided to use this small solution, taking advantage of Rails column naming conventions:
scope :field_before_and_on_date, -> (field, time) do
column_name = field.to_s.parameterize.underscore
where("#{column_name} <= ?", time.end_of_day)
end
# Takes advantage of:
> "); and delete everything(); stuff(".parameterize.underscore
=> "and_delete_everything_stuff"
It's limited but the concept would work for a type cast too.
One problem I often run into in Rails is this:
Let's say I have an invoices table with a date and a days column.
How can I retrieve all invoices which are due?
class Invoice < ActiveRecord::Base
def self.due
where("due_date > ?", Date.today) # this doesn't work because there is no database column "due_date"
end
private
def due_date
date + days
end
end
Can anybody tell me how to do this without having to add a database column due_date to my invoices table?
Thanks for any help.
In PostgreSQL, adding an integer to a date adds that many days:
date '2001-09-28' + integer '7' = date '2001-10-05'
so you can simply say:
where('due_date + days > :today', :today => Date.today)
However, SQLite doesn't really have a date type at all, it stores dates as ISO 8601 strings. That means that adding a number to a date will end up concatenating the strings and that's sort of useless. SQLite does have a date function though:
date(timestring, modifier, modifier, ...)
[...]
All five date and time functions take a time string as an argument. The time string is followed by zero or more modifiers.
so you can say things like date('2014-01-22', '+ 11 days') to do your date arithmetic. That leaves you with this:
where("date(due_date, '+' || days || ' days') > :today", :today => Date.today)
Thankfully, ISO 8601 date strings compare properly as strings so > still works.
Now you're stuck with two versions of the same simple query. You could check what sort of thing self.connection is to differentiate between dev/SQLite and production/PostgreSQL or you could look at Rails.env.production?. This of course leaves a hole in your test suite.
I think you should stop developing on top of SQLite if you intend on deploying on top of PostgreSQL and you should do that right now to minimize the pain and suffering. The truth is that any non-trivial application will be wedded to the database you use in production or you will have to expend significant effort (including running your test suite against all the different databases you use) to maintain database portability. Database independence is a nice idea in theory but wholly impractical unless someone is prepared to cover the non-trivial costs (in time and treasure) that such independence requires. ORMs won't protect you from the differences between databases unless your application is yet another "15 minute blog" toy.
You could do something like:
class Invoice < ActiveRecord::Base
def self.due
Invoice.all.select { |invoice| invoice.due_date > Date.today }
end
private
def due_date
date + days
end
end
can anyone please tell me how to write query in select_value.
I have tried,
ActiveRecord::Base.connection.select_value("select count(*) from leave_details where status= 'Pending' and 'employeedetails_id'=25")
but it showing error
invalid input syntax for integer: "employeedetails_id".
I am using PostgreSQL.
Single quotes are used to quote strings in PostgreSQL (and every other SQL database that even pretends to respect the SQL standard) so you're saying something like this:
some_string = some_integer
when you do this:
'employeedetails_id'=25
and that doesn't make any sense: you can't compare strings and integers without an explicit type cast. You don't need to quote that identifier at all:
ActiveRecord::Base.connection.select_value(%q{
select count(*)
from leave_details
where status = 'Pending'
and employeedetails_id = 25
})
If you even do need to quote an identifier (perhaps it is case sensitive or contains spaces), then you'd use double quotes with PostgreSQL.
Apparently you created your column as "EmployeeDetails_id" so that it is case sensitive. That means that you always have to use that case and you always have to double quote it:
ActiveRecord::Base.connection.select_value(%q{
select count(*)
from leave_details
where status = 'Pending'
and "EmployeeDetails_id" = 25
})
I'd recommend reworking your table to not use mixed case identifiers:
They go against standard Ruby/Rails naming.
They force you to double quote the mixed case column names everywhere you use them.
They go against standard PostgreSQL practice.
This is going to trip you up over and over again.
Executing SQL directly isn't really The Rails Way, and you lose any database portability by doing it that way.
You should create a model for leave_details. E.g.
rails g model LeaveDetails status:string employeedetails_id:integer
Then, the code would be:
LeaveDetails.where({ :status => 'Pending', :employeedetails_id => 25 }).count
Dealing with a legacy database, I've come across a column in a SQL Server database where the date is stored as a decimal. E.g. 2011-04-23 is stored as 20110423.0.
Is there a general ActiveRecord mechanism for dealing with "weird" column storage conventions? Enum-like columns where they're actually stored as integers is another case that might also make use of the same mechanism, but I can't quite find what I'm looking for.
It seems like serialization gets me partly there:
class Thing < ActiveRecord::Base
class DecimalDate
def load(date)
if date.is_a? Numeric
y,m,d = /^(\d\d\d\d)(\d\d)(\d\d)/.match(date.to_s)[1..3].map(&:to_i)
Date.civil(y,m,d)
end
end
def dump(date)
date ? date.strftime('%Y%m%d').to_i : 0
end
end
serialize :weird_date, DecimalDate
end
rails c
> Thing.first.weird_date
=> Sun, 02 Jan 2011
But, the illusion is thin. The column doesn't "know" that it's a date stored as a decimal. E.g. comparisons fail:
rails c
> Thing.where('weird_date > ?', 1.week.ago)
...
ActiveRecord::StatementInvalid: ... Error converting data type varchar to numeric.:
If your are forced to deal with legacy data I see two possibilities to manage it.
1 From your database
You can "convert" your data by making a view of your table which convert your (date) fields on the fly. Then you make a trigger (before insert/update) on this view which convert your data back to your old format. Finally you tell ActiveRecord to use your view instead of your table.
2 From your application (Rails)
Find a way to tell ActiveRecord to do the same job. Have you already tried to manage it with AR callbacks with after_initialize and before_save? More informations here