Dealing with a legacy database, I've come across a column in a SQL Server database where the date is stored as a decimal. E.g. 2011-04-23 is stored as 20110423.0.
Is there a general ActiveRecord mechanism for dealing with "weird" column storage conventions? Enum-like columns where they're actually stored as integers is another case that might also make use of the same mechanism, but I can't quite find what I'm looking for.
It seems like serialization gets me partly there:
class Thing < ActiveRecord::Base
class DecimalDate
def load(date)
if date.is_a? Numeric
y,m,d = /^(\d\d\d\d)(\d\d)(\d\d)/.match(date.to_s)[1..3].map(&:to_i)
Date.civil(y,m,d)
end
end
def dump(date)
date ? date.strftime('%Y%m%d').to_i : 0
end
end
serialize :weird_date, DecimalDate
end
rails c
> Thing.first.weird_date
=> Sun, 02 Jan 2011
But, the illusion is thin. The column doesn't "know" that it's a date stored as a decimal. E.g. comparisons fail:
rails c
> Thing.where('weird_date > ?', 1.week.ago)
...
ActiveRecord::StatementInvalid: ... Error converting data type varchar to numeric.:
If your are forced to deal with legacy data I see two possibilities to manage it.
1 From your database
You can "convert" your data by making a view of your table which convert your (date) fields on the fly. Then you make a trigger (before insert/update) on this view which convert your data back to your old format. Finally you tell ActiveRecord to use your view instead of your table.
2 From your application (Rails)
Find a way to tell ActiveRecord to do the same job. Have you already tried to manage it with AR callbacks with after_initialize and before_save? More informations here
Related
I have Order model in which I have datetime column start and int columns arriving_dur, drop_off_dur, etc.. which are durations in seconds from start
Then in my model I have
class Order < ApplicationRecord
def finish_time
self.start + self.arriving_duration + self.drop_off_duration
end
# other def something_time ... end
end
I want to be able to do this:
Order.where(finish_time: Time.now..(Time.now+2.hours) )
But of course I can't, because there's no such column finish_time. How can I achieve such result?
I've read 4 possible solutions on SA:
eager load all orders and select it with filter - that would not work well if there were more orders
have parametrized scope for each time I need but that means soo much code duplication
have sql function for each time and bind it to model with select() - it's just pain
somehow use http://api.rubyonrails.org/classes/ActiveRecord/Attributes/ClassMethods.html#method-i-attribute ? But I have no idea how to use it for my case or whether it even solves the problem I have.
Do you have any idea or some 'best practice' how to solve this?
Thanks!
You have different options to implement this behaviour.
Add an additional finish_time column and update it whenever you update/create your time values. This could be done in rails (with either before_validation or after_save callbacks) or as psql triggers.
class Order < ApplicationRecord
before_validation :update_finish_time
private
def update_finish_time
self.finish_time = start_time + arriving_duration.seconds + drop_off_duration.seconds
end
end
This is especially useful when you need finish_time in many places throughout your app. It has the downside that you need to manage that column with extra code and it stores data you actually already have. The upside is that you can easily create an index on that column should you ever have many orders and need to search on it.
An option could be to implement the finish-time update as a postgresql trigger instead of in rails. This has the benefit of being independent from your rails application (e.g. when other sources/scripts access your db too) but has the downside of splitting your business logic into many places (ruby code, postgres code).
Your second option is adding a virtual column just for your query.
def orders_within_the_next_2_hours
finishing_orders = Order.select("*, (start_time + (arriving_duration + drop_off_duration) * interval '1 second') AS finish_time")
Order.from("(#{finishing_orders.to_sql}) AS orders").where(finish_time: Time.now..(Time.now+2.hours) )
end
The code above creates the SQL query for finishing_order which is the order table with the additional finish_time column. In the second line we use that finishing_orders SQL as the FROM clause ("cleverly" aliased to orders so rails is happy). This way we can query finish_time as if it was a normal column.
The SQL is written for relatively old postgresql versions (I guess it works for 9.3+). If you use make_interval instead of multiplying with interval '1 second' the SQL might be a little more readable (but needs newer postgresql version, 9.4+ I think).
My controller creates something called start_time. When I print start_time's value before it's added to a LittleClassSession hash, here's what I get:
22:45:00
Okay, it looks like a value with the type time. After it's added to the hash, I ask the controller what the :start_time value is.
#little_class_session = LittleClassSession.new({
...
:start_time => start_time
})
puts #little_class_session.start_time
Here's what it puts:
2000-01-01 22:45:00 UTC
It appears to be formatted like a datetime, but asking what the .class of the start_time attribute is returns:
Time
The LittleClassSession start_time column is a time in the table (I can verify this by checking the type in the Rails console) but was a datetime when the model was created.
What could be causing this?
While your database may support a "time" column (meaning just a time with no date information), Rails by default does not (largely because neither does Ruby's standard library -- even a Time contains date information). As such, when you assign it to your model, Rails is coercing it into the type it knows how to deal with, DateTime. So, you have a few options:
Ignore the date part of the time when you use it.
Use a gem like tod to deal with your time-only types, and follow the guidelines in the README for hooking it up to Rails.
Store start_time_hour and start_time_minutes in two separate columns, and work with them as needed (e.g, Date.current + start_time_hour.hours + start_time_minutes.minutes).
Hope that helps!
Ruby 2.1.5
Rails 4.1
I've inherited some code with where some of the columns have spaces in them. How do I access these columns from a view. Example:
Table: expenses
Columns: Jan 2015
Feb 2015
In expenses_controller.rb, I have
#epxenses = Expense
In views/expenses/index.html.erb, the following would not work:
#expenses.each do |e|
e.jan 2010
Any ideas?
You can access it using [] or attributes[]:
expense = Expense.last
expense['Jan 2015']
expense.attributes['Jan 2015']
Rails/Active record should not generate migrations with spaces in the field names, or if you specifically tell it to I would be curious as to why because it will likely be inconvenient going forward. If you are hooking up to a database that was created outside of AR, maybe you would benefit from aliasing the field names in your data models.
alias_attribute :new_column_name, :column_name_in_db
One problem I often run into in Rails is this:
Let's say I have an invoices table with a date and a days column.
How can I retrieve all invoices which are due?
class Invoice < ActiveRecord::Base
def self.due
where("due_date > ?", Date.today) # this doesn't work because there is no database column "due_date"
end
private
def due_date
date + days
end
end
Can anybody tell me how to do this without having to add a database column due_date to my invoices table?
Thanks for any help.
In PostgreSQL, adding an integer to a date adds that many days:
date '2001-09-28' + integer '7' = date '2001-10-05'
so you can simply say:
where('due_date + days > :today', :today => Date.today)
However, SQLite doesn't really have a date type at all, it stores dates as ISO 8601 strings. That means that adding a number to a date will end up concatenating the strings and that's sort of useless. SQLite does have a date function though:
date(timestring, modifier, modifier, ...)
[...]
All five date and time functions take a time string as an argument. The time string is followed by zero or more modifiers.
so you can say things like date('2014-01-22', '+ 11 days') to do your date arithmetic. That leaves you with this:
where("date(due_date, '+' || days || ' days') > :today", :today => Date.today)
Thankfully, ISO 8601 date strings compare properly as strings so > still works.
Now you're stuck with two versions of the same simple query. You could check what sort of thing self.connection is to differentiate between dev/SQLite and production/PostgreSQL or you could look at Rails.env.production?. This of course leaves a hole in your test suite.
I think you should stop developing on top of SQLite if you intend on deploying on top of PostgreSQL and you should do that right now to minimize the pain and suffering. The truth is that any non-trivial application will be wedded to the database you use in production or you will have to expend significant effort (including running your test suite against all the different databases you use) to maintain database portability. Database independence is a nice idea in theory but wholly impractical unless someone is prepared to cover the non-trivial costs (in time and treasure) that such independence requires. ORMs won't protect you from the differences between databases unless your application is yet another "15 minute blog" toy.
You could do something like:
class Invoice < ActiveRecord::Base
def self.due
Invoice.all.select { |invoice| invoice.due_date > Date.today }
end
private
def due_date
date + days
end
end
I'm trying to find a database agnostic way of comparing dates with active record queries. I've the following query:
UserRole.where("(effective_end_date - effective_start_date) > ?", 900.seconds)
This works fine on MySQL but produces an error on PG as the sql it generates doesn't contain the 'interval' syntax. From the console:
←[1m←[36mUserRole Load (2.0ms)←[0m ←[1mSELECT "user_roles".* FROM "user_roles" WHERE "user_roles"."effective_end_date" IS NULL AND ((effective_end_d
ate - effective_start_date) > '--- 900
...
')←[0m
ActiveRecord::StatementInvalid: PG::Error: ERROR: invalid input syntax for type interval: "--- 900
When I run this with the to_sql I option I get:
irb(main):001:0> UserRole.where("effective_end_date - effective_start_date) > ?", 900.seconds).to_sql
=> "SELECT \"user_roles\".* FROM \"user_roles\" WHERE \"user_roles\".\"effective_end_date\" IS NULL AND (effective_end_date - effective_start_date) >
'--- 900\n...\n')"
All help appreciated.
If your effective_end_date and effective_start_date columns really are dates then your query is pointless because dates have a minimum resolution of one day and 900s is quite a bit smaller than 86400s (AKA 25*60*60 or 1 day). So I'll assume that your "date" columns are actually datetime (AKA timestamp) columns; if this is true then you might want to rename the columns to avoid confusion during maintenance, effectively_starts_at and effectively_ends_at would probably be good matches for the usual Rails conventions. If this assumption is invalid then you should change your column types or stop using 900s.
Back to the real problem. ActiveRecord converts Ruby values to SQL values using the ActiveRecord::ConnectionAdapters::Quoting#quote method:
def quote(value, column = nil)
# records are quoted as their primary key
return value.quoted_id if value.respond_to?(:quoted_id)
case value
#...
else
"'#{quote_string(YAML.dump(value))}'"
end
end
So if you try to use something as a value for a placeholder and there isn't any specific handling built in for that type, then you get YAML (a bizarre choice of defaults IMO). Also, 900.seconds is an ActiveSupport::Duration object (despite what 900.seconds.class says) and the case value has no branch for ActiveSupport::Duration so 900.seconds will get YAMLified.
The PostgreSQL adapter provides its own quote in ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#quote but that doesn't know about ActiveSupport::Duration either. The MySQL adapter's quote is also ignorant of ActiveSupport::Duration. You could monkey patch some sense into these quote methods. Something like this in an initializer:
class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter
# Grab an alias for the standard quote method
alias :std_quote :quote
# Bludgeon some sense into things
def quote(value, column = nil)
return "interval '#{value.to_i} seconds'" if(value.is_a?(ActiveSupport::Duration))
std_quote(value, column)
end
end
With that patch in place, you get intervals that PostgreSQL understands when you use an ActiveSupport::Duration:
> Model.where('a - b > ?', 900.seconds).to_sql
=> "SELECT \"models\".* FROM \"models\" WHERE (a - b > interval '900 seconds')"
> Model.where('a - b > ?', 11.days).to_sql
=> "SELECT \"models\".* FROM \"models\" WHERE (a - b > interval '950400 seconds')"
If you add a similar patch to the MySQL adapter's quote (which is left as an exercise for the reader), then things like:
UserRole.where("(effective_end_date - effective_start_date) > ?", 900.seconds)
will do The Right Thing in both PostgreSQL and MySQL and your code won't have to worry about it.
That said, developing and deploying on different databases is a really bad idea that will make Santa Claus cry and go looking for some coal (possibly laced with arsenic, possibly radioactive) for your stocking. So don't do that.
If on the other hand you're trying to build database-agnostic software, then you're in for some happy fun times! Database portability is largely a myth and database-agnostic software always means writing your own portability layer on top of the ORM and database interfaces that your platform provides. You will have to exhaustively test everything on each database you plan to support, everyone pays lip service to the SQL Standard but no one seems to fully support it and everyone has their own extensions and quirks to worry about. You will end up writing your own portability layer that will consist of a mixture of utility methods and monkey patches.