I'm using SQLIte3 v5.0.2 and TypeORM v0.2.38. I created a date column on a SQLIte table using date DATETIME DEFAULT CURRENT_TIMESTAMP.
When I search the database for rows between dates, it wasn't pickup up rows on the last day so I added a time to my BETWEEN query: WHERE date BETWEEN '2021-01-01 00:00:00' AND '2021-01-31 11:59:59' to include rows on last day.
As soon as I add the time part for the Between query for the date/time, it appears to load the field into entities using the local timezone. Without the time part it doesn't do this but also doesn't find the last days rows. I looked in the database and saw the date times are being stored with my local timezone (-7hrs) instead of UTC as I've seen is meant to be the case on several Github issues for TypeORM.
How can I ensure the dates are saved and loaded only as UTC so they don't change when I query rows in TypeORM? I'm considering writing a custom transformer for the entity field but if its supposed to be UTC by default then is this a bug or am I doing something wrong?
Related
I have two tables both of which have a timestamp field [TIMESTAMP_TZ] and when I perform a join based on this timestamp field the plan in snowflake DB shows an auto conversion on these timestamps into LTZ. Ex
(TO_TIMESTAMP_LTZ(CAG.LOAD_DATE_UTC) = TO_TIMESTAMP_LTZ(PIT.CSAT_AGREEMENT_LDTS))
Any reason why this is happening?
TIMESTAMP_TZ means your timestamp is linked to a time zone and TIMESTAMP_LTZ is your local timezone. Probably the timezones of your two timestamps are different and thus Snowflake converts them automatically to your local timezone to match them correctly.
I have weird situation with my PostgreSQL db date value.
On my web site I have calendar for selecting a date and when I select some future date like "2018-09-23" in PostgreSQL table column it is saved as "2018-09-22 22:00:00"?
Obviously I am missing something. On web site all the time it shows okay time "2018-09-23" but at the table it is minus one day as you see above. Why?
Rails stores DateTime fields in UTC, but without marking their time zone as UTC. This corresponds to the timestamp without time zone type in postgres. So if your time zone is +2, it'll store the time as UTC (+0).
In Rails, Time.zone will return the current local timezone (you can add logic to change this by user, for example). When persisting a datetime, Rails will automatically convert the current Time.zone to UTC. However, it doesn't use the Postgres type that actually includes the time zone data, so it relies on convention to convert back and forth to the user's time zone.
If you really only care about the date, use the date type in your migration instead of Timestamp or DateTime.
Times and dates have a lot of subtle quirks and the "right" behavior depends on your use case. In some applications, you need to deal with "local" time when considering date transitions, and sometimes you need to finesse your application or database logic to think in terms of local time and sometimes you care about UTC time.
I am building a rails app, where the user picks up a date from a date picker and a time from the time picker. Both the date and time have been formatted using moment js to show the date and time in the following way:
moment().format('LL'); //January 23,2017
moment().format('LTS'); //1:17:54 PM
I read this answer with guidelines about selection of a proper column type.
Is there documentation for the Rails column types?
Ideally, I should be using :date, :time or :timestamp for this. But since the dates are formatted, should I be using :string instead?
Which would be the correct and appropriate column type to use in this situation?
If you want to store a time reference in your database you should use one of the types the database offers you. I'll explain this using MySQL (which is the one I have used the most) but the explanation should be similar in other database servers.
If you use a timestamp column you will be using just 4 bytes of storage, which is always a good new since it makes smaller indexes, uses less memory in temporal tables during the internal database operations and so on. However, timestamp has a smaller range than datetime so you will only be able to store values from year 1970 up to year 2038 more or less
If you use datetime you will be able to store a wider range (from year 1001 to year 9999) with the same precision (second). The bad consequence is that a higher range needs more memory, making it a bit slower.
There are some other differences between these two column types that don't fit in this answer, but you should keep an eye on before deciding.
If you use varchar, which is the default column type for text attributes in Ruby on Rails, you will be forced to convert from text to datetime and vice-versa every time you need to use that field. In addition, ordering or filtering on that column will be very inefficient because the database will need to convert all strings into dates before filtering or sorting, making it impossible to use indexes on that column.
If you need sub-second precision, you can use bigint to meet your requirements, as MySQL does not provide a date specific type for this purpose
In general, I recommend using timestamp if your application requirements fit the timestamp limitation. Otherwise, use datetime, but I strongly discourage you to use varchar for this purpose.
EDIT: Formatting
The way you store dates in database is completely different from the way you display it to the user. You can create a DateTime object using DateTime.new(year, month, day, hour, minute, second) and assign that object to your model. By the time you save it into database, ActiveRecord will be in charge of converting the DateTime object into the appropiate database format.
In order to display a value that is already stored in database in a specific format (in a view, API response, etc.) you can hava a look at other posts like this one.
You can have a timestamp column in your database, and then parse the request to a ruby datetime object like this:
d = Time.parse(params[:date])
t = Time.new(params[:time])
dt = DateTime.new(d.year, d.month, d.day, t.hour, t.min, t.sec, t.zone)
#now simply use dt to your datetime column
On Postgres you can save a ruby DateTime object straight into a postgres timestamp field, e.g
User.first.update_attribute('updated_at', dt )
Another option is to concatenate your date and time strings into one and then u can do a one-liner:
User.last.update_attribute('created_at', Time.parse('January 23,2017 1:17:54 PM'))
I'm pretty sure this will work on MySQL datetime or timestamp as well.
Credit to david grayson Ruby: combine Date and Time objects into a DateTime
I have table which have a datetime field named date. When doing a POST in order to insert a new row, the date sent from the client (browser) looks like 2015-11-20T14:30:00+10:00 which is actually a correct date and timezone.
However, inside Postgres this date has been inserted as 2015-11-20 04:30:00.000000, which as you can see, is not at all the same as above. I know the problem is related to the timezone. But I cannot seems to figure out a fix.
For information, I have configured my app timezone :
class Application < Rails::Application
config.time_zone = 'Brisbane'
end
Ideas?
2015-11-20T14:30:00+10:00 means that the local time of 14:30 is 10 hours ahead of UTC. Your database field reflects the correct UTC value of 04:30. This is often the desired behavior, especially if the value represent a timestamp - the date and time something occured (past tense).
In PostgreSQL, there are two different types of timestamp fields (reference)
The TIMESTAMP WITH TIME ZONE field accepts an input that contains a time zone offset. It then converts the value to UTC for storage. On retrieval, it uses the session's timezone setting.
The TIMESTAMP, or TIMESTAMP WITHOUT TIME ZONE simply stores the date and time given, ignoring any offset, and not converting to UTC.
Most of the time, you should indeed use TIMESTAMP WITH TIME ZONE. You should only use TIMESTAMP WITHOUT TIME ZONE if you need to retain the local date and time value, such as in scheduling of future events and calculation of business hours. And for those scenarios, it often makes more sense to split date and time into separate DATE and TIME fields.
One last thing - if you can avoid it, avoid using Rails time zones and use standard tzdb zones. "Australia/Brisbane" is the full tzdb identifier equivalent to the Rails "Brisbane" time zone. Refer to the section on Rails time zones at the bottom of the timezone tag wiki.
I found this gem to be incredibly useful and easy for correctly setting the time https://github.com/kbaum/browser-timezone-rails
Rails stores created_at and updated_at timestamps in UTC time.
But I am using these fields to filter and store (and do a lot of other stuff with the records) based on these fields, so it's important that when I call created_at and updated_at attributes, I get timestamps in my time zone.
As told by the following two SO questions, I can do that by configuring the time zone in the environment.rb file.
However, that's now working for me.
Because I am fetching the records from the database, based on created_at fields (which are in UTC) so naturally wrong ones are coming out, and then I am displaying them, so the dates are displayed in UTC, again, not what I want.
Is there a way to change in what time-zones the rails stores the dates?
OR
Is there a workaround for how I can achieve the database-fetching and displaying, without making a call to Rails object.created_at attribute?
Rails and timezone in created_at
Rails Time zone issue
I wonder why I even asked the question on this forum, because the workaround seemed to be pretty simple. But it did take me eight hours to reach it :)
1) While fetching records from the database and displaying them, I could simply do
select CONVERT_TZ(date) as date from table;
instead of doing
select date from table;
CONVERT_TZ() is a mysql specific function, so it worked for me. Not a universal solution though.
2) Secondly, when I had to fetch the records from a date range given in any other time-zone, I could do one of the two things.
One - I could replace [user-entered-date] 00:00:00 with [user-entered-date] 07:00:00 for the date-range's starting point and [user-entered-date] 59:59:59 with [user-entered-date + 1.day] 07:00:00 for the range's ending point. Note this would require me to use DateTime objects instead of using Date objects.
Two - In the where clause, I could do
CONVERT_TZ(registrations.created_at) <= [user/entered/date]
instead of
(registrations.created_at) <= [user/entered/date]