I have a model where the data is stored in json format in a jsonb column in postgres.
I want to sort the output by a data field using an activerecord query.
Model.all.order("json_data -> 'date'")
gives me an output but orders it alphabetically based on the date string.
Is there an easy way I can sort this as a date?
Note: The dates are in the following format:
"Fri, 24 Jun 2016 04:13:26 -0700"
If the date is in a sensible format Postgres will deal with this automatically.
Model.all.order("(json_data ->> 'date')::timestamp with time zone DESC")
or
Model.all.order("(json_data ->> 'date')::timestamptz DESC")
If your date field string is a little unorthodox, you can do the following
Model.all.order("to_timestamp(json_data->>'date','Dy, DD Mon YYYY HH24:MI:SS ') DESC")
Details here
Note the ->> there to output the string rather than the json object.
You can of course just create an extra column and store your information there as per #Uzbekjon's answer below.
Is there an easy way I can sort this as a date?
Not as part of the jsonb field, since JSON doesn't know anything about dates.
So, an easy alternative would be to store them as a separate table column.
If you really have to store them as an element of your json field, then I would suggest one of the two options:
Store your field as timestamp. Since, json fields in postgresql support numeric values and it could (potentially) optimize the sorting.
Store your date in ISO 8601 format string. It would sort correctly, even if it is a string.
Related
I'm building an iOS app where I want to retrieve all the values from my database between two dates that the user picks. So for example, I want all the rows from the 1st of March to the 5th of March. Would look something like
SELECT * FROM MAIN WHERE DATE = '01/03/2020' AND ENDS ='05/03/2020'
So from that I would hope to retrieve all data from the 1st,2nd,3rd,4th and 5th of march. Any ideas on how to do this?
Thank you
Try to use comparison operators like:
DATE >= '01/03/2020' AND DATE <= '05/03/2020'
There are two issues:
Date types:
As Datatypes In SQLite Version 3 says:
2.2. Date and Time Datatype
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
So storing dates in a dd/MM/yyyy format (using the DateFormatter capitalization convention) is problematic because in the absence of a native date type, it’s going to store them as strings, and therefore all comparisons will be done alphabetically, not chronologically, sorting values like 03/10/2009 (or nonsense strings like 02foobar, for that matter) in between the strings 01/05/2020 and 05/05/2020.
If, however you store them as yyyy-MM-dd, then it just so happens that alphabetical comparisons will yield chronologically correct comparisons, too.
SQL syntax:
Once you have your dates in your database in a format that is comparable, then if you have all of your dates in a single column, you can use the BETWEEN syntax. For example, let’s say you stored all of your dates in yyyy-MM-dd format, then you could do things like:
SELECT * FROM main WHERE date BETWEEN '2020-03-01' AND '2020-03-05';
But needless to say, you can’t use this pattern (or any comparison operators other than equality) as long as your dates are stored in dd/MM/yyyy format.
If you want to show all the data that has values of column "date" between this two dates then:
Select *
from MAIN
where `date` between '01.03.2020' and '05.03.2020';
If you want to show all the data that has values of column "ends" between this two dates then:
Select *
from MAIN
where ends between '01.03.2020' and '05.03.2020';
If you want to show all the data that has values of columns "date" and "ends" between this two dates then:
Select *
from MAIN
where ends between '01.03.2020' and '05.03.2020'
and `date` between '01.03.2020' and '05.03.2020';
Here is a demo
I'm using Rails. I've stored a count by month in a postgres db as a hash using hstore.
The stored hash is formatted as follows:
{"2017-03-01 00:00:00 UTC"=>"10", "2017-04-01 00:00:00 UTC"=>"3"}
I'm struggling to find a great way to retrieve specific month counts from this hash due to the date format used for the key.
QUESTION
What is the best way to format a string to match the current hash key date format?
For example for March in the Hash the key is "2017-03-01 00:00:00 UTC"
However, a new DateTime for March 1 2017 is formatted as "2017-03-01T00:00:00+00:00"
Or is it best to change the format of how I am storing the hash in the first place?
If you need a timestamp in a specific format, the standard tool to use is DateTime#strftime (all the time-ish classes will have a strftime method and they all behave the same). In your case:
some_datetime.utc.strftime('%Y-%m-%d %H:%M:%S %Z')
And hooking that up to ActiveRecord:
Model.where('your_hstore -> :key', :key => some_datetime.utc.strftime('%Y-%m-%d %H:%M:%S %Z'))
Or:
Model.where('your_hstore -> ?', some_datetime.utc.strftime('%Y-%m-%d %H:%M:%S %Z'))
%Z should be the "Time zone abbreviation name" and for me it produces strings like 'UTC', 'PDT', ... If your strftime (which almost certainly is just a wrapper around the system's libc version of strftime) doesn't produce the strings that you want then you have some options:
Drop the timezone completely if it will always be UTC. Then the keys would look like 2017-03-01 00:00:00 and you'd use '%Y-%m-%d %H:%M:%S' as your strftime format string.
If they keys are actually just dates as they appear to be, then use dates and drop the time-of-day. Then your keys would look like 2017-03-01, you'd use Date instances in Ruby rather than DateTimes, and you'd say some_date.strftime('%Y-%m-%d') or some_date.iso8601 in Ruby to get your hstore keys.
If you are using non-UTC timezones, then convert everything to UTC and go with 1 or 2.
If you don't want any of the above, switch to numeric timezone offsets (2017-05-10 18:05:57 +0000, 2017-05-10 18:06:48 +00:00, ...) and use %z, %:z, or %::z in the strftime format string (see the docs for difference between these three).
These of course require reworking any data you already have in the database but it is best to get the out of the way sooner rather than later.
I have a model with :birth_date of type date.
I've tried to put a string like 3 janvier 1968 (French language) into that field and somehow in database I saw that PostgreSQL or someone else converted it into a date!
I also tried some other dates like 3 février 1968 or like 3 fevrier 1968 which didn't work and turned out to be NULL in db.
I can't find information about this feature anywhere. How does this work?
Rails knows that attribute is a Date from the database definition, so it converts the string you give it to a Date. If you create a new instance of your model in the Rails console and assign to birth_date, you can show that it's already a Date even before you save it to the database:
m = Model.new # Use your model name
m.birth_date = "3 février 1968"
m.birth_date.class
The console should tell you that m.birth_date is a Date.
So the conversion to Date is done before you save the model to the database. Rails defines a String::to_date method that calls the Ruby ::Date.parse method, which converts various human-readable date strings into a Date (https://ruby-doc.org/stdlib-2.3.1/libdoc/date/rdoc/Date.html#method-c-parse). In the Rails source, you'll see that whatever you assign to a Date attribute is converted to a Date with the to_date method. So when you assign a String, it happens via String::to_date which calls Date.parse.
As you mentioned in your comment, Date.parse seems to take a fairly loose approach to the months when they're spelled out. I tried a variety of English, French, and Spanish months in Date.parse, and as long as the first three letters of the non-English month are the same as the English month, Date.parse will convert them. But if the first three letters are different, then Date.parse throws an error.
if you have a column in the database as type 'date', it will only save as a date. Rails does it's best to convert a string into a recognized date if possible. You should always pass the 'birth_date' data as a date (i.e. use a date_field). Otherwise, if you REALLY want to store it as a string, the birth_date column must be of type string in the database
I am using Delphi 7, and Titan BTrieve to open a Pervasive Table.
It is a TtbTable component.
I am trying to apply the filter on a TimeStamp field with my code as follows:
Date:=InputDate;
DateString:=FormatDateTime('DD/MM/YYYY HH:NN:SS', InputDate);
Table1.Filter:='UPDATEDON > '+chr(39)+DAteString+chr(39);
Table1.Filtered:=True;
The problem is that the filter results are incorrect. It returns records that are before the do not match the filter criteria.
the Table1.Filter, filters the data in TDataSet not in btrieve/pervasive.
the problem is the date in string format... you must use the formar YYYY-MM-DD and not DD-MM-YYYY because the string compare.
in a string compare 17-06-2012 is grater than 16-07-2012, (17>16)
From the look of it, you are comparing strings, not dates.
What version of PSQL are you using? Is the UPDATEDON field defined as a Timestamp in the Btrieve database? If it's a timestamp, values are stored in 8-byte unsigned values representing septaseconds (10^-7 second) since January 1, 0001 in a Gregorian calendar, Coordinated Universal Time (UTC). It is not stored as a string.
Btrieve / PSQL stores dates in the 'YYYY-MM-DD' format.
WHat does a value from the UPDATEDON field look like? To use it as a filter, you need to make sure the filter value looks the same.
I am a parsing a file into a sqlite database that contains dates in the YYYY-MM-DD format. I want to store the entries into sqlite in such a way that I can sort the entries by date (strings not cutting it). What is the normal protocol for storing and ordering dates in sqlite? Should convert the dates into a number. Is there a way to convert YYYY-MM-DD dates into timestamps?
SQLite supports "DATE" in table creation. (More about that later.)
CREATE TABLE test (dt DATE PRIMARY KEY);
INSERT INTO "test" VALUES('2012-01-01');
INSERT INTO "test" VALUES('2012-01-02');
INSERT INTO "test" VALUES('2012-01-03');
SELECT dt FROM test ORDER BY dt;
2012-01-01
2012-01-02
2012-01-03
Values in the form yyyy-mm-dd sort correctly as either a string or a date. That's one reason yyyy-mm-dd is an international standard.
But SQLite doesn't use data types in the way most database workers expect it. Data storage is based on storage classes instead. For example, SQLite allows this.
INSERT INTO test VALUES ('Oh, bugger.');
SELECT * FROM test ORDER BY dt;
2012-01-01
2012-01-02
2012-01-03
Oh, bugger.
It also allows different date "formats" (actually, values) in a single column. Its behavior is quite unlike standard SQL engines.
INSERT INTO test VALUES ('01/02/2012');
SELECT * FROM test ORDER BY dt;
01/02/2012
2012-01-01
2012-01-02
2012-01-03
Oh, bugger.
You don't have to do anything special to store a timestamp in a date column. (Although I'd rather see you declare the column as timestamp, myself.)
INSERT INTO test VALUES ('2012-01-01 11:00:00');
SELECT * FROM test ORDER BY dt;
2012-01-01
2012-01-01 11:00:00
2012-01-02
2012-01-03
Oh, bugger.
SQLite will try to do the Right Thing as long as you feed consistent data into it. And it will sort dates correctly if you use the standard format.
Instead of storing date in format "YYYY-MM-DD", store the time-stamp of that date and that will help you to sorting the table.
If You want to Current TimeStamp then use
SELECT strftime('%s','now');
If You want toYYYY-MM-DD date TimeStamp then use
SELECT strftime('%s','YYYY-MM-DD');
where %s=seconds since 1970-01-01
i have the date field store in this way DD/MM/YYYY.
For sorting the date ( date field is a string ) i have to convert it before order it.
select (substr(date, 7, 4) || '-' || substr(date, 4, 2) || '-' || substr(date, 1, 2)) as new_date from work_hour order by new_date desc