I need to count entries grouped by date (field has TIMESTAMP type in database).
I wrote code
sql = "
SELECT COUNT(id) AS cnt, TO_CHAR(closed_on, 'YYYY-mm-dd') AS closed_on, issues.status_id
FROM issues
WHERE closed_on IS NOT NULL AND closed_on <= '#{end_from_created_date}'
AND created_on <= '#{end_from_created_date}'
GROUP BY closed_on, status_id
"
This query work on postrgesql, but it does not work on sqlite, because sqlite does not have TO_CHAR function
Solution should be compatible with mysql, postgresql, sqlite.
I can use different sql for different DB. But it is not good idea.
May be I can use functions from ActiveRecord, but i do not find solutions in docs
SQLite uses several date formats; PostgreSQL uses similar formats and more.
Both understand YYYY-MM-DD HH:MM:SS.
The solution is
sql = "SELECT COUNT(id) AS cnt, DATE(closed_on) AS closed_on, issues.status_id FROM issues WHERE closed_on IS NOT NULL AND closed_on <= '#{end_from_created_date}' AND created_on <= '#{end_from_created_date}' GROUP BY DATE(closed_on), status_id"
Related
I am getting this error in the pg production mode, but its working fine in sqlite3 development mode.
ActiveRecord::StatementInvalid in ManagementController#index
PG::Error: ERROR: column "estates.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = ...
^
: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = 'Mazzey' GROUP BY user_id
#myestate = Estate.where(:Mgmt => current_user.Company).group(:user_id).all
If user_id is the PRIMARY KEY then you need to upgrade PostgreSQL; newer versions will correctly handle grouping by the primary key.
If user_id is neither unique nor the primary key for the 'estates' relation in question, then this query doesn't make much sense, since PostgreSQL has no way to know which value to return for each column of estates where multiple rows share the same user_id. You must use an aggregate function that expresses what you want, like min, max, avg, string_agg, array_agg, etc or add the column(s) of interest to the GROUP BY.
Alternately you can rephrase the query to use DISTINCT ON and an ORDER BY if you really do want to pick a somewhat arbitrary row, though I really doubt it's possible to express that via ActiveRecord.
Some databases - including SQLite and MySQL - will just pick an arbitrary row. This is considered incorrect and unsafe by the PostgreSQL team, so PostgreSQL follows the SQL standard and considers such queries to be errors.
If you have:
col1 col2
fred 42
bob 9
fred 44
fred 99
and you do:
SELECT col1, col2 FROM mytable GROUP BY col1;
then it's obvious that you should get the row:
bob 9
but what about the result for fred? There is no single correct answer to pick, so the database will refuse to execute such unsafe queries. If you wanted the greatest col2 for any col1 you'd use the max aggregate:
SELECT col1, max(col2) AS max_col2 FROM mytable GROUP BY col1;
I recently moved from MySQL to PostgreSQL and encountered the same issue. Just for reference, the best approach I've found is to use DISTINCT ON as suggested in this SO answer:
Elegant PostgreSQL Group by for Ruby on Rails / ActiveRecord
This will let you get one record for each unique value in your chosen column that matches the other query conditions:
MyModel.where(:some_col => value).select("DISTINCT ON (unique_col) *")
I prefer DISTINCT ON because I can still get all the other column values in the row. DISTINCT alone will only return the value of that specific column.
After often receiving the error myself I realised that Rails (I am using rails 4) automatically adds an 'order by id' at the end of your grouping query. This often results in the error above. So make sure you append your own .order(:group_by_column) at the end of your Rails query. Hence you will have something like this:
#problems = Problem.select('problems.username, sum(problems.weight) as weight_sum').group('problems.username').order('problems.username')
#myestate1 = Estate.where(:Mgmt => current_user.Company)
#myestate = #myestate1.select("DISTINCT(user_id)")
this is what I did.
I have a view which is exporting some columns, one column is called
'created_at' type:"timestamp without timezone" format: "2014-03-20 12:46:36.590253"
In rails I have a method, which is getting data from the view and is trying to filter the data by a date. I tried rapping created_at into date() but is still not working.
Any ideas?
return ActiveRecord::Base.connection.select_all("
select * from db_functions where date(created_at) >= #{date_p} AND date(created_at) <= #{date_p}")
PG::UndefinedFunction: ERROR: operator does not exist: date >= integer
LINE 2: ...select * from db_functions where date(created_at) >= 2014-03...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
The first noticeable problem is that your time is not quoted. This is causing the time to be treated as integer. To fix this, you could simply wrap date_p with quotes or use ActiveReocrd::ConnectionAdapters#quote as:
conn = ActiveRecord::Base.connection
return conn.select_all("select * from db_functions
where date(created_at) >= #{conn.quote(date_p)}
AND date(created_at) <= #{conn.quote(date_p)}")
Another option, instead of converting each created_at to date in where clause, you could modify date_p to be beginning of the day value and remove the "date" conversion altogether. Also, instead of using values directly in the query, it's better to use prepared statements (Linked article is a couple of years old, but explains prepared statements clearly with examples).
Then there is also the task of modifying the date time parameter to beginning of day. Given that date_p is a string and not a time, here is what you can do:
date_p = Time.zone.parse("2014-03-20 12:46:36.590253").beginning_of_day.utc
return ActiveRecord::Base.connection.select_all("select * from db_functions
where created_at >= ?
AND created_at <= ?", date_p, date_p)
Background:
Writing a Ruby on Rails app that is connect to MS SQL DB(don't ask why, its something I cant get around), the DB is quiet large and can have up to 2/3 million rows.
There is one main columns that matter for me at the moment and that is 'TimeUTC' and the table is called ApplicationLog and the query will be in the ApplicationLogController.
Problem:
I want to write a query that i can pass in two dates and it will group all the records by day using the 'TimeUTC' column and gives me a total of all the records for those days in between those two dates.
I have the SQL Query:
DECLARE #StartDate DateTime = '2014-01-04 00:00:00'
DECLARE #EndDate DateTime = '2014-02-04 23:59:59'
select (dateadd(DAY,0, datediff(day,0, TimeUtc))) as [DATE], count(*)
from applicationlog (nolock)
where TimeUtc between #StartDate and #EndDate
group by dateadd(DAY,0, datediff(day,0, TimeUtc))
order by [DATE] desc
I try starting with something like this:
#results = ApplicationLog.select((dateadd(DAY,0, datediff(day,0, TimeUtc)) as [date], count(*)).
group(dateadd(DAY,0, datediff(day,0, TimeUtc))).order(date desc)
Now I am a newbie at this so I could be so far off the track its not funny, but any help would be great. Am I even going about this the right way, is there a better way??
Try with the following code, which uses Arel code with some SQL embedded.
class ApplicationLog < ActiveRecord::Base
def self.between(range)
columns = Arel.sql('dateadd(DAY,0,datediff(day,0,TimeUtc)) as date, COUNT(*)')
conditions = arel_table['TimeUTC'].in(range)
query = arel_table.project(columns).where(conditions).group([1]).order('date desc')
ActiveRecord::Base.connection.execute(query.to_sql)
end
end
Then use ApplicationLog.between(1.week.ago..Time.now).
I'm trying to figure out how to do a query where created_at.year == a given year, and created_at.month equals a given month.
However I can't figure out what I'm doing wrong.
Model.where("'created_at.month' = ? AND 'created_at.year' = ?", 7,2013)
results in nothing being shown.
However when I try Model.first.created_at.month ==7 and
Model.first.created_at.year ==2013 I get true for both.
Therefore theoretically my query should be at least be returning my first record.
Anyone know what I'm doing wrong or any alternative way to find records created on specific months?
Note that in my views the month / year will be parameters but for the purposes of this example I used actual values.
using ruby 1.9.3
rails 3.2.13
You can use the extract SQL function, that will extract the month and year of the timestamp:
Model.where('extract(year from created_at) = ? and extract(month from created_at) = ?', '2013','7')
This query should give you the desired result.
created_at is a timestamp; it is not a set of discrete fields in the database. created_at.year and such don't exist in your DB; it's simply a single timestamp field. When you call #model.created_at.year, Rails is loading the created_at field from the database, and creating a Time object from it, which has a #year method you can call.
What you want is to query on a range of dates:
Model.where("created_at >= ? and created_at < ?", Time.mktime(2013, 7), Time.mktime(2013, 8))
This will find any Model with a created_at timestamp in July 2013.
How do I group by hour in Postgres & Rails? I've read through quite a few SO answers but I'm getting errors.
This works when grouping by date:
Model.group("date(updated_at)").count
Then I tried the following for hour but they didn't work:
Model.group("hour(updated_at)").count
Model.group("date_format(updated_at, '%H')").count
Model.group("extract(hour from updated_at)").count
Once I've found certain hours I need to update, how would I then get the models with those hours? I.e:
Model.where("hour(updated_at) = ?", 5)
You could try the following
Model.group("DATE_PART('hour', updated_at)").count
UPDATE:
How to find records
Model.where("DATE_PART('hour', updated_at) = ?", 5)
PostgreSQL presents two ways to extract a part of date/time:
EXTRACT is SQL standard function that exists in other SQL based databases.
Model.group("EXTRACT(hour FROM created_at)").count
Model.where("EXTRACT(hour FROM created_at) = ?", 5)
date_part function
Model.group("date_part('hour', updated_at)").count
Model.where("date_part('hour', updated_at) = ?", 5)
The official documentation.
If you want both the date and the hour, you can use date_trunc:
Model.group("date_trunc('hour', created_at)").count
And if you want the answers in order:
Model.order("date_trunc('hour', created_at)").group("date_trunc('hour', created_at)").count
Yeah it's weird that you can't just do order(:created_at) but oh well.