Hi
I changed my database from mySql to PostgreSQL and get an error every time I use query with :order statement
For example the following code works perfectly in MySQL
Hour.sum("working_hours",:conditions=>['project_id=? AND reported_date=?',project,h.reported_date],:order=>"reported_date
But gives me an error in PostgreSQL
PGError: ERROR: column "hours.reported_date" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ...rted_date='2010-10-06 00:00:00.000000') ORDER BY reported_d..
: SELECT sum("hours".working_hours) AS sum_working_hours FROM "hours" WHERE (project_id=1 AND reported_date='2010-10-06 00:00:00.000000') ORDER BY reported_date
If I delete the order statement then the query works ok
I will most appreciate any help on this subject
PostreSQL is stricter to the SQL standard than MySQL is.
SQL states that if you ORDER by a column, that column must be SELECTed and appear in the GROUP BY clause.
Try this:
Hour.sum("working_hours",:conditions=>['project_id=? AND reported_date=?',project,h.reported_date], :order=>"reported_date", :group_by => "working_hours"
Related
I am trying to order all the recipes in my database by the number of likes they have received. Likes are polymorphic and belong to :likeable while a recipe has many likes.
My query works for SQLite3, but when I upload to Heroku using PostgreSQL it seems to break things.
function is as follows:
Recipe.select('*').joins(:likes).group('recipes.id').order('COUNT(likes.likeable_id)')
And the error that Heroku gives me when I try to run the website:
ActionView::Template::Error (PG::GroupingError: ERROR: column "likes.id" must appear in the GROUP BY clause or be used in an aggregate function
Everything compiles, but the homepage uses that scope function so I get a server error right away.
You need to explicitly select recipies.*:
Recipe.select(
Recipe.arel_table[:*],
Likes.arel_table[:*].count.as('likes_count')
)
.joins(:likes)
.group(:id)
.order(:likes_count)
Selecting the count is really optional - you can skip .select entirely and just fetch the aggregate in the order clause:
Recipe.joins(:likes)
.group(:id)
.order(Likes.arel_table[:*].count)
You cannot select * from grouping by.
for most SQL-dabases (Postgres, newer Mysql, ...) you can only use SELET columns in a GROUP BY:
columns you've grouped by, and that are transient by the grouped column (e.g. grouping recipes.id can also select recipes.title)
And aggregated columns (count, sum, max)
Try:
Recipe.select('recipies.*').joins(:likes).group(:id).order('COUNT(likes.likeable_id)')
I'm studying rails at the moment and i'm trying to group results of a query by field.
the query is being created by:
#orders = Order.includes(:product).where(user_id: current_user.id, paid: 0).group(:product_id)
It works fine on sqlite (dev env) but it throws a fatal error while on production (postgreSQL).
Error is:
ActionView::Template::Error (PG::GroupingError: ERROR: column "orders.id" must appear in the GROUP BY clause or be used in an aggregate function
Hmm.. what am i doing wrong here?
Thanks
If you want to group in postgres, you need to either directly select that column or you need to call an aggregate function that uses this column. Count is such an aggregate function and you can achieve that in Rails via
#product_counts = Order.where(user: current_user, paid: 0).group(:product_id).count
Postgres automatically executes Count on the column you grouped by, which is product_id in this case. This will give you a hash like this:
{ product_id => count }
For more information, see here: https://www.tutorialspoint.com/postgresql/postgresql_group_by.htm
I am getting this error in the pg production mode, but its working fine in sqlite3 development mode.
ActiveRecord::StatementInvalid in ManagementController#index
PG::Error: ERROR: column "estates.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = ...
^
: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = 'Mazzey' GROUP BY user_id
#myestate = Estate.where(:Mgmt => current_user.Company).group(:user_id).all
If user_id is the PRIMARY KEY then you need to upgrade PostgreSQL; newer versions will correctly handle grouping by the primary key.
If user_id is neither unique nor the primary key for the 'estates' relation in question, then this query doesn't make much sense, since PostgreSQL has no way to know which value to return for each column of estates where multiple rows share the same user_id. You must use an aggregate function that expresses what you want, like min, max, avg, string_agg, array_agg, etc or add the column(s) of interest to the GROUP BY.
Alternately you can rephrase the query to use DISTINCT ON and an ORDER BY if you really do want to pick a somewhat arbitrary row, though I really doubt it's possible to express that via ActiveRecord.
Some databases - including SQLite and MySQL - will just pick an arbitrary row. This is considered incorrect and unsafe by the PostgreSQL team, so PostgreSQL follows the SQL standard and considers such queries to be errors.
If you have:
col1 col2
fred 42
bob 9
fred 44
fred 99
and you do:
SELECT col1, col2 FROM mytable GROUP BY col1;
then it's obvious that you should get the row:
bob 9
but what about the result for fred? There is no single correct answer to pick, so the database will refuse to execute such unsafe queries. If you wanted the greatest col2 for any col1 you'd use the max aggregate:
SELECT col1, max(col2) AS max_col2 FROM mytable GROUP BY col1;
I recently moved from MySQL to PostgreSQL and encountered the same issue. Just for reference, the best approach I've found is to use DISTINCT ON as suggested in this SO answer:
Elegant PostgreSQL Group by for Ruby on Rails / ActiveRecord
This will let you get one record for each unique value in your chosen column that matches the other query conditions:
MyModel.where(:some_col => value).select("DISTINCT ON (unique_col) *")
I prefer DISTINCT ON because I can still get all the other column values in the row. DISTINCT alone will only return the value of that specific column.
After often receiving the error myself I realised that Rails (I am using rails 4) automatically adds an 'order by id' at the end of your grouping query. This often results in the error above. So make sure you append your own .order(:group_by_column) at the end of your Rails query. Hence you will have something like this:
#problems = Problem.select('problems.username, sum(problems.weight) as weight_sum').group('problems.username').order('problems.username')
#myestate1 = Estate.where(:Mgmt => current_user.Company)
#myestate = #myestate1.select("DISTINCT(user_id)")
this is what I did.
In SQLite (development) I don't have any errors, but in production with Postgres I get the following error. I don't really understand the error.
PG::Error: ERROR: column "commits.updated_at" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ...mmits"."user_id" = 1 GROUP BY mission_id ORDER BY updated_at...
^
: SELECT COUNT(*) AS count_all, mission_id AS mission_id FROM "commits" WHERE "commits"."user_id" = 1 GROUP BY mission_id ORDER BY updated_at DESC
My controller method:
def show
#user = User.find(params[:id])
#commits = #user.commits.order("updated_at DESC").page(params[:page]).per(25)
#missions_commits = #commits.group("mission_id").count.length
end
UPDATE:
So i digged further into this PostgreSQL specific annoyance and I am surprised that this exception is not mentioned in the Ruby on Rails Guide.
I am using psql (PostgreSQL) 9.1.11
So from what I understand, I need to specify which column that should be used whenever you use the GROUP_BY clause. I thought using SELECT would help, which can be annoying if you need to SELECT a lot of columns.
Interesting discussion here
Anyways, when I look at the error, everytime the cursor is pointed to updated_at. In the SQL query, rails will always ORDER BY updated_at. So I have tried this horrible query:
#commits.group("mission_id, date(updated_at)")
.select("date(updated_at), count(mission_id)")
.having("count(mission_id) > 0")
.order("count(mission_id)").length
which gives me the following SQL
SELECT date(updated_at), count(mission_id)
FROM "commits"
WHERE "commits"."user_id" = 1
GROUP BY mission_id, date(updated_at)
HAVING count(mission_id) > 0
ORDER BY updated_at DESC, count(mission_id)
LIMIT 25 OFFSET 0
the error is the same.
Note that no matter what it will ORDER BY updated_at, even if I wanted to order by something else.
Also I don't want to group the records by updated_at just by mission_id.
This PostgreSQL error is just misleading and has little explanation to solving it. I have tried many formulas from the stackoverflow sidebar, nothing works and always the same error.
UPDATE 2:
So I got it to work, but it needs to group the updated_at because of the automatic ORDER BY updated_at. How do I count only by mission_id?
#missions_commits = #commits.group("mission_id, updated_at").count("mission_id").size
I guest you want to show general number of distinct Missions related with Commits, anyway it won't be number on page.
Try this:
#commits = #user.commits.order("updated_at DESC").page(params[:page]).per(25)
#missions_commits = #user.commits.distinct.count(:mission_id)
However if you want to get the number of distinct Missions on page I suppose it should be:
#missions_commits = #commits.collect(&:mission_id).uniq.count
Update
In Rails 3, distinct did not exist, but pure SQL counting should be used this way:
#missions_commits = #user.commits.count(:mission_id, distinct: true)
See the docs for PostgreSQL GROUP BY here:
http://www.postgresql.org/docs/9.3/interactive/sql-select.html#SQL-GROUPBY
Basically, unlike Sqlite (and MySQL) postgres requires that any columns selected or ordered on must appear in an aggregate function or the group by clause.
If you think it through, you'll see that this actually makes sense. Sqlite/MySQL cheat under the hood and silently drop those fields (not sure that's technically what happens).
Or thinking about it another way if you are grouping by a field, what's the point of ordering it? How would that even make sense unless you also had an aggregate function on the ordered field?
I have this statement to choose a given number of random records from the database that match certain set of tags. I am using act-as-taggable-on gem for tagging.
questions=Question.select('questions.*,RANDOM()').tagged_with(tags,:any=>true).order("RANDOM()").limit(test.no_of_questions)
I am using postgresql and I get the following error.
PG::Error: ERROR: syntax error at or near "DISTINCT"
LINE 1: SELECT questions.*,RANDOM(), DISTINCT questions.* FROM "que...
^
: SELECT questions.*,RANDOM(), DISTINCT questions.* FROM "questions" JOIN taggings quest_taggings_14e6dd0 ON quest_taggings_14e6dd0.taggable_id = "questions".id AND quest_taggings_14e6dd0.taggable_type = 'Question' WHERE (quest_taggings_14e6dd0.tag_id = 1) ORDER BY RANDOM() LIMIT 3
I am not sure what to do because I think the Distinct part is inserted by the tagging gem.
Thanks in advance!
You can do it along these lines
Question.tagged_with(tags,:any=>"true").order("RANDOM()").limit(limit_number)
update
As shown here http://archives.postgresql.org/pgsql-sql/2010-10/msg00004.php
You need to put the DISTINCT and the ORDER BY in separate query levels.
Apparently, the above Query would work with other DB but not Postgresql
Nevermind Ive found the solution.
I did this:
questions=Question.tagged_with(tags,:any=>true)
questions=questions.select('questions.*,RANDOM()').order("RANDOM()").limit(test.no_of_questions)
This way the questions.* wasnt included twice in the select statement.