I have a rails app that uses tagging and sqlite i am trying to deploy as a school assignment. when i deploy to Heroku (it converts to Postgresql) I get an error when trying to run it. It has to do with some differences between SQLITE and POSTGRESQL(from the logs it seems)
There are two tables being joined, tags and tagging which store tags related to posts.
Rails code:
self.select("name, count(taggings.tag_id) as count")
.joins(:taggings).group("taggings.tag_id")
Here is SQLite generated sql captured from console:
Tag Load (0.2ms) SELECT name, count(taggings.tag_id) as count
FROM "tags" INNER JOIN "taggings" ON "taggings"."tag_id" = "tags"."id"
GROUP BY taggings.tag_id
Here is the Heroku log error based on that statement running under Postgresql:
2015-06-20T03:25:00.360433+00:00 app[web.1]: **PG::GroupingError**: ERROR: column "tags.name" must appear in the GROUP BY clause or be used in an aggregate function
2015-06-20T03:25:00.360435+00:00 app[web.1]: LINE 1: SELECT name, count(taggings.tag_id) as count FROM "tags" INN...
2015-06-20T03:25:00.360417+00:00 app[web.1]: Tag Load (2.5ms) SELECT name, count(taggings.tag_id) as count FROM "tags" INNER JOIN "taggings" ON "taggings"."tag_id" = "tags"."id" GROUP BY taggings.tag_id
I am currently experimenting trying to find a solution but could use some help thanks!
I am using postgres 9.3 and rails 4.1
Postgres requires that non-aggregate fields you your SELECT clause appear in your GROUP clause, so it's unhappy that you're selecting name but grouping by taggings.tag_id. Try grouping by name instead.
Actually I found the exact same issue and solution here after i posted. The error pointed to exactly what needed to be done. Include name somewhere in the syntax...I just needed an example. Here was the final code.
Tag.select("tags.id, tags.name,count(taggings.tag_id) as count").
joins(:taggings).group("taggings.tag_id, tags.id, tags.name")
self.select("name, count(taggings.tag_id) as count").joins(:taggings).group("name, taggings.tag_id")
easy
Related
I'm running the following query in Rails 5, with the goal of finding the user with the most Pitches:
User
.select("users.*, COUNT(user_id) as pitch_count")
.unscoped
.joins("LEFT JOIN pitches AS pitches ON pitches.user_id = users.id")
.group("pitch.user_id")
.order("pitch_count DESC")
.limit(5)
But I'm getting the error:
Caused by PG::UndefinedColumn: ERROR: column "pitch_count" does not exist
Why isn't the query orderable by pitch_count?
Problem is in the unscoped method. It removes all previously defined scopes including select statement. See the following example:
User.select(:full_name, :email).unscoped.to_sql
# => SELECT "users".* FROM "users"
User.unscoped.select(:full_name, :email).to_sql
# => SELECT "users"."full_name", "users"."email" FROM "users"
See the difference? unscoped called after select definition completely removed every thing defined in the select.
For you this means that you should modify your code to call unscoped right after the model name:
User
.unscoped
.select("users.*, COUNT(user_id) as pitch_count")
.joins("LEFT JOIN pitches AS pitches ON pitches.user_id = users.id")
.group("pitch.user_id")
.order("pitch_count DESC")
.limit(5)
Note: new lines added mostly for readability but it should work like this in your ruby files. If you want to execute it in the rails console. You will have to remove new lines
Btw. you still might get error that column "user.id" must appear in the GROUP BY clause or be used in an aggregate function. It should be fixed by modifying group statement to use users.id instead of pitch.user_id:
.group("users.id")
I suggest you use counter_cache to make it easy to maintain and good for performance as well. By adding counter cache, you can get the user record with most pitches by User.reorder(pitches_count: :desc).first.
I have a model, Student, that with a habtm relationship with another model, Group. The following code:
Student.ransack(groups_id_eq: 22839).result
Produces the following SQL:
SELECT "students".* FROM "students"
LEFT OUTER JOIN "groups_students" ON "groups_students"."student_id" = "students"."id"
LEFT OUTER JOIN "groups" ON "groups"."id" = "groups_students"."group_id"
WHERE "groups"."id" = NULL
Notice that it's converting 22839 into NULL
This was working before upgrading to rails 5.
I have another model, User, with the same habtm relationship with groups. When I try Student.ransack(groups_id_eq: 22839).result it works and produces the desired SQL. The difference between two models is that Student has a uuid id, whereas User
Edit: This does appear to be a bug and I've submitted an issue with a more comprehensive gist.
I have resolved the issue by monkey patching ActiveRecord AliasTracker.
See my gh issue for the full explanation.
I have an actions table with over 450,000 records. I want to join the actions table on the users table (it actually joins two other tables, one of which is joined on the other, and the other being joined on the users table, before joining the actions table.) The sql query looks like this:
SELECT "users".* FROM "users" INNER JOIN "campaigns" ON "campaigns"."user_id" = "users"."id" INNER JOIN "books" ON "books"."campaign_id" = "campaigns"."id" INNER JOIN "actions" ON "actions"."book_id" = "books"."id" AND "actions"."type" IN ('Impression')
However, this query in rails causes my app to hang because of the large number of records in the actions table.
How should I be handling this?
There are several problems with this approach:
You are fetching the same users several times (users multiplied by
number of actions)
You are fetching all the users with corresponding actions at once. It means big memory consumption and thus frequent garbage collection
You are fetching all the attributes for users. I guess, you do not need all them all
You have made comment about ordering users by their order count. Do you do this in Ruby code ? If yes, then it's the 4th problem. Big problem, indeed
So I'd propose to use group() method for grouping records or just plain SQL like
SELECT "users".id, count(*) as actions_cnt
FROM "users"
INNER JOIN "campaigns" ON "campaigns"."user_id" = "users"."id"
INNER JOIN "books" ON "books"."campaign_id" = "campaigns"."id"
INNER JOIN "actions" ON "actions"."book_id" = "books"."id" AND "actions"."type" IN ('Impression')
GROUP BY
"users".id
If there are many users in your app then I'd propose to add "OFFSET #{offset} LIMIT #{limit}" to fetch records in batches.
Finally, you can directly specify columns that you need so that memory footprint will be not so large.
I am trying to put my app live on heroku but I am running into a problem that it doesn't like me using backticks (`) in my sql queries. Here is the error from the log:
2011-10-29T18:28:26+00:00 app[web.1]: UTER JOIN "events_users" ON "events_users"."event_id" = "events"."id" LEFT OUTER JOIN "users" ON "users"."id" = "events_users"."user_id" WHERE (`users`.id IN (2,4,17,1)) ORDER BY events.event_date DESC):
It works on my local machine because i am using sqlite but it is not working on heroku. So I have two questions:
1) Is there something else I can use instead of the backtick?
2) Is postgres a sqlite alternative I should be using so that my heroku deployment matches my local machine?
You should be able to use both with quotes and without quotes. Quotes are acceptable for columns, and single ticks for values. Resulting in something like:
OUTER JOIN "events_users" ON "events_users"."event_id" = "events"."id" LEFT OUTER JOIN "users" ON "users"."id" = "events_users"."user_id" WHERE ("users"."id" IN (2,4,17,1)) ORDER BY events.event_date DESC)
sqlite is acceptable for local development, though if you do want exact parity you could setup postgres locally to ensure that you're creating code that runs identically.
I'm attempting a transition from MySQL to what seems to be the stricter and less Rails-friendly PostgreSQL.
I'm running into this funny conundrum:
irb(main):015:0> puts w.to_sql
SELECT DISTINCT ON (vendors.id) vendors.*
FROM "vendors"
INNER JOIN "locations" ON "locations"."locatable_id" = "vendors"."id"
AND "locations"."locatable_type" = 'Vendor'
WHERE (locations.latitude IS NOT NULL AND locations.longitude IS NOT NULL)
But...
irb(main):017:0> puts w.order('vendors.id').to_sql
SELECT * FROM (
SELECT DISTINCT ON (vendors.id) vendors.*
FROM "vendors"
INNER JOIN "locations" ON "locations"."locatable_id" = "vendors"."id"
AND "locations"."locatable_type" = 'Vendor'
WHERE (locations.latitude IS NOT NULL AND locations.longitude IS NOT NULL)
) AS id_list ORDER BY id_list.alias_0
This, despite the fact that just adding ORDER BY vendors.id works just fine as a valid PostgreSQL query. Instead of just adding that, it does something super funny and produces an invalid query at the end of the day:
ActiveRecord::StatementInvalid: PGError: ERROR: column id_list.alias_0 does not exist
LINE 1: ...tions.longitude IS NOT NULL)) AS id_list ORDER BY id_list.al...
Any clue what I should look at?
I've run into the same problem. Looks like it was a bug in Arel causing DISTINCT ON not to play nice with ORDER when using PostgreSQL.
The original bug is no longer visible since Rails has moved from Lighthouse to github issues but you can take a look at the Google cache. There's a comment on a more recent pull request which indicates the issue was fixed after Rails 3.0.7 was released. Looks like it will be in 3.1 but I couldn't verify as some of the gems I'm using are not yet compatible.
The current work-around is to use find_by_sql.