We use Heroku review apps for QAing our PRs on Github. At some point a couple months ago we did some housekeeping and deleted a lot of old migration files and I think that is causing issues with the deploy on Heroku.
Here is my app.json file -
{
"name": "myapp",
"scripts": {
"postdeploy": "rake db:migrate db:seed db:fixtures:repopulate"
},
"env": {
"WEB_CONCURRENCY": "1",
"HEROKU_APP_NAME": {
"required": true
}
},
"formation": {
},
"addons": [
"heroku-redis",
{
"plan": "heroku-postgresql",
"options": {
"version": "9.6"
}
}
],
"buildpacks": [
]
}
Here is the results of running heroku releases:output -
Migrating to PopulateOldPasswordTable (20180305160642)
(1.3ms) BEGIN
== 20180305160642 PopulateOldPasswordTable: migrating =========================
User Load (2.5ms) SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1 [["LIMIT", 1000]]
(1.2ms) ROLLBACK
(1.4ms) SELECT pg_advisory_unlock(5078732648953702200)
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:
PG::UndefinedTable: ERROR: relation "users" does not exist
LINE 1: SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIM...
^
: SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `async_exec'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `block in exec_no_cache'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:589:in `block in log'
/app/vendor/bundle/ruby/2.3.0/gems/activesupport-5.0.2/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:583:in `log'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `exec_no_cache'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/postgresql_adapter.rb:587:in `execute_and_clear'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:103:in `exec_query'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:373:in `select'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:41:in `select_all'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/abstract/query_cache.rb:95:in `select_all'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/querying.rb:39:in `find_by_sql'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation.rb:702:in `exec_queries'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation.rb:583:in `load'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation.rb:260:in `records'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/batches.rb:200:in `block in in_batches'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/batches.rb:198:in `loop'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/batches.rb:198:in `in_batches'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/batches.rb:120:in `find_in_batches'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/relation/batches.rb:58:in `find_each'
/app/vendor/bundle/ruby/2.3.0/gems/activerecord-5.0.2/lib/active_record/querying.rb:9:in `find_each'
/app/db/migrate/20180305160642_populate_old_password_table.rb:3:in `change'
It sounds like you deleted a migration that hadn't been run in all environments. You really don't want to do that.
The simplest way to recover, if the database doesn't contain anything of value, would be to destroy it and recreate it from scratch -- that'll use your schema.rb file, which should better match your other working environments.
If that isn't an option, consider bringing the old migration(s) back [via your git history], at least until you've finished running them everywhere.
Related
I'm converting my Rails project to multi-tenant and I'm using Apartment.
I followed the directions on their README and the video they link to
My apartment.rb looks like this:
require 'apartment/elevators/subdomain'
Apartment.configure do |config|
config.excluded_models = ['Project', 'User']
config.tenant_names = lambda { Project.pluck :subdomain }
config.use_schemas = true
config.persistent_schemas = %w{ public }
end
Rails.application.config.middleware.use Apartment::Elevators::Subdomain
My Project model looks like this:
class Project < ApplicationRecord
after_create :create_tenant
private
def create_tenant
Apartment::Tenant.create(subdomain)
end
end
I can run db:migrate and db:setup no problem. It creates all my tables in the public schema then.
But, when I try to create a new Project, I get this (from rails c)
C:\code\vulnerability-history>rails c
Loading development environment (Rails 5.0.1)
irb(main):001:0> Project.create(name: 'foo', subdomain: 'bar')
(0.0ms) BEGIN
SQL (1.0ms) INSERT INTO "public"."projects" ("name", "subdomain") VALUES ($1, $2) RETURNING "id" [["name", "foo"], ["subdomain", "bar"]]
(1.0ms) CREATE SCHEMA "bar"
SQL (1.0ms) CREATE EXTENSION IF NOT EXISTS "plpgsql"
(2.0ms) DROP TABLE "commit_filepaths" CASCADE
(14.0ms) CREATE TABLE "commit_filepaths" ("id" serial primary key, "commit_id" integer NOT NULL, "filepath_id" integer NOT NULL, "total_churn" integer DEFAULT 0 NOT NULL)
(1.0ms) DROP TABLE "commits" CASCADE
(11.0ms) CREATE TABLE "commits" ("id" serial primary key, "commit_hash" character varying NOT NULL, "author_id" integer NOT NULL, "message" character varying NOT NULL, "date_created" timestamp NOT NULL, "notes" jsonb DEFAULT '{}' NOT NULL)
(1.0ms) DROP TABLE "developers" CASCADE
(11.0ms) CREATE TABLE "developers" ("id" serial primary key, "email" character varying NOT NULL)
(1.5ms) DROP TABLE "events" CASCADE
(9.4ms) CREATE TABLE "events" ("id" serial primary key, "detail_type" character varying NOT NULL, "detail_id" integer NOT NULL, "title_template" character varying DEFAULT ':title:' NOT NULL, "description_template" character varying DEFAULT ':description:' NOT NULL, "type_template" character varying DEFAULT ':type:' NOT NULL, "date_template" character varying DEFAULT ':date:' NOT NULL, "style_id" integer NOT NULL)
(1.0ms) DROP TABLE "filepaths" CASCADE
(11.0ms) CREATE TABLE "filepaths" ("id" serial primary key, "filepath" character varying NOT NULL, "notes" jsonb DEFAULT '{}' NOT NULL)
(1.0ms) DROP TABLE "fixes" CASCADE
(11.0ms) CREATE TABLE "fixes" ("id" serial primary key, "commit_id" integer NOT NULL, "vulnerability_id" integer NOT NULL, "notes" jsonb DEFAULT '{}' NOT NULL)
(1.0ms) DROP TABLE "projects" CASCADE
(13.0ms) CREATE TABLE "projects" ("id" serial primary key, "name" character varying NOT NULL, "subdomain" character varying NOT NULL)
(1.0ms) DROP TABLE "releases" CASCADE
(10.6ms) CREATE TABLE "releases" ("id" serial primary key, "number" integer NOT NULL, "date_released" timestamp NOT NULL, "project" character varying NOT NULL, "notes" jsonb NOT NULL)
(1.0ms) DROP TABLE "styles" CASCADE
(11.3ms) CREATE TABLE "styles" ("id" serial primary key, "name" character varying NOT NULL, "color" character varying DEFAULT '#ffffff' NOT NULL, "icon" character varying DEFAULT 'stars' NOT NULL)
(1.0ms) DROP TABLE "users" CASCADE
(11.4ms) CREATE TABLE "users" ("id" serial primary key, "username" character varying NOT NULL)
(1.0ms) DROP TABLE "vccs" CASCADE
(10.0ms) CREATE TABLE "vccs" ("id" serial primary key, "commit_id" integer NOT NULL, "vulnerability_id" integer NOT NULL, "notes" jsonb DEFAULT '{}' NOT NULL)
(1.0ms) DROP TABLE "vulnerabilities" CASCADE
(12.3ms) CREATE TABLE "vulnerabilities" ("id" serial primary key, "cve" character varying NOT NULL, "announced" timestamp NOT NULL, "description" character varying DEFAULT '' NOT NULL, "notes" jsonb DEFAULT '{}' NOT NULL)
(1.0ms) SELECT version FROM "schema_migrations"
ActiveRecord::InternalMetadata Load (1.0ms) SELECT "ar_internal_metadata".* FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = $1 LIMIT $2 [["key", :environment], ["LIMIT", 1]]
(0.0ms) ROLLBACK
ActiveRecord::StatementInvalid: PG::InFailedSqlTransaction: ERROR: current transaction is aborted, commands ignored until end of transaction block
: SET search_path TO "public", "public"
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:98:in `async_exec'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:98:in `block in execute'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/abstract_adapter.rb:589:in `block in log'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activesupport-5.0.1/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/abstract_adapter.rb:583:in `log'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:97:in `execute'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/postgresql/schema_statements.rb:315:in `schema_search_path='
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/postgresql_adapter.rb:40:in `reset'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/abstract_adapter.rb:109:in `rescue in ensure in switch'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/abstract_adapter.rb:109:in `ensure in switch'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/abstract_adapter.rb:109:in `switch'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/abstract_adapter.rb:26:in `block in create'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:97:in `__run_callbacks__'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:750:in `_run_create_callbacks'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activesupport-5.0.1/lib/active_support/callbacks.rb:90:in `run_callbacks'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/apartment-1.2.0/lib/apartment/adapters/abstract_adapter.rb:23:in `create'
... 22 levels...
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/connection_adapters/abstract/database_statements.rb:232:in `transaction'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/transactions.rb:211:in `transaction'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/transactions.rb:392:in `with_transaction_returning_status'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/transactions.rb:319:in `block in save'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/transactions.rb:334:in `rollback_active_record_state!'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/transactions.rb:318:in `save'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/suppressor.rb:41:in `save'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/activerecord-5.0.1/lib/active_record/persistence.rb:34:in `create'
from (irb):1
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/railties-5.0.1/lib/rails/commands/console.rb:65:in `start'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/railties-5.0.1/lib/rails/commands/console_helper.rb:9:in `start'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:78:in `console'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/railties-5.0.1/lib/rails/commands.rb:18:in `<top (required)>'
from bin/rails:4:in `require'
from bin/rails:4:in `<main>'
I did notice this issue with a similar stacktrace, but the workaround doesn't make sense to me.
I'm using:
Ruby 2.3.1 (2016-04-26 patchlevel 112) [x64-mingw32]
apartment 1.2.0
Rails 5.0.1
If the log is to be trusted, it appears to be failing on:
SELECT "ar_internal_metadata".* FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = $1 LIMIT $2 [["key", :environment], ["LIMIT", 1]]
Does your db/schema.rb contain this table? It's a new system table in Rails 5.
I have tested the new rails code on c9.io and it is working fine to select the record when the emailReceived field is false on Invitation table,
After I updated the new code into Heroku, it is failed to do it. And showing the error message.
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column
"emailreceived" does not exist
I used Invitation.column_names and can find the "emailReceived". Does the database corrupted? Here is the console information for your reference. Please help to advise how to fix it.
irb(main):005:0> Invitation.column_names
=> ["id", "church", "telephone1", "telephone2", "worshipTime", "contactName", "contactNumber", "priest", "preacher", "emailReceived", "acceptPromote", "promotionTime", "other", "created_at", "updated_at"]
irb(main):006:0> Invitation.where("emailReceived = false")
Invitation Load (3.0ms) SELECT "invitations".* FROM "invitations" WHERE (emailReceived = false)
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column "emailreceived" does not exist
LINE 1: SELECT "invitations".* FROM "invitations" WHERE (emailReceiv...
^
HINT: Perhaps you meant to reference the column "invitations.emailReceived".
: SELECT "invitations".* FROM "invitations" WHERE (emailReceived = false)
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in async_exec'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql_adapter.rb:598:inblock in exec_no_cache'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/abstract_adapter.rb:566:in block in log'
from /app/vendor/bundle/ruby/2.2.0/gems/activesupport-5.0.0.1/lib/active_support/notifications/instrumenter.rb:21:ininstrument'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/abstract_adapter.rb:560:in log'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql_adapter.rb:598:inexec_no_cache'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql_adapter.rb:585:in execute_and_clear'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:103:inexec_query'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/abstract/database_statements.rb:373:in select'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/abstract/database_statements.rb:41:inselect_all'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in select_all'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/querying.rb:39:infind_by_sql'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/relation.rb:699:in exec_queries'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/relation.rb:580:inload'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/relation.rb:260:in records'
from /app/vendor/bundle/ruby/2.2.0/gems/activerecord-5.0.0.1/lib/active_record/relation.rb:683:ininspect'
from /app/vendor/bundle/ruby/2.2.0/gems/railties-5.0.0.1/lib/rails/commands/console.rb:65:in start'
from /app/vendor/bundle/ruby/2.2.0/gems/railties-5.0.0.1/lib/rails/commands/console_helper.rb:9:instart'
from /app/vendor/bundle/ruby/2.2.0/gems/railties-5.0.0.1/lib/rails/commands/commands_tasks.rb:78:in console'
from /app/vendor/bundle/ruby/2.2.0/gems/railties-5.0.0.1/lib/rails/commands/commands_tasks.rb:49:inrun_command!'
from /app/vendor/bundle/ruby/2.2.0/gems/railties-5.0.0.1/lib/rails/commands.rb:18:in <top (required)>'
from bin/rails:8:inrequire'
from bin/rails:8:in `'irb(main):007:0>
One more information on this case, I run "heroku run rake --trace db:migrate VERSION=20161007083159" and get the below result. And it still doesn't work
heroku run rake --trace db:migrate VERSION=20161007083159
Running rake --trace db:migrate VERSION=20161007083159 on ⬢ shrouded-scrubland-30708... up, run.1694
** Invoke db:migrate (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:migrate
ActiveRecord::SchemaMigration Load (0.6ms) SELECT "schema_migrations".* FROM "schema_migrations"
ActiveRecord::SchemaMigration Load (0.5ms) SELECT "schema_migrations".* FROM "schema_migrations"
(0.5ms) SELECT pg_try_advisory_lock(129277373589159705);
ActiveRecord::SchemaMigration Load (0.5ms) SELECT "schema_migrations".* FROM "schema_migrations"
ActiveRecord::InternalMetadata Load (0.8ms) SELECT "ar_internal_metadata".* FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = $1 LIMIT $2 [["key", :environment], ["LIMIT", 1]]
(0.5ms) BEGIN
(0.4ms) COMMIT
(0.5ms) SELECT pg_advisory_unlock(129277373589159705)
** Invoke db:_dump (first_time)
** Execute db:_dump
Thanks
Patrick Lee
Eventually, I found that the problem is occurred due to the case sensitive of postgresql. So I changed the code from Invitation.where("emailReceived is ? OR emailReceived = ?", nil,false) to Invitation.where("emailReceived" => nil) on model.
But one more question, how can use OR for Invitation.where("emailReceived" => nil) Invitation.where("emailReceived" => false)?
Thanks
Patrick Lee
Model Layout
Article (has_many :attachments, :comments, :tags)
- id: string
Comment (has_many :attachments)
- id: integer
- article_id: string
Attachment
- id: integer
- reference_type: string
- reference_id: string
Tag
- id: integer
- article_id: string
Erroneous Query
In the rails console I run:
Article.includes(:tags, :comments => :attachments).references(:tags)
And the resulting error is:
Article.includes(:tags, :comments => :attachments).references(:tags)
SQL (0.7ms) SELECT "articles"."id" AS t0_r0, "articles"."created_at" AS t0_r1, "articles"."updated_at" AS t0_r2, "tags"."id" AS t1_r0, "tags"."article_id" AS t1_r1, "tags"."created_at" AS t1_r2, "tags"."updated_at" AS t1_r3, "comments"."id" AS t2_r0, "comments"."article_id" AS t2_r1, "comments"."created_at" AS t2_r2, "comments"."updated_at" AS t2_r3, "attachments"."id" AS t3_r0, "attachments"."reference_type" AS t3_r1, "attachments"."reference_id" AS t3_r2, "attachments"."created_at" AS t3_r3, "attachments"."updated_at" AS t3_r4 FROM "articles" LEFT OUTER JOIN "tags" ON "tags"."article_id" = "articles"."id" LEFT OUTER JOIN "comments" ON "comments"."article_id" = "articles"."id" LEFT OUTER JOIN "attachments" ON "attachments"."reference_id" = "comments"."id" AND "attachments"."reference_type" = $1 [["reference_type", "Comment"]]
ActiveRecord::StatementInvalid: PG::UndefinedFunction: ERROR: operator does not exist: character varying = integer
LINE 1: ...OIN "attachments" ON "attachments"."reference_id" = "comment_...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
: SELECT "articles"."id" AS t0_r0, "articles"."created_at" AS t0_r1, "articles"."updated_at" AS t0_r2, "tags"."id" AS t1_r0, "tags"."article_id" AS t1_r1, "tags"."created_at" AS t1_r2, "tags"."updated_at" AS t1_r3, "comments"."id" AS t2_r0, "comments"."article_id" AS t2_r1, "comments"."created_at" AS t2_r2, "comments"."updated_at" AS t2_r3, "attachments"."id" AS t3_r0, "attachments"."reference_type" AS t3_r1, "attachments"."reference_id" AS t3_r2, "attachments"."created_at" AS t3_r3, "attachments"."updated_at" AS t3_r4 FROM "articles" LEFT OUTER JOIN "tags" ON "tags"."article_id" = "articles"."id" LEFT OUTER JOIN "comments" ON "comments"."article_id" = "articles"."id" LEFT OUTER JOIN "attachments" ON "attachments"."reference_id" = "comments"."id" AND "attachments"."reference_type" = $1
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `async_exec'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `block in exec_no_cache'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:566:in `block in log'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:560:in `log'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `exec_no_cache'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:587:in `execute_and_clear'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:103:in `exec_query'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/abstract/database_statements.rb:373:in `select'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/abstract/database_statements.rb:41:in `select_all'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/relation/finder_methods.rb:389:in `find_with_associations'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/relation.rb:699:in `exec_queries'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/relation.rb:580:in `load'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/relation.rb:260:in `records'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activerecord-5.0.0/lib/active_record/relation.rb:683:in `inspect'
... 1 levels...
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/railties-5.0.0/lib/rails/commands/console_helper.rb:9:in `start'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/railties-5.0.0/lib/rails/commands/commands_tasks.rb:78:in `console'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/railties-5.0.0/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/railties-5.0.0/lib/rails/commands.rb:18:in `<top (required)>'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:293:in `require'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:293:in `block in require'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:259:in `load_dependency'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:293:in `require'
from /Users/jeremy/Documents/Synack/Code/experiments/joins-test/bin/rails:9:in `<top (required)>'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:287:in `load'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:287:in `block in load'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:259:in `load_dependency'
from /Users/jeremy/.rvm/gems/ruby-2.2.2#rails5/gems/activesupport-5.0.0/lib/active_support/dependencies.rb:287:in `load'
from /Users/jeremy/.rvm/rubies/ruby-2.2.2/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /Users/jeremy/.rvm/rubies/ruby-2.2.2/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
Successful Query
Without a .references() clause, the query works properly, and fetches all models separately, creating four separate requests.
In the rails console I run:
Article.includes(:tags, :comments => :attachments)
And I see:
Article.includes(:tags, :comments => :attachments)
Article Load (5.1ms) SELECT "articles".* FROM "articles"
Tag Load (0.3ms) SELECT "tags".* FROM "tags" WHERE "tags"."article_id" = 'seedacorn-123'
Comment Load (0.2ms) SELECT "comments".* FROM "comments" WHERE "comments"."article_id" = 'seedacorn-123'
Attachment Load (0.3ms) SELECT "attachments".* FROM "attachments" WHERE "attachments"."reference_type" = $1 AND "attachments"."reference_id" IN ('1', '2') [["reference_type", "Comment"]]
=> #<ActiveRecord::Relation [#<article id: "seedacorn-123", created_at: "2016-07-15 22:41:27", updated_at: "2016-07-15 22:41:27">]>
The Question
So I guess, what I'm really wondering, is why ActiveRecord adding all of the included models to the JOIN rather than just those that I added to the .references()? Is this intended as a performance benefit? Or is this needed for some other reason?
I did not add :comments or :comments => :attachments to the .references(), but they were still added to the JOIN. This is causing a typecasting error, but I'm wondering why this is even happening in the first place?
Ask
If you know how to resolve this with some kind of creative solution, I'm trying desperately to figure it out. The core problems is that my polymorphic Attachment model can reference other models with either String or Integer primary keys, and so its reference_type is a string. Then this is not getting properly typecast when querying other models, in the case where JOIN's are introduced.
Found an answer to this question after posting an issue on github.com/rail/rails.
As it turns out, this is known (and unfortunately expected) behavior in Rails. If you require that an association be loaded with a separate query and not as part of a JOIN, you can use preload instead of includes, which will always load the association as in a separate query.
Likewise, if you want to force an association to load as part of a JOIN, and you're not using includes, you can replace references with eager_load.
So
Article.includes(:tags, :comments => :attachments).references(:tags)
Would become
Article.preload(:tags, :comments => :attachments).eager_load(:tags)
I'm trying to create a graph with the count of users registered on my website every day.
I use:
User.group('date(created_at)').count
I'm following this suggestion but I have an error:
#users.group("DATE_TRUNC('day', created_at)").count
(0.6ms) SELECT COUNT(*) AS count_all, DATE_TRUNC('day', created_at) AS date_trunc_day_created_at FROM "users" GROUP BY DATE_TRUNC('day', created_at) ORDER BY created_at DESC
PG::GroupingError: ERROR: column "users.created_at" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ... GROUP BY DATE_TRUNC('day', created_at) ORDER BY created_at...
^
: SELECT COUNT(*) AS count_all, DATE_TRUNC('day', created_at) AS date_trunc_day_created_at FROM "users" GROUP BY DATE_TRUNC('day', created_at) ORDER BY created_at DESC
ActiveRecord::StatementInvalid: PG::GroupingError: ERROR: column "users.created_at" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: ... GROUP BY DATE_TRUNC('day', created_at) ORDER BY created_at...
^
: SELECT COUNT(*) AS count_all, DATE_TRUNC('day', created_at) AS date_trunc_day_created_at FROM "users" GROUP BY DATE_TRUNC('day', created_at) ORDER BY created_at DESC
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `async_exec'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_no_cache'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:138:in `block in exec_query'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:425:in `block in log'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activesupport-4.0.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:420:in `log'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:137:in `exec_query'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:885:in `select'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract/database_statements.rb:24:in `select_all'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/connection_adapters/abstract/query_cache.rb:63:in `select_all'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/relation/calculations.rb:318:in `execute_grouped_calculation'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/relation/calculations.rb:222:in `perform_calculation'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/relation/calculations.rb:108:in `calculate'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-deprecated_finders-1.0.3/lib/active_record/deprecated_finders/relation.rb:84:in `calculate'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/relation/calculations.rb:111:in `calculate'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-deprecated_finders-1.0.3/lib/active_record/deprecated_finders/relation.rb:84:in `calculate'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/activerecord-4.0.0/lib/active_record/relation/calculations.rb:24:in `count'
from (irb):21
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/railties-4.0.0/lib/rails/commands/console.rb:90:in `start'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/railties-4.0.0/lib/rails/commands/console.rb:9:in `start'
from /Users/scrivoaroby/.rvm/gems/ruby-2.0.0-p353/gems/railties-4.0.0/lib/rails/commands.rb:64:in `<top (required)>'
from bin/rails:4:in `require'
from bin/rUser.group('date(created_at)')
Are you using order somewhere? (In a scope or earlier query maybe.)
User.group('date(created_at)').count #=> should return a hash with {date => (no. of users)}
But, as far as this is concerned, we don't know what #users is. Therefore, I'd suggest you to unscope it in this case since you only want the count.
#users.unscoped.group("DATE_TRUNC('day', created_at)").count
I use find_by_sql and within a second it gives me back a response:
Report.find_by_sql("SELECT min(reports.time) FROM reports WHERE unit_id = '#{Unit.find(3007).id}' AND driver_id = '#{Driver.find(2).id}' AND time >= '#{beginning}' AND time <= '#{ending}'")
Unit Load (111.1ms) SELECT "units".* FROM "units" WHERE "units"."id" = $1 LIMIT 1 [["id", 3007]]
Driver Load (98.0ms) SELECT "drivers".* FROM "drivers" WHERE "drivers"."id" = $1 LIMIT 1 [["id", 2]]
Report Load (59.6ms) SELECT min(reports.time) FROM reports WHERE unit_id = '3007' AND driver_id = '2' AND time >= '2013-03-01 00:00:00 UTC
' AND time <= '2013-03-31 23:59:59 UTC'
=> [#<Report >]
Then I try to use a scope with activerecord relation methods to build the same query:
scope :start_driver_time, lambda { |unit, driver, start_time, end_time|
where("unit_id = ? AND
driver_id = ? AND
time >= ? AND
time <= ?",
unit.id, driver.id, start_time, end_time)
.order("time asc")
.minimum(:time)
}
1.9.3p0 :012 > Report.start_driver_time(Unit.find(3007), Driver.find(2), beginning, ending)
Unit Load (48.4ms) SELECT "units".* FROM "units" WHERE "units"."id" = $1 LIMIT 1 [["id", 3007]]
Driver Load (46.5ms) SELECT "drivers".* FROM "drivers" WHERE "drivers"."id" = $1 LIMIT 1 [["id", 2]]
(49.3ms) SELECT MIN("reports"."time") AS min_id FROM "reports" WHERE (unit_id = 3007 AND
driver_id = 2 AND
time >= '2013-03-01 00:00:00.000000' AND
time <= '2013-03-31 23:59:59.999999')
But after 45 minutes the query is still running.
When I press control + c this is the output:
^C Report Load (30484.0ms) SELECT "reports".* FROM "reports"
ActiveRecord::StatementInvalid: IRB::Abort: abort then interrupt!: SELECT "reports".* FROM "reports"
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:88:in `irb_abort'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:257:in `signal_handle'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:65:in `block in start'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:1151:in `call'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:1151:in `async_exec'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:1151:in `exec_no_cache'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:664:in `block in exec_query'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activesupport-3.2.5/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:663:in `exec_query'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/postgresql_adapter.rb:1246:in `select'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/abstract/database_statements.rb:18:in `select_all'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/connection_adapters/abstract/query_cache.rb:63:in `select_all'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/querying.rb:38:in `block in find_by_sql'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/activerecord-3.2.5/lib/active_record/explain.rb:40:in `logging_query_plan'
... 12 levels...
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:273:in `signal_status'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:156:in `block in eval_input'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb/ruby-lex.rb:243:in `block (2 levels) in each_top_level_statement'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `loop'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `block in each_top_level_statement'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `catch'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `each_top_level_statement'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:155:in `eval_input'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:70:in `block in start'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:69:in `catch'
from /Users/johnmerlino/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/irb.rb:69:in `start'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.5/lib/rails/commands/console.rb:47:in `start'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.5/lib/rails/commands/console.rb:8:in `start'
from /Users/johnmerlino/.rvm/gems/ruby-1.9.3-p0/gems/railties-3.2.5/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'Maybe IRB bug!
I think that scopes are usually meant to return a custom subset / join of the model, which may be causing the SELECT "reports".* FROM "reports" query after the others complete. Since it looks like you're on Rails 3 or higher, I'd replace the scope with a class method (the Rails Guide says "Using a class method is the preferred way to accept arguments for scopes.").