I am getting an error when I try to migrate my db. I don't entirely remember how I got here, but I believe I:
created new branch, scaffolded 'Requests', db:migrated, switched back to master, and merged branch
created another branch, did some stuff, db:migrated, and everything was working fine.
pulled from heroku postgres database so i could test out if things worked with actual data. then tried db migrating, but gave me this error:
rake db:migrate
== CreateRequests: migrating =================================================
-- create_table(:requests)
NOTICE: CREATE TABLE will create implicit sequence "requests_id_seq1" for serial column "requests.id"
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "requests" already exists
: CREATE TABLE "requests" ("id" serial primary key, "title" character varying(255), "content" text, "category" character varying(255), "status" character varying(255), "requested_track_id" integer, "created_at" timestamp, "updated_at" timestamp)
Any ideas?
I'm not sure exactly what pull strategy you used, but if we make two reasonable assumptions about your pull strategy:
it doesn't drop the database but just overwrites tables, since this requires less permissions.
it is operating in a sort of 'archive mode', meaning it doesn't drop tables on the destination just because they don't exist on the source. Think rsync; you have to specify --delete to get what might be your expected behavior with that utility.
If your steps are correct, then what happened is you overwrote the schema_migrations table, so Rails thinks you haven't added the table yet, but neither did your heroku pull drop the table because of #2 above.
Don't create another migration!!! This will fail on everyone elses' computer except yours, but will only run on yours once.
Instead, run rails dbconsole and execute something like DROP TABLE 'requests' (I forget the postgres syntax, might not be exactly that). Then you can run your migrations.
There is another way to avoid dropping a table with data in it.
What I do in those cases is to check which migration is failing.
Suppose you have a file db/migrate/20130908214222_create_requests.rb, and for some reason, ActiveRecord failed in the past when stored this migration in its "tracking system".
To be sure that this is the case, just find, in the schema_migrations table, a row containing a number like this example: 20130908214222
If that row does not exist, you just have to insert a new one:
INSERT INTO schema_migrations(
version
) VALUES (
20130908214222
);
Next time you run rake db:migrate, ActiveRecord will omit this step, and will continue migrating to end without complications.
Related
After upgrading to Rails 5 my schema file keeps getting altered when running db:migrate. Rails is changing:
create_table "flightlessons", force: :cascade do |t|
to:
create_table "flightlessons", id: :integer, default: -> { "nextval('lessons_id_seq'::regclass)" }, force: :cascade do |t|
It only occurs on this one model. Why is rails implementing nextval on this particular model? And, why is it getting the model name wrong (lessons_id_seq should be flightlessons_id_seq). Manually changing it to flightlessons_id_seq, however, results in the same no relation error.
PG::UndefinedTable: ERROR: relation "lessons_id_seq" does not exist
To proceed, I simply alter the schema.rb file back to what that line 'should' be. Then, I can migrate or test:prepare or whatever until the next time rails alters it back to using the nextval method.
Thank you for any insight into this.
This is a bit long of an answer, so I've broken it into sections. Buckle up!
My theory
My guess is that your development database does contain the lessons_id_seq sequence, and that its definition of flightlessons.id is set to depend on it (i.e., exactly what Rails is putting into your schema file).
How and why? You likely renamed the lessons table to flightlessons at some point in the past, but that rename didn't change the sequence that the table depended on -- and since schema.rb does not record sequences, the lessons_id_seq sequence does not get copied to your test database, and thus you get this error.
To verify my theory, run rails db and try the following commands:
\d lessons_id_seq
This should return the definition of that sequence. Then, try:
\d flightlessons
And look at the definition of the id column. I expect it to include DEFAULT nextval('lessons_id_seq').
Fixes
The easiest way to fix this is to switch to using structure.sql instead of schema.rb (see the docs). This will carry over the exact state of your database and avoid any interference or interpretation by Rails, which is what's causing your current issue. I always recommend structure.sql for production systems.
However, you can also go into your development database and change the sequence name:
ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;
ALTER TABLE flightlessons ALTER COLUMN id SET DEFAULT nextval('flightlessons_id_seq');
This would be a terrible idea on a production system, but if your issue is just local, it should rectify your current database state with your schema.rb and thus address your current problem. You may wish to encode that into a migration, if you want rails db:drop db:create db:migrate to work on a fresh app.
Why now?
The behavior where Rails is dumping out the default value for your table's primary key may very well be new in Rails 5. Previously, Rails may have just trusted that your ID column had a sane default, and ignored whatever value it actually saw. But I haven't done the research to see if that's true or not.
The simplest fix is to just to rename the sequence in production to match the current table name. E.g. in a production Rails console:
ActiveRecord::Base.connection.execute("ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;")
Turns out it's fine to just rename it, if you haven't done anything fancy with the sequence (like implementing your own Postgres function that references it by name).
Apparently the table points to the sequence by ID, not by name, so the rename is instant and with no ill effects that we could see. More details here: https://dba.stackexchange.com/questions/265569/how-can-i-safely-rename-a-sequence-in-postgresql-ideally-without-downtime
We tried it on staging first, and verified that the ID sequence kept on ticking after making the change in staging and production. Everything just worked.
(Also see Robert Nubel's fantastic answer for more details on what's going on.)
One of my migration files is referencing another table/model that will will be created further down the migration sequence.
Postgres doesn't like that:
PG::UndefinedTable: ERROR: relation "users" does not exist
So I wonder if there are any potential problems to manually reorder the migration files (by inventing new timestamps/prefixes)?
The affected tables are already down migrated.
When you run rake db:migrate command it compares schema_migrations table and migration files located in db/migrate folder. All the migrations which were not executed receive MigrationClass#up call then.
So starting from the point when your code is already published and/or migrations are run by other users, changing your migrations timestamps/names may lead to unprocessable migration procedure (as schema_migrations will treat a migration with changed timestamp as new, unprocessed one, and try to process it "again"). Possible workaround for this would be to comment the contents of up method for a while and uncomment it back after migrations are done. For fun you can also manipulate schema_migrations table directly from your db console (adding or removing necessary records). Both of these ways smells like a hack though.
Until then... Everything should work flawlessly.
This is what worked for me for this same situation, even though it's a hack.
Rails runs migrations in order of the timestamp, and Rails tracks which migrations have been run by the timestamp part of the migration file name, but not by the rest of the file name. So if you need to change the order of two migrations because the earlier one references the later one you can simply switch the 14 digit timestamp portion of the filenames by renaming both migration files. If the timestamp is off by even one digit Rails will think it's a new migration so write them down before changing them.
I have a local PSQL database setup and am having issues getting rake db:migrate to function properly. For instance, my database has already had its high_school column name changed to high_school_name, but rake db:migrate is failing, and the status reveals this:
up 20130307043554 Adding seo tables
up 20130307185401 Create admin notes
up 20130307185402 Move admin notes to comments
up 20130308160956 Add active flad to users
up 20130308214928 Add column public to users table
up 20130325203837 Add duration to videos
up 20130326171803 Update duration of videos
down 20130410145000 Fix high school name
up 20130410145028 Add high school id to users
up 20130410161705 Convert units for stats
up 20130410164209 ********** NO FILE **********
up 20130416142844 Add column coach id to users
Why is a migration in the middle of the migration order failing/being read as "down"? Here's the error:
Migrating to AddingSeoTables (20130307043554)
Migrating to CreateAdminNotes (20130307185401)
Migrating to MoveAdminNotesToComments (20130307185402)
Migrating to AddActiveFladToUsers (20130308160956)
Migrating to AddColumnPublicToUsersTable (20130308214928)
Migrating to AddDurationToVideos (20130325203837)
Migrating to UpdateDurationOfVideos (20130326171803)
Migrating to FixHighSchoolName (20130410145000)
(0.1ms) BEGIN
== FixHighSchoolName: migrating ==============================================
-- rename_column(:users, :high_school, :high_school_name)
(0.4ms) ALTER TABLE "users" RENAME COLUMN "high_school" TO "high_school_name"
PG::Error: ERROR: column "high_school" does not exist
: ALTER TABLE "users" RENAME COLUMN "high_school" TO "high_school_name"
(0.1ms) ROLLBACK
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: column "high_school" does not exist
: ALTER TABLE "users" RENAME COLUMN "high_school" TO "high_school_name"
Obviously, there is a schema issue or something. Schema is tracked via the remote branch though, so I don't know where the issue lies.
I suspect that the down was not able to handle the column rename change correctly, unless just adding or removing columns.
One option you may wish to consider is actually changing the name with SQL. For instance if a migration to change high_school to high_school_name is failing and the database itself currently has high_school_name, rename that (in SQL) to high_school and then try and run the migration. This may be one option for you.
Add resque nil to at the end of the row where column is renaming
I've ended up with 9 migrations effectively duplicated. (I think this is because I installed/updated Gems and/or pulled in their migrations on both my dev and production machines, but am not entirely sure at this stage.)
I moved out one set of the duplicated 9 from the rails directories on the production server, but now that I want to db:migrate on production in order to run another migration, I'm getting:
$ bundle exec rake db:migrate RAILS_ENV=production
[DEPRECATION WARNING] Nested I18n namespace lookup under "activerecord.attributes.checkout" is no longer supported
== CreatePages: migrating ====================================================
-- create_table(:pages)
rake aborted!
An error has occurred, all later migrations canceled:
Mysql2::Error: Table 'pages' already exists: CREATE TABLE `pages` (`id` int(11) DEFAULT NULL auto_increment PRIMARY KEY, `title` varchar(255), `body` text, `slug` varchar(255), `created_at` datetime, `updated_at` datetime) ENGINE=InnoDB
This is because the migrations have effectively already been run.
I'd rather avoid doing db:migrate:down and db:migrate:up for each one - I think this will mean data in the production database is lost. (A couple of static pages in Spree in this case.)
Is there a way I can tell this installation of Rails to forget all outstanding migrations, effectively marking all outstanding migrations as done?
I solved it like this:
Go to the conflicting migration file.
Erase the content and save it.
Run rake db:migrate
Ctrl+Z the file to the previous state.
This was an special case, because I had copied the DB from another app, and I had conflicting migrations, and stuff.
You can add the timestamps of the migrations to the schema_migrations table. However why is that table missing from the database or missing the rows it needs?
It is likely to be the case that this particular migration has got half way through and failed, thus when you are trying to run it again the first part of the migration will not work as it has been done previously. This is a limitation of MySQL as it can't rollback migration changes that fail part of the way through. Postgres on the other hand can rollback structural changes to the database thus avoiding this issue.
By default rake db:migrate runs all the pending migrations. So, in order to get your migrations right.. for the sake ..comment out those migrations and afterwards revert those back to normal. It will ensure that you are ok in future migrations.
You can also, in a Rails console, use the ActiveRecord::SchemaMigration model to add the timestamps into the schema_migrations table.
ActiveRecord::SchemaMigration.create! version: '20210724133241'
I have a migration that creates a postres sequence for auto incrementing a primary identifier, and then executes a statement for altering the column and specifying the default value:
execute 'CREATE SEQUENCE "ServiceAvailability_ID_seq";'
execute <<-SQL
ALTER TABLE "ServiceAvailability"
ALTER COLUMN "ID" set DEFAULT NEXTVAL('ServiceAvailability_ID_seq');
SQL
If I run db:migrate everything seems to work, in that no errors are returned, however, if I run the rails application I get:
Mnull value in column "ID" violates not-null constraint
I have discovered by executing the sql statement in the migration manually, that this error is because the alter statement isn't working, or isn't being executed.
If I manually execute the following statement:
CREATE SEQUENCE "ServiceAvailability_ID_seq;
I get:
error : ERROR: relation "serviceavailability_id_seq" already exists
Which means the migration successfully created the sequence! However, if I manually run:
ALTER TABLE "ServiceProvider"
ALTER COLUMN "ID" set DEFAULT NEXTVAL('ServiceProvider_ID_seq');
SQL
It runs successfully and creates the default NEXTVAL.
So the question is, why is the migration file creating the sequence with the first execute statement, but not altering the table in the second execute? (Remembering, no errors are output on running db:migrate)
Thank you and apologies for tl:dr
I separated the creation of sequences and the altering of tables into two migrations.
When running:
rake db:migrate
The sequences would not be created nor the tables altered and the rake would run successfully.
If however, I ran the migrations seperately:
rake db:migrate VERSION=1
rake db:migrate VERSION=2
The sequences would be created, and the tables altered as expected.