After upgrading to Rails 5 my schema file keeps getting altered when running db:migrate. Rails is changing:
create_table "flightlessons", force: :cascade do |t|
to:
create_table "flightlessons", id: :integer, default: -> { "nextval('lessons_id_seq'::regclass)" }, force: :cascade do |t|
It only occurs on this one model. Why is rails implementing nextval on this particular model? And, why is it getting the model name wrong (lessons_id_seq should be flightlessons_id_seq). Manually changing it to flightlessons_id_seq, however, results in the same no relation error.
PG::UndefinedTable: ERROR: relation "lessons_id_seq" does not exist
To proceed, I simply alter the schema.rb file back to what that line 'should' be. Then, I can migrate or test:prepare or whatever until the next time rails alters it back to using the nextval method.
Thank you for any insight into this.
This is a bit long of an answer, so I've broken it into sections. Buckle up!
My theory
My guess is that your development database does contain the lessons_id_seq sequence, and that its definition of flightlessons.id is set to depend on it (i.e., exactly what Rails is putting into your schema file).
How and why? You likely renamed the lessons table to flightlessons at some point in the past, but that rename didn't change the sequence that the table depended on -- and since schema.rb does not record sequences, the lessons_id_seq sequence does not get copied to your test database, and thus you get this error.
To verify my theory, run rails db and try the following commands:
\d lessons_id_seq
This should return the definition of that sequence. Then, try:
\d flightlessons
And look at the definition of the id column. I expect it to include DEFAULT nextval('lessons_id_seq').
Fixes
The easiest way to fix this is to switch to using structure.sql instead of schema.rb (see the docs). This will carry over the exact state of your database and avoid any interference or interpretation by Rails, which is what's causing your current issue. I always recommend structure.sql for production systems.
However, you can also go into your development database and change the sequence name:
ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;
ALTER TABLE flightlessons ALTER COLUMN id SET DEFAULT nextval('flightlessons_id_seq');
This would be a terrible idea on a production system, but if your issue is just local, it should rectify your current database state with your schema.rb and thus address your current problem. You may wish to encode that into a migration, if you want rails db:drop db:create db:migrate to work on a fresh app.
Why now?
The behavior where Rails is dumping out the default value for your table's primary key may very well be new in Rails 5. Previously, Rails may have just trusted that your ID column had a sane default, and ignored whatever value it actually saw. But I haven't done the research to see if that's true or not.
The simplest fix is to just to rename the sequence in production to match the current table name. E.g. in a production Rails console:
ActiveRecord::Base.connection.execute("ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;")
Turns out it's fine to just rename it, if you haven't done anything fancy with the sequence (like implementing your own Postgres function that references it by name).
Apparently the table points to the sequence by ID, not by name, so the rename is instant and with no ill effects that we could see. More details here: https://dba.stackexchange.com/questions/265569/how-can-i-safely-rename-a-sequence-in-postgresql-ideally-without-downtime
We tried it on staging first, and verified that the ID sequence kept on ticking after making the change in staging and production. Everything just worked.
(Also see Robert Nubel's fantastic answer for more details on what's going on.)
Related
Like some others, I am having trouble with an error
"ActiveRecord::StatementInvalid: SQLite3::SQLException: object name
reserved for internal use: sqlite_sp_functions: CREATE TABLE
"sqlite_sp_functions" ("name" text, "text" text)"
when running rake test on a Rails project.
The offending lines in schema.rb are:
create_table "sqlite_sp_functions", id: false, force: true do |t|
t.text "name"
t.text "text"
end
The suggestions on the previous query about this involved editing schema.rb or deleting that file & regenerating it, but schema.rb (and the offending code) are regenerated on each migration (plus, I don't want to delete Rails-generated code without knowing the implications), so that's not really a satisfactory solution.
So what is the sqlite_sp_functions table for, and how can I get Rails to generate a schema.rb file that doesn't break rake test for the project?
as per https://stackoverflow.com/a/25077833/601782:
Add the following config line to config/application.rb or in config/environments/development.rb:
ActiveRecord::SchemaDumper.ignore_tables = /^sqlite_*/
This will ignore all tables starting with "sqlite_" in your database during the schema dump.
Try removing the offending lines from your schema.rb and then issue:
RAILS_ENV=test rake db:reset
which will completely nuke your testing database but you shouldn't care anyway. You are not supposed to run migrations for test environments. Migrations are meant to be small (reversible) steps for environments that hold data (such as production and sometimes staging/dev).
The preferred way to handle your testing DB is to use db:schema:load as part of your testing routine which will of course delete all your DB data.
Again: Please do not try this in development mode (if you have data that you set up manually) and definitely not in production.
Other than that, you could probably delete the whole sqlite_sp_functions table from your SQLite testing DB and get rid of the issue altogether. I don't think that this has anything to do with Rails (nor is Rails generating that). I believe that SQLite created this table and your schema just picks it up (as it should). Apparently the table is used to hold stored procedures of some kind.
I am getting an error when I try to migrate my db. I don't entirely remember how I got here, but I believe I:
created new branch, scaffolded 'Requests', db:migrated, switched back to master, and merged branch
created another branch, did some stuff, db:migrated, and everything was working fine.
pulled from heroku postgres database so i could test out if things worked with actual data. then tried db migrating, but gave me this error:
rake db:migrate
== CreateRequests: migrating =================================================
-- create_table(:requests)
NOTICE: CREATE TABLE will create implicit sequence "requests_id_seq1" for serial column "requests.id"
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "requests" already exists
: CREATE TABLE "requests" ("id" serial primary key, "title" character varying(255), "content" text, "category" character varying(255), "status" character varying(255), "requested_track_id" integer, "created_at" timestamp, "updated_at" timestamp)
Any ideas?
I'm not sure exactly what pull strategy you used, but if we make two reasonable assumptions about your pull strategy:
it doesn't drop the database but just overwrites tables, since this requires less permissions.
it is operating in a sort of 'archive mode', meaning it doesn't drop tables on the destination just because they don't exist on the source. Think rsync; you have to specify --delete to get what might be your expected behavior with that utility.
If your steps are correct, then what happened is you overwrote the schema_migrations table, so Rails thinks you haven't added the table yet, but neither did your heroku pull drop the table because of #2 above.
Don't create another migration!!! This will fail on everyone elses' computer except yours, but will only run on yours once.
Instead, run rails dbconsole and execute something like DROP TABLE 'requests' (I forget the postgres syntax, might not be exactly that). Then you can run your migrations.
There is another way to avoid dropping a table with data in it.
What I do in those cases is to check which migration is failing.
Suppose you have a file db/migrate/20130908214222_create_requests.rb, and for some reason, ActiveRecord failed in the past when stored this migration in its "tracking system".
To be sure that this is the case, just find, in the schema_migrations table, a row containing a number like this example: 20130908214222
If that row does not exist, you just have to insert a new one:
INSERT INTO schema_migrations(
version
) VALUES (
20130908214222
);
Next time you run rake db:migrate, ActiveRecord will omit this step, and will continue migrating to end without complications.
I am in the process of learning Rails and I've ran into an interesting problem tonight.
I was creating a migration that would require an index on the foreign key:
Whenever I would run 'bundle exec rake db:migrate', I would receive this console error:
It looks as if it was trying to create the index before it was creating the reference.
The reason I believe this is because when I change the "subject" reference to a symbol:
The migration then suddenly works as expected!
This may just be the fact that I'm a total newby, but are symbols actually processed faster by Ruby than strings are?
Just curious - thanks!
This isn't a "faster" problem, or a problem of speed. The migrations are executed one line at a time, in order. The way you had it specified before simply didn't create the column correctly, hence when it got to the line where you create the index, the names didn't match up.
My guess is, with the string version it created the column name exactly as you spelled it, "subject" as opposed to subject_id when you use a symbol. Either way, you definitely had a name mismatch between when the column was created, and when the index was being built.
Always use symbols for this in your migrations, and you should be fine. Always check your schema.rb file, or browse the database using a GUI tool, after a migration to make sure the columns got created the way you expect, and with the data types you think they are, and you should be good.
Now that Rails has timestamped migrations, the single version number at the top of /db/schema.rb seems pointless. Sometimes the version number ends up incorrect when dealing with multiple developers or multiple branches.
Does Rails even utilize that :version parameter anymore?
And is there any harm in it being incorrect (as in: it doesn't reflect the timestamp of most recently applied commit)?
Example:
ActiveRecord::Schema.define(:version => 20100417022947) do
# schema definition ...
end
Actually, the version is much more important than this. The code you've cited is actually only a small part of what assume_migrated_upto_version does. The real effect of the migration version is that all prior migrations (as found in the db/migrate directory) are assumed to have been run. (So yes, it does what the function name suggests.)
This has some interesting implications, particularly in the case where multiple people commit new migrations at the same time.
If you version your schema.rb, which is what the Rails team recommends, you're okay. You're 100% guaranteed to have a conflict (the schema version), and the committing/merging user has to resolve it, by merging their changes and setting the :version to the highest of the two. Hopefully they do this merge correctly.
Some projects choose to avoid this continual conflict issue by keeping the schema.rb out of version control. They might rely solely on migrations, or keep a separate version-controlled copy of the schema that they occasionally update.
The problem occurs if someone creates a migration with a timestamp prior to your schema.rb's :version. If you db:migrate, you'll apply their migration, your schema.rb will be updated (but retain the same, higher :version), and everything is fine. But if you should happen to db:schema:load (or db:reset) instead, you'll not only be missing their migration, but assume_migrated_upto_version will mark their migration as having been applied.
The best solution at this point is probably to require that users re-timestamp their migrations to the time of their merge.
Ideally, I would prefer if schema.rb actually contained a list of applied migration numbers rather than an assume-up-to-here :version. But I doubt this will happen -- the Rails team seems to believe the problem is adequately solved by checking in the schema.rb file.
I decided to investigate myself. It turns out that because of the timestamped migrations, the only thing Rails does with that number is assume that the migration with that particular timestamp has already been applied and thus create the appropriate entry in the schema_migration table if it doesn't exist.
from: /lib/active_record/connection_adapters/abstract/schema_statements.rb
def assume_migrated_upto_version(version, migrations_path = ActiveRecord::Migrator.migrations_path)
# other code ...
unless migrated.include?(version)
execute "INSERT INTO #{sm_table} (version) VALUES ('#{version}')"
end
# ...
I'm currently running a Rails migration where I am adding a datatype specific to Postgres, the tsvector. It holds search information in the form that Postgres expects for its built-in text searching capabilities.
This is the line from my migration:
t.column "search_vectors", :tsvector
Everything seems to be working fine, and the search works with it. However, when I opened up schema.rb, this is what I got:
Could not dump table "users" because of following StandardError
Unknown type 'tsvector' for column 'search_vectors'
This is preventing me from running unit tests on the user table, and also strikes me as really dangerous looking given that the schema.rb is supposed to be the authoritative definition of my database.
I notice there are a number of Rails plugins that seem to use the same approach of storing the tsvector like I would expect, such as tsearchable. Am I really just stuck without testing and without an authoritative definition of my database?
FYI for anyone who happens across this page, I fixed this by adding this (actually uncommenting it) to my Rails config:
config.active_record.schema_format = :sql
Have you tried specifying the type as a string instead of a symbol?
t.column "search_vectors", "tsvector"
If that doesn't work then you might need to drop down to database-specific SQL:
def self.up
execute "--Put your PostgreSQL specific SQL statements here"
end