I am a little confused how migrations affect schema.rb file?
For example if I write a migration to rename a table column and run that migratin and even do a schema:load rake task too, then when I open my schema.rb file should it automatically be changed to have that new column name? or should I manually change it in there too?
Also my create_table*.rb files that create the original tables. They automatically have a t.timestamp field defined in them that creates those two created_at and updated_at fields in the schema, so if I want to remove those is it enough to just alter the create_table*.rb file and take out t.timestamp from them? and run the migration? or again I should manually alter shcema.rb file too?
So if someone can explain a little bit how there work together would be great.
The schema file is automatically altered when you run migrations. You should never have to manually edit it.
See this Rails Guide for moer information.
Related
After upgrading to Rails 5 my schema file keeps getting altered when running db:migrate. Rails is changing:
create_table "flightlessons", force: :cascade do |t|
to:
create_table "flightlessons", id: :integer, default: -> { "nextval('lessons_id_seq'::regclass)" }, force: :cascade do |t|
It only occurs on this one model. Why is rails implementing nextval on this particular model? And, why is it getting the model name wrong (lessons_id_seq should be flightlessons_id_seq). Manually changing it to flightlessons_id_seq, however, results in the same no relation error.
PG::UndefinedTable: ERROR: relation "lessons_id_seq" does not exist
To proceed, I simply alter the schema.rb file back to what that line 'should' be. Then, I can migrate or test:prepare or whatever until the next time rails alters it back to using the nextval method.
Thank you for any insight into this.
This is a bit long of an answer, so I've broken it into sections. Buckle up!
My theory
My guess is that your development database does contain the lessons_id_seq sequence, and that its definition of flightlessons.id is set to depend on it (i.e., exactly what Rails is putting into your schema file).
How and why? You likely renamed the lessons table to flightlessons at some point in the past, but that rename didn't change the sequence that the table depended on -- and since schema.rb does not record sequences, the lessons_id_seq sequence does not get copied to your test database, and thus you get this error.
To verify my theory, run rails db and try the following commands:
\d lessons_id_seq
This should return the definition of that sequence. Then, try:
\d flightlessons
And look at the definition of the id column. I expect it to include DEFAULT nextval('lessons_id_seq').
Fixes
The easiest way to fix this is to switch to using structure.sql instead of schema.rb (see the docs). This will carry over the exact state of your database and avoid any interference or interpretation by Rails, which is what's causing your current issue. I always recommend structure.sql for production systems.
However, you can also go into your development database and change the sequence name:
ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;
ALTER TABLE flightlessons ALTER COLUMN id SET DEFAULT nextval('flightlessons_id_seq');
This would be a terrible idea on a production system, but if your issue is just local, it should rectify your current database state with your schema.rb and thus address your current problem. You may wish to encode that into a migration, if you want rails db:drop db:create db:migrate to work on a fresh app.
Why now?
The behavior where Rails is dumping out the default value for your table's primary key may very well be new in Rails 5. Previously, Rails may have just trusted that your ID column had a sane default, and ignored whatever value it actually saw. But I haven't done the research to see if that's true or not.
The simplest fix is to just to rename the sequence in production to match the current table name. E.g. in a production Rails console:
ActiveRecord::Base.connection.execute("ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;")
Turns out it's fine to just rename it, if you haven't done anything fancy with the sequence (like implementing your own Postgres function that references it by name).
Apparently the table points to the sequence by ID, not by name, so the rename is instant and with no ill effects that we could see. More details here: https://dba.stackexchange.com/questions/265569/how-can-i-safely-rename-a-sequence-in-postgresql-ideally-without-downtime
We tried it on staging first, and verified that the ID sequence kept on ticking after making the change in staging and production. Everything just worked.
(Also see Robert Nubel's fantastic answer for more details on what's going on.)
One of my migration files is referencing another table/model that will will be created further down the migration sequence.
Postgres doesn't like that:
PG::UndefinedTable: ERROR: relation "users" does not exist
So I wonder if there are any potential problems to manually reorder the migration files (by inventing new timestamps/prefixes)?
The affected tables are already down migrated.
When you run rake db:migrate command it compares schema_migrations table and migration files located in db/migrate folder. All the migrations which were not executed receive MigrationClass#up call then.
So starting from the point when your code is already published and/or migrations are run by other users, changing your migrations timestamps/names may lead to unprocessable migration procedure (as schema_migrations will treat a migration with changed timestamp as new, unprocessed one, and try to process it "again"). Possible workaround for this would be to comment the contents of up method for a while and uncomment it back after migrations are done. For fun you can also manipulate schema_migrations table directly from your db console (adding or removing necessary records). Both of these ways smells like a hack though.
Until then... Everything should work flawlessly.
This is what worked for me for this same situation, even though it's a hack.
Rails runs migrations in order of the timestamp, and Rails tracks which migrations have been run by the timestamp part of the migration file name, but not by the rest of the file name. So if you need to change the order of two migrations because the earlier one references the later one you can simply switch the 14 digit timestamp portion of the filenames by renaming both migration files. If the timestamp is off by even one digit Rails will think it's a new migration so write them down before changing them.
What is happening under the hood when executing this?
rails generate migration create_menu_categories_and_menu_items
Why can't I just create a file in the migrations directory? It seems I have to use the generate command in order for it to actually run, but the only file I see changed is the file created by the above command.
You can, but you need to get the time stamp at the beginning of the file name, so that rails can figure out that it's a "pending migration" and run it. It does this by storing the time stamp (or version in migration parlance) in a special database table (schema_migrations) and anything newer than the last run migration is pending.
Otherwise there's nothing special about the file itself
You dont even have to create a separate migration file. You can just add a column to your table by adding the object to to your current migration class like this:
t.string :name
Then run a rake db:migrate
Hope this helps.
Like some others, I am having trouble with an error
"ActiveRecord::StatementInvalid: SQLite3::SQLException: object name
reserved for internal use: sqlite_sp_functions: CREATE TABLE
"sqlite_sp_functions" ("name" text, "text" text)"
when running rake test on a Rails project.
The offending lines in schema.rb are:
create_table "sqlite_sp_functions", id: false, force: true do |t|
t.text "name"
t.text "text"
end
The suggestions on the previous query about this involved editing schema.rb or deleting that file & regenerating it, but schema.rb (and the offending code) are regenerated on each migration (plus, I don't want to delete Rails-generated code without knowing the implications), so that's not really a satisfactory solution.
So what is the sqlite_sp_functions table for, and how can I get Rails to generate a schema.rb file that doesn't break rake test for the project?
as per https://stackoverflow.com/a/25077833/601782:
Add the following config line to config/application.rb or in config/environments/development.rb:
ActiveRecord::SchemaDumper.ignore_tables = /^sqlite_*/
This will ignore all tables starting with "sqlite_" in your database during the schema dump.
Try removing the offending lines from your schema.rb and then issue:
RAILS_ENV=test rake db:reset
which will completely nuke your testing database but you shouldn't care anyway. You are not supposed to run migrations for test environments. Migrations are meant to be small (reversible) steps for environments that hold data (such as production and sometimes staging/dev).
The preferred way to handle your testing DB is to use db:schema:load as part of your testing routine which will of course delete all your DB data.
Again: Please do not try this in development mode (if you have data that you set up manually) and definitely not in production.
Other than that, you could probably delete the whole sqlite_sp_functions table from your SQLite testing DB and get rid of the issue altogether. I don't think that this has anything to do with Rails (nor is Rails generating that). I believe that SQLite created this table and your schema just picks it up (as it should). Apparently the table is used to hold stored procedures of some kind.
If I edit my shema, and run db:migrate the database doesn't changed, but if I clear to version 0, and recall migration it's works, but I lost all database data.
What's wrong?
That's how db:migrate works. It maintains a table in the database called schema_migrations that keeps track of the migration timestamps (i.e. if your file is called 20090807152224_create_widgets.rb, the 20090807152224 part is the timestamp -- and the line that will get added to your schema_migrations table).
You're not supposed to modify the schema.rb file by hand -- that file gets autogenerated as a result of db:migrate.
The thinking in rails is that if you want to make a change to your schema, you're going to generate a new migration with those changes and then run db:migrate (which, as a result, will update the schema.rb file appropriately).
When you say you are updating your schema, does that mean you're updating the db/schema.rb file, or actual migrations?
If you're updating the schema.rb file, you should note that it will have no effect because the file is auto-generated.
See the comment at the top of the file:
# This file is auto-generated from the current state of the database. Instead of editing this file,
# please use the migrations feature of Active Record to incrementally modify your database, and
# then regenerate this schema definition.
It sounds like you were changing a migration file.
Don't change migration files. Add new ones. You can have migrations that change say, the type of a column. There are times when changing old migrations may be helpful, but don't do it unless you know the consequences. As others have mentioned, don't change the schema either, but I don't think you were doing that.