Now that Rails has timestamped migrations, the single version number at the top of /db/schema.rb seems pointless. Sometimes the version number ends up incorrect when dealing with multiple developers or multiple branches.
Does Rails even utilize that :version parameter anymore?
And is there any harm in it being incorrect (as in: it doesn't reflect the timestamp of most recently applied commit)?
Example:
ActiveRecord::Schema.define(:version => 20100417022947) do
# schema definition ...
end
Actually, the version is much more important than this. The code you've cited is actually only a small part of what assume_migrated_upto_version does. The real effect of the migration version is that all prior migrations (as found in the db/migrate directory) are assumed to have been run. (So yes, it does what the function name suggests.)
This has some interesting implications, particularly in the case where multiple people commit new migrations at the same time.
If you version your schema.rb, which is what the Rails team recommends, you're okay. You're 100% guaranteed to have a conflict (the schema version), and the committing/merging user has to resolve it, by merging their changes and setting the :version to the highest of the two. Hopefully they do this merge correctly.
Some projects choose to avoid this continual conflict issue by keeping the schema.rb out of version control. They might rely solely on migrations, or keep a separate version-controlled copy of the schema that they occasionally update.
The problem occurs if someone creates a migration with a timestamp prior to your schema.rb's :version. If you db:migrate, you'll apply their migration, your schema.rb will be updated (but retain the same, higher :version), and everything is fine. But if you should happen to db:schema:load (or db:reset) instead, you'll not only be missing their migration, but assume_migrated_upto_version will mark their migration as having been applied.
The best solution at this point is probably to require that users re-timestamp their migrations to the time of their merge.
Ideally, I would prefer if schema.rb actually contained a list of applied migration numbers rather than an assume-up-to-here :version. But I doubt this will happen -- the Rails team seems to believe the problem is adequately solved by checking in the schema.rb file.
I decided to investigate myself. It turns out that because of the timestamped migrations, the only thing Rails does with that number is assume that the migration with that particular timestamp has already been applied and thus create the appropriate entry in the schema_migration table if it doesn't exist.
from: /lib/active_record/connection_adapters/abstract/schema_statements.rb
def assume_migrated_upto_version(version, migrations_path = ActiveRecord::Migrator.migrations_path)
# other code ...
unless migrated.include?(version)
execute "INSERT INTO #{sm_table} (version) VALUES ('#{version}')"
end
# ...
Related
I've just reviewed a colleague's work and there is an empty EF migration in his PR (up and down methods contain no code). There is another migration after this with DB modifications.
I believe that this empty migration can be deleted on the basis that it does absolutely nothing. There should be a corresponding entry in the _MigrationHistory table, which can also be deleted safely, in my opinion.
My colleagues believe that it should be left in there "just in case".
Are there any EF experts who can say which approach is best, and why?
Тhe truth is somewhere in the middle. You can delete it, but do not delete the migration file directly.
-> Removing A Migration
The following command removes a migration:
[Command Line]
dotnet ef migrations remove
[Package Manager Console]
remove-migration
You will use this command to remove the latest migration. This will remove the class file that was generated for the latest migration and it will also revert the ModelSnapshot file back to the state of the previous migration. If there are no pending migrations, an exception will be raised. If you want to remove a migration that has been committed, you must reverse the migration first (see below).
You should always use commands to remove a migration instead of simply deleting the migration code file, otherwise the snapshot and migrations will fall out of sync with eachother. Then future migrations will be based on a model that is incorrect. Nevertheless, the remove command will recognise if migration files have been deleted and will revert the snapshot accordingly.
If you need to remove a migration that was generated before the most recent one, you must remove all sunsequent migrations first, then adjust the model and then create a new migration to cater for the changes.
Sores: https://www.learnentityframeworkcore.com/migrations
After upgrading to Rails 5 my schema file keeps getting altered when running db:migrate. Rails is changing:
create_table "flightlessons", force: :cascade do |t|
to:
create_table "flightlessons", id: :integer, default: -> { "nextval('lessons_id_seq'::regclass)" }, force: :cascade do |t|
It only occurs on this one model. Why is rails implementing nextval on this particular model? And, why is it getting the model name wrong (lessons_id_seq should be flightlessons_id_seq). Manually changing it to flightlessons_id_seq, however, results in the same no relation error.
PG::UndefinedTable: ERROR: relation "lessons_id_seq" does not exist
To proceed, I simply alter the schema.rb file back to what that line 'should' be. Then, I can migrate or test:prepare or whatever until the next time rails alters it back to using the nextval method.
Thank you for any insight into this.
This is a bit long of an answer, so I've broken it into sections. Buckle up!
My theory
My guess is that your development database does contain the lessons_id_seq sequence, and that its definition of flightlessons.id is set to depend on it (i.e., exactly what Rails is putting into your schema file).
How and why? You likely renamed the lessons table to flightlessons at some point in the past, but that rename didn't change the sequence that the table depended on -- and since schema.rb does not record sequences, the lessons_id_seq sequence does not get copied to your test database, and thus you get this error.
To verify my theory, run rails db and try the following commands:
\d lessons_id_seq
This should return the definition of that sequence. Then, try:
\d flightlessons
And look at the definition of the id column. I expect it to include DEFAULT nextval('lessons_id_seq').
Fixes
The easiest way to fix this is to switch to using structure.sql instead of schema.rb (see the docs). This will carry over the exact state of your database and avoid any interference or interpretation by Rails, which is what's causing your current issue. I always recommend structure.sql for production systems.
However, you can also go into your development database and change the sequence name:
ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;
ALTER TABLE flightlessons ALTER COLUMN id SET DEFAULT nextval('flightlessons_id_seq');
This would be a terrible idea on a production system, but if your issue is just local, it should rectify your current database state with your schema.rb and thus address your current problem. You may wish to encode that into a migration, if you want rails db:drop db:create db:migrate to work on a fresh app.
Why now?
The behavior where Rails is dumping out the default value for your table's primary key may very well be new in Rails 5. Previously, Rails may have just trusted that your ID column had a sane default, and ignored whatever value it actually saw. But I haven't done the research to see if that's true or not.
The simplest fix is to just to rename the sequence in production to match the current table name. E.g. in a production Rails console:
ActiveRecord::Base.connection.execute("ALTER SEQUENCE lessons_id_seq RENAME TO flightlessons_id_seq;")
Turns out it's fine to just rename it, if you haven't done anything fancy with the sequence (like implementing your own Postgres function that references it by name).
Apparently the table points to the sequence by ID, not by name, so the rename is instant and with no ill effects that we could see. More details here: https://dba.stackexchange.com/questions/265569/how-can-i-safely-rename-a-sequence-in-postgresql-ideally-without-downtime
We tried it on staging first, and verified that the ID sequence kept on ticking after making the change in staging and production. Everything just worked.
(Also see Robert Nubel's fantastic answer for more details on what's going on.)
One of my migration files is referencing another table/model that will will be created further down the migration sequence.
Postgres doesn't like that:
PG::UndefinedTable: ERROR: relation "users" does not exist
So I wonder if there are any potential problems to manually reorder the migration files (by inventing new timestamps/prefixes)?
The affected tables are already down migrated.
When you run rake db:migrate command it compares schema_migrations table and migration files located in db/migrate folder. All the migrations which were not executed receive MigrationClass#up call then.
So starting from the point when your code is already published and/or migrations are run by other users, changing your migrations timestamps/names may lead to unprocessable migration procedure (as schema_migrations will treat a migration with changed timestamp as new, unprocessed one, and try to process it "again"). Possible workaround for this would be to comment the contents of up method for a while and uncomment it back after migrations are done. For fun you can also manipulate schema_migrations table directly from your db console (adding or removing necessary records). Both of these ways smells like a hack though.
Until then... Everything should work flawlessly.
This is what worked for me for this same situation, even though it's a hack.
Rails runs migrations in order of the timestamp, and Rails tracks which migrations have been run by the timestamp part of the migration file name, but not by the rest of the file name. So if you need to change the order of two migrations because the earlier one references the later one you can simply switch the 14 digit timestamp portion of the filenames by renaming both migration files. If the timestamp is off by even one digit Rails will think it's a new migration so write them down before changing them.
Working on a 3.0 rails project for a client and I have a sensitive migration that I need to run off a live production server. Its essentially suppose to down case all the State abbreviations in the DB, FL -> fl, PA -> pa etc...
I can't test locally due to restrictions:
does calling the wording of the migration effect anything? I know it does with add and create etc but not sure when updating info like this.
rails g migration UpdateStateAbbreviation
def self.up
say_with_time "Updating states abbreviation..." do
State.find(:all).each do |s|
tmp = s.abbreviation.downcase
s.update_attribute :abbreviation, tmp
end
end end
Rake db:migrate
One very important rule with migrations is to never reference models in your migrations. This might seem like an academic concern, but at some point in the future you may not have a State model at all, and when you remove app/models/state.rb then this migration will not work.
A properly constructed migration will execute properly regardless of changes in the future. Whatever it does may be later un-done, there's nothing wrong with that, but setting it up for failure is never a good idea.
You can do this downcasing operation in your database using a string function and something like:
execute "UPDATE states SET abbreviation=LOWER(abbreviation)"
Using models in the migration causes all sorts of problems. This goes for using your model to pre-populate certain key records as well. Use seeds.rb if you must, or even better, a rake task to do it for you.
As a note, if you can't test locally you have a very flawed development process. You should always run and test your migrations, both up and down where applicable, to ensure that they work correctly. Where you can't get actual production data for reasons of security or privacy, work with your DBA to get a scrubbed, non-sensitive version for testing purposes. State names should not be confidential, for instance.
I am in the process of learning Rails and I've ran into an interesting problem tonight.
I was creating a migration that would require an index on the foreign key:
Whenever I would run 'bundle exec rake db:migrate', I would receive this console error:
It looks as if it was trying to create the index before it was creating the reference.
The reason I believe this is because when I change the "subject" reference to a symbol:
The migration then suddenly works as expected!
This may just be the fact that I'm a total newby, but are symbols actually processed faster by Ruby than strings are?
Just curious - thanks!
This isn't a "faster" problem, or a problem of speed. The migrations are executed one line at a time, in order. The way you had it specified before simply didn't create the column correctly, hence when it got to the line where you create the index, the names didn't match up.
My guess is, with the string version it created the column name exactly as you spelled it, "subject" as opposed to subject_id when you use a symbol. Either way, you definitely had a name mismatch between when the column was created, and when the index was being built.
Always use symbols for this in your migrations, and you should be fine. Always check your schema.rb file, or browse the database using a GUI tool, after a migration to make sure the columns got created the way you expect, and with the data types you think they are, and you should be good.