I'm looking to generate a migration file from the schema.rb. is it possible?
I have many migration files at the moment and would like to combine everything into one master migration file.
I also think i may have accidentally deleted a migration file at some point.
thanks for any help
You could copy and paste schema.rb into a migration and back-date it (e.g. change the date) so that no existing databases will run it. After you create this migration you can delete all your old migrations.
I disagree with Andrew that you should never delete migrations. Migrations break unexpectedly all the time based on model classes changing and it is very non-trivial to fix them. Since I'm sure you are using version control, you can always look back in the history if you need them for reference.
There's no need to do this. For new installations you should be running rake db:schema:load, not rake db:migrate, this will load the schema into the database, which is faster than running all the migrations.
You should never delete migrations, and certainly not combine them. As for accidentally deleting one, you should be using a version control system, such as Git.
Related
I added a migration in branch "add_dogs" with migration db/migrate/20221220155010_create_dogs.rb, and ran db:migrate.
Later on, I changed branches (without a merge), and ultimately abandoned the "new_dogs" branch.
Later later on, I checked out "add_cats" branch with db/migrate/20221101010101_create_cats.rb, and ran db:migrate. So far, all is well.
But then I tweak the "add_cats" migration (before committing anything), and ran db:rollback so I can run it again. I get this error:
ActiveRecord::UnknownMigrationVersionError:
No migration with version number 20221220155010.
I can still run db:migrate on new migrations just fine, but not db:rollback or db:migrate:redo.
This makes sense, because the database has a record of applying 20221220155010, but that migration file no longer exists, so there is no way to roll it back.
How can I get past this?
Here are three ways to deal with a missing migration file, depending on your needs and access:
For a quick temporary fix, you can roll back just the migration you're currently editing so you can run it again. This may be useful if the other migration is still in the pipeline on the other branch and both eventually will get merged.
rake db:migrate:down VERSION=20230101010101
// This is the version of the migration you WANT to rollback, not the missing one.
If the missing migration will never come back, you want a permanent fix. The simplest way is to remove that record from the database. You can do this from your favorite SQL client, rails console, etc. (I suppose you could even write a migration to do that, but that seems mighty sketchy.)
DELETE FROM schema_migrations WHERE version = '20221220155010'
-- This is the version of the migration that is MISSING, not the one you are working on.
If you don't have direct access to the database for whatever reason, you can give Rails a placebo to rollback. Ensure the timestamp in the filename matches the missing migration's version number.
Create a file named db/migrations/20221220155010_just_kidding.rb:
class JustKidding < ActiveRecord::Migration
def change
# nothing to see here.
end
end
Then, rails db:rollback will roll back that no-op migration and delete 20221220155010 from the schema_migrations table. You can now delete the placebo migration forever and you'll be in good shape as far as running migrations and rollbacks.
However...don't forget that the effects of the old migration are still in your schema. Maybe you're stuck with a new, unused 'dogs' table or an extra column on a table. Maybe that's benign on your dev box, but you certainly don't want that cruft on a production environment. All the advice in this answer assumes you're on a throw-away environment and that the effects of the old migration aren't a problem. Tearing down your whole database and rebuilding may become a more attractive option in this case.
One of the realy take-aways here is... don't let this happen in the first place! Ideally, you should rollback any new, uncommitted migrations before changing away from a branch. But...things happen...
p.s. If there is a way to do this from the command line, I'd love to learn it. I'm imagining something like rails db:migrate:delete VERSION=20230101010101 might be handy in a hackish kind of way.
I have several dozens Rails DB migrations which were written over a year. Is there a way to aggregate them to one migration so that I will just see a full DDL statement for the database as it exists now? I just need the current snaphot without all the history of how we got to it.
It is possible, but probably not a good idea to aggregate the migrations!
Maybe ask:
Why do you want to do this?
How often do you really need to migrate all the way to VERSION=0 and then back up again?
Is something really broken? (if not, then don't fix it)
I've had the same problem once.. I ended up just re-ordering my migrations, because changes in the schema caused it to not correctly migrate up/down anymore. I would be hesitant to do that again.
If you have migrations which just add fields or indexes, then maybe you can combine them with the main migration for the model -- but beware that you can't reproduce old situations anymore, e.g. older DB-dumps may not be compatible with what migration number they should be compatible with -- that is probably the biggest argument against aggregating...
Technically, you can dump the schema and then load it directly - that is one way:
rake db:schema:dump
then create a single new migration with the contents of the schema dump file db/schema.rb
Here are some similar questions:
Rebase Rails migrations in a long running project
Deleting/"Rebasing" rails migrations
Way to "flatten" Rails migrations?
Should I flatten Rails migrations?
P.S.: I found it useful to stick with the old migration numbering scheme, where the migrations do not use timestamps - for me this works better (is easier to see in which order they are).
e.g. in your config/application.rb file:
config.active_record.timestamped_migrations = false
You should never be using all the migrations to get a database up and running. The current schema.rb is always what the DB looks like 'presently'.
It's good practice to periodically just truncate your migrations if you have a ton of them in there. We finally did that with one of our larger applications, removing a good 50 migrations from the folder because the only thing that matters is schema.rb. Migrations are just that, a way to migrate and make changes to an existing state of the database. They should only ever have to be run once.
You can simply load the current schema into the DB.
rake db:schema:load RAILS_ENV=[production, test, etc.]
This will take the schema.rb file's version of the schema, and load it into the DB without running individual migrations.
NOTE: if you have migrations that put data into the DB (e.g. default values, for example), that data will not be added to the DB.
If you need to load default values into your DB, that might be better done via a custom rake task, independent of migrations.
Should we include schema.rb while commiting to GIT? or should we ignore it? what is the right approach?
Well the standard schema.rb file for Rails 2 has this at the end of the comment block at the top of the file:
# It's strongly recommended to check this file into your version control system.
The Rails 3 schema.rb that I have kicking around says the same thing. I think the comment says it all.
Update in response to comments:
Yes, mistakes can be made and you can get conflicting changes and bugs mangling your schema.rb but that's why you want it under revision control, revision control allows you to keep track of everything and backup when needed. There is only one thing in your entire source tree that specifies your database schema and that is schema.rb. Your database schema absolutely is a critical artifact and anything that important needs to be tracked in revision control.
Any update/merge problems with schema.rb should be sorted out just by sorting out your conflicting migrations so schema.rb will get fixed as a side effect of fixing the real problem.
Yes, schema.rb is a generated file but it is only generated in the sense that your text editor generates your pancake.rb model file or an unedited scaffold file is generated.
Yes, you could rebuild your schema.rb file by building a new database and then running all of your migrations. But, you should clear out your old migrations now and then to avoid having to check hundreds of migration files every time you rake db:migrate so "rebuild and run all the migrations" often isn't an option in a highly active project.
Yes. The schema file is used to configure your database when using rake db:reset and other commands. Migrations should only be used when changing the database schema and will always result in a new schema file.
Well, it's not included on .gitignore by default. So, I think that you would not have a problem including it(I do in my projects, without any problem).
I don't commit this file to Git, because it create when I launch rake db:migrate.
If I will commit this file to Git, I cannot pull new changes from server after each db:migrate.
I have been running a big Rails application for over 2 years and, day by day, my ActiveRecord migration folder has been growing up to over 150 files.
There are very old models, no longer available in the application, still referenced in the migrations. I was thinking to remove them.
What do you think? Do you usually purge old migrations from your codebase?
The Rails 4 Way page 177:
Sebastian says...
A little-known fact is that you can remove old migration files (while
still keeping newer ones) to keep the db/migrate folder to a
manageable size. You can move the older migrations to a
db/archived_migrations folder or something like that. Once you do trim
the size of your migrations folder, use the rake db:reset task to
(re-)create your database from db/schema.rb and load the seeds into
your current environment.
Once I hit a major site release, I'll roll the migrations into one and start fresh. I feel dirty once the migration version numbers get up around 75.
I occasionally purge all migrations, which have already been applied in production and I see at least 2 reasons for this:
More manageable folder: it is easier to spot a new migration.
Cleaner text search results: global text search within a project does not lead to tons of useless matches because of some 3-year-old migration when someone added or removed some column which anyway does not exist anymore.
They are relatively small, so I would choose to keep them, just for the record.
You should write your migrations without referencing models, or other parts of application, because they'll come back to you haunting ;)
Check out these guidelines:
http://guides.rubyonrails.org/migrations.html#using-models-in-your-migrations
Personally I like to keep things tidy in the migrations files. I think once you have pushed all your changes into prod you should really look at archiving the migrations. The only difficulty I have faced with this is that when Travis runs it runs a db:migrate, so these are the steps I have used:
Move historic migrations from /db/migrate/ to /db/archive/release-x.y/
Create a new migration file manually using the version number from the last run migration in the /db/archive/release-x.y directory and change the description to something like from_previous_version. Using the old version number means that it won't run on your prod machine and mess up.
Copy the schema.rb contents from inside the ActiveRecord::Schema.define(version: 20141010044951) do section and paste into the change method of your from_previous_version changelog
Check all that in and Robert should be your parent's brother.
The only other consideration would be if your migrations create any data (my test scenarios contain all their own data so I don't have this issue)
Why? Unless there is some kind of problem with disk space, I don't see a good reason for deleting them. I guess if you are absolutely certain that you are never going to roll back anything ever again, than you can. However, it seems like saving a few KB of disk space to do this wouldn't be worth it. Also, if you just want to delete the migrations that refer to old models, you have to look through them all by hand to make sure you don't delete anything that is still used in your app. Lots of effort for little gain, to me.
See http://edgeguides.rubyonrails.org/active_record_migrations.html#schema-dumping-and-you
Migrations are not a representation of the database: either structure.sql or schema.rb is. Migrations are also not a good place for setting/initializing data. db/seeds or a rake task are better for that kind of task.
So what are migrations? In my opinion they are instructions for how to change the database schema - either forwards or backwards (via a rollback). Unless there is a problem, they should be run only in the following cases:
On my local development machine as a way to test the migration itself and write the schema/structure file.
On colleague developer machines as a way to change the schema without dropping the database.
On production machines as a way to change the schema without dropping the database.
Once run they should be irrelevant. Of course mistakes happen, so you definitely want to keep migrations around for a few months in case you need to rollback.
CI environments do not ever need to run migrations. It slows down your CI environment and is error prone (just like the Rails guide says). Since your test environments only have ephemeral data, you should instead be using rake db:setup, which will load from the schema.rb/structure.sql and completely ignore your migration files.
If you're using source control, there is no benefit in keeping old migrations around; they are part of the source history. It might make sense to put them in an archive folder if that's your cup of coffee.
With that all being said, I strongly think it makes sense to purge old migrations, for the following reasons:
They could contain code that is so old it will no longer run (like if you removed a model). This creates a trap for other developers who want to run rake db:migrate.
They will slow down grep-like tasks and are irrelevant past a certain age.
Why are they irrelevant? Once more for two reasons: the history is stored in your source control and the actual database structure is stored in structure.sql/schema.rb. My rule of thumb is that migrations older than about 12 months are completely irrelevant. I delete them. If there were some reason why I wanted to rollback a migration older than that I'm confident that the database has changed enough in that time to warrant writing a new migration to perform that task.
So how do you get rid of the migrations? These are the steps I follow:
Delete the migration files
Write a rake task to delete their corresponding rows in the schema_migrations table of your database.
Run rake db:migrate to regenerate structure.sql/schema.rb.
Validate that the only thing changed in structure.sql/schema.rb is removed lines corresponding to each of the migrations you deleted.
Deploy, then run the rake task from step 2 on production.
Make sure other developers run the rake task from step 2 on their machines.
The second item is necessary to keep schema/structure accurate, which, again, is the only thing that actually matters here.
It's fine to remove old migrations once you're comfortable they won't be needed. The purpose of migrations is to have a tool for making and rolling back database changes. Once the changes have been made and in production for a couple of months, odds are you're unlikely to need them again. I find that after a while they're just cruft that clutters up your repo, searches, and file navigation.
Some people will run the migrations from scratch to reload their dev database, but that's not really what they're intended for. You can use rake db:schema:load to load the latest schema, and rake db:seed to populate it with seed data. rake db:reset does both for you. If you've got database extensions that can't be dumped to schema.rb then you can use the sql schema format for ActiveRecord and run rake db:structure:load instead.
Yes. I guess if you have completely removed any model and related table also from database, then it is worth to put it in migration. If model reference in migration does not depend on any other thing, then you can delete it. Although that migration is never going to run again as it has already run and even if you don't delete it from existing migration, then whenever you will migrate database fresh, it cause a problem.
So better it to remove that reference from migration. And refactore/minimize migrations to one or two file before big release to live database.
I agree, no value in 100+ migrations, the history is a mess, there is no easy way of tracking history on a single table and it adds clutter to your file finding. Simply Muda IMO :)
Here's a 3-step guide to squash all migrations into identical schema as production:
Step1: schema from production
# launch rails console in production
stream = StringIO.new
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, stream); nil
stream.rewind
puts stream.read
This is copy-pasteable to migrations, minus the obvious header
Step 2: making the migrations without it being run in production
This is important. Use the last migration and change it's name and content. ActiveRecord stors the datetime number in it's schema_migrations table so it knows what it has run and not. Reuse the last and it'll think it has already run.
Example: rename 20161202212203_this_is_the_last_migration -> 20161202212203_schema_of_20161203.rb
And put the schema there.
Step 3: verify and troubleshoot
Locally, rake db:drop, rake db:create, rake db:migrate
Verify that schema is identical. One issue we encountered was datetime "now()" in schema, here's the best solution I could find for that: https://stackoverflow.com/a/40840867/252799
So this is my first real Ruby on Rails project. I've learned my lesson -- I didn't make all changes using migrations so things are a bit messed up.
What's the best way to start over with new migration files and rebuild the schema, etc? My project is too far along to rebuild the entire project, but not far enough along to where I care about losing the migrations I have thus far. I also don't mind losing the data in the database. I was trying to rollback to the beginning but some of it is failing.
I know this is a bad state to be in, but lesson learned.
EDIT:
I just deleted all the migrations files and rebuilt the schema file with db:schema:dump.
I assume this puts me in a clean state with my existing database, just lost migrations.
if you want to migrate some steps back you can
rake db:rollback STEP=2
That command will migrate your database 2 migrations back.
If you need more help with rake commands, jus type
rake -T
That command will list all the tasks you have in you application.
If you are not concerned about losing data then do
rake db:purge
It should just drop your database
Your schema.rb file should contain the actual schema from your database. You could use it as a starting point to create you migrations. You could create a new migration for each table with the :force => true parameter to overwrite the old table. Afterwards you could just delete the old migrations (you would probably also need to delete their entries from schema_migrations table).
Another options would be just updating the old migrations to match your current schema.