What are best practices for migration management?
For instance, when debugging a migration, do you edit the original migration or add an edit migration before committing to the repository? Thanks!
I tend to edit the original migration as long as it is a) the last migration and b) not in source control. This presents a clean migration path for all other consumers of the code. The important thing is that your migrations should be able to run without error from whatever database state is the earliest that you can expect to encounter.
Start testing your migrations.
http://blog.carbonfive.com/2011/01/27/start-testing-your-migrations-right-now/
If you are working with multiple developers, editing an existing migration can be dangerous.
If your coworker has already migrated the original migration, then when he updates he will not pick up the new code and hilarity will ensue. This is a very difficult issue to track down. Error on the side of being a good denizen and just create an updated migration.
Only ever edit an existing migration if you can verify that it has not yet been run by other developers or some automated build setup. To be on the safe side, you shouldn't edit a commited migration file unless the bug was so severe that the migration wouldn't run in the first place (in which case why did you commit it?)
Also, special care has to be taken with migrations calling code from elsewhere in your application so that when they are run, they are run using the correct version of the code. Otherwise subtle changes in your models can really screw up your earlier migrations.
Even after reading this and the answers below, I just learned the hard way. Not to edit the original. You end up losing track your development process and it's hard to get back under control.
Related
I've started working with rails recently and I have noticed that during development you can end up with a lot of migration files as your ideas change and your database with it. Is there a convention in rails to avoid accumulating a lot of migrations and deleting redundant ones (such as one migration adding a table and then the next deleting it)? Or, is it preferred to keep all your migrations and never delete them? Thanks.
If you are working on a team it is preferred and you always should keep all the migrations.
Suppose you push a migration to Git and later found out that it needs some change. So you revert and delete the migration and create a new one and push the new changes. It works well in your machine. But if somehow your previous faulty migrations are run by any of your team mate and he pull the new code he will have to run the new migrations which will break the applications in your teammates machine.
But if you are working solo and want to get a bit tidy then you can carefully delete migrations after reverting them (Not preferred).
The primary job of migration files is to alter the database schema. Deleting redundant ones (such as one migration adding a table and then the next deleting it) will not cause any problems and it free up some disk space and slug size.
Is having a long list of migration files often common when building a web application? I seem to be adding up a long list of migration files because I keep forgetting or keep thinking of adding an extra column to a 'already' migrated database table.
Having a long list of migration files is normal. It's one of the best features of rails. Think of them as layers(like an onion) that you stack on top of each other. If you add a new column or table and then you decide that you don't want it anymore you can rollback(peel away) the latest changes. As long as you have the migration files you can move back and forth easily(don't recommend moving much but you get the point). REMEMBER DO NOT DELETE migration files once they are raked unless you do a rollback. When you rollback and delete a migration file make absolutely SURE you are at the right layer(rollback point).
why? because for example when someone clones your app and runs your migration file it goes through all the migration files from beginning to end. if something in the middle is messed up or deleted you won't be able to create the database because it goes through ALL the steps. Hope it helps.
It may be bad habit, but I migrate down using rake db:migrate VERSION=0, then change the respective migration that has (for instance) the users, and finally migrate the database using rake db:migrate. That way i have less of a mess, and know exactly which migration does what to what model. It's cleaner, but I guess this technique can be used only at the beginning of the webapp.
Hope this helps.
One more point to be noted, from rails docs
If you need to create the application database on another system, you should be using db:schema:load, not running all the migrations from scratch. The latter is a flawed and unsustainable approach (the more migrations you'll amass, the slower it'll run and the greater likelihood for issues).
It's strongly recommended to check this file(schema.rb) into your version control system.
IMO, if you do the migrations correctly it should have to have a long list of migrations. Because every little change you have to do via migration. As you said, proper way is to add a new migration when you need to alter a table.
So as I mentioned I believe having more and more migrations means you are doing it correct. (Because most of the time when you need alterations to to an existing table you simple cannot drop the table and re-create it).
But having said that it always a good idea to run rake db:migrate every now and then (for an isolated db) just to make sure your migrations are working as a group.
I'm working on a project that's accumulated hundreds of migrations, and I'm unsure what to do with them long term. I suppose they're no hurting anything, but it seems strange to keep around a bunch of old files for incremental migrations, some of them creating tables that are later removed.
So far, I've seen three possibilities:
Leave them alone. They're not hurting anything.
Just delete them. I don't see much harm in doing this, since a new developer would be probably be starting with schema load anyway, not migrations.
Delete them all, create a new one with a timestamp matching an old merge, and create a new merge from your schema. This seems very clean, but I'm not sure who would actually use it.
I'm inclined to just delete them, but I'm curious if there's a big pitfall I'm missing.
In my opinion, as soon as every database on the project, especially the production, are at least at version '201xxxxxxxx', it should be fine to delete migrations before that version. They are not technically necessary anymore.
After that, if you want to play archaeology with your database history, you can still use your version control system.
With Git for example, you can use the following commands to have a quick look over the past :
git log --name-only db/migrate/ #to list commit involving migrations + migration filename
git show xxxxx db/migrate #to see the code of commit xxxxx's migration(s)
Alternatively, you can browse repository history of schema.rb, identify a commit and see the corresponding migration content with the command above.
If you prefer to have a lighter db/migrate and use Version control, I would go for a little cleanup.
If you find it more convenient to have the whole migrations history directly available because it's easier to browse, I would go for option 1.
Note: it is very likely that old migrations do not make sense with the current application code. For example, some migrations may refer to class or methods that don't exist anymore.
Using version control to checkout the application at the time the migration was written could also avoid some confusion.
Personally, I lean toward option 1: It's generally true that at some point in any project, the schema is what matters, and the migrations are just a curiosity, but you're right when you say they're not hurting anything. Theoretically the old migrations could be useful for someone who wanted to go back and see how the database was organized at some point in the past.
I don't know of any serious pitfalls in deleting them, but I also don't see an advantage to doing so, unless it's the saving of time scrolling past them when you want to edit a new migration.
I don't think the effort of putting together a single migration that duplicates the schema is beneficial - it's extra work, and that's what the schema is for anyway.
If you work on a project large enough for long enough, there will come a time when you look at all of those extra migrations with disdain and wonder how so many can exist.
You think to yourself, "just delete them...they don't do anything." This is a perfectly logical and normal thought process (especially as a Rails developer, we just love to minimize code and make things efficient!), but don't let the dark side tempt you.
By deleting migrations, you are deleting the historical record of your application's data model, and worse, the logical path you took to get to your current model. This history can help you remember why you did what you did, and didn't do what you didn't do.
Yes, we've all been guilty of deleting migrations from time to time. But you must resist the temptation as the net benefit would only be a few kB and a cleaner migration folder.
Remember:
Those who delete migrations are condemned to repeat them!
In one of our former projects we dropped all migrations, created a new one which would truncate schema_migrations table with manual sql and then copied db/schema.rb contents to it.
Surely this migration is irreversible, however it allowed us to get rid of hundreds of old valueless migrations but still be able to re-create db not from db schema only but from migrations as well.
How do you go about changing column names and types in your rails app? Do you create a new migration to make the changes, or do you rollback, edit your migration file, and then migrate again?
What's the "proper" way to do this in Rails?
It sort of depends on when this happened in your development cycle, If you recently made the change and haven't pushed it out into a public repo, then you indeed might want to do the rollback thing and then edit the migration files and migrate again, just to keep things clean. But if it's a change to a migration that's a few migrations back then you should create a new migration that changes the rows and columns to the "new" old values.
Well, to undo a migration you typically want to roll it back:
bundle exec rake db:rollback
Where VERSION= can also be specified. If you wanted to change it to something entirely new, you would make a new migration. Typically you shouldn't be touching old migrations at all.
Rails gives you the choice of creating a migration and changing it a various of ways. So there is no "ruby developers" technique of choice.
However, every time you create a migration, a file is created, and it represents the history of your development. There are a lot of cases that a simpler file should be uselfull, like a code that other ruby beginners would have to look at and modify. On other cases may be necessary to an advanced developer understand changes and improvements made on the code, specially if it is a code running for a long time (some years maybe).
In Rails, migrations have a down method by default for reverting a migration. In what scenario would I ever want to revert a migration, though?
Some thoughts:
Whether in development or production, I always have a snapshot of my database to go back to, before I even run the migrations. Especially for migrations which perform data conversion, I find in most cases that reverting a snapshot is even faster than reverting a migration. (So I would never do it in a rush!)
If a migration were to fail, it would either:
fail with an exception on a non-transactional database, and thus leave the database broken, or
fail with an exception and roll back the transaction, and thus there would be no need to revert otherwise.
If the changes made are in production (or late in development), and later turn out to be a mistake, I would fix my mistake in a new migration. I would not revert the old one. In development, I'd simply delete the migration.
I also find that the down method introduces extra code in which I repeat myself, and thus may introduce new bugs. This is against the DRY principle.
So I'm curious about the pros, because I can't think of any.
In development, it is easy and fast to incrementally "improve" migrations by using the down method automatically. Eg
Create a migration and migrate to it
Realize you need to make a change
Migrate to the ver prior to your new migration by using db:migrate with a version
Improve/fix your migration
Rerun the migration task
Your method of taking snapshots works fine. But rails includes the same effect auto-magically using the "down" migration techniques. Works with all db's, tastes great
Added:
For production, I agree that a down migration shouldn't be needed. But sometimes mistakes happen and you need to roll back. The down migration path gives you a first, and quick opportunity to fix things in an emergency situation during an upgrade that goes wrong.
-- it is much faster to try a down migration in an emergency than to restore the db using a checkpoint.
The "down" migration used for DB Rollbacks is in place so that every action has an equal and opposite action. It takes the onus off of the developer to maintain database snapshots, and allows them to use code to achieve the same ends. As Larry K said,
they're good for situations like so:
Add a database column called 'resubmitted', it's a boolean.
Product owner says they can resubmit multiple times, so change that column needs to be an int
Now, if you're 10 or 15 migrations deep, it's easier to just write a new one instead of losing all the dev data in the new tables/columns by doing a rollback. However, if you've just written that migration, it's cleaner and less cluttered to roll back, change the migration, and re-run it.
The other extremely useful feature of rollbacks is this:
Developer 1 has his own Dev DB. He writes a migration and runs it.
Developer 1 commits his migration to source control
Developer 2 has her own Dev DB. She writes a migration and runs it.
Developer 2 updates from source control
Developer 2 tries to run migrations but can't, as her local DB says "the latest migration has already been run", since her migration (the latest one), has technically already been run. Now she needs to rollback, then do a db:migrate to get all migrations in her local DB.
The idea of running a down migration in production horrifies me. Back when the preferred way to rollback all migrations was rake db:migrate VERSION=0 I would do this all the time in development. However, then I got paranoid that because it was committed to muscle memory I would accidentally type this on a production server when I meant to simply migrate.
Because of this paranoia, I add the following to all of my down methods.
def self.down
if Rails.env.production?
raise ActiveRecord::IrreversibleMigration
else
drop_table :foo_bars
end
end
This way, it still works in development, but I can't accidentally nuke my production database from orbit while half-asleep at 2:00 AM.
When a migration hasn't had the expected result, it is better to roll back and rewrite it than to keep the failed migration in the code.