I have a migration that's breaking in the middle of a couple of schema changes. When it breaks an exception is thrown and rake db:migrate exits, leaving my database in a half-migrated state.
How can I set it up so that the migration automatically reverts just the changes that have run so far? I'd like to do so globally when in development mode. Surely someone out there has a better way than embedding each successive AR::Migration::ClassMethod in a begin; rescue =>e opposite_action; end block.
Perhaps a common example is in order:
#2010010100000000_made_a_typo.rb
class MadeATypo < ActiveRecord::Migration
def self.up
rename_column :birds, :url, :photo_file_name
rename_column :birds, :genius, :species #typo on :genius => :genus
end
def self.down
rename_column :birds, :photo_file_name, :url
rename_column :birds, :species, :genius
end
end
This migration will fail on the second line with "column genius not found", but not record the migration number in the schema_migrations table. I'd like it if it called
rename_column :birds, :photo_file_name, :url #this is a revert of the first line
before the exception was passed out of MadeATypo.up.
Responses to comments:
I understand that mysql might not have support for DDL transactions, I'm looking for a more application-level solution which (probably) uses AR::Migration itself. Surely someone has created a plugin which captures method calls to the main AR:M:ClassMethods and can rewind them in most cases if an exception occurs during a migration.
I don't have a solution, but the main problem is that DDL statements can't be transactioned (at least in MySQL, I don't know if that's a general thing).
So, because migrations are treated as atomic up/down actions, there's no easy way to undo "half" a migration - it's not easy to work out which parts of the down correspond to which parts of the up
I don't know of any way to do this currently, but the code for reversible migrations coming to Rails 3.1 looks like a good base to build this feature on. See these links:
http://edgerails.info/articles/what-s-new-in-edge-rails/2011/05/06/reversible-migrations/index.html
https://github.com/rails/rails/commit/47017bd1697d6b4d6780356a403f91536eacd689
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/migration/command_recorder.rb
The way I'm thinking you could implement this is to run the migration against the recorder (as in https://github.com/rails/rails/commit/47017bd1697d6b4d6780356a403f91536eacd689#L0R337), then switch back to the live connection and run the recorder forward one command at a time, tracking how many have completed. All you need, then, is to wrap that forward running loop in a begin-rescue-end that executes as many inverse commands as you successfully executed forward commands, if there's an exception.
Related
I've a migration in my Rails app, that I'd like to run only if a particular rake task has been run, otherwise I'll lose a bunch of data. Following is something that I'd like to do:
if has_rake_task_been_run?
remove_column :transactions, :paid_by
end
Currently, I couldn't find anyway, instead of assuring this thing manually. Is there any work around for it?
Using rake task for data migration is an extremely risky idea. Couple of reasons not to do this:
Even if you manage to find out whether your rake task has finished or not, your migration will still be marked as completed and you won't be able to replay it. Only way around is raising an exception in your migration.
No, you won't be able to rollback that migration neither. If rake task finishes after the migration has run, rollback will try to add already existing column.
Setting up your database from scratch by new devs will become painful as hell, as they will need to know which rake tasks are to be run when. Not to mentioned that rake db:migrate executes all migrations.
You're polluting your rake task list with non-reusable tasks
It seems that what you're doing is just a regular data migration, so all the stuff done by your rake task should be in fact part of your migration. That will even allow you to make a reversible data migration (in majority of cases).
Note however that data migrations are not that simple as regular scheme-only migrations. Because your migration should be completely independent on your code (as they are to work in the future, even when migrated model is completely removed from your codebase), so it is a common practice to redefine the models you are about to use in your migrations (only the bits required fro the migration). This is not that simple as it sounds, unfortunately, and honestly I am still looking for a perfect solution to that. The best I've seen so far is simple (I'm assuming that paid_by used to be string and you changed it paid_by_id, which references the user):
class YOURMIGRATIONNAME < ActiveRecord::Migration
class Transaction < ActiveRecord::Base
belongs_to :paid_by, class_name: "User"
end
class User < ActiveRecord::Base
end
def up
add_column :transaction, :paid_by_id, :integer
Transaction.transaction do # for speed
Transaction.find_each do |t|
t.paid_by_id = User.find_by(username: t[:paid_by])
t.save! # Always banged save in migration!
end
end
remove_column :paid_by
end
def down
add_column :transaction, :paid_by, :string
Transactions.transaction do
Transaction.find_each do |t|
t[:paid_by] = t.paid_by && t.paid_by.username
t.save!
end
end
remove_column :transactions, :paid_by_id
end
The only downfall of using the code above is that it won't work well if any of those models is using STI (I've made that mistake once, took a while to find out what's wrong). The work around is to define it outside of the migration class, but then those classes are available across all migrations and can be affected with your actual model code (especially in production when all the models are preloaded). In short, data migration with STI is something I am still looking into at the moment.
In case somebody would come here, We succcessfully used after_party rails library. With a simple mechanism, the library maintains rake tasks that have been executed and it becomes easy to perform migration tasks.
Among my migrations, I have 3 that pertain to this question. In order, there is CreateEvents < ActiveRecord::Migration. The next down the line is CreateYears < ActiveRecord::Migration. Then the last is AddYearIdToEvents < ActiveRecord::Migration. The last one looks like this...
class AddYearIdToEvents < ActiveRecord::Migration
def change
add_column :events, :year_id, :integer, null: false, index: true
end
end
Now the problem is, whenever I try to drop the entire database (not near deploying to production), I get an obvious error of
ERROR: cannot drop table years because other objects depend on it
DETAIL: constraint events_year_id_fk on table events depends on table seasons
HINT: Use DROP ... CASCADE to drop the dependent objects too.
Now, I'm not a rails expert, but I believe I need to define a def down in that last migration. Rather than def change, do I need a def up and def down? The def down to drop this particular column.
If so, how can I do this. This migration is 20 migrations old. You can't just edit a migration like that can you? Do I add a new migration and just specify a def down? Or is the answer something completely different?
Have you tried rolling back your migrations? This undoes your migrations in reverse order, i.e. starting with the most recent migration.
$ rake db:rollback
If you have 20 migrations and you'd like to rollback all of them, you can use STEP=20
$ rake db:rollback STEP=20
Once you rollback past a migration that you'd like to change, you can change it.
In addition, you should be able to use rake db:drop, which drops the entire database rather than going through each migration in reverse order. If you'd like to drop the database, then recreate it and re-run all the migrations, you can run rake db:reset. Keep in mind this also runs rake db:seed if you have a db/seed.rb file.
And in response to your question of "Rather than def change, do I need a def up and def down?", the answer is no. Newer versions of Rails use def change since (among other reasons) it makes it easier to edit migrations once they are generated, i.e. if you spell a column name incorrectly when adding a column to a table.
I can't figure out why my migration worked but doesn't show up in corresponding model...
Ran a migration to add field to a table:
def up
add_column :quick_tests, :trace_route_data, :text, :null => true
end
def down
remove_column :quick_tests, :trace_route_data
end
Looked in schema.rb, and it's there.
Added :trace_route_data to attr_accessible in QuickTest model
Opened up rails console by doing bundle exec rails c, ran QuickTest.new and I can see the :trace_route_data field.
But, when I execute the same QuickTest.new statement when paused in the QuickTestController#show method, the field isn't there.
Why is this field showing up in the rails console but not my actual app??
I suspect something went wrong during the migration such that it never got applied to the database. Rails uses the actual schema in the database to build model objects, so the fact that your model doesn't show the change means the change is almost certainly not in the database (despite being in the schema.rb).
I'd recommend you rollback your migration and run it again in verbose mode. This should either show you the error, or at least show you the SQL being run. In that case, you can run the SQL manually at the db console and see what happens then.
I am just wondering what is the correct behaviour when creating concurrent indexes in Rails.
I am using this in my migration file:
disable_ddl_transaction!
def change
add_index :table_name, :field_name, algorithm: :concurrently
end
This should create a postgres concurrent index.
My question is: When running rake db:migrate Is the correct behaviour to wait for the creation of the index? I mean, the migration will wait until the index is created? Or it should end and delegate postgres that responsibility?
P.S. It is a really big index.
I found out it is the correct behavior.
Regardless the migration is waiting, the index will be created concurrently as expected.
Writes and reads are allowed in creation time.
I have a pretty old migration on a legacy app by a friend that contains this snippet:
class MakeChangesToProductionDbToMatchWhatItShouldBe < ActiveRecord::Migration
def self.up
# For some reason ActiveRecord::Base.connection.tables.sort().each blows up
['adjustments',
'accounts',
...## more rows of classes here ###...
'product_types'].each do |table|
t = table.singularize.camelize.constantize.new
if t.attributes.include?('created_at')
change_column( table.to_sym, :created_at, :datetime, :null => false ) rescue puts "#{table} doesnt have created_at"
end
if t.attributes.include?('updated_at')
change_column( table.to_sym, :updated_at, :datetime, :null => false ) rescue puts "#{table} doesnt have updated_at"
end
end
This old migration is now conflicting with a new migration I wrote to remove two of the tables mentioned in this long list, which is now causing any deployment to error upon rake db:migrate.
What's the correct kind of migration or down action to write to address this migration and get db:migrate working again?
There are a few different best practices that can help, but at the end of the day there's no good way to always upgrade a database from an arbitrary point without stepping through the codebase along as you run migrations (speaking of which, why is there not already a rake task to do this?).
Always include a migration-namespaced copy of the models you're working on. Example below.
When building a database from scratch, do not run migrations…use db:schema:load which will re-create the last snapshot of the database.
Don't give your migrations ridiculous and aggression fueled titles like MakeChangesToProductionDbToMatchWhatItShouldBe.
Avoid making assumptions, when writing migrations, about the environment they will be run in. This includes specifying table names, database drivers, environment variables, etc.
Write down actions when you write up actions whenever a down action is feasible. It's usually much easier (especially on esoteric or complex migrations) when the series of transformations is fresh in your head.
For this specific case, there's an argument to be made for declaring “Migration Bankruptcy” — clearing out some or all existing migrations (or refactoring and coalescing into a new one) to achieve the desired database state. When you do this you are no longer backwards compatible so it is not to be taken lightly, but there are times it is the appropriate move.