Sometimes, data migrations are required. As time passes, code changes and migrations using your domain model are no longer valid and migrations fail. What are the best practices for migrating data?
I tried make an example to clarify the problem:
Consider this. You have a migration
class ChangeFromPartnerAppliedToAppliedAt < ActiveRecord::Migration
def up
User.all.each do |user|
user.applied_at = user.partner_application_at
user.save
end
end
this runs perfectly fine, of course. Later, you need a schema change
class AddAcceptanceConfirmedAt < ActiveRecord::Migration
def change
add_column :users, :acceptance_confirmed_at, :datetime
end
end
class User < ActiveRecord::Base
before_save :do_something_with_acceptance_confirmed_at
end
For you, no problem. It runs perfectly. But if your coworker pulls both these today, not having run the first migration yet, he'll get this error on running the first migration:
rake aborted!
An error has occurred, this and all later migrations canceled:
undefined method `acceptance_confirmed_at=' for #<User:0x007f85902346d8>
That's not being a team player, he'll be fixing the bug you introduced. What should you have done?
This is a perfect example of the Using Models in Your Migrations
class ChangeFromPartnerAppliedToAppliedAt < ActiveRecord::Migration
class User < ActiveRecord::Base
end
def up
User.all.each do |user|
user.applied_at = user.partner_application_at
user.save
end
end
Edited after Mischa's comment
class ChangeFromPartnerAppliedToAppliedAt < ActiveRecord::Migration
class User < ActiveRecord::Base
end
def up
User.update_all('applied_at = partner_application_at')
end
end
Best practice is: don't use models in migrations. Migrations change the way AR maps, so do not use them at all. Do it all with SQL. This way it will always work.
This:
User.all.each do |user|
user.applied_at = user.partner_application_at
user.save
end
I would do like this
update "UPDATE users SET applied_at=partner_application_at"
Some times 'migrating data' could not be performed as a part of schema migration, like discussed above. Sometimes 'migrating data' means 'fix historical data inconstancies' or 'update your Solr/Elasticsearch' index, so its a complex task. For these kind of tasks, check out this gem https://github.com/OffgridElectric/rails-data-migrations
This gem was designed to decouple Rails schema migrations from data migrations, so it wont cause downtimes at deploy time and make it easy to manage in overall
Related
I’m running into an issue where one of my migration is failing. This is how the existing migration looks like.
class AddColumnToTwitterPosts < ActiveRecord::Migration[6.0]
def up
add_column :twitter_posts, :status, :string, default: "new"
add_index :twitter_posts, :status
add_default_status_to_existing_posts
end
def down
remove_column :twitter_posts, :status
end
private
def add_default_status_to_existing_posts
TwitterPost.find_each do |post|
post.update!(status: "new")
end
end
end
Now I have moved model TwitterPost to namespace Twitter::Post.
So whenever this migration runs it fails to find this model. How can I make sure that rails picks up on the updated namespace instead of the old model name specified in the migration?
Actually you should not put any "data migrations" inside of your "schema migrations". This is considered bad practice, exactly for the reason which causes a problem for you now. One way to solve your problem is to use Rake tasks, as suggested by Thoughtbot. If you need to update data after a specific migration, you create a rake task doing the updates you need.
This way, your database schema migrations will always work since they don't depend on the presence of any specific model defined in your app.
A rake task might in your case look like:
namespace :twitter_posts do
desc "Update twitter posts"
task update_status_new: :environment do
puts "Going to update #{TwitterPost.count} twitter_posts"
ActiveRecord::Base.transaction do
TwitterPost.update_all(status: "new")
end
puts " All done now!"
end
end
i need to convert some data, but not sure how to do it.
I have a professional model, which had an foreign key. We decided that wasn't enough and changed the "has many" to a HABTM model, but now in the production environment, i need to convert the data from the foo_id field to the professional_foo joint table.
The "add table" migration will be executed before the "drop column" one, but how should i set up a conversion knowing that i have databases in use that use the old form and i will have new setups of the system that will be made straight to the last code version and because that, wont need any conversion to be done. (initializes scripts on newest version are already fixed.
I recommend doing the conversion of data in the migration between the add table and the drop column. This migration should be run when the new code is deployed so the new code will work with the new data structure and new installations that don't have any data yet will just run the migration very fast since there won't be data to convert.
The migration would look something like:
class OldProfessional < ActiveRecord::Base
self.table_name = "professionals"
has_many :foos
end
class NewProfessional < ActiveRecord::Base
self.table_name = "professionals"
has_and_belongs_to_many :foos
end
class MigrateFoosToHasAndBelongsToMany < ActiveRecord::Migration
def up
OldProfessional.all.each do |old_pro|
new_pro = NewProfessinoal.find(old_pro.id)
old_pro.foos.each do |foo|
new_pro.foos << foo
end
new_pro.save!
end
end
def down
NewProfessional.all.each do |new_pro|
old_pro = OldProfessional.find(new_pro.id)
new_pro.foos.each do |foo|
old_pro.foos << foo
end
old_pro.save!
end
end
end
I want to rename a table... (any table.)
I tried this line of code:
ActiveRecord::ConnectionAdapters::SchemaStatements.rename_table(old_name, new_name)
Here's the weird thing. I know I got it working the first time, but now I get this error: undefined method `rename_table' for ActiveRecord::ConnectionAdapters::SchemaStatements:Module
Was there something I need to set?
Remember that in Rails >= 3.1 you can use the change method.
class RenameOldTableToNewTable < ActiveRecord::Migration
def change
rename_table :old_table_name, :new_table_name
end
end
You would typically do this sort of thing in a migration:
class RenameFoo < ActiveRecord::Migration
def self.up
rename_table :foo, :bar
end
def self.down
rename_table :bar, :foo
end
end
.rename_table is an instance method, not a class method, so calling Class.method isn't going to work. Instead you'll have to create an instance of the class, and call the method on the instance, like this: Class.new.method.
[EDIT]
In this instance, ActiveRecord::ConnectionAdapters::SchemaStatements isn't even a class (as pointed out by cam), which means that you can't even create an instance of it as per what I said above. And even if you used cam's example of class Foo; include ActiveRecord::ConnectionAdapters::SchemaStatements; def bar; rename_table; end; end;, it still wouldn't work as rename_table raises an exception.
On the other hand, ActiveRecord::ConnectionAdapters::MysqlAdapter is a class, and it is likely this class you'd have to use to rename your table (or SQLite or PostgreSQL, depending on what database you're using). Now, as it happens, ActiveRecord::ConnectionAdapters::MysqlAdapter is already accessible through Model.connection, so you should be completely able to do Model.connection.rename_table, using any model in your application.
[/EDIT]
However, if you wish to permanently rename a table, I would suggest using a migration to do it. It's easy and the preferred way of manipulating your database structure with Rails. Here's how to do it:
# Commandline
rails generate migration rename_my_table
# In db/migrate/[timestamp]_rename_my_table.rb:
class RenameMyTable < ActiveRecord::Migration
def self.up
rename_table :my_table, :my_new_table
end
def self.down
rename_table :my_new_table, :my_table
end
end
Then, you can run your migration with rake db:migrate (which calls the self.up method), and use rake db:rollback (which calls self.down) to undo the migration.
ActiveRecord::Migration.rename_table(:old_table_name, :new_table_name)
I'd like to know which is the preferred way to add records to a database table in a Rails Migration. I've read on Ola Bini's book (Jruby on Rails) that he does something like this:
class CreateProductCategories < ActiveRecord::Migration
#defines the AR class
class ProductType < ActiveRecord::Base; end
def self.up
#CREATE THE TABLES...
load_data
end
def self.load_data
#Use AR object to create default data
ProductType.create(:name => "type")
end
end
This is nice and clean but for some reason, doesn't work on the lasts versions of rails...
The question is, how do you populate the database with default data (like users or something)?
Thanks!
The Rails API documentation for migrations shows a simpler way to achieve this.
http://api.rubyonrails.org/classes/ActiveRecord/Migration.html
class CreateProductCategories < ActiveRecord::Migration
def self.up
create_table "product_categories" do |t|
t.string name
# etc.
end
# Now populate the category list with default data
ProductCategory.create :name => 'Books', ...
ProductCategory.create :name => 'Games', ... # Etc.
# The "down" method takes care of the data because it
# drops the whole table.
end
def self.down
drop_table "product_categories"
end
end
Tested on Rails 2.3.0, but this should work for many earlier versions too.
You could use fixtures for that. It means having a yaml file somewhere with the data you want to insert.
Here is a changeset I committed for this in one of my app:
db/migrate/004_load_profiles.rb
require 'active_record/fixtures'
class LoadProfiles < ActiveRecord::Migration
def self.up
down()
directory = File.join(File.dirname(__FILE__), "init_data")
Fixtures.create_fixtures(directory, "profiles")
end
def self.down
Profile.delete_all
end
end
db/migrate/init_data/profiles.yaml
admin:
name: Admin
value: 1
normal:
name: Normal user
value: 2
You could also define in your seeds.rb file, for instance:
Grid.create :ref_code => 'one' , :name => 'Grade Única'
and after run:
rake db:seed
your migrations have access to all your models, so you shouldn't be creating a class inside the migration.
I am using the latest rails, and I can confirm that the example you posted definitely OUGHT to work.
However, migrations are a special beast. As long as you are clear, I don't see anything wrong with an ActiveRecord::Base.connection.execute("INSERT INTO product_types (name) VALUES ('type1'), ('type2')").
The advantage to this is, you can easily generate it by using some kind of GUI or web front-end to populate your starting data, and then doing a mysqldump -uroot database_name.product_types.
Whatever makes things easiest for the kind of person who's going to be executing your migrations and maintaining the product.
You should really not use
ProductType.create
in your migrations.
I have done similar but in the long run they are not guaranteed to work.
When you run the migration the model class you are using is the one at the time you run the migration, not the one at the time you created the migration. You will have to be sure you never change your model in such a way to stop you migration from running.
You are much better off running SQL for example:
[{name: 'Type', ..}, .. ].each do |type|
execute("INSERT INTO product_types (name) VALUES ('#{type[:name]} .. )
end
I have a sequence of migrations in a rails app which includes the following steps:
Create basic version of the 'user' model
Create an instance of this model - there needs to be at least one initial user in my system so that you can log in and start using it
Update the 'user' model to add a new field / column.
Now I'm using "validates_inclusion_of" on this new field/column. This worked fine on my initial development machine, which already had a database with these migrations applied. However, if I go to a fresh machine and run all the migrations, step 2 fails, because validates_inclusion_of fails, because the field from migration 3 hasn't been added to the model class yet.
As a workaround, I can comment out the "validates_..." line, run the migrations, and uncomment it, but that's not nice.
Better would be to re-order my migrations so the user creation (step 2) comes last, after all columns have been added.
I'm a rails newbie though, so I thought I'd ask what the preferred way to handle this situation is :)
The easiest way to avoid this issue is to use rake db:schema:load on the second machine, instead of db:migrate. rake db:schema:load uses schema.rb to load the most current version of your schema, as opposed to migrating it up form scratch.
If you run into this issue when deploying to a production machine (where preserving data is important), you'll probably have to consolidate your migrations into a single file without conflicts.
You can declare a class with the same name inside the migration, it will override your app/models one:
class YourMigration < ActiveRecord::Migration
class User < ActiveRecord::Base; end
def self.up
# User.create(:name => 'admin')
end
end
Unfortunately, your IDE may try to autocomplete based on this class (Netbeans does) and you can't use your model logic in there (except if you duplicate it).
I'm having to do this right now. Building upon BiHi's advice, I'm loading the model manually then redefining methods where I need to.
load(File.join(RAILS_ROOT,"app/models/user.rb"))
class User < ActiveRecord::Base
def before_validation; nil; end # clear out the breaking before_validation
def column1; "hello"; end # satisfy validates_inclusion_of :column1
end
In your migration, you can save your user skipping ActiveRecord validation:
class YourMigration < ActiveRecord::Migration
def up
user = User.new(name: 'admin')
user.save(validate: false)
end
end