Can Rails Migrations be used to convert data? - ruby-on-rails

I'm trying to convert a column in my Rails app, for arguments sake let's pretend I'm trying to change the age column in my users table to a string representation rather than an int.
In my migration I have this;
def.self up
add_column :users, :age_text, :string
users = User.find(:all)
users.each do |u|
u.age_text = convert_to_text(u.age)
u.save
end
end
def self.convert_to_text(number)
#code here to convert 1 to 'one' etc
end
But it doesn't seem to be working, is what I'm attempting here even possible with migrations?

What you're trying to do is possible, and I would say the correct thing to do.
You need, though, to reload the column info for the model classes you're updating in the migration, so that Rails knows about the new columns. Try this:
def.self up
add_column :users, :age_text, :string
User.reset_column_information
users = User.find(:all)
users.each do |u|
u.age_text = convert_to_text(u.age)
u.save
end
end
On a separate note, please note that if your table is large, doing updates one by one will take a looong time.. Be careful with that.

Since I'm new here I can't comment on the above so I'll add my own answer.
GENERALLY manipulating data in migrations is a BAD idea. Migrations with direct model access can get stuck if the model logic changes.
Imagine in your second migration you've added a new column. You want to seed that column with new data.
Let's also say a few weeks later you add a new validation to the model - a validation that operates on a field that does not yet exist in your second migration. if you ever were to construct the database from migration 0, you'd have some problems.
I strongly suggest using migrations to alter columns and other means to manage database data, especially when moving to production.

Here is an example migration I ran to convert data. You can easily convert it to use integers instead of strings. Making the conversion in SQL is much faster than loading each row in Rails.
class ConvertCommentTextToText < ActiveRecord::Migration
def up
add_column :comments, :text_tmp, :text
# copy text column
execute <<-SQL
update comments set text_tmp = text
SQL
remove_column :comments, :text
rename_column :comments, :text_tmp, :text
end
def down
add_column :comments, :text_tmp, :string
# copy text column
execute <<-SQL
update comments set text_tmp = text
SQL
remove_column :comments, :text
rename_column :comments, :text_tmp, :text
end
end
And to test it:
rake db:migrate
rake db:rollback
rake db:migrate

I would say that if you can "undo" the imported data when rolling back the migration version, then it's appropriate to put imports into the migration.
For example, I have a migration which sets up a lot of lookup tables and other meta-data. The data for these tables are populated during this phase. As the data for these lookup tables changes, I create new YAML files storing the meta-data and load those files in subsequent migrations (and un-do those YAMLS, re-loading the previous YAML file when backing out of a migration version). This is pretty clean. I have files (in different well-defined folders in my case) with these files:
002_setup_meta_data.rb
002_meta_data.yaml
007_change_meta_data.rb
007_meta_data.yaml
If you're importing "production" data from another system into transactional (non-static) tables, then I would say using migrations is not appropriate. Then I would follow Brian Hogan's advice of using rake tasks.

Related

How to add a new column to every model?

with what migration can I add a column to every model at once? This would be handy almost every time you forgot to add a field anyway because most of the time a field you want to add is not limited to one model only. Like "email" In this case I forgot to add an "order" field.
Thanks
You can do as follows -
def change
tables = [:users, :products, :addresses]
tables.each do |table_name|
add_column table_name, :order, :integer
end
end
It's called a group migration
Answering your first question "With what migration can I add a column to every model at once?". Answer: None. Rails migrations are a way to alter database schemas over time in a consistent way.
Rails migrations are Ruby classes using Rails methods as instructions to modify your database as needed. So your question could be better formulated as "How can I create a migration to add a column to every model at once?"
IMHO I don't think there's going to be a specific method to do this, as the requeriment is pretty custom, but, depending in your Rails version you can get all the ApplicationRecord.descendants:
Zeitwerk::Loader.eager_load_all
ApplicationRecord.descendants.map { |table| table.name.downcase.pluralize }.each do |table|
add_column table, :logdate, :datetime
end
Or all those tables from the database that can be safe_constantized:
ActiveRecord::Base.connection.tables.map do |table|
table.classify.safe_constantize
end.reject(&:nil?).each do |table|
add_column table, :logdate, :datetime
end
That way you get the name of each table/model and use it as the first argument for add_column.
The difference is that in Rails 6 the default code loader is Zeitwerk, so you can eager load all the project dependencies. In other versions you could do the same but using Rails.application.eager_load!.
The second version would work without having to load the models as dependencies as it makes a query asking for their tables and then maps them as constants.

Rails Migration Column Added But Not Populated

I'm trying to write a very simple Rails migration that adds a new column to the table and populates it with a combination of data from other columns. However, the column gets added but the column value is nil for every record. What am I doing wrong?
class AddNameToPermissions < ActiveRecord::Migration
def change
add_column :auth_permissions, :name, :string
Auth::Permission.reset_column_information
Auth::Permission.all.each do |permission|
target_name = permission.target_symbol || permission.target_class
permission.name = permission.action << ", " << target_name
permission.save
end
end
end
Calling permission.save will execute all the callbacks including validators and maybe a validator is preventing the record to be saved. To skip validators you can use update_column instead of save
permission.update_column(:name, permission.action << ", " << target_name)
Also I would recommend you a couple of tips:
Use Auth::Permission.find_each instead of Auth::Permission.all.each (this loads all the records at once in memory)
Try to update the permission in a rake task and use only the migration to modify the table (it is the recommended good practice way)
Unless you are using the default option in a migration, it is usually advisable to do that in a custom rake task. Content Migrations are a bit of an anti pattern. The main Logic you want there is anything that has to deal with the schema of your database. Run your migration with only the add_column line, and then write a rake task to transfer the values from one table to the other.

Rails: One migration works when I run it individually, but not in a series with other migrations

I have a series of migrations for refactoring on the upcoming release. Some of the migrations are used to move columns around.
When I run the migrations all at once, the specific column in the following example is never copied to the new place.
def up
add_column :buy_topics, :price, :integer
say_with_time 'Move price and value to buy/topics' do
bar = ProgressBar.new(Buy::Topic.count)
Buy::Topic.includes{topic}.find_each do |topic|
topic.price = topic.topic.price
topic.save
bar.increment!
end
end
remove_column :topics, :price
end
Even though the values are not copied, the columns were still created and removed successfully though.
However if I run all the migration up to the the one prior to this (e.g. rake db:migrate VERSION=XXXXXXXXn-1), and then manually run this one particular migration (rake db:migrate VERSION=XXXXXXXXn), the values are copied across.
So in summary, if I run that migration individually it works, otherwise it will not work
What could be the reason behind this?
Try doing this:
Buy::Topic.reset_column_information
For more information, look at "Using a model after changing its table" on [this page][http://api.rubyonrails.org/classes/ActiveRecord/Migration.html].

Temporary index name too long in Rails migration

I've got a problem trying to rollback one of my migration. It seems as if Rails is generating a temporary table for the migration, with temporary indices. My actual index on this table is less than 64 characters, but whenever Rails tries to create a temporary index for it, it turns into a name longer than 64 characters, and throws an error.
Here's my simple migration:
class AddColumnNameToPrices < ActiveRecord::Migration
def self.up
add_column :prices, :column_name, :decimal
end
def self.down
remove_column :prices, :column_name
end
end
Here's the error I'm getting:
== AddColumnNameToPrices: reverting ============================================
-- remove_column(:prices, :column_name)
rake aborted!
An error has occurred, this and all later migrations canceled:
Index name 'temp_index_altered_prices_on_column_and_other_column_and_third_column' on table 'altered_prices' is too long; the limit is 64 characters
I've changed the column names, but the example is still there. I can just make my change in a second migration, but that still means I can't rollback migrations on this table. I can rename the index in a new migration, but that still locks me out of this single migration.
Does anyone have ideas on how to get around this problem?
It looks like your database schema actually has index called prices_on_column_and_other_column_and_third_column. You have probably defined the index in your previous play with migrations. But than just removed index definition from migrations.
If it is true you have 2 options:
The simplier one (works if you code is not in production). You can
recreate database from scratch using migrations (not from
db/schema.rb) by calling rake db:drop db:create db:migrate. Make sure that you do not create this index with long name in other migration files. If you do, add :name => 'short_index_name' options to add_index call to make rails generate shorter name for the index.
If you experience this problem on a production database it is a bit more complicated. You might need to manually drop the index from the database console.
Had this problem today and fixed it by changing the migration to include the dropping and adding of the index causing the long name issue. This way the alteration is not tracked while I am altering the column type (that is where the really long name is caused)
I added the following:
class FixBadColumnTypeInNotifications < ActiveRecord::Migration
def change
# replace string ID with integer so it is Postgres friendly
remove_index :notifications, ["notifiable_id","notifiable_type"]
change_column :notifications, :notifiable_id, :integer
# shortened index name
add_index "notifications", ["notifiable_id","notifiable_type"], :name => "notifs_on_poly_id_and_type"
end
end

How might you clone a database table via Rails migration?

I want a migration to create a clone of an existing table by just suffixing the name, including all the indexes from the original table.
So there's a "snapshots" table and I want to create "snapshots_temp" as an exact copy of the table (not the data, just the table schema, but including the indexes).
I could just copy and paste the block out of the schema.rb file and manually rename it.
But I'm not sure by the time this migration is applied if the definition from schema.rb will still be accurate. Another developer might have changed the table and I don't want to have to update my migration script.
So how might I get the schema of the table at runtime? Essentially, how does 'rake schema:dump' reverse engineer the table so I can do the same in my migration? (but changing the table name).
Try doing it with pure SQL. This will do what you want:
CREATE TABLE new_tbl LIKE orig_tbl;
In Rails 4 & PostgreSQL, create a new migration and insert:
ActiveRecord::Base.connection.execute("CREATE TABLE clone_table_name AS SELECT * FROM source_table_name;")
This will create the clone with the exact structure of the original table, and populate the new table with old values.
More info: http://www.postgresql.org/docs/9.0/static/sql-createtableas.html
This will do. It's not perfect, because it won't copy table options or indices. If you do have any table options set, you will have to add them to this migration manually.
To copy indices you'll have to formulate a SQL query to select them, and then process them into new add_index directives. That's a little beyond my knowledge. But this works for copying the structure.
class CopyTableSchema < ActiveRecord::Migration
def self.up
create_table :new_models do |t|
Model.columns.each do |column|
next if column.name == "id" # already created by create_table
t.send(column.type.to_sym, column.name.to_sym, :null => column.null,
:limit => column.limit, :default => column.default, :scale => column.scale,
:precision => column.precision)
end
end
# copy data
Model.all.each do |m|
NewModel.create m.attributes
end
end
def self.down
drop_table :new_models
end
end
Copy the tables entry from your projects db/schema.rb straight into your migration. Just change the table name and your good to go.
It looks like this logic is wrapped up in ActiveRecord::SchemaDumper but it only exposes an all-in-one "dump" method and you can't dump just a specific table (the "table" method is private).

Resources