Here is the problem:
I have Ruby on Rails project that has a table that have almost 100k rows and I have a binary column and I want to make changes in the content data on this column.
So I must iterate over those 100k rows, making changes on that row on particular column, saving it back on database.
But I must keep track of changes because these changes could fail and I should have someway to re-start data change from where I stopped.
Here is what I thought of a way of doing it:
Create a Migration to have a table MigrationTrack to track all records that have being migrated
Create a model of the above migration
Create a rake task that grabs all 100k from TableToUpdate and iterate over them, saving data back to row and save its ID on MigrationTrack. Create a logic to have a join on TableToUpdate and MigrationTrack to filter only ids that I haven't updated yet
After above migration finished create another migration to drop MigrationTrack table and remove its model.
Is there any other "Railsh way" to do that? Anyone have done such change?
Thanks
I would do it like this:
Add and deploy a migration adding a new column with the desired data type to the database table.
Add code to your model that save the value from the old column into the new column too.
Run a rake task or a simple one-liner in the console that touches all records to make sure the code introduced in step one ran on each record.
After this step, you can manually verify if all records in the database have both columns set as expected.
Switch using the new attribute instead of the old attribute in the code.
Drop the old column.
For simple cases, try running a simple view to check how it will turn out to be, for example, if your migration is
change_column :table, :boolean_field, 'integer USING CASE boolean_field THEN ...'
then you try do a simple select query with your cast, if you need more safey, you can create 'up' and 'down' methods on your migrations, then you can create a backup table on up, and on down, you can revert the values
Are migrations instructions that change a model? Will I make several migrations, or will there be one migration per table? For instance, let's say if I want to change "username" to "admin_username". Does this call for a migration? Then, let's say I decide I want to add "age". Do I have to make a new migration, or do I just add it to the aforementioned migration?
You can make the quantity of migrations you want. My advise, try to plan your project and see how your model needs to be in order to avoid migrations. The migration feature is a great way to add, delete, rename fields to your model among other things.
If you need to change the username of the model User then you run rails g migration renameUsernameInUsers and add the necessary code to change the name of that field in you recently created migration file.
If some minutes later you realise that you also need to add a field you can make a new migration with your needs and add a field or add multiple fields in just one migration.
Remember, in order to apply your migrations you need to run rake db:migrate. If you modify the code in your migration files already migrated with this command, it will not have effect in your model. You will need a new migration.
In conclusion, you can(will) make many migrations for a model.
To go further on this topic you can see this official page for migrations in rails.
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
I created a model "user" using the generate scaffold user command line. Next I made some changes to the database using generate migration command line and then updated the database using rake db:migration . Now I want to make my model in sync with the database fields automatically. Is it possible through a command line and if yes, how ? If not, what are the other options.
thanks
Your application find automacaly the change in your database if you don't specify a attr_accessible in your model. It make SQL requests to see all fields in the table. It's slower but simpler.
I want to delete a table in my rails app but i cannot use rollback, because i created this table a long time ago, and i have a LOT of other tables created since that one.
How is supposed to be named a drop table migration file and is there a way to genrate it with rails generate?
Create one more migration to drop the table. The class should have the method
def self.up
drop_table :table_name
end
Be careful as you will not be able to rollback to get all the data you will lose while dropping the table.