Here is the problem:
I have Ruby on Rails project that has a table that have almost 100k rows and I have a binary column and I want to make changes in the content data on this column.
So I must iterate over those 100k rows, making changes on that row on particular column, saving it back on database.
But I must keep track of changes because these changes could fail and I should have someway to re-start data change from where I stopped.
Here is what I thought of a way of doing it:
Create a Migration to have a table MigrationTrack to track all records that have being migrated
Create a model of the above migration
Create a rake task that grabs all 100k from TableToUpdate and iterate over them, saving data back to row and save its ID on MigrationTrack. Create a logic to have a join on TableToUpdate and MigrationTrack to filter only ids that I haven't updated yet
After above migration finished create another migration to drop MigrationTrack table and remove its model.
Is there any other "Railsh way" to do that? Anyone have done such change?
Thanks
I would do it like this:
Add and deploy a migration adding a new column with the desired data type to the database table.
Add code to your model that save the value from the old column into the new column too.
Run a rake task or a simple one-liner in the console that touches all records to make sure the code introduced in step one ran on each record.
After this step, you can manually verify if all records in the database have both columns set as expected.
Switch using the new attribute instead of the old attribute in the code.
Drop the old column.
For simple cases, try running a simple view to check how it will turn out to be, for example, if your migration is
change_column :table, :boolean_field, 'integer USING CASE boolean_field THEN ...'
then you try do a simple select query with your cast, if you need more safey, you can create 'up' and 'down' methods on your migrations, then you can create a backup table on up, and on down, you can revert the values
Are migrations instructions that change a model? Will I make several migrations, or will there be one migration per table? For instance, let's say if I want to change "username" to "admin_username". Does this call for a migration? Then, let's say I decide I want to add "age". Do I have to make a new migration, or do I just add it to the aforementioned migration?
You can make the quantity of migrations you want. My advise, try to plan your project and see how your model needs to be in order to avoid migrations. The migration feature is a great way to add, delete, rename fields to your model among other things.
If you need to change the username of the model User then you run rails g migration renameUsernameInUsers and add the necessary code to change the name of that field in you recently created migration file.
If some minutes later you realise that you also need to add a field you can make a new migration with your needs and add a field or add multiple fields in just one migration.
Remember, in order to apply your migrations you need to run rake db:migrate. If you modify the code in your migration files already migrated with this command, it will not have effect in your model. You will need a new migration.
In conclusion, you can(will) make many migrations for a model.
To go further on this topic you can see this official page for migrations in rails.
Is there a way to specify the fields and relationships in the model and run a command to the rails create tables in the database based on the models, and then I can create or remove other fielda, change some relationships and run this command again and rails do update the tables in the database? I thought the migrate did it but after I read about it seems that it only creates the tables once and has no concept of model / db sync that i need.
No you need to create a new migration anytime you need to update the database. Changing some code in the model will not generate a migration.
No, but if you want a simpler way to write your migrations etc, I suggest you read more about the generators :)
For example :
rails generate model user first_name:string last_name:string birth_date:datetime friend:references
will directly generate the migration with the correct fields :)
And if you want to add a field after, you can write it like :
rails generate migration AddPseudoToUsers pseudo:string
And it will be correctly written normally :)
This is the doc
http://guides.rubyonrails.org/migrations.html#creating-a-migration
http://guides.rubyonrails.org/command_line.html#rails-generate
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
For instance, when I generate an Event model, the table automatically sets to the public schema. How do I specify it to get set to a different schema?
Furthermore, how do you alter the schema of an existing table? Perhaps move it to a different schema?
Thank you!
Disclaimer: I don't know rails, so I'm going to give very postgresql-oriented answers here. For the first part of your question, there is quite possibly a much better way to do this, by making rails specify the schema when creating tables.
In PostgreSQL, tables are searched for in schemas according to the search_path setting. This is set by default to "$user",public. Tables are created in the first schema found in the search path that exists. So if you connect as "my_user", it will try to create tables in "my_user", and fall back to creating them in "public" if "my_user" doesn't exist.
So one approach is to update the "search_path" setting used for the user you connect to the database to make schema changes. For example you can say ALTER USER my_user SET search_path = my_app, public. If you then create a "my_app" schema then subsequent CREATE TABLE foo(...) commands executed by "my_user" will put the new table into "my_app".
You can change the schema of a table using ALTER TABLE foo SET SCHEMA my_app.
Create a migration to generate your new schema. ActiveRecord can't update you schema to you it's the pattern system. You can try sequel or DataMapper if you want update you schema from your code.