I need to rename a column in a Rails project that has enabled strong_migrations. Im looking into the official guide and it specified the following steps to doing it:
Create a new column.
Write to both columns.
Backfill data from the old column to the new column.
Move reads from the old column to the new column.
Stop writing to the old column.
Drop the old column.
So as far as I understand I need about 4 PRs to simply change the name of the column.
This PRs would be:
Create the migration for the new table and start using the new field.
Backfill data and move reads from the old column to the new one.
Add self.ignored_columns = ["some_column"] to the model and remove the old logic that use that old column.
Create the migration to drop off the old column.
It's that right? Or Im missing something and maybe some of this PRs are not necessary?
Can be done with 3. With ways to migrate the data (a and b)
Add the new column and code that will start writing to it. a) Add a worker that will backfill the data after the deploy
Change code to start reading the new column and add the old column to ignored columns. b) Backfill the data in a migration before deploy.
Remove the old column
Data migration style will depend on the size of your database. If it's small enough, you can do it in the migration during deploy. If it's bigger, you'll have to migrate the data beforehand. Either a worker or a console script. In this case it's still a good idea to have the backfill in a migration as well to ensure the data is migrated everywhere (local db of every developer, staging environments and so on).
Related
Is there a quick/easy way to make a migration that adds a new column between two existing columns?
Note: I googled and couldn't find an obvious answer. But I also am curious if it's even good practice (since if for some reason other column/s were removed, then the migration may fail?)
If you haven't commited your changes to production you can reorder migrations by rolling them back and then changing the timestamps in the file name.
If thats not an alternative you can actually re-order the columns on some databases directly with SQL even if its not part of the migrations DSL. Migrations are after all really just a DSL to create SQL strings and run them in a repeatable way across environments.
If you can't generate the SQL you want with the DSL you can always use execute to execute a raw DSL string.
# MySQL example
class ReorderYourTableName < ActiveRecord::Migration[5.2]
def change
execute "alter table yourTableName change column yourColumnName yourColumnName dataType after yourSpecificColumnName;"
end
end
However on some DBs you can't actually reorder the columns without extensive steps of creating new columns and shuffling the data around.
My app is in production and I want to manage user data when user updates the app without loss of their data, how can I achieve this with sqflite. Explicitly I want to add a column and delete another.
You can probably add a column using raw sql, but sqlite (and thus sqflite) doesn't support dropping a column. For that you would need to do the following:
increase the database version number
in onUpgrade copy the old database columns to a temporary table
delete the original table
create a new table using the original table name but with the right schema
copy the data from the temp table
delete the temp table
Sorry, this isn't a full answer, but it is the direction I would go if I were in your situation.
I have the same problem and found this article which seems to be a good solution.
I am working on a web app that already has a schema in place (aka in prod) with a certain number of tables (A, B, C...).
Table A has an attribute that corresponds to an enum from table B. Problem is that I can have only one item of that enum list in my dedicated column in table A. But I want my objects from table A to have many of them. So I created a join table A_B and a has_many through association with my table B.
The first consequence is that I need to fill my join table with data from the previous schema architecture. To be clearer, they were objects from table A that were associated with one element of the table B enum. I need to report these simple relationships (only one element from enum list in table B is associated with table A objects) in my newly created join table.
Here's the type of things I'd like to do:
list_of_ids = []
Model_A.where(attribute: 0).each { |r| list_of_ids << r.id }
a.each { |el| A_B.create(tableA_id: el, tableB_id: 0) }
Where should I write and execute these lines of code that will update my data ?
As stated in my comments, I would put this "data-update" logic in the same migration file of the join table creation.
Why in a migration file?
the data conversion needs to be done only once, AFTER you created the join table and BEFORE you remove the column which holds the foreign key. If you do this data conversion after the column removal, you will get an error saying that your code is trying to access column that does not exist anymore.
the migration is responsible for changing the DB structure AND for the data integrity.
Why not a rake task?
rake tasks are meant to be run several times, not only once. The usual tasks are "send_emails", "update_expiration_dates", "compute_cache", "close_inactive_users_account", etc.
the data you have before your conversion has_many -> HABTM have to be updated to follow the new structure. the rake task could not be ran, and then your data would not be updated, therefore you would loose the association between your models (removing the foreign_key column before running the rake task would make you loose this data).
your data-conversion logic must happen after the migration creating the join table and BEFORE the migration removing the foreign key's column. There are clear way to say to your rails app: "do this migration, then stop, run this task, then do this migration". It will run the migrations consecutively. If you have these 2 migrations pending, they will be run at once unless you specify otherwise (which is not common at all), and then you rake task will be useless because it relies on the fact that the foreign key column still exists.
In addition to MrYoshiji's answer (too long for comments):
Migrations are intended for one-off changes to both database schemas and the data therein. The official guide mentions this, too.
In your case, populating the join table is an appropriate thing to do in a migration. Otherwise data would be left in an invalid or incorrect state vis-a-vis the schema and model changes you have made. Running migrations is a typical step in deploying a Rails app. Since you probably do not want to deploy without updating the data, having it as part of a migration is a great solution. In contrast, if you created a custom rake task to update the data then you would need to remember to run it manually after deploying or add it as a deployment step, neither of which is a very good option.
Running Rails 4 with Postgres 9.4 in development and production. I've got a large table that has grown by migrations over time. Because of the many different columns on the table, I want to reorder things so that the columns are grouped more logically. In other words, some column elements naturally group together based on what information they capture.
I found a discussion on using after: in a migration to reorder columns (using ALTER TABLE in SQL), at this stack overflow discussion. I then went ahead and set up a migration to do that. After running the migration I notice that my schema.rb file hasn't changed. Looking at the columns in the database (postgres on development), nothing has changed there either.
Further research led me to the Postgres wiki, which states there is currently no support for altering column position in Postgres.
This isn't mission critical, but it would make life easier. My question is:
If I simply move lines in the schema.rb file up or down in order to position them as desired, will that cause any problems? Since I expect to provision any new database with rake db:schema:load, it wouldn't seem like that should break anything. Further, since the previous migrations aren't materially changed (just the order of columns they output to schema.rb) everything should be internally consistent, no?
If that's a bad idea, can I simply go back into the earlier migrations, add the syntax for after: :some_column to each column element so they correctly set up the schema.rb file?
My assumption is that when I rebuild my production database from scratch with seed data, it will use the schema structure to correctly create the table in the desired order. Right now it's not a real, deployed database with end users, so it doesn't seem like an issue. Using option #1 above seems like the easiest solution, as long as it doesn't break anything.
Many people new to postgresql often ask if it has support for altering
column positions within a table. Currently it does not; if you want to
change column positions, you must either recreate the table, or add
new columns and move data. The idea of allowing re-ordering of column
position is not one the postgresql developers are against, it is more
a case where no one has stepped forward to do the work.
ref
If I simply move lines in the schema.rb file up or down in order to
position them as desired, will that cause any problems?
no. It does nothing with column order in PG.
If that's a bad idea, can I simply go back into the earlier migrations, add the syntax for after: :some_column to each column element so they correctly set up the schema.rb file?
after: :some_column option does nothing with column order if you use PG
The order of your columns does not matter to Ruby or Rails.
As to whether it matters to any code you wrote, only you and your tests can answer that. If you're only using ActiveRecord and not doing any straight SQL that references column number, you should be fine.
As a workaround, you can use annotate to document your models in the code and then reorder the created comment there.
Of course, that will probably be overwritten by annotate once you run it again. And it will only make your life easier if you look at the table structure, but it won't help if you manually run SQL queries with SELECT * in your db.
Using Code First EF6, the database table already includes a column that the Entity does not (I previously only accessed it as a navigation property). Now in order to hopefully speed up performance, I want to add the other, existing column so I can use in in my predicates. Do I need to worry about using migrations?
Yes and no. You don't need to use migrations to create the column, obviously; however, EF may still see it as a mismatch between the model and the DB, because it bases its knowledge off of what's in the __MigrationHistory table. Try scaffolding the migration, and if it wants to add the column, call Update-Database -Script, then execute just the INSERT INTO __MigrationHistory part in order to convince EF that your DB is up to date.