Is there a quick/easy way to make a migration that adds a new column between two existing columns?
Note: I googled and couldn't find an obvious answer. But I also am curious if it's even good practice (since if for some reason other column/s were removed, then the migration may fail?)
If you haven't commited your changes to production you can reorder migrations by rolling them back and then changing the timestamps in the file name.
If thats not an alternative you can actually re-order the columns on some databases directly with SQL even if its not part of the migrations DSL. Migrations are after all really just a DSL to create SQL strings and run them in a repeatable way across environments.
If you can't generate the SQL you want with the DSL you can always use execute to execute a raw DSL string.
# MySQL example
class ReorderYourTableName < ActiveRecord::Migration[5.2]
def change
execute "alter table yourTableName change column yourColumnName yourColumnName dataType after yourSpecificColumnName;"
end
end
However on some DBs you can't actually reorder the columns without extensive steps of creating new columns and shuffling the data around.
Related
I need to rename a column in a Rails project that has enabled strong_migrations. Im looking into the official guide and it specified the following steps to doing it:
Create a new column.
Write to both columns.
Backfill data from the old column to the new column.
Move reads from the old column to the new column.
Stop writing to the old column.
Drop the old column.
So as far as I understand I need about 4 PRs to simply change the name of the column.
This PRs would be:
Create the migration for the new table and start using the new field.
Backfill data and move reads from the old column to the new one.
Add self.ignored_columns = ["some_column"] to the model and remove the old logic that use that old column.
Create the migration to drop off the old column.
It's that right? Or Im missing something and maybe some of this PRs are not necessary?
Can be done with 3. With ways to migrate the data (a and b)
Add the new column and code that will start writing to it. a) Add a worker that will backfill the data after the deploy
Change code to start reading the new column and add the old column to ignored columns. b) Backfill the data in a migration before deploy.
Remove the old column
Data migration style will depend on the size of your database. If it's small enough, you can do it in the migration during deploy. If it's bigger, you'll have to migrate the data beforehand. Either a worker or a console script. In this case it's still a good idea to have the backfill in a migration as well to ensure the data is migrated everywhere (local db of every developer, staging environments and so on).
We have a SAAS platform written in Rails using the postgres' schema based multitenancy and Apartment gem. The different schemas are identical, with same number of tables and same columns in each table. We want to migrate to foreign key based multitenant system where we want to merge all the records from different schemas into a single schema, identifying each record with a tenant_id. What is the proper way of merging all the records from the different schemas, and preserving the foreign key relationships.
This is a situation that will need care. I think (I could be wrong) that the best approach is to add to all tables tenant_id and original_id ... before attempting migration populate original_id in all tables with id of that record. Essentially this is to have a record of what the value of id was before the merging.
After merge you can then run a rake task that rebuilds the associations. So if you had...
class Foo
has_many :bars
Your migration script would do (after migration)
Bar.all.each do |bar|
foo = Foo.find_by(tenant_id: bar.tenant_id, original_id: bar.foo_id)
bar.update_column(:foo_id, foo.id)
end
You'd need to do something similar for every relation, so it's a bit of a slog.
Hopefully, someone else will come up with a better solution.
NOTE THIS IS NOT IDEMPOTENT. If it errors, you can't restart it except by redoing the merging completely.
I am working on a web app that already has a schema in place (aka in prod) with a certain number of tables (A, B, C...).
Table A has an attribute that corresponds to an enum from table B. Problem is that I can have only one item of that enum list in my dedicated column in table A. But I want my objects from table A to have many of them. So I created a join table A_B and a has_many through association with my table B.
The first consequence is that I need to fill my join table with data from the previous schema architecture. To be clearer, they were objects from table A that were associated with one element of the table B enum. I need to report these simple relationships (only one element from enum list in table B is associated with table A objects) in my newly created join table.
Here's the type of things I'd like to do:
list_of_ids = []
Model_A.where(attribute: 0).each { |r| list_of_ids << r.id }
a.each { |el| A_B.create(tableA_id: el, tableB_id: 0) }
Where should I write and execute these lines of code that will update my data ?
As stated in my comments, I would put this "data-update" logic in the same migration file of the join table creation.
Why in a migration file?
the data conversion needs to be done only once, AFTER you created the join table and BEFORE you remove the column which holds the foreign key. If you do this data conversion after the column removal, you will get an error saying that your code is trying to access column that does not exist anymore.
the migration is responsible for changing the DB structure AND for the data integrity.
Why not a rake task?
rake tasks are meant to be run several times, not only once. The usual tasks are "send_emails", "update_expiration_dates", "compute_cache", "close_inactive_users_account", etc.
the data you have before your conversion has_many -> HABTM have to be updated to follow the new structure. the rake task could not be ran, and then your data would not be updated, therefore you would loose the association between your models (removing the foreign_key column before running the rake task would make you loose this data).
your data-conversion logic must happen after the migration creating the join table and BEFORE the migration removing the foreign key's column. There are clear way to say to your rails app: "do this migration, then stop, run this task, then do this migration". It will run the migrations consecutively. If you have these 2 migrations pending, they will be run at once unless you specify otherwise (which is not common at all), and then you rake task will be useless because it relies on the fact that the foreign key column still exists.
In addition to MrYoshiji's answer (too long for comments):
Migrations are intended for one-off changes to both database schemas and the data therein. The official guide mentions this, too.
In your case, populating the join table is an appropriate thing to do in a migration. Otherwise data would be left in an invalid or incorrect state vis-a-vis the schema and model changes you have made. Running migrations is a typical step in deploying a Rails app. Since you probably do not want to deploy without updating the data, having it as part of a migration is a great solution. In contrast, if you created a custom rake task to update the data then you would need to remember to run it manually after deploying or add it as a deployment step, neither of which is a very good option.
Running Rails 4 with Postgres 9.4 in development and production. I've got a large table that has grown by migrations over time. Because of the many different columns on the table, I want to reorder things so that the columns are grouped more logically. In other words, some column elements naturally group together based on what information they capture.
I found a discussion on using after: in a migration to reorder columns (using ALTER TABLE in SQL), at this stack overflow discussion. I then went ahead and set up a migration to do that. After running the migration I notice that my schema.rb file hasn't changed. Looking at the columns in the database (postgres on development), nothing has changed there either.
Further research led me to the Postgres wiki, which states there is currently no support for altering column position in Postgres.
This isn't mission critical, but it would make life easier. My question is:
If I simply move lines in the schema.rb file up or down in order to position them as desired, will that cause any problems? Since I expect to provision any new database with rake db:schema:load, it wouldn't seem like that should break anything. Further, since the previous migrations aren't materially changed (just the order of columns they output to schema.rb) everything should be internally consistent, no?
If that's a bad idea, can I simply go back into the earlier migrations, add the syntax for after: :some_column to each column element so they correctly set up the schema.rb file?
My assumption is that when I rebuild my production database from scratch with seed data, it will use the schema structure to correctly create the table in the desired order. Right now it's not a real, deployed database with end users, so it doesn't seem like an issue. Using option #1 above seems like the easiest solution, as long as it doesn't break anything.
Many people new to postgresql often ask if it has support for altering
column positions within a table. Currently it does not; if you want to
change column positions, you must either recreate the table, or add
new columns and move data. The idea of allowing re-ordering of column
position is not one the postgresql developers are against, it is more
a case where no one has stepped forward to do the work.
ref
If I simply move lines in the schema.rb file up or down in order to
position them as desired, will that cause any problems?
no. It does nothing with column order in PG.
If that's a bad idea, can I simply go back into the earlier migrations, add the syntax for after: :some_column to each column element so they correctly set up the schema.rb file?
after: :some_column option does nothing with column order if you use PG
The order of your columns does not matter to Ruby or Rails.
As to whether it matters to any code you wrote, only you and your tests can answer that. If you're only using ActiveRecord and not doing any straight SQL that references column number, you should be fine.
As a workaround, you can use annotate to document your models in the code and then reorder the created comment there.
Of course, that will probably be overwritten by annotate once you run it again. And it will only make your life easier if you look at the table structure, but it won't help if you manually run SQL queries with SELECT * in your db.
I would like to run a migration to create a User table that starts counting from 10000 for the user_id column. I would like to avoid using DB specific syntax via the execute method, because my production and development servers aren't the same.
Is there a rails way to do this?
If you need to do this in a database-independent way, I don't see any easier option than creating 10000 Users in your migration, and then doing "truncate table users" or just deleting users with id < 10000. This will probably take a little bit of time.
In postgres you can redefine starting number of the sequence responsible for generating these IDs. In MySQL you can do it also, but differently: "ALTER TABLE theTableInQuestion AUTO_INCREMENT=1234"
Of course you could add this functionality to ActiveRecord::Migration class, and provide implementation in DB-specific drivers.
All depends on how far you want to go with this :)