I'm looking for some advise on the following:
You have an app live with customers and real data
While developing new features, you need to add a column to lets say the projects table
This new column is a UID of some type which is generating by the model using a before_save
This all works fine for new projects moving forward. But all existing projects are nil for that column and everything breaks.
How do you handle his in the Rails world?
Thanks
You could simply create a rake task that pulls in all projects without a UID and and one to each project.
After you run the migration run the task. All of your projects should now have a UID.
I think this should be handled within the migration script, rather than a Rake task.
If I understand correctly, it'll only ever need to be performed once, at the time the column is added to historical records. In my mind, a migration script shouldn't leave the app with a broken data set. Migrations are designed for more than just schema changes.
Here's an example:
def self.up
change_table :projects do |t|
t.integer 'new_column'
end
Project.reset_column_information
Project.all.each do |project|
project.new_column = some_value
project.save
end
end
The reset_column_information method makes Rails aware of the new column you just added.
Related
I have a Rails 6 application used in production.
When I upgrade my app, I add migrations for new tables and/or columns.
What is the best way if I have to add new default datas ?
I use seed file for initial datas, but is there something like migrations (incremental) to add datas ? I know I can add datas in migrations but I would prefer another place for that.
[EDIT]
Do someone know this gem : https://github.com/ilyakatz/data-migrate ?
It seems to do what I am looking for.
In some projects we use to put data updates into db/data-updates/<UTC TimeStamp>_add_thing.rb. We created an own generator for those, called by rails generate data_update AddThing to create a migration calling / executing the contents from the corresponding update file.
Based on that you can put those default datas into db/default-data with similar logic for your new defaults.
You don't need to call them by mirgrations, you also may run them as rails runner db/default-data/123_xyz.rb in your differnt environments.
But if the new defaults are mandatory for your busines logic you should keep them as close as possible to the inventing migration!
If someone is looking for a simple solution, data_migrate does exactly what I wanted.
https://github.com/ilyakatz/data-migrate
Data migrations are stored in db/data.
Data migrations are included in schema migration workflow with simple commands as :
rails db:migrate:with_data
If you can read spanish, this is an interesting documentation on data_migrate : https://la-guia.platan.us/stack/ruby/rails/data_migrate
When it is possible, you should avoid using ActiveRecord models in data migrations. Models are subject to change and if, for example, you have added validations it is possible that your data migrations will not pass.
This is well explained here : https://medium.com/#jeffcoh23/why-you-should-avoid-activerecord-when-using-ruby-on-rails-data-migrate-gem-2651739395d9
A good solution is to call a model created just for data migration :
class AddVisitorRoleToUsers < ActiveRecord::Migration[6.0]
class MigrationUser < ApplicationRecord
self.table_name = :users
end
def up
MigrationUser.all.each do |user|
user.update!(role: 'visitor')
# This will NOT call User model validations
end
end
def down
raise ActiveRecord::IrreversibleMigration
end
end
I'm a SQL pro and a rails newbie.
I'm having trouble understanding how best to use the rails database abstraction. Things I can do from a SQL command line in seconds seems overtly laborious in rails.
For example:
I generated a scaffold with several models and then tried to generate migrations to add associates to those models.
The generated migrations used 'create table' which won't work because the table already exists.
So I can either drop and recreate the table or use SQL 'ALTER TABLE' statements in the migration which makes me think I should have just created the database model by hand in the first place.
What are the benefits of using the rails data abstraction as apposed to doing the SQL with modeling tools and just using schema:dump and schema:load?
Well there are many reasons why you want to use migrations over than just writing your create table and ALTER TABLE in the console.
1.When I add a new table to column to a table there is a record that it happened and the rest of the developers on the team will know about it on the next commit with out having to send out email with complicated instructions to the ones that are not SQL pro
2.When you use a migration and you want to change databases from MySql to Postgres or anything else all you have to do is change the connection script
you can rollback your changes
Example:
class AddSsl < ActiveRecord::Migration
def up
add_column :accounts, :ssl_enabled, :boolean, default: true
end
def down
remove_column :accounts, :ssl_enabled
end
end
there are much more that you can do with migrations.
I recommenced looking at
http://api.rubyonrails.org/classes/ActiveRecord/Migration.html
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
I'm attempting to write my own timestamps method that gets run during the migration. The one that is in place now adds a NOT_NULL constraint on the field, and I really really don't want that.
The problem I have is that I have a multi schema'd database. Where each major client gets their own schema. When we on-board a new client we create a new tenant record then run a migration for a newly minted schema.
The new schema is supposed to be an exact copy of the tables in the other schemas, except of course with no data.
The last migration I ran was using a slightly older version of rails. Still in the 3's but a smidge older. When it created the timestamps they were NULLable.
When I ran migration the other day (on a new rails)... Well all the fields are now NOT_NULL
I have code that was developed with the idea that updated_at was only populated when the record was updated... not when it was created. (third party apps and database "functions" create the records)..
The third party apps and database functions that create records are falling down on the new schema...
I've gone in and removed all the NOT_NULL constraints on all the tables manually, but I don't want to have to write the cleanup right into my migration task, so that all future tables are corrected..
I figured the best thing to do was to override the timestamps method that was changed, back to one that didn't break existing code.
So there's the reason I need to revert/override..
My question now is... How do I override the method. I can't see a clear class path to it and I'm not exactly sure how to override it..
Put this in a monkey patch... Easy as!
class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::TableDefinition
def timestamps(*args)
options = args.extract_options!
column(:created_at, :datetime, options)
column(:updated_at, :datetime, options)
end
end
As Maniek said. Updates to rails will be ignored because of this "fix".
But his initial offering does the same. Also to accommodate his fix, you'd need to go back through ol' migrations and replace "timestamps" with the new code. Add to that that you'd have to replace all future auto-generated migration too.
I don't think that fits well with DRY.. Nor does it fit in SPOT.
Just B carefulllll!
what's wrong with:
create_table :foo do |t|
t.text :bar
t.datetime :created_at
t.datetime :updated_at
end
?
I need to make changes to an in-use production database. Just adding a few columns. I've made the changes to the dev database with migrations. What is the best way to update the production database while preserving the existing data and not disrupting operation too much?
It's MYSQL and I will be needing to add data to the columns as well for already existing records. One column can have a default value (it's boolean) but the other is a timestamp and should have an arbitrary backdated value. The row counts are not huge.
So if I use migrations how do I add data and how do I get it to just do the two (or three - I add data -latest migrations on the production db when it wasn't initially built via migrations (I believe they used the schema instead)?
I always follow this procedure:
Dump prod database with mysqldump command
Populate dev / test database with dump using mysql command
Run migrations in dev / test
Check migration worked
Dump prod database with mysqldump command (as it may have changed) keeping backup on server
Run migrations on prod (using capristano)
Test migration has worked on prod
Drink beer (while watching error logs)
It sounds like you're in a state where the production db schema doesn't exactly match what you're using in dev (although it's not totally clear). I would draw a line in the sand, and get that prod db in a better state. Essentially what you want to do is make sure that the prod db has a "schema_info" table that lists any migrations that you >don't< ever want to run in production. Then you can add migrations to your hearts content and they'll work against the production db.
Once you've done that you can write migrations that add schema changes or add data, but one thing you need to be really careful about is that if you add data using a migration, you must define the model within the migration itself, like this:
class AddSomeColumnsToUserTable < ActiveRecord::Migration
class User < ActiveRecord::Base; end
def self.up
add_column :users, :super_cool, :boolean, :default => :false
u = User.find_by_login('cameron')
u.super_cool = true
u.save
end
def self.down
remove_column :users, :super_cool
end
end
The reason for this is that in the future, you might remove the model altogether, during some refactoring or other. If you don't define the user class on line "User.find_by_login..." the migration will throw an exception which is a big pain.
Is there a reason you are not using the same migrations you used in your dev environment?
Adding a column with add_column in a migration should be non-destructive: it will generate a "ALTER TABLE" statement. If you know what you're going to put into the columns once created, you can fill in the values within the migration (you may choose a less time-consuming alternative if the row counts are large).
Removing or altering the definition of a column is, I think, platform-dependent: some will allow deletion of a column in place, others will perform a rename-create-select-drop sequence of commands.
To get more specific, we need more information: what kind of migration are you looking at, what platform are you running on, do you need to set values as part of the migration? Stuff like that would help a lot - just edit the question, which will push it back up the list.