In my Rails 4 app I would like to collapse my migration files into one large file (similar to schema.rb) as it's time to do some housekeeping but I'm not sure on how to access the table in the database that stores migration data so that when I run a migration I don't receive any errors/conflicts.
Question How can I access and delete the data in the table that stores migration data?
for fun, you can also manipulate these in the console by making a model class for them...
class SchemaMigration < ActiveRecord::Base; self.primary_key = :version; end
then you can do SchemaMigration.all, SchemaMigration.last.delete, etc.
Really just a substitute for using SQL, and it is very rare that you would need to mess around at this low level… generally a bad idea but cool to see how to do it :)
Another solution could be to access it through:
ActiveRecord::SchemaMigration
The answer given by David didn't work in my context.
The schema_migrations table holds the revision numbers; with the last record being the most recently executed migration. You can just manipulate these records manually.
to get the last version:
ActiveRecord::SchemaMigration.last.version
or all versions:
ActiveRecord::SchemaMigration.all.map(&:version)
Not sure why you want to do this but here you go:
ActiveRecord::Migrator.get_all_versions
I've had to do some cleanup of the sort: accumulation of seemingly trivial migrations create such pollution that things stop making sense.
As a last phase of development (not recommended once in production), you can clear out the schema_migrations table, consolidate your migrations (one-to-one with classes) and create a new table (beware: running migrate has different behaviours, depending on mysql vs postgresql)
#david-lowenfels answer is perfect for this context.
All this, naturally, assumes you haven't made errors in keys, indices, defaults. This is a serious task, but not an insensible one at the end of a development phase.
Related
I have a join table in rails that has a few entries that need to be deleted.
lets say the join table is named 'products_variants'
I found out i have a few entries in this join table that were created erroneously a while ago. I have their IDs, so i could go in phpmyadmin and delete them, but I want to make a migration to do it in case anyone uses an older database (which has happened before).
Since I don't have a ruby object representing this join table I cant do something like:
ProductsVariants.find(id_array)
How would i go about deleting these entries in a rails migration?
You can create AR class for this table inside of migration and use it for delete record.
How would you do it from the console? Whatever that is, put it in the migration. For example
class Cleanup < ActiveRecord::Migration
def up
execute("delete from product_variants where id in (1,2,3)")
end
def down
end
end
Barring a solution like maxd's answer, you can also delete them via plain 'ol SQL. If you already have the list of ids, you can do something like this:
ActiveRecord::Base.connection.execute("DELETE FROM products_variants WHERE id IN (...)")
Where ... is the list of ids to delete.
Semi-pointless side-note: Technically speaking, data manipulation is not typically done in the migrations for various reason; one of them being that you're usually not necessarily guaranteed that all (or even any) migrations will be run by your colleagues (speaking very generally here), or in the case of new local project setups (meaning, you've just pulled down the project code and are setting it up locally for the first time).
While it doesn't sound like this is an issue in your scenario, if you want to do this the Rails-y way, one alternative would be to put this in a Rake task, so that you or others can execute it as needed, rather than relying on the migrations.
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
I'm changing a relationship in my database from has_many to a has_many :through. So right now I have:
class Brand < Ar::Base
has_many :products
end
class Product < AR::Base
belongs_to :brand
end
and I'm going to add a join table.
But of course I need to update the database with data after that. I have seen that it is not good practice to do this in the confines of the migration. Where is the best place to perform this, knowing that I have to then run another migration after the data update is complete (i.e. removing the original brand_id column from the products table)?
Unless I misunderstand your question, the migration is the place to make that transformation. The purpose of the migration is to change your schema and migrate existing data to use the schema. Migrations capture the temporal aspect of layering schema changes, so that you can go forward and backward in time without leaving data in an inconsistent state. If you were to migrate your rows anywhere else, you have no guarantee that when that code runs, the schema is as it was when you wrote your migration code.
I believe that you will find support for my position in the examples on the Active Record Migrations api documentation. You might be confusing migrations with populating seed data (rake db:seed) which is handled in db/seeds.rb.
One-time changes like this can be done with ruby code in the migration. Migrations aren't just for schema changes. The idea is that migrations, by there version/datecode, are ensured to only be run once.
I think you should at least include the call to the code (maybe a rake task) that runs the data manipulation within the migration, since you have to run the 2nd migration right after the data manipulation.
If it were me, I would create a rake task that manipulates the data. This will at least remove the code from the migration and allow you to run it manually if it were ever necessary. Then code your migration and include a call to that rake task. I honestly don't see what the big deal is about not using data manipulation in a migration. Especially when you have to do things in a specific order as you are doing. They are tied so closely together, so why completely separate them?
I've been writing some migrations lately which fall under the Irreversible Migration umbrella. But they aren't end of the world irreversible. You could roll them back if you want. The scenario I have at the moment is changing a one to many relationship to a many to many relationship. It involves dropping a column and making a new join table. (as well as two lines in the models).
I was thinking, instead of aborting the down migration, I could say something like "This migration is [INSERT SCARY MESSAGE HERE], are you sure you want to proceed? Y/N" and then roll back the migration if they choose to? Just put the migration inside an if statement?
It's easy enough to make migrations irreversible, and usually there's good reason (e.g. data can't be recovered). Do these issues usually get resolved by just writing a migration which does it manually?
In my noobish mind it'd be nice to have a happy medium. Is it wise? Maybe I just don't understand when to make them non-reversible in the first place.
I always try to make the migration reversible if possible. The only time I think I've run into problems is when you go from a coarsely defined data model to a finer grained on, and then back again. I don't see any reason to not use your solution though, depending of course, on the consequences of the migration. There is also nothing stopping the person running the down migration from commenting out your raised error and writing their own code to reverse the migration, but it is far safer for you, the person writing the data model change to know how to transform back to the previous state instead of them guessing.
Just stumbled over this old post here - as I struggled somehow over the same question.
I had the other case: moving from many-to-many (HABTM) to one-to-many. Of course, I wanted to delete the join table afterwards. I was really afraid that I would forget to copy over the data from the join table during deployment. So I decided to include a "warning" migration:
class DataMigrationWarning < ActiveRecord::Migration
def change
puts("********************** Data Migration Warning **********************")
puts("Dont forgett to save the data.")
puts("Next UP migration will delete table XYZ.")
puts("Next DOWN migration will delete field A in table BCD.")
puts("press y for continue.")
puts("press anything else for stopping.")
if STDIN.gets.chomp == "y"
puts("Ok then!")
else
fail
end
# More detailed explanation...
end
end
The command line will then just show all the things in there and waits for an input from the user. y will just go to next migration. all other inputs will stop the migration and all following.
The process looked in the end like:
Migration: Create new field for new belong_to
Warning migration
Migration: Delete old join table
I'm building a Rails application using MongoDB as the back-end and MongoMapper as the ORM tool. Suppose in version 1, I define the following model:
class SomeModel
include MongoMapper::Document
key :some_key, String
end
Later in version 2, I realize that I need a new required key on the model. So, in version 2, SomeModel now looks like this:
class SomeModel
include MongoMapper::Document
key :some_key, String
key :some_new_key, String, :required => true
end
How do I migrate all my existing data to include some_new_key? Assume that I know how to set a reasonable default value for all the existing documents. Taking this a step further, suppose that in version 3, I realize that I really don't need some_key at all. So, now the model looks like this
class SomeModel
include MongoMapper::Document
key :some_new_key, String, :required => true
end
But all the existing records in my database have values set for some_key, and it's just wasting space at this point. How do I reclaim that space?
With ActiveRecord, I would have just created migrations to add the initial values of some_new_key (in the version1 -> version2 migration) and to delete the values for some_key (in the version2 -> version3 migration).
What's the appropriate way to do this with MongoDB/MongoMapper? It seems to me that some method of tracking which migrations have been run is still necessary. Does such a thing exist?
EDITED: I think people are missing the point of my question. There are times where you want to be able to run a script on a database to change or restructure the data in it. I gave two examples above, one where a new required key was added and one where a key can be removed and space can be reclaimed. How do you manage running these scripts? ActiveRecord migrations give you an easy way to run these scripts and to determine what scripts have already been run and what scripts have not been run. I can obviously write a Mongo script that does any update on the database, but what I'm looking for is a framework like migrations that lets me track which upgrade scripts have already been run.
Check out Mongrations... I just finished reading about it and it looks like what you're after.
http://terrbear.org/?p=249
http://github.com/terrbear/mongrations
Cheers! Kapslok
One option is to use the update operation to update all of your data at once. Multi update is new in the development releases so you'll need to use one of those.
You can try this contraption I just made, but it only works with mongoid and rails 3 (beta 3) at the moment. http://github.com/adacosta/mongoid_rails_migrations . It'll be upgraded to rails 3 when it goes final.
Also another gem for MongoMapper migrations https://github.com/alexeypetrushin/mongo_mapper_ext
Mongrations is a super old gem, completely deprecated. I recommend NOT using it.
Exodus is a really cool migration framework for Mongo, that might be what you want:
https://github.com/ThomasAlxDmy/Exodus
We just build this one: https://github.com/eberhara/mongration - it is a regular node module (you can find it on npm).
We needed a good mongodb migration framework, but could not find any - so we built one.
It has lot's of better features than the regular migration frameworks:
Checksum (issues an error when a previosuly ran migration does not match its old version)
Persists migration state to mongo (there is no regular state file)
Full support to replica sets
Automatic handle rollbacks (developers must specify the rollback procedures)
Ability to run multiple migrations (sync or async) at the same time
Ability to run migrations against different databases at the same time
Hope it helps!
Clint,
You can write code to do updates -- though it seems that for updating a record based on its own fields is not supported.
In such a case, I did the following and ran it against the server:
------------------------------
records = Patient.all()
records.each do |p|
encounters = p.encounters
if encounters.nil? || encounters.empty?
mra = p.updated_at
#puts "\tpatient...#{mra}"
else
mra = encounters.last.created_at
#puts "\tencounter...#{mra}"
end
old = p.most_recent_activity
p.most_recent_activity = mra
p.save!
puts "#{p.last_name} mra: #{old} now: #{mra}"
end
------------------------------
I bet you could hook into Activerecord::Miration to automate and track your "migration" scripts.
MongoDB is a schema-less database. That's why there are no migrations. In the database itself, it doesn't matter whether the objects have the key :some_key or the key :some_other_key at any time.
MongoMapper tries to enforce some restrictions on this, but since the database is so flexible, you will have to maintain those restrictions yourself. If you need a key on every object, make sure you run a script to update those keys on pre-existing objects, or handle the case of an object that doesn't have that key as you come across them.
I am fairly new to MongoDB myself, but as far as I can see, due to the flexibility of the schema-less db this is how you will need to handle it.