I made a new Rails 3 app today, added a simple migration, and for some reason, nothing happens when I do rake db:migrate. It simply pauses a few seconds, then returns to the command prompt, with no errors or anything. Schema.rb and the database stay empty.
Any ideas what could be going on? I've made many apps and never had this problem. Everything is a totally standard setup too.
There's a few reasons why your migrations won't run, but the most common is that the system is already under the impression that all the migrations you've defined have already run.
Each migration creates an entry in the schema_migrations table with the version column corresponding to the identifier number. If you want to force a migration to re-run you can usually back it out and retry it. For example, if you had 20100421175455_create_things.rb then you would re-run it using:
rake db:migrate:redo VERSION=20100421175455
A common situation is that your migration has failed to run in the first place, that it generated an exception for instance, and yet Rails still considers it complete. To forcibly re-run a migration, delete the corresponding record from the schema_migrations table and run rake db:migrate again.
One way to avoid this kind of problem in the future is to define your migrations with an automatic back-out procedure:
class CreateThings < ActiveRecord::Migration
def self.up
# ... (migration) ...
rescue
# If an exception occurs, back out of this migration, but ignore any
# exceptions generated there. Do the best you can.
self.down rescue nil
# Re-raise this exception for diagnostic purposes.
raise
end
end
If you have a mistake in your migration you will see the exception listed on the console. Since the migration has automatically been rolled back you should be able to run it again and again until you get it right.
I faced the same problem. I did a kind of a short hack that helped me. I am posting it just in case anyone wants a short and sweet solution. I agree with what Tadman is saying
"the system is already under the impression that all the migrations
you've defined have already run"
What I did was to change the name of the migrate file in the /app_folder/db/migrate folder. I think the numeric part in the name of the ruby migrate file is the time at which the file was created.
You can add say 1 , to the filename, every time you want to re-run the migrate. After changing the name drop/delete the table (I used mysql command line tool for deleting) and then run rake db:migrate and the migrations should be done.
Calling spring stop might solve your problems.
Well, I found out what was causing my problem. I'm using the slim_scrooge gem and commenting it out makes everything proceed normally. Don't know why though...
I faced similar problem today while migrating plugin for Redmine using
rake redmine:plugins:migrate RAILS_ENV=production NAME=plugin_name
where plugin_name is actually plugin name defined in init.rb of the plugin.
I struggled for 4 hours and finally figured out that my plugin directory name was not the same as the plugin name (note redmine_ prefix):
~/redmine/plugins/redmine_plugin_name
So, make sure your plugin is placed in folder named after plugin name. I believe it applies to other rails apps as well.
Related
I added a migration in branch "add_dogs" with migration db/migrate/20221220155010_create_dogs.rb, and ran db:migrate.
Later on, I changed branches (without a merge), and ultimately abandoned the "new_dogs" branch.
Later later on, I checked out "add_cats" branch with db/migrate/20221101010101_create_cats.rb, and ran db:migrate. So far, all is well.
But then I tweak the "add_cats" migration (before committing anything), and ran db:rollback so I can run it again. I get this error:
ActiveRecord::UnknownMigrationVersionError:
No migration with version number 20221220155010.
I can still run db:migrate on new migrations just fine, but not db:rollback or db:migrate:redo.
This makes sense, because the database has a record of applying 20221220155010, but that migration file no longer exists, so there is no way to roll it back.
How can I get past this?
Here are three ways to deal with a missing migration file, depending on your needs and access:
For a quick temporary fix, you can roll back just the migration you're currently editing so you can run it again. This may be useful if the other migration is still in the pipeline on the other branch and both eventually will get merged.
rake db:migrate:down VERSION=20230101010101
// This is the version of the migration you WANT to rollback, not the missing one.
If the missing migration will never come back, you want a permanent fix. The simplest way is to remove that record from the database. You can do this from your favorite SQL client, rails console, etc. (I suppose you could even write a migration to do that, but that seems mighty sketchy.)
DELETE FROM schema_migrations WHERE version = '20221220155010'
-- This is the version of the migration that is MISSING, not the one you are working on.
If you don't have direct access to the database for whatever reason, you can give Rails a placebo to rollback. Ensure the timestamp in the filename matches the missing migration's version number.
Create a file named db/migrations/20221220155010_just_kidding.rb:
class JustKidding < ActiveRecord::Migration
def change
# nothing to see here.
end
end
Then, rails db:rollback will roll back that no-op migration and delete 20221220155010 from the schema_migrations table. You can now delete the placebo migration forever and you'll be in good shape as far as running migrations and rollbacks.
However...don't forget that the effects of the old migration are still in your schema. Maybe you're stuck with a new, unused 'dogs' table or an extra column on a table. Maybe that's benign on your dev box, but you certainly don't want that cruft on a production environment. All the advice in this answer assumes you're on a throw-away environment and that the effects of the old migration aren't a problem. Tearing down your whole database and rebuilding may become a more attractive option in this case.
One of the realy take-aways here is... don't let this happen in the first place! Ideally, you should rollback any new, uncommitted migrations before changing away from a branch. But...things happen...
p.s. If there is a way to do this from the command line, I'd love to learn it. I'm imagining something like rails db:migrate:delete VERSION=20230101010101 might be handy in a hackish kind of way.
A company I recently started working for have, at some point in time, managed to delete one of their migration files - or at least that is what I believe. This file created a table of let's say users, but this table has since been removed.
I would normally not mind had it not been for the fact that building the project locally will fail unless I temporarily shoehorn in an early migration that includes this table creation, this is because there are later migrations that both modify columns on this supposed users table and eventually removes the table. What I have done recently is just add it to an existing migration file that was creating some other table, let's say products, which I have been "carrying" around from branch to branch being real careful that I don't commit it.
It looks something like this (examples):
class AddProductsTable < ActiveRecord::Migration
def change
create_table :products do |t|
end
end
end
Which I have changed to:
class AddProductsTable < ActiveRecord::Migration
def change
create_table :products do |t|
end
create_table :users do |t|
end
end
end
I am essentially wondering if committing this would do any harm on production. After all, the migration has already been run so it wouldn't execute it again? I'm very tempted to just do this as it's getting old real fast to always remember to recreate or lug this piece of code around when I need to rebuild, but I don't feel confident enough that something bad won't happen nor have I found much in the way of answers on Google.
PS., I'm the only developer on the project currently and they don't have any other developers who I could ask this question to.
If you haven't already ran the migration, then no problem editing the migration file. Otherwise:
Quoting from rails guides:
In general, editing existing migrations is not a good idea. You will be creating extra work for yourself and your co-workers and cause major headaches if the existing version of the migration has already been run on production machines. Instead, you should write a new migration that performs the changes you require. Editing a freshly generated migration that has not yet been committed to source control (or, more generally, which has not been propagated beyond your development machine) is relatively harmless.
Source: Changing existing migrations
As long as the migration has been ran in production, this should not cause any issues in the production environment. You could even delete it if you must (Note: not recommended if you have other team members as they will face the issue you are facing if they haven't already ran the migration locally).
To check whether a migration has been ran in a given environment you can use the following rake command:
RAILS_ENV=<environment> bundle exec rake db:migrate:status
For example, doing this in development you will get this:
RAILS_ENV=development bundle exec rake db:migrate:status
up 20190201100000 Add soups table
up 20190301100000 ********** NO FILE **********
up 20190601100000 Add supplies table
up 20190501100000 Remove food column from supplies table
up 20190601100000 Add test table
up 20190701100000 Add foo column to test
up 20190801100000 Add products table
down 20191119030000 I havent ran this
The above tells you which migrations have been ran (indicated by up) and which haven't been ran (indicated by down). It also tells you whether there is a file associated with the migration or if the file has been deleted (see the one above with no file). When you deploy your project to production, the down migrations will be ran so modifying the up migrations will not do anything.
I recommend you run the following in production to make sure the migration you are modifying has the down status.
RAILS_ENV=production bundle exec rake db:migrate:status
I'm doing my first project on RoR I decided to use Devise gem for the authentication everything is working fine in Localhost, right now i'm able to sing_up, log_in, reset pass and some other default features that comes with devise.
What I don't understand is how everything gets setup once i push the project into production environment. Let me explain better.
-I see that devise generates db/schema.rb file based on the user.rb model that devise auto generate, these schema i assume needs to be created in the production db.
I have to push the project into git repo but..
1) How the code gets executed once is published
2) How the user table gets created
3) where do I need to specify the db connection (db, user, pass)
In general how is the process to migrate into production env?
I appreciate your help.
In rails db schema is controlled using migrations. Migrations are usually very small files living in db/migrations folder. Each of them defines a single subclass of ActiveRecord::Migration class.
Each migration is supposed to go both-ways, so it has to implement both up and down method (for rollback). In majority of the changes those are obvious (like when up action creates a table, down action should destroy it). For migrations like this you can use change function - rails will create down function automatically (assuming that all actions are reversible - some of the methods even accept extra arguments to make it possible like remove_column :table, :field, :integer. :integer bit might seem redundant here, however the action is irreversible without that information).
You run your migrations using db:migrate rake task. Effectively it will run all the migration which has not been run against current database. When it is done, it will dump your current database into db/schema.rb file. This file is used only as a snapshot of your current database schema, which is of great help if you need to get back to previous version of your application (which is why schema.rb should always be checked into your source control). It is also possible to recreate your database directly from that file rake db:schema:load, however it will cost you all your data.
If you check your database (any environment), you will notice there is one extra table called schema_migrations which does not appear in your schema file. This table contains all the migrations that has been executed against this database. This way rails will not retry migrations that already has been run.
There are many advantages of using migration system:
It is easy for every developer to bring their local database up to the latest schema.
It is easy to rollback your latest changes if deployment failed. (Not always the case)
So how to use it in production? You simply has to point the rake task to the right database:
RAILS_ENV=production rake db:migrate
You can find your database configuration in config/database.yml file. Note that this is the best practice not to store that file in source control and in case of production it is not advisable to store your database password there. Normally you will need to set it through environment variable.
It is important to remember couple of things about migrations:
Never, ever change the existing migration if there is a chance that someone has already run it. Especially, do not modify migrations that has run in production unless you are 100% sure what you are doing (for example fixing rollback). Since database remembers which migrations has run, it might lead to getting your local db out of sync with production db and hence lead to bugs.
Never ever change database schema manually - every change has to be made through migrations.
Well depending on where you're hosting your code you should still do a rake db:migrate to make the changes to the production DB.
If deployed on Heroku, run the following in the console:
heroku run rake db:migrate
Hope that answers your question?
In other words I'd like to know the best way to make sure that self.down actually rolls back self.up before running the migration in question.
What can I do if I need to rollback a migration but self.down doesn't serve the purpose?
Which is the best practice when dealing with potentially destructive migrations?
Just a database backup?
Thanks,
Duccio.
You should be developing on a development database which should not contain live data. Therefore it should not matter if the data is destroyed as you can easily generate it again?
A database backup might be appropriate if you find yourself in a situation where your development data is important but not ideal.
Typically migrations should contain only schema changes. In that case it should be very safe & easy to run the migrations in the dev/test environment. If something goes wrong you can alway re-create the database and populate it with some test data. But if you have some data related migrations to be tested, things might go wrong when you actually run them on production.
In that case as you mentioned database backup is what you should rely on. Come with a proper & quick restore mechanism before deploying.
To make sure the migrations behave as you want you should experiment in your development environment.
Run the command
rake -T
to show you the available tasks such as
rake db:migrate
or
rake db:rollback
Each migration runs inside a transaction. Keep that in mind. That means that if something would go wrong inside a single migration, the migration is rolled back (and if there are any following they are not executed).
To test migrations, be it up or down I insert a lot of puts-statements, to check everything has worked has supposed to, and then in my last line I raise an exception. This will make rails think the migration has failed, and it will rollback the operation (as if it never happened).
When I am sure everything works as it should, I remove the raise line and let the migration really work.
In your case, you would test with the raise, remove the raise and NOT run it again I assume :)
Hope this helps.
I have been running a big Rails application for over 2 years and, day by day, my ActiveRecord migration folder has been growing up to over 150 files.
There are very old models, no longer available in the application, still referenced in the migrations. I was thinking to remove them.
What do you think? Do you usually purge old migrations from your codebase?
The Rails 4 Way page 177:
Sebastian says...
A little-known fact is that you can remove old migration files (while
still keeping newer ones) to keep the db/migrate folder to a
manageable size. You can move the older migrations to a
db/archived_migrations folder or something like that. Once you do trim
the size of your migrations folder, use the rake db:reset task to
(re-)create your database from db/schema.rb and load the seeds into
your current environment.
Once I hit a major site release, I'll roll the migrations into one and start fresh. I feel dirty once the migration version numbers get up around 75.
I occasionally purge all migrations, which have already been applied in production and I see at least 2 reasons for this:
More manageable folder: it is easier to spot a new migration.
Cleaner text search results: global text search within a project does not lead to tons of useless matches because of some 3-year-old migration when someone added or removed some column which anyway does not exist anymore.
They are relatively small, so I would choose to keep them, just for the record.
You should write your migrations without referencing models, or other parts of application, because they'll come back to you haunting ;)
Check out these guidelines:
http://guides.rubyonrails.org/migrations.html#using-models-in-your-migrations
Personally I like to keep things tidy in the migrations files. I think once you have pushed all your changes into prod you should really look at archiving the migrations. The only difficulty I have faced with this is that when Travis runs it runs a db:migrate, so these are the steps I have used:
Move historic migrations from /db/migrate/ to /db/archive/release-x.y/
Create a new migration file manually using the version number from the last run migration in the /db/archive/release-x.y directory and change the description to something like from_previous_version. Using the old version number means that it won't run on your prod machine and mess up.
Copy the schema.rb contents from inside the ActiveRecord::Schema.define(version: 20141010044951) do section and paste into the change method of your from_previous_version changelog
Check all that in and Robert should be your parent's brother.
The only other consideration would be if your migrations create any data (my test scenarios contain all their own data so I don't have this issue)
Why? Unless there is some kind of problem with disk space, I don't see a good reason for deleting them. I guess if you are absolutely certain that you are never going to roll back anything ever again, than you can. However, it seems like saving a few KB of disk space to do this wouldn't be worth it. Also, if you just want to delete the migrations that refer to old models, you have to look through them all by hand to make sure you don't delete anything that is still used in your app. Lots of effort for little gain, to me.
See http://edgeguides.rubyonrails.org/active_record_migrations.html#schema-dumping-and-you
Migrations are not a representation of the database: either structure.sql or schema.rb is. Migrations are also not a good place for setting/initializing data. db/seeds or a rake task are better for that kind of task.
So what are migrations? In my opinion they are instructions for how to change the database schema - either forwards or backwards (via a rollback). Unless there is a problem, they should be run only in the following cases:
On my local development machine as a way to test the migration itself and write the schema/structure file.
On colleague developer machines as a way to change the schema without dropping the database.
On production machines as a way to change the schema without dropping the database.
Once run they should be irrelevant. Of course mistakes happen, so you definitely want to keep migrations around for a few months in case you need to rollback.
CI environments do not ever need to run migrations. It slows down your CI environment and is error prone (just like the Rails guide says). Since your test environments only have ephemeral data, you should instead be using rake db:setup, which will load from the schema.rb/structure.sql and completely ignore your migration files.
If you're using source control, there is no benefit in keeping old migrations around; they are part of the source history. It might make sense to put them in an archive folder if that's your cup of coffee.
With that all being said, I strongly think it makes sense to purge old migrations, for the following reasons:
They could contain code that is so old it will no longer run (like if you removed a model). This creates a trap for other developers who want to run rake db:migrate.
They will slow down grep-like tasks and are irrelevant past a certain age.
Why are they irrelevant? Once more for two reasons: the history is stored in your source control and the actual database structure is stored in structure.sql/schema.rb. My rule of thumb is that migrations older than about 12 months are completely irrelevant. I delete them. If there were some reason why I wanted to rollback a migration older than that I'm confident that the database has changed enough in that time to warrant writing a new migration to perform that task.
So how do you get rid of the migrations? These are the steps I follow:
Delete the migration files
Write a rake task to delete their corresponding rows in the schema_migrations table of your database.
Run rake db:migrate to regenerate structure.sql/schema.rb.
Validate that the only thing changed in structure.sql/schema.rb is removed lines corresponding to each of the migrations you deleted.
Deploy, then run the rake task from step 2 on production.
Make sure other developers run the rake task from step 2 on their machines.
The second item is necessary to keep schema/structure accurate, which, again, is the only thing that actually matters here.
It's fine to remove old migrations once you're comfortable they won't be needed. The purpose of migrations is to have a tool for making and rolling back database changes. Once the changes have been made and in production for a couple of months, odds are you're unlikely to need them again. I find that after a while they're just cruft that clutters up your repo, searches, and file navigation.
Some people will run the migrations from scratch to reload their dev database, but that's not really what they're intended for. You can use rake db:schema:load to load the latest schema, and rake db:seed to populate it with seed data. rake db:reset does both for you. If you've got database extensions that can't be dumped to schema.rb then you can use the sql schema format for ActiveRecord and run rake db:structure:load instead.
Yes. I guess if you have completely removed any model and related table also from database, then it is worth to put it in migration. If model reference in migration does not depend on any other thing, then you can delete it. Although that migration is never going to run again as it has already run and even if you don't delete it from existing migration, then whenever you will migrate database fresh, it cause a problem.
So better it to remove that reference from migration. And refactore/minimize migrations to one or two file before big release to live database.
I agree, no value in 100+ migrations, the history is a mess, there is no easy way of tracking history on a single table and it adds clutter to your file finding. Simply Muda IMO :)
Here's a 3-step guide to squash all migrations into identical schema as production:
Step1: schema from production
# launch rails console in production
stream = StringIO.new
ActiveRecord::SchemaDumper.dump(ActiveRecord::Base.connection, stream); nil
stream.rewind
puts stream.read
This is copy-pasteable to migrations, minus the obvious header
Step 2: making the migrations without it being run in production
This is important. Use the last migration and change it's name and content. ActiveRecord stors the datetime number in it's schema_migrations table so it knows what it has run and not. Reuse the last and it'll think it has already run.
Example: rename 20161202212203_this_is_the_last_migration -> 20161202212203_schema_of_20161203.rb
And put the schema there.
Step 3: verify and troubleshoot
Locally, rake db:drop, rake db:create, rake db:migrate
Verify that schema is identical. One issue we encountered was datetime "now()" in schema, here's the best solution I could find for that: https://stackoverflow.com/a/40840867/252799