I have an application that requires a sequence to be present in the database. I have a migration that does the following:
class CreateSequence < ActiveRecord::Migration
def self.up
execute "CREATE SEQUENCE sequence"
end
def self.down
execute "DROP SEQUENCE sequence"
end
end
This does not modify the schema.rb and thus breaks rake db:setup. How can I force the schema to include the sequence?
Note: The sequence exists after running rake db:migrate.
Rails Migrations because they aim toward a schema of tables and fields, instead of a complete database representation including stored procedures, functions, seed data.
When you run rake db:setup, this will create the db, load the schema and then load the seed data.
A few solutions for you to consider:
Choice 1: create your own rake task that does these migrations independent of the Rails Migration up/down. Rails Migrations are just normal classes, and you can make use of them however you like. For example:
rake db:create_sequence
Choice 2: run your specific migration after you load the schema like this:
rake db:setup
rake db:migrate:up VERSION=20080906120000
Choice 3: create your sequence as seed data, because it's essentially providing data (rather than altering the schema).
db/seeds.rb
Choice 4 and my personal preference: run the migrations up to a known good point, including your sequence, and save that blank database. Change rake db:setup to clone that blank database. This is a bit trickier and it sacrifices some capabilities - having all migrations be reversible, having migrations work on top of multiple database vendors, etc. In my experience these are fine tradeoffs. For example:
rake db:fresh #=> clones the blank database, which you store in version control
All the above suggestions are good. however, I think I found a better solution. basically in your development.rb put
config.active_record.schema_format = :sql
For more info see my answer to this issue -
rake test not copying development postgres db with sequences
Check out the pg_sequencer gem. It manages Pg sequences for you as you wish. The one flaw that I can see right now is that it doesn't play nicely with db/schema.rb -- Rails will generate a CREATE SEQUENCE for your tables with a serial field, and pg_sequencer will also generate a sequence itself. (Working to fix that.)
Related
Problem statement:
Let's say I created a new column column_1 in a table table_1 through rails migration. And now I want to populate the data into column_1 doing some computation but it's a one time task.
Approaches:
Is it preferred to do it through migration. or should I add it in seeds and then populate it and remove it back again
or is there any other best practise.
Generally, we use Rails Migrations to migrate our application’s schema, and Rake tasks to migrate our production data.
There have only been a few cases where we have used Rails Migrations to ensure that a data migration took place as a part of the deployment.
In all other cases, using a Rake task provides us with more flexibility, and less maintenance.
See detailed explanation here
Even though there are different approaches, it is better to do that in in Rake or Migrations but not seed.
Rake tasks:
Rake tasks are generally preferred to do maintenance or data migration jobs over a collection of data.
Example of rake:
lib/tasks/addData.rake
desc "TODO"
task :my_task1 => :environment do
User.update_all(status: 'active')
end
Example of doing it in migration:
If you add status field to user:
class AddStatusToUser < ActiveRecord::Migration
def up
add_column :users, :status, :string
User.update_all(status: 'active')
end
def down
remove_column :users, :status
end
end
Why Not seeds:
Seeds are generally like database Dump, seed files are used to fill the data into Database for the first time when the application is started. So that the application is kickstarted with different data.
Examples are Application settings, Application Configurations, Admin users etc.
This is actually an opinion based question
Is it preferred to do it through migration. or should I add it in seeds and then populate it and remove it back again
It depends on how long it's gonna take.
1. Migration:
If it's one-time task go with migration but only if the task is going to run for few minutes.
2. Rake Task:
If the task is one-time but it might take a few hours it should be a rake task, not a migration.
One time task ? Definitely will go with migration as it will only executed when the migration take place. And won't be executed afterwards
Rake task is considered as too much, since you will need it to run only once. The task script remain there until you decide to remove it. But, it is totally doesn't make sense ( to build something which will be removed in the near future, unless, for testing purposes in some special cases )
If you're asking about best practices, people will tend to have different approach. It will depend on each case that we are trying to solve. But, in common, there are some cases which are shareable.
I have 3 different schemas in one Rails application. My preparation of the test database using rake db:test:prepare fails with:
psql:/Users/me/myapp/db/structure.sql:7417: ERROR: relation "schema_migrations" does not exist
LINE 1: INSERT INTO schema_migrations (version) VALUES ('20131213203...
That's because it is not proper setting the Postgres search_path before doing all the insertions to the schema_migrations table. I haven't messed with this code in about 8 months and can't remember what I did. I haven't the faintest idea of how I even got those other schema to dump.
You may want to try rake db:structure:dump and / or rake db:schema:dump and try re-running rake db:test:prepare. The former should create the structure.sql and the later the schema.db
I was able to accomplish what I needed by doing two things:
Overriding the purge task under db:test in AR's railties to call a custom drop_database_objects method.
Using a little-known attribute in my database.yml: schema_search_path: public
The first thing lets me drop only the database objects I want, leaving my other support databases intact.
The second thing just creates the structure from my main database, and doesn't try to create the structure from the other databases. It looks like a bug in that it structure:dump doesn't set the schema search path appropriately at the end of the structure.sql script, right before the inserts into the schema_migrations table in a multi-schema instance. These fixes make that not necessary.
The problem is the following:
I have db/seed.rb full of initial data.
One of migrations depends on data this seed provides.
I'm trying to deploy my app from empty db.
Result is:
RAILS_ENV=production rake db:migrate - fails due to lack of initial data
RAILS_ENV=production rake db:seed - fails due to pending migrations
I wanted to somehow tell rake to ignore pending migrations, but unable to do it so far.
UPDATE (due to additional experience)
Sometimes migrations and model code goes out of sync, so migrations are not being run.
To avoid this problem recently used redefining of model in migrations:
# reset all callbacks, hooks, etc for this model
class MyAwesomeModel < ActiveRecord::Base
end
class DoSomethingCool < ActiveRecord::Migration
def change
...
end
end
I am not very sure if this will help you. But I was looking for something and found this question. So it looks like this might help:
In RAILS_ROOT/config/environments/development.rb
Set the following setting to false:
# NOTE: original version is `:page_load`
config.active_record.migration_error = false
In my situation it now does not show the pending migration error anymore. Should work for rake tasks and console for the same environment as well.
Source in rails/rails
Rename the migration dependent on the data from:
20140730091353_migration_name.rb
to
.20140730091353_migration_name.rb
(add a dot at the start of the filename)
Then run rake db:seed (it will no longer complain on the pending migrations) and then rename back the migration.
If you have more migrations following after, you have to rename all of them or just move it temporary away.
Rails stores migration information in a table called schema_migrations.
You can add the version from your migration into that table to skip a specific migration.
The version is the number string which comes before the description in the file name.
[version]_Create_Awesome.rb
I had a similar issue. I commented out the add_column lines and ran the rake db:migrate commands and then removed the comment when I will need it for the testing or production environment.
There is no way unless you monkey patch the Rails code. I strongly advise you to fix your migrations instead.
A migration should not depend on the existence of some data in the database. It can depend on a previous migration, but of course absolutely not on the data on the db.
If you came across the "pending migrations" issue when trying to seed your data from within a running Rails application, you can simply call this directly which avoids the abort_if_pending_migrations check:
ActiveRecord::Tasks::DatabaseTasks.load_seed
See where seeds are actually called from within ActiveRecord:
https://github.com/rails/rails/blob/v6.0.3.2/activerecord/lib/active_record/railties/databases.rake#L331
and see the DatabaseTasks docs:
https://apidock.com/rails/v6.0.0/ActiveRecord/Tasks/DatabaseTasks
https://apidock.com/rails/v6.0.0/ActiveRecord/Tasks/DatabaseTasks/load_seed
I have a question regarding my migrations in rails.
Normally when i want to add a colum to a model for i dont make extra migrations but instead i perform this steps:
rake db:rollback
next i change the migration file in db/migrations and rerune:
rake db:migrate
The biggest problem is that when i do this i loose my data.
Previous i wrote migrations from the command line with for example
rake g migration Add_Column_House_to_Users house:string
The problem with this approach is that my db/migrations folder afterwards get very large and not very clear! I mean at the end i dont know wich variables the object has! Im not an expert in rails and would like to ask you how to keep the overview over the migrations!Thanks
Just a minor thought - I just use the file db/migrate/schema.rb to determine whats in the database as opposed to tracking through the migrations
You definitely shouldn't use db:rollback with a table with existing data.
I have a few production RonR apps with a ton of data and there are 100+ entries in the migrations table and adding new migrations to tweak tables is the rails way to do things. Not sure what you mean by lucid, but your schema and data model are going to change over time and that is ok and expected.
One tip. The migrations are great, but they are just the beginning, you can include complex logic as needed to fix your existing data (like so)
Changing data in existing table:
def up
add_column :rsvps, :column_name_id, :integer
update_data
end
def update_data
rsvps = Rsvp.where("other_column is not null")
rsvps.each do |rsvp|
invite = Blah.find(rsvp.example_id)
...
rsvp.save
end
end
Another tip: backup your production database often (should do this anyway), but use it to test all of your migrations before deploying. I run scripts like this all the time for local testing:
mysql -u root -ppassword
drop database mydatabase_dev;
create database mydatabase_dev;
use mydatabase_dev;
source /var/www/bak/mydatabase_backup_2013-10-04-16.28.06.sql
exit
rake db:migrate
I have 4 related migrations in my Rails app:
First 3 migrations create one table each in the self.up and and drops them in their respective self.down methods.
4th migration runs a rake task that loads data to all three (in self.up). I am thinking what to put in self.down of this migration to delete the rows from the 3 tables?
Am I doing it wrong? Probably should have created one migration where I create 3 tables and load the data in self.up and just drop all three tables in the self.down method?
This is wrong!
Migrations should be used only for migrating your Data Model not your Data.
Rails 2.3.4 adds 'seeds'. This is a file in db/seeds.rb contains ruby code to 'bootstrap' your database. This is a great way to create semi-static content for your database like categories, look-up tables or user accounts.
You can then load the seed data with a simple rake task
rake db:seed
There isn't really a right or wrong way to do this. Based on what you've done, the 4th migration should just unload the data in its down method. Each down should only undo the actions of the up.
Loading data in the migration that creates the table is certainly not required. If you have 3 tables that create has_many or belongs_to relationships then it would make sense to put the data in a separate migration so you can create the relationships and then use them in your data load.
All that aside, if you have a rake task to load data, why bother with a migration that runs that rake task? Just make running the rake task part of your install, or just use your rake task to load or unload some demonstration data.
Generally I keep my migrations schema focused, and I'd put any data loading in a rake task I call independently.