How to load schema and data for my production rails app into a different machine.
I would like to converge all migrations done so far into a single migration file and point the new instance to a snapshot of the prod db. How to solve
Just copy and paste the schema and do rake db:schema:load. You can also paste the schema.rb contents into a migration, but make sure to delete all other migrations. However keep in mind that there is really no reason to do so, just leave the old migrations as they are.
To import test data use this gem: https://github.com/ludicast/yaml_db .
Related
I have created an application with Ruby and Rails. The thing is that when I was develpoing it, I had some problems with the migrations, because I created them but with a wrong syntax. What happened is that I deleted some of the files because sold migrations that didn´t work had the same name than the new ones, but in the middle of that I accidentally deleted some of the migrations (obviously after running rails db:migrate) that the project uses actually. So for instance, i have the Service table, which is related to the Reservation table because Service has reservation_id, but i don´t have the migration file that says AddReservationIdToService.
So now I want to use Heroku for production. the thing is that O have to change to postgresql because Heroku doesn't support sqlite. So i have to run the de:migrate again to create the tables and relationships in the new DB, but I need the files that I explained that I deleted. The question is:
Can I create the migrations manually, so when i run db:migrate for postgres the full structure of the database is created without lacking relations?
You don't really need the migrations to recreate the existing DB -- in fact it's not a good idea to try for a couple of reasons (including the missing migration file problem you encountered). You can simply run:
bin/rails db:schema:load
to populate a new database from the existing schema. If for some reason you haven't got a db/schema.rb checked under version control you can run:
bin/rails db:schema:dump
against the sqlite version to re-create a fresh schema file from the database.
You can also keep your migrations list tidy by occasionally zapping really old migrations, since all the cumulative changes are captured in the schema file.
Yes, you might create another couple of migration files.
Certify you have now the tables you wish locally with your sqlite. Draw these table in a piece of paper (or where it be the best fr you), then check this official API documentation of Rails.
Delete all migrations made before and create another according to the tables you drew.
The workflow is gonna be like:
1) "I need to create a table called Reservation, where is it shown on the documentation?"
2) "I need a table called Service, where is it shown on the documentation?
3) "I need to add a column with a foreign key to service named reservaton_id, how does this documentation says it?
For all this steps above, create the correspondent migration file as you normally have done.
The main difference here is not to run the migration locally. Instead, push your new version app to your heroku remote branch and there you run the migration, like:
heroku run rails db:migrate
Remember to not run this same migration locally because you already have these tables locally.
The last two advise is:
1) If your migration doesn't go as you expect, don't delete the migration file. Instead, run rails db:rollback and try again.
2) Keep tracking your migration files on the same branch of your version control.
I have been developing a project in Ruby on Rails. During the development I have generated tons of migration files for my project. Sometimes I have added and deleted columns from different tables.
Is there a way that I could consolidate all the migrations from multiple files into a single file?
TL;DR
What you need isn't a consolidated set of migrations; it's a single schema file and an optional seeds.rb file. Rails generally maintains the schema automagically when you run migrations, so you already have most of what you should need with the possible exception of seed data as described below.
Use the Schema, Not Migrations
In general, you shouldn't be maintaining a large pool of migrations. Instead, you should periodically clear out your migrations, and use schema.rb or schema.sql to (re)create a database. The Rails guides specifically state:
There is no need (and it is error prone) to deploy a new instance of an app by replaying the entire migration history. It is much simpler and faster to just load into the database a description of the current schema...Because schema dumps are the authoritative source for your database schema, it is strongly recommended that you check them into source control.
You should therefore be using bin/rails db:schema:load rather than running migrations, or run the associated Rake task on older versions of Rails.
Data Migrations
While you can use migrations to fix or munge data related to a recent schema change, data migrations (if used at all) should be temporary artifacts. Data migrations are almost never idempotent, so you shouldn't be maintaining data migrations long-term. The guide says:
Some people use migrations to add data to the database...However, Rails has a 'seeds' feature that should be used for seeding a database with initial data. It's a really simple feature: just fill up db/seeds.rb with some Ruby code, and run rake db:seed...This is generally a much cleaner way to set up the database of a blank application.
Database seed data should be loaded with bin/rails db:seed (or the associated Rake task) rather than maintaining the data in migrations.
There is a gem that purports to do exactly what you describe in the question - check out Squasher.
From the README:
"Squasher compresses old migrations in a Rails application. ... Squasher removes all the migrations and creates a single migration with the final database state of the specified date (the new migration will look like a schema)."
You'll have to do the merge manually.
But if you want only a single file, there is db/schema.rb. It contains a snapshot of current database schema. You can load it directly in database if you want.
In section Rails Database Migrations of Ruby on Rails Guides, there is one line saying that
The db:reset task will drop the database, recreate it and load the current
schema into it. This is not the same as running all the migrations.
Can anyone tell me where exactly they are different and why it is more error prone to replay the migration history?
I'm fairly new to Ruby on Rails. Thanks in advance.
The schema file contains the current structure of your database. When you load it, you are guaranteed to have the exact schema in your db that is in the file. Migrations were designed to make incremental changes in the database. You may add a table, then some columns, and then remove the table in three separate migrations. There's no need to go through all this when the schema already knows that the table no longer exists.
On why they are error prone, I'm not totally sure. The one thing I can think of is that migrations can be used to make changes to data and not just the structure.
Running rake db:reset will rebuild the structure of your database from schema.db, which essentially works as a cached version of your migrated database structure. Running all your migrations, on the other hand, applies the migrations one by one, which may include arbitrary code to accomodate for changes to the database (e.g. prepopulate an added counter cache column).
It can be more error prone to replay the migration history, since it is the product of changes to both the structure and data of the database. If the developers haven't been careful, it might not apply cleanly to a fresh environment (e.g. the migration assumes an old version of a model). On the other hand, schema.db can get out of sync if you edit a migration once you've migrated (a useful trick to avoid migration explosion during development). In that case, you need to run rake db:migrate:reset.
At some point in my rails development I started making database changes (e.g. dropping or altering columns/tables) without using rails migrations. So now I get errors when I try to deploy my rails app from scratch.
blaine#blaine-laptop ~/tmp/rbjacolyte $ rake db:migrate
(in /home/blaine/tmp/rbjacolyte)
== AddHashToTrack: migrating =================================================
-- add_column(:tracks, :hash, :string)
rake aborted!
An error has occurred, all later migrations canceled:
Mysql::Error: Table 'jacolyte_dev_tmp.tracks' doesn't exist: ALTER TABLE `tracks` ADD `hash` varchar(255)
(See full trace by running task with --trace)
How can I sync my production and development environments with migrations after I've mucked it up by using raw SQL? I want to deploy my rails application without database errors, and I don't want to start from scratch.
The data in the production and development environments match, but the migrations fail. I want a way to 'start from scratch.'
Could I simply delete all of the migrations that I have, and then just start using migrations from now on?
The shortcut way: manually add an entry to schema_migrations for a timestamp that represents a baseline. You can add migrations after that and as long as they don't make any bad assumptions about the db schema they should be able to run just fine. You won't be able to migrate backwards, but that's not a huge problem.
The bigger problem is that you won't be able to make a DB from scratch, which gets to be a pain longer term.
The fix for that is to delete all your existing migrations and create a new one that creates the existing schema. Manually delete everything from the schema_migrations table and put in an entry for this one new migration. After that, you can create new migrations that build on this new baseline and they should apply just fine. You should be able to bootstrap new databases in the normal fashion.
As long as your direct SQL is contained in Rails migrations, there's no problem with using it. Just make sure you implement both the #up and #down methods and you should be good. We've actually taken to using raw SQL as a best practice to avoid problems when models are changed later on. Something like
Foo.create(:name => 'bar')
seems innocuous, until the User model is modified to have
validates_presence_of :baz
At which point the new migration will run against an existing database, but that earlier migration that created the table and added the dummy entry will fail because User fails validation. Just using
execute("insert into foos (name) values ('bar')")
will work fine as long as the later migrations properly populate any new columns they add.
Maybe you could just get rid of all your current migrations, and use rake db:schema:dump to create a new schema.rb file, and manually edit your production database to reflect the changes you've made so far?
I like Veeti's suggestion, with a modification: rake db:schema:dump, then move that file to your development machine. Flatten your Rails migrations so far (see this SO thread on that), get rid of most of your migrations, and re-work your migrations to work, given your new schema.
Get this working on your dev machine, commit and deploy.
If the existing production data is compatible with the development database schema, then I would:
Dump the production data to a file using a program such as mysqldump
Drop the production database
Recreate the production database
Run the migrations against the production database, specifying VERSION=0
Import the production data from the file created at step one
If the schemas aren't compatible then you might be able to follow this process but you'll have to edit the SQL in the file created in the first step to take account of the schema differences.
So this is my first real Ruby on Rails project. I've learned my lesson -- I didn't make all changes using migrations so things are a bit messed up.
What's the best way to start over with new migration files and rebuild the schema, etc? My project is too far along to rebuild the entire project, but not far enough along to where I care about losing the migrations I have thus far. I also don't mind losing the data in the database. I was trying to rollback to the beginning but some of it is failing.
I know this is a bad state to be in, but lesson learned.
EDIT:
I just deleted all the migrations files and rebuilt the schema file with db:schema:dump.
I assume this puts me in a clean state with my existing database, just lost migrations.
if you want to migrate some steps back you can
rake db:rollback STEP=2
That command will migrate your database 2 migrations back.
If you need more help with rake commands, jus type
rake -T
That command will list all the tasks you have in you application.
If you are not concerned about losing data then do
rake db:purge
It should just drop your database
Your schema.rb file should contain the actual schema from your database. You could use it as a starting point to create you migrations. You could create a new migration for each table with the :force => true parameter to overwrite the old table. Afterwards you could just delete the old migrations (you would probably also need to delete their entries from schema_migrations table).
Another options would be just updating the old migrations to match your current schema.