Is there a good way to merge rails db migration files into 1 file per table besides splitting schema.rb manually?
Most of my migration file was created during development and does not represent real data changes. For historical reasons those files will be still accessible on source control system. I feel uncomfortable keeping those unnecessary files.
Well, i can imagine that you want to have a clean start. While being in project development mode for your first version release you don't want all the separate migration files. Although they can't hurt obviously.
Basically what you can do, is this.
FIRST BACKUP your schema and data.
The db/schema.rb contains (or should contain) the latest version of your schema. Otherwise run:
rake db:schema:dump
Now you can clean your migration folder.
Then run:
rake db:drop
rake db:schema:load
The last command runs the db/schema.rb and create a new schema. This should bring you to the last version of your database.
show db task
rake -T db
You can use Squasher gem to merge the migrations all olds into one.
Don't bother. The old migration files are not doing any harm, and they may make maintenance easier. Leave them as they are.
Related
My understanding is that whoever created the migrations should've also updated schema.rb. Since I've pulled the migrations, I should've also pulled the updated schema.rb. However, once in a while, schema.rb updates after I run bundle exec rake db:migrate.
My current workflow is:
git pull --rebase origin master --prune
rails s
Rails tells me to migrate
bundle exec rake db:migrate
Realize schema.rb updated
At this point, I'm pretty sure I'm not supposed to check in the updated schema.rb. I'd manually revert it through git checkout origin/master db/schema.rb.
So what went wrong in this case? Did a co-worker forget to run migrations after creating them? Did I do something wrong?
As far as I know schema can change after running rails db:migrate because of:
A co-worker did not commit the schema.rb so when you fetched and run the migrations you get the diff
A different DB version is running on your local machine. Based on db configuration schema may be changed accordingly.
Running git diff will help you to understand what is going.
schema.rb retains two key sets of data:
a description of all the tables in your app database structure
a list of all the migrations that have been applied.
if a new developer were to join your team, they should be able to run rake db:schema:load and get an up-to-date database structure straight away. That's far more efficient and reliable than expecting them to run through all the migrations manually.
Running rake db:migrate, even if there are no outstanding migrations that need running, will always regenerate db/schema.rb. Most of the time you won't notice because the file will be the same – but you may get differences in whitespace formatting or column order.
The best practice (IMHO) should always be to check in the updated db/schema.rb in the same commit as any migrations you've added.
When fetching or pulling branches to your local machine, running rake db:migrate will apply whatever new migrations need to be run based on the records in your local database's schema_migrations table. After that, your new db/schema.rb should be the same as the one you pulled down – but if it isn't, git diff will show you what the difference is.
You can then make a judgement call as to what the best course of action is. If the only difference is cosmetic, I personally tend to revert the unstaged changes and leave the committed version untouched until the next migration.
All the above also applies if you have switched to a SQL-based structure file (db/structure.sql) by specifying config.active_record.schema_format = :sql in config/application.rb.
Every time I commit code that has a migration, for some reason, I get a bunch of schema changes that I didn't write, that came from previous PRs.
For example, i'll write a migration to add a column on User...but after running the migration, the schema file will include 10 changes from previous old code that isn't in the current branch at all.
How do I fix this?
The schema file reflect to the database schema. I think you had changed the schema at previous old code but didn't recover(rollback) it, deleted it and start coding for new migration.
The thing you shloud do is eliminating diff between code and datebase.
Solution:
Checkout to your old branch and rollback the schema change by runningrake db:migrate:down VERSION=20161106xxxxxx.
or
In current branch, run rake db:rollback STEP=n rollback schema change done by current branch
Then checkout co old branch execute rake db:rollback STEP=m to rollback schema change by old branch.
Checkout back to current branch, and run rake db:migrate, and you will not see the extra changes in schema file.
reference:
http://edgeguides.rubyonrails.org/active_record_migrations.html#rolling-back
http://edgeguides.rubyonrails.org/active_record_migrations.html#running-specific-migrations
There are two possibilities:
You haven't deleted the code for the previous migrations that you are trying to neglect from the schema.rb file.
You're very new to rails and you tried deleting fields from the schema.rb file manually, thinking it would synchronize with your database.
Either way: delete all the migration files you don't want if you haven't already, then simply rollback your database to the original empty version using the command:
rake db:rollback VERSION=0
Then: Now that you have the right migration files, migrate to your databse using the command:
rake db:migrate
This should give you an accurate schema.rb file
When working on a project, I keep track of all the changes I make to a database in a notepad file. Then, later, I manually write all the changes in rails' db migration file.
But it should be possible to compare the schema of a backup of my database, with the new version of my database, and automatically detect the differences. And generate the rails db migration file automatically.
Is there a tool that can compare two database schema's and automatically generate rails' db migration files?
As far as I'm aware, there's no tool that will do it automatically, however, you can get most of the way there just using rake db:schema:dump with source control.
Create a new Rails project and do the following:
Update database.yml to connect to your first database.
Use rake db:schema:dump to populate schema.rb and commit schema.rb to git.
Update database.yml to connect to your second database and again run rake db:schema:dump
Use git diff on schema.rb to compare the changes. This can easily be mapped to a migration.
The benefit of using source control is that you can then test the migration by comparing schema.rb after the migration runs to the schema dump of the second database.
Some projects that I try and learn from don't have a schema.rb file, why is that?
Can this be generated somehow?
sure, do the rake db:migrate (if you didn't make it already) and rake db:schema:dump.
I personally don't commit my schema.rb, that's because when you have more than one developer working on the same project, there is always merge problem, since this file change always.
check the .gitignore file and see if schema.rb is there, if you really want to push it to your repository, comment or remove this entry from your .gitignore
As VP said, rake db:migrate will dump schema.rb after migrating. But if you just want to dump the schema:
rake db:schema:dump
Ignoring db/schema.rb is a contentious issue among developers. Personally, I don't have a problem ignoring it as it's caused issues for me in the past and I have no compelling reason to include it in version control.
I don't want to add schema.rb to .gitignore, because I want to be able to load a new database schema from that file. However, keeping it checked in is causing all sorts of spurious conflicts that are easily resolved by a fresh db:migrate:reset.
Basically I want a way to:
Keep schema.rb in the repository for deploy-time database setup
Keep schema.rb in '.gitignore' for general development
There would be one or two people responsible for updating schema.rb and knowing that it was correct.
Is there a way I can have my cake and eat it, too?
I'm afraid the magic solution you're looking for does not exist. This file is normally managed in version control, then for any conflicts on the version line just choose the later of the two dates. As long as you're also running all of the associated migrations nothing should get out of sync this way. If two developers have caused modifications to a similar area of schema.rb and you get conflicts in addition to the version then you are faced with a normal merge conflict resolution, but in my opinion these are normally easy to understand and resolve. I hope this helps some!
One other thing you can do is use:
git update-index --assume-unchanged /path/schema.rb
This will keep the file in the repository but won't track changes. you can switch the tracking anytime by using:
git update-index --no-assume-unchanged /path/schema.rb
What has worked really well for me is to delete and .gitignore schema.rb and then have it regenerated for each developer when they rake db:migrate.
You can still achieve what you wanted without migrating from 0 and risking broken migrations from years ago by simply doing a "roll-up" of the migrations periodically. You can do this by:
Run all outstanding migrations with rake db:migrate
Taking the contents of your schema.rb in the ActiveRecord::Schema.define block
Paste it into your initial_schema migration inside def up (overwriting what's already there)
Delete all other migrations
Now your initial_schema migration is your starting point for new systems and you don't have to worry about conflicts in schema.rb that may not be resolved correctly. It's not magical, but it works.
Would it be sufficient to do a rake db:dump in a pre-commit git hook?
The following won't necessarily fix (1) or (2), but it might take care of the merging issue, and then maybe (1) and (2) go away.
Instead of using .gitignore, use separate branches: Develop which omits schema.rb and Test and Deploy which include schema.rb. Only make code changes in the Develop branches and never merge from Test into Develop. Keep schema.rb in a separate branch:
Developer A
Develop --------
Local Schema \ Your Repo
Test ---------> Dev A
---------> Dev B
Developer B / Master
Develop -------- Schema
Local Schema Test
Test Deploy
In Git, branches are pointers to collections of file contents, so they can include or exclude particular files as well as track file versions. This makes them flexible tools for building your particular workflow.
You could define a merge strategy.
I've found this solution, but dont remember the source
[merge "railsschema"]
name = newer Rails schema version
driver = "ruby -e '\n\
system %(git), %(merge-file), %(--marker-size=%L), %(%A), %(%O), %(%B)\n\
b = File.read(%(%A))\n\
b.sub!(/^<+ .*\\nActiveRecord::Schema\\.define.:version => (\\d+). do\\n=+\\nActiveRecord::Schema\\.define.:version => (\\d+). do\\n>+ .*/) do\n\
%(ActiveRecord::Schema.define(:version => #{[$1, $2].max}) do)\n\
end\n\
File.open(%(%A), %(w)) {|f| f.write(b)}\n\
exit 1 if b.include?(%(<)*%L)'"
put this "somewhere" and
git-config --global core.attributesfile "somewhere"
I built a gem to solve this problem.
It sorts columns, index names and foreign keys, removes excess whitespace and runs Rubocop for some formatting to unify the output of your schema.rb file.
https://github.com/jakeonrails/fix-db-schema-conflicts
After you add it to your Gemfile you just run rake db:migrate or rake db:schema:dump like normal.
Commit schema.rb file.
Run git pull (or continue with what you're doing)
Every time you migrate the database, the schema.rb file updates and appears in git status. When working on something and occasionally doing git pull, this can be annoying because you have to commit schema.rb file before pulling to resolve conflict. This means that every time you migrate the database, you need to commit schema.rb file.
schema.rb should be tracked Git, of course.
I've just released this gem that can solve an issue with "conflicts" between branches for good.
The idea of that gem is simple. It keeps all migrated migrations inside tmp folder so that Git ignores them. It's just only your local story. These files are needed to roll back the "unknown" migrations being in another branch. Now, whenever you have an inconsistent DB schema due to running migrations in some other branch just run rails db:migrate inside the current branch and it will fix the issue automatically. The gem does all this magic automatically for you.