I came across a problem where I was working on two branches on a rails project and each project has a migration to add a column. At the time, rake db:migrate:reset cause a problem and I solely relied on my schema.rb to correctly represent the state of my database. At one point, I came across a problem where a column added by branch A got into the schema of branch B. Since migrate:reset was not an option, I resorted to manually editing the schema file to. I committed this change that basically deleted the column from branch A that I did not need in branch B's schema.rb.
Problem came after I have merged branch A into master. When I tried to rebase branch B to master, I still had that commit in B to delete the column (which now has become relevant because it is in master) in the schema file. Git did not see a conflict for this and auto-merged it. At the end of my rebase, I found that my schema is inconsistent with what I have in master.
My fix is to edit the schema file again and manually add the previously deleted column back to the schema file. My question is: Is this considered unconventional? dangerous? hacky?
Right now it involves one column but if this involved multiple column deletions/additions the (dangerous?) solution could lead to more problems and db/schema.rb inconsistency.
It is generally considered a bad practice to edit your schema.rb file.
According to the Rails Guide on Migrations:
Migrations, mighty as they may be, are not the authoritative source for your database schema. That role falls to either db/schema.rb or an SQL file which Active Record generates by examining the database. They are not designed to be edited, they just represent the current state of the database.
schema.rb gets updated every time you run a new migration:
Note that running the db:migrate also invokes the db:schema:dump task, which will update your db/schema.rb file to match the structure of your database.
I'd recommend just spending some time to sort things out and get the schema.rb file back on track and correct up to the latest set of migrations.
Related
Rails 6.1.4
Ruby 2.7
Postgresql 14
A dozen or so migrations, one schema.rb file.
I edited a migration, but did not change the migration id. The result is super weird behavior and I wanted to get input on the best approach.
After I incorrectly edited my migrations, I commited and pushed my feature. A team member pulled the feature and ran the migration on their machine. After they did, no matter the branch, the schema would include the changes I added when I originally modified it. But if they were on a different branch than mine, the actual migration files did not have those changes!!
I tried reverting my commit history to pre-migration editing with no luck. This is how I know it's a db issue, albeit caused from git.
So basically, after every migration, a specific model in the schema gets 4 added columns. No matter what, and it's not in a migration file on rails.
And thats the issue.
My question:
How would you go about solving this without resetting the db?
My current approach/best guess:
Lingering state in the db gets generated in the schema.rb file.
If its not in a migration, the only place schema.rb can get the info is from db.
How do I reset the state on stuff in general?
Either rebuild from scratch, or 'install a copy'. From scratch is not an option :)
If I wanted to install a copy, would it be a wise path to:
Revert changes from any migrations after pulling, delete branch.
Pull down fresh copy of branch
DO NOT MIGRATE - Instead, rails db:schema:load
This should copy over the db structure and effectively overwrite any lingering ghost state.
Rails db:migrate -> this will update migrations,
if you did everything right only the schema version number should change
Now things are synced, continue to db:migrate as normal moving forward.
I did this on my local machine and was successful, but I am curious..
Am I understanding this process correctly? Is there an easier way?
Using Rails and Git, if I run a migration in one branch, the table is now in the database. Now if I check out a branch in which I don't want that migration, the table is still committed to the schema.rb, because it gets dumped from the database.
Is there a way to prevent the schema from getting dumped from the database? For example is it possible to maybe have the schema be generated by running the migrations instead of by dumping the tables from the DB? It's really annoying to me, and to the other people in my company.
When you switch branches, either create a brand new DB and load an empty schema, or rollback migrations to the common anscestor, then migrate up again once you're in the new branch.
There are some tools and scripts that can help with this, but I haven't used any of them personally so I can't vouch for them. Here's a couple that I found after googling:
https://github.com/fhwang/migreazy
http://www.mobalean.com/blog/2013/03/03/keep-database-in-sync-with-current-branch
The schema file changing is only the symptom of the true problem: rails is dynamically building the properties of your models from the schema of the database. Sweeping the schema.rb file under the rug will be insufficient because rails will still be making assumptions about your models based on a database that has the incorrect schema for the branch you're currently working in.
Currently I'm working with a huge rails application and multiple branches with each a new feature for this application.
It happens a lot a feature will require migrations, which shouldn't be a problem until you merge it with master: schema.rb got updated with the information of your dev database!
To clarify:
1. Branch A has migration create_table_x
2. Branch B has migration create_table_y
3. Branch A adds another create_table_z and runs db:migrate
4. You want to merge Branch A with Master and you see table_x, table_y and table_z in the schema.rb of Branch A.
It's not an option to reset+seed the database before every migration in a branch or create a database per branch. Due to the huge size of 2 GB SQL data, it would not be workable.
My question:
Is it really required to keep schema.rb in the repository since it gets rebuilt every migration?
If so, is it possible to build schema off the migrations instead of database dump?
you should keep the schema within your repo. running the migration will fix merge conflicts within your schema.rb file for you.
my simple take on your questions
Is it Required? not really, but good practice.
"It's strongly recommended to check this file into your version
control system." -schema.rb
It is possible? yes but you may want to ask yourself if you really save any time by doing so, either manually or building the schema off your migrations elsewhere and pushing it in.
you ge tthe added benefit of using
rake db:schema:load
and
rake db:schema:dump
http://tbaggery.com/2010/10/24/reduce-your-rails-schema-conflicts.html
Keeping schema.rb in the repo is important, because you want a new developer (or your test environment) to be able to generate a new database just from schema.rb. When you have a lot of migrations, they may not all continue to run, especially if they rely on model classes that don't exist, or have changed, or have different validations in effect than when the migration was first run. When you run your tests, you want to dump and recreate your database from schema.rb, not by re-running all the migrations every time you run the full test suite.
If you're working on two different feature branches simultaneously, and changing the database structure in both, I think schema.rb is the least of your problems. If you rename or remove a column in branch B, branch A is going to break whenever it references the old column. If all they're doing is creating new tables, it's not as bad, but then you want schema.rb to be updated when you merge from master into A, so that A doesn't try to run the migration a second time and fail because the new table already exists.
If that doesn't answer your question, maybe you could explain your workflow a little more?
Fresh Temporary DB as a quick workaround
For example, do the following whenever you need pretty db/schema.rb on particular branch:
git checkout -- db/schema.rb
switch to the different development database, i.e. update config/database.yml
rake db:drop && rake db:setup
rake db:migrate
commit your pretty db/schema.rb
revert changes in config/database.yaml
So, I have an app that I am experimenting with.
The current state is good on my develop branch. I want to install spree, but that comes with many migrations and changes to the db.
So I made a new branch, just for Spree, based on my develop branch.
I installed the gem, ran the migrations, and all that stuff.
However, I messed up a few stuff, so I wanted to revert to my develop branch and deleted the spree branch.
I assumed that my schema.rb would return to normal and so would my DB.
But then I look in my Schema.rb and I see a lot of Spree tables.
Not only that, but I also see other tables in my Schema.rb that I know I had created a migration for and 'dropped' in the spree branch.
So, for instance, I have an orders table in my Schema.rb that I know should have been dropped. A bit confused, I decide to create a new migration to drop the orders table along with the other tables that I dropped before installing spree and I get a PG error saying that the table doesn't exist.
So....it seems that right now, my schema.rb and DB don't sync up.
And to make matters worse, my schema.rb and my db/migrate folder doesn't sync up.
I was going to manually delete the create_table statements in my schema.rb for the tables that shouldn't be there - but then it's all a mess.
How do I dig myself out of this hole and how do I prevent it in the future?
In Summary
DB
Pre-Spree State
Table A
Table B
Table C
Spree State
Table A (removed)
Table B (removed)
Table C
Spree_Table_A
Spree_Table_B
Post-Spree State
Table A (removed)
Table B (removed)
Table C
Spree_Table_A
Spree_Table_B
Basically identical to the Spree State
DB/Migrate Folder
Pre-Spree State
Migration for Table A
Migration for Table B
Migration for Table C
Spree State
Migration to Remove Table A
Migration to Remove Table B
Migration for Table C
Migration for Spree_Table_A
Migration for Spree_Table_B
Post-Spree State
Migration for Table A
Migration for Table B
Migration for Table C
Note that the migrations for the 'Post-Spree State' don't sync up with the state of the DB of the 'Post-Spree State'.
Converted comments to answer since it was getting quite lengthy:
It's safer to have a separate database for any new branch, even if it's just a snapshot of the development database. You will need to restore the migrates (if possible) from your spree branch and run the down on them to get your database back to where you had it.
rake db:migrate VERSION=xxx
Where xxx is equal to the number of the last migrate before spree
I'm not a git expert, but if your branch was deleted through git then this thread may help you restore it. If you only deleted it locally then you should be able to just check it out again.
Git: Recover deleted (remote) branch
When you switch back to your develop branch, git should check out your original schema.rb (if this isn't happening, I think you should recheck the git command you're using). However, it won't automatically bring your DB back into line with where it was when you created spree.
If you no longer have the required migration files to get your DB state back to one that corresponds to the commit you're checking out of git, you should use rake db:reset to get them to match up. This will dump the current state of the DB and rebuild it using schema.rb (then apply the appropriate seed file).
This is a little general I know, but it's been bugging the hell out of me. I've been working on lots of rails projects remotely with Git and every time I do a git pull and see that there is some sort of data change (migration, or schema.rb change) I do a rake db:migrate.
These generally run fine and I can continue working. But if you do a git pull and then git status, your working directory is clean (obviously) then do a rake db:migrate (obviously when there are changes) and another git status and all the sudden your db/schema.rb has changed. I have been just doing a git checkout immediately to reset back to the latest committed version of the schema.rb file, but why should this be necessary?! What is rails doing? Updating a timestamp? I can't seem to figure out what the diff is but maybe I'm just missing something?
The schema enables machines to run rake db:schema:load when being setup for the first time instead of having to run the migrations, which can go out of date if models are renamed or removed, etc. It's supposed to update after a migration, and you always want to latest version checked into source control.
The order of attributes in the dump reflects the order of the attributes in the database, and that can get out of sync if one person has been playing around locally with migrations, running them forwards and backwards manually, and editing things to get them just so. It's possible to create a state where the attribute order in the pusher's schema.rb is different from what everyone else will see when they run the migrations.
If it's easy to recreate your development data, just rebuild the database from schema.rb - then everyone is back in sync (but remember you can't reload the data from a SQL dump that also creates the table - that will recreate the problem. it has to be a data-only dump/load). In the worst case, you can create a migration to delete the column and another to re-add it.
schema.rb reflects your database schema so when you migrate(with changes) it follow that your schema also changes to reflect your db change. I usually add schema.rb to our gitignore, along with database.yml (probably because instead of using schema:load like the one below, i usually do a sql dump when cloning an existing app - but that's just me)