So, I have an app that I am experimenting with.
The current state is good on my develop branch. I want to install spree, but that comes with many migrations and changes to the db.
So I made a new branch, just for Spree, based on my develop branch.
I installed the gem, ran the migrations, and all that stuff.
However, I messed up a few stuff, so I wanted to revert to my develop branch and deleted the spree branch.
I assumed that my schema.rb would return to normal and so would my DB.
But then I look in my Schema.rb and I see a lot of Spree tables.
Not only that, but I also see other tables in my Schema.rb that I know I had created a migration for and 'dropped' in the spree branch.
So, for instance, I have an orders table in my Schema.rb that I know should have been dropped. A bit confused, I decide to create a new migration to drop the orders table along with the other tables that I dropped before installing spree and I get a PG error saying that the table doesn't exist.
So....it seems that right now, my schema.rb and DB don't sync up.
And to make matters worse, my schema.rb and my db/migrate folder doesn't sync up.
I was going to manually delete the create_table statements in my schema.rb for the tables that shouldn't be there - but then it's all a mess.
How do I dig myself out of this hole and how do I prevent it in the future?
In Summary
DB
Pre-Spree State
Table A
Table B
Table C
Spree State
Table A (removed)
Table B (removed)
Table C
Spree_Table_A
Spree_Table_B
Post-Spree State
Table A (removed)
Table B (removed)
Table C
Spree_Table_A
Spree_Table_B
Basically identical to the Spree State
DB/Migrate Folder
Pre-Spree State
Migration for Table A
Migration for Table B
Migration for Table C
Spree State
Migration to Remove Table A
Migration to Remove Table B
Migration for Table C
Migration for Spree_Table_A
Migration for Spree_Table_B
Post-Spree State
Migration for Table A
Migration for Table B
Migration for Table C
Note that the migrations for the 'Post-Spree State' don't sync up with the state of the DB of the 'Post-Spree State'.
Converted comments to answer since it was getting quite lengthy:
It's safer to have a separate database for any new branch, even if it's just a snapshot of the development database. You will need to restore the migrates (if possible) from your spree branch and run the down on them to get your database back to where you had it.
rake db:migrate VERSION=xxx
Where xxx is equal to the number of the last migrate before spree
I'm not a git expert, but if your branch was deleted through git then this thread may help you restore it. If you only deleted it locally then you should be able to just check it out again.
Git: Recover deleted (remote) branch
When you switch back to your develop branch, git should check out your original schema.rb (if this isn't happening, I think you should recheck the git command you're using). However, it won't automatically bring your DB back into line with where it was when you created spree.
If you no longer have the required migration files to get your DB state back to one that corresponds to the commit you're checking out of git, you should use rake db:reset to get them to match up. This will dump the current state of the DB and rebuild it using schema.rb (then apply the appropriate seed file).
Related
Rails 6.1.4
Ruby 2.7
Postgresql 14
A dozen or so migrations, one schema.rb file.
I edited a migration, but did not change the migration id. The result is super weird behavior and I wanted to get input on the best approach.
After I incorrectly edited my migrations, I commited and pushed my feature. A team member pulled the feature and ran the migration on their machine. After they did, no matter the branch, the schema would include the changes I added when I originally modified it. But if they were on a different branch than mine, the actual migration files did not have those changes!!
I tried reverting my commit history to pre-migration editing with no luck. This is how I know it's a db issue, albeit caused from git.
So basically, after every migration, a specific model in the schema gets 4 added columns. No matter what, and it's not in a migration file on rails.
And thats the issue.
My question:
How would you go about solving this without resetting the db?
My current approach/best guess:
Lingering state in the db gets generated in the schema.rb file.
If its not in a migration, the only place schema.rb can get the info is from db.
How do I reset the state on stuff in general?
Either rebuild from scratch, or 'install a copy'. From scratch is not an option :)
If I wanted to install a copy, would it be a wise path to:
Revert changes from any migrations after pulling, delete branch.
Pull down fresh copy of branch
DO NOT MIGRATE - Instead, rails db:schema:load
This should copy over the db structure and effectively overwrite any lingering ghost state.
Rails db:migrate -> this will update migrations,
if you did everything right only the schema version number should change
Now things are synced, continue to db:migrate as normal moving forward.
I did this on my local machine and was successful, but I am curious..
Am I understanding this process correctly? Is there an easier way?
I am trying to deploy a new release of a project to my staging server, but the schema_migrations table in the database is inexplicably empty.
It is now trying to run all migrations while deploying, causing issues since the other tables exist and are intact.
Instead of dropping/recreating the database and losing all my data(though inconvenient, a valid option), is it possible to generate the schema_migrations table without dropping?
Two points:
There's nothing wrong with deleting migrations that aren't needed anymore, you're not supposed to keep them around forever.
You can manually update the schema_migrations table if you have to.
As far as (1) goes, I delete migrations (provided they've been applied everywhere) that are old than a month or so. Keeping around years of clutter in db/migrate is pointless. Migrating down and then back up tends to be more of a mess than it is worth for all but the most recent migrations; if you need to test something old, dump your current development database and start with a fresh one based on an older schema.rb. Besides, if you really need an old migration then you can always dig it out of revision control or simply recreate it based on diffs between versions of db/schema.rb (or db/structure.sql).
Of course, if you don't keep db/schema.rb in revision control then you're doing it wrong and you a looming disaster on your hands.
For (2), the schema_migrations table is very simple:
create table schema_migrations (
version varchar not null primary key
)
So you can connect to PostgreSQL using psql, create the table by hand, and then do a bunch of insert into schema_migrations (version) values (...) to cover the historic migrations that should have been run in your staging environment, and then db:migrate as normal to get things up to date. Then you can spend as much time as you want doing a post mortem to figure out what happened to your staging database's schema_migrations table.
If you go this route, you might want to have a look at the ar_internal_metadata table if your version of Rails uses it.
You will, of course, backup your staging database before doing any of this. And you will want to examine your production database to make sure it isn't suffering the same problems.
Sometimes you have to get your hands dirty to make things go.
My problem is that I at some point was doing some spiking and I migrated my database with a migration that I had created in the spike branch. Then I switched back to my master branch(I never committed the migration). Now that migration is perpetually in my database and this is making it impossible to migrate my database.
when I run
ActiveRecord::Migrator.get_all_versions
in the console my deleted migration file timestamp is in this array. I would like to remove this version from the migrator. Also I cannot drop the database.
If the database can be blown away, then the quickest way would be to drop the database, create it afresh, and reapply the migrations, with the following steps:
rake db:drop
rake db:create db:migrate
If the database has useful information & can't be blown away, then it is tricky. Only possible option would be to go to the database level, and undo the changes made by the migration.
Figure out which tables/columns were added/deleted in the migration, and then run sql scripts to reverse those changes.
In addition, delete the specific row corresponding to this migration from schema_migrations table, with the following query:
delete from schema_migrations where version = '201503......'
Currently I'm working with a huge rails application and multiple branches with each a new feature for this application.
It happens a lot a feature will require migrations, which shouldn't be a problem until you merge it with master: schema.rb got updated with the information of your dev database!
To clarify:
1. Branch A has migration create_table_x
2. Branch B has migration create_table_y
3. Branch A adds another create_table_z and runs db:migrate
4. You want to merge Branch A with Master and you see table_x, table_y and table_z in the schema.rb of Branch A.
It's not an option to reset+seed the database before every migration in a branch or create a database per branch. Due to the huge size of 2 GB SQL data, it would not be workable.
My question:
Is it really required to keep schema.rb in the repository since it gets rebuilt every migration?
If so, is it possible to build schema off the migrations instead of database dump?
you should keep the schema within your repo. running the migration will fix merge conflicts within your schema.rb file for you.
my simple take on your questions
Is it Required? not really, but good practice.
"It's strongly recommended to check this file into your version
control system." -schema.rb
It is possible? yes but you may want to ask yourself if you really save any time by doing so, either manually or building the schema off your migrations elsewhere and pushing it in.
you ge tthe added benefit of using
rake db:schema:load
and
rake db:schema:dump
http://tbaggery.com/2010/10/24/reduce-your-rails-schema-conflicts.html
Keeping schema.rb in the repo is important, because you want a new developer (or your test environment) to be able to generate a new database just from schema.rb. When you have a lot of migrations, they may not all continue to run, especially if they rely on model classes that don't exist, or have changed, or have different validations in effect than when the migration was first run. When you run your tests, you want to dump and recreate your database from schema.rb, not by re-running all the migrations every time you run the full test suite.
If you're working on two different feature branches simultaneously, and changing the database structure in both, I think schema.rb is the least of your problems. If you rename or remove a column in branch B, branch A is going to break whenever it references the old column. If all they're doing is creating new tables, it's not as bad, but then you want schema.rb to be updated when you merge from master into A, so that A doesn't try to run the migration a second time and fail because the new table already exists.
If that doesn't answer your question, maybe you could explain your workflow a little more?
Fresh Temporary DB as a quick workaround
For example, do the following whenever you need pretty db/schema.rb on particular branch:
git checkout -- db/schema.rb
switch to the different development database, i.e. update config/database.yml
rake db:drop && rake db:setup
rake db:migrate
commit your pretty db/schema.rb
revert changes in config/database.yaml
I came across a problem where I was working on two branches on a rails project and each project has a migration to add a column. At the time, rake db:migrate:reset cause a problem and I solely relied on my schema.rb to correctly represent the state of my database. At one point, I came across a problem where a column added by branch A got into the schema of branch B. Since migrate:reset was not an option, I resorted to manually editing the schema file to. I committed this change that basically deleted the column from branch A that I did not need in branch B's schema.rb.
Problem came after I have merged branch A into master. When I tried to rebase branch B to master, I still had that commit in B to delete the column (which now has become relevant because it is in master) in the schema file. Git did not see a conflict for this and auto-merged it. At the end of my rebase, I found that my schema is inconsistent with what I have in master.
My fix is to edit the schema file again and manually add the previously deleted column back to the schema file. My question is: Is this considered unconventional? dangerous? hacky?
Right now it involves one column but if this involved multiple column deletions/additions the (dangerous?) solution could lead to more problems and db/schema.rb inconsistency.
It is generally considered a bad practice to edit your schema.rb file.
According to the Rails Guide on Migrations:
Migrations, mighty as they may be, are not the authoritative source for your database schema. That role falls to either db/schema.rb or an SQL file which Active Record generates by examining the database. They are not designed to be edited, they just represent the current state of the database.
schema.rb gets updated every time you run a new migration:
Note that running the db:migrate also invokes the db:schema:dump task, which will update your db/schema.rb file to match the structure of your database.
I'd recommend just spending some time to sort things out and get the schema.rb file back on track and correct up to the latest set of migrations.