my issue appears to pertain to mainly docker.
I got frustrated with a table that I had so I deleted it plus all associated migrations from my rails migrations.
I did make a zip file of repository before I did this so that I wouldn't lose all my work, I'm not sure if this can be affecting anything but I thought I would put it out there just in case because all of those files do technically exist somewhere else.
When I do rails db:migrate:status those migrations no longer appear on the list, but they still exist in schema.rb
What I have tried:
I deleted my branch that the original migration is in and started in a new one.
I tried just deleting it from schema.rb because it didn't exist anymore, but when I tried to create a table with the same name (because I need the table to be named that) it says the migration failed because that table already exists.
So then I restarted my docker container with docker restart <container name*> and tried running the migration again... same error.
Next I rebuilt my database, same error.
After that failed I deleted the docker container and rebuilt it from scratch.. same error.
I then deleted the repository from my computer, deleted the docker image, shut down and restarted my computer. Re-cloned the repository, rebuilt the container and still the same error.
It just keeps reappearing in the schema.
Sure I could create a new table with a different name, but I don't want the schema for a table that doesn't exist.
What should my next move be?
If I understood correctly, you had run migrations to update your schema.rb file, then you removed (dropped) a database table, maybe using your database software directly, and then you deleted the related migration files.
I don't think this has anything to do with Docker but everything with how Rails works with databases.
First, Rails will not know if you remove tables or make any other changes to your database directly using your database software and SQL (DDL), so the schema file is not updated based on it.
Second, Rails will not know about removing any migration files from the disk, and the schema file is not updated based on your changes.
You will always have to use migrations to keep your database structure up-to-date.
Your options
All of the below options only operate on your local database.
Option 1: Resetting and recreating your local database using Rails
Assumptions: you can delete your local database
You could try resetting your database with
bin/rails db:reset
that basically runs bin/rails db:drop db:setup that is described as:
The bin/rails db:setup command will create the database, load the schema, and initialize it with the seed data.
This should recreate the database structure using your schema.rb and it should bring you back the frustrating table.
Option 2: Resetting and recreating your local database by restoring it from a backup
Assumptions: you have a backup, you can delete your local database
You could try restoring or importing the latest backup using your database software. Most likely, you would need to drop the database, first, but it depends on the tools used and their options available.
This should result in the structure you have in your schema.rb and it should give you back the frustrating table with any data you had in the backup snapshot.
Option 3: Recreating only the missing table in your local database by importing it from a backup
Assumptions: you have a backup, you do not want to delete your local database
You could try restoring or importing only the missing table using your database software.
This should result in the structure you have in your schema.rb and it should give you back the frustrating table. It will also import any data for that table you had in your backup snapshot.
Depending on your table definition(s), you may need to update something in the data, though, especially if other tables reference this one or this one has references to others. The data you imported might be wrong so you may want to set some references to NULL, for example, or fix them, whichever suits you better.
Option 4: Recreating only the missing table in your local database manually
Assumptions: you do not have a backup, you do not want to delete your local database
You could try recreating just the frustrating table manually without any data.
You could do this based on your schema.rb by translating it to SQL DDL yourself. A faster approach would be to ask one of your team members or colleagues, if any, who has an up-to-date version of the database structure to share you the SQL DDL for that table so that you could run it locally using your database software, for example.
This would result in the structure you have in your schema.rb and it should give you back the frustrating table.
Next steps: Finally, no matter which option you decided to implement to restore your database structure, you should create a new migration in which you call drop_table to remove (drop) the frustrating table correctly. That will remove the table definition from the schema.rb as well so that you can later create a new migration to recreate the table with the same name.
Related
I am new to Rails and I am facing the following problem. There were different tables in the existing project schema such as questionnaire, test, answers, questions, etc. But they did not have the columns of the creation date and the update date, and there were no models and views yet. So I decided to delete them and recreate them. Unfortunately I did it directly through the pgAdmin. Then I recreated each model with a scaffold generator. Everything was going fine until I got to the Questions model. For this model, I forgot to delete the table from the database, which I decided to do. But when I returned to the terminal and tried again to migrate the creation of the question table, I came across the same error message that the table already exists. Moreover, in the schema, I found that all tables manually removed from the database are still present.
What I tried:
I wrote migrations to drop tables. Migrations go through, tables disappear from the schema. Then I tried to write again table creation migrations, they are also started and tables are added to the schema, but they are not created in the database itself. But the application is still accessing the database, retrieving data, inserting it (into other tables that I have not touched on).
How can I return everything to the original? Migrations do not work in the project and I do not understand how to fix it. And I would not really like to dump the database, since it contains a lot of data.
Also in the project there are old migrations of test creation, test results, etc., which I deleted manually through the PG Admin. Should I delete them?
db/migrate
I found a bug. The problem was that the project turns out to be using docker (did not fully understand what it is). But as I understand it, for any update of the project, it must be reloaded through 4 commands in the terminal. Therefore, migrations did not go through the terminal. they had to be done by the docker himself. I expected to see the error logs right in the terminal, and they were issued by docker. I should have connected to the docker logs and read errors from there. There, he swore in plain text about the impossibility of migrating due to the lack of tables. I rolled back migrations to the state before the problem, removed migrations conflicting with the current state of the database and created new ones, according to the rails conventions. In the end, everything worked.
I have created an application with Ruby and Rails. The thing is that when I was develpoing it, I had some problems with the migrations, because I created them but with a wrong syntax. What happened is that I deleted some of the files because sold migrations that didn´t work had the same name than the new ones, but in the middle of that I accidentally deleted some of the migrations (obviously after running rails db:migrate) that the project uses actually. So for instance, i have the Service table, which is related to the Reservation table because Service has reservation_id, but i don´t have the migration file that says AddReservationIdToService.
So now I want to use Heroku for production. the thing is that O have to change to postgresql because Heroku doesn't support sqlite. So i have to run the de:migrate again to create the tables and relationships in the new DB, but I need the files that I explained that I deleted. The question is:
Can I create the migrations manually, so when i run db:migrate for postgres the full structure of the database is created without lacking relations?
You don't really need the migrations to recreate the existing DB -- in fact it's not a good idea to try for a couple of reasons (including the missing migration file problem you encountered). You can simply run:
bin/rails db:schema:load
to populate a new database from the existing schema. If for some reason you haven't got a db/schema.rb checked under version control you can run:
bin/rails db:schema:dump
against the sqlite version to re-create a fresh schema file from the database.
You can also keep your migrations list tidy by occasionally zapping really old migrations, since all the cumulative changes are captured in the schema file.
Yes, you might create another couple of migration files.
Certify you have now the tables you wish locally with your sqlite. Draw these table in a piece of paper (or where it be the best fr you), then check this official API documentation of Rails.
Delete all migrations made before and create another according to the tables you drew.
The workflow is gonna be like:
1) "I need to create a table called Reservation, where is it shown on the documentation?"
2) "I need a table called Service, where is it shown on the documentation?
3) "I need to add a column with a foreign key to service named reservaton_id, how does this documentation says it?
For all this steps above, create the correspondent migration file as you normally have done.
The main difference here is not to run the migration locally. Instead, push your new version app to your heroku remote branch and there you run the migration, like:
heroku run rails db:migrate
Remember to not run this same migration locally because you already have these tables locally.
The last two advise is:
1) If your migration doesn't go as you expect, don't delete the migration file. Instead, run rails db:rollback and try again.
2) Keep tracking your migration files on the same branch of your version control.
I've started working with rails recently and I have noticed that during development you can end up with a lot of migration files as your ideas change and your database with it. Is there a convention in rails to avoid accumulating a lot of migrations and deleting redundant ones (such as one migration adding a table and then the next deleting it)? Or, is it preferred to keep all your migrations and never delete them? Thanks.
If you are working on a team it is preferred and you always should keep all the migrations.
Suppose you push a migration to Git and later found out that it needs some change. So you revert and delete the migration and create a new one and push the new changes. It works well in your machine. But if somehow your previous faulty migrations are run by any of your team mate and he pull the new code he will have to run the new migrations which will break the applications in your teammates machine.
But if you are working solo and want to get a bit tidy then you can carefully delete migrations after reverting them (Not preferred).
The primary job of migration files is to alter the database schema. Deleting redundant ones (such as one migration adding a table and then the next deleting it) will not cause any problems and it free up some disk space and slug size.
QUESTION:
Is there any way I can avoid running current migrations on a remote database already configured correctly, while ALSO being allowed to apply future migrations to it?
Context:
I have an existing rails app with plenty of migrations, that, up until now, has been using a local postgres database. A remote postgres database which SHOULD mostly match up to the structure of my local database exists.
When I try to connect to it, I get a "pending migrations" error. Attempting to run my migrations gets errors about tables already existing.
I want to skip all current migrations, but UNLIKE a lot of similar questions I'm seeing, I want to make sure my future migrations will work on this remote database.
Edit:
I followed this answer:
Rails 4 how to ignore pending migrations
And inserted my own current scheme version number into the remove database's schema_migrations table. But the pending migration remains.
I can confirm trying to run the migration gets me an error of a table already existing. This table is the FIRST migration past the remote databases previously most recent version in the schema table. It appears to be ignoring my inserted current most recent version.
You have to add all migration version numbers that you want to ignore to schema_migrations
Other technique that may be also applicable - migration squashing with something like squasher or manually. Point is in combining all old migrations that are not to be rolled back into one.
In section Rails Database Migrations of Ruby on Rails Guides, there is one line saying that
The db:reset task will drop the database, recreate it and load the current
schema into it. This is not the same as running all the migrations.
Can anyone tell me where exactly they are different and why it is more error prone to replay the migration history?
I'm fairly new to Ruby on Rails. Thanks in advance.
The schema file contains the current structure of your database. When you load it, you are guaranteed to have the exact schema in your db that is in the file. Migrations were designed to make incremental changes in the database. You may add a table, then some columns, and then remove the table in three separate migrations. There's no need to go through all this when the schema already knows that the table no longer exists.
On why they are error prone, I'm not totally sure. The one thing I can think of is that migrations can be used to make changes to data and not just the structure.
Running rake db:reset will rebuild the structure of your database from schema.db, which essentially works as a cached version of your migrated database structure. Running all your migrations, on the other hand, applies the migrations one by one, which may include arbitrary code to accomodate for changes to the database (e.g. prepopulate an added counter cache column).
It can be more error prone to replay the migration history, since it is the product of changes to both the structure and data of the database. If the developers haven't been careful, it might not apply cleanly to a fresh environment (e.g. the migration assumes an old version of a model). On the other hand, schema.db can get out of sync if you edit a migration once you've migrated (a useful trick to avoid migration explosion during development). In that case, you need to run rake db:migrate:reset.