I have a development env and a beta env for the app that I am building. Yesterday I came across a strange error.
I wrote a migration to change one of the tables and it worked fine on my dev env. Once I deployed the changes to the beta env it ran and but when I access the page it started giving me trouble. The new columns that I added were all undefined columns when it comes to the beta environment. So looked at the schema for the column on the beta side using column_names function on the class and it still had the old columns while logging into mysql there and checking the fields in the table shows me the new ones.
Anyone have any idea why is the schema not updating while the database was updated. Is there anyway to update the schema for a class, like some function or something.
I'm having this problem too. So far, the only solution I've found is to rename the table.
Related
I have been building our grails app to AWS Elastic Beanstalk through Jenkins for awhile now without issue, jumping and building between branches for years. This became an issue, though, when adding the grails test suite into the build.
I set up a test database for jenkins to use itself and let grails populate all the table data on its own, it worked for several months until recently when I decided to deploy a branch that was around 6 months old to one of our development environments. As you can guess, a branch 6 months old was missing some columns that were in more recent releases, and hence in the database, so grails deleted those columns, and tested and deployed without issue.
The problem arose when I went to deploy a more recent branch to a different environment, and grails test-app started failing due to sql errors because the app was trying to use a column that didn't exist on that table.
I dug into it further and discovered in the logs that when grails should have been trying to update the tables because they already existed and just needed a column added, it was trying to insert the tables instead. Obviously this caused issue with the tables already existing and the database not being updated.
Does anyone have any knowledge on how to force grails test-app to update the database tables instead of try to insert them? This has never happened in the use of the app, so I know this is localized to an issue with the test-suite, but the documentation on it is kind of bad, especially for grails 2.3.11 so I can't find anything.
This ended up being due to the old branch that was deployed having a different dbCreate value for the test environment, so for some reason when it made it's changes with the dbCreate property of "update" then switched back to "create-drop" it no longer could drop the tables before re-adding them.
I am new to Rails and I am facing the following problem. There were different tables in the existing project schema such as questionnaire, test, answers, questions, etc. But they did not have the columns of the creation date and the update date, and there were no models and views yet. So I decided to delete them and recreate them. Unfortunately I did it directly through the pgAdmin. Then I recreated each model with a scaffold generator. Everything was going fine until I got to the Questions model. For this model, I forgot to delete the table from the database, which I decided to do. But when I returned to the terminal and tried again to migrate the creation of the question table, I came across the same error message that the table already exists. Moreover, in the schema, I found that all tables manually removed from the database are still present.
What I tried:
I wrote migrations to drop tables. Migrations go through, tables disappear from the schema. Then I tried to write again table creation migrations, they are also started and tables are added to the schema, but they are not created in the database itself. But the application is still accessing the database, retrieving data, inserting it (into other tables that I have not touched on).
How can I return everything to the original? Migrations do not work in the project and I do not understand how to fix it. And I would not really like to dump the database, since it contains a lot of data.
Also in the project there are old migrations of test creation, test results, etc., which I deleted manually through the PG Admin. Should I delete them?
db/migrate
I found a bug. The problem was that the project turns out to be using docker (did not fully understand what it is). But as I understand it, for any update of the project, it must be reloaded through 4 commands in the terminal. Therefore, migrations did not go through the terminal. they had to be done by the docker himself. I expected to see the error logs right in the terminal, and they were issued by docker. I should have connected to the docker logs and read errors from there. There, he swore in plain text about the impossibility of migrating due to the lack of tables. I rolled back migrations to the state before the problem, removed migrations conflicting with the current state of the database and created new ones, according to the rails conventions. In the end, everything worked.
For a Ruby-on-Rails app that I'm hosting through Heroku, I recently downloaded a backup, restored it locally and then added data to the database from outside sources. That all worked fine.
After the updated database was pushed to Heroku using heroku pg:push <localdbname> HEROKU_DATABASE_URL --app <appname> the app was working fine and I could see the data that was newly added.
However today when I tried to log in to the app on Heroku, I was getting error messages. In my log file I saw this error:
ActionView::Template::Error (Unknown primary key for table ...
According to this SO post: Getting "Unknown primary key for table" while the ID is there
one user was able to get around this problem by resetting and pushing the database to Heroku several times. However, that hasn't worked for me. I've tried to reset and push the database at least 4 times now.
One possibility is that my local postgres database is using version 9.6 and the app is using version 9.4. The Heroku database is a Hobby-Basic database. There is documentation from Heroku on how to upgrade the Postgres version but it's not totally clear from their example what I would need to do. I'm guessing that I basically need to create a new database, copy the data from the old to the new and then destroy the old. Has anyone done this before? Is there a fee associated with doing so? And has upgrading the version of Postgres remotely fixed this issue for other people?
I have ran into similar issues before. I was creating a postgres database outside of rails in RazorSQL. I imported data from outside sources and when I ran it locally everything worked fine. When it came to deployment I ran into all sorts of issues.
I created tables outside of rails thus no migrations were created. I had to recreate the tables by deleting the current model and generating a new model which is a copy of the old one only this time a migration is created for example:
rails generate model ad name:string description:text price:decimal
seller_id:integer email:string img_url:string
I needed to import data from the database I had already created and before I did anything I actually created a seed file using this gem https://github.com/rroblak/seed_dump. All I had to do when I created the model again (remember to delete the model and recreate it) was run rake db:seedand it pulled the data in.
3.The last thing I had to do was insert the assocations in the models for example in a owner model putting in has many: customersto connect the customer model.
Another scenario was I ran a query in RazorSQL and generated a new table from the query and imported it into postgres. The problem was it was only a table from results thus it did not have a primary key. I had to manually create that in postgres using ALTER TABLE test1 ADD COLUMN id SERIAL PRIMARY KEY; This gave me an auto incrementing primary key field and I have to give credit to this answer which helped me https://stackoverflow.com/a/2944561/7039895.
Hope this helps.
According to the docs, you several choices of how to upgrade your database. pg:copy and pg:upgrade.
In your case, I recommand pg:copy: this is easier but required your database to be "off" while your doing the upgrade. This should not be an issue since your database is not currently working.
pg:upgrade should only be used when the downtime required for a PG copy
upgrade is unacceptably long for your business.
Upgrade with PG copy: (All the steps are explain in depth in the link)
Provision new database
You need to create a new database, it will be automatically using the last version of postgresql for heroku (in your case 9.6)
heroku addons:create heroku-postgresql:standard-0
Prevent new database updates
Stop you current database from writing mode to avoid corrupted data while copying to the new
heroku maintenance:on
Transfer data to new database
You need to copy all the data from the old DB to the new.
heroku pg:copy DATABASE_URL HEROKU_POSTGRESQL_PINK --app sushi
Promote new database
You need to notify heroku that you will use the new DB and not the old one.
heroku pg:promote HEROKU_POSTGRESQL_PINK
Last step: Make application active
Everything should be good by now, just make your DB active so it can save new query.
heroku maintenance:off
As you can see, all the steps are straightforward. Therefore, if upgrading you version of postgreSQL doesn't solve the issue, you can still switch back to the old one before removing it.
Pricing
I think it depends of your subscription already (Hobby, Standard or Premium) https://www.heroku.com/pricing. It may be free, or cheap, but I think it also depends of the size of your DB.
To be sure it won't cost you too much, I'm sure you can ask directly to Heroku support https://devcenter.heroku.com/articles/paid-support.
I'm considering using Ruby on Rails for my next project. Understanding the deployment of a rails website is easy enough to understand (sounds like I'll be using Phusion Passenger)
But now I'm trying to figure out the database. I see a lot about "database migrations", which allow me to update the database using ruby code. I also see that I'm allowed to create both an up and down variant of these migrations.
However, I can only fathom how this works cleanly in a single direction. Imagine if I suddenly say "The color column cannot be null". So, the up will make it required and give all NULL entries a default value. But what will the down do? If you care about it being identical to how it started, you can't just set the default values back to NULL.
This doesn't really matter much for releases to production. That will likely just be done in a single direction (in the up direction). However, I want to use Gerrit for code reviews as well as setting up a bot to run a build before allowing check-ins...
So how could that work? From one code review to the next, the build server will check out the new set of code, and run the migrations? But when this happens, it won't even retain the migration code from before, so how could it run the down steps? As an simpler example, I do not see how I could check out an old version of the code and "db migrate" backwards.
Yes, you can't check out an old version of the code and then run a down migration from a newer version of the code. You would need to run the down migration before rolling back to the older code.
There are many, many cases where a down migration is just not practical or possible. That's not necessarily a bad thing. It just means that you have defined a 'point of no return', where you won't be able to restore your database to an earlier state.
Migrations like creating a table or adding a column are easily reversed by simply destroying that table or removing that column. However, if you are doing something more complex, such as adding default values or moving data around, then you can tell Rails that it's not possible to reverse this migration:
def down
raise ActiveRecord::IrreversibleMigration
end
I would recommend that Gerrit should never assume anything about the database. It should start with a fresh database each time a new version is deployed, and run db:migrate to run all your migrations. You can use gems like factory_girl to populate your app with demo data for testing purposes.
I'm not a Rails developer (currently) so please forgive my ignorance on this.
One thing I've always liked about Rails is migrations and how it fills a need that's common across all languages and platforms. With that said, I am curious to understand what a certain scenario would result with the changes made in 2.1.
Rails 2.1 and higher, from what I can tell, made two changes to the migrations logic. The first was to use timestamp based script names when generated in order to reduce the probability of 2 developers working on the same file at the same time before adding the file to source control. So instead of 002_test.rb, it is now 20090729123456_test.rb when the script is generated.
The second item was that the Schema_Info table was replaced with the Schema_Migrations table that presented a list of migrations and not just the latest version number.
Looking through the Rails source, I noticed that it took the "current version" of the schema as the max version found in the Schema_Migration table.
Here's the scenario I'm trying to figure out:
Developer A generates a new script: 20090729120000_test.rb.
Developer B generates a new script: 20090729130000_test.rb.
Developer B migrates his script to the database first by not specifying the version number and assuming that Developer A's script isn't added yet.
What happens when Developer A adds his script and tries to migrate to the latest version since his script version (based on the time stamp) is less than the currently applied version now?
I'm not positive, but I believe that he would have to do a "rake db:rollback" to undo the Developer B migration, then run "rake db:migrate" to do both of them in the proper order. Of course, if two developers are working independently on tables that require no integration with one another (as this case shows, since Developer B didn't have to wait for Developer A to run his migration), developer A can simply add one to the timestamp of Developer B's migration and it will be in proper order once again.
The short answer is: don't worry about it.
rake db:migrate will attempt to run any migrations that are not found in the schema_migrations table. It doesn't matter if there are newer migrations that have already been run.
If B is dependent on A and must be run in that order, then you might have a problem, but that's an issue between the developers.