For some reason heroku pgbackups:restore won't work on my rails app,
I'm creating my dumpfile using the command:
pg_dump -F c -v -U john -h localhost project_development > -f dumpfile.dump
I then upload it to dropbox, as the heroku documentation says it needs to be needs to uploaded somewhere with an HTTP-accessible URL. After I've uploaded the file to dropbox, I then run the command
heroku pgbackups:restore HEROKU_POSTGRESQL_RED_URL 'https://dl.dropboxusercontent.com/content_link/JgHkmg0iv8vIMKWqlN6medJqjClPeWYl2AQqJWV8IeEiujnfYvPdtvq7xh26dvCQ'
the output of which is
Retrieving... done
Restoring... done
HEROKU_POSTGRESQL_RED_URL is my database url and the line in the single quotes is a raw version of my dumpfile(the dropbox documentation showed how to turn the dl file into a raw file).
But whenever go to my apps homepage I get an error, I can see from running heroku logs that the error is a database error
PG::UndefinedTable: ERROR: relation "listings" does not exist
LINE 1: SELECT "listings".* FROM "listings" ORDER BY listings.pri..
Also, when I run heroku pg:info I can see that nothing has actually been inserted into the database
=== HEROKU_POSTGRESQL_RED_URL (DATABASE_URL)
Plan: Hobby-dev
Status: Available
Connections: 2/20
PG Version: 9.3.4
Created: 2014-10-21 15:24 UTC
Data Size: 7.0 MB
Tables: 0
Rows: 0/10000 (In compliance)
Fork/Follow: Unsupported
Rollback: Unsupported
Can anyone tell me why my database isn't being updated? Thanks.
I tested this on a blank hobby dev database with heroku. You don't need to include the the no-owner and no-acl options (-x -O) but it's considered good practice since those alter statements will fail.
The problem with your dump command is the stray >
pg_dump -F c -v -O -x -U john -h localhost project_development -f dumpfile.dump
Upload the resulting file to dropbox or s3 and try the restore again. If there are errors you can run
heroku logs -n 100 --ps pgbackups
To see the log output from the pg_restore command.
Related
I am going through Heroku's getting started tutorial. As far as I know, Postgres is fully updated and everything is installed appropriately. Interestingly, the app works online when I deploy it to heroku, but when I try to run the app locally, not so much! I am getting a:
PG::ConnectionBad
could not translate host name "myname" to address: Temporary failure in name resolution
### Convenience alias for PG::Connection.new.
def self::connect( *args )
return PG::Connection.new( *args )
end
I have not touched any information in the files created by Heroku for this tutorial - so what could be wrong?
The following commands were run:
heroku login
git clone https://github.com/heroku/ruby-getting-started.git
cd ruby-getting-started
heroku create
git push heroku main
heroku open (this gets to the app okay) heroku logs --tail
which psql
bundle install
export DATABASE_URL=postgres://$(whoami) bundle exec rake db:create db:migrate (Here is where the error is)
heroku local web
(Error here) http://localhost:5000/
Debug Postgres connection
Confirm that Postgres exists in yout local machine, is running and it have a user that maches with you machine username.
Try to run this command in your local machine:
psql -U $(whoami) -d postgres
The output should be something like this:
psql (9.2.24, server 10.14)
WARNING: psql version 9.2, server version 10.0.
Some psql features might not work.
Type "help" for help.
postgres=#
Then, confirm that a database matching with your username exists with "\l".
postgres=#\l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
----------------+----------+----------+-------------+-------------+-----------------------
myuser | odoo | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
tip: you can close with "\q"
postgres=#\q
if no error, try replacing the value of DATABASE_URL with one of the following...
postgresql://
postgresql://localhost
postgresql://localhost:5433
postgresql://localhost/mydb
postgresql://user#localhost
If this is not successful, then you must review the way that postgres is installed.
...Or manually create a superadmin User with a password inside Postgres.
The conection string pattern is
postgres://user:password#host:port
example...
export DATABASE_URL=postgres://db_user:123456#localhost:5432
DATABASE_URL=postgres://$(whoami)
This is trying to connect to a computer which happens to have the same name as your local user account. Unsurprisingly, it doesn't exist, hence the error.
Here's the command to use if you are trying to make a backup or create a dump file from your local Postgres DB.
Mainly you need to change --host to --port, and that's it:
pg_dump --format t --no-owner --no-privileges --verbose --username postgres --password --dbname <DATABASE_NAME> --port <PORT, usually 5432> --file <THE FILE NAME YOU WANT>.dump
*if you are on Windows, use CMD instead of git bash. idk why.
Last night I was asked to take down a client's website and close out their accounts as they are going out of business. I made sure to go to the Heroku Dashboard, go into the Heroku Postgres dashboard, Durability > Create Manual Backup.
I then downloaded the back up to my computer just in case something came up.
Once I had the backup downloaded, I deleted everything off of Heroku.
Well sure enough they messaged me today because they wanted to look at some data. However I closed the project completely. Now I'm left with this backup file format extension, so I don't know what to do with it. Something like: "1c02eb4e-d4ac-4631-9b61-742e9ea42659"
I have Postgres installed on my dev machine and I still have a dev version of the Ruby on Rails project set up. Is there a Heroku CLI command I can use to replace my dev database with this backup? Or is there a GUI program I can open this up in and write some queries? I'm on Mac Mojave.
Here is the command to load the dump file on development
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U [username] -d [database name] infile
Relevant documentation: https://devcenter.heroku.com/articles/heroku-postgres-import-export#restore-to-local-database
I have been trying to deploy a local database which has some data into heroku. My app's name is myFirstBlog. But after deploying when I refresh my app, it says something went wrong. After so troubeshooting. I found out this:
Before exporting data into heroku database, I ran the command
heroku pg:info
And the output was:
Plan:Hobby-dev
Status:Available
Connections:1/20
PG Version: 9.4.1
Created: 2015-07-09 08:20 UTC
Data Size:6.6 MB
Tables: 3
Rows: 2/10000 (In compliance)
Fork/Follow: Unsupported
Rollback: Unsupported
And after I exported data to heroku and ran the same command, output was:
Plan: Hobby-dev
Status: Available
Connections:1/20
PG Version:9.4.1
Created:2015-07-09 08:20 UTC
Data Size:6.5 MB
Tables:0
Rows: 0/10000 (In compliance)
Fork/Follow:Unsupported
Rollback:Unsupported
The number of tables becomes 0 after export. Why is it happenning?
This is how I am exporting my local database's data to heroku:
PGPASSWORD="password" pg_dump -Fc --no-acl --no-owner -h localhost -U aditya9509 myFirstBlog_development > backup.dump //This command dumps the data in the backup.dump
Then I saved backup.dump in my github account because the tutorials stated that in order to export data from local database to heroku, it must be at a location which can be retrieved using a http protocol. I did not understand why but I did what it said.
Then finally I ran this command:
heroku pg:backups restore "http://github.com/aditya9509/rubyOnRails/blob/master/backup.dump" DATABASE -a stark-beach-9626
The "stark-beach-9626 is the name of the app given by heroku.
After running this command, when I access the app, it shows "something went wrong". What am I missing here?
P.S. I am new to ruby on rails so please be as simple as you can when you answer. I have been busting my head to solve thi problem for hours now. Also, let me know if you need some additional info. I gave all the info I thought was relevant.
You are trying to restore the git blob and not the raw file, use https://github.com/aditya9509/rubyOnRails/raw/master/backup.dump for the url and it will work.
I'm trying to download a file (a copy) of our staging DB. I'm using pgbackups following these docs.
I created a backup, than got the public-url using heroku pgbackups:url b462 --app staging-appname, which worked.
Than I run the command curl -o latest.dump heroku pg:backups <public-url> b462 -a staging-appname and get curl: no URL specified!
Where it says <public-url> I've copied the actual long public-url, as in https://s3.amazonaws.com/reallly-long-url-with-acess-key-ect.
What am I missing here?
The following will create the backup:
heroku pg:backups capture --app name-of-the-app
Using the following information printed in terminal
---backup---> b001
In the following command
heroku pg:backups public-url b001 -a name-of-the-app
Will give you the url of the dump
Try just the following to get the latest file immediately after running heroku pgbackups:capture --app=staging-appname:
curl -o latest.dump `heroku pgbackups:url --app=staging-appname`
The b.. is required only if you are looking for a specific older file.
I was also struggling with this doc, and Prakash's suggestion didn't work for me (error complained about not being able to resolve 'heroku' which isn't part of the URL, so the syntax is seems to be bad, at least for the version of the pg or pgbackup gem I'm using.)
I also tried wget, as suggested in the heroku doc, to no avail.
When I'm stumped I try to think it down to the most simple elements, which led me to a far simpler solution than is provided by the doc...
I used the command in Rafael's answer to generate a temporary URL to retrieve the dump file:
heroku pg:backups public-url b001 -a name-of-the-app
Then, instead of plugging the resulting URL into the curl command, I just pasted that URL into Chrome's address bar and downloaded the file using my web browser.
The download was successful, the db file is valid.
With a little more sugar, here an example of shell script to download mysql dump from heroku and to restore it as a local database
backup_name=$1
if [ -z "$backup_name" ]
then
echo "You must supplied heroku backup name (ex: b001)"
return
fi
echo "Loading '$backup_name' database from heroku"
heroku pg:backups public-url $backup_name -a your-app-name > last_url.txt
curl -o latest.dump `cat last_url.txt`
dropdb local-db-name
createdb local-db-name
pg_restore -c -d local-db-name latest.dump
You need to give the heroku backup name as input.
I have a staging app full of data that I want to use to populate my currently empty production database. Currently I am trying to use pg:transfer. What is the correct syntax to use?
Addresses:
Staging app: afternoon-oasis-XXXX
Production app: warm-springs-XXXX, or postgres://long-database-url.compute-1.amazonaws.com:XXX/XXXXXXXX
The documentation states:
#documentation
$ heroku pg:transfer --to `heroku config:get DATABASE_URL -a app-staging` --confirm someapp
I have tried
$ heroku pg:transfer -t postgres://long-database-url.compute-1.amazonaws.com:XXX/XXXXXXXX -f JADE
This should be pulling from JADE, but the confirm message that appears to indicate that JADE, my afternoon-oasis app, my intended source, is going to be altered:
WARNING: Destructive Action
! This command will affect the app: afternoon-oasis-XXXX
! To proceed, type "afternoon-oasis-XXXX" or re-run this command with --confirm afternoon-oasis-XXXX
Why would Heroku be altering the source database? Or am I getting the syntax wrong?
Thanks in advance.
Second Update
You need the pgbackups addon for this but it is free. Sorry forgot that.
This addon will backup your postgres database every so often which is great if you ever need to recover data.
To add it just run heroku addons:add pgbackups:auto-week -a warm-springs-XXXX
Also add pgbackups to staging app
heroku addons:add pgbackups:auto-week -a afternoon-oasis-XXXX
Then run
heroku pgbackups:capture -a afternoon-oasis-XXXX to backups the latest
Finally you can run
heroku pgbackups:restore DATABASE_URL `heroku pgbackups:url -a afternoon-oasis-XXXX` -a warm-springs-XXXX
This command first gets the url of your staging app db backup and the pulls it to your production.
Original
Instead of using pg:transfer try doing something like this:
heroku pgbackups:restore DATABASE_URL 'INPUT YOUR STAGING DATABASE URL' -a warm-springs-XXXX
Let me know if you've got any questions. Then you can just type the confirm message with prompted or add the --confirm warm-springs-XXXX to the end of the command above.