I'm trying to import my production Heroku database into my development machine.
My local db is PostgreSQL.
First, I'm exporting the dump from Heroku to my machine
curl -o latest.dump `heroku pgbackups:url`
Then, I try to drop the local db with rake db:drop and then I create the empty database again by using rake db:create.
The problem I'm getting is when actually trying to import the dump to the database
psql -d app_development -U myusername -f mydumpfile.sql
I begin seeing errors like this
psql:latest.dump:24: ERROR: syntax error at or near "PGDMP"
LINE 1: PGDMP
^
psql:latest.dump:28: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:36: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:40: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:45: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:49: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
...
psql:latest.dump:1601: invalid command \S4???(?̭?A?|c?e0<00K?A?}FϚ?????A(??~?t?I?????G(? K???l??k"?H?ȁ?ͲS?,N*?[(#??a5J??j}
psql:latest.dump:1602: invalid command \??k???|??w???h?
psql:latest.dump:1603: invalid command \=??????o?h?
psql:latest.dump:1609: invalid command \????^.?????????E???/-???+??>#?ؚE?.2)Ȯ&???? g????"7},_??]?:?f?Tr|o???)?p????h?KO?08[Rqu???|3?cW?ڮ?ahbm??H?H8??$???2?a?-أ
psql:latest.dump:1613: invalid command \D!qVS???L??*??R??I!???
psql:latest.dump:1614: invalid command \??-?}Q
psql:latest.dump:12565: ERROR: invalid byte sequence for encoding "UTF8": 0xb0
Any idea what is happening this and how to solve it?
You see errors because psql tries to interpret SQL queries when you're actually giving him a compressed dump (that's what heroku uses).
While you can't read the dump, pg_restore -O latest.dump gives you valid SQL you could pipe to psql but the easy solution is the following one :
pg_restore -O -d app_development latest.dump
Notes :
Use -O because you probably don't use the random username of your remote heroku postgres db.
Heroku doesn't recommend to use taps but I don't know how really risky it is.
Follow these 4 simple steps in your terminal(Heroku Dev Center):
Create a backup copy of your database:
$ heroku pg:backups capture DATABASE_NAME
Download the copy from Heroku (to your local machine) using curl:
$ curl -o latest.dump `heroku pg:backups public-url`
Load it*:
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME and choose the desired database from your config/database.yml file.
DATABASE_NAME can be your development/test/production db (Ex. mydb_development)
That's it!
I wanted to avoid having to set up Postgres on my local machine (blowing away and recreating the database is a pain if you're just looking for quick instructions). I put together some exact instructions for doing this with a local Postgres database running Docker. I'm adding a link here, since Google kept bringing me to this question (and it's a possible solution, though probably not what you're looking for):
https://gist.github.com/locofocos/badd43131f14b3e40c760741d5a26471
Heroku export the .dump extension file of the db to import in any of the relational DB by having its own norms and conditions.
While importing it to the local postgres DB, first you download the file latest.dump into your local machine and then run
pg_restore -h localhost -p 5432 -U postgres_username -d db_name -v latest.dump
and restart the rails server.
Late 2021 update for the highest voted answer to date (works great):
$ rails db:drop db:create db:migrate
$ heroku pg:backups capture DATABASE_URL
$ curl -o latest.dump heroku pg:backups public-url
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME on your local machine
DATABASE_NAME can be your development/test/production db (Ex. rails_react_bootstrap_development) from your config/database.yml file.
DATABASE_URL is not a variable or example code you need to set. It is a valid heroku option
Related
While exporting databases using mysqldump like this,
mysqldump -u mysqluser -p mysqlpassword databasename > /tmp/databasename.sql
Will this command also export stored procedures that listed using the following command,
SHOW PROCEDURE STATUS WHERE db = 'databasename';
If not, how to export mysql database using mysqldump along with its associated stored procedures from the Linux terminal? Also note that i cannot use phpMyAdmin for this purpose.
Try this.
mysqldump -u mysqluser -p mysqlpassword --routines databasename > /tmp/databasename.sql
Refer this link : http://www.ducea.com/2007/07/25/dumping-mysql-stored-procedures-functions-and-triggers/
We can use the -R flag as a substitute for --routines flag while dumping as the other answer suggest.
mysqldump -u mysqluser -p mysqlpassword -R databasename > /tmp/databasename.sql
Just trying to set up rails app on ubuntu to do some low level dev work without going through the hassle of bothering our main devs. However at the very end of the process, I run rails s command. I get landing page on localhost:3000 and can get through the pages without direct call to database. However when I go to any page that gets the data from db, it gives me following error:
ActiveRecord::StatementInvalid in WantsController#new
PG::Error: ERROR: relation "geometry_columns" does not exist
LINE 1: SELECT * FROM geometry_columns WHERE f_table_name='wants'
^
: SELECT * FROM geometry_columns WHERE f_table_name='wants'
Now, no idea what I'm doing wrong. Here are the steps I'm taking:
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" >> /etc/apt/sources.list'
wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install postgresql-9.3-postgis pgadmin3 postgresql-contrib postgresql-server-dev-9.3
sudo -u postgres psql
postgres=# CREATE EXTENSION adminpack;
postgres=# CREATE EXTENSION postgis;
postgres=# \q
postgres=# CREATE USER abc WITH PASSWORD 'abc';
postgres=# CREATE DATABASE abc;
postgres=# CREATE DATABASE abc_development;
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U abc -d abc abc-20131217-09.pgdmp
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U abc -d abc_development abc_development-20131217-09.pgdmp
Then navigate to cloned git folder and execute bundle and rails s
Here's the database.yml:
development: &dev
# adapter: postgresql
database: abc_development
username: abc
password: abc
host: localhost
encoding: utf8
postgis_extension: true
schema_search_path: public,postgis
adapter: postgis
encoding: utf8
postgis_extension: postgis
production:
adapter: postgis
database: abc
username: abc
password: oTSZ1gQdwsFXWIUZsehj
host: localhost
encoding: utf8
postgis_extension: true
schema_search_path: public,postgis
postgis_extension: postgis
Any ideas where it is going wrong? Help would be much appreciated.
Okay, I solved it somehow. Don't really know what did the trick but basically, it involved giving superuser privileges to "abc" user, then running db:create, db:migrate, importing db file from our site backups and running migrations again.
The error message is telling you that those tables do not exist in your database. You need to create migrations(basically just hunks of sql that rails can generate for you) then run rake db:migrate to run the migrations on your database.
You can read more about migrations at the rails guide to migrations.
I do the following on my server:
pg_dump -O -c register_production > register.sql
Then, after copying register.sql to my local environment, I try:
psql register_development < register.sql
This appears to work, but when I try to launch the Rails site locally, I get this:
PG::UndefinedTable: ERROR: relation "list_items" does not exist at character 28
How can I restore everything (including relations) from the server db to my local dev db?
I use this command to save my database:
pg_dump -F c -v -U postgres -h localhost <database_name> -f /tmp/<filename>.psql
And this to restore it:
pg_restore -c -C -F c -v -U postgres /tmp/<filename>.psql
This dumps the database in Postgres' custom format (-F c) which is compressed by default and allows for reordering of its contents. -C -c will drop the database if it exists already and then recreate it, helpful in your case. And -v specifies verbose so you can see exactly what's happening when this goes on.
Does the register_development database exist before you run the psql command? Because that form will not create it for you.
See http://www.postgresql.org/docs/8.1/static/backup.html#BACKUP-DUMP-RESTORE for more information.
In this folder called my_backup I have a mongodb database dump with all my models/collections for example:
admins.bson
admins.metadata.json
categories.bson
categories.metadata.json
pages.bson
pages.metadata.json
.
.
.
I have a database called ubuntu_development on mongodb. I am working with rails 3 + mongoid
How can I import/restore all models/collections from the folder my_backup to my database ubuntu_development
Thank you very much!
Execute this command from the console (in this case):
mongorestore my_backup --db ubuntu_development
mongodbrestore is followed by my_backup, which is the folder name where the previous dump of the database is saved.
--db ubuntu_development specifies the database name where we want to restore the data.
To import .bson files
mongorestore -d db_name -c collection_name path/file.bson
Incase only for a single collection.Try this:
mongorestore --drop -d db_name -c collection_name path/file.bson
To import .json files
mongoimport --db db_name --collection collection_name --file name.json
You have to run this mongorestore command via cmd and not on Mongo Shell... Have a look at below command on...
Run this command on cmd (not on Mongo shell)
>path\to\mongorestore.exe -d dbname -c collection_name path\to\same\collection.bson
Here path\to\mongorestore.exe is path of mongorestore.exe inside bin folder of mongodb. dbname is name of databse. collection_name is name of collection.bson. path\to\same\collection.bson is the path up to that collection.
Now from mongo shell you can verify that database is created or not (If it does not exist, database with same name will be created with collection).
Trying to copy a production db with:
heroku db:pull postgres://postgres:#localhost/prod_20120717 \
--app my_app --confirm my_app
but I get:
$ heroku db:pull postgres://postgres:#localhost/prod_20120717 --app my_app \
--confirm my_app
Loaded Taps v0.3.24
Warning: Data in the database 'postgres://postgres:#localhost/prod_20120717' \
will be overwritten and will not be recoverable.
! App not found.
heroku is probably misconfigured. Try to do this heroku login and redo your login.
In addition, please use pgbackups, not pg:pull. https://devcenter.heroku.com/articles/pgbackups