I do the following on my server:
pg_dump -O -c register_production > register.sql
Then, after copying register.sql to my local environment, I try:
psql register_development < register.sql
This appears to work, but when I try to launch the Rails site locally, I get this:
PG::UndefinedTable: ERROR: relation "list_items" does not exist at character 28
How can I restore everything (including relations) from the server db to my local dev db?
I use this command to save my database:
pg_dump -F c -v -U postgres -h localhost <database_name> -f /tmp/<filename>.psql
And this to restore it:
pg_restore -c -C -F c -v -U postgres /tmp/<filename>.psql
This dumps the database in Postgres' custom format (-F c) which is compressed by default and allows for reordering of its contents. -C -c will drop the database if it exists already and then recreate it, helpful in your case. And -v specifies verbose so you can see exactly what's happening when this goes on.
Does the register_development database exist before you run the psql command? Because that form will not create it for you.
See http://www.postgresql.org/docs/8.1/static/backup.html#BACKUP-DUMP-RESTORE for more information.
Related
version: '3.7'
services:
pgdump:
image: postgres:alpine
command: pg_dump -f "backup-`date -u -Iseconds`.pg_restore" $DATABASE_URL
This produces a file named
backup-`date -u -Iseconds`.pg_restore
instead of the desired
backup-2021-04-14T16:42:54+00:00.pg_restore.
I also tried:
command: pg_dump -f backup-`date -u -Iseconds`.pg_restore $DATABASE_URL
command: pg_dump -f "backup-${date -u -Iseconds}.pg_restore" $DATABASE_URL
command: pg_dump -f backup-${date -u -Iseconds}.pg_restore $DATABASE_URL
All of them yield different errors.
As of April 2021 command substitution is not supported by docker-compose according to this GitHub issue.
As a workaround in my use case, one could either use native docker run commands, where substitution works or use an .env file.
Current command
The date command itself is incorrect. Try running it on its own
date -u -Iseconds
echo `date -u -Iseconds`
From your command, I presume you want date in UTC seconds since epoch? Epoch by itself is UTC. So you just need seconds since Epoch. No need for -u parameter.
Solution
Here's the correct command in two forms:
A.
command: pg_dump -f "backup-`date +'%s'`.pg_restore" $DATABASE_URL
B.
command: pg_dump -f "backup-$(date +'%s').pg_restore" $DATABASE_URL
Explanation
There are multiple things to watch out for in the command you provided:
Notice the double quotes around the file name? This means you cannot nest another double-quote within the original outer pair without escaping the inner ones with \. Another alternative option is to use as many single-quote pairs you want within a pair of double-quotes. See this answer and this excerpt about 2.2.2 Single-Quotes and 2.2.3 Double-Quotes.
For string interpolation, you can use either $() or `` notation. But NOT within single-quotes as I said.
As a dry-run test, create a file directly with said notations:
vi "backup-`date +'%s'`.txt"
vi "backup-$(date +'%s').txt"
As for date format. Both GNU/date BSD/date accept %s to represent seconds since Epoch. Find "%s" in ss64 or man7 or cyberciti.
Docker-related, watch out what command does. Source:
command overrides the the default command declared by the container image (i.e. by Dockerfile's CMD).
You can create the filename and store it as a variable with shell command before doing the pg_dump:
version: '3.7'
services:
pgdump:
image: postgres:alpine
entrypoint: ["/bin/sh","-c"]
command: >
"FILENAME=backup-`date -u -Iseconds`.pg_restore
&& pg_dump -f $$FILENAME $$DATABASE_URL"
Successfully tested against Docker image for postgres 13.6.
While exporting databases using mysqldump like this,
mysqldump -u mysqluser -p mysqlpassword databasename > /tmp/databasename.sql
Will this command also export stored procedures that listed using the following command,
SHOW PROCEDURE STATUS WHERE db = 'databasename';
If not, how to export mysql database using mysqldump along with its associated stored procedures from the Linux terminal? Also note that i cannot use phpMyAdmin for this purpose.
Try this.
mysqldump -u mysqluser -p mysqlpassword --routines databasename > /tmp/databasename.sql
Refer this link : http://www.ducea.com/2007/07/25/dumping-mysql-stored-procedures-functions-and-triggers/
We can use the -R flag as a substitute for --routines flag while dumping as the other answer suggest.
mysqldump -u mysqluser -p mysqlpassword -R databasename > /tmp/databasename.sql
In this folder called my_backup I have a mongodb database dump with all my models/collections for example:
admins.bson
admins.metadata.json
categories.bson
categories.metadata.json
pages.bson
pages.metadata.json
.
.
.
I have a database called ubuntu_development on mongodb. I am working with rails 3 + mongoid
How can I import/restore all models/collections from the folder my_backup to my database ubuntu_development
Thank you very much!
Execute this command from the console (in this case):
mongorestore my_backup --db ubuntu_development
mongodbrestore is followed by my_backup, which is the folder name where the previous dump of the database is saved.
--db ubuntu_development specifies the database name where we want to restore the data.
To import .bson files
mongorestore -d db_name -c collection_name path/file.bson
Incase only for a single collection.Try this:
mongorestore --drop -d db_name -c collection_name path/file.bson
To import .json files
mongoimport --db db_name --collection collection_name --file name.json
You have to run this mongorestore command via cmd and not on Mongo Shell... Have a look at below command on...
Run this command on cmd (not on Mongo shell)
>path\to\mongorestore.exe -d dbname -c collection_name path\to\same\collection.bson
Here path\to\mongorestore.exe is path of mongorestore.exe inside bin folder of mongodb. dbname is name of databse. collection_name is name of collection.bson. path\to\same\collection.bson is the path up to that collection.
Now from mongo shell you can verify that database is created or not (If it does not exist, database with same name will be created with collection).
I'm trying to import my production Heroku database into my development machine.
My local db is PostgreSQL.
First, I'm exporting the dump from Heroku to my machine
curl -o latest.dump `heroku pgbackups:url`
Then, I try to drop the local db with rake db:drop and then I create the empty database again by using rake db:create.
The problem I'm getting is when actually trying to import the dump to the database
psql -d app_development -U myusername -f mydumpfile.sql
I begin seeing errors like this
psql:latest.dump:24: ERROR: syntax error at or near "PGDMP"
LINE 1: PGDMP
^
psql:latest.dump:28: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:36: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:40: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
^
psql:latest.dump:45: ERROR: syntax error at or near ""
LINE 1: id integer NOT NULL,
^
psql:latest.dump:49: ERROR: syntax error at or near ""
LINE 1: INCREMENT BY 1
...
psql:latest.dump:1601: invalid command \S4???(?̭?A?|c?e0<00K?A?}FϚ?????A(??~?t?I?????G(? K???l??k"?H?ȁ?ͲS?,N*?[(#??a5J??j}
psql:latest.dump:1602: invalid command \??k???|??w???h?
psql:latest.dump:1603: invalid command \=??????o?h?
psql:latest.dump:1609: invalid command \????^.?????????E???/-???+??>#?ؚE?.2)Ȯ&???? g????"7},_??]?:?f?Tr|o???)?p????h?KO?08[Rqu???|3?cW?ڮ?ahbm??H?H8??$???2?a?-أ
psql:latest.dump:1613: invalid command \D!qVS???L??*??R??I!???
psql:latest.dump:1614: invalid command \??-?}Q
psql:latest.dump:12565: ERROR: invalid byte sequence for encoding "UTF8": 0xb0
Any idea what is happening this and how to solve it?
You see errors because psql tries to interpret SQL queries when you're actually giving him a compressed dump (that's what heroku uses).
While you can't read the dump, pg_restore -O latest.dump gives you valid SQL you could pipe to psql but the easy solution is the following one :
pg_restore -O -d app_development latest.dump
Notes :
Use -O because you probably don't use the random username of your remote heroku postgres db.
Heroku doesn't recommend to use taps but I don't know how really risky it is.
Follow these 4 simple steps in your terminal(Heroku Dev Center):
Create a backup copy of your database:
$ heroku pg:backups capture DATABASE_NAME
Download the copy from Heroku (to your local machine) using curl:
$ curl -o latest.dump `heroku pg:backups public-url`
Load it*:
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME and choose the desired database from your config/database.yml file.
DATABASE_NAME can be your development/test/production db (Ex. mydb_development)
That's it!
I wanted to avoid having to set up Postgres on my local machine (blowing away and recreating the database is a pain if you're just looking for quick instructions). I put together some exact instructions for doing this with a local Postgres database running Docker. I'm adding a link here, since Google kept bringing me to this question (and it's a possible solution, though probably not what you're looking for):
https://gist.github.com/locofocos/badd43131f14b3e40c760741d5a26471
Heroku export the .dump extension file of the db to import in any of the relational DB by having its own norms and conditions.
While importing it to the local postgres DB, first you download the file latest.dump into your local machine and then run
pg_restore -h localhost -p 5432 -U postgres_username -d db_name -v latest.dump
and restart the rails server.
Late 2021 update for the highest voted answer to date (works great):
$ rails db:drop db:create db:migrate
$ heroku pg:backups capture DATABASE_URL
$ curl -o latest.dump heroku pg:backups public-url
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U YOUR_USERNAME -d DATABASE_NAME latest.dump
get YOUR_USERNAME on your local machine
DATABASE_NAME can be your development/test/production db (Ex. rails_react_bootstrap_development) from your config/database.yml file.
DATABASE_URL is not a variable or example code you need to set. It is a valid heroku option
we have been uising capistrano to roll out changes for some time (it was set up by a previous coder). as IT have decided to remove his workspace from perforce ive created a new one in my name and figured it would just work, but it seems to be rolling back on [deploy:update_code].
ive chekced the usual username / password errors, and with any of these incorrect the error is
Perforce password (P4PASSWD) invalid or unset.
or
p4client is incorrect or unset
so im quite confident it isnt that.
any ideas? ill paste the error message in ommiting any details! thanks a lot
failed: "sh -c 'p4 -p p4SERVER:1666 -u P4Username -P p4Password -c P4Workspace sync -f #298781 && cp -rf p4 -p p4SERVER:1666 -u P4Username -P p4Password -c P4Workspace client -o | grep ^Root | cut -f2 /home/ubuntu/clan/releases/20110905145323 && (echo 298781 > /home/ubuntu/clan/releases/20110905145323/REVISION)'" on ipaddressofserver
I managed to resolve this problem. the issue was that when creating the workspace the "host" was specified as being my local machine, removing this optional param allowed it to deploy correctly