Why is `mongorestore` not restoring one of my collections - ruby-on-rails

I have rails app which requires mongo dump of a test database, which I restore using something like
mongorestore -d test_database dump/test_databse
when I run this command from the terminal everything works fine
$ mongo test_database
MongoDB shell version: 2.4.12
connecting to: test_database
> db.users_user.count()
50
> db.users_posts.count()
100
but when I run the same command using Ruby
system "mongorestore -d test_database dump/test_databse"
one of the collections users_posts is not inserted
$ mongo test_database
MongoDB shell version: 2.4.12
connecting to: test_database
> db.users_user.count()
50
> db.users_posts.count()
0
What's going on here? Is it a permissions issue? I am stumped.

Related

How to take pg dump from testing server to my pc

I am trying to take pg dump from testing server to my local machine, i have been logged into my server
root#myproject-staging-development1:~/myproject/current#
Which command should i write in here to get pg dump into my local machine?
Dump Your PostgreSQL Database
Step 1
SSH to the staging/production server.
Step 2
Dump the desired database:
pg_dump database_name > database_name_20160527.sql
You can name your dump as you wish - I'm using dates to distinguish multiple dumps.
Step 3
Leave SSH and download your new SQL file using SCP.
scp login#host:path_to_dir_with/database_name_20160527.sql database_name_20160527.sql
This command logs you into your remote server using SSH and downloads a given file to the local directory specified by you. If you give no path to the local directory, the dump will be saved in your current working dir.
Example:
scp marcin#8.8.8.8:/home/my_app/backups/my_app_database_20160527.sql my_app_database_20160527.sql
Restore Your PostgreSQL Dump
Step 1
If you want to use the current localhost database, you must drop it first:
psql template1 -c 'drop database database_name;'
Step 2
Create a new database on the localhost:
psql template1 -c 'create database database_name with owner your_user_name;
Step 3
And write your dump into the database:
psql database_name < database_name_20160527.sql
Source
You can run the pg_dump via the ssh command so you have a one-liner:
filename="tmp/backup_$(date +%Y-%m-%d_%H-%M-%S).sql"
ssh user#IP \
"pg_dump --no-owner postgresql://user:pass#127.0.0.1/dbname" \
>"$filename"

Connecting to postgres from rails

I've installed postgres via homebrew.
brew info postgres ⏎
postgresql: stable 9.6.2 (bottled), HEAD
Object-relational database system
When I try and rake db:create
I get the error:
"could not connect to server: Connection refused"
If I run
ps aux | grep postgres
=>
But running
brew services start postgresql ⏎
Service postgresql already started, use brew services restart postgresql to restart.
I'm on OSX 10.11.6
Why can't rails connect to postgrsql?

How to import PG dump from digitalocean to local machine?

I am new to rails.
I want to take backup of pg database from digitalocean to my local machine. How I take dump of that and migrate to my local machine
To use pg_dump,
First, for the target machine(remote machine with database you want to dump), two steps to make the machine receive pg_dump requests:
1.Add or edit the following line in your postgresql.conf :(in my experience, the location maybe /etc/postgresql/9.3/main/postgresql.conf, replace 9.3 with your psql version. If nobody change the file before, you add the line below to the end of the file)
listen_addresses = '*'
2.Add the following line as the first line of file 'pg_hba.conf'. (in my experience, the location like /etc/postgresql/9.3/main/pg_hba.conf) It allows access to all databases for all users with an encrypted password:
# TYPE DATABASEUSER CIDR-ADDRESS METHOD
host all all all md5
After those two steps, type in the terminal:
/etc/init.d/postgresql start
At last, in your local machine, you should figure out the target database's user(or owner) who can read it:
You can achieve this by ssh to connect that machine and step into psql console
sudo su - postgres && psql
and type
\l
to see the db owner.
Finally you can use pg_dump in your local machine to dump the database.Like :
pg_dump -f dump_name -h host_ip -d database_name -U database_user -p 5432 -W
then input the user's password, and wait for the long time for dumping the db.
Hope you make it~
First you need to create backup then download dump from digital ocean and the run these commands on console.
Download dump using SCP.
1-pg_dump dbname > outfile
2-pg_restore --verbose --clean --jobs=4 --disable-triggers --no-acl --no-owner -h localhost -U user_name -d database_name outfile.dump

error with postgresql datababse : Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

When I run the rake db:migrate or run the rails s command, I get the same error:
Error : could not connect to server:
No such file or directory Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I get the error in the browser when I try rails s.
This is my database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
database: books_development
test:
<<: *default
database: books_test
production:
<<: *default
database: books_production
username: abd
password: <%= ENV['BOOKS_DATABASE_PASSWORD'] %>
Note : I have the databases books_development; books_test
; and the postresql are running without problems when I try sudo /etc/init.d/postgresql start
I did run:
create database books_development;
create database books_test;
in the psql console. And it said that it's done successfully
I tried a lot of solutions and I spent yesterday looking for a solution and no solution in the related questions solved my error.
I have postgresql-9.4 (the latest) and xubuntu 14.04
Any Ideas?
The convention for PostgreSQL packaged for Debian or Debian derivatives such as Ubuntu is to use /var/run/postgresql as the directory for Unix domain sockets. On the other hand the convention for self-compiled postgres client libs is to use /tmp, unless self-configured otherwise.
So the usual root cause of this mismatch between both is a mix of self-compiled client-side stuff with pre-compiled server-side packages (even if client and server are installed on the same machine, client-side and server-side are still distinct and can be out of sync).
Soft-linking from /tmp to this directory as suggested by the asker works except that the link will be lost at every reboot, because in general /tmp is emptied on reboot.
A better option would be to add as an entry in database.yml:
either host: /tmp if the real socket path is /tmp (self-compiled server, packaged client)
or host: /var/run/postgresql if the real socket path /var/run/postgresql/ (packaged server, self-compiled client).
When the value in the host field starts with a slash character, the postgres library knows that it's the location of a directory for local sockets rather than a hostname. The filename inside the directory .s.PGSQL.portnumber is generated and must not be specified, only the directory.
Another possibility is to configure the self-compiled software packages as closely as possible to Debian, overriding the defaults as they do.
I had the same Is the server running locally and accepting connections on Unix domain socket “/var/run/postgresql/.s.PGSQL.5432”? error when typing psql into the postgres user in Ubuntu 14.04. I could not find an existing working solution.
The short answer for me was: my install made a var/pgsql_socket directory but no configuration files knew about it.
1) Find the postgres.conf file (it was in etc/postgresql/9.6/main for me)
2) change to listen_addresses = '*'
3) add another unix socket directory
unix_socket_directories = '/var/run/postgresql, /var/pgsql_socket' # comma-separated list of directories
4) at this point, sudo service postgresql start attempted to start but did not have authority to create the lock file.
* The PostgreSQL server failed to start. Please check the log output:
2016-10-05 17:14:55 CEST [28472-1] FATAL: could not create lock file "/var/pgsql_socket/.s.PGSQL.5432.lock": Permission denied
2016-10-05 17:14:55 CEST [28472-2] LOG: database system is shut down
5) Change permissions ( found from Mark Berry's comment here )
$ sudo chown root.postgres /var/pgsql_socket
$ sudo chmod g+wx /var/pgsql_socket
6) sudo service postgresql start sudo -i -u postgres psql
That finally worked for me
That means your Postgres server is not running.
Check Postgres Service status from Terminal
sudo service postgresql status
Enable Postgres Service, If not started
sudo service postgresql start
OR
sudo service postgresql restart
Now your command should work, If Postgres Service is successfully started.
I solved It . I Just created a softlink using :
sudo ln -s /tmp/.s.PGSQL.5432 /var/run/postgresql/.s.PGSQL.5432
and then edited the
/etc/postgresql/9.4/main/pg_hba.conf
( If you have another version of postgresql you have to change 9.4 in the path)
From:
local all postgres peer
To:
local all postgres md5
Solution:
Try this
export LC_ALL="en_US.UTF-8"
and this. (9.3 is my current PostgreSQL version. Write your version!)
sudo pg_createcluster 9.3 main --start
The exact same symptom can be caused by a stale lock file /var/run/postgresql/.s.PGSQL.5432.lock. One of the symptoms of this is psql reporting
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
even though there is clearly a socket with this path available as reported by netstat -lp --protocol=unix | grep postgres
The problem can be solved by removing the lock file and restarting postgresql. This is definitely less invasive than a purge and re-install.
sudo rm /var/run/postgresql/.s.PGSQL.5432.lock
sudo service postgresql restart
On Mac OS X I usually get this error when my computer shuts down incorrectly, for example, due to power failure.
The solution I use is pretty simple and works 100% of the time:
# Find the postgres config folder
cd /usr/local/var/postgres
# remove file
rm postmaster.pid
# restart postgres
brew services restart postgres
Running pg_lsclusters will list all the postgres clusters running on your device
eg:
Ver Cluster Port Status Owner Data directory Log file
9.6 main 5432 online postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
if the status is down run
#format is pg_ctlcluster <version> <cluster> <action>
sudo pg_ctlcluster 9.6 main start
If this process is not successfull it will throw the error.
My error was(You can see the error log on /var/log/postgresql/postgresql-9.6-main.log)
FATAL: could not access private key file "/etc/ssl/private/ssl-cert-snakeoil.key": Permission denied
Try adding `postgres` user to the group `ssl-cert`
make sure that postgres is the owner of /var/lib/postgresql/version_no/main
eg: sudo chown postgres -R /var/lib/postgresql/9.6/main/
It happened to me and it turned out that I removed erroneously the Postgres user from "ssl-cert" group. Run the below code to fix the user group issue and fixing the permissions
#set user to group back with
sudo gpasswd -a postgres ssl-cert
# Fixed ownership and mode
sudo chown root:ssl-cert /etc/ssl/private/ssl-cert-snakeoil.key
sudo chmod 740 /etc/ssl/private/ssl-cert-snakeoil.key
# now postgresql starts! (and install command doesn't fail anymore)
sudo service postgres restart
When I run into this error, my Postgres server was actually listening on a different port (5433) and not 5432.
To solve this, add
port: 5433
to your database.yml file to instruct rails to use the same

Postgres sh: /usr/local/var/postgres/server.log: No such file or directory

I recently had to reinstall homebrew and postgres and now postgres isn't working with rails.
First when trying to run rails s, I get this:
/Users/lee/.rvm/gems/ruby-1.9.3-p125#achievex/gems/activerecord-3.2.6/lib/active_record/connection_adapters/postgresql_adapter.rb:1206:in `initialize': could not connect to server: Connection refused (PG::Error)
So I try starting postgres and I get this:
pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
server starting
sh: /usr/local/var/postgres/server.log: No such file or directory
It appears that postgres is running because of this:
ps aux | grep postgreslee 1099 0.3 0.0 2425480 188 s000 R+ 5:48PM 0:00.01 grep postgres
I'm totally at a loss here as to why I can't get it running.
First, ps | grep is in fact pulling your grep out of the process tree. Postgres is not running. This sort of thing happens when the directory to put the log in doesn't exist. So probably you are pointing it at a non-existant directory.
This could happen if you don't have a data directory yet.
Recommendations:
Start by making sure your data directory exists. search for pg_hba.conf and see if it is anywhere on your system. If it is then there is a good chance it is in your data directory (if it is in etc, then look for base/1/PG_VERSION This will be in your data directory one level up from base).
if it exists already change your postgresql.conf to set logging setting appropriately and drop the -l option.
If no data directory exists, use the initdb command to make one.

Resources