Related
Using docker-desktop on macOS.
I'm trying to run a node following the instructions on this page.
The database name is node, which is the same as the username: node. The user has access to the database and can log in using psql client.
Connection strings I've tried in the .env file:
postgresql://node#localhost/node
postgresql://node:password#localhost/node
postgresql://node:password#localhost:5432/node
postgresql://node:password#127.0.0.1:5432/node
postgresql://node:password#127.0.0.1/node
When I run the start command: cd ~/.chainlink-kovan && docker run -p 6688:6688 -v ~/.chainlink-kovan:/chainlink -it --env-file=.env smartcontract/chainlink local n , using docker-desktop on macOS, I get the following stack trace:
2020-09-15T14:24:41Z [INFO] Starting Chainlink Node 0.8.15 at commit a904730bd62c7174b80a2c4ccf885de3e78e3971 cmd/local_client.go:50
2020-09-15T14:24:41Z [INFO] SGX enclave *NOT* loaded cmd/enclave.go:11
2020-09-15T14:24:41Z [INFO] This version of chainlink was not built with support for SGX tasks cmd/enclave.go:12
2020-09-15T14:24:41Z [INFO] Locking postgres for exclusive access with 500ms timeout orm/orm.go:69
2020-09-15T14:24:41Z [ERROR] unable to lock ORM: dial tcp 127.0.0.1:5432: connect: connection refused logger/default.go:139 stacktrace=github.com/smartcontractkit/chainlink/core/logger.Error
/chainlink/core/logger/default.go:117
...
Does anyone know how I can resolve this?
The problem probably caused by the fact that your chainlink database has been locked with Exclusive Lock and before stopping node that locks never removed.
What you do in this situation (as what works for me) is use PgAdmin Ui or similar way to find all Locks then find the Exclusive Lock that is held on the chainlink database and note down its Process id or ids (if multiple exclusive locks there are on chainlink DB)
Log in to your pg client and run SELECT pg_terminate_backend(<pid>) or SELECT pg_cancel_backend(<pid>); Enter PID of those locks here without quotes and meanwhile keep refreshing on pg admin URL to see if those processes stopped If stopped then rerun your chainlink node.
The problem is with docker networking.
Add --network host to the docker run command so that it is:
cd ~/.chainlink-kovan && docker run -p 6688:6688 -v ~/.chainlink-kovan:/chainlink -it --env-file=.env smartcontract/chainlink --network host local n
This fixes the issue.
Installed Docker on Mac and trying to run Vespa on Docker following steps specified in following link
https://docs.vespa.ai/documentation/vespa-quick-start.html
I did n't had any issues till step 4. I see vespa container running after step 2 and step 3 returned 200 OK response.
But Step 5 failed to return 200 OK response. Below is the command I ran on my terminal
curl -s --head http://localhost:8080/ApplicationStatus
I keep getting
curl: (52) Empty reply from server whenever I run without -s option.
So I tried to see listening ports inside my vespa container and don't see anything for 8080 but can see for 19071(used in step 3)
➜ ~ docker exec vespa bash -c 'netstat -vatn| grep 8080'
➜ ~ docker exec vespa bash -c 'netstat -vatn| grep 19071'
tcp 0 0 0.0.0.0:19071 0.0.0.0:* LISTEN
Below doc has info related to vespa ports
https://docs.vespa.ai/documentation/reference/files-processes-and-ports.html
I'm assuming port 8080 should be active after docker run(step 2 of quick start link) and can be accessed outside container as port mapping is done.
But I don't see 8080 port active inside container in first place.
A'm I missing something. Do I need to perform any additional step than mentioned in quick start? FYI I installed Jenkins inside my docker and was able to access outside container via port mapping. But not sure why it's not working with vespa.I have been trying from quiet sometime but no progress. Please advice me if I'm missing something here.
You have too low memory for your docker container, "Minimum 6GB memory dedicated to Docker (the default is 2GB on Macs).". See https://docs.vespa.ai/documentation/vespa-quick-start.html
The deadlock detector warnings and failure to get configuration from configuration server (which is likely oom killed) indicates that you are too low on memory.
My guess is that your jdisc container had not finished initialize or did not initialize properly? Did you try to check the log?
docker exec vespa bash -c '/opt/vespa/bin/vespa-logfmt /opt/vespa/logs/vespa/vespa.log'
This should tell you if there was something wrong. When it is ready to receive requests you would see something like this:
[2018-12-10 06:30:37.854] INFO : container Container.org.eclipse.jetty.server.AbstractConnector Started SearchServer#79afa369{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
[2018-12-10 06:30:37.857] INFO : container Container.org.eclipse.jetty.server.Server Started #10280ms
[2018-12-10 06:30:37.857] INFO : container Container.com.yahoo.container.jdisc.ConfiguredApplication Switching to the latest deployed set of configurations and components. Application switch number: 0
[2018-12-10 06:30:37.859] INFO : container Container.com.yahoo.container.jdisc.ConfiguredApplication Initializing new set of configurations and components. Application switch number: 1
Before anything, please note that I have found several similar questions on Stack Overflow and articles all over the web, but none of those helped me fix my issue:
PG Error could not connect to server: Connection refused Is the server running on port 5432?
PG::ConnectionBad - could not connect to server: Connection refused
psql: could not connect to server: Connection refused
Now, here is the issue:
I have a Rails app that works like a charm.
With my collaborator, we use GitHub to work together.
We have a master and an mvp branches.
I recently updated my git version with Homebrew (Mac).
We use Foreman to start our app locally.
Now, when I try to launch the app locally, I get the following error:
PG::ConnectionBad at /
could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
I tried to reboot my computers several times.
I also checked the content of /usr/local/var/postgres:
PG_VERSION pg_dynshmem pg_multixact pg_snapshots pg_tblspc postgresql.conf
base pg_hba.conf pg_notify pg_stat pg_twophase postmaster.opts
global pg_ident.conf pg_replslot pg_stat_tmp pg_xlog server.log
pg_clog pg_logical pg_serial pg_subtrans postgresql.auto.conf
As you can see, there is no postmaster.pid file in there.
Any idea how I could fix this?
run postgres -D /usr/local/var/postgres and you should see something like:
FATAL: lock file "postmaster.pid" already exists
HINT: Is another postmaster (PID 379) running in data directory "/usr/local/var/postgres"?
Then run kill -9 PID in HINT
And you should be good to go.
You most likely ran out of battery and your postgresql server didn't shutdown correctly.
The easiest workaround is to download the official postgresql app and launch it: it will force the server to start (http://postgresapp.com/)
Most likely it's because your system shutdown unexpectedly
Try
postgres -D /usr/local/var/postgres
You might see
FATAL: lock file "postmaster.pid" already exists
HINT: Is another postmaster (PID 449) running in data directory "/usr/local/var/postgres"?
Then try
kill -9 PID
example
kill -9 419
And it should start postgres normally
The postgresql server might be down and the solution might be as simple as running:
sudo service postgresql start
which fixed the issue for me.
This could be caused by the pid file created for postgres which has not been deleted due to unexpected shutdown. To fix this, remove this pid file.
Find the postgres data directory. On a MAC using homebrew it is /usr/local/var/postgres/, other systems it might be /usr/var/postgres/
Remove pid file by running:
rm postmaster.pid
Restart postgress. On Mac, run:
brew services restart postgresql
I had almost just as same error with my Ruby on Rails application running postgresql(mac). This worked for me:
brew services restart postgresql
This worked in my case:
brew uninstall postgresql
rm -fr /usr/local/var/postgres/
brew install postgresql
In my case PostgreSQL updates from version 13.4 to 14 in background, so it fixes by:
brew postgresql-upgrade-database
In other case the problem fixed by:
rm -rf /usr/local/var/postgres/postmaster.pid
or
rm -rf /opt/homebrew/var/postgres/postmaster.pid
Restart service postgresql:
brew services restart postgresql
PS:
How can you understand what is the problem?
For first see what service is not correct started:
brew services list
For second show file postgres.log, where will be the error:
tail -f /usr/local/var/log/postgres.log
or
tail -f /opt/homebrew/var/log/postgres*
And so find answer by this error's text
I resolved the issue via this command
pg_ctl -D /usr/local/var/postgres start
At times, you might get this error
pg_ctl: another server might be running; trying to start server anyway
So, try running the following command and then run the first command given above.
pg_ctl -D /usr/local/var/postgres stop
Step 1:
cd /etc/postgresql/12/main/
open file named postgresql.conf
sudo nano postgresql.conf
add this line to that file
listen_addresses = '*'
then open file named pg_hba.conf
sudo nano pg_hba.conf
and add this line to that file
host all all 0.0.0.0/0 md5
It allows access to all databases for all users with an encrypted password
restart your server
sudo /etc/init.d/postgresql restart
This is how I solved my problem:
see the status of services
brew services list
and the output was :
Name Status User Plist
postgresql error myuser /Users/myuser/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
unbound stopped
I change the file name in this path, you can also remove it
mv /Users/myuser/Library/LaunchAgents/homebrew.mxcl.postgresql.plist /Users/myuser/Library/LaunchAgents/homebrew.mxcl.postgresql.plist_temp
and then reboot the os
sudo reboot
after booting I started the postgresql and it worked.
brew services start postgresql
Find postgresql#10 service directory
$ ls /usr/local/var/postgresql#10
Find file postmaster.pid and delete that file
$ rm -f postmaster.pid
Restart postgres service using
$ brew services restart postgresql#10
This worked for me:
run
sudo lsof -i :<port_number>
after that it will display the PID which is currently attached to the process.
After that run sudo kill -9 <PID>
if that doesn't work,
try the solution offered by user8376606 it would definitely work!
If you want to restart Postgresql on Linux, then you have to use the following command.
/etc/init.d/postgresql restart
In my case
I have changed the port in postgresql.conf file
and restart postgresql services in
Run => service.msc => Restart
now retry
First I tried
lsof -wni tcp:5432
but it doesn't show any PID number.
Second I tried
Postgres -D /usr/local/var/postgres
and it showed that server is listening.
So I just restarted my mac to restore all ports back and it worked for me.
For Docker users: In my case it was caused by excessive docker image size. You can remove unused data using prune command:
docker system prune --all --force --volumes
Warning: as per manual (docker system prune --help):
This will remove:
all stopped containers
all networks not used by at least one container
all dangling images
all dangling build cache
I encountered a similar problem when I was trying to connect my Django application to PostgreSQL database.
I wrote my Dockerfile with instructions to setup the Django project followed by instructions to install PostgreSQL and run Django server in my docker-compose.yml.
I defined two services in my docker-compose-yml.
services:
postgres:
image: "postgres:latest"
environment:
- POSTGRES_DB=abc
- POSTGRES_USER=abc
- POSTGRES_PASSWORD=abc
volumes:
- pg_data:/var/lib/postgresql/data/
django:
build: .
command: python /code/manage.py runserver 0.0.0.0:8004
volumes:
- .:/app
ports:
- 8004:8004
depends_on:
- postgres
Unfortunately whenever I used to run docker-compose up then same err. used to pop up.
And this is how my database was defined in Django settings.py.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'abc',
'USER': 'abc',
'PASSWORD': 'abc',
'HOST': '127.0.0.1',
'PORT': '5432',
'OPTIONS': {
'client_encoding': 'UTF8',
},
}
}
So, In the end I made use of docker-compose networking which means if I change the host of my database to postgres which is defined as a service in docker-compose.yml will do the wonders.
So, Replacing 'HOST': '127.0.0.1' => 'HOST': 'postgres' did wonders for me.
After replacement this is how your Database config in settings.py will look like.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'abc',
'USER': 'abc',
'PASSWORD': 'abc',
'HOST': 'postgres',
'PORT': '5432',
'OPTIONS': {
'client_encoding': 'UTF8',
},
}
}
I often encounter this problem on windows,the way I solved the problem is Service - Click PostgreSQL Database Server 8.3 - Click the second tab "log in" - choose the first line "the local system account".
It also gives the same error if you just stop your PostgreSQL app. You just need to start it again. (PostgreSQL 11)
I have faced the same issue and I was unable to start the postgresql server and was unable to access my db even after giving password, and I have been doing all the possible ways.
This solution worked for me,
For the Ubuntu users:
Through command line, type the following commands:
1.service --status-all (which gives list of all services and their status. where "+" refers to running and "-" refers that the service is no longer running)
check for postgresql status, if its "-" then type the following command
2.systemctl start postgresql (starts the server again)
refresh the postgresql page in browser, and it works
For the Windows users:
Search for services, where we can see list of services and the right click on postgresql, click on start and server works perfectly fine.
In my case I forgot to change the database from postgres (on my production) back to sqlite3 which I was using for development.
It's working for me >>Node.Js App
user#MacBook-Pro % sudo lsof -i :5430
Output
COMMAND PID user FD TYPE DEVICE SIZE/OFF NODE NAME
node 7885 user 21u IPv6 0x2e7d89f6118f95b9 0t0 TCP *:radec-corp (LISTEN)
Kill the PID
user#MacBook-Pro % sudo kill -9 7885
One more test
user#MacBook-Pro % sudo lsof -i :5430
user#MacBook-Pro % "No more running PID for the port 5430"
In my case, on a Ruby on Rails project, I removed a .pid file from the folder tmp/pids and restart the system.
Had the same issue. I checked that my database.yml file, (dev mode) host was pointing to 5433. I updated it to 5432 and it worked.
Just in case someone needs this for windows, read on.
On windows hit the Windows button + R
then enter services.msc and look for postgresql-x64-14, Right click it and click start
Then go back to your PgAdmin4 for windows and then enter your master password if asked.
From here, you should be able to proceed as usual with viewing of the db schemas.
Also, for Django, restart your server with CTR+C then python manage.py runserver (assuming you're working inside a virtual env)
Good luck
ps -ef|grep postgres
Then kill the process with PID
sudo kill -9 PID
Then start the postgresql
sudo service postgresql start
In my case I when it happens to me I need to do the following steps:
1º Step
Log in postgres user:
#sudo su postgres
2º Step
Run the following steps: /opt/PostgreSQL/10/bin/postgres -D /opt/PostgreSQL/10/data -r /usr/local/var/postgres/server.log
Explanation:
We access the utility of postgres located at /opt/PostgreSQL/10/bin/ in your case could be another but identify where it's.
After this step we tell to the utility of postgres where the it's data folder is by using the option -D, this data folder contains all necessary configuration of postgres server.
The option -r we tell to postgres where to send stdout and stderr to given file, in my case the file that I used is /usr/local/var/postgres/server.log
Note:
I'm using Postgre 10
Linux Ubuntu
I have configure and installed couchdb 1.6.1 on ubuntu 10.04. While configuring and installing it said successfully done. But when I start couchdb it gives me error.
When start first..
Apache CouchDB 1.6.1 (LogLevel=info) is starting.
Apache CouchDB has started. Time to relax.
**[info] [<0.32.0>] Apache CouchDB has started on http://127.0.0.1:5984/**
When starting again
Apache CouchDB 1.6.1 (LogLevel=info) is starting.
**Failure to start Mochiweb: eaddrinuse**
[error] [<0.127.0>] {error_report,<0.31.0>,
{<0.127.0>,crash_report,[[{initial_call,
{mochiweb_socket_server,init,['Argument__1']}},
{pid,<0.127.0>},
{registered_name,[]},
{error_info,
{exit,eaddrinuse,
[{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_secondary_services,couch_server_sup,
<0.32.0>]},
{messages,[]},
{links,[<0.95.0>]},
{dictionary,[]},
{trap_exit,true},
{status,running},
{heap_size,987},
{stack_size,24},
{reductions,467}],
[]]}}
{"init terminating in do_boot",{{badmatch,{error,{bad_return,{{couch_app,start,[normal,["/usr/local/etc/couchdb/default.ini","/usr/local/etc/couchdb/local.ini"]]},{'EXIT',{{badmatch,{error,shutdown}},[{couch_server_sup,start_server,1},{application_master,start_it_old,4}]}}}}}},[{couch,start,0},{init,start_it,1},{init,start_em,1}]}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
in local.ini I changed
Port to 5983 and also ip to 0.0.0.0 from 127.0.0.1
Nothing helped
When i run sudo netstat -tulpn
I get following output
tcp 0 0 127.0.0.1:5984 0.0.0.0:* LISTEN 21688/beam.smp
couchdb -s says Apache CouchDB is not running.
Thanks in advance
If you came here because you recently upgrade you CouchDB 2.0, make sure you updated your local.ini config file. The section for the web server configuration has changed (the name changed from something like httpd to chttpd). Because of that the port directive wasn't being picked up and it was defaulting to 80 which was the same as the web server.
Based on the error message entries provided, CouchDB is already up & running on your machine/ server with port 5984
**Failure to start Mochiweb: eaddrinuse**
That's why When you run sudo netstat -tulpn command on Linux based machine, you are getting the following output :
tcp 0 0 127.0.0.1:5984 0.0.0.0:* LISTEN 21688/beam.smp
Solutions :
couchdb is not a service script in the way you're trying to use it. It is just trying to start up again and failing because one is already running (erlang beam.smp process is still running).
Check & kill all CouchDB processes using following commands :
/bin/ps aux | grep couchdb | grep -v grep | awk '{print $2}' | xargs killl
Start CouchDB again using following command using root linux user :
service couchdb start
Note :
When you enter couchdb -s command, You are getting the message like Apache CouchDB is not running.
When you enter couchdb -d command, You are getting the message like Apache CouchDB is not running.
Reason for above outputs, You are executing above mentioned commands as non privileged linux users accounts.
Try to execute above mentioned commands as root Or couchDB running linux users accounts.
Even i faced same problem. I just did..
sudo apt-get update
sudo apt-get install couchdb
Note:i just re-installed couchdb now it's working fine. before re-install i did below configuration.
Install ICU and use locate to find the icu-config command:
locate icu-config
sudo apt-get install libicu-dev
for more details refer below link
https://wiki.apache.org/couchdb/Error_messages
I face the same problem...
Luckily there's a fix in the Couchdb Error messages documentation.
https://wiki.apache.org/couchdb/Error_messages
Problem
$ couchdb
Apache CouchDB 0.9.0a747640 (LogLevel=info) is starting.
Failure to start Mochiweb: eaddrinuse
{"init terminating in do_boot",{{badmatch,{error,shutdown}},[{couch_server_sup,start_server,1},{erl_eval,do_apply,5},{erl_eval,exprs,5},{init,start_it,1},{init,start_em,1}]}}
Solution
Edit your /etc/couchdb/couch.ini file and change the Port setting to an available port.
But mine didn't have couch.ini but had default.ini.
I changed that to another port that i guessed should be free and it worked
I have encountered this error in starting couchdb on ubuntu 16.04
The reason being erlang running on the port 5984, you would be something like below when running the command "netstat -tulpn"
0 0 127.0.0.1:5984 0.0.0.0:* LISTEN 13967/beam.smp
open the file /etc/couchdb/default.ini in super user mode and change the httpd port to some other port which is available and you would be able to start couchdb without any fail.
I am trying to run ruby on rails under passenger with apache2 under fedora 19 and I got this error in log:
[Tue Feb 25 09:37:52.367683 2014] [passenger:error] [pid 2779] ***
Passenger could not be initialized because of this error: Unable to
start the Phusion Passenger watchdog because it encountered the
following error during startup: Cannot change the directory
'/tmp/passenger.1.0.2779/generation-1/buffered_uploads' its UID to 48
and GID to 48: Operation not permitted (errno=1)
That directory (/tmp/passenger.1.0.2779) doesn't even exist. I think that problem is with selinux. I tried to solve it about 4 hours. Httpd is running under user apache and group apache, I tried:
cat /var/log/audit/audit.log | grep passenger | audit2allow -M
passenger semodule -i passenger.pp
but still nothing.
In your case, you should switch SELinux into Permissive mode at first, then try to capture the audit log from starting Apache to run your application.1
Once you got the home page of your application, you can build your custom policy with the logs.
Switch SELinux into Permissive mode and clean audit.log
]# setenforce 0
]# rm /var/log/audit/audit.log
]# service auditd restart
Restart Apache
]# service httpd restart
Try to open your application with a web browser
It might give more information about what is happenning when you application is running.
Make a custom policy module to allow these actions
]# mkdir work
]# cd work
]# grep httpd /var/log/audit/audit.log | audit2allow -M passenger
]# ls
passenger.pp passenger.te
Load postgrey policy module using the 'semodule' command into the current SELinux policy:
]# semodule -i passenger.pp
]# setenforce 1
Restart Apache
]# service httpd restart
References:
http://wiki.centos.org/HowTos/SELinux#head-faa96b3fdd922004cdb988c1989e56191c257c01
I ran into a similar error, with a startup error about being unable to create a directory that did not exist. (logs, not tmp, but same sort of thing) I, too, battled with it for an hour and couldn't make sense of it. I created/deleted/chmod the directory many ways without success.
The fix for me was to change the parameters to passenger-start. Initially, my Docker container started passenger with:
exec bundle exec passenger start --auto --disable-security-update-check --min-instances 20 --max-pool-size 20 --max-request-queue-size 500
I removed all parameters, leaving just this:
exec bundle exec passenger start
At this point, passenger could create the log folder and file, and all was well. I could have restored the params at this point, but we decided they were not needed for the development environment so left them out moving ahead.
In hindsight, I have a hunch that I deleted the log directory while a file in it was still open, and the file system persisted that condition in some way. But that's just a hunch. Perhaps simply rebooting my Mac would have fixed it...