Connection reset by peer when hitting Docker container - docker

I'm having a problem, where I can't send network requests to a Docker container I've created. I've exposed the correct ports, so I'm not sure what other issues could be at fault here.
I have a server running in container alice at localhost:10009:
$ docker exec -it alice bash
bash-4.4# curl localhost:10009
curl: (52) Empty reply from server
Port 10009 is exposed from my container:
$ docker port alice
10009/tcp -> 0.0.0.0:10009
When doing the same curl from my host machine I get a different message:
$ curl localhost:10009
curl: (56) Recv failure: Connection reset by peer

I would check to see if the server application is configured to only listen to requests coming from its "localhost", this check depends on the type of server that you're using which is not mentioned.
an easy check is to start your container as follows:
docker run --network host -d yourimagename
You don't need to worry about port mapping since you're using the host network
then try to curl, if that works, then you'll just need to review your server listening IP setting.
curl localhost:10009

I think there are some problems with #Bouzid Zitouni's answer, according to Docker official documentation:
this is the same level of isolation as if the nginx process were running directly on the Docker host and not in a container
However, if you use the --network host you will not have isolated networking in the container, and the host networking driver only works on Linux hosts.
The problem of Connection refused/reset happens because your Server is listening on 127.0.0.1 inside the container and the port forwarding is going to external IP of the container (e.g. 172.17.0.2).
Solution
In your case you need to run a new container making your server to listen on all interfaces. Example using python http.server :
docker run -p 8000:8000 -it python:3.7-slim python3 -m http.server --bind 0.0.0.0
Note
The option --bind 0.0.0.0 it's specific option of http.server. Probally your server has other ways to specify this.
References:
https://pythonspeed.com/articles/docker-connection-refused/
https://docs.docker.com/network/network-tutorial-host/

I would like to expand on #Bouzid Zitouni's answer. It seems there is indeed an issue with the address(es) the server binds to.
Connection reset by peer usually indicates that one has defined a port mapping for the container that does not point to a listening server. Here is an example to illustrate this:
docker run -p 10009:10009 -it ubuntu bash
Install nmap in container:
apt-get update && apt install -y nmap
Run ncat (localhost only)
# ncat -v --listen localhost 10009
...
Ncat: Listening on 127.0.0.1:10009
Run curl on host:
# curl localhost:10009
curl: (56) Recv failure: Connection reset by peer
You actually get the same result even if you don't have any server process at all.
Run ncat (all IPs)
# ncat -v --listen 10009
...
Ncat: Listening on :::10009
Ncat: Listening on 0.0.0.0:10009
Curl on host connects successfully. Hope that helps.

I faced the same error with the docker container running locally on my machine/laptop.
I ran multiple containers and was using the same port number say 8080 for each container run.
After killing all docker process and restarting docker i am now able to connect to the container on the mentioned port 8080 in my case.
$ sudo service docker stop
Warning: Stopping docker.service, but it can still be activated by:
docker.socket
$ sudo service docker start

Related

Docker: Failed to connect to localhost port 8080 after 8 ms: Connection refused

I pulled a python docker image using:
docker pull python:3.8
Then I started a simple http server with:
docker run -ti -p 8080:8080 900972ffeecd python -m http.server 8080
If I docker exec into the container and run curl localhost:8080 I get the response I expect, but if I run the same curl command from the host machine it fails.
Shouldn't using -p 8080:8080 be enough to expose the port?
Potentially relevant info: I'm using colima on MacOS.
I hate to admit, but restarting my machine sent the issue away.

Cannot connect to Podman container outside of host machine

I have been trying to create a Grafana container on my Tumbleweed server using Podman. I used NetworkManager instead of wickedd on this server. I of course published port 3000 when I ran the container:
sudo podman run -d -p 3000:3000 --name=grafana_hub -v grafana-storage:/var/lib/grafana grafana/grafana
and whitelisted the port in firewalld:
sudo firewall-cmd --zone=public --add-port=3000/tcp
but no dice.
I can access the web server with curl http://localhost:3000 on the host or even curl http://<host LAN IP>:3000 from the host, however if I run the latter on another machine on the LAN, it times out. I am at a loss here. Is there something different about Podman networking from Docker I am missing?

Why is my docker registry refusing connections from VMs in my local network but not the host?

For context - I am attempting to deploy OKD in an air-gapped environment, which requires mirroring an image registry. This private, secured registry is then pulled from by other machines in the network during the installation process.
To describe the environment - the host machine where the registry container is running is running Centos 7.6. The other machines are all VMs running Fedora coreOS in using libvirt. The VMs and the host are connected using a virtual network created using libvirt which includes DHCP settings (dnsmasq) for the VMs to give them static IPs. The host machine also hosts the DNS server, which, as far as I can tell is configured properly, as I can ping every machine from every other machine using its fully qualified domain name and access specific ports (such as the port the apache server listens on). Podman is used instead of Docker for container management for OKD, but as far as I can tell the commands are exactly the same.
I have the registry running in the air-gapped environment using the following command:
sudo podman run --name mirror-registry -p 5000:5000 -v /opt/registry/data:/var/lib/registry:z \
-v /opt/registry/auth:/auth:z -v /opt/registry/certs:/certs:z -e REGISTRY_AUTH=htpasswd \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.pem -e REGISTRY_HTTP_TLS_KEY=/certs/registry-key.pem \
-d docker.io/library/registry:latest
It is accessible using curl -u username:password https://host-machine.example.local:5000/v2/_catalog which returns {"repositories":[]}. I believe this confirms that my TLS and authorization configurations are correct. However, if I transfer the ca.pem file (used to sign the SSL certificates the registry uses) over to one of the VM's on the virtual network, and attempt to use the same curl command, I get an error:
connect to 192.168.x.x port 5000 failed: Connection refused
Failed to connect to host-machine.example.local port 5000: Connection refused
Closing connection 0
This is quite strange to me, as I've been able to use this method to communicate with the registry from the VMs in the past, and I'm not sure what has changed.
After some further digging, it seems like there is some sort of issue with the port itself, but I can't be sure where the issue is stemming from. For example, If I run sudo netstat -tulpn | grep LISTEN on the host, I receive a line indicating that podman (conmon) is listening on the correct port:
tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 48337/conmon
but if I test whether the port is accessible from the VM, (nc -zvw5 192.168.x.x 5000) I get a similar error: Ncat: Connection refused. If I use the same test on any of the other listening ports on the host, it indicates successful connections to those ports.
Please note, I have completely disabled firewalld, so as far as I know, all ports are open.
I'm not sure if the issue is with my DNS settings, or the virtual network, or with the registry itself and I'm not quite sure how to further diagnose the issue. Any insight would be much appreciated.

Trying to connect to mariadb running in docker

I have a database running in a docker container. It does not publish mariadb's port 3306.
Now I want to remotely log in to the docker host, connect to the container and access the database from my laptop
laptop ---> dockerhost ---> container
in order to access the database with gUI tools like DbVisualizer.
The idea is to open a connection with socat, but I'm stuck. Basically something like:
socat TCP4-LISTEN:3306 EXEC:'ssh dockerhost sudo docker exec container "socat - TCP:localhost:3306"'
The last attempt failed with "Unexpected exception encountered during query." in DbVisualizer and "2019/09/10 12:19:54 socat[17462] E write(6, 0x7f9985803c00, 114): Broken pipe" in the shell.
The command was (broken for readability):
socat TCP4-LISTEN:3306,forever,reuseaddr,fork \
exec:'
ssh dockerhost \
sudo docker exec container "
socat STDIO TCP:localhost:3306,forever,reuseaddr,fork
"
'
I hope someone can pinpoint what I do wrong or tell me how I can achieve my goal.
Delete and restart your container with a docker run -p or Docker Compose ports: option that will make it visible outside of Docker space. (You're storing the actual database data in a volume, right? On restart it will keep using the data from the volume.)
If you're comfortable with the container being accessed directly from off-host, then you can use an ordinary port invocation -p 3306:3306 and then reach it using dockerhost as the host name and the first port number as the port number.
If you still want to require the ssh tunnel, you can cause the port to be bound to the Docker host's localhost interface, and then use ssh port forwarding.
dockerhost$ docker run -p 127.0.0.1:3306:3306 -v ... mysql
laptop$ ssh -L 3306:localhost:3306 dockerhost
laptop$ mysql -h 127.0.0.1
docker exec in many ways is equivalent to ssh root#... and is not the normal way to interact with a network-accessible service.

Connection refused when try to connect http server in docker daemon container

I am using boot2docker. I run one image at the daemon mode which starts grunt server at port 3000. This is the command I used to start it up.
That image has already exposed port 3000
docker run -d -P --name dummy image_name grunt server
docker ps
3af4ba19c539 image_name:latest "grunt server" 54 minutes ago Up 54 minutes 0.0.0.0:45000->3000/tcp dummy
and then run into the same container to "curl" the web server.
docker exec -it 3af4ba19c539 /bin/bash
curl localhost:3000
It gets the html.
However, when I try to connect it in my mac pc. It said "Connection refused."
curl $(boot2docker ip):45000
//curl: (7) Failed connect to 192.168.59.103:45000; Connection refused
I try to solve this problem by using VBoxManage, but it is not working either
VBoxManage modifyvm "boot2docker-vm" --natpf1 "tcp-port45000,tcp,,45000,,45000"
May I know how to solve this problem. Many thanks in advance
I need to see the source of your application to be sure, but I'm willing to bet you've bound to the local loopback interface (127.0.0.1 or localhost) in your application. If you instead bind to 0.0.0.0 to listen to all interfaces, you should find it is accessible from the outside.

Resources