OS: Ubuntu 20.04 LTS
Docker version 20.10.7, build 20.10.7-0ubuntu1~20.04.2
Tried this (minimal way to reproduce), the previous day it works normally:
docker run --name nginx1 -p 8089:80 -d nginx:alpine
Works normally:
docker ps
CONTAINER ID IMAGE COMMAND CREATED
74faabf44b8e nginx:alpine "/docker-entrypoint.…" 5 minutes ago
STATUS PORTS NAMES
Up 5 minutes 0.0.0.0:8089->80/tcp, :::8089->80/tcp nginx1
curl inside docker works:
docker exec -it 74faabf44b8e sh
/ # curl localhost
<!DOCTYPE html>
<html>
<head>
....
But when accessed from the host:
netstat -antpel | grep 8089
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN 0 82748695 -
tcp6 0 0 :::8089 :::* LISTEN 0 82751546 -
curl -v localhost:8089
* Trying 127.0.0.1:8089...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8089 (#0)
> GET / HTTP/1.1
> Host: localhost:8089
> User-Agent: curl/7.68.0
> Accept: */*
>
--> stuck without a response, after a while:
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
What's the possible cause of this?
This also doesn't work for another container (PostgreSQL), seems the docker-proxy does not forward back the response from docker?
already tried:
reinstall docker.io
delete all containers and re-create
docker network prune
Ubuntu version: 22.04 LTS
Docker version: 20.10.17, build 100c701
Installation procedure: https://docs.docker.com/engine/install/ubuntu/
I am encountering the same problem as you on more recent versions of Ubuntu & Docker: Everything seems to be working fine except that the host is not able to reach the container on port, as defined by the -p <docker-host-port>:<docker-container-port> option.
curl http://localhost:<port> works from within the container, but not from outside despite port binding ...
I tried to restart docker daemon, restart machine, remove container, recreate container, remove image, recreate image, uninstall completely docker, reinstall docker from scratch. I just don't understand why Docker is acting funky like this on this specific instance (Ubuntu 22.04 + Docker).
Related
I've run into an issue that seems similar too this one; https://forums.docker.com/t/cant-access-service-in-swarm/63876. My setup is a little bit different though and I haven't found a solution to my problem yet.
The minimal, reproducible example
Build a swarm cluster between atleast 3 Ubuntu 20.04 docker swarm managers.
Deploy a service docker service create --name test_web --replicas 3 --publish published=8080,target=80 nginxdemos/hello
Check that the containers and services were created properly and observe the failure of connecting to that service:
demi-ubu01:~/stacks$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d4a12a3c5448 nginxdemos/hello:latest "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp test_web.2.yul33wdycarig3qoxnehgrjrz
demi-ubu01:~/stacks$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
0yqd7gvggwuh test_web replicated 3/3 nginxdemos/hello:latest *:8080->80/tcp
# External test:
demi-ubu01:~/stacks$ curl -I 10.100.4.5:8080
curl: (7) Failed to connect to 10.100.4.5 port 8080: Connection refused
# Inside container to published service port:
demi-ubu01:~/stacks$ docker exec -it d4a12a3c5448 wget http://test_web:8080
Connecting to test_web:8080 (10.0.4.2:8080)
wget: can't connect to remote host (10.0.4.2): Host is unreachable
# Inside container to apps exposed port:
demi-ubu01:~/stacks$ docker exec -it d4a12a3c5448 wget http://localhost:80
Connecting to localhost:80 (127.0.0.1:80)
index.html 100% |****************************| 7217 0:00:00 ETA
The expected result of the first curl command should be a Status 200 Ok.
The detailed report
My setup is 4 nodes in total. They are identical Ubuntu 20.04 KVM virtual machines all on the same network. There are no firewalls between them. I have 3 Managers and 1 Worker (which i've only added as a step during troubleshooting).
:~/stacks$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
kcm5v64psntjxngnqkfdj1jzh * demi-ubu01 Ready Active Reachable 20.10.1
uo3rljg6ax5qkjm898pyym9t1 demi-ubu02 Ready Active Leader 20.10.1
pysnl8sohdp4fv67gui156z4k demi-ubu03 Ready Active Reachable 20.10.1
rp2otsqpnxkgbmxbpkv21yjs6 demi-ubu04 Ready Active 20.10.1
I can run a container normally and reach it on the local host fine.
demi-ubu01:~/stacks$ docker run -p 8080:80 -d nginxdemos/hello
de4d0a937710acb1d6d8ae3b7eb9175860b6614dfd9ce92bc972efe619ae095f
demi-ubu01:~/stacks$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
de4d0a937710 nginxdemos/hello "nginx -g 'daemon of…" 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp pedantic_wiles
demi-ubu01:~/stacks$ curl -I 10.100.4.5:8080
HTTP/1.1 200 OK
Server: nginx/1.13.8
Date: Sat, 19 Dec 2020 17:59:23 GMT
Content-Type: text/html
Connection: keep-alive
Expires: Sat, 19 Dec 2020 17:59:22 GMT
Cache-Control: no-cache
However the same app deployed as a service using the following compose file:
demi-ubu01:~/stacks$ cat test.yml
version: "3.6"
services:
web:
image: nginxdemos/hello:latest
deploy:
replicas: 3
resources:
limits:
cpus: "0.1"
memory: 50M
restart_policy:
condition: on-failure
ports:
- target: 80
published: 8080
protocol: tcp
mode: ingress
networks:
- webnet
networks:
webnet:
driver: overlay
It does not become reachable from any of the hosts at all:
demi-ubu01:~/stacks$ docker stack deploy -c test.yml test
Creating network test_webnet
Creating service test_web
demi-ubu01:~/stacks$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
05030ef897a1 nginxdemos/hello:latest "nginx -g 'daemon of…" 10 seconds ago Up 7 seconds 80/tcp test_web.1.kobrpkp68f2qbs4jhd6o8aebg
# Trying on all of the hosts in the cluster. No firewalls here.
demi-ubu01:~/stacks$ curl -I 10.100.4.5:8080
curl: (7) Failed to connect to 10.100.4.5 port 8080: Connection refused
demi-ubu01:~/stacks$ curl -I 10.100.4.9:8080
curl: (7) Failed to connect to 10.100.4.9 port 8080: Connection refused
demi-ubu01:~/stacks$ curl -I 10.100.4.10:8080
curl: (7) Failed to connect to 10.100.4.10 port 8080: Connection refused
demi-ubu01:~/stacks$ curl -I 10.100.4.11:8080
curl: (7) Failed to connect to 10.100.4.11 port 8080: Connection refused
demi-ubu01:~/stacks$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
elvfm7o4v4zo test_web replicated 3/3 nginxdemos/hello:latest *:8080->80/tcp
I also don't see any port bindings being made on those hosts at all, so it doesn't look like any ports are being published.
INeed2Poo#demi-ubu01:~/stacks$ docker service inspect test_web
[
## https://pastebin.com/WqqyDnVS ##
]
demi-ubu01:~/stacks$ netstat -na | grep LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:49152 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:24007 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
demi-ubu01:~/stacks$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6e5f7e7cebc3 bridge bridge local
7a1155f87a62 docker_gwbridge bridge local
ab32da8ac1ec host host local
46id8wzw4ayf ingress overlay swarm
a24a40ef78f4 none null local
d9l7msysdx8m test_webnet overlay swarm
INeed2Poo#demi-ubu01:~/stacks$ docker network inspect 46id8wzw4ayf
[
https://pastebin.com/JPA0ZBjE
]
I also can't reach the service while exec'ed into a container for that service. Execing into a container, I'm able to hit the LOCAL app port, however I cannot hit the service by name. The container CAN resolve the service name.
## Testing the app's service from the local container fails:
demi-ubu01:~/stacks$ docker exec -it 05030ef897a1 wget http://test_web:8080
Connecting to test_web:8080 (10.0.4.2:8080)
wget: can't connect to remote host (10.0.4.2): Host is unreachable
## Testing the app's local port from the local container is sucessful:
demi-ubu01:~/stacks$ docker exec -it 05030ef897a1 wget http://localhost:80
Connecting to localhost:80 (127.0.0.1:80)
index.html 100% |****************************| 7217 0:00:00 ETA
demi-ubu01:~/stacks$ docker --version
Docker version 20.10.1, build 831ebea
I've changed the default-addr-pool for the swarm cluster from the original 10.0.0.0/8 network:
demi-ubu01:~$ docker info --format '{{json .Swarm.Cluster.DefaultAddrPool}}'
["10.135.0.0/16"]
I've gone and made sure that I'm not using any overlapping networks that might be causing this and have gone so far as to completely redeploy the cluster. I've just about exhausted all of my troubleshooting idea's. Any Idea's?
Edit: Update: I redeployed using Ubuntu 18.04 as my base image, and the same exact setup on that (deployed using ansible) seems to work fine... So this is an issue with the current version of Docker on Ubuntu 20.04.
Let me add my response from the docker forum here as well, as it is high likely the solution:
Is it safe to assume that 10.100.4.5 is one of your nodes ip?
The default address pool is 10.0.0.0/8, see: docker info --format '{{json .Swarm.Cluster.DefaultAddrPool}}'
If this is the case, you might find this blog post helpful - you can safely ignore that it refers to Docker EE, the problem and solution is valid for Docker CE as well. You need to alter default-addr-pool either when initiating the swarm or by modifying each node’s /etc/docker/daemon.json configuration file (and restart the daemon then).
Following the tutorial on https://docs.docker.com/get-started/part2/.
I start my docker container with docker run -p 4000:80 friendlyhello
and see
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:8088/ (Press CTRL+C to quit)
But it's inaccessible from the expected path of localhost:4000.
$ curl http://localhost:4000/
curl: (7) Failed to connect to localhost port 4000: Connection refused
$ curl http://127.0.0.1:4000/
curl: (7) Failed to connect to 127.0.0.1 port 4000: Connection refused
Okay, so maybe it's not on my local host. Getting the container ID I retrieve the IP with
docker inspect --format '{{ .NetworkSettings.IPAddress }}' 7e5bace5f69c
and it returns 172.17.0.2 but no luck! curl continues to give the same responses. I can confirm something is running on 4000....
lsof -i :4000
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 94812 travis 18u IPv6 0x7516cbae76f408b5 0t0 TCP *:terabase (LISTEN)
I'm pulling my hair out on this. I've read through the troubleshooting guide and can confirm
* not on a proxy
* don't use a custom dns
* I'm having issues connecting to docker, not docker connecting to my pip server.
Running the app.py with python app.py the server starts and I'm able to hit it. What am I missing?
Did you accidentally put port=8088 at the bottom of your app.py file? When you are running this the last line of your output is saying that your python app is exposed on port 8088 not 80.
To confirm you can run either modify the app.py file and rebuild the image, or alternatively you could run: docker run -p 4000:8088 friendlyhello which would map your local port 4000 to 8088 in the container.
Try to run it using:
docker run -p 4000:8088 friendlyhello
As you can see from the logs, your app starts on port 8088, but you connect 4000 to 80 where on 80, nothing is actually listening.
getting this error while curl the application ip
curl (56) Recv failure: Connection reset by peer - when hitting docker container
Do a small check by running:
docker run --network host -d <image>
if curl works well with this setting, please make sure that:
You're mapping the host port to the container's port correctly:
docker run -p host_port:container_port <image>
Your service application (running in the container) is running on localhost or 0.0.0.0 and not something like 127.0.0.1
I GOT the same error
umesh#ubuntu:~/projects1$ curl -i localhost:49161
curl: (56) Recv failure: Connection reset by peer
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
in my case it was due wrong port no
|---MY Projects--my working folder
--------|Dockerfile ---port defined 8080
--------|index.js-----port defined 3000
--------|package.json
then i was running ::::
docker run -p 49160:8080 -d umesh1/node-web-app1
so as the application was running in port 3000 in index.js it was not able to connect to the application got the error as u were getting
So TO SOLVE THE PROBLEM
deleted the last container/image that was created my worong port
just change the port no of INDEX.JS
|---MY Projects--my working folder
--------|Dockerfile ---port defined 8080
--------|index.js-----port defined 8080
--------|package.json
then build the new image
docker build -t umesh1/node-web-app1 .
running the image in daemon mode with exposed port
docker run -p 49160:8080 -d umesh1/node-web-app1
THUS MY APPLICATION WAS RUNNING without any error listing on port 49161
I have same when bind to port that is not lissened by any service inside container.
So check -p option
-p 9200:9265
-p <port in container>:<port in host os to be binded to>
EDIT
Turned out to a problem with the image, I tried another one and it works fine
I'm trying to run Pgadmin 4 as server mode using Docker on Debian 9. I have followed the instructions on https://hub.docker.com/r/dpage/pgadmin4/ I start it by the following command
docker run -p 5050:5050 -e "PGADMIN_DEFAULT_EMAIL=myemail#gmail.com" -e "PGADMIN_DEFAULT_PASSWORD=a12345678" -d dpage/pgadmin4
I don't get any errors, and docker ps shows the status as below
root#poweredge:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c4b11e4bceb7 dpage/pgadmin4 "/bin/bash /entry.sh" 12 seconds ago Up 10 seconds 80/tcp, 443/tcp, 0.0.0.0:5050->5050/tcp upbeat_jackson
But when I go to serverip:5050 nothing loads. Any idea what the problem may be here?
On the local machine when I execute curl http://localhost:5050 I get Connection reset by peer if the docker instance is running
root#poweredge:~# curl http://localhost:5050
curl: (56) Recv failure: Connection reset by peer
if I stop the Docker instance, I get
root#poweredge:~# curl http://localhost:5050
curl: (7) Failed to connect to localhost port 5050: Connection refused
PgAdmin 4 docker container has exposed port 80 and 443 by default. You can checck the Dockerfile here https://github.com/postgres/pgadmin4/blob/master/pkg/docker/Dockerfile
So the port mapping parameter in the command has to be updated (-p host_port: container_port)
Below is the updated command to access pgadmin4 via http (port 80)
docker run -p 5050:80 -e "PGADMIN_DEFAULT_EMAIL=myemail#gmail.com" -e "PGADMIN_DEFAULT_PASSWORD=a12345678" -d dpage/pgadmin4
After starting the container you should be able to access it via http://localhost:5050
Are you trying to access it out side your virtual box? If yes, check if you have port forwarding rules of your Virtual machine set correctly:
I'm running my docker container with:
docker run -d sequenceiq/hadoop-docker:2.6.0
The Dockerfile is here.
After it is started on my mac - I'm running docker ps and getting:
6bfa4f2fd3b5 sequenceiq/hadoop-docker:2.6.0 "/etc/bootstrap.sh -d" 4 minutes ago Up 4 minutes 22/tcp, 8030-8033/tcp, 8040/tcp, 8042/tcp, 8088/tcp, 49707/tcp, 50010/tcp, 50020/tcp, 50070/tcp, 50075/tcp, 50090/tcp kind_hawking
Then I'm running
ssh -v localhost -p 22
and I'm getting
OpenSSH_7.4p1, LibreSSL 2.5.0
debug1: Reading configuration data /Users/User/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to localhost [::1] port 22.
debug1: connect to address ::1 port 22: Connection refused
debug1: Connecting to localhost [127.0.0.1] port 22.
debug1: connect to address 127.0.0.1 port 22: Connection refused
ssh: connect to host localhost port 22: Connection refused
Assumptions: I think this is not a duplicate of the other centos sshd questions as this is a different centos version. (For those that are similar - it is doing what the potentially similar question is asking and it is not working).
My question is: How to get my docker centos sshd passwordless server running?
Edit:
#Andrew has been super-helpful in helping me refine my question - so here goes.
Here is my updated Dockerfile
FROM sequenceiq/hadoop-docker:2.6.0
CMD ["/etc/bootstrap.sh", "-d"]
# Hdfs ports
EXPOSE 50010 50020 50070 50075 50090 8020 9000
# Mapred ports
EXPOSE 10020 19888
#Yarn ports
EXPOSE 8030 8031 8032 8033 8040 8042 8088
#Other ports
EXPOSE 49707 2122
EXPOSE 9000
EXPOSE 2022
Now I'm building this with:
sudo docker build -t my-hdfs .
Then I'm running this with:
sudo docker run -d -p my-hdfs
Then I'm checking the processes with:
sudo docker ps
with a result like:
d9c9855cfaf0 my-hdfs "/etc/bootstrap.sh -d" 2 minutes ago
Up 2 minutes 0.0.0.0:32801->22/tcp, 0.0.0.0:32800->2022/tcp,
0.0.0.0:32799->2122/tcp, 0.0.0.0:32798->8020/tcp, 0.0.0.0:32797->8030/tcp,
0.0.0.0:32796->8031/tcp, 0.0.0.0:32795->8032/tcp, 0.0.0.0:32794->8033/tcp,
0.0.0.0:32793->8040/tcp, 0.0.0.0:32792->8042/tcp, 0.0.0.0:32791->8088/tcp,
0.0.0.0:32790->9000/tcp, 0.0.0.0:32789->10020/tcp, 0.0.0.0:32788->19888/tcp,
0.0.0.0:32787->49707/tcp, 0.0.0.0:32786->50010/tcp, 0.0.0.0:32785->50020/tcp,
0.0.0.0:32784->50070/tcp, 0.0.0.0:32783->50075/tcp, 0.0.0.0:32782->50090/tcp
agitated_curran
Then to get the IP address I'm running:
docker inspect --format '{{ .NetworkSettings.IPAddress }}' d9c9855cfaf0
with a result like
172.17.0.3
Then I'm testing it with:
ssh -v 172.17.0.3 -p 32800
This gives a result:
OpenSSH_7.4p1, LibreSSL 2.5.0
debug1: Reading configuration data /Users/User/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 172.17.0.3 [172.17.0.3] port 32800.
debug1: connect to address 172.17.0.3 port 32800: Operation timed out
ssh: connect to host 172.17.0.3 port 32800: Operation timed out
My question is: How to get my docker centos sshd passwordless server running?
You are trying to connect to you local ssh server instead of container. To connect to any port inside container, you need to expose and publish it and possibly map it to another one, especially in case when you want to run multiple similar containers on different ports on the same host. See Expose.
So in your case your command should be
docker run -p 2222:22 -d sequenceiq/hadoop-docker:2.6.0
And ssh command
ssh -v localhost -p 2222
Exposing docker port (as seen in your linked docker file) makes it accessible
to other docker containers, but not to your host machine. To understand difference between exposed and published ports see this question
However, when i tried to connect to port 2222 it haven't worked. Looking at Dockerfile of 2.6.0 version, i've found that it has a bug, where sshd configured to listen on port 2122, but exposed port is 22, as can be seen here. Also, when i'm tried to build a lastest Dockerfile you provided, it failed at step 31, so you might want to inverstigate further.
Edit after question update:
Look at docker ps output you provided, and on Dockerfile. sshd configured to listen on port 2122 (if you haven’t changed that though since we don't have a complete dockerfile of yours), and in output we see
0.0.0.0:32799->2122/tcp
0.0.0.0:32800->2022/tcp
You should connect as ssh -v localhost -p 32799 instead of 32800 since nothing is listening on port 2022 inside container