Docker Compose: Expose not working - docker

docker-ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
83b1503d2e7c app_nginx "nginx -g 'daemon ..." 2 hours ago Up 2 hours 0.0.0.0:80->80/tcp app_nginx_1
c9dd2231e554 app_web "/home/start.sh" 2 hours ago Up 2 hours 8000/tcp app_web_1
baad0fb1fabf app_gremlin "/start.sh" 2 hours ago Up 2 hours 8182/tcp app_gremlin_1
b663a5f026bc postgres:9.5.1 "docker-entrypoint..." 25 hours ago Up 2 hours 5432/tcp app_db_1
They all work fine:
app_nginx connects well with app_web
app_web connects well with postgres
No working file:
app_web is not able to connect with app_gremlin
docker-compose.yaml
version: '3'
services:
db:
image: postgres:9.5.12
web:
build: .
expose:
- "8000"
depends_on:
- gremlin
command: /home/start.sh
nginx:
build: ./nginx
links:
- web
ports:
- "80:80"
command: nginx -g 'daemon off;'
gremlin:
build: ./gremlin
expose:
- "8182"
command: /start.sh
Errors:
Basically I am not able to connect to gremlin container from my app_web container.
All below have been executed inside web_app container
curl:
root#49a8f08a7b82:/# curl 0.0.0.0:8182
curl: (7) Failed to connect to 0.0.0.0 port 8182: Connection refused
netstat
root#49a8f08a7b82:/# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.11:42681 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN
udp 0 0 127.0.0.11:54232 0.0.0.0:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
nmap
root#49a8f08a7b82:/# nmap -p 8182 0.0.0.0
Starting Nmap 7.60 ( https://nmap.org ) at 2018-06-22 09:28 UTC
Nmap scan report for 0.0.0.0
Host is up.
PORT STATE SERVICE
8182/tcp filtered vmware-fdm
Nmap done: 1 IP address (1 host up) scanned in 2.19 seconds
nslookup
root#88626de0c056:/# nslookup app_gremlin_1
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: app_gremlin_1
Address: 172.19.0.3
Experimenting:
For Gremlin container I did,
ports:
- "8182:8182"
Then from Host I can connect to gremlin container BUT still no connection between web and gremlin container
I am working on creating a re-creating sample Docker file (minimal stuff to recreate the issue) meanwhile anyone has any idea what the issue might be?

curl 0.0.0.0:8182
The 0.0.0.0 address is a wild card that tells an app to listen on all network interfaces, you do not connect to this interface as a client. For container to container communication, you need:
containers on the same user generated network (compose does this for you by default)
connect to the name of the service (or container name)
connect to the port inside the other container, not the published port.
In your case, the command should be:
curl http://gremlin:8182
Networking is namespaced in apps running inside containers, so each container gets it's open loopback interface and ip address on a bridge network. So moving an app into containers means you need to listen on 0.0.0.0 and connect to the bridge ip using DNS.
You should also remove links and depends_on from your Dockerfile, they don't apply in version 3. Links have long since been deprecated in favor of shared networks. And depends_on doesn't work in swarm mode along with probably not doing what you wanted since it never checked for the target app to be running, only the start of that container to have been kicked off.
One last note, expose doesn't affect the ability to communicate between containers on common networks or publish ports on the host. Expose simply sets meta data on the image that is documentation between the person creating the image and the person running the image. Applications are not required to use that value, but it's a good habit to make your app default to that value for the benefit of downstream users. Because of its role, unless you have another app checking for the exposed port list, like a self updating reverse proxy, there's no need to expose the port in the compose file unless you're giving the compose file to another person and they need the documentation.

There is no link configured in the docker-compose.yaml between web and gremlin. Try to use the following:
version: '3'
services:
db:
image: postgres:9.5.12
web:
links:
- gremlin
build: .
expose:
- "8000"
depends_on:
- gremlin
command: /home/start.sh
nginx:
build: ./nginx
links:
- web
ports:
- "80:80"
command: nginx -g 'daemon off;'
gremlin:
build: ./gremlin
expose:
- "8182"
command: /start.sh

Related

docker-compose docker containers connection refused

I have a container running a service on port 5001, and I am trying to access that service from another container. Whenever I try this I get a connection refused error. I have my docker-compose.yaml file configured like below.
version: "3"
services:
server:
build:
context: ./server
ports:
- 5001:5001
networks:
- local
client:
build:
context: ./client/wanderingreader
ports:
- 3000:3000
depends_on:
- server
networks:
- local
networks:
local:
driver: bridge
I am able to ping the server from the client using ping server, but nmap shows all ports are closed.
/wanderingreader # nmap server
Starting Nmap 7.92 ( https://nmap.org ) at 2022-06-26 17:54 UTC
Nmap scan report for server (172.22.0.2)
Host is up (0.0000060s latency).
rDNS record for 172.22.0.2: wanderingreader-server-1.wanderingreader_local
All 1000 scanned ports on server (172.22.0.2) are in ignored states.
Not shown: 1000 closed tcp ports (reset)
MAC Address: 02:42:AC:16:00:02 (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 0.32 seconds
This is the output of a docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3b0c46283abb wanderingreader_client "docker-entrypoint.s…" 5 minutes ago Up 5 minutes 0.0.0.0:3000->3000/tcp, 5001/tcp wanderingreader-client-1
8d96da5e2629 wanderingreader_server "/bin/sh -c 'python …" 5 minutes ago Up 5 minutes 0.0.0.0:5001->5001/tcp wanderingreader-server-1
I have tried other solutions, but I can't seem to get it to work properly. Any help is greatly appreciated.

How to expose redis-server port started using "webdis docker image" to host machine

I want to monitor redis running in webdis docker container.
I use telegraf which collects redis stats but, telegraf is installed on host machine and it cannot connect to redis as it is running inside docker on 6379 port.
I tried to map docker port 6379 on which redis is running inside docker with hosts 6379 port so telegraf can listen to redis metrices, but telegraf cannot listen as connection breaks.
when I use telnet on host, I get connection closed by foreign host error.
telnet 127.0.0.1 6379
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
Also, I am able to connect to webdis port on host machine, which is running on port 7379 inside wedis container.
To start webdis I am using following command : "docker run -d -p 8080:7379 -p 6379:6379 webdis"
Further to debug, I checked that redis inside webdis container is running on interface 127.0.0.1:6379
I checked that it should be running on 0.0.0.0:6379 in-order for port mapping to work properly.
How can I run redis inside webdis image on 0.0.0.0:6379?
Is there any other way I can monitor redis server running inside webdis container?
I tried to start redis-server inside webdis container by binding it to 0.0.0.0 using redis.conf file, but it still binds with 127.0.0.1
To which docker image are you refering. Is it this one? https://github.com/anapsix/docker-webdis/
If yes, when checking the Dockerfile, it does not include redis itself but in docker-compose.yaml there is a redis service include. This one does not expose ports which you need to connect to redis from outside of the container.
You need to change redis service to the following:
...
redis:
image: 'anapsix/redis:latest'
ports:
- '6379:6379'
I have this problem recently ago and I solve it.
webdis.Dockerfile
FROM nicolas/webdis:0.1.19
EXPOSE 6379
EXPOSE 7379
RUN sed -i "s/127.0.0.1/0.0.0.0/g" /etc/redis.conf
docker-compose.yaml
version: "3.8"
services:
webdis:
build:
context: .
dockerfile: webdis.Dockerfile
image: webdis_with_redis_expose
container_name: webdis
restart: unless-stopped
ports:
- "6379:6379"
- "7379:7379"
then execute docker-compose up

Can't enable ssl by docker-letsencrypt-nginx-proxy-companion

I want to enable ssl by docker-letsencrypt-nginx-proxy-companion.
This is the docker-compose.yml
version: "3.3"
services:
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- certs:/etc/nginx/certs:ro
- vhostd:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: always
db:
# ---
wordpress:
# ---
environment:
# ---
VIRTUAL_HOST: blog.ironsand.net
LETSENCRYPT_HOST: blog.ironsand.net
LETSENCRYPT_EMAIL: mymail#example.com
restart: always
letsencrypt-nginx-proxy-companion:
container_name: letsencrypt
image: jrcs/letsencrypt-nginx-proxy-companion
volumes:
- certs:/etc/nginx/certs
- vhostd:/etc/nginx/vhost.d
- html:/usr/share/nginx/html
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
NGINX_PROXY_CONTAINER: nginx-proxy
restart: always
networks:
default:
external:
name: nginx-proxy
volumes:
certs:
vhostd:
html:
docker logs letsencrypt shows that a certificate exists already.
/etc/nginx/certs/blog.ironsand.net /app
Creating/renewal blog.ironsand.net certificates... (blog.ironsand.net)
2020-04-09 00:03:23,711:INFO:simp_le:1581: Certificates already exist and renewal is not necessary, exiting with status code 1.
/app
But ACME challenge returns nothing. (failure?)
$ docker exec letsencrypt bash -c 'echo "Hello world!" > /usr/share/nginx/html/.well-known/acme-challenge/hello-world'
$
The port 443 is listning, but the port is closed from outside.
// in remote server
$ sudo lsof -i:443
[sudo] password for ubuntu:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
docker-pr 10910 root 4u IPv6 633694 0t0 TCP *:https (LISTEN)
// from local pc
❯ nmap -p 443 blog.ironsand.net
Starting Nmap 7.80 ( https://nmap.org ) at 2020-04-09 09:44 JST
Nmap scan report for blog.ironsand.net (153.127.40.107)
Host is up (0.035s latency).
rDNS record for 153.127.40.107: ik1-418-41103.vs.sakura.ne.jp
PORT STATE SERVICE
443/tcp closed https
Nmap done: 1 IP address (1 host up) scanned in 0.21 seconds
I'm using packet filtering, but it's open for 80 and 443, and I'm not using firewall.
How can I investigate more where the problem exists?
I can't solve your problem directly, but I can wrote some hints, so can solve your problem.
Your command return nothing.
bash -c 'echo "Hello world!" > /usr/share/nginx/html/.well-known/acme-challenge/hello-world'
This comand only writes "Hello world!" to the location and normally return nothing. See https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Redirections
Look inside of certs-folder.
Have a look into the certs folder and maybe clean them up. Check that the folder was mounted corretly in your nginx-container. Take a bash into the container and check the ssl-folder.
Check that the firewall nothing breaks up
From outside is no connection possibly? What is from the inside? Login on your docker-host and check the connection from there (maybe openssl and curl are your friends),
Don't use SSL inside container.
I often see problems when sombody tries to use ssl with ACME-images and "wild mounting and shared volumes". But I heard never about problems, when the same people using a normal reverse proxy. I explain a good setup bellow.
So just remove the whole letscrypt-code from your container and close the 443 port of your container.
(Additionally you can switch to a non-root-image and expose only ports which doesn't need root-privileges.)
Then install nginx on your host and setup a reverse proxy (something like proxy_pass 127.0.0.1:8080). And now install certbot and start it. It helps you and is straight-forward.
The certbot can also maintain your certificates.

docker compose up nginx reverse proxy not adding containers to docker0 bridge

After I carry out docker-compose up, it starts the containers.
when I do docker ps I get the below, which tells me that the containers are running. However when I do docker network inspect bridge the result shows me that there are no containers part of the docker0 bridge.
When I then carry out docker run meanchat_myserver it actually does show up on docker0 and I am also getting the data that the server is running on port 3000.
Which I don't get by using docker-compose.
What am I doing wrong here?
I am reading that when I use docker0 I can only refer to IP's to connect to other containers and not the name. Can I assume the ip's don't change on the containers and that this works without issue on deploying the app in production?
02cf08b1c3da d57f06ba9c68 "npm start" 33 minutes ago Up 33 minutes 4200/tcp meanchat_client_1
e257063c9e21 meanchat_myserver "npm start" 33 minutes ago Up 33 minutes 3000/tcp meanchat_myserver_1
02441c2e43f5 e114a298eabd "npm start" About an ago Up 33 minutes 0.0.0.0:80->80/tcp meanchat_nginx_1
88d9841d2553 mongo "docker-entrypoint..." 3 hours ago Up 3 hours 27017/tcp meanchat_mongo_1
compose
version: '3'
services:
# Build the container using the client Dockerfile
client:
build: ./
# This line maps the contents of the client folder into the container.
volumes:
- ./:/usr/src/app
myserver:
build: ./express-server
volumes:
- ./:/usr/src/app
depends_on:
- mongo
nginx:
build: ./nginx
# Map Nginx port 80 to the local machine's port 80
ports:
- "80:80"
# Link the client container so that Nginx will have access to it
mongo:
environment:
- AUTH=yes
- MONGO_INITDB_ROOT_USERNAME=superAdmin
- MONGO_INITDB_ROOT_PASSWORD=admin123
- MONGO_INITDB_DATABASE=d0c4ae452a5c
image: mongo
volumes:
- /var/mongodata/data:/data/db
By default Compose sets up a single network for your app.
For more detail, refer this link.
This means containers with compose won't be located in default bridge network by default.
You can check which network the containers with compose are using with the command.
docker inspect $container_name -f "{{.NetworkSettings.Networks}}"
However, If you want containers to be in default bridge network, you can use network_mode.
services:
service_name:
# other options....
network_mode: bridge

Keepalived/Haproxy docker-compose connection refused

Docker Version: Version 17.03.0-ce-mac2 (15654)
OS: macOS Sierra
I am trying to setup an HA environment using docker-compose. A quick overview of the what the topology might look like is that I will have at least two instances of keepalived and haproxy running, the haproxy will be in front of multiple servers. However, in this posting I make reference to only one instance of keepalived, haproxy, and server for simplification.
The problem that I have right now is that I am unable to direct traffic to the virtual IP address that I assign to keepalived. For testing purposes in my docker compose file I have a client that tries to communicate using the VIP, and it results in a connection refused error.
dial tcp 192.168.99.120:80: getsockopt: connection refused
However, if I reach out directly to haproxy there is not a connection issue. Furthermore, I can communicate directly to the haproxy from host but not to keepalived.
I feel like this has something to do with how networks work in docker but I am pretty new to using docker and have not been able to track down the issue. Any help would be much appreciated.
My configuration files are all included below.
docker-compose.yml:
version: '2'
services:
keepalived1:
image: neoassist/docker-keepalived:latest
container_name: keepalived1
volumes:
- "./keepalived.conf:/etc/keepalived/keepalived.conf"
environment:
- VIRTUAL_IP=192.168.99.120
- VIRTUAL_MASK=24
- VRID=1
- CHECK_IP=any
- CHECK_PORT=80
- INTERFACE=eth0
entrypoint: sh -c 'sleep 4;/usr/bin/keepalived.sh'
network_mode: "host"
cap_drop:
- NET_ADMIN
privileged: true
haproxy1:
image: haproxy:latest
container_name: haproxy1
ports:
- 7054:7054
volumes:
- "./haproxy1.cfg:/usr/local/etc/haproxy/haproxy.cfg"
environment:
- EXPOSE=7054
links:
- fabric-ca-server1:fabric-ca-server1
fabric-ca-server1:
image: hyperledger/fabric-ca
container_name: fabric-ca-server1
ports:
- 7051:7054
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
volumes:
- "./fabric-ca-server:/etc/hyperledger/fabric-ca-server"
command: sh -c "fabric-ca-server start -d -b admin:adminpw"
admin-client:
image: hyperledger/fabric-ca
container_name: admin-client
network_mode: "host"
command: sh -c "sleep 14;fabric-ca-client enroll -d -u http://admin:adminpw#192.168.99.120"
haproxy.cfg
global
maxconn 4096
defaults
mode http
maxconn 2000
timeout connect 5000
timeout client 50000
timeout server 50000
frontend server
bind *:7054
mode tcp
default_backend server_cluster
backend server_cluster
balance source
mode tcp
option tcpka
server server1 fabric-ca-server1:7054
keepalived.conf
vrrp_script haproxy {
script "pidof haproxy"
interval 2
weight 2
}
vrrp_instance haproxy_1 {
virtual_router_id 1
advert_int 1
interface eth0
nopreempt
state BACKUP
virtual_ipaddress {
192.168.99.120/24 dev eth0
}
track_script {
haproxy
}
}
ifconfig from my mac has:
vboxnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 0a:00:27:00:00:00
inet 192.168.99.1 netmask 0xffffff00 broadcast 192.168.99.255
I don't think this will ever work with Docker for Mac because it actually uses a VM under the covers to run your Docker containers. This should work on a system with native Docker support if you try using host networking rather than bridge networking.
My suggestion would be to look at either Docker swarm mode (not the standalone Docker swarm) or Kubernetes which both provide mechanisms to scale services and provide load balancing across them via a single address

Resources