Why docker-compose do not use port binding as expected? - docker

I'm using this docker-compose config for running a classic nodejs app + mysql + redis:
version: '3'
services:
app:
container_name: app
restart: always
build:
context: ./app/
depends_on:
- db
- redis
- elasticsearch
links:
- db
- redis
- elasticsearch
ports:
- "127.0.0.1:${APP_PORT}:${APP_PORT}"
environment:
- "PORT=${APP_PORT}"
- "MYSQL_PORT=${MYSQL_PORT}"
- "REDIS_PORT=${REDIS_PORT}"
redis:
container_name: redis
image: "redis:3-alpine"
restart: always
ports:
- "127.0.0.1:${REDIS_PORT}:6379"
db:
container_name: db
image: mariadb:10.0.30
restart: always
volumes:
- "${MYSQL_DATA_LOCATION}/data:/var/lib/mysql:delegated"
- "${MYSQL_DATA_LOCATION}/my.cnf:/etc/mysql/my.cnf:ro"
ports:
- "127.0.0.1:${MYSQL_PORT}:3306"
environment:
- "MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}"
- "MYSQL_USER=${MYSQL_USER}"
- "MYSQL_PASSWORD=${MYSQL_PASSWORD}"
- "MYSQL_DATABASE=${MYSQL_DATABASE}"
I'm also using a .env file containing the following values:
APP_PORT=5000
MYSQL_PORT=3306
REDIS_PORT=6379
And I start the application with
docker-compose up -d
Everything runs fine this way. I can change the mysql port without error, but if I change the redis port to something else than the original port, I have the following error in the app, when I use this config:
redis.createClient({host: 'redis', port: process.env.REDIS_PORT });
I get the following message:
Error: Redis connection to redis:5555 failed - connect ECONNREFUSED 172.18.0.3:5555
Whereas the connection between the db and the app works correctly :s
In docker ps I can see that the machine started with the expected config:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9c594c6d98b8 nginx:mainline-alpine "nginx -g 'daemon of…" 33 seconds ago Up 39 seconds 0.0.0.0:2000->80/tcp webproxy
62d1f917d7ff app "npm run docker-debug" 36 seconds ago Up 42 seconds 127.0.0.1:5000->5000/tcp, 127.0.0.1:9222->9222/tcp app
7c982d543a7d mariadb:10.0.30 "docker-entrypoint.s…" 39 seconds ago Up 45 seconds 127.0.0.1:4000->3306/tcp db
9dd13967b37d redis:3 "docker-entrypoint.s…" 39 seconds ago Up 44 seconds 127.0.0.1:5555->6379/tcp redis
I can see here that the db is mapped from 4000 to 3306, which works, but the redis mapping from 5555 to 6379 is ignored. And if I manually change the port in the app connection to port 6379, it does work...
Is there something missing in my configuration ?

the redis is running inside the container on port 6379 period (if you dont change the port in its config). On your host you map port 5555 to redis container 6379. All services in docker-compose are visible to each other and you dont even need to EXPOSE those ports to host. In your app settings if you access the redis by redis://redis:6379 will work or with the exposed port redis://HOST_IP:5555 will work too

Related

How can I load balance an internal service (with many instances) using docker-compose?

I hope you can help me, thanks in advance!
I have a docker-compose file with three services traefik, app and service-x. app can have many instances and I implemented a load balancer / reverse proxy using traefik to it, and it is working fine.
app calls internally to service-x and service-x can also have many instances, but when app calls http://service-x it always do it to the same instance (the first one), so I wonder if there is a way to implement a load balance mechanism, so when I call service-x it is distributed among all the instances?
this is the simplified docker-compose file:
services:
traefik:
image: "traefik"
restart: always
command:
- "--api=true"
...
ports:
- "80:80"
- "443:443"
app:
image: app:latest
labels:
- "traefik.enable=true"
...
ports:
- "3000"
restart: on-failure
links:
- service-x
service-x:
image: service-x:latest
ports:
- "8001"
I wonder if there is a way to implement a load balance mechanism, so when I call service-x it is distributed among all the instances?
That is the way docker-compose behaves by default. If you're not seeing that behavior, I suspect it's your use of the deprecated links parameter. You haven't need that for years now; when you attach containers to the same user-defined network (which docker-compose does by default), they can refer to each other by name using the DNS service maintained automatically by Docker.
Also -- and this isn't directly related to your question -- you don't need ports entries for the services that are only used internally.
For a practical example, take a look at this docker-compose.yaml:
version: "3"
services:
proxy:
image: traefik:latest
command:
- --api.insecure=true
- --providers.docker
- --accesslog=true
- --accesslog.filepath=/dev/stderr
ports:
- "127.0.0.2:80:80"
- "127.0.0.2:443:443"
- "127.0.0.2:8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
app:
image: docker.io/nginx:mainline
volumes:
- "./default.conf:/etc/nginx/conf.d/default.conf"
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`example.com`)"
service-x:
image: docker.io/containous/whoami:latest
In the above, service-x is running this image, which runs a webserver that reports its hostname and some other metadata.
The role of your app service is played by nginx, using this configuration:
server {
listen 80;
server_name localhost;
location / {
proxy_pass "http://service-x";
proxy_set_header Host $http_host;
}
}
With the above configuration, a request to http://127.0.02 with a Host header of example.com will go to the app service, which will then proxy the request to the service-x service.
If we bring up multiple instance of service-x, like this:
docker-compose up --scale service-x=3
So that we have:
$ docker ps -f label=com.docker.compose.project=proxy-example
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e2cbcb3e4e36 containous/whoami:latest "/whoami" 6 minutes ago Up 5 minutes 80/tcp proxy2_service-x_2
48655b9f0dd3 containous/whoami:latest "/whoami" 6 minutes ago Up 5 minutes 80/tcp proxy2_service-x_3
f6cdd1cfe908 nginx:mainline "/docker-entrypoint.…" 10 minutes ago Up 5 minutes 80/tcp proxy2_app_1
d65f996e7113 containous/whoami:latest "/whoami" 22 minutes ago Up 5 minutes 80/tcp proxy2_service-x_1
af1310113b6a traefik:latest "/entrypoint.sh --ap…" 22 minutes ago Up 5 minutes 127.0.0.2:80->80/tcp, 127.0.0.2:443->443/tcp, 127.0.0.2:8080->8080/tcp proxy2_proxy_1
And then make multiple requests through Traefik:
for x in {1..10}; do
curl -sf --resolve example.com:80:127.0.0.2 http://example.com/ | grep Hostname
done
We see that the requests are rotating among the service-x containers:
Hostname: e2cbcb3e4e36
Hostname: 48655b9f0dd3
Hostname: d65f996e7113
Hostname: e2cbcb3e4e36
Hostname: 48655b9f0dd3
Hostname: d65f996e7113
Hostname: e2cbcb3e4e36
Hostname: 48655b9f0dd3
Hostname: d65f996e7113
Hostname: e2cbcb3e4e36
Notes:
Why 127.0.0.2? Because I already have a service listening on 127.0.0.1 on port 80.

Unable to telnet to MariaDB container

I'm using docker-compose to run MariaDB and it is working fine. I am fetching jasper server and maria DB docker images and running them. When I telnet the jasper server image, it responds correctly, but when I telnet to MariaDB, it says:
telnet localhost 3306
Trying ::1...
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
What I might be doing wrong?
Here is the output of sudo docker ps -a:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9e759f106006 bitnami/jasperreports:7 "/app-entrypoint.sh …" 21 minutes ago Up 21 minutes 0.0.0.0:9093->8080/tcp, 0.0.0.0:443->8443/tcp ceyedev_jasperreports_1
9242e52f6af8 bitnami/mariadb:10.3 "/opt/bitnami/script…" 21 minutes ago Up 21 minutes 3306/tcp ceyedev_mariadb_1
Here is my docker compose file:
version: '2'
services:
mariadb:
image: 'bitnami/mariadb:10.3'
environment:
- MARIADB_USER=bn_jasperreports
- MARIADB_DATABASE=bitnami_jasperreports
- ALLOW_EMPTY_PASSWORD=yes
volumes:
- 'mariadb_data:/bitnami'
jasperreports:
image: 'bitnami/jasperreports:7'
environment:
- MARIADB_HOST=mariadb
- MARIADB_PORT_NUMBER=3306
- JASPERREPORTS_DATABASE_USER=bn_jasperreports
- JASPERREPORTS_DATABASE_NAME=bitnami_jasperreports
- ALLOW_EMPTY_PASSWORD=yes
ports:
- '9093:8080'
- '443:8443'
volumes:
- 'jasperreports_data:/bitnami'
depends_on:
- mariadb
volumes:
mariadb_data:
driver: local
jasperreports_data:
driver: local
You have to open the ports in your Docker compose file (that thing you posted is called a Docker Compose file, not Dockerfile which is the one containing the commands to build a Docker image).
In the mariadb section make it like this:
services:
mariadb:
image: 'bitnami/mariadb:10.3'
environment:
- MARIADB_USER=bn_jasperreports
- MARIADB_DATABASE=bitnami_jasperreports
- ALLOW_EMPTY_PASSWORD=yes
volumes:
- 'mariadb_data:/bitnami'
ports:
- 3306
This way, the 3306 port of MariaDB will be exposed to your local computer. This means:
that you may access MariaDB through the 3306 port
that ANYONE with direct network access to your computer (i.e. local IP address) will be able to access MariaDB through port 3306.
Bear in mind those two things regarding your system security.

How to setup Redis Commander and Docker?

I've tried the 'with docker' docs here but it's not working from the localhost:7000, localhost:8081, or any other port I use. What am I missing?
REDIS_PORT=6379
### Redis ################################################
redis:
container_name: redis
hostname: redis
build: ./redis
volumes:
- ${DATA_PATH_HOST}/redis:/data
ports:
- "${REDIS_PORT}:6379"
networks:
- backend
### REDISCOMMANDER ################################################
redis-commander:
container_name: rediscommander
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379
ports:
- "7000:80"
networks:
- frontend
- backend
depends_on:
- redis
Docker ps gives me:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
042c9a2e918a rediscommander/redis-commander:latest "/usr/bin/dumb-init …" About a minute ago Up About a minute (healthy) 8081/tcp, 0.0.0.0:7000->80/tcp rediscommander
86bc8c1ca5ff laradock_redis "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:6379->6379/tcp redis
Docker logs rediscommander gives me:
$ docker logs rediscommander
Creating custom redis-commander config '/redis-commander/config/local-production.json'.
Parsing 1 REDIS_HOSTS into custom redis-commander config '/redis-commander/config/local-production.json'.
node ./bin/redis-commander
Using scan instead of keys
No Save: false
listening on 0.0.0.0:8081
access with browser at http://127.0.0.1:8081
Redis Connection redis:6379 using Redis DB #0
Redis commader is listening on port 8081 in container. That is why you should change port binding to
ports:
- "7000:8081"
in redis commander block and access it via localhost:7000.

Unexpected unmapped port 80 in docker container

when i run docker ps i am seeing port 80 unmapped on the apphub-ui container, i am expecting it to be mapped to 4200, which i also see. I dont understand why 80/tcp is present, it is creating connection refused issues in my app.
Here is the docker ps output
docker ps 10s +59%
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
22cb2b9c28d4 swimlane/apphub-ui:latest "/usr/bin/reefer -t …" 20 seconds ago Up 17 seconds 80/tcp, 0.0.0.0:80->4200/tcp apphub_apphub-ui_1
19a78f93bfe7 swimlane/apphub-api:latest "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 0.0.0.0:3000->3000/tcp apphub_apphub-api_1
7b3478e2dcd5 mongo:3.6 "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 0.0.0.0:27017->27017/tcp apphub_mongo_1
My docker-compose file is as follows
version: '3.6'
volumes:
mongo:
services:
mongo:
image: mongo:3.6
ports:
- 27017:27017
volumes:
- mongo:/data/db
apphub-api:
image: swimlane/apphub-api:latest
ports:
- 3000:3000
depends_on:
- mongo
apphub-ui:
image: swimlane/apphub-ui:latest
ports:
- 80:4200
depends_on:
- apphub-api
environment:
- APPHUB_API_HOST=apphub-api
My app runs fine if i change the mapped ports for the ui feature to 4200:4200, and i dont see the stray unmapped 80
I'm having the feeling that you want to map the port 4200 on your host to the port 80 of your container. If that's the case, you should switch the values. i.e.
ports:
- 4200:80

Why can't I connect to my local docker-compose container on Windows 10?

I'm trying to dockerize a Python application, for which I've been following this tutorial. The tutorial is from April 2015 and still uses Docker Machine, which, judging from this answer, is no longer necessary to run Docker containers locally on Windows.
I got it working with Docker Machine before, and was able to see the web app and interact with it. But now I'm trying to get this working without Docker Machine, with Docker version 17.06.0-ce, build 02c1d87, on Windows 10.
Here's the docker-compose.yml:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
volumes:
- /usr/src/app/static
env_file: .env
command: /usr/local/bin/gunicorn -w 2 -b :8000 app:app
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
volumes:
- /www/static
volumes_from:
- web
links:
- web:web
data:
image: postgres:latest
volumes:
- /var/lib/postgresql
command: "true"
postgres:
restart: always
image: postgres:latest
volumes_from:
- data
ports:
- "5432:5432"
I started the containers:
$ docker-compose up -d
Creating polly_data_1 ...
Creating polly_data_1 ... done
Creating polly_postgres_1 ...
Creating polly_postgres_1 ... done
Creating polly_web_1 ...
Creating polly_web_1 ... done
Creating polly_nginx_1 ...
Creating polly_nginx_1 ... done
Then, when I run docker ps, it shows the following three containers running:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9b2c1048f3a5 polly_nginx "/usr/sbin/nginx" 4 seconds ago Up 3 seconds 0.0.0.0:80->80/tcp polly_nginx_1
d561ac5b901a polly_web "/usr/local/bin/gu..." 5 seconds ago Up 4 seconds 8000/tcp polly_web_1
ecb029d6ec3a postgres:latest "docker-entrypoint..." 7 seconds ago Up 5 seconds 0.0.0.0:5432->5432/tcp polly_postgres_1
(At this point, navigating to http://localhost:8000/ in Chrome already yields ERR_CONNECTION_REFUSED.)
I then ran the script to set up the database, as per the tutorial (extra //s because I'm using Git Bash on Windows 10):
$ docker-compose run web ///usr/local/bin/python create_db.py
Starting polly_data_1 ...
Starting polly_data_1 ... done
Starting polly_postgres_1 ... done
Now when I run docker ps, it shows the following four containers running:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a129c12f5982 polly_web "//usr/local/bin/p..." 5 seconds ago Up Less than a second 8000/tcp polly_web_run_1
9b2c1048f3a5 polly_nginx "/usr/sbin/nginx" 16 seconds ago Up 15 seconds 0.0.0.0:80->80/tcp polly_nginx_1
d561ac5b901a polly_web "/usr/local/bin/gu..." 17 seconds ago Up 16 seconds 8000/tcp polly_web_1
ecb029d6ec3a postgres:latest "docker-entrypoint..." 19 seconds ago Up 17 seconds 0.0.0.0:5432->5432/tcp polly_postgres_1
And localhost:8000 is still refusing to connect. The web container exposes port 8000, so I don't get why I can't connect to it.
How can I get this working so I can access the web app in the web container locally?
Just change:
expose:
- "8000"
By
ports:
- "8000:8000"
Btw http://localhost:80 is not working?
Regards
Turns out, as suggested by Carlos and 200_OK as part of their answers and comments, it was working as intended - it was running at port 80, not 8000.
Web exposes port 8000 internally inside the container. But that port is not mapped to your host machine port.
I think the problem is in your command. The option is -p, not -b.
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
volumes:
- /usr/src/app/static
env_file: .env
command: /usr/local/bin/gunicorn -w 2 -p :8000 app:app

Resources