Docker swarm can't reach worker nodes - docker

Question

So, i have set up a docker swarm and connected a worker on it and deployed a stack with 4 services:
generator, which will be located at the worker node
API & scheduler, both will be located on the manager node
proxy, that will be located on the manager node, accepting requests and redirecting it though the other 3
This is my stack file:
version: "3.7"
services:
generator:
image: musicorum/generator:latest
restart: always
environment:
- 'XXXX=XXXX'
deploy:
resources:
reservations:
memory: 860M
placement:
constraints:
- "node.labels.generator==yes"
ports:
- 5000:5000
networks:
- proxy_ext
- netg
volumes:
- type: bind
source: /home/musicorum/cache
target: /usr/src/app/cache
api:
image: musicorum/api:latest
restart: always
environment:
- 'XXXX=XXXX'
networks:
- proxy_ext
ports:
- 4500:4500
deploy:
placement:
constraints:
- "node.labels.generator!=yes"
scheduler:
image: musicorum/scheduler:latest
restart: always
environment:
- 'XXXX=XXXX'
ports:
- 6500:6500
networks:
- proxy_ext
deploy:
placement:
constraints:
- "node.labels.generator!=yes"
proxy:
image: nginx:latest
restart: always
networks:
- proxy_ext
- netg
ports:
- 80:80
- 443:443
configs:
- source: nginx_4
target: /etc/nginx/conf.d/default.conf
- source: sslcrt
target: /etc/ssl/musicorumapp/ssl.crt
- source: sslkey
target: /etc/ssl/musicorumapp/ssl.key
depends_on:
- scheduler
- api
- generator
deploy:
placement:
constraints:
- "node.labels.generator!=yes"
configs:
nginx_4:
external: true
sslcrt:
external: true
sslkey:
external: true
networks:
proxy_ext:
external: true
netg:
driver: overlay
attachable: true
As you can see, the they are connected on the same network, i even created proxy_ext and netg to double-check the connection, but Nginx give this message when start up:
/docker-entrypoint.sh: Configuration complete; ready for start up
2020/07/07 13:32:17 [emerg] 1#1: host not found in upstream "musicorum_generator" in /etc/nginx/conf.d/default.conf:30
nginx: [emerg] host not found in upstream "musicorum_generator" in /etc/nginx/conf.d/default.conf:30
I don't know why the Nginx, at the manager node, can't reach out to the generator container, at the worker node. If it helps, here's my default.conf:
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/musicorumapp/ssl.crt;
ssl_certificate_key /etc/ssl/musicorumapp/ssl.key;
server_name api.musicorumapp.com;
location / {
proxy_pass http://musicorum_api:4500/;
}
}
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/musicorumapp/ssl.crt;
ssl_certificate_key /etc/ssl/musicorumapp/ssl.key;
server_name scheduler.musicorumapp.com;
location / {
proxy_pass http://musicorum_scheduler:6500/;
}
}
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/musicorumapp/ssl.crt;
ssl_certificate_key /etc/ssl/musicorumapp/ssl.key;
server_name generator.musicorumapp.com;
location / {
proxy_pass http://musicorum_generator:5000/;
}
}

Answer 1

In your default.conf you need to reference the services by their service name. This is the name that the internal DNS will resolve.
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/musicorumapp/ssl.crt;
ssl_certificate_key /etc/ssl/musicorumapp/ssl.key;
server_name api.musicorumapp.com;
location / {
proxy_pass http://api:4500/; <------ 'api' is the service name
}
}
You only would need to prefix the name of your stack if the reverse proxy server was running external to your stack's network, but since they are all on the same network, the DNS will resolve the service name alone.
You may also remove the ports: 8000:8000 on all of your apps (except reverse proxy) in your stack yaml file because you want to route traffic through your reverse proxy, not bind the port to the host. This could lead to security vulnerabilities as well. There are no port restrictions within a docker network. If an application is listening on 8000, your reverse proxy can contact it with http://service-name:8000 within the stack's overlay network.