traefik not working when a service is in multiple overlay networks - docker-swarm

version: '3.2'
services:
traefik:
image: "traefik:latest"
command:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.docker=true
- --providers.docker.swarmMode=true
- --providers.docker.exposedbydefault=false
- --providers.docker.network=public
- --api
- --log.level=ERROR
ports:
- "80:80"
- "443:443"
networks:
- public
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
agent:
image: portainer/agent
environment:
# REQUIRED: Should be equal to the service name prefixed by "tasks." when
# deployed inside an overlay network
AGENT_CLUSTER_ADDR: tasks.agent
# AGENT_PORT: 9001
# LOG_LEVEL: debug
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/volumes:/var/lib/docker/volumes
networks:
- agent_network
deploy:
mode: global
placement:
constraints: [node.platform.os == linux]
portainer:
image: portainer/portainer-ce:2.0.0
command: -H tcp://tasks.agent:9001 --tlsskipverify
volumes:
- data:/data
networks:
- public
- agent_network
deploy:
mode: replicated
replicas: 1
placement:
constraints: [node.role == manager]
labels:
- "traefik.enable=true"
- "traefik.http.routers.portainer.rule=Host(`portainer.yourdomain.com`)"
- "traefik.http.routers.portainer.entrypoints=web"
- "traefik.http.services.portainer.loadbalancer.server.port=9000"
- "traefik.http.routers.portainer.service=portainer"
# Edge
- "traefik.http.routers.edge.rule=Host(`edge.yourdomain.com`)"
- "traefik.http.routers.edge.entrypoints=web"
- "traefik.http.services.edge.loadbalancer.server.port=8000"
- "traefik.http.routers.edge.service=edge"
networks:
public:
external: true
agent_network:
external: true
volumes:
We can see that "portainer" is in both public and agent_network overlay networks. And it has two IP addresses, 10.0.38.7 and 10.0.39.7.
traefik is only in public network, it has IP address 10.0.38.6.
The problem is, from traefik web UI, it refers "portainer" as 10.0.39.7 instead of 10.0.38.7. So that it fails to work.
Any solution to this?

Related

Wordpress service is not showing in Traefik

I am trying to deploy Wordpress application on docker swarm stack, behind Traefik reverse proxy, I wanted to use Nginx for reverse proxying but as the the Wordpress is deployed with 2 replicas I am facing session time out issue, therefore I am trying the use Traefik instaed to configure sticky session later.
I have deployed the Traefik service successfully and can access the dashboard, but the Wordpress is not being proxied and it is not showing in the services list on the dashboard.
Traefik Dashboard Screenshot
Traefik Dashboard 2
Traefik Proxy yaml File:
version: '3.3'
services:
traefik:
image: traefik:v2.2
ports:
- 80:80
- 443:443
deploy:
placement:
constraints:
# Make the traefik service run only on the node with this label
# as the node with it has the volume for the certificates
- node.labels.traefik-public.traefik-public-certificates == true
labels:
- traefik.enable=true
- traefik.docker.network=traefik-public
- traefik.constraint-label=traefik-public
- traefik.http.middlewares.admin-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
- traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-http.entrypoints=http
- traefik.http.routers.traefik-public-http.middlewares=https-redirect
- traefik.http.routers.traefik-public-https.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-https.entrypoints=https
- traefik.http.routers.traefik-public-https.tls=true
- traefik.http.routers.traefik-public-https.service=api#internal
- traefik.http.routers.traefik-public-https.tls.certresolver=le
- traefik.http.routers.traefik-public-https.middlewares=admin-auth
- traefik.http.services.traefik-public.loadbalancer.server.port=8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- traefik-public-certificates:/certificates
command:
- --providers.docker
- --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
- --providers.docker.exposedbydefault=false
- --providers.docker.swarmmode
- --entrypoints.http.address=:80
- --entrypoints.https.address=:443
- --certificatesresolvers.le.acme.email=${EMAIL?Variable not set}
- --certificatesresolvers.le.acme.storage=/certificates/acme.json
- --certificatesresolvers.le.acme.tlschallenge=true
- --accesslog
- --log
- --api
networks:
- traefik-public
volumes:
traefik-public-certificates:
networks:
traefik-public:
external: true
Wordpress yaml File:
version: "3.4"
services:
db:
image: mariadb
secrets:
- db_user
- db_pass
environment:
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_pass
MYSQL_USER_FILE: /run/secrets/db_user
MYSQL_PASSWORD_FILE: /run/secrets/db_pass
MYSQL_DATABASE_NAME: wpdb
ports:
- 3306:3306
networks:
- backend
volumes:
- db-data:/var/lib/mysql
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 60s
wp:
image: wordpress
secrets:
- db_user
- db_pass
depends_on:
- db
labels:
- traefik.enable=true
- traefik.constraint-label=traefik-public
- traefik.docker.network=traefik-public
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
- traefik.http.routers.wp.rule=Host(`example.com`)
- traefik.http.routers.wp.entrypoints=http
- traefik.http.routers.wp.middlewares=https-redirect
- traefik.http.routers.wp-secured.rule=Host(`example.com`)
- traefik.http.routers.wp-secured.entrypoints=https
- traefik.http.routers.wp-secured.tls=true
- traefik.http.routers.wp-secured.tls.certresolver=le
- traefik.http.services.wp.loadbalancer.server.port=8080
environment:
WORDPRESS_DB_HOST: 192.168.20.30:3306 # node IP
WORDPRESS_DB_USER_FILE: /run/secrets/db_user
WORDPRESS_DB_PASSWORD_FILE: /run/secrets/db_pass
WORDPRESS_DB_NAME: wpdb
networks:
- backend
- traefik-public
volumes:
- wp-data:/var/www/html
deploy:
replicas: 2
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 60s
networks:
backend:
external: false
traefik-public:
external: true
volumes:
wp-data:
db-data:
secrets:
db_user:
file: ./db_user.txt
db_pass:
file: ./db_pass.txt
```

Traefik Dashboard returns always 404 in Docker Swarm deployment

I'm trying my best to get Traefik dashboard available through http://gateway.localhost/dashboard/, but I'm always getting a 404 response* from Traefik. Can s.o. please review my stack file and tell me, why it's not working?
I tried it on my server with a valid domain, but it's either working there or on localhost with Docker Desktop in Swarm mode. The WhoAmI service can be reached through http://localhost which is correct.
docker stack deploy -c traefik.yml traefik
*404 is returned for these routes too: http://gateway.localhost, http://gateway.localhost/dashboard
traefik.yml:
version: '3'
services:
reverse-proxy:
image: traefik:v2.5
command:
- "--providers.docker.swarmmode=true"
- "--providers.docker.exposedByDefault=false"
- "--api.dashboard=true"
- "--entrypoints.web.address=:80"
# Logging
- "--accesslog"
- "--log.level=INFO"
ports:
- "80:80"
deploy:
labels:
#Because Swarm API does not support automatic way
- "traefik.http.services.reverse-proxy.loadbalancer.server.port=80"
#Dashboard
- "traefik.http.routers.dashboard.rule=Host(`gateway.localhost`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
- "traefik.http.routers.dashboard.service=api#internal"
- "traefik.http.routers.dashboard.entrypoints=web"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/,test2:$$apr1$$d9hr9HBB$$4HxwgUir3HP4EsggP/QNo0"
placement:
constraints:
- node.role == manager
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
whoami:
image: traefik/whoami
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`localhost`)"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.services.whoami.loadbalancer.server.port=80"
You need to enable traefik for the container with the traefik.enable=true label:
version: '3'
services:
reverse-proxy:
image: traefik:v2.5
command:
- "--providers.docker.swarmmode=true"
- "--providers.docker.exposedByDefault=false"
- "--api.dashboard=true"
- "--entrypoints.web.address=:80"
# Logging
- "--accesslog"
- "--log.level=INFO"
ports:
- "80:80"
deploy:
labels:
######## add the following label to enable traefik #######
- "traefik.enable=true"
#Because Swarm API does not support automatic way
- "traefik.http.services.reverse-proxy.loadbalancer.server.port=80"
#Dashboard
- "traefik.http.routers.dashboard.rule=Host(`gateway.localhost`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
- "traefik.http.routers.dashboard.service=api#internal"
- "traefik.http.routers.dashboard.entrypoints=web"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/,test2:$$apr1$$d9hr9HBB$$4HxwgUir3HP4EsggP/QNo0"
placement:
constraints:
- node.role == manager
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
whoami:
image: traefik/whoami
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`localhost`)"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.services.whoami.loadbalancer.server.port=80"

traefik portainer swarm 404

when i try to reach endpoint for portainer or traefik i get 404 not found error.
It's work well if i setup in docker and not docker swarm. I believe i miss config something like balancer.
something i miss with thetraefik.http.services.<service_name>.loadbalancer.server.port
Without swarm config:
# traefik.yml
api:
dashboard: true
entryPoints:
http:
address: ":80"
https:
address: ":443"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
certificatesResolvers:
http:
acme:
email: kevin.gaulin#gmail.com
storage: acme.json
httpChallenge:
entryPoint: http
#docker-compose.yml
version: '3'
services:
traefik:
image: traefik:v2.3.7
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- traefik-proxy
ports:
- 80:80
- 443:443
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik.yml:/traefik.yml:ro
- ./acme.json:/acme.json
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.routers.traefik.rule=Host(`traefik2.${DOMAIN}`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${USERNAME}:${HASHED_PASSWORD}"
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=https"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik2.${DOMAIN}`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=http"
- "traefik.http.routers.traefik-secure.service=api#internal"
portainer:
image: portainer/portainer-ce:latest
container_name: portainer
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- traefik-proxy
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./portainer-data:/data
labels:
- "traefik.enable=true"
- "traefik.http.routers.portainer.entrypoints=http"
- "traefik.http.routers.portainer.rule=Host(`portainer2.${DOMAIN}`)"
- "traefik.http.middlewares.portainer-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.portainer.middlewares=portainer-https-redirect"
- "traefik.http.routers.portainer-secure.entrypoints=https"
- "traefik.http.routers.portainer-secure.rule=Host(`portainer2.${DOMAIN}`)"
- "traefik.http.routers.portainer-secure.tls=true"
- "traefik.http.routers.portainer-secure.tls.certresolver=http"
- "traefik.http.routers.portainer-secure.service=portainer"
- "traefik.http.services.portainer.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik-proxy"
networks:
traefik-proxy:
external: true
Swarm config
# traefik.yml
api:
dashboard: true
entryPoints:
http:
address: ":80"
https:
address: ":443"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
swarmmode: true
watch: false
certificatesResolvers:
http:
acme:
email: kevin.gaulin#gmail.com
storage: acme.json
httpChallenge:
entryPoint: http
log:
level: DEBUG
# docker-compose.yml
version: '3'
services:
traefik:
image: traefik:v2.3.7
networks:
- traefik-proxy
ports:
- 80:80
- 443:443
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik.yml:/traefik.yml:ro
- ./acme.json:/acme.json
deploy:
mode: replicated
replicas: 1
placement:
constraints: [node.role == manager]
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.routers.traefik.rule=Host(`traefik2.${DOMAIN}`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${USERNAME}:${HASHED_PASSWORD}"
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=https"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik2.${DOMAIN}`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=http"
- "traefik.http.routers.traefik-secure.service=api#internal"
- "traefik.http.services.traefik.loadbalancer.server.port=8080"
agent:
image: portainer/agent
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/volumes:/var/lib/docker/volumes
networks:
- agent_network
deploy:
mode: global
placement:
constraints: [node.platform.os == linux]
portainer:
image: portainer/portainer-ce:latest
command: -H tcp://tasks.agent:9001 --tlsskipverify
networks:
- traefik-proxy
- agent_network
volumes:
- /etc/localtime:/etc/localtime:ro
- ./portainer-data:/data
deploy:
mode: replicated
replicas: 1
placement:
constraints: [node.role == manager]
labels:
- "traefik.enable=true"
- "traefik.http.routers.portainer.entrypoints=http"
- "traefik.http.routers.portainer.rule=Host(`portainer2.${DOMAIN}`)"
- "traefik.http.middlewares.portainer-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.portainer.middlewares=portainer-https-redirect"
- "traefik.http.routers.portainer-secure.entrypoints=https"
- "traefik.http.routers.portainer-secure.rule=Host(`portainer2.${DOMAIN}`)"
- "traefik.http.routers.portainer-secure.tls=true"
- "traefik.http.routers.portainer-secure.tls.certresolver=http"
- "traefik.http.routers.portainer-secure.service=portainer"
- "traefik.http.services.portainer.loadbalancer.server.port=9000"
- "traefik.docker.network=traefik-proxy"
networks:
traefik-proxy:
driver: overlay
external: true
agent_network:
driver: overlay

How can I use Sticky sessions on Docker using Traefik v2.3?

How can I use Sticky sessions in Traefik v2.3?
When I enable the below line in my docker compose file, my docker services (e.g. http://192.168.0.1:9086) don't work.
- "traefik.http.services.mynginximage2.loadBalancer.sticky.cookie=true"
Do you have any idea why a sticky session doesn't work?
And is it possible to make a session affinity by the labels of a service?
The following is my docker compose file:
version: "3.8"
services:
traefik:
image: traefik:v2.3
deploy:
mode: global
networks:
- traefik-net
command:
#- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.mynginximage2_ep.address=:8086"
ports:
- 80:80
- 9086:8086
- 8080:8080
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
whoami:
image: "traefik/whoami"
deploy:
replicas: 2
networks:
- traefik-net
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.routers.whoami.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
#- "traefik.http.services.whoami.loadBalancer.sticky.cookie=true"
- "traefik.http.services.whoami-service.loadbalancer.server.port=80"
mynginximage2:
image: mynginximage2
deploy:
replicas: 2
networks:
- traefik-net
#ports:
# - 9080:8086
labels:
- "traefik.enable=true"
- "traefik.http.routers.mynginximage2.entrypoints=mynginximage2_ep"
- "traefik.http.routers.mynginximage2.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
#- "traefik.http.services.mynginximage2.loadBalancer.sticky.cookie=true"
- "traefik.http.services.mynginximage2-service.loadbalancer.server.port=8086"
networks:
traefik-net:
external: true
name: traefik-net
I found the issue why my Traefik routers don't work when I enable a sticky session feature.
The solution is that each service has a different cookie name. So, I added cookie names. For example:
version: "3.8"
services:
#...
whoami:
#...
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.routers.whoami.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
- "traefik.http.services.whoami-service.loadBalancer.sticky.cookie=true"
- "traefik.http.services.whoami-service.loadBalancer.sticky.cookie.name=whoami_cookie_name"
- "traefik.http.services.whoami-service.loadbalancer.server.port=80"
mynginximage2:
#...
labels:
- "traefik.enable=true"
- "traefik.http.routers.mynginximage2.entrypoints=mynginximage2_ep"
- "traefik.http.routers.mynginximage2.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
- "traefik.http.services.mynginximage2-service.loadBalancer.sticky.cookie=true"
- "traefik.http.services.mynginximage2-service.loadBalancer.sticky.cookie.name=mynginximage2_cookie_name"
- "traefik.http.services.mynginximage2-service.loadbalancer.server.port=8086"
#...

Traefik + Consul not generaitng SSL certificates in replicated mode, using TRAEFIK DEFAULT CERT

I have a setup of 1 master and 2 worker nodes running docker swarm. I deployed Traefik + Consul using the setup below:
version: '3.3'
services:
consul-leader:
image: consul:latest
command:
- agent
- -server
- -client=0.0.0.0
- -bootstrap
- -ui
environment:
CONSUL_BIND_INTERFACE: eth0
CONSUL_LOCAL_CONFIG: '{"leave_on_terminate": true}'
volumes:
- consul-data-leader:/consul/data
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
labels:
traefik.tags: traefik-public
traefik.redirectorservice.frontend.redirect.entryPoint: https
traefik.webservice.frontend.entryPoints: https
traefik.redirectorservice.frontend.entryPoints: http
traefik.docker.network: traefik-public
traefik.enable: 'true'
traefik.frontend.auth.basic.users: admin:$apr1$lKAo73kT$xlahD.KLANH8ZbMaDXDsC.
traefik.port: '8500'
traefik.frontend.rule: Host:consul.live.mydomain.app
consul-replica:
image: consul:latest
command:
- agent
- -server
- -client=0.0.0.0
- -retry-join=consul-leader
environment:
CONSUL_BIND_INTERFACE: eth0
CONSUL_LOCAL_CONFIG: '{"leave_on_terminate": true}'
volumes:
- consul-data-replica:/consul/data
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
replicas: 3
traefik:
image: traefik:v1.7
command:
- --docker
- --docker.swarmmode
- --docker.watch
- --docker.exposedbydefault=false
- --constraints=tag==traefik-public
- --entrypoints=Name:http Address::80
- --entrypoints=Name:https Address::443 TLS
- --consul
- --consul.endpoint=consul-leader:8500
- --acme
- --acme.email=me.me#me.com
- --acme.storage=traefik/acme/account
- --acme.entryPoint=https
- --acme.httpChallenge.entryPoint=http
- --acme.onhostrule=true
- --acme.acmelogging=true
- --logLevel=INFO
- --accessLog
- --api
ports:
- 80:80
- 443:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
replicas: 3
labels:
traefik.tags: traefik-public
traefik.redirectorservice.frontend.redirect.entryPoint: https
traefik.webservice.frontend.entryPoints: https
traefik.redirectorservice.frontend.entryPoints: http
traefik.docker.network: traefik-public
traefik.enable: 'true'
traefik.frontend.auth.basic.users: admin:$apr1$lKAo73kT$xlahD.KLANH8ZbMaDXDsC.
traefik.port: '8080'
traefik.frontend.rule: Host:traefik.live.mydomain.app
placement:
constraints:
- node.role == manager
networks:
default:
driver: overlay
traefik-public:
external: true
volumes:
consul-data-replica:
driver: local
When I deploy a service or stack in GLOBAL mode, everything seems to work fine. However, when I try to deploy stack using replicated mode, SSL certificate is not generated and Traefik is using TRAEFIK DEFAULT CERT instead. Can anyone tell me what I'm doing wrong ?

Resources