Traefik + Consul not generaitng SSL certificates in replicated mode, using TRAEFIK DEFAULT CERT - docker

I have a setup of 1 master and 2 worker nodes running docker swarm. I deployed Traefik + Consul using the setup below:
version: '3.3'
services:
consul-leader:
image: consul:latest
command:
- agent
- -server
- -client=0.0.0.0
- -bootstrap
- -ui
environment:
CONSUL_BIND_INTERFACE: eth0
CONSUL_LOCAL_CONFIG: '{"leave_on_terminate": true}'
volumes:
- consul-data-leader:/consul/data
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
labels:
traefik.tags: traefik-public
traefik.redirectorservice.frontend.redirect.entryPoint: https
traefik.webservice.frontend.entryPoints: https
traefik.redirectorservice.frontend.entryPoints: http
traefik.docker.network: traefik-public
traefik.enable: 'true'
traefik.frontend.auth.basic.users: admin:$apr1$lKAo73kT$xlahD.KLANH8ZbMaDXDsC.
traefik.port: '8500'
traefik.frontend.rule: Host:consul.live.mydomain.app
consul-replica:
image: consul:latest
command:
- agent
- -server
- -client=0.0.0.0
- -retry-join=consul-leader
environment:
CONSUL_BIND_INTERFACE: eth0
CONSUL_LOCAL_CONFIG: '{"leave_on_terminate": true}'
volumes:
- consul-data-replica:/consul/data
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
replicas: 3
traefik:
image: traefik:v1.7
command:
- --docker
- --docker.swarmmode
- --docker.watch
- --docker.exposedbydefault=false
- --constraints=tag==traefik-public
- --entrypoints=Name:http Address::80
- --entrypoints=Name:https Address::443 TLS
- --consul
- --consul.endpoint=consul-leader:8500
- --acme
- --acme.email=me.me#me.com
- --acme.storage=traefik/acme/account
- --acme.entryPoint=https
- --acme.httpChallenge.entryPoint=http
- --acme.onhostrule=true
- --acme.acmelogging=true
- --logLevel=INFO
- --accessLog
- --api
ports:
- 80:80
- 443:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- default
- traefik-public
logging:
driver: json-file
deploy:
replicas: 3
labels:
traefik.tags: traefik-public
traefik.redirectorservice.frontend.redirect.entryPoint: https
traefik.webservice.frontend.entryPoints: https
traefik.redirectorservice.frontend.entryPoints: http
traefik.docker.network: traefik-public
traefik.enable: 'true'
traefik.frontend.auth.basic.users: admin:$apr1$lKAo73kT$xlahD.KLANH8ZbMaDXDsC.
traefik.port: '8080'
traefik.frontend.rule: Host:traefik.live.mydomain.app
placement:
constraints:
- node.role == manager
networks:
default:
driver: overlay
traefik-public:
external: true
volumes:
consul-data-replica:
driver: local
When I deploy a service or stack in GLOBAL mode, everything seems to work fine. However, when I try to deploy stack using replicated mode, SSL certificate is not generated and Traefik is using TRAEFIK DEFAULT CERT instead. Can anyone tell me what I'm doing wrong ?

Related

Wordpress service is not showing in Traefik

I am trying to deploy Wordpress application on docker swarm stack, behind Traefik reverse proxy, I wanted to use Nginx for reverse proxying but as the the Wordpress is deployed with 2 replicas I am facing session time out issue, therefore I am trying the use Traefik instaed to configure sticky session later.
I have deployed the Traefik service successfully and can access the dashboard, but the Wordpress is not being proxied and it is not showing in the services list on the dashboard.
Traefik Dashboard Screenshot
Traefik Dashboard 2
Traefik Proxy yaml File:
version: '3.3'
services:
traefik:
image: traefik:v2.2
ports:
- 80:80
- 443:443
deploy:
placement:
constraints:
# Make the traefik service run only on the node with this label
# as the node with it has the volume for the certificates
- node.labels.traefik-public.traefik-public-certificates == true
labels:
- traefik.enable=true
- traefik.docker.network=traefik-public
- traefik.constraint-label=traefik-public
- traefik.http.middlewares.admin-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
- traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-http.entrypoints=http
- traefik.http.routers.traefik-public-http.middlewares=https-redirect
- traefik.http.routers.traefik-public-https.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-https.entrypoints=https
- traefik.http.routers.traefik-public-https.tls=true
- traefik.http.routers.traefik-public-https.service=api#internal
- traefik.http.routers.traefik-public-https.tls.certresolver=le
- traefik.http.routers.traefik-public-https.middlewares=admin-auth
- traefik.http.services.traefik-public.loadbalancer.server.port=8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- traefik-public-certificates:/certificates
command:
- --providers.docker
- --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
- --providers.docker.exposedbydefault=false
- --providers.docker.swarmmode
- --entrypoints.http.address=:80
- --entrypoints.https.address=:443
- --certificatesresolvers.le.acme.email=${EMAIL?Variable not set}
- --certificatesresolvers.le.acme.storage=/certificates/acme.json
- --certificatesresolvers.le.acme.tlschallenge=true
- --accesslog
- --log
- --api
networks:
- traefik-public
volumes:
traefik-public-certificates:
networks:
traefik-public:
external: true
Wordpress yaml File:
version: "3.4"
services:
db:
image: mariadb
secrets:
- db_user
- db_pass
environment:
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_pass
MYSQL_USER_FILE: /run/secrets/db_user
MYSQL_PASSWORD_FILE: /run/secrets/db_pass
MYSQL_DATABASE_NAME: wpdb
ports:
- 3306:3306
networks:
- backend
volumes:
- db-data:/var/lib/mysql
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 60s
wp:
image: wordpress
secrets:
- db_user
- db_pass
depends_on:
- db
labels:
- traefik.enable=true
- traefik.constraint-label=traefik-public
- traefik.docker.network=traefik-public
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
- traefik.http.routers.wp.rule=Host(`example.com`)
- traefik.http.routers.wp.entrypoints=http
- traefik.http.routers.wp.middlewares=https-redirect
- traefik.http.routers.wp-secured.rule=Host(`example.com`)
- traefik.http.routers.wp-secured.entrypoints=https
- traefik.http.routers.wp-secured.tls=true
- traefik.http.routers.wp-secured.tls.certresolver=le
- traefik.http.services.wp.loadbalancer.server.port=8080
environment:
WORDPRESS_DB_HOST: 192.168.20.30:3306 # node IP
WORDPRESS_DB_USER_FILE: /run/secrets/db_user
WORDPRESS_DB_PASSWORD_FILE: /run/secrets/db_pass
WORDPRESS_DB_NAME: wpdb
networks:
- backend
- traefik-public
volumes:
- wp-data:/var/www/html
deploy:
replicas: 2
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 3
window: 60s
networks:
backend:
external: false
traefik-public:
external: true
volumes:
wp-data:
db-data:
secrets:
db_user:
file: ./db_user.txt
db_pass:
file: ./db_pass.txt
```

traefik not working when a service is in multiple overlay networks

version: '3.2'
services:
traefik:
image: "traefik:latest"
command:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.docker=true
- --providers.docker.swarmMode=true
- --providers.docker.exposedbydefault=false
- --providers.docker.network=public
- --api
- --log.level=ERROR
ports:
- "80:80"
- "443:443"
networks:
- public
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
agent:
image: portainer/agent
environment:
# REQUIRED: Should be equal to the service name prefixed by "tasks." when
# deployed inside an overlay network
AGENT_CLUSTER_ADDR: tasks.agent
# AGENT_PORT: 9001
# LOG_LEVEL: debug
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/volumes:/var/lib/docker/volumes
networks:
- agent_network
deploy:
mode: global
placement:
constraints: [node.platform.os == linux]
portainer:
image: portainer/portainer-ce:2.0.0
command: -H tcp://tasks.agent:9001 --tlsskipverify
volumes:
- data:/data
networks:
- public
- agent_network
deploy:
mode: replicated
replicas: 1
placement:
constraints: [node.role == manager]
labels:
- "traefik.enable=true"
- "traefik.http.routers.portainer.rule=Host(`portainer.yourdomain.com`)"
- "traefik.http.routers.portainer.entrypoints=web"
- "traefik.http.services.portainer.loadbalancer.server.port=9000"
- "traefik.http.routers.portainer.service=portainer"
# Edge
- "traefik.http.routers.edge.rule=Host(`edge.yourdomain.com`)"
- "traefik.http.routers.edge.entrypoints=web"
- "traefik.http.services.edge.loadbalancer.server.port=8000"
- "traefik.http.routers.edge.service=edge"
networks:
public:
external: true
agent_network:
external: true
volumes:
We can see that "portainer" is in both public and agent_network overlay networks. And it has two IP addresses, 10.0.38.7 and 10.0.39.7.
traefik is only in public network, it has IP address 10.0.38.6.
The problem is, from traefik web UI, it refers "portainer" as 10.0.39.7 instead of 10.0.38.7. So that it fails to work.
Any solution to this?

How can I use Sticky sessions on Docker using Traefik v2.3?

How can I use Sticky sessions in Traefik v2.3?
When I enable the below line in my docker compose file, my docker services (e.g. http://192.168.0.1:9086) don't work.
- "traefik.http.services.mynginximage2.loadBalancer.sticky.cookie=true"
Do you have any idea why a sticky session doesn't work?
And is it possible to make a session affinity by the labels of a service?
The following is my docker compose file:
version: "3.8"
services:
traefik:
image: traefik:v2.3
deploy:
mode: global
networks:
- traefik-net
command:
#- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.mynginximage2_ep.address=:8086"
ports:
- 80:80
- 9086:8086
- 8080:8080
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
whoami:
image: "traefik/whoami"
deploy:
replicas: 2
networks:
- traefik-net
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.routers.whoami.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
#- "traefik.http.services.whoami.loadBalancer.sticky.cookie=true"
- "traefik.http.services.whoami-service.loadbalancer.server.port=80"
mynginximage2:
image: mynginximage2
deploy:
replicas: 2
networks:
- traefik-net
#ports:
# - 9080:8086
labels:
- "traefik.enable=true"
- "traefik.http.routers.mynginximage2.entrypoints=mynginximage2_ep"
- "traefik.http.routers.mynginximage2.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
#- "traefik.http.services.mynginximage2.loadBalancer.sticky.cookie=true"
- "traefik.http.services.mynginximage2-service.loadbalancer.server.port=8086"
networks:
traefik-net:
external: true
name: traefik-net
I found the issue why my Traefik routers don't work when I enable a sticky session feature.
The solution is that each service has a different cookie name. So, I added cookie names. For example:
version: "3.8"
services:
#...
whoami:
#...
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.entrypoints=web"
- "traefik.http.routers.whoami.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
- "traefik.http.services.whoami-service.loadBalancer.sticky.cookie=true"
- "traefik.http.services.whoami-service.loadBalancer.sticky.cookie.name=whoami_cookie_name"
- "traefik.http.services.whoami-service.loadbalancer.server.port=80"
mynginximage2:
#...
labels:
- "traefik.enable=true"
- "traefik.http.routers.mynginximage2.entrypoints=mynginximage2_ep"
- "traefik.http.routers.mynginximage2.rule=Host(`192.168.0.1`) || Host(`192.168.0.2`)"
- "traefik.http.services.mynginximage2-service.loadBalancer.sticky.cookie=true"
- "traefik.http.services.mynginximage2-service.loadBalancer.sticky.cookie.name=mynginximage2_cookie_name"
- "traefik.http.services.mynginximage2-service.loadbalancer.server.port=8086"
#...

Swarm: Traefik returning 404 on compose services

I have two devices running Docker; an Intel NUC and a Raspberry Pi. My NUC is used as a mediaplayer/mediaserver. This also is the manager node. The Pi is being used as Home Assistant and MQTT machine and is set as worker node. I wanted to add them to a swarm so I could use Traefik for reverse proxy and HTTPS on both machines.
NUC:
1 docker-compose file for Traefik, Consul and Portainer.
1 docker-compose file for my media apps (Sabnzbd, Transmission-vpn, Sonarr, Radarr etc).
Pi:
1 docker-compose file for Home Assistant, MQTT etc.
Traefik and Portainer are up and running. I got them setup with `docker stack deploy`. Next I tried to setup my media apps, but they don't need to be connected with the Pi so I tried `docker compose`. Portainer shows the apps are running, but when I go to their subdomain Traefik returns 404 page not found. This makes me conclude that apps running outside the swarm, but connected to Traefik don't work. They also don't show up in the Traefik dashboard.
docker-compose.traefik.yml - 'docker stack deploy'
version: '3.7'
networks:
traefik_proxy:
external: true
agent-network:
attachable: true
volumes:
consul-data-leader:
consul-data-replica:
portainer-data:
services:
consul-leader:
image: consul
command: agent -server -client=0.0.0.0 -bootstrap -ui
volumes:
- consul-data-leader:/consul/data
environment:
- CONSUL_BIND_INTERFACE=eth0
- 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}'
networks:
- traefik_proxy
deploy:
labels:
- traefik.frontend.rule=Host:consul.${DOMAINNAME?Variable DOMAINNAME not set}
- traefik.enable=true
- traefik.port=8500
- traefik.tags=${TRAEFIK_PUBLIC_TAG:-traefik-public}
- traefik.docker.network=traefik_proxy
- traefik.frontend.entryPoints=http,https
- traefik.frontend.redirect.entryPoint=https
- traefik.frontend.auth.forward.address=http://oauth:4181
- traefik.frontend.auth.forward.authResponseHeaders=X-Forwarded-User
- traefik.frontend.auth.forward.trustForwardHeader=true
consul-replica:
image: consul
command: agent -server -client=0.0.0.0 -retry-join="consul-leader"
volumes:
- consul-data-replica:/consul/data
environment:
- CONSUL_BIND_INTERFACE=eth0
- 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true}'
networks:
- traefik_proxy
deploy:
replicas: ${CONSUL_REPLICAS:-3}
placement:
preferences:
- spread: node.id
traefik:
image: traefik:v1.7
hostname: traefik
restart: always
networks:
- traefik_proxy
ports:
- target: 80
published: 80
- target: 443
published: 443
- target: 8080
published: 8145
deploy:
replicas: ${TRAEFIK_REPLICAS:-3}
placement:
constraints:
- node.role == manager
preferences:
- spread: node.id
labels:
traefik.enable: 'true'
traefik.backend: traefik
traefik.protocol: http
traefik.port: 8080
traefik.tags: traefik-public
traefik.frontend.rule: Host:traefik.${DOMAINNAME}
traefik.frontend.headers.SSLHost: traefik.${DOMAINNAME}
traefik.docker.network: traefik_proxy
traefik.frontend.passHostHeader: 'true'
traefik.frontend.headers.SSLForceHost: 'true'
traefik.frontend.headers.SSLRedirect: 'true'
traefik.frontend.headers.browserXSSFilter: 'true'
traefik.frontend.headers.contentTypeNosniff: 'true'
traefik.frontend.headers.forceSTSHeader: 'true'
traefik.frontend.headers.STSSeconds: 315360000
traefik.frontend.headers.STSIncludeSubdomains: 'true'
traefik.frontend.headers.STSPreload: 'true'
traefik.frontend.headers.customResponseHeaders: X-Robots-Tag:noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex
traefik.frontend.headers.customFrameOptionsValue: 'allow-from https:${DOMAINNAME}'
traefik.frontend.auth.forward.address: 'http://oauth:4181'
traefik.frontend.auth.forward.authResponseHeaders: X-Forwarded-User
traefik.frontend.auth.forward.trustForwardHeader: 'true'
domainname: ${DOMAINNAME}
dns:
- 1.1.1.1
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ${USERDIR}/docker/traefik:/etc/traefik
- ${USERDIR}/docker/shared:/shared
environment:
CF_API_EMAIL: ${CLOUDFLARE_EMAIL}
CF_API_KEY: ${CLOUDFLARE_API_KEY}
command:
#- "storeconfig" #This is the push to consul, secondary traefik must be created and interfaced to this traefik. Remove this traefik's open ports, it shuts down once consul is messaged.
- '--logLevel=INFO'
- '--InsecureSkipVerify=true' #for unifi controller to not throw internal server error message
- '--api'
- '--api.entrypoint=apiport'
- '--defaultentrypoints=http,https'
- '--entrypoints=Name:http Address::80 Redirect.EntryPoint:https'
- '--entrypoints=Name:https Address::443 TLS TLS.SniStrict:true TLS.MinVersion:VersionTLS12 CipherSuites:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256'
- '--entrypoints=Name:apiport Address::8080'
- '--file'
- '--file.directory=/etc/traefik/rules/'
- '--file.watch=true'
- '--acme'
- '--acme.storage=/etc/traefik/acme/acme.json'
- '--acme.entryPoint=https'
# not yet ready?
# - "--acme.TLS-ALPN-01=true"
- '--acme.dnsChallenge=true'
- '--acme.dnsChallenge.provider=cloudflare'
- '--acme.dnsChallenge.delayBeforeCheck=60'
- '--acme.dnsChallenge.resolvers=1.1.1.1,1.0.0.1'
- '--acme.onHostRule=true'
- '--acme.email=admin#${DOMAINNAME}'
- '--acme.acmeLogging=true'
- '--acme.domains=${DOMAINNAME},*.${DOMAINNAME},'
- '--acme.KeyType=RSA4096'
#Let's Encrypt's staging server,
#caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
- '--docker'
- '--docker.swarmMode'
- '--docker.domain=${DOMAINNAME}'
- '--docker.watch'
- '--docker.exposedbydefault=false'
#- "--consul"
#- "--consul.endpoint=consul:8500"
#- "--consul.prefix=traefik"
- '--retry'
- 'resolvers=[192,168.1.1:53,1.1.1.1:53,]'
depends_on:
- consul-leader
docker-compose.media.yml - 'docker compose'
sabnzbd:
image: linuxserver/sabnzbd
container_name: sabnzbd
restart: always
network_mode: service:transmission-vpn
# depends_on:
# - transmission-vpn
# ports:
# - '${SABNZBD_PORT}:8080'
volumes:
- ${USERDIR}/docker/sabnzbd:/config
- /media/Data/Downloads:/Downloads
# - ${USERDIR}/Downloads/incomplete:/incomplete-downloads
environment:
PUID: ${PUID}
PGID: ${PGID}
TZ: ${TZ}
UMASK_SET: 002
deploy:
replicas: 1
labels:
traefik.enable: 'true'
traefik.backend: sabnzbd
traefik.protocol: http
traefik.port: 8080
traefik.tags: traefik_proxy
traefik.frontend.rule: Host:sabnzbd.${DOMAINNAME}
# traefik.frontend.rule: Host:${DOMAINNAME}; PathPrefix: /sabnzbd
traefik.frontend.headers.SSLHost: sabnzbd.${DOMAINNAME}
traefik.docker.network: traefik_proxy
traefik.frontend.passHostHeader: 'true'
traefik.frontend.headers.SSLForceHost: 'true'
traefik.frontend.headers.SSLRedirect: 'true'
traefik.frontend.headers.browserXSSFilter: 'true'
traefik.frontend.headers.contentTypeNosniff: 'true'
traefik.frontend.headers.forceSTSHeader: 'true'
traefik.frontend.headers.STSSeconds: 315360000
traefik.frontend.headers.STSIncludeSubdomains: 'true'
traefik.frontend.headers.STSPreload: 'true'
traefik.frontend.headers.customResponseHeaders: X-Robots-Tag:noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex
# traefik.frontend.headers.frameDeny: "true" #customFrameOptionsValue overrides this
traefik.frontend.headers.customFrameOptionsValue: 'allow-from https:${DOMAINNAME}'
traefik.frontend.auth.forward.address: 'http://oauth:4181'
traefik.frontend.auth.forward.authResponseHeaders: X-Forwarded-User
traefik.frontend.auth.forward.trustForwardHeader: 'true'
​
​
I already tried multiple things like removing the deploy command and just using labels etc but that didn't help at all. My Traefik logs also don't show anything that might be saying what's going wrong.
Are you running de .env file to set the environment variables? Because the feature .env is not currently supported by docker stack. You must source manually the .env running export $(cat .env) before running docker stack.

Traefik not detecting containers running in network_mode=host (192.168.99.x)

i am running traefik container in dockertoolbox with default network bridge
and one more container running on network_mode=host
but the traefik is detecting the service with 127.0.0.1 instead of DockerHost
IP= 192.168.99.x
can anyone help me with this
version: '3.7'
services:
reverse_proxy:
image: traefik
command: --api --docker --docker.domain=docker.localhost --logLevel=DEBUG
ports:
- "81:80"
- "8081:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- backend
whoami:
image: containous/whoami
labels:
- "traefik.frontend.rule=Host:whoami.localhost"
- "traefik.enable=true"
- "traefik.backend=whoami"
- "traefik.port=80"
network_mode: host
restart:
always
networks:
backend:
driver: bridge
NOTE: using dockertoolbox in windows 10

Resources