Traefik requests sub-domain specific certificate instead of wildcard? - docker

Question

I have the following docker-compose.yml file
version: "3.8"
secrets:
loopia_api_user:
file: "./traefik/secrets/loopia_api_user.secret"
loopia_api_password:
file: "./traefik/secrets/loopia_api_password.secret"
networks:
dockersocket:
driver: bridge
internal: true
traefik:
external: true # this network has to be created once before starting:
name: traefik # docker network create traefik
services:
docker-socket-proxy:
image: tecnativa/docker-socket-proxy
container_name: docker-socket-proxy
restart: unless-stopped
mem_limit: 128M
cpus: 0.5
networks:
- dockersocket
expose:
- 2375
environment:
CONTAINERS: 1
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
traefik:
image: "traefik:${TRAEFIK_VERSION}"
container_name: "traefik"
restart: unless-stopped
read_only: true
mem_limit: 2G
cpus: 0.75
security_opt:
- no-new-privileges:true
depends_on:
- docker-socket-proxy
secrets:
- "loopia_api_user"
- "loopia_api_password"
command:
- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
#- "--entrypoints.web.http.redirections.entryPoint.to=websecure" # Redirect http to https
#- "--entrypoints.web.http.redirections.entrypoint.scheme=https"
# Https configuration
- "--entrypoints.websecure.http.tls=true"
- "--entrypoints.websecure.http.tls.domains[0].main=${DOMAIN}"
- "--entrypoints.websecure.http.tls.domains[0].sans=*.${DOMAIN}"
- "--entrypoints.websecure.http.tls.certresolver=loopia"
# Lets encrypt Loopia dns challange
- "--certificatesresolvers.loopia.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.loopia.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53"
- "--certificatesresolvers.loopia.acme.dnschallenge=true"
- "--certificatesresolvers.loopia.acme.dnschallenge.provider=loopia"
- "--certificatesresolvers.loopia.acme.email=${POSTMASTER_EMAIL}"
- "--certificatesresolvers.loopia.acme.storage=/letsencrypt/acme.json"
# Use the docker socket proxy
- "--providers.docker.endpoint=tcp://docker-socket-proxy:2375" # using Docker Socket Proxy instead of docker socket for improved security
- "--providers.docker.network=traefik" # Defines a default docker network to use for connections to all containers.
# Logs
- --accesslog.filepath=/logs/access.log
- --accesslog.format=json
- --accesslog.fields.defaultMode=keep
- --accesslog.fields.headers.defaultMode=keep
- --log.filepath=/logs/traefik.log
environment:
- LOOPIA_API_USER_FILE=/run/secrets/loopia_api_user
- LOOPIA_API_PASSWORD_FILE=/run/secrets/loopia_api_password
- DOMAIN
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- ./traefik/logs:/logs
- ./traefik/dynamic_conf:/dynamic_conf:ro
- ./traefik/letsencrypt:/letsencrypt
networks:
- dockersocket
- traefik
whoami:
image: "traefik/whoami"
container_name: "simple-service"
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.${DOMAIN}`)"
- "traefik.http.routers.whoami.tls.certresolver=loopia"
- "traefik.http.routers.whoami.entrypoints=websecure"
networks:
- traefik
TRAEFIK_VERSION is v2.9.6.
When I start my containers traefik requests a certificate as expected but does so for whoami.${DOMAIN} instead of *.${DOMAIN} what am I missing?
I filtered the log somewhat so if any line is missing tell me and ill fetch it.
time="2023-01-29T21:12:23+01:00" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"websecure\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"tls\":{\"certResolver\":\"loopia\",\"domains\":[{\"main\":\"example.com\",\"sans\":[\"*.example.com\"]}]}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"tcp://docker-socket-proxy:2375\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"network\":\"traefik\",\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"directory\":\"/dynamic_conf/\",\"watch\":true}},\"api\":{\"insecure\":true,\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"filePath\":\"/logs/traefik.log\",\"format\":\"common\"},\"accessLog\":{\"filePath\":\"/logs/access.log\",\"format\":\"json\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"keep\"}}},\"certificatesResolvers\":{\"loopia\":{\"acme\":{\"email\":\"peter#example.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"loopia\",\"resolvers\":[\"1.1.1.1:53\",\"8.8.8.8:53\"]}}}}}"
time="2023-01-29T21:12:23+01:00" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-01-29T21:12:23+01:00" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-01-29T21:12:23+01:00" level=info msg="Starting provider *acme.Provider"
time="2023-01-29T21:12:23+01:00" level=debug msg="*acme.Provider provider configuration: {\"email\":\"peter#example.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"loopia\",\"resolvers\":[\"1.1.1.1:53\",\"8.8.8.8:53\"]},\"ResolverName\":\"loopia\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"middlewares\":{\"https-redirect\":{\"redirectScheme\":{\"scheme\":\"https\"}},\"local-only\":{\"ipWhiteList\":{\"sourceRange\":[\"127.0.0.1/32\",\"192.168.0.0/24\",\"172.20.0.0/24\"]}},\"securedheaders\":{\"headers\":{\"customResponseHeaders\":{\"X-Robots-Tag\":\"none,noarchive,nosnippet,notranslate,noimageindex,\",\"server\":\"\"},\"sslRedirect\":true,\"stsSeconds\":63072000,\"stsIncludeSubdomains\":true,\"stsPreload\":true,\"forceSTSHeader\":true,\"customFrameOptionsValue\":\"SAMEORIGIN\",\"contentTypeNosniff\":true,\"browserXssFilter\":true,\"referrerPolicy\":\"same-origin\",\"featurePolicy\":\"camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';\"}}}},\"tcp\":{},\"udp\":{},\"tls\":{\"options\":{\"default\":{\"minVersion\":\"VersionTLS13\",\"clientAuth\":{},\"sniStrict\":true,\"alpnProtocols\":[\"h2\",\"http/1.1\",\"acme-tls/1\"]},\"mintls13\":{\"minVersion\":\"VersionTLS13\",\"clientAuth\":{},\"sniStrict\":true,\"alpnProtocols\":[\"h2\",\"http/1.1\",\"acme-tls/1\"]}}}}" providerName=file
time="2023-01-29T21:12:23+01:00" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=info msg="Testing certificate renew..." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api#internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect#internal\",\"dashboard_stripprefix#internal\"],\"service\":\"dashboard#internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645},\"web-to-websecure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-websecure\"],\"service\":\"noop#internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}},\"redirect-web-to-websecure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"models\":{\"websecure\":{\"tls\":{\"certResolver\":\"loopia\",\"domains\":[{\"main\":\"example.com\",\"sans\":[\"*.example.com\"]}]}}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami-docker\",\"rule\":\"Host(`whoami.example.com`)\",\"tls\":{\"certResolver\":\"loopia\"}}},\"services\":{\"whoami-docker\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-01-29T21:12:23+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:12:23+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:12:23+01:00" level=debug msg="Adding route for whoami.example.com with TLS options default" entryPointName=websecure
time="2023-01-29T21:12:23+01:00" level=debug msg="Trying to challenge certificate for domain [whoami.example.com] found in HostSNI rule" providerName=loopia.acme routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:12:23+01:00" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.example.com\"]..." routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Domains [\"whoami.example.com\"] need ACME certificates generation for domains \"whoami.example.com\"." routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:23+01:00" level=debug msg="Loading ACME certificates [whoami.example.com]..." providerName=loopia.acme routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:12:24+01:00" level=debug msg="Building ACME client..." providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=debug msg="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=info msg=Register... providerName=loopia.acme
time="2023-01-29T21:12:24+01:00" level=debug msg="legolog: [INFO] acme: Registering account for peter#example.com"
time="2023-01-29T21:12:25+01:00" level=debug msg="Using DNS Challenge provider: loopia" providerName=loopia.acme
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Obtaining bundled SAN certificate"
time="2023-01-29T21:12:25+01:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami-docker\",\"rule\":\"Host(`whoami.example.com`)\",\"tls\":{\"certResolver\":\"loopia\"}}},\"services\":{\"whoami-docker\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.20.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/5153870843"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Could not find solver for: tls-alpn-01"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Could not find solver for: http-01"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: use dns-01 solver"
time="2023-01-29T21:12:25+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Preparing to solve DNS-01"
time="2023-01-29T21:12:26+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Trying to solve DNS-01"
time="2023-01-29T21:12:26+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Checking DNS record propagation using [1.1.1.1:53 8.8.8.8:53]"
time="2023-01-29T21:13:31+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Cleaning DNS-01 challenge"
time="2023-01-29T21:13:32+01:00" level=debug msg="legolog: [INFO] [whoami.example.com] acme: Validations succeeded; requesting certificates"
time="2023-01-29T21:13:33+01:00" level=debug msg="Certificates obtained for domains [whoami.example.com]" providerName=loopia.acme routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory"
time="2023-01-29T21:13:33+01:00" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=loopia.acme
time="2023-01-29T21:13:33+01:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-01-29T21:13:33+01:00" level=debug msg="Adding route for whoami.example.com with TLS options default" entryPointName=websecure
time="2023-01-29T21:13:33+01:00" level=debug msg="Trying to challenge certificate for domain [whoami.example.com] found in HostSNI rule" routerName=whoami#docker rule="Host(`whoami.example.com`)" ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme
time="2023-01-29T21:13:33+01:00" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.example.com\"]..." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme routerName=whoami#docker rule="Host(`whoami.example.com`)"
time="2023-01-29T21:13:33+01:00" level=debug msg="No ACME certificate generation required for domains [\"whoami.example.com\"]." ACME CA="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=loopia.acme routerName=whoami#docker rule="Host(`whoami.example.com`)"
acme.json
{
"loopia": {
"Account": {
"Email": "peter#example.com",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:peter#example.com"
]
},
"uri": "https://acme-staging-v02.api.letsencrypt.org/acme/acct/85276103"
},
"PrivateKey": "....",
"KeyType": "4096"
},
"Certificates": [
{
"domain": {
"main": "whoami.example.com"
},
"certificate": "....",
"key": "....",
"Store": "default"
}
]
}
}
Edit: I even tried to add to add a second certificate but that just generated 2 certificates instead.

Answer 1

Adding the following to the traefik container solved the issue
labels:
- "traefik.enable=true"
- 'traefik.http.routers.wildcard-certs.tls.certresolver=loopia'
- 'traefik.http.routers.wildcard-certs.tls.domains[0].main=${DOMAIN}'
- 'traefik.http.routers.wildcard-certs.tls.domains[0].sans=*.${DOMAIN}'
But is
- "--entrypoints.websecure.http.tls.domains[0].main=${DOMAIN}"
- "--entrypoints.websecure.http.tls.domains[0].sans=*.${DOMAIN}"
- "--entrypoints.websecure.http.tls.certresolver=loopia"
Incorrect? (most blog entries online use those commands)