Traefik ACME DNS challenge not working with docker - docker

Question

I'm trying to configure Traefik as a proxy for docker containers running on DigitalOcean servers.
Here's my Traefik container configuration:
version: '2'
services:
traefik:
image: traefik
restart: always
command: --docker
ports:
- 80:80
- 443:443
networks:
- proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- $PWD/traefik.toml:/traefik.toml
- $PWD/acme.json:/acme.json
container_name: traefik
environment:
DO_AUTH_TOKEN: abcd
labels:
- traefik.frontend.rule=Host:monitor.example.com
- traefik.port=8080
networks:
proxy:
external: true
And traefik.toml,
defaultEntryPoints = ["http", "https"]
[web]
address = ":8080"
[web.auth.basic]
users = ["admin:secretpassword"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[acme]
email = "lakshmi#example.com"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
onDemand = false
[acme.dnsChallenge]
provider = "digitalocean"
delayBeforeCheck = 0
When I try to access https://monitor.example.com, I get this error:
traefik | time="2018-05-29T15:35:32Z" level=error msg="Unable to obtain ACME certificate for domains \"monitor.example.com\" detected thanks to rule \"Host:monitor.example.com\" : cannot obtain certificates: acme: Error -> One or more domains had a problem:\n[monitor.example.com] Error presenting token: HTTP 403: forbidden: You do not have access for the attempted action.\n"
I have given a valid DO token and pointed monitor.example.com to the VM running Traefik. Am I missing any step?

Answer 1

I was getting a 403 because Traefik was trying to write a TXT entry for ACME DNS challenge in my DigitalOcean domain using a read-only token. I changed it to a read-write token and it worked fine.

Answer 2

For anyone else having this issue, make sure acme.json has 600 permissions. Don't create or touch acme.json yourself. Let Traefik create it. After the pod is created, check permissions on acme.json.
The problem I found is Traefik creates acme.json and sets it to 600. After running upgrade, acme.json changed to 660 and starting giving the 'unknown resolver letsencrypt' error. The fix was having to uncomment the 'initContainers' lines in the values.yml in the Traefik Helm chart. Basically it sets permissions to 600 before startup. Hacky but works.
deployment:
enabled: true
# Can be either Deployment or DaemonSet
kind: Deployment
replicas: 1
annotations: {}
labels: {}
podAnnotations: {}
podLabels: {}
additionalContainers: []
volumeMounts:
- name: csi-pvc
initContainers:
- name: volume-permissions
image: busybox:1.31.1
command: ["sh", "-c", "chmod -Rv 600 /data/*"]
volumeMounts:
- name: csi-pvc
mountPath: /data
dnsPolicy: ClusterFirstWithHostNet
imagePullSecrets: []