How can I properly setup basic traefik reverse proxy? - docker

Question

Assume my current public IP is 101.15.14.71, I have a domain called example.com which I configured using cloudflare and I created multiple DNS entry pointing to my public ip.
Eg:
1) new1.example.com - 101.15.14.71
2) new2.example.com - 101.15.14.71
3) new3.example.com - 101.15.14.71
Now, Here's my example project structure,
├── myapp
│   ├── app
│   │   └── main.py
│   ├── docker-compose.yml
│   └── Dockerfile
├── myapp1
│   ├── app
│   │   └── main.py
│   ├── docker-compose.yml
│   └── Dockerfile
└── traefik
├── acme.json
├── docker-compose.yml
├── traefik_dynamic.toml
└── traefik.toml
Here I have two fastAPIs (i.e., myapp, myapp1)
Here's the example code I have in main.py in both myapp and myapp1, Its exactly same but return staement is different that's all
from fastapi import FastAPI
app = FastAPI()
#app.get("/")
def read_main():
return {"message": "Hello world for my project myapp"}
Here's my Dockerfile for myapp and myapp1, here too both are exactly same but the only difference is I deploy myapp on 7777 and myapp1 on 7778 in different containers
FROM ubuntu:latest
ARG DEBIAN_FRONTEND=noninteractive
RUN apt update && apt upgrade -y
RUN apt install -y -q build-essential python3-pip python3-dev
# python dependencies
RUN pip3 install -U pip setuptools wheel
RUN pip3 install gunicorn fastapi uvloop httptools "uvicorn[standard]"
# copy required files
RUN bash -c 'mkdir -p /app'
COPY ./app /app
ENTRYPOINT /usr/local/bin/gunicorn \
-b 0.0.0.0:7777 \ # this line I use for myapp dockerfile
-b 0.0.0.0:7778 \ # this line I change for myapp1 dockerfile
-w 1 \
-k uvicorn.workers.UvicornWorker app.main:app \
--chdir /app
Here's my docker-compose.yml file for myapp and myapp1, here also I have exactly same but only difference is I change the port,
services:
myapp: # I use this line for myapp docker-compose file
myapp1: # I use this line for myapp1 docker-compose file
build: .
restart: always
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik_public"
- "traefik.backend=myapp" # I use this line for myapp docker-compose file
- "traefik.backend=myapp1" # I use this line for myapp1 docker-compose file
- "traefik.frontend.rule=Host:new2.example.com" # I use this for myapp compose file
- "traefik.frontend.rule=Host:new3.example.com" # I use this for myapp1 compose file
- "traefik.port=7777" # I use this line for myapp docker-compose file
- "traefik.port=7778" # I use this line for myapp1 docker-compose file
networks:
- traefik_public
networks:
traefik_public:
external: true
Now coming to traefik folder,
acme.json # I created it using nano acme.json command with nothing in it,
but did chmod 600 acme.json for proper permissions.
traefik_dynamic.toml
[http]
[http.routers]
[http.routers.route0]
entryPoints = ["web"]
middlewares = ["my-basic-auth"]
service = "api#internal"
rule = "Host(`new1.example.com`)"
[http.routers.route0.tls]
certResolver = "myresolver"
[http.middlewares.test-auth.basicAuth]
users = [
["admin:your_encrypted_password"]
]
traefik.toml
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.web.http]
[entryPoints.web.http.redirections]
[entryPoints.web.http.redirections.entryPoint]
to = "websecure"
scheme = "https"
[entryPoints.websecure]
address = ":443"
[api]
dashboard = true
[certificatesResolvers.myresolver.acme]
email = "reallygoodtraefik#gmail.com"
storage= "acme.json"
[certificatesResolvers.myresolver.acme.httpChallenge]
entryPoint = "web"
[providers]
[providers.docker]
watch = true
network = "web"
[providers.file]
filename = "traefik_dynamic.toml"
docker-compose.yml
services:
traefik:
image: traefik:latest
ports:
- 80:80
- 443:443
- 8080:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
- ./traefik_dynamic.toml:/traefik_dynamic.toml
networks:
- web
networks:
web:
These are the details about my files, what I am trying to achieve here is,
I want to setup traefik and traefik dashboard with basic authentication, and I deploy two of my fastapi services,
myapp 7777, I need to access this app via new2.example.com
myapp1 7778, I need to access this app via new3.example.com
traefik dashboard, I need to access this via new1.example.com
All of these should be https and also has certification autorenew enabled.
I got all these from online articles for latest version of traefik. But the problem is this is not working. I used docker-compose to build and deploy the traefik and I open the api dashboard. It is asking for password and user (basic auth I setup) I entered my user details I setup in traefik_dynamic.toml but its not working.
Where did I do wrong? Please help me correcting mistakes in my configuration. I am really interested to learn more about this.
Error Update:
traefik_1 | time="2021-06-16T01:51:16Z" level=error msg="Unable to obtain ACME certificate for domains \"new1.example.com\": unable to generate a certificate for the domains [new1.example.com]: error: one or more domains had a problem:\n[new1.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new1.example.com/.well-known/acme-challenge/mu85LkYEjlvnbDI-wM2xMaRFO1QsPDNjepTDb47dWF0 [2606:4700:3032::6815:55c4]: 404\n" rule="Host(`new1.example.com`)" routerName=api#docker providerName=myresolver.acme
traefik_1 | time="2021-06-16T01:51:19Z" level=error msg="Unable to obtain ACME certificate for domains \"new2.example.com\": unable to generate a certificate for the domains [new2.example.com]: error: one or more domains had a problem:\n[new2.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new2.example.com/.well-known/acme-challenge/ykiCAEpJeQ1qgVdeFtSRo3q-ATTwgKdRdGHUs2kgIsY [2606:4700:3031::ac43:d1e9]: 404\n" providerName=myresolver.acme routerName=myapp1#docker rule="Host(`new2.example.com`)"
traefik_1 | time="2021-06-16T01:51:20Z" level=error msg="Unable to obtain ACME certificate for domains \"new3.example.com\": unable to generate a certificate for the domains [new3.example.com]: error: one or more domains had a problem:\n[new3.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new3.example.com/.well-known/acme-challenge/BUZWuWdNd2XAXwXCwkeqe5-PHb8cGV8V6UtzeLaKryE [2606:4700:3031::ac43:d1e9]: 404\n" providerName=myresolver.acme routerName=myapp#docker rule="Host(`new3.example.com`)"

Answer 1

You only need one docker-compose file for all the services, and there is no need to define one for each container.
The project structure you should be using should be something like:
├── docker-compose.yml
├── myapp
│   ├── .dockerignore
│   ├── Dockerfile
│   └── app
│   └── main.py
├── myapp1
│   ├── .dockerignore
│   ├── Dockerfile
│   └── app
│   └── main.py
└── traefik
├── acme.json
└── traefik.yml
When creating containers, unless they are to be used for development purposes, it is recommended to not use a full-blown image, like ubuntu. Specifically for your purposes I would recommend a python image, such as python:3.7-slim.
Not sure if you are using this for development or production purposes, but you could also use volumes to mount the app directories inside the containers (especially useful if you are using this for development), and only use one Dockerfile for both myapp and myapp1, customizing it via environment variables.
Since you are already using traefik's dynamic configuration, I will do most of the setup for the container configuration via docker labels in the docker-compose.yml file.
Your dockerfile for myapp and myapp1 will be very similar at this point, but I've kept them as seperate ones, since you may need to make changes to them depending on the requirements of your apps in the future. I've used an environment variable for the port, which can allow you to change the port from your docker-compose.yml file.
You can use the following Dockerfile (./myapp/Dockerfile and ./myapp1/Dockerfile):
FROM python:3.7-slim
ARG DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
RUN pip3 install -U pip setuptools wheel && \
pip3 install gunicorn fastapi uvloop httptools "uvicorn[standard]"
COPY . /app
ENV PORT=7777 # and 7778 for myapp1
ENTRYPOINT /usr/local/bin/gunicorn -b 0.0.0.0:$PORT -w 1 -k uvicorn.workers.UvicornWorker app.main:app --chdir /app
Note: you should really be using something like poetry or a requirements.txt file for your app dependencies.
The .dockerignore file (./myapp/.dockerignore and ./myapp1/.dockerignore) should contain:
Dockerfile
Since the whole directory is being copied inside the container and you don't need the Dockerfile to be in there.
Your main traefik config (./traefik/traefik.yml) can be something like:
providers:
docker:
exposedByDefault: false
global:
checkNewVersion: false
sendAnonymousUsage: false
api: {}
accessLog: {}
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: "websecure"
scheme: "https"
websecure:
address: ":443"
ping:
entryPoint: "websecure"
certificatesResolvers:
myresolver:
acme:
caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
email: "example#example.com"
storage: "/etc/traefik/acme.json"
httpChallenge:
entryPoint: "web"
Note: The above acme config will use the stage letsencrypt server. Make sure all the details are correct, and remove caServer after you've tested that everything works, in order to communicate with the letsencrypt production server.
Your ./docker-compose.yml file should be something like:
version: "3.9"
services:
myapp:
build:
context: ./myapp
dockerfile: ./Dockerfile
image: myapp
depends_on:
- traefik
expose:
- 7777
labels:
- "traefik.enable=true"
- "traefik.http.routers.myapp.tls=true"
- "traefik.http.routers.myapp.tls.certResolver=myresolver"
- "traefik.http.routers.myapp.entrypoints=websecure"
- "traefik.http.routers.myapp.rule=Host(`new2.example.com`)"
- "traefik.http.services.myapp.loadbalancer.server.port=7777"
myapp1:
build:
context: ./myapp1
dockerfile: ./Dockerfile
image: myapp1
depends_on:
- traefik
expose:
- 7778
labels:
- "traefik.enable=true"
- "traefik.http.routers.myapp1.tls=true"
- "traefik.http.routers.myapp1.tls.certResolver=myresolver"
- "traefik.http.routers.myapp1.entrypoints=websecure"
- "traefik.http.routers.myapp1.rule=Host(`new3.example.com`)"
- "traefik.http.services.myapp1.loadbalancer.server.port=7778"
traefik:
image: traefik:v2.4
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/traefik.yml:/etc/traefik/traefik.yml
- ./traefik/acme.json:/etc/traefik/acme.json
ports:
- 80:80
- 443:443
labels:
- "traefik.enable=true"
- "traefik.http.routers.api.tls=true"
- "traefik.http.routers.api.tls.certResolver=myresolver"
- "traefik.http.routers.api.entrypoints=websecure"
- "traefik.http.routers.api.rule=Host(`new1.example.com`)"
- "traefik.http.routers.api.service=api#internal"
- "traefik.http.routers.api.middlewares=myAuth"
- "traefik.http.middlewares.myAuth.basicAuth.users=admin:$$apr1$$4zjvsq3w$$fLCqJddLvrIZA.CCoGE2E." # generate with htpasswd. replace $ with $$
You can generate the password by using the command:
htpasswd -n admin | sed 's/\$/\$\$/g'
Note: If you need a literal dollar sign in the docker-compose file you need to use $$ as documented here.
Issuing docker-compose up in the directory should bring all the services up, and working as expected.
The above should work for you based on the details you have provided, but can be further improved at multiple points, depending on your needs.
Moreover, having the credentials for the traefik dashboard in the docker-compose.yml file is probably not the best, and you may want to use docker secrets for it. You can also add healthchecks and consider placing myapp and myapp1 into a seperate internal network.
If you want to get further into it, I propose that you start with Get started with Docker Compose and also read: Dockerfile reference and Compose file version 3 reference