Traefik + cloudflared with full strict tunnel on docker - docker

I have a VM which run multiple containers all linked to one docker network.
Traefik (as reverse proxy & load balancer)
cloudflared as tunnel
whoami (for testing purposes)
and some containers like photoprism, nextcloud, node-red,...
I generated an origin cert via Cloudflare which has been added to Traefik.
In Cloudflare, I have a subdomain which points via the tunnel to https://172.16.10.11 (ip from the VM). This causes an unsecure connection (IP SAN applied -> I don't think this is possible on a private ip?). When I disable TLS verification on Cloudflare, it works. However, I am trying to set this up properly. Next,I tried pointing my domain towards https://localhost. the cloudflared service running in a container cannot reach any other services as these are located other containers.
I was thinking, what if I run the cloudflared service within the Traefik container, I believe I can reach Traefik via localhost?
Do you have any advice on how to achieve a secure tunnel with cert verification? Or is this not realistic when self-hosting?
Current docker compose:
version: '3'
services:
traefik:
image: traefik:latest
command:
- --log.level=debug
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --serverstransport.insecureskipverify
- --providers.file.filename=/etc/traefik/dynamic_conf.yml
- --providers.file.watch=true
ports:
- "8080:8080"
- "443:443"
- "80:80"
networks:
- proxy_network
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- traefik-data:/etc/traefik
labels:
- traefik.enable=true
- traefik.docker.network=proxy_network
- traefik.http.routers.traefik.rule=Host(`${DOMAINNAME_TRAEFIK}`)
- traefik.http.routers.traefik.entrypoints=web
- traefik.http.routers.traefik.service=traefik
- traefik.http.services.traefik.loadbalancer.server.port=8080
tunnel:
container_name: cloudflared-tunnel
image: cloudflare/cloudflared
#restart: unless-stopped
networks:
- proxy_network
command: tunnel --no-autoupdate run --token ${CLOUDFLARED_TOKEN}
whoami:
image: traefik/whoami
container_name: whoami1
command:
# It tells whoami to start listening on 2001 instead of 80
- --port=2000
- --name=iamfoo
networks:
- proxy_network
labels:
- traefik.enable=true
- traefik.http.routers.whoami.rule=Host(`${DOMAINNAME}`)
- traefik.http.routers.whoami.entrypoints=websecure
- traefik.http.routers.whoami.tls=true
- traefik.http.routers.whoami.service=whoami
- traefik.http.services.whoami.loadbalancer.server.port=2000
volumes:
traefik-data:
driver: local
networks:
proxy_network:
name: proxy_network
external: true
I expect a secure tunnel solution and to make sure that this architecture is setup in a good way.

Related

ioBroker with Node-RED behind traefik on Synlogy NAS with Docker

To control some Tasmota driven WiFi Sockets and some other stuff I want to install a Docker based SmartHome central on my Synology DS218+.
This installation is to be reachable only from inside my LAN while some other Docker containers on my NAS are accessible from the Internet.
So I decided to use a docker-compose Setup based on a Traefik-script with one single Traefik-container and a SmartHome-script with some SmartHome related containers (both scripts see below).
During a step-wise installation I first implemented the ioBroker container, finished the initial Setup and installed the Node-RED adapter.
After that I added a Mosquitto container to my SmartHome-script and a dependency to let the ioBroker container start after Mosquitto.
All containers of the above setup come up without any problems but ioBroker is the only Service that's accessible.
Whether my Tasmota-deices nor ioBroker seem to have access to Mosquitto and when I try to start the Node-RED-instance, I get an Error "404 page not found"
Traefik-script:
version: "3.9"
services:
traefik:
image: traefik:v2.4
command:
- --log.level=ERROR
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=web-secure
- --entrypoints.web.http.redirections.entrypoint.scheme=https
- --entrypoints.web-secure.address=:443
- --entrypoints.web-secure.http.tls.certresolver=lets-encrypt
- --entrypoints.something.address=:1234
...
- --entrypoints.node-red.address=:1880
- --entrypoints.mosquitto.address=:1883
- --entrypoints.iobroker.address=:8081
...
- --entrypoints.something-different.address=:23456
- --entrypoints.something-different.http.redirections.entrypoint.to=something-different
- --entrypoints.something-different.http.redirections.entrypoint.scheme=https
- --providers.docker=true
- --providers.docker.endpoint=unix:///var/run/docker.sock
- --providers.file.directory=/etc/traefik/dynamic/
- --providers.file.watch=true
- --certificatesresolvers.lets-encrypt.acme.email=my.email#internet.com
- --certificatesresolvers.lets-encrypt.acme.storage=/etc/traefik/acme.json
- --certificatesresolvers.lets-encrypt.acme.tlschallenge=true
restart:
- unless-stopped
ports:
- 80:80
- 443:443
- 1234:1234
...
- 1880:1880
- 1883:1883
- 8081:8081
...
- 23456:23456
volumes:
- /etc/localtime:/etc/localtime:ro
- ${PWD}/traefik:/etc/traefik
- /var/run/docker.sock:/var/run/docker.sock:ro
labels:
- traefik.enable=false
networks:
- traefik
networks:
traefik:
external: false
driver: bridge
name: traefik
SmartHome-script
version: "3.9"
services:
mosquitto:
image: eclipse-mosquitto:latest
restart:
- unless-stopped
volumes:
- ${PWD}/mosquitto-config:/mosquitto/config
- ${PWD}/mosquitto-data:/mosquitto/data
- ${PWD}/mosquitto-log:/mosquitto/log
labels:
- traefik.enable=true
- traefik.tcp.routers.mosquitto.entrypoints=mosquitto
- traefik.tcp.routers.mosquitto.rule=HostSNI(`my.synology.nas.local`)
- traefik.tcp.routers.mosquitto.service=svc-mosquitto
- traefik.tcp.services.svc-mosquitto.loadbalancer.server.port=1883
networks:
- traefik
iobroker:
image: iobroker/iobroker:latest
restart:
- unless-stopped
depends_on:
- mosquitto
environment:
- LANG=de_DE.UTF‑8
- LANGUAGE=de_DE:de
- LC_ALL=de_DE.UTF-8
- TZ=Europe/Berlin
volumes:
- ${PWD}/iobroker-data:/opt/iobroker
labels:
- traefik.enable=true
- traefik.http.routers.iobroker.entrypoints=iobroker
- traefik.http.routers.iobroker.rule=Host(`my.synology.nas.local`)
networks:
- traefik
networks:
traefik:
external: true
I suspect that the inaccessible Mosquitto-server is related to the "labels" section of the Mosquitto-container, because this is the first time I try to use TCP routing.
The inaccessible Node-RED instance within ioBroker might be related to using more than one HTTP-port with this container but I have no idea, where to begin troubleshooting.
What's the "correct way" to handle such use cases in docker-compose scripts respectively in Traefik?
Thanx in advance for your hints!
Lanzi

Mailcow Reverse Proxy using Traefik not routing to the correct Nginx Service

I am trying to follow the community documentation on mailcow dockerized and I am using Traefik as my load balancer.
I have successfully obtained a SSL certificate and the certdump is working as expected when I check the logs.
The issue I have having is that the nginx-mailcow container is not receiving the requests when I visit mail.example.com. My Traefik logs show this:
level=error msg="entryPoint \"secure\" doesn't exist" routerName=moo#docker entryPointName=secure
level=error msg="no valid entryPoint for this router" routerName=moo#docker
My docker-compose.override.yml looks like this (not much different to the community documentation):
version: '2.1'
services:
nginx-mailcow:
networks:
traefik:
web:
labels:
- traefik.enable=true
- traefik.http.routers.moo.rule=Host(`${MAILCOW_HOSTNAME}`)
- traefik.http.routers.moo.tls=true
- traefik.http.routers.moo.tls.certresolver=godaddy
- traefik.http.routers.moo.middlewares=redirect#file
- traefik.http.routers.moo.service=nginx-mailcow
- traefik.http.services.moo.loadBalancer.passHostHeader=true
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.headers.customrequestheaders.X-Forwarded-Proto=https
- traefik.http.routers.moo.middlewares=https-redirect
- traefik.http.services.moo.loadbalancer.server.port=80
- traefik.http.routers.moo.entrypoints=secure
- traefik.docker.network=web
certdumper:
image: humenius/traefik-certs-dumper
container_name: traefik_certdumper
network_mode: none
volumes:
- acme:/traefik:ro
- ./data/assets/ssl/:/output:rw
environment:
- DOMAIN=${MAILCOW_HOSTNAME}
networks:
traefik:
external: true
web:
external: true
volumes:
acme:
name: "traefik_acme"
Can anyone see what I am doing wrong?
I have also tried with only:
labels:
- traefik.enable=true
- traefik.http.routers.moo.rule=Host(`${MAILCOW_HOSTNAME}`)
- traefik.http.routers.moo.tls=true
- traefik.http.routers.moo.tls.certresolver=godaddy
- traefik.http.services.moo.loadbalancer.server.port=80
- traefik.http.routers.moo.entrypoints=secure
- traefik.docker.network=web
This still did not work.
When you define a name for a service, you must use the same name in your service configuration, like this:
traefik.http.routers.moo.service=nginx-mailcow
traefik.http.services.moonginx-mailcow.loadBalancer.passHostHeader=true
The loadBalancer.servers (notice the s in servers) doesn't have a port key, only url:
traefik.http.services.moo.loadbalancer.server.port=80
traefik.http.services.nginx-mailcow.loadbalancer.servers.url=['http://nginx-mailcow:80']
But as you are using defaults, you can omit all of the above =)
One more thing, I don't know how your Traefik container is configured but if your Traefik is running with a defined traefik network (internal) and a web network (public), you should use the traefik network in your Mailcow container as you want to route all external traffic through Traefik.
traefik.docker.network=web
labels:
- traefik.enable=true
- traefik.http.routers.moo.rule=Host(`${MAILCOW_HOSTNAME}`)
- traefik.http.routers.moo.tls=true
- traefik.http.routers.moo.tls.certresolver=godaddy
- traefik.http.routers.moo.middlewares=redirect#file
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.headers.customrequestheaders.X-Forwarded-Proto=https
- traefik.http.routers.moo.middlewares=https-redirect
- traefik.http.routers.moo.entrypoints=secure
- traefik.docker.network=traefik
#30daysofstackoverflow

Traefik not updating when services are deployed to docker swarm

I have a traefik environment running in docker. Originally I was running services in standard containers. I am not deploying containers to docker swarm and I have done this for traefik too, where the container is only deployed to my swarm manager.
For some reason, traefik successfully registers the host name I have given it, and can access that fine.
However, when I deploy any other service to the swarm, traefik doesn't pick it up.
There is one other service that has partially worked. I have deployed heimdall to docker swarm which can be access from gateway.docker.swarm:8091 but I don't want the port either.
My traefik compose file is as follows:
version: '3.3'
networks:
swarm-network:
driver: overlay
services:
traefik:
# The official v2 Traefik docker image
image: traefik
deploy:
placement:
constraints:
- node.role == manager
labels:
- "traefik.enable=true"
- "traefik.docker.network=pi_swarm-network"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.services.traefik.loadbalancer.server.port=8080"
- "traefik.http.routers.traefik.rule=Host(`traefik.docker.swarm`)"
# Enables the web UI and tells Traefik to listen to docker
command:
- '--api.insecure=true'
- '--providers.docker=true'
- '--providers.docker.swarmmode=true'
- '--providers.docker.defaultRule=Host("docker.swarm")'
- '--providers.docker.watch=true'
- '--providers.docker.swarmModeRefreshSeconds=15s'
# Metrics configuration for influx db.
- '--metrics=true'
- '--metrics.influxdb=true'
- '--metrics.influxdb.address=192.168.8.122:8086'
- '--metrics.influxdb.protocol=http'
- '--metrics.influxdb.database=traefik'
- '--metrics.influxdb.addEntryPointsLabels=true'
- '--metrics.influxdb.addServicesLabels=true'
- '--metrics.influxdb.pushInterval=10s'
# Tracing
- '--tracing=true'
- '--tracing.zipkin=true'
- '--tracing.zipkin.httpEndpoint=http://192.168.8.117:9411/api/v2/spans'
- '--log'
- '--accesslog'
ports:
# The HTTP port
- "80:80"
# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
networks:
- swarm-network
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
An example of another service I am running is heimdall which has the compose file of the following:
version: "3"
networks:
swarm-network:
external:
name: pi_swarm-network
services:
heimdall:
image: ghcr.io/linuxserver/heimdall
environment:
- PUID=1000
- PGID=1000
- TZ=Europe/London
deploy:
placement:
constraints:
- node.labels.tier == web
labels:
- "traefik.enable=true"
- "traefik.docker.network=pi_swarm-network"
- "traefik.http.routers.heimdall.entrypoints=http"
- "traefik.http.services.heimdall.loadbalancer.server.port=8091"
- "traefik.http.routers.heimdall.rule=Host(`gateway.docker.swarm`)"
ports:
- 8091:80
restart: unless-stopped
networks:
- swarm-network
Can anyone see what I'm doing wrong?
I have figured out the problem.
In my compose file, I was using "traefik.http.services.heimdall.loadbalancer.entrypoints=http"
as well as
"traefik.http.routers.heimdall.entrypoints=http"
this was incorrect and needed to just be
"traefik.http.routers.heimdall.entrypoints=http"
For heimdall, I was also targetting the external port of 8091, whereas I actually needed to target the internal port of 80

Is it possible to share ports between NGINX and Docker Container?

I have Ubuntu 18:04/NGINX VPS where I have a bunch of Laravel project blocks, all use ssl (certbot).
I wanted to deploy Nextcloud via Docker Compose on the same VPS:
version: "3"
services:
proxy:
image: jwilder/nginx-proxy:alpine
labels:
# labels needed by lets encrypt to identify container to generate certs in
- "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true"
container_name: nextcloud-proxy
networks:
- nextcloud_network
ports:
- 80:80
- 443:443
volumes:
- ./proxy/conf.d:/etc/nginx/conf.d:rw
- ./proxy/vhost.d:/etc/nginx/vhost.d:rw
- ./proxy/html:/usr/share/nginx/html:rw
- ./proxy/certs:/etc/nginx/certs:ro
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: unless-stopped
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion:v1.12.1
container_name: nextcloud-letsencrypt
depends_on:
- proxy
networks:
- nextcloud_network
volumes:
- ./proxy/certs:/etc/nginx/certs:rw
- ./proxy/vhost.d:/etc/nginx/vhost.d:rw
- ./proxy/html:/usr/share/nginx/html:rw
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
restart: unless-stopped
db:
image: mariadb:10.5.1
container_name: nextcloud-mariadb
networks:
- nextcloud_network
volumes:
- ./db:/var/lib/mysql
- ./dbdumps:/var/dbdumps
- /etc/localtime:/etc/localtime:ro
environment:
- MYSQL_ROOT_PASSWORD=... # set me
- MYSQL_PASSWORD=... # set me
- MYSQL_DATABASE=... # set me
- MYSQL_USER=... # set me
restart: unless-stopped
redis:
container_name: nextcloud-redis
image: redis:5.0.8
restart: unless-stopped
networks:
- nextcloud_network
volumes:
- ./redis/data:/data
command: ["redis-server", "--appendonly yes"]
app:
image: nextcloud:18.0.2
container_name: nextcloud-app
networks:
- nextcloud_network
depends_on:
- letsencrypt
- proxy
- redis
- db
volumes:
- ./nextcloud:/var/www/html
- ./app/config:/var/www/html/config
- ./app/custom_apps:/var/www/html/custom_apps
- ./app/data:/var/www/html/data
- ./app/themes:/var/www/html/themes
- /etc/localtime:/etc/localtime:ro
environment:
- VIRTUAL_HOST=YOURDOMAINHERE # set me
- LETSENCRYPT_HOST=YOURDOMAINHERE # set me
- LETSENCRYPT_EMAIL=you#example.com # set me
restart: unless-stopped
networks:
nextcloud_network:
driver: bridge
When I run this I get:
ERROR: for 3f210d699b80_nextcloud-proxy Cannot start service proxy: driver failed programming
external connectivity on endpoint nextcloud-proxy
(2d76e425c94abb95da70a7d903bf8830d4e9192a512e17db1b39f76da85c7b97): Error starting userland proxy:
listen tcp 0.0.0.0:443: bind: address already in use
ERROR: for proxy Cannot start service proxy: driver failed programming external connectivity on
endpoint nextcloud-proxy (2d76e425c94abb95da70a7d903bf8830d4e9192a512e17db1b39f76da85c7b97): Error
starting userland proxy: listen tcp 0.0.0.0:443: bind: address already in use
ERROR: Encountered errors while bringing up the project.
Because this port is already in use.
If I stop NGINX on VPS and run docker-compose up -d again, everything is ok and Nextcloud service is accessible via URL.
I tried to change outside ports to
- 8080:80
- 4444:443
And rebuild it. Then I don't see the above error but everything is messed up - the url point to wrong domain...
Is it possible to tweak the proxy container settings somehow to resolve this?
2 services are unable to listen to the same port as you have found. Your laravel applications are already listening on ports 80/443, so when start your nextcloud containers, it won't be able to bind to those ports.
You'll have to have your jwilder/nginx-proxy:alpine act as a proxy to both the nextcloud container and the laravel servers. This can be done via your nginx configurations and mount it to your container (which you seem to be using the ./proxy/ directory):
https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/
Although, if your VPS is able to have 2 IP addresses, then you are able to bind the laravel applications to one interface and your nextcloud proxy to the other which will also solve your problem. The first method is better practice as would allow you to scale your server better without having to add another IP address per-application.
https://docs.docker.com/config/containers/container-networking/

Docker Swarm + Traefik: Expose Traefik GUI through frontend rule; Service / Container port redirection

I am trying to use Traefik with Docker Swarm backend, and I am using the stack file below:
version: "3"
services:
traefik:
image: traefik:1.5
command: --web --docker --docker.swarmmode --docker.watch --docker.domain=sample.com --logLevel=DEBUG
deploy:
placement:
constraints: [node.role==manager]
restart_policy:
condition: on-failure
labels:
- "traefik.port=8080"
- "traefik.docker.network=sample-network"
- "traefik.frontend.rule=Host:traefik.sample.com"
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/null:/traefik.toml
networks:
- sample-network
portainer:
image: portainer/portainer:latest
command: --no-auth -H unix:///var/run/docker.sock
deploy:
placement:
constraints: [node.role == manager]
labels:
- "traefik.portainer.port=7777"
- "traefik.docker.network=sample-network"
- "traefik.frontend.rule=Host:portainer.sample.com"
ports:
- "7777:9000"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- sample-network
networks:
sample-network:
I have 2 questions that I am trying to wrap my head around:
1) (Exposing Traefik dashboard through frontend rule) I can access Traefik's web dashboard on sample.com:8080, yet I cannot access it through traefik.sample.com.
2) (Port Redirection on containers/services) I can access Portainer GUI through sample.com:7777, yet I cannot access it through portainer.sample.com. I am more curious of port redirection, because how will I setup 2 services in a single stack file if I encounter 2 images publishing to the same port? My service label declarations will clash at traefik.port=XXXX
You don´t need the traefik labels on the traefik service itself. It´s accessed from the outside over the specified ports:
ports:
- "80:80"
- "8080:8080"
- "443:443"
On the portainer service you don´t need the port mappings because you probably want to route the request with traefik.
Because traefik and portainer are in the same docker network traefik can access portainer on every port.
Therefore the port for traefik have to match the real portainer port:
labels:
- "traefik.port=9000"
- "traefik.docker.network=sample-network"
- "traefik.frontend.rule=Host:portainer.sample.com"
In the current setup you have to request traefik with Host:portainer.sample.com.
You can test it with
curl --verbose --header 'Host: portainer.sample.com' 'http://<DockerhostIp>:80'
Edit: Updated curl
Edit 2: Reaction to the edit of PO
The portainer.sample.com DNS entry will have to point to your docker host. Then traefik will route it to the correct container.
An alternative is to specifiy a traefik prefix:
"traefik.frontend.rule=Host:site1.org;PathPrefixStrip: /sub/"
With the rule all requests on site1.org/sub will routed to this specific service/container.
Have a look at
Traefik-Docker-Sample
Edit 3:
The self route for the dashboard/webui should work with:
labels:
- "traefik.port=8080"
- "traefik.docker.network=sample-network"
- "traefik.frontend.rule=Host:traefik.sample.com"
Just be sure that you have a DNS entry for traefik.sample.com.
To check if the traefik setup works you can also run
curl --verbose -H Host:traefik.sample.com <DockerHostIp>

Resources