multiple docker compose files with traefik (v2.1) and database networks - docker

I would like to build a docker landscape. I use a container with a traefik (v2. 1) image and a mysql container for multiple databases.
traefik/docker-compose.yml
version: "3.3"
services:
traefik:
image: "traefik:v2.1"
container_name: "traefik"
restart: always
command:
- "--log.level=DEBUG"
- "--api=true"
- "--api.dashboard=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.network=proxy"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--entrypoints.traefik-dashboard.address=:8080"
- "--certificatesresolvers.devnik-resolver.acme.httpchallenge=true"
- "--certificatesresolvers.devnik-resolver.acme.httpchallenge.entrypoint=web"
#- "--certificatesresolvers.devnik-resolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.devnik-resolver.acme.email=####"
- "--certificatesresolvers.devnik-resolver.acme.storage=/letsencrypt/acme.json"
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- "./letsencrypt:/letsencrypt"
- "./data:/etc/traefik"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
- "proxy"
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.rule=Host(`devnik.dev`)"
- "traefik.http.routers.traefik.entrypoints=traefik-dashboard"
- "traefik.http.routers.traefik.tls.certresolver=devnik-resolver"
#basic auth
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.usersfile=/etc/traefik/.htpasswd"
#Docker Networks
networks:
proxy:
database/docker-compose.yml
version: "3.3"
services:
#MySQL Service
mysql:
image: mysql:5.7
container_name: mysql
restart: always
ports:
- "3306:3306"
volumes:
#persist data
- ./mysqldata/:/var/lib/mysql/
- ./init:/docker-entrypoint-initdb.d
networks:
- "mysql"
environment:
MYSQL_ROOT_PASSWORD: ####
TZ: Europe/Berlin
#Docker Networks
networks:
mysql:
driver: bridge
For the structure I want to control all projects via multiple docker-compose files. These containers should run on the same network as the traefik container and some with the mysql container.
This also works for the following case (but only sometimes)
dev-releases/docker-compose.yml
version: "3.3"
services:
backend:
image: "registry.gitlab.com/devnik/dev-releases-backend/master:latest"
container_name: "dev-releases-backend"
restart: always
volumes:
#laravel logs
- "./logs/backend:/app/storage/logs"
#cron logs
- "./logs/backend/cron.log:/var/log/cron.log"
labels:
- "traefik.enable=true"
- "traefik.http.routers.dev-releases-backend.rule=Host(`dev-releases.backend.devnik.dev`)"
- "traefik.http.routers.dev-releases-backend.entrypoints=websecure"
- "traefik.http.routers.dev-releases-backend.tls.certresolver=devnik-resolver"
networks:
- proxy
- mysql
environment:
TZ: Europe/Berlin
#Docker Networks
networks:
proxy:
external:
name: "traefik_proxy"
mysql:
external:
name: "database_mysql"
As soon as I restart the containers in dev-releases/ via docker-compose up -d I get the typical error "Gateway timeout" when calling them in the browser.
As soon as I comment the network networks: #- mysql and restart the docker-compose in dev-releases/ it works again.
My guess is that I have not configured the external networks correctly. Is it not possible to use 2 external networks?
I'd like some container have access to the 'mysql' network but it should not be accessible for the whole traefik network.
Let me know if you need more information
EDIT (26.03.2020)
I make it running.
I put all my containers into one network "proxy". It seems mysql also have to be in the proxy network.
So I add following to database/docker-compose.yml
networks:
proxy:
external:
name: "traefik_proxy"
And removed the database_mysql network out of dev-releases/docker-compose.yml

based on the names of the files, your mysql network should be mysql_mysql.
you can verify this by executing
$> docker network ls
You are also missing a couple of labels for your services such as
traefik command line
- '--providers.docker.watch=true'
- '--providers.docker.swarmMode=true'
labels
- traefik.docker.network=proxy
- traefik.http.services.dev-releases-backend.loadbalancer.server.port=yourport
- traefik.http.routers.dev-releases-backend.service=mailcatcher
You can check this for more info

Related

ioBroker with Node-RED behind traefik on Synlogy NAS with Docker

To control some Tasmota driven WiFi Sockets and some other stuff I want to install a Docker based SmartHome central on my Synology DS218+.
This installation is to be reachable only from inside my LAN while some other Docker containers on my NAS are accessible from the Internet.
So I decided to use a docker-compose Setup based on a Traefik-script with one single Traefik-container and a SmartHome-script with some SmartHome related containers (both scripts see below).
During a step-wise installation I first implemented the ioBroker container, finished the initial Setup and installed the Node-RED adapter.
After that I added a Mosquitto container to my SmartHome-script and a dependency to let the ioBroker container start after Mosquitto.
All containers of the above setup come up without any problems but ioBroker is the only Service that's accessible.
Whether my Tasmota-deices nor ioBroker seem to have access to Mosquitto and when I try to start the Node-RED-instance, I get an Error "404 page not found"
Traefik-script:
version: "3.9"
services:
traefik:
image: traefik:v2.4
command:
- --log.level=ERROR
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=web-secure
- --entrypoints.web.http.redirections.entrypoint.scheme=https
- --entrypoints.web-secure.address=:443
- --entrypoints.web-secure.http.tls.certresolver=lets-encrypt
- --entrypoints.something.address=:1234
...
- --entrypoints.node-red.address=:1880
- --entrypoints.mosquitto.address=:1883
- --entrypoints.iobroker.address=:8081
...
- --entrypoints.something-different.address=:23456
- --entrypoints.something-different.http.redirections.entrypoint.to=something-different
- --entrypoints.something-different.http.redirections.entrypoint.scheme=https
- --providers.docker=true
- --providers.docker.endpoint=unix:///var/run/docker.sock
- --providers.file.directory=/etc/traefik/dynamic/
- --providers.file.watch=true
- --certificatesresolvers.lets-encrypt.acme.email=my.email#internet.com
- --certificatesresolvers.lets-encrypt.acme.storage=/etc/traefik/acme.json
- --certificatesresolvers.lets-encrypt.acme.tlschallenge=true
restart:
- unless-stopped
ports:
- 80:80
- 443:443
- 1234:1234
...
- 1880:1880
- 1883:1883
- 8081:8081
...
- 23456:23456
volumes:
- /etc/localtime:/etc/localtime:ro
- ${PWD}/traefik:/etc/traefik
- /var/run/docker.sock:/var/run/docker.sock:ro
labels:
- traefik.enable=false
networks:
- traefik
networks:
traefik:
external: false
driver: bridge
name: traefik
SmartHome-script
version: "3.9"
services:
mosquitto:
image: eclipse-mosquitto:latest
restart:
- unless-stopped
volumes:
- ${PWD}/mosquitto-config:/mosquitto/config
- ${PWD}/mosquitto-data:/mosquitto/data
- ${PWD}/mosquitto-log:/mosquitto/log
labels:
- traefik.enable=true
- traefik.tcp.routers.mosquitto.entrypoints=mosquitto
- traefik.tcp.routers.mosquitto.rule=HostSNI(`my.synology.nas.local`)
- traefik.tcp.routers.mosquitto.service=svc-mosquitto
- traefik.tcp.services.svc-mosquitto.loadbalancer.server.port=1883
networks:
- traefik
iobroker:
image: iobroker/iobroker:latest
restart:
- unless-stopped
depends_on:
- mosquitto
environment:
- LANG=de_DE.UTF‑8
- LANGUAGE=de_DE:de
- LC_ALL=de_DE.UTF-8
- TZ=Europe/Berlin
volumes:
- ${PWD}/iobroker-data:/opt/iobroker
labels:
- traefik.enable=true
- traefik.http.routers.iobroker.entrypoints=iobroker
- traefik.http.routers.iobroker.rule=Host(`my.synology.nas.local`)
networks:
- traefik
networks:
traefik:
external: true
I suspect that the inaccessible Mosquitto-server is related to the "labels" section of the Mosquitto-container, because this is the first time I try to use TCP routing.
The inaccessible Node-RED instance within ioBroker might be related to using more than one HTTP-port with this container but I have no idea, where to begin troubleshooting.
What's the "correct way" to handle such use cases in docker-compose scripts respectively in Traefik?
Thanx in advance for your hints!
Lanzi

Traefik 2 network between 2 containers results in Gateway Timeout errors

I'm trying to set up 2 docker containers with docker-compose, 1 is a Traefik proxy and the other is a Vikunja kanban board container.
They both have their own docker-compose file. I can start the containers and the Traefik dashboard doesn't show any issues but when I open the URL in a browser I only get a Gateway Timeout error.
I have been looking at similar questions on here and different platforms and in nearly all other cases the issue was that they were placed on 2 different networks. However, I added a networks directive to the Traefik docker-compose.yml and still have this problem, unless I'm using them wrong.
The docker-compose file for the Vikunja container
(adapted from https://vikunja.io/docs/full-docker-example/)
version: '3'
services:
api:
image: vikunja/api
environment:
VIKUNJA_DATABASE_HOST: db
VIKUNJA_DATABASE_PASSWORD: REDACTED
VIKUNJA_DATABASE_TYPE: mysql
VIKUNJA_DATABASE_USER: vikunja
VIKUNJA_DATABASE_DATABASE: vikunja
VIKUNJA_SERVICE_JWTSECRET: REDACTED
VIKUNJA_SERVICE_FRONTENDURL: REDACTED
volumes:
- ./files:/app/vikunja/files
networks:
- web
- default
depends_on:
- db
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-api.rule=Host(`subdomain.domain.de`) && PathPrefix(`/api/v1`, `/dav/`, `/.well-known/`)"
- "traefik.http.routers.vikunja-api.entrypoints=websecure"
- "traefik.http.routers.vikunja-api.tls.certResolver=myresolver"
frontend:
image: vikunja/frontend
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-frontend.rule=Host(`subdomain.domain.de`)"
- "traefik.http.routers.vikunja-frontend.entrypoints=websecure"
- "traefik.http.routers.vikunja-frontend.tls.certResolver=myresolver"
networks:
- web
- default
restart: unless-stopped
db:
image: mariadb:10
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
environment:
MYSQL_ROOT_PASSWORD: REDACTED
MYSQL_USER: vikunja
MYSQL_PASSWORD: REDACTED
MYSQL_DATABASE: vikunja
volumes:
- ./db:/var/lib/mysql
restart: unless-stopped
command: --max-connections=1000
networks:
- web
networks:
web:
external: true
The network directives for the api and frontend services in the Vikunja docker-compose.yml were present in the template (I added one for the db service for testing but it didn't have any effect).
networks:
- web
After getting a docker error about the network not being found I created it via docker network create web
The docker-compose file for the Traefik container
version: '3'
services:
traefik:
image: traefik:v2.8
ports:
- "80:80"
- "443:443"
- "8080:8080" # dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./letsencrypt:/letsencrypt
- ./traefik.http.yml:/etc/traefik/traefik.yml
networks:
- web
networks:
web:
external: true
I've tried adding the Traefik service to the Vikunja docker-compose.yml in one file but that didn't have any effect either.
I'm thankful for any pointers.
For debugging you could try to configure all container to use the host network to enusre they are realy on the same netwok.
i had a similar issue trying to run two different dockers and getting a
"Gateway Timeout". My issue was solved after changing the mapping in the second docker for traefik and accessing the site with :84 at the end (http://sitename:84)
traefik:
image: traefik:v2.0
container_name: "${PROJECT_NAME}_traefik"
command: --api.insecure=true --providers.docker
ports:
- '84:80'
- '8084:8080'

docker compose: restrict internet access

I want to run with a container that is a copy of a production container, so I want to restrict access to the internet to prevent that call production servers.
But I need to access the container from the host machine with internet access
This is what I am trying to do:
version: '2.1'
services:
proxy:
image: traefik
command: --api.insecure=true --providers.docker
networks:
- no-internet
- internet
ports:
- "80:80"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
prod-service:
image: ....
depends_on:
- db
ports:
- "8094:8094"
labels:
- "traefik.http.routers.blog.rule=Host(`localhost`)"
- "traefik.port=8094"
networks:
- no-internet
db:
container_name: db
image: postgres:11
hostname: ap-db
expose:
- 5433
ports:
- 5433:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
networks:
- no-internet
- internet
networks:
internet:
driver: bridge
no-internet:
internal: true
driver: bridge
But the trafic configuration is not working for me.
What is the best option to do this?
the answers I found do not take into account the access from the host machine, the container without internet is isolated
I appreciate any advice

Jira & Docker & Traefik Setup

I'm first time Traefik user and I successfully configured this docker compose setup for Jira with Traefik and Let's Encrypt Cert.
My problem is that Jira must be able to connect to his self. Their are some Jira Services like Gadgets that loads it's data via JavaScript from via his own address over http. This typ of service does not work for me. Their is a support documents that describes this problems and also shows solutions for this. But I don't know how to setup this up correctly with Traefik/Docker. https://confluence.atlassian.com/jirakb/how-to-fix-gadget-titles-showing-as-__msg_gadget-813697086.html
Your help would be great. Thanks a lot!
version: '3'
services:
reverse-proxy:
image: traefik # The official Traefik docker image
command: --docker # Enables the web UI and tells Traefik to listen to docker --api
ports:
- "80:80" # The HTTP port
- "443:443" # The HTTPS port
- "8081:8080" # The Web UI (enabled by --api)
hostname: traefik
restart: unless-stopped
domainname: ${DOMAINNAME}
networks:
- frontend
- backend
labels:
- "traefik.enable=false"
- "traefik.frontend.rule=Host:traefik.${DOMAINNAME}"
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- /etc/compose/traefik:/etc/traefik
- /etc/compose/shared:/shared
jira:
image: dchevell/jira-software:${JIRAVERSION}
ports:
- 8080:8080
networks:
- backend
restart: unless-stopped
volumes:
- /data/files/jira/data:/var/atlassian/application-data/jira
environment:
- JVM_MAXIMUM_MEMORY=2048m
- JVM_MINIMUM_MEMORY=768m
- CATALINA_CONNECTOR_PROXYNAME=jira.${DOMAINNAME}
- CATALINA_CONNECTOR_PROXYPORT=443
- CATALINA_CONNECTOR_SCHEME=https
- CATALINA_CONNECTOR_SECURE=true
depends_on:
- jira-postgresql
links:
- "jira-postgresql:database"
labels:
- "traefik.enable=true"
- "traefik.backend=jira"
- "traefik.frontend.rule=Host:jira.${DOMAINNAME}"
- "traefik.port=8080"
jira-postgresql:
image: postgres:9.6.11-alpine
networks:
- backend
ports:
- 5432:5432
restart: unless-stopped
volumes:
- /data/index/postgresql/data/:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=jira
- POSTGRES_USER=jira
- POSTGRES_DB=jira
labels:
- "traefik.enable=false"
# Portainer
portainer:
image: portainer/portainer
container_name: portainer
restart: always
ports:
- 9000:9000
command: -H unix:///var/run/docker.sock
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./etc-portainer/data:/data
environment:
TZ: ${TZ}
labels:
- "traefik.enable=false"
networks:
frontend:
external:
name: frontend
backend:
driver: bridge
Configuration I got working with apps over secure - not super intuitive, but it looks like it accepts redirects secure traffic properly. I've got mine using acme on godaddy for certs, and it appears to be functioning properly over https with a forced recirect:
Forced redirect for reference:
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
And the dockerfile that I made to get things deployed properly:
version: '3'
services:
jira:
image: dchevell/jira-software:8.1.0
deploy:
restart_policy:
condition: on-failure
labels:
- traefik.frontend.rule=Host:jira.mydomain.com
- traefik.enable=true
- traefik.port=8080
ports:
- "8080"
networks:
- traefik-pub
- jiranet
environment:
- CATALINA_CONNECTOR_PROXYNAME=jira.mydomain.com
- CATALINA_CONNECTOR_PROXYPORT=443
- CATALINA_CONNECTOR_SCHEME=https
- CATALINA_CONNECTOR_SECURE=true
jira-postgresql:
image: postgres:11.2-alpine
networks:
- jiranet
ports:
- "5432"
volumes:
- jira-postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=supersecret
- POSTGRES_USER=secret_user
- POSTGRES_DB=jira_db
labels:
- "traefik.enable=false"
volumes:
jira-postgres-data:
networks:
traefik-pub:
external: true
jiranet:
driver: overlay
This still required manual configuration of the database - I may one day take the time to build my own jira dockerfile that accepts the database config already, but with this one working, I don't see much point in pre-configuring the database connection when it's 20 seconds of extra work vs. rebuilding a dockerfile that I haven't written myself.

Traefik with Docker Networks

I really don't get how to use traefik with docker networks.
I try to run "wekan" kanban. If I bind ports to host, it works perfectly, so it really is about adressing it through traefik. Here is my docker-config:
version: '2'
services:
wekandb:
image: mongo:3.2.14
container_name: wekan-db
command: mongod --smallfiles --oplogSize 128
networks:
- wekan-tier
expose:
- 27017
volumes:
- wekan-db:/data/db
- wekan-db-dump:/dump
wekan:
image: wekanteam/wekan:latest
container_name: wekan-app
networks:
- wekan-tier
# ports:
# - 8081:80
environment:
- MONGO_URL=mongodb://wekandb:27017/wekan
- ROOT_URL=https://wekan.domain.com
depends_on:
- wekandb
labels:
- "traefik.port=80"
- "traefik.backend=wekan"
- "traefik.frontend.rule=Host:wekan.domain.com"
- "traefik.docker.network=wekan_wekan-tier"
volumes:
wekan-db:
driver: local
wekan-db-dump:
driver: local
networks:
wekan-tier:
driver: bridge
I can't seem to find a way to access the damn thing... Your answer will be greatly appreciated, not only will it allow me to run Wekan, but also to update my older services where I used linking:linking instead of Docker Networks - Linking being now deprecated.
I believe you have more than one issue here.
First, in your compose you don't have Traefik service, it is OK, Traefik will be able to see containers from the services here, but Traefik will not be able to send the request to it, because Traefik service and wekan service does not share the same network.
So to fix that you need to create an specific network to Traefik and set it in your compose file also.
Example:
$ docker network create traefik-net
$ docker service --name traefik --network traefik-net .... traefik ....
Second, you need to define the network Traefik will use to connect with your service, this network must be one shared with Traefik service.
So your wekan service needs to be like this:
wekan:
image: wekanteam/wekan:latest
container_name: wekan-app
networks:
- wekan-tier
- traefik-net
environment:
- MONGO_URL=mongodb://wekandb:27017/wekan
- ROOT_URL=https://wekan.domain.com
depends_on:
- wekandb
labels:
- "traefik.port=80"
- "traefik.backend=wekan"
- "traefik.frontend.rule=Host:wekan.domain.com"
- "traefik.docker.network=traefik-net"
I have modified your docker-compose file to make it work:
version: '3'
services:
web:
image: wekanteam/wekan:latest
networks:
- wekan-tier
environment:
- MONGO_URL=mongodb://wekandb:27017/wekan
- ROOT_URL=https://wekan.domain.com
labels:
- "traefik.port=80"
- "traefik.docker.network=wekan_wekan-tier"
wekandb:
image: mongo:3.2
command: mongod --smallfiles --oplogSize 128
networks:
- wekan-tier
expose:
- 27017
volumes:
- wekan-db:/data/db
- wekan-db-dump:/dump
traefik:
image: 'traefik:1.6'
command: --web --docker --docker.watch --docker.domain=local --logLevel=DEBUG
labels:
- traefik.docker.network=wekan-tier
- traefik.port=8080
ports:
- '80:80'
- '8080:8080'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
networks:
- wekan-tier
volumes:
wekan-db:
driver: local
wekan-db-dump:
driver: local
networks:
wekan-tier:
driver: bridge
Now start containers with the following command:
$ docker-compose -p wekan up -d
To check that traefik is working go to http://localhost:8080/, if you have problems stop your apache server using $ service apache2 stop. On the other hand, if you can see the traefik interface then add the following line to your /etc/hosts file:
127.0.0.1 web.wekan.local
Now go to http://web.wekan.local and you should see Wekan login page :)

Resources