traefik with docker breaks when I setup networks - docker

I'm trying to setup docker networks with traefik on an existing website.
Before my tries, it had this:
version: "3"
services:
database:
build:
context: ./database
environment:
MYSQL_DATABASE: '${MYSQL_DATABASE}'
MYSQL_USER: '${MYSQL_USER}'
MYSQL_PASSWORD: '${MYSQL_PASSWORD}'
MYSQL_ROOT_PASSWORD: '${MYSQL_ROOT_PASSWORD}'
volumes:
- ./database/data:/var/lib/mysql
restart: always
php-http:
build:
context: ../
dockerfile: ./docker/php-apache/Dockerfile
args:
MAIN_DOMAIN: '${MAIN_DOMAIN}'
ALL_DOMAINS: '${ALL_DOMAINS}'
PROJECT_FOLDER_NAME: '${PROJECT_FOLDER_NAME}'
WEBSITE_USER_PASSWORD: '${WEBSITE_USER_PASSWORD}'
depends_on:
- database
- mailserver
volumes:
- ./apachelogs:/var/log/apache2
- ./apachelogs/auth.log:/var/log/auth.log
- './symfonylogs:/var/www/html/mywebsite/var/log/'
labels:
- traefik.http.routers.php-http.tls=true
- traefik.http.routers.php-http.tls.certresolver=letsencrypt
- traefik.http.services.php-http.loadbalancer.server.port=80
- traefik.enable=true
- traefik.http.routers.php-http.rule=Host(`mystuff.com`, `en.mystuff.com`)
- 'traefik.http.routers.php-http.tls.domains[0].main=mystuff.com'
- 'traefik.http.routers.php-http.tls.domains[1].main=en.mystuff.com'
restart: always
mailserver:
[doesntmatter]
traefik:
image: traefik:v2.9
command:
- --providers.docker
- --providers.docker.exposedByDefault=false
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.web.http.redirections.entryPoint.to=websecure
- --entrypoints.web.http.redirections.entryPoint.scheme=https
- --entrypoints.web.http.redirections.entrypoint.permanent=true
- --certificatesresolvers.letsencrypt.acme.email=heyho#gmail.com
- --certificatesresolvers.letsencrypt.acme.storage=acme.json
- --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web
ports:
- 80:80
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./acme.json:/acme.json
It works fine.
Then I tried doing this :
docker network create web
And in the yml:
networks:
web:
external: true
internal:
external: false
For php-http:
networks:
- internal
- web
and (I tried without and with it)
- "traefik.docker.network=web"
In database and mailserver :
networks:
- internal
In traefik:
networks:
- web
and (tried without and with it)
- "traefik.docker.network=web"
It didn't work at all, my website wasn't accessible anymore.
Then as said there : https://doc.traefik.io/traefik/user-guides/docker-compose/basic-example/
I tried :
networks:
web: {}
Then in php-http and traefik:
networks:
- web
It didn't work either. Their example (with whoami) works on my server. (Tried with a local curl). Like always, this makes me hate sysadmin very much, does anyone has any clue on what's wrong there? It doesn't make anysense to me. I followed everything, tried everything.
Thank you

Related

Why am I unable to route to my API backend with Traefik

I had two container frontend (nginx :80) and backend (nodejs :3000).
I'm trying to redirect all path to my frontend : localhost/* to my frontend
Except one path to my backend API : localhost/v1/* to my backend
I secure my database container (mongodb) by allowing only communication with my backend
Here is my docker-compose.yml (I'm only using this)
version: '3'
services:
traefik:
image: traefik:v2.3
container_name: traefik
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
- "8080:8080"
- "443:443"
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
frontend:
image: registry.gitlab.com/test/frontend
container_name: frontend
build:
context: ../frontend/.
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=PathPrefix(`/`)
- traefik.http.routers.frontend.entrypoints=web
networks:
- traefik-network
backend:
image: registry.gitlab.com/test/backend
container_name: backend
build:
context: ../backend/.
labels:
- traefik.enable=true
- traefik.http.routers.backend.rule=PathPrefix(`/v1`)
- traefik.http.routers.backend.service=backend
- traefik.http.routers.backend.entrypoints=web
- traefik.http.services.backend.loadbalancer.server.port=3000
command: yarn start
environment:
- MONGODB_URL=mongodb://mongodb:27017/backend
depends_on:
- mongodb
volumes:
- ../backend/.:/usr/src/backend
networks:
- traefik-network
- backend-network
mongodb:
image: mongo:4.2.1-bionic
container_name: mongodb
ports:
- 27017:27017
volumes:
- dbdata:/data/db
networks:
- backend-network
volumes:
dbdata:
networks:
backend-network:
traefik-network:
The problem is...
If the frontend (backend and traefik too) is turned on
the paths to localhost/* work (this is what I want),
but the paths to localhost/v1/* don't work (Problem here!).
If the frontend is turned off but traefik and backend is turned on
the paths to localhost/* don't work (of course, that's right),
but the paths to localhost/v1/* work (of course, this is what I want).
I've tried a lot of solutions but nothing seems to work the way I want it to.
What did I misunderstand?
Thanks for helping,
Have a nice day
Try to add the following labels to the backend service
- "traefik.http.routers.backend.rule=Host(`servicex.me`) && Path(`/v1`)"
and frontend
- traefik.http.routers.frontend.rule=Host(`servicex.me`)
you also need to add this line to your /etc/hosts
127.0.0.1 servicex.me
and make sure that you stop and start the services
Complete Example
version: '3'
services:
traefik:
image: traefik:v2.3
container_name: traefik
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
- "8080:8080"
- "443:443"
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
frontend:
image: registry.gitlab.com/test/frontend
container_name: frontend
build:
context: ../frontend/.
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=Host(`servicex.me`)
- traefik.http.routers.frontend.entrypoints=web
- traefik.http.routers.frontend.service=frontend
- traefik.http.services.frontend.loadbalancer.server.port=80
networks:
- traefik-network
backend:
image: registry.gitlab.com/test/backend
container_name: backend
build:
context: ../backend/.
labels:
- traefik.enable=true
- "traefik.http.routers.backend.rule=Host(`servicex.me`) && Path(`/v1`)"
- traefik.http.routers.backend.service=backend
- traefik.http.routers.backend.entrypoints=web
- traefik.http.services.backend.loadbalancer.server.port=3000
command: yarn start
environment:
- MONGODB_URL=mongodb://mongodb:27017/backend
depends_on:
- mongodb
volumes:
- ../backend/.:/usr/src/backend
networks:
- traefik-network
- backend-network
mongodb:
image: mongo:4.2.1-bionic
container_name: mongodb
ports:
- 27017:27017
volumes:
- dbdata:/data/db
networks:
- backend-network
volumes:
dbdata:
networks:
backend-network:
traefik-network:
BTW, why do you need both traefik and nginx (Both are doing the same job), it would be better if you can replace one with another.
I added this label to my containers
traefik.docker.network=traefik-network
It works fine now

Multi Docker-Compose projects sharing service

I am attempting to create multiple Docker-Compose projects and have them share a single service, a database server. I'm currently unable to get projects to use my mariadb instance. I'm not sure if my issues is in traefik or docker. I can get wordpress to recognize the database if it is included in the same docker-compose, but as I would ultimately like to not have multiple instances of a database server running, I would prefer it to be it's own project.
# ~/docker/traefik/docker-compose.yml
version: "3.3"
services:
traefik:
image: "traefik:v2.2"
container_name: "traefik"
command:
- "--api=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.dnschallenge=true"
- "--certificatesresolvers.myresolver.acme.dnschallenge.provider=namedotcom"
- "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.myresolver.acme.email=<MY-EMAIL>"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
networks:
t2_proxy:
ports:
- "80:80"
- "443:443"
environment:
- "NAMECOM_USERNAME=<My-USERNAME>
- "NAMECOM_API_TOKEN=<My-Token>
labels:
# Dashboard
- "traefik.enable=true"
- "traefik.http.routers.traefik.rule=Host(`api.mydomain.com`)"
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.middlewares=admin"
- "traefik.http.routers.traefik.tls.certresolver=myresolver"
- "traefik.http.routers.traefik.entrypoints=websecure"
- "traefik.http.middlewares.admin.basicauth.users=myuser:mypasswordhash/"
# middleware redirect
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
# global redirect to https
- "traefik.http.routers.redirs.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.redirs.entrypoints=web"
- "traefik.http.routers.redirs.middlewares=redirect-to-https"
volumes:
- "./letsencrypt:/letsencrypt"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
t2_proxy:
# ~/docker/db/docker-compose.yml
version: '3.3'
services:
mysql:
image: mariadb
container_name: "mariadb"
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
MYSQL_DATABASE: wordpressDB
MYSQL_USER: wordpressUSER
MYSQL_PASSWORD: wordpressPW
networks:
default:
ports:
- 3306:3306
adminer:
image: adminer
restart: always
container_name: "adminer"
labels:
- "traefik.enable=true"
- "traefik.http.routers.adminer.rule=PathPrefix(`/adminer`)"
- "traefik.http.routers.adminer.entrypoints=websecure"
- "traefik.http.routers.adminer.tls.certresolver=myresolver"
networks:
- default
- traefik_t2_proxy
networks:
default:
traefik_t2_proxy:
external: true
# ~/docker/wordpress/docker-compose.yml
version: '3.3'
services:
wordpress:
image: wordpress:latest
restart: always
container_name: "mySite"
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpressUSER
WORDPRESS_DB_PASSWORD: wordpressPW
WORDPRESS_DB_NAME: wordpressDB
networks:
traefik_t2_proxy:
db_default:
labels:
- "traefik.enable=true"
- "traefik.http.routers.wp2.rule=PathPrefix(`/wordpress`)"
- "traefik.http.routers.wp2.entrypoints=websecure"
- "traefik.http.routers.wp2.tls.certresolver=myresolver"
volumes:
db_data: {}
networks:
traefik_t2_proxy:
external: true
db_default:
external: true
You can have multiple docker-compose files, with the same network and run them together as below:
docker-compose -f docker-compose-1.yml -f docker-compose-2.yml up -d
But if you need the services to recognize eachother, the networks should be the same. Give your traefik_t2_proxy network to mysql too.

Setting up docker auto build to use docker-compose file

I am trying to set up auto builds using docker cloud/docker hub. It is always looking for Dockerfile when I have a docker-compose.yml. I am unable to find any option to change this. I am wondering whether this isn't possible or am I missing something?
This is my docker-compose.yml
version: '3'
services:
reverse-proxy:
image: traefik
ports:
- "80:80"
- "443:443"
- "${TRAEFIK_DASHBOARD_PORT}:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/traefik.toml:/etc/traefik/traefik.toml
- ./traefik/certs/journal.crt:/certs/journal.crt
- ./traefik/certs/journal.key:/certs/journal.key
networks:
- web
prisma:
image: prismagraphql/prisma:1.8
restart: always
ports:
- "${PRISMA_PORT}"
networks:
- web
environment:
PRISMA_CONFIG: |
port: ${PRISMA_PORT}
managementApiSecret: ${PRISMA_MANAGEMENT_API_SECRET}
databases:
default:
connector: postgres
host: ${PRISMA_DB_HOST}
port: ${PRISMA_DB_PORT}
database: ${PRISMA_DB}
user: ${PRISMA_DB_USER}
password: ${PRISMA_DB_PASSWORD}
migrations: ${PRISMA_ENABLE_MIGRATION}
graphql-server:
build:
context: ./graphql-server/
args:
- PORT=${GRAPHQL_SERVER_PORT}
networks:
- web
ports:
- "${GRAPHQL_SERVER_PORT}"
volumes:
- ./graphql-server:/usr/src/app
depends_on:
- prisma
command: ["./wait-for-it.sh", "prisma:${PRISMA_PORT}", "--", "./bootstrap.sh"]
environment:
- PRISMA_SERVICE_NAME=prisma
- PRISMA_PORT
- GRAPHQL_SERVER_PORT
- APOLLO_ENGINE_KEY
- PRISMA_ENDPOINT
- PRISMA_MANAGEMENT_API_SECRET
labels:
- "traefik.backend=graphql"
- "traefik.frontend.rule=Host:api.journal.com"
- "traefik.enable=true"
- "traefik.port=8080"
- "traefik.docker.network=web"
react-client:
build:
context: ./react-client/
args:
- PORT=${REACT_CLIENT_PORT}
ports:
- "${REACT_CLIENT_PORT}"
volumes:
- ./react-client:/usr/src/app
depends_on:
- graphql-server
environment:
- GRAPHQL_SERVER_PORT
- REACT_CLIENT_PORT
networks:
- web
networks:
web:
external: true
Both docker hub and docker cloud are trying to get only the dockerfile and not docker-compose. I also saw a post mentioning docker-compose should be used only for running and not for building; so I am not sure whether I am doing something wrong.

Handling server redirection with traefik

I have a http server running in a docker. Accessing the server at / redirects it to /web. This works fine locally. I have setup traefik to connect to the docker through xxxxxxx.com domain. However, this results in a 404 page not found but when I try xxxxxxxx.com/web it works. How does traefik handle this kind of redirects? Thanks in advance.
Here's my docker-compose.yml file:
version: "3"
networks:
proxy:
external: true
internal:
external: false
services:
web:
restart: always
image: odoo:10.0
labels:
- traefik.backend=web
- traefik.frontend.rule=Host:portal.sironirestaurant.com
- traefik.docker.network=proxy
- traefik.port=8069
networks:
- internal
- proxy
depends_on:
- db
ports:
- 8069:8069
volumes:
- odoo-web-data:/var/lib/odoo
- ./config:/etc/odoo
- ./addons:/mnt/extra-addons
db:
restart: always
image: postgres:9.4
networks:
- internal
labels:
- traefik.enable=false
environment:
- POSTGRES_PASSWORD=xxxxxxx
- POSTGRES_USER=xxxxxx
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- odoo-db-data:/var/lib/postgresql/data/pgdata
volumes:
odoo-web-data:
odoo-db-data:

Jwilder nginx proxy - 503 after docker compose structure update

I'm using jwilder/nginx-proxy with separate docker-compose.yaml. It looks like this:
proxy:
image: jwilder/nginx-proxy
restart: always
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/conf.d/proxy.conf:/etc/nginx/conf.d/proxy.conf:ro
- /Users/marcin/Docker/local_share/certificates:/etc/nginx/certs:ro
ports:
- "80:80"
- "443:443"
container_name: proxy
I'm using it for quite a long time and it's working fine when my project docker-compose.yaml looks like this:
web:
build: /Users/marcin/Docker/definitions/php-nginx/php-7.1-ubuntu
volumes:
- /Users/marcin/Docker/projects/test.local/html/:/usr/share/nginx/html/
- /Users/marcin/Docker/projects/test.local/nginx/conf.d/:/etc/nginx/conf.d/
- /Users/marcin/Docker/projects/test.local/nginx/log/:/var/log/nginx/
- /Users/marcin/Docker/projects/test.local/supervisor/conf.d/:/etc/supervisor/conf.d/
- /Users/marcin/Docker/projects/test.local/supervisor/log/:/var/log/supervisor/
- /Users/marcin/Docker/projects/test.local/cron/:/root/.cron/
- /Users/marcin/Docker/local_share/:/root/.local_share/
- /Users/marcin/Docker/local_share/certificates/:/usr/share/nginx/certificates/
working_dir: /usr/share/nginx/html/
links:
- db
container_name: test.php
hostname: test.local
ports:
- "336:22"
- "8081:80"
- "18080:443"
environment:
- VIRTUAL_HOST=test.local
- CERT_NAME=default
- HTTPS_METHOD=noredirect
db:
build: /Users/marcin/Docker/definitions/mysql/5.7
environment:
- MYSQL_ROOT_PASSWORD=pass
- MYSQL_DATABASE=
- MYSQL_USER=
- MYSQL_PASSWORD=
expose:
- 3306
volumes:
- /Users/marcin/Docker/projects/test.local/mysql/data/:/var/lib/mysql/
- /Users/marcin/Docker/projects/test.local/mysql/conf.d/:/etc/mysql/conf.d/source
- /Users/marcin/Docker/projects/test.local/mysql/log/:/var/log/mysql/
ports:
- "33060:3306"
container_name: test.db
hostname: test.local
I can access site without any problem using http://test.local or https://test.local what is expected.
However I had to update my file structure to newer version:
version: "3.2"
services:
web:
build: /Users/marcin/Docker/definitions/php-nginx/php-7.1-ubuntu
volumes:
- /Users/marcin/Docker/projects/test.local/html/:/usr/share/nginx/html/
- /Users/marcin/Docker/projects/test.local/nginx/conf.d/:/etc/nginx/conf.d/
- /Users/marcin/Docker/projects/test.local/nginx/log/:/var/log/nginx/
- /Users/marcin/Docker/projects/test.local/supervisor/conf.d/:/etc/supervisor/conf.d/
- /Users/marcin/Docker/projects/test.local/supervisor/log/:/var/log/supervisor/
- /Users/marcin/Docker/projects/test.local/cron/:/root/.cron/
- /Users/marcin/Docker/local_share/:/root/.local_share/
- /Users/marcin/Docker/local_share/certificates/:/usr/share/nginx/certificates/
working_dir: /usr/share/nginx/html/
links:
- db
container_name: test.php
hostname: test.local
ports:
- "336:22"
- "8081:80"
- "18080:443"
environment:
- VIRTUAL_HOST=test.local
- CERT_NAME=default
- HTTPS_METHOD=noredirect
db:
build: /Users/marcin/Docker/definitions/mysql/5.7
environment:
- MYSQL_ROOT_PASSWORD=pass
- MYSQL_DATABASE=
- MYSQL_USER=
- MYSQL_PASSWORD=
expose:
- 3306
volumes:
- /Users/marcin/Docker/projects/test.local/mysql/data/:/var/lib/mysql/
- /Users/marcin/Docker/projects/test.local/mysql/conf.d/:/etc/mysql/conf.d/source
- /Users/marcin/Docker/projects/test.local/mysql/log/:/var/log/mysql/
ports:
- "33060:3306"
container_name: test.db
hostname: test.local
and after that it seems not to work. I can access site using ip and port without a problem, but I cannot longer use domain to access it. When I try I'm getting:
503 Service Temporarily Unavailable
nginx/1.13.8
And this is for sure from jwilder nginx (and not the nginx in project).
So the question is - where should I put environment variables to make it work? It seems that when they are placed as they are at the moment they are not read by proxy.
The 503 indicates that the nginx-proxy container can see your container running in docker and it has the configuration needed for nginx to route traffic to it, but it is unable to connect to that container over the docker network. For container-to-container networking to work, you need to have a common docker network defined. You should first run the following to create a network:
docker network create proxy
Then update your nginx-proxy compose file to use the network (this should also be upgraded to at least a v2 syntax, I've gone with 3.2 to match your other file):
version: "3.2"
networks:
proxy:
external: true
services:
proxy:
image: jwilder/nginx-proxy
restart: always
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/conf.d/proxy.conf:/etc/nginx/conf.d/proxy.conf:ro
- /Users/marcin/Docker/local_share/certificates:/etc/nginx/certs:ro
ports:
- "80:80"
- "443:443"
container_name: proxy
networks:
- proxy
And then do something similar for your application:
version: "3.2"
networks:
proxy:
external: true
services:
web:
build: /Users/marcin/Docker/definitions/php-nginx/php-7.1-ubuntu
volumes:
- /Users/marcin/Docker/projects/test.local/html/:/usr/share/nginx/html/
- /Users/marcin/Docker/projects/test.local/nginx/conf.d/:/etc/nginx/conf.d/
- /Users/marcin/Docker/projects/test.local/nginx/log/:/var/log/nginx/
- /Users/marcin/Docker/projects/test.local/supervisor/conf.d/:/etc/supervisor/conf.d/
- /Users/marcin/Docker/projects/test.local/supervisor/log/:/var/log/supervisor/
- /Users/marcin/Docker/projects/test.local/cron/:/root/.cron/
- /Users/marcin/Docker/local_share/:/root/.local_share/
- /Users/marcin/Docker/local_share/certificates/:/usr/share/nginx/certificates/
working_dir: /usr/share/nginx/html/
links:
- db
container_name: test.php
hostname: test.local
ports:
- "336:22"
- "8081:80"
- "18080:443"
environment:
- VIRTUAL_HOST=test.local
- CERT_NAME=default
- HTTPS_METHOD=noredirect
networks:
- proxy
- default
db:
build: /Users/marcin/Docker/definitions/mysql/5.7
environment:
- MYSQL_ROOT_PASSWORD=pass
- MYSQL_DATABASE=
- MYSQL_USER=
- MYSQL_PASSWORD=
expose:
- 3306
volumes:
- /Users/marcin/Docker/projects/test.local/mysql/data/:/var/lib/mysql/
- /Users/marcin/Docker/projects/test.local/mysql/conf.d/:/etc/mysql/conf.d/source
- /Users/marcin/Docker/projects/test.local/mysql/log/:/var/log/mysql/
ports:
- "33060:3306"
container_name: test.db
hostname: test.local
If you were upgrading from a v1 syntax (without a version defined), you will find that docker switches from running everything on the same network without dns to running each compose project or stack on a dedicated network with dns. To run your apps on other networks, you'll need to explicitly configure that. In the above example, only the web container was placed on the proxy network, and both are on the default network created for this project or stack.

Resources