Can't get correct container IP using traefik - docker

When using Traefik and Docker-compose, I would like to get the container IP to perform IP-based filtering but instead get the docker network gateway IP.
Here is the results of a curl request from the curl-client container:
docker-compose exec curl-client curl https://whoami.domain.name
Hostname: 608f3dcaf7d9
IP: 127.0.0.1
IP: 172.18.0.2
GET / HTTP/1.1
Host: whoami.domain.name
User-Agent: curl/7.58.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.18.0.1
X-Forwarded-Host: whoami.domain.name
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 88756553599b
X-Real-Ip: 172.18.0.1
Here, 172.18.0.1 is the gateway for the traefik_net network. Instead, I would expect to see 172.18.0.9 in the X-Forwarded-For field, as it is the IP of the curl-client container:
docker-compose exec curl-client cat /etc/hosts
172.18.0.9 34f7b6e5472f
I've also tried using the 'traefik.frontend.whiteList.useXForwardedFor=true' option without success.
traefik.toml
logLevel = "ERROR"
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.dashboard]
address = ":8080"
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[api]
entrypoint="dashboard"
[acme]
email = "something#aaa.com"
storage = "acme.json"
entryPoint = "https"
[acme.dnsChallenge]
provider = "ovh"
delayBeforeCheck = 0
[[acme.domains]]
main = "*.domain.name"
[docker]
domain = "domain.name"
watch = true
network = "traefik_net"
docker-compose.yml
version: '3'
services:
traefik_proxy:
image: traefik:alpine
container_name: traefik
networks:
- traefik_net
ports:
- 80:80
- 443:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
restart: unless-stopped
environment:
- OVH_ENDPOINT=ovh-eu
- OVH_APPLICATION_KEY=secretsecret
- OVH_APPLICATION_SECRET=secretsecret
- OVH_CONSUMER_KEY=secretsecret
labels:
- 'traefik.frontend.rule=Host:traefik.domain.name'
- 'traefik.port=8080'
- 'traefik.backend=traefik'
whoami:
image: containous/whoami
container_name: whoami
networks:
- traefik_net
labels:
- 'traefik.frontend.rule=Host:whoami.domain.name'
curl-client:
image: ubuntu
networks:
- traefik_net
command: sleep infinity
networks:
traefik_net:
external: true
Edit: The domain name is resolved using the following dnsmasq.conf:
domain-needed
expand-hosts
bogus-priv
interface=eno1
domain=domain.name
cache-size=1024
listen-address=127.0.0.1
bind-interfaces
dhcp-range=10.0.10.10,10.0.10.100,24h
dhcp-option=3,10.0.10.1
dhcp-authoritative
server=208.67.222.222
server=208.67.220.220
address=/domain.name/10.0.10.3

After some investigation it seems that Traefik is not the problem here, the inability to access the container IP is due to the way Docker manages its internal network (see the following comments: https://github.com/containous/traefik/issues/4352 and https://github.com/docker/for-mac/issues/180).
I was able to achieve my goal of whitelisting internal connections by running my openvpn container in nework_host mode, this way the client is assigned an IP by the system directly.

Setting the ports configuration of the docker-compose file as follows should work:
ports:
- target: 80
published: 80
mode: host
- target: 443
published: 443
mode: host
This ports capability is only available for docker-compose file format =>3.2

Related

Can't get multiple services running behind traefik on the same host

I have two services as shown below:
version: '3'
services:
# reverse proxy
traefik:
image: traefik:1.7-alpine
ports:
- 8080:8080 # Access through port 8080 e.g. localhost:8080 -> traefik dashboard
- 9000:80 # Access through port 9000 e.g. localhost:9000 -> actual service (flask and any others)
- 9001:80 # Access through port 9001 e.g. localhost:9001 -> actual service (flask and any others)
- 443:443 # Access through port 443 e.g. localhost:443 -> unused until we get https going
networks:
- traefik
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# - ./acme.json:/acme.json # ignored for production version
# - ./traefik.toml:/traefik.toml # ignored for production version
container_name: traefik
command: --docker --api --docker.domain=local.docker
restart: unless-stopped
# flask app 1
flaskapp1:
# build the Dockerfile
build:
context: ./flaskapp1
dockerfile: Dockerfile
container_name: flaskapp1
restart: always
command: >
gunicorn -b 0.0.0.0:5000
--access-logfile -
--timeout=1200
--reload
"flaskapp1.app:create_app()"
volumes:
- '.:/flaskapp1'
networks:
- traefik
# exposing a port to the other services (note, this is not publishing the port to the world)
expose: ['5000']
# these labels override the traefik configure file so we can setup traefik here, and not deal with a .toml file
labels:
- traefik.enable=true
- traefik.backend=flaskapp1
- traefik.docker.network=traefik
- traefik.frontend.rule=Host:localhost
- traefik.flaskapp1.port=5000
# flask app 2
flaskapp2:
# build the Dockerfile
build:
context: ./flaskapp2
dockerfile: Dockerfile
container_name: flaskapp2
restart: always
command: >
gunicorn -b 0.0.0.0:5000
--access-logfile -
--timeout=600
--reload
"hairByElli.app:create_app()"
volumes:
- '.:/flaskapp2'
networks:
- traefik
# exposing a port to the other docker services (note, this is not publishing the port to the world)
expose: ['5000']
# these labels override the traefik config file so we can setup traefik here, and not deal with a .toml file
labels:
- traefik.enable=true
- traefik.backend=flaskapp2
- traefik.docker.network=traefik
- traefik.frontend.rule=Host:localhost:9001
# - traefik.frontend.rule=Host:localhost;PathPrefixStrip:/app2
- traefik.flaskapp2.port=5000
networks:
traefik:
external: false
As you can see, this is my localhost testing machine. Ubuntu 18.04. So I'm currently trying to get to localhost:9000 and localhost:9001 as my entry points whilst I get this working properly.
My flaskapp1 works perfectly. My flaskapp2 is giving a 404 error.
Here is my toml:
defaultEntryPoints = ["http", "https"]
logLevel = "error"
debug = false
# WEB interface of Traefik - it will show web page with overview of frontend and backend configurations
[api]
dashboard = false
address = ":8080"
# Connection to docker host system (docker.sock)
[docker]
domain = "local.docker"
watch = true
# This will hide all docker containers that don't have explicitly
# set label to "enable"
exposedbydefault = false
# Force HTTPS
[entryPoints]
[entryPoints.http]
address = ":9000"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.http]
address = ":9001"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
What am I doing wrong here? I mean, I don't mind if it comes in at 9000 / 9001 or some other entry point. It's just so I can test them on the same machine until going live. At which point I'll have separate domains pointing to them.
Any help would be greatly appreciated.

Bad gateway on traefik docker

I build traefik with cloudflare CDN. I used docker container run command to execute my docker container execute by Drone CI. I have an issue when I successfully built docker container which leads to bad gateway on subdomain.
docker-compose.yml
version: '3'
services:
traefik:
image: traefik:latest
container_name: traefik
restart: always
domainname: ${DOMAINNAME}
networks:
- traefik_proxy
ports:
- "80:80"
- "443:443"
- "8080:8080"
environment:
- CF_API_EMAIL=${CLOUDFLARE_EMAIL}
- CF_API_KEY=${CLOUDFLARE_API_KEY}
labels:
- "traefik.enable=true"
- "traefik.backend=traefik"
- "traefik.frontend.rule=Host:monitor.${DOMAINNAME}"
- "traefik.port=8080"
- "traefik.docker.network=traefik_proxy"
- "traefik.frontend.headers.SSLRedirect=true"
- "traefik.frontend.headers.STSSeconds=315360000"
- "traefik.frontend.headers.browserXSSFilter=true"
- "traefik.frontend.headers.contentTypeNosniff=true"
- "traefik.frontend.headers.forceSTSHeader=true"
- "traefik.frontend.headers.SSLHost=example.com"
- "traefik.frontend.headers.STSIncludeSubdomains=true"
- "traefik.frontend.headers.STSPreload=true"
- "traefik.frontend.headers.frameDeny=true"
- "traefik.frontend.auth.basic.users:${HTTP_USERNAME}:${HTTP_PASSWORD}"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /etc/traefik:/etc/traefik
- /etc/docker/shared:/shared
networks:
traefik_proxy:
external:
name: traefik_proxy
Traefik.toml
nsecureSkipVerify = true
defaultEntryPoints = ["https", "http"]
# WEB interface of Traefik - it will show web page with overview of frontend and backend configurations
[api]
entryPoint = "traefik"
dashboard = true
address = ":8080"
# Force HTTPS
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
# Let's encrypt configuration
[acme]
email = "example#gmail.com" #any email id will work
storage="acme.json"
entryPoint = "https"
acmeLogging=true
onDemand = false #create certificate when container is created
[acme.dnsChallenge]
provider = "cloudflare"
delayBeforeCheck = 300
[[acme.domains]]
main = "example.com"
[[acme.domains]]
main = "*.example.com"
# Connection to docker host system (docker.sock)
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "example.com"
watch = true
# This will hide all docker containers that don't have explicitly
# set label to "enable"
exposedbydefault = false
Command I used to run the docker container execute by Drone:
docker container run -d --name example-development --restart=unless-
stopped --label "traefik.backend=example-development" --label
"traefik.frontend.rule=Host:subdomain.example.com" --label
"traefik.enable=false" --label "traefik.port=6611" --expose 6611
cloud.canister.io:5000/username/repo
My docker container is listening to http://127.0.0.1:6611
Above codes examples lead to Error 504 Gateway Timeout.
Traefik needs to have a common network with the containers it is connecting to. In this case, you need to run containers with --net=traefik_proxy.
If you're container is on multiple networks, you'll also need the label traefik.docker.network=traefik_proxy to tell traefik which of those networks to use.

Configure Traefik in front of two docker containers, all on port 80

I try to run three docker containers on one host. Traaefik is one of the containers to proxy traffic to the other containers.
My first goal is to reach each container through a dedicated hostname on port 80. Traefik ui should be available only through a hostname and on port 80 also, having some sort of authentication.
Using only a docker-compose.yml, I can reach all three containers using the hostnames, all on port 80. But to add authentication, I guess I need to introduce a traefik.toml. But this gives me troubles.
Next goal would be to introduce SSL using let's encrypt on all three hosts.But first things first...
Working solution with three hosts, all on port 80, lacking authorization for Traefik UI:
version: "2"
networks:
web:
services:
prox:
image: containous/traefik:latest # The official Traefik docker image
command: --api --docker # Enables the web UI and tells Træfik to listen to docker
restart: unless-stopped
ports:
- "80:80" # The HTTP port
labels:
- "traefik.port=8080"
- "traefik.backend=traefikception"
- "traefik.frontend.rule=Host:traefik.test.com"
- "traefik.enable=true"
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
networks:
- web
seafile_1:
image: seafileltd/seafile
container_name: seafile_1
restart: unless-stopped
environment:
SEAFILE_ADMIN_EMAIL: me#test.com
SEAFILE_ADMIN_PASSWORD: ####
SEAFILE_SERVER_HOSTNAME: 1.test.com
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:1.test.com
- traefik.port=80
- traefik.backend=seafile_1
- traefik.docker.network=web
volumes:
- /opt/seafile-data/ttt_1:/shared
networks:
- web
seafile_2:
image: seafileltd/seafile
container_name: seafile_2
restart: unless-stopped
environment:
SEAFILE_ADMIN_EMAIL: me#test2.com
SEAFILE_ADMIN_PASSWORD: #####
SEAFILE_SERVER_HOSTNAME: 2.test2.com
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:2.test2.com
- traefik.port=80
- traefik.backend=seafile_1
- traefik.docker.network=web
volumes:
- /opt/seafile-data/ttt_2:/shared
networks:
- web
Adding the following traefik.toml:
defaultEntryPoints = ["http"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.proxy]
address=":80"
[entryPoints.proxy.auth]
[entryPoints.proxy.auth.basic]
users = [
"joh:$apr1$RKdHyOKO$QDK1EKB4UJbsda7CXfPfK0",
]
[api]
entrypoint="proxy"
I get lot's of the following errors in the log, none of the containers is reachable from outside:
prox_1 | time="2018-06-17T19:23:26Z" level=fatal msg="Error preparing server: listen tcp :8080: bind: address already in use"
prox_1 | time="2018-06-17T19:24:26Z" level=error msg="Error opening listener listen tcp :8080: bind: address already in use"
prox_1 | time="2018-06-17T19:24:26Z" level=fatal msg="Error preparing server: listen tcp :8080: bind: address already in use"
I am pretty sure I need to adapt my docker-compose.yml and move settings to traefik.toml, but I cannot get my head around how to to that.
Thanks in advance!!
With the help of traefik support on slack I was able to solve this.
you may not have more than one entryPoint per Port
The Authorization can be configured in the docker-compose.yml
add acme.json and configure https and Let's encrypt only in traefik.toml
In /opt/traefik put the following three files:
acme.json:
may be empty but must be well secured:
touch acme.json
chmod 600 acme.json
docker-compose.yml:
version: "2"
networks:
web:
services:
prox:
image: containous/traefik:latest # The official Traefik docker image
command: --api --docker # Enables the web UI and tells Træfik to listen to docker
restart: unless-stopped
ports:
- "80:80"
- "443:443"
# - "8080:8080" # Don't want this port open (on all hostnames!)
labels:
- "traefik.port=8080"
- "traefik.backend=traefikception"
- "traefik.frontend.rule=Host:traefik.example.me"
- "traefik.enable=true"
- "traefik.frontend.auth.basic=admin:$$ert2$$RKdHyOKO$$QDK1EKB4UJbsda7CXfPfK0"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock" # So that Traefik can listen to the Docker events
- "./traefik.toml:/traefik.toml"
- "./acme.json:/acme.json"
networks:
- web
seafile_org1:
image: seafileltd/seafile
container_name: seafile_org1
restart: unless-stopped
environment:
SEAFILE_ADMIN_EMAIL: mail#mail.me
SEAFILE_ADMIN_PASSWORD: ####
SEAFILE_SERVER_HOSTNAME: org1.example.me
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:org1.example.me
- traefik.port=80
- traefik.backend=seafile_org1
- traefik.docker.network=web
volumes:
- /opt/seafile-data/org1:/shared
networks:
- web
seafile_org2:
image: seafileltd/seafile
container_name: seafile_org2
restart: unless-stopped
environment:
SEAFILE_ADMIN_EMAIL: mail#mail.com
SEAFILE_ADMIN_PASSWORD: ####
SEAFILE_SERVER_HOSTNAME: org2.example.com
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:org2.example.com
- traefik.port=80
- traefik.backend=seafile_org2
- traefik.docker.network=web
volumes:
- /opt/seafile-data/org2:/shared
networks:
- web
get what you need to put as value to traefik.frontend.auth.basic issuing:
htpasswd -n admin
traefik.toml:
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[retry]
[api]
dashboard = true
# Enable ACME (Let's Encrypt): automatic SSL.
[acme]
email = "you#mail.com"
storage = "acme.json"
entryPoint = "https"
# If true, display debug log messages from the acme client library.
# acmeLogging = true
# Enable certificate generation on frontends host rules.
onHostRule = true
# CA server to use.
# Uncomment the line to use Let's Encrypt's staging server,
# leave commented to go to prod.
caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
# Use a HTTP-01 ACME challenge.
# Optional (but recommended)
[acme.httpChallenge]
entryPoint = "http"
This uses Let's encrypt staging environment to get you three certs. Comment the line with caServer to get the real certs! Recreate an empty acme.json as well!
The seafile-data is stored in
/opt/seafile-data/org1
and
/opt/seafile-data/org2
respectively.
In /opt/traefik you can start the system:
docker-compose up -d
and watch the logs with
docker-compose logs
Startup takes some time on first run to setup seafile, get the certs,...
Your hosts should be reachable, giving no SSL errors or warnings on
http://traefik.example.me (Asking your credentials to see the page)
http://org1.example.me
http://org2.example.com
What's left to do is to edit the ccnet.conf file in each of the seafile installation directories (/opt/seafile-data/org1/seafile/conf/ccnet.conf) and change the protocol to "http" and remove the port ":8000" from SERVICE_URL so that shared links are correct for that setup as well. The line should read:
SERVICE_URL = https://org1.example.me
You can do it all in the Docker Stacks file:
version: "3.7"
services:
traefik:
image: traefik:1.7.13
command: >
--api
--docker
--docker.swarmmode
--docker.watch
--docker.exposedbydefault=false
# --debug=true
--loglevel=error # debug
--defaultentrypoints=https,http
--entryPoints="Name:http Address::80 Redirect.EntryPoint:https"
--entryPoints="Name:https Address::443 TLS"
--retry
--acme=true
--acme.entrypoint=https
--acme.httpchallenge
--acme.httpchallenge.entrypoint=http
--acme.domains="..."
--acme.email="..."
--acme.storage=/certs/acme.json
ports:
- 80:80 # HTTP
- 443:443 # HTTPS
- 8080:8080 # The Web UI (enabled by --api)
volumes:
- acme:/certs
- /var/run/docker.sock:/var/run/docker.sock:ro
whoami:
image: containous/whoami
deploy:
labels:
traefik.frontend.rule: Path:/whoami
traefik.enable: "true"
traefik.port: 80
volumes:
acme:

Not redirected to SSL/acme.json empty

I am using traefik as reverse proxy for cryptpad. Traefik is running in a docker container. I used this docs to solve this issue: https://github.com/xwiki-labs/cryptpad/wiki/Docker-(with-Nginx-and-Traefik)#configuring-traefik
That's what I did:
$ mkdir traefik && cd traefik
$ wget https://raw.githubusercontent.com/containous/traefik/master/traefik.sample.toml
$ mv traefik.sample.toml traefik.toml
$ touch acme.json && chmod 600 acme.json
I added this to traefik.toml:
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[retry]
[acme]
email = "me#example.org"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"
Created docker-compose.yml
version: '3'
services:
traefik:
image: traefik
command: --docker
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik/traefik.toml:/traefik.toml
- ./traefik/acme.json:/acme.json
ports:
- 80:80
- 443:443
networks:
- proxy
container_name: traefik
networks:
proxy:
external: true
And created/started traefik with docker-compose up -dll .
Well - for http it works fine so far - but I am not being redirected to https. And if I would, the connection would be reset (https://cryptpad.mydomain.org -> connection refused).
I checked if something is listening for http/https:
tcp LISTEN 0 128 :::http :::*
tcp LISTEN 0 128 :::https :::*
And tried to telnet it:
[root#cryptpad ~]# telnet localhost 443
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
What did I miss?

Certificate not automatically created - Docker-Compose, Traefik, Let's Encrypt,

I have set up my first home-docker-stack:
DDNS account --> No chance to get subdomains --> Use ports.
Configure a https-proxy so that https-configuration is not required for each container --> Use traefik.
Now I have a traefik.toml with the following content:
defaultEntryPoints = ["http", "https"]
logLevel = "DEBUG"
debug = true
[web]
address = ":8080"
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[acme]
email = "MY_EMAIL_ADDRESS"
storage = "/etc/traefik/acme/acme.json"
entryPoint = "https"
onDemand = true
OnHostRule = false
[acme.httpChallenge]
entryPoint = "http"
[docker]
domain = "MY_DOMAIN"
watch = true
And a docker-compose.yml looking as follows:
version: '3.4'
services:
db:
image: mariadb:10.1
restart: always
volumes:
- db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=SOMEPASSWORD
env_file:
- db.env
wordpress:
image: wordpress:apache
restart: always
ports:
- "8001:80"
environment:
- WORDPRESS_DB_HOST=db
- WORDPRESS_DB_PASSWORD=SOMEPASSWORD
- WORDPRESS_DB_NAME=wordpress
depends_on:
- db
networks:
- default
- traefik-net
deploy:
replicas: 1
labels:
- "traefik.enable=true"
- "traefik.port=8001"
- "traefik.docker.network=traefik-net"
traefik:
image: traefik:1.5-alpine
restart: always
networks:
- traefik-net
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /home/traefik/traefik.toml:/etc/traefik/traefik.toml:ro
- /home/traefik/acme:/etc/traefik/acme
privileged: true
container_name: traefik
volumes:
db:
networks:
traefik-net:
If I now try to reach my traefik-instance as follows:
https://MY_DOMAIN/
The client returns (untrusted self signed certificate):
Dem Zertifikat wird nicht vertraut, weil es vom Aussteller selbst signiert wurde.
The traefik log contains:
traefik | time="2018-03-19T13:29:29Z" level=debug msg="Looking for provided certificate to validate MY_DOMAIN..."
traefik | time="2018-03-19T13:29:29Z" level=debug msg="No provided certificate found for domains MY_DOMAIN, get ACME certificate."
traefik | time="2018-03-19T13:29:29Z" level=debug msg="Looking for an existing ACME challenge for MY_DOMAIN..."
traefik | time="2018-03-19T13:29:29Z" level=debug msg="http2: server: error reading preface from client 80.129.18.33:44700: remote error: tls: unknown certificate authority"
Why might the generation of the certificate not start? What do I have to fix?
Traefik will access your container through the docker network, in the docker network the ports you have set in the ports mapping in the docker-compose file means nothing. ports is just to map the container port to one of the hosts ports.
So the port that you should tell traefik to use (through the labels) is 80, the port that the webserver listens to. And you could remove the port mapping, because that's just to the host, if Traefik have ports open for http, it will route the requests to your container (through the exposed port, which should be 80).
As long as your container is in the network, the exposed ports will be available for all other containers in the network.
Another note:
You are in the docker-compose file using a 3.x version. All 3.x versions are swarm specific, so stick to 2.x for none-swarm files.

Resources