HTTPS-Redirect with Traefik behind Aws Loadbalancer - docker

I'm trying to redirect all incoming Traefik from http to https, for a web application which gets served out of a docker container with a custom port.
If I build this docker compose file, and scale the application everything works as expected. I'm able to request http and https of the application, but I try to accomplish that only https get served and http gets redirected to https.
Since I use a Docker-Compose file, I don't have a Traefik.toml, and try to accomplish this without one.
Docker Compose:
traefik:
image: traefik:latest
command:
- "--api"
- "--docker"
- "--docker.domain=example.com"
- "--logLevel=DEBUG"
- "--docker.watch"
labels:
- "traefik.enable=true"
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/null:/traefik.toml
application:
image: application
command: web
tty: false
stdin_open: true
restart: always
expose:
- "8081"
labels:
- "traefik.backend=application"
- "traefik.frontend.rule=HostRegexp:{subdomain:[a-z]+}.example.com"
- "traefik.frontend.priority=1"
- "traefik.enable=true"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
I try'd different variations on the application container, such as:
- "traefik.frontend.entryPoints=http,https"
- "traefik.frontend.redirect.entryPoint=https"
- "traefik.frontend.headers.SSLRedirect=true"
But the maximum I could accomplish was a to many redirects response, with the SSLRedirect label, and without I get the following from traefik and neither http or https requests get forwarded correctly.
level=error msg="Recovered from panic in http handler: runtime error: invalid memory address or nil pointer dereference"
Can anyone push me in the right direction?
Thanks in advance ;)
I run under the following Settings
user:~$ docker --version
Docker version 1.13.1, build 092cba3
user:~$ docker-compose --version
docker-compose version 1.8.0
Docker PS Response
IMAGE COMMAND ... PORTS NAMES
application "dotnet Web..." ... 8081/tcp components_application_1
traefik:latest "/traefik --api --..." ... 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:8080->8080/tcp components_traefik_1
Infrasturcture Setup
aws-elb => vpc => ec2...ecn
traefik per instance,
n applications per instance

This only works until traefik v1.7, after v2.* you need another config setup, which i haven't figured out yet
After a deeper research, i found the solution myself.
The problem was a missing label on the application Container,
after i added
- "traefik.frontend.headers.SSLProxyHeaders=X-Forwarded-Proto: https"
- "traefik.frontend.headers.SSLRedirect=true"
on my application containers it worked like a charm with a clear 301 redirect.
Why the need of the header, in default the aws-elb takes a https request and forwards it with a HTTP(80) to the connected Instance, during this process the elb adds the X-Forwarded-Proto: https Header to the request.
Since traefik doesn't know that it is running behind an elb it does the redirect over and over again. But the Header stops this behavior.

Related

Traefik with Docker-Compose not working as expected

I am fairly new to using traefik, so I might be totally missing something simple, but I have the following docker-compose.yaml:
version: '3.8'
services:
reverse-proxy:
container_name: reverse_proxy
restart: unless-stopped
image: traefik:v2.0
command:
- --entrypoints.web.address=:80
- --entrypoints.web-secure.address=:443
- --api.insecure=true
- --providers.file.directory=/conf/
- --providers.file.watch=true
- --providers.docker=true
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./scripts/certificates/conf/:/conf/
- ./scripts/certificates/ssl/:/certs/
networks:
- bnkrl.io
labels:
- "traefik.enable=true"
- "traefik.http.routers.api.rule=Host(`traefik.bnkrl.io`)"
- "traefik.docker.network=bnkrl.io"
bankroll:
container_name: bankroll
build:
context: .
ports:
- "3000"
volumes:
- .:/usr/src/app
command: yarn start
networks:
- bnkrl.io
labels:
- "traefik.http.routers.bankroll.rule=Host(`bankroll.bnkrl.io`)"
- "traefik.docker.network=bnkrl.io"
- "traefik.http.services.bankroll.loadbalancer.server.port=3000"
- "traefik.http.routers.bankroll-https.rule=Host(`bankroll.bnkrl.io`)"
- "traefik.http.routers.bankroll-https.tls=true"
networks:
bnkrl.io:
external: true
But for some reason the following is happening:
Running curl when ssh'd into my bankroll container gives the following:
/usr/src/app# curl bankroll.bnkrl.io
curl: (7) Failed to connect to bankroll.bnkrl.io port 80: Connection refused
Despite having - "traefik.http.services.bankroll.loadbalancer.server.port=3000" label set up.
I am also unable to hit traefik from my application container:
curl traefik.bnkrl.io
curl: (6) Could not resolve host: traefik.bnkrl.io
Despite my expectation to be able to do so since they are both on the same network.
Any help with understanding what I might be doing wrong would be greatly appreciated! My application (bankroll) is a very basic hello-world react app, but I don't think any of the details around that are relevant to the issue I'm facing.
EDIT: I am also not seeing any error logs on traefik side of things.
You are using host names that are not declared and therefore are unreachable.
To reach a container from another container, you need to use the service name, for example, if you connect to bankroll from the reverse-proxy it will hit the other service.
While if you want to access them from the host machine, you will have to publish the ports (which you did, it's all the stuff in ports in your Docker-compose file) and access from localhost or from your machine local IP address instead of traefik.bnkrl.io.
If you want to access from traefik.bnkrl.io, you will have to declare this host name, and point it to the place where the Docker containers are running from.
So either a DNS record in the domain bnkrl.io pointing to your local machine, or a HOSTS file entry in your computer pointing to 127.0.0.1.
Another note: For SSL you are going to need a valid certificate to use for the host name. While in local development, you can use the self-signed certificate provided by Traefik, but you may have to install it in the computer connecting to the service, or allow untrusted certificates from your browser, or wherever you are making the requests from (some browsers no longer support using self-signed certificates). For SSL on the Internet you will need to look at things like Let's Encrypt.

Redirect http to https using docker-compose

I'm having a jenkins container running on my gcp instance and want to redirect to https when someone enters a url with http extension.
Here my docker-compose file
version: '3.8'
services:
jenkins:
image: jenkins/jenkins
container_name: jenkins-docker
restart: always
privileged: true
user: root
ports:
- 80:80
- 443:8443
- 50000:50000
volumes:
- ./jenkins_home:/var/jenkins_home
- ../opt/cert/dcsjenkins.jks:/var/lib/jenkins/dcsjenkins.jks
environment:
JAVA_OPTS: -Duser.timezone=CET -Xmx2048m -Djava.awt.headless=true
JENKINS_OPTS: --httpPort=-1 --httpsPort=8443
Solution 1
Try setting the --httpsRedirectHttp flag on server startup. It requires both HTTP and HTTPs ports defined.
--httpsRedirectHttp = redirect http requests to https (requires both --httpPort and --httpsPort)
All available flags can be found here.
Solution 2
The other option is to deploy a reverse proxy before the Jenkins container. If you are already using an LB in GCP to expose your Jenkins instance, you can simply add a rule there to redirect HTTP traffic.

Getting 502 Bad Gateway when trying to configure HTTPS on Traefik

I got a basic Traefik 2 setup working for HTTP here...
Now I'm trying to get HTTPS working, and basic auth for the dashboard with TLS and redirects...
docker_compose.yml:
version: '3.8'
networks:
myweb:
external: true
services:
proxy:
image: traefik:v2.3.0-rc4-windowsservercore-1809
container_name: traefik
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
# Mount the certs drive
- ./traefik-ssl-certs/:c:/certs/
# Mount the config folder
- ./traefik-config/:c:/config/
# Mount the host docker engine pipe ("docker volume ls")
- source: '\\.\pipe\docker_engine'
target: '\\.\pipe\docker_engine'
type: npipe
command:
- "--api.insecure=true"
# Register the traefik config directory as per: https://docs.traefik.io/providers/file/#directory
- --providers.file.directory=c:/config/
- --providers.file.watch=true
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
# Redirect http to https
- --entrypoints.web.http.redirections.entryPoint.to=websecure
- --entrypoints.web.http.redirections.entryPoint.scheme=https
- --entrypoints.web.http.redirections.entrypoint.permanent=true
# Configure Docker provider
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.endpoint=npipe:////./pipe/docker_engine"
- "--providers.docker.network=myweb"
- "--providers.docker.watch=true"
networks:
- myweb
labels:
- traefik.http.routers.api.service=api#internal
- traefik.enable=true
- traefik.docker.network=myweb
remoteling:
image: remoteling:latest
container_name: remoteling
networks:
- myweb
labels:
- traefik.enable=true
- traefik.docker.network=myweb
- traefik.http.middlewares.http2https.redirectscheme.scheme=https
- traefik.http.routers.remoteling-http.middlewares=http2https
- traefik.http.routers.remoteling-http.rule=Host(`example.com`) || Host(`example.example.com`)
- traefik.http.routers.remoteling-http.entrypoints=web
- traefik.http.services.remoteling.loadbalancer.server.port=443
- traefik.http.routers.remoteling-https.rule=Host(`example.com`) || Host(`example.example.com`)
- traefik.http.routers.remoteling-https.entrypoints=websecure
- traefik.http.routers.remoteling-https.tls=true
depends_on:
- proxy
From powershell I'm running:
docker network create -d nat myweb
docker-compose -p myweb up
In my working dir I have the following file structure:
./docker-compose.yml
./traefik-config/traefik.yml
./traefik-ssl-certs/example.com.crt
./traefik-ssl-certs/example.com.key
./traefik-ssl-certs/example.example.com.key
./traefik-ssl-certs/example.example.com.key
And the traefik.yml file looks like this:
tls:
certificates:
- certFile: c:/certs/example.example.com.crt
keyFile: c:/certs/example.example.com.key
- certFile: c:/certs/example.com.crt
keyFile: c:/certs/example.com.key
The traefik dashboard shows my service defined as a loadbalancer, which is good (is there any other type of service other than loadbalancer? I'm not sure how else to define the port other than via a loadbalancer). Clicking on the service gives me the local network IP - and when I visit that IP my site loads just fine (albeit with an SSL cert mismatch warning - expected).
However when I try to visit https://example.com or https://example.example.com - I get a 502 Bad Gateway. Any idea why I'd be getting this?
The browser shows the SSL certs are valid, with no warnings, so I assume my certificates config is fine.
The http to https redirect seems to be working, if I visit http://example.com it forwards me to https://example.com
What is wrong with my routing config to cause these Bad Gateways?
I have solved my problem! There were a few issues:
I had configured a TLS cert in my Dockerfile, back from when I was running the image as a single service on my server. So my image already had port 443 bound with the TLS cert. I think that caused issues when the Traefik router tried to configure TLS for the service. So I had to rebuild my image, removing the TLS. I also removed the code that required HTTPS attributes too, that is now the responsibility of traefik.
Image only needs to expose 1 port: My understanding now is that my web application should only run through port 80 (e.g. don't even need port 443 exposed in the image's firewall) and the Traefik router configures and handles the TLS/443 etc through port 80.
I hadn't actually defined my HTTP and HTTPS services properly. I have shared my docker-compose.yml file below - notice for both traefik and my other service I have separated the labels into sections: defining the service, routing for HTTP, routing for HTTPS, redirect middleware, and basic auth for the traefik dashboard. I couldn't find any good documentation or tutorials that really broke down the necessary aspects for traefik 2.0 and categorized them clearly.
The service definition label / loadbalancer port should point to whatever port the image's service is hosted from, e.g. port 80 in most cases, or in the case of the traefik service 8080. But crucially it doesn't need to also point to 443 for https.
In summary my (verbose) learning notes afterall are, you need to:
Define the service again as a label: Didn't work for me without the service definition label. I had to add a service label pointing to the name of the service under which the image is defined.
Define routers for http and https: Whatever text you put after the traefik.http.routers.YOUR_ROUTER_NAME becomes your router. It wasn't clear to me I needed separate routers for both http and https. You must do this for each service (e.g. traefik, whoami, remoteling, etc.)
Define the routers' entrypoints: AFAIK the name you put after the entrypoint in the traefik command defines a new entrypoint that you can use for other services. So in the traefik service definition you'd have --entrypoints.WEBNAME.address=:80 and --entrypoints.WEBSECURENAME.address=:443 (replacing those caps with your own name to use throughout the docker-compose.yml file.
Define the routers' domain names to capture: Just like you define routers for each http and https you must define the domains to capture for both of these routers, even just the same domain/path.
Add tls for https router: For the https router you need the tls.true label.
Separate middleware redirection definitions for traefik and each of your services: I've read that you can declare a global one, but AFAIK each service must opt-in by assigning the middleware to the http router.
Providing your own SSL certs on Docker for Windows: There was precious little information about running Docker for Windows with your own SSL certs. I have a batch script in my 'traefik' working dir with a subfolder containing the SSL certs (crt and key files). I mount this as the first volume, - ./traefik-ssl-certs/:c:/certs/. Then, in my working dir I have another folder called traefik-config containing my traefik.yml file (details in the question above). In linux everyone seems to just mount the config file directly, but mounting files doesn't work in Windows so I had to mount as a folder instead, then I used the command providers.file.directory=c:/config/, which tells traefik to look for the config file in there. The config file provides the location of the SSL certs for traefik to load. If you enable TLS for a router, Traefik will automatically use any cert that matches the domain you've specified on that router.
Basic Auth: Must define the middleware as one label, and then assign that middleware to your https router. And I suppose if you're not using HTTPS redirect then you could assign it to your http router but it wouldn't be secure obviously.
docker-compose.yml:
version: '3.8'
networks:
myweb:
external: true
services:
proxy:
image: traefik:v2.3.0-rc4-windowsservercore-1809
container_name: traefik
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
# Mount the certs drive
- ./traefik-ssl-certs/:c:/certs/
# Mount the config folder
- ./traefik-config/:c:/config/
# Mount the host docker engine pipe ("docker volume ls")
- source: '\\.\pipe\docker_engine'
target: '\\.\pipe\docker_engine'
type: npipe
command:
- --api=true
- --api.dashboard=true
- --api.insecure=false
# Register the traefik config directory as per: https://docs.traefik.io/providers/file/#directory
- --providers.file.directory=c:/config/
- --providers.file.watch=true
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
# Configure Docker provider
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --providers.docker.endpoint=npipe:////./pipe/docker_engine
- --providers.docker.network=myweb
- --providers.docker.watch=true
networks:
- myweb
labels:
- traefik.enable=true
- traefik.docker.network=myweb
# Define the service
- traefik.http.services.proxy.loadbalancer.server.port=8080
# Routing for dashboard HTTP
- traefik.http.routers.dash-http.service=api#internal
- traefik.http.routers.dash-http.rule=Host(`example.com`)
- traefik.http.routers.dash-http.entrypoints=web
# Routing for dashboard HTTPS
- traefik.http.routers.dash-https.service=api#internal
- traefik.http.routers.dash-https.rule=Host(`example.com`)
- traefik.http.routers.dash-https.entrypoints=websecure
- traefik.http.routers.dash-https.tls=true
# Http-to-Https redirect Middleware
- traefik.http.middlewares.dash-http2https.redirectscheme.scheme=https
- traefik.http.middlewares.dash-http2https.redirectscheme.permanent=true
- traefik.http.routers.dash-http.middlewares=dash-http2https
# BasicAuth for dashboard
# Windows doesn't have htpasswd command so I generated one here: https://hostingcanada.org/htpasswd-generator/
# As per Traefik documentation, escaped single $ char with $$ for the yml parser
# user/pass = admin/testpassword
- traefik.http.middlewares.api-auth.basicauth.users=admin:$$2y$$10$$mfWQ11K16V6gVK.8Y6q1Eeh765NZscmjCrjJlAtaWubEsjU8HLYOO
- traefik.http.routers.dash-https.middlewares=api-auth
remoteling:
image: remoteling:latest
container_name: remoteling
networks:
- myweb
labels:
- traefik.enable=true
- traefik.docker.network=myweb
# Define the service
- traefik.http.services.remoteling.loadbalancer.server.port=80
# Routing for remoteling HTTP
- traefik.http.routers.remoteling-http.service=remoteling
- traefik.http.routers.remoteling-http.entrypoints=web
- traefik.http.routers.remoteling-http.rule=Host(`services.example.com`)
# Routing for remoteling HTTPS
- traefik.http.routers.remoteling-https.service=remoteling
- traefik.http.routers.remoteling-https.entrypoints=websecure
- traefik.http.routers.remoteling-https.rule=Host(`services.example.com`)
- traefik.http.routers.remoteling-https.tls=true
# Http-to-Https redirect Middleware
- traefik.http.middlewares.remoteling-http2https.redirectscheme.scheme=https
- traefik.http.middlewares.remoteling-http2https.redirectscheme.permanent=true
- traefik.http.routers.remoteling-http.middlewares=remoteling-http2https
depends_on:
- proxy
Hopefully someone else finds that useful.

Traefik + Docker for Windows: Failed to create a client for docker, error: protocol not available & Provider connection error protocol not available

I'm having troubles getting a basic Traefik routing setup to work.
My goal is to get basic routing with two helloworld apps (each different to tell apart), both on port 80, e.g.:
demo1.localhost -> helloworld1
demo2.localhost -> helloworld2
Each of the images works fine if I run them via docker run in isolation.
Using Powershell from my project dir, /app, when I run docker-compose up I get the following:
The Traefik service launches, I can visit the dashboard just fine but the routing table doesn't show my routes. demo1 and demo2 launch just fine, but obviously I can't connect to them because the routing isn't working.
Even though the services all launch successfully - I repeatedly get the following errors:
traefik | ... "Failed to create a client for docker, error: protocol not available" providerName=docker
traefik | ... "Provider connection error protocol not available, retrying ..." providerName=docker
I've included my docker-compose.yml file below, which is the only file in my dir, /app.
docker-compose.yml:
# app/docker-compose.yml
version: '3.8'
networks:
myweb:
driver: nat
services:
proxy:
image: traefik:v2.3.0-rc4-windowsservercore-1809
container_name: traefik
ports:
- "80:80"
- "8080:8080"
volumes:
- source: '\\.\pipe\docker_engine'
target: '\\.\pipe\docker_engine'
type: npipe
command:
- "--api.insecure=true"
- "--providers.docker"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
networks:
- myweb
labels:
- "traefik.http.routers.dashboard.rule=Host(`dash.localhost`)"
- "traefik.docker.network=app_myweb"
demo1:
image: helloworld:1
container_name: demo1
labels:
- "traefik.enable=true"
- "traefik.docker.network=app_myweb"
- "traefik.port=80"
- "traefik.http.routers.demo1.rule=Host(`demo1.localhost`)"
# Have tried this below, doesn't help.
# volumes:
# - source: '\\.\pipe\docker_engine'
# target: '\\.\pipe\docker_engine'
# type: npipe
networks:
- myweb
depends_on:
- proxy
demo2:
image: helloworld:2
container_name: demo2
labels:
- "traefik.enable=true"
- "traefik.docker.network=app_myweb"
- "traefik.port=80"
- "traefik.http.routers.demo2.rule=Host(`demo2.localhost`)"
networks:
- myweb
depends_on:
- proxy
I saw a suggestion somewhere that I should enable the setting "Expose daemon on tcp://localhost:2375 without TLS" in Docker Desktop settings, which I have done but doesn't help.
My setup is:
Docker Desktop (v19.03.12) for Windows
Docker using Windows Containers
Windows 10 (10.0.18363 Build 18363)
Question #1:
Anybody have any idea what might be causing the problem?
Question #2:
Notice in my file I also have a route set up for the dashboard, to route from dash.localhost to localhost:8080/dashboard, but even that doesn't work. Any idea how to get that working? Do I need to tell it to route from 80->8080 for the dashboard?
According to a ticket on their GitHub you seem to be:
Missing --providers.docker.endpoint=npipe:////./pipe/docker_engine in Traefik command line
Sharing \\.\pipe\docker_engine when Docker is expecting .\pipe\docker_engine
Try making those two changes and see if that helps Traefik connect to your Docker daemon. None of your routes will work until Traefik can talk to Docker to read the labels of your containers.

Traefik configuration with Docker not working

I am trying to set up a sample application with the Traefik reverse proxy in Docker.
I am using Traefik v2.2 for this project which has significant differences from Traefik.v1.0.
Here is my docker-compose.yml file:
version: '3'
services:
traefik:
# The official v2 Traefik docker image
image: traefik:v2.2
# Enables the web UI and tells Traefik to listen to docker
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
# The HTTP port
- "89:80"
# The Web UI (enabled by --api.insecure=true)
- "8089:8080"
volumes:
# So that Traefik can listen to the Docker events
- "/var/run/docker.sock:/var/run/docker.sock:ro"
whoami:
# A container that exposes an API to show its IP address
image: containous/whoami
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
- "traefik.http.routers.whoami.entrypoints=web"
I can access Traefik's dashboard when I go to localhost:8089 on my web browser, but I cannot access the whoami application when I type in the whoami.localhost address on my web browser. I'm just wondering if there is anything I need to change before I can access it, or do I need to change the host from whoami.localhost to localhost:3000 since that's the port I want to access the application in.
One problem I am spotting is that you exposed container port 80 of the traefik container to the host port 89. If you type in whoami.localhost in your web browser, your browser is going to search for an application on host port 80 at that address (since localhost maps natively to port 80), but it is not going to find anything there, because it can only be found at port 89. From my understanding, you should be able to access the application via the command line with the command curl -H Host:whoami.localhost http://127.0.0.1:89. Unfortunately, I am unsure how the URL whoami.localhost:89 is handled by your browser respectively by your DNS.
You can to modify the docker-compose.yml file this way:
version: "3"
services:
traefik:
# The official v2 Traefik docker image
image: traefik:v2.2
# Enables the web UI and tells Traefik to listen to docker
command:
- --api.insecure=true
- --providers.docker=true
ports:
# The HTTP port
- "89:80"
# The Web UI (enabled by --api.insecure=true)
- "8089:8080"
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
whoami:
# A container that exposes an API to show its IP address
image: containous/whoami
labels:
- traefik.http.routers.whoami.rule=Host(`whoami.localhost`)
And then you can access the application on your command terminal by typing in:
curl -H Host:whoami.localhost http://127.0.0.1:89
Note: whoami.localhost can be whoami.docker.localhost or app.localhost or whatever you want. The thing here is that you should localhost attached to the end, except if you're adding a Fully Qualifies Domain name (FQDN).
That's all.
I hope this helps

Resources