Related
I have an IngressRouteTCP configured for Traefik running in an AKS cluster (behind an Azure load balancer). I'm trying to do routing based on SNI, rather than on the hostname header. The certificate is generated by Cloudflare for test.example.com.
As you can see below, it doesn't work. What does work, is setting a TLSOption with alpnProtocols to http/1.1. But that would default to http/1 as I understand it. My application supports http2, so I'd prefer to use that.
I'm not sure why it fails? Is it Traefik, curl or the application?
Testing it with curl -svk --connect-to test.example.com:443:my-azure-load-balancer.cloudapp.azure.com:443 https://test.example.com gives this:
* Connecting to hostname: my-azure-load-balancer.cloudapp.azure.com
* Connecting to port: 443
* Trying x.x.x.x:443...
* TCP_NODELAY set
* Connected to my-azure-load-balancer.cloudapp.azure.com (x.x.x.x) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: O=CloudFlare, Inc.; OU=CloudFlare Origin CA; CN=CloudFlare Origin Certificate
* start date: Sep 20 12:40:00 2022 GMT
* expire date: Sep 16 12:40:00 2037 GMT
* issuer: C=US; O=CloudFlare, Inc.; OU=CloudFlare Origin SSL Certificate Authority; L=San Francisco; ST=California
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x558d30129860)
> GET / HTTP/2
> Host: test.example.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* http2 error: Remote peer returned unexpected data while we expected SETTINGS frame. Perhaps, peer does not support HTTP/2 properly.
* Connection #0 to host my-azure-load-balancer.cloudapp.azure.com left intact
I'm using Traefik version 2.9.1
This is my Kubernetes configuration
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
spec:
replicas: 2
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: traefik/whoami:v1.6.0
imagePullPolicy: IfNotPresent
---
apiVersion: v1
kind: Service
metadata:
name: whoami
labels:
app: whoami
spec:
type: ClusterIP
ports:
- port: 80
name: whoami
selector:
app: whoami
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: whoami
spec:
entryPoints:
- websecure
routes:
- match: HostSNI(`test.example.com`)
services:
- name: whoami
port: 80
tls:
secretName: cloudflare-cert
These are the only logs related to this that I can find in the logs (the second log line doesn't always come though):
traefik-c757597b9-2xv65 time="2022-10-31T08:52:59Z" level=debug msg="Handling connection from 10.9.3.227:61988 to 10.9.3.73:80"
traefik-c757597b9-2xv65 time="2022-10-31T08:52:59Z" level=debug msg="Error during connection: read tcp 10.9.3.58:34400->10.9.3.73:80: read: connection reset by peer"
Those IPs are:
10.9.3.227 - The Kubernetes node
10.9.3.58 - Traefik
10.9.3.73 - The service for the whoami pod
I initially tested this behind Cloudflare, and received this:
I am attempting to have Traefik serve as a reverse proxy for services running in Docker containers. I've been following the documentation that Traefik provides and have a small docker environment configured via docker compose that successfully serves data via HTTP. Traefik sits behind HAProxy running in TCP mode forwarding packets received from the Internet to Traefik.
However when I tried to add a new router for serving the same content via HTTPS, I receive the following esoteric (to me) error when I run a curl directed to https://my.domain.tld/: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Full curl output:
curl -v https://my.domain.tld/
* Trying <IP Address of Domain>...
* TCP_NODELAY set
* Connected to my.domain.tld (<IP Address of Domain>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
When I attempt to browse to the site via Firefox (web browser) I receive an error code of SSL_ERROR_RX_RECORD_TOO_LONG. When googling this error I was unable to find a post that seemed to have my specific issue.
Below is the docker-compose for the setup I am using to configure the applications
version: "3.9"
secrets:
cloudflare_dns_token:
file: ./secrets/cf_dns_api_token.txt
networks:
socket_proxy:
name: socket_proxy
driver: bridge
ipam:
config:
- subnet: 192.168.0.0/24
container_bridge:
name: container_bridge
driver: bridge
ipam:
config:
- subnet: 192.168.1.0/24
services:
socket-proxy:
image: tecnativa/docker-socket-proxy
container_name: socket-proxy
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
socket_proxy:
ipv4_address: 192.168.0.2 # Static IP address
environment:
EVENTS: 1
PING: 1
VERSION: 1
CONTAINERS: 1
NETWORKS: 1
traefik:
# The official v2 Traefik docker image
image: traefik:v2.8.1
container_name: traefik-proxy
command:
# Log Level for Traefik
- "--log.level=DEBUG"
# Enables the web UI
- "--api.insecure=true"
# Traefik enables docker as the provider to look for services
- "--providers.docker=true"
# Traefik will use the Docker Socket proxy to communicate with the docker socket
- "--providers.docker.endpoint=tcp://192.168.0.2:2375"
# Traefik will not expose services if they aren't labled for export
- "--providers.docker.exposedByDefault=false"
# Port where Traefik will listen for web (http) traffic for routing
- "--entrypoints.web.address=:80"
# Port where Traefik will listen for web secure (https) traffic for routing
- "--entrypoints.websecure.address=:443"
# Trust Proxy Protocol Packets from only the listed IP address
- "--entryPoints.web.proxyProtocol.trustedIPs=10.0.8.1/32"
# Trust Proxy Protocol Packets from only the listed IP address
- "--entryPoints.websecure.proxyProtocol.trustedIPs=10.0.8.1/32"
# Enable a ACME DNS challenge named "letsencrypt"
- "--certificatesresolvers.letsencrypt.acme.dnschallenge=true"
# Tell Traefik which provider to use for DNS Challenge
- "--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare"
# Staging environment for let's encrypt for testing
- "--certificatesresolvers.letsencrypt.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
# Email to provide to let's encrypt
- "--certificatesresolvers.letsencrypt.acme.email=${EMAIL}"
# Tell Traefik to store the certificate on a path under our volume
- "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
networks:
# Tells Traefik to connect to both the socket proxy network and the container bridge network where the other containers will be connected
socket_proxy:
container_bridge:
ports:
# The HTTP port
- "80:80"
# The HTTPS port
- "443:443"
# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
secrets:
- "cloudflare_dns_token"
environment:
# To Be Removed Need Secret Working Properly
- "CF_DNS_API_TOKEN=${CF_DNS_TOKEN}"
#- "CF_DNS_API_TOKEN=/run/secrets/cloudflare_dns_token"
volumes:
# Create a letsencrypt dir within the folder where the docker-compose file is
- "./letsencrypt:/letsencrypt"
whoami:
image: traefik/whoami
container_name: whoami-server
networks:
container_bridge:
labels:
# Tells Traefik to proxy to the service (container)
- "traefik.enable=true"
#####################################################################
#
# Labels for HTTPS Proxying
#
#####################################################################
# Explicitly stating 'whoami-secure' route is HTTPS
- "traefik.http.routers.whoami-secure.tls=true"
# Rule for determing when to route requests to this service for the secure http router
- "traefik.http.routers.whoami-secure.rule=Host(`whoami.${FQDN}`)"
# Entry point for requests to this service for the secure http router
- "traefik.http.routers.whoami-secure.entrypoints=websecure"
# Uses the Host rule to define which certificate to issue
- "traefik.http.routers.whoami-secure.tls.certresolver=letsencrypt"
#####################################################################
#
# Labels for HTTP Proxying
#
#####################################################################
# Rule for determing when to route requests to this service for the unsecure http router
- "traefik.http.routers.whoami.rule=Host(`whoami.${FQDN}`)"
# Entry point for requests to this service for the unsecure http router
- "traefik.http.routers.whoami.entrypoints=web"
My expectation is that Traefik would gracefully handle the request via HTTPS and manage the TLS handshake without issue. I can confirm that Traefik is able to successfully generate a certificate via Let's Encrypt DNS Challenge for Cloudflare. I am using the Let's Encrypt staging environment at the moment so I did expect an error about the certificate being served as invalid, but it seems that TLS handshake errors out before a determination of validity.
EDIT #1: Running OpenSSL and Wireshark
OpenSSL returns the following when run ```openssl s_client -connect my.domain.tld:443``
CONNECTED(00000003)
140330304906560:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:331:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 315 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
Wireshark logs show the following:
3 TCP packets preceding the first TLSv1 Client Hello
The Client Hello is acknowledged by the server
Then the server returns a HTTP 400 error - Bad Request
Wireshark dump of error
Transmission Control Protocol, Src Port: 443, Dst Port: 50085, Seq: 1, Ack: 518, Len: 207
Source Port: 443
Destination Port: 50085
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 207]
Sequence Number: 1 (relative sequence number)
Sequence Number (raw): 1488204866
[Next Sequence Number: 208 (relative sequence number)]
Acknowledgment Number: 518 (relative ack number)
Acknowledgment number (raw): 500352812
0101 .... = Header Length: 20 bytes (5)
Flags: 0x018 (PSH, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 1... = Push: Set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······AP···]
Window: 501
[Calculated window size: 64128]
[Window size scaling factor: 128]
Checksum: 0x1155 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
[SEQ/ACK analysis]
TCP payload (207 bytes)
Hypertext Transfer Protocol
[Expert Info (Warning/Security): Unencrypted HTTP protocol detected over encrypted port, could indicate a dangerous misconfiguration.]
[Unencrypted HTTP protocol detected over encrypted port, could indicate a dangerous misconfiguration.]
[Severity level: Warning]
[Group: Security]
HTTP/1.1 400 Bad request\r\n
[Expert Info (Chat/Sequence): HTTP/1.1 400 Bad request\r\n]
[HTTP/1.1 400 Bad request\r\n]
[Severity level: Chat]
[Group: Sequence]
Response Version: HTTP/1.1
Status Code: 400
[Status Code Description: Bad Request]
Response Phrase: Bad request
Content-length: 90\r\n
Cache-Control: no-cache\r\n
Connection: close\r\n
Content-Type: text/html\r\n
\r\n
[HTTP response 1/1]
File Data: 90 bytes
5 packets later the connection is reset
Wireguard dump of connection reset
Transmission Control Protocol, Src Port: 443, Dst Port: 50085, Seq: 208, Len: 0
Source Port: 443
Destination Port: 50085
[Stream index: 0]
[Conversation completeness: Complete, WITH_DATA (63)]
[TCP Segment Len: 0]
Sequence Number: 208 (relative sequence number)
Sequence Number (raw): 1488205073
[Next Sequence Number: 208 (relative sequence number)]
Acknowledgment Number: 0
Acknowledgment number (raw): 0
0101 .... = Header Length: 20 bytes (5)
Flags: 0x004 (RST)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...0 .... = Acknowledgment: Not set
.... .... 0... = Push: Not set
.... .... .1.. = Reset: Set
[Expert Info (Warning/Sequence): Connection reset (RST)]
[Connection reset (RST)]
[Severity level: Warning]
[Group: Sequence]
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·········R··]
Window: 0
[Calculated window size: 0]
[Window size scaling factor: 128]
Checksum: 0x6dc9 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
[Timestamps]
I am running rabbitmq:3.10.7-management for AMQPS on a VM and I am using chained TLS-Certificates.
When I configure Rabbitmq according to the How-To-TLS website from Rabbitmq I get no errors starting the container but I can't connect via TLS and get an immediate Connection-Error. Depending on which Tool I use or where I use it I get a connection reset by peer. But it seems like the handshake does not work at all.
When I enable TCP I can just connect fine with a non-TLS-client.
The Rabbitmq part of the Docker-Compose looks like this:
rabbitmq:
restart: unless-stopped
hostname: rabbitmq
image: rabbitmq:3.10.7-management
networks:
- traefik
ports:
- "5672:5672"
- "5671:5671"
logging:
options:
max-size: "10m"
max-file: "3"
volumes:
- ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
- ./certs:/certs
- /data/rabbitmq_data:/var/lib/rabbitmq
labels:
- "traefik.enable=true"
- "traefik.http.routers.rabbitmq-secure.priority=150"
- "traefik.http.services.rabbitmq-secure.loadbalancer.server.port=15672"
- "traefik.http.routers.rabbitmq-secure.rule=Host(`<myDomain>`) && PathPrefix(`/`)"
- "traefik.http.routers.rabbitmq-secure.entrypoints=web-secure"
- "traefik.http.routers.rabbitmq-secure.tls=true"
- "traefik.http.routers.rabbitmq-secure.tls.options=myTLSOptions#file"
- "traefik.http.routers.rabbitmq.rule=Host(`<myDomain>`) && PathPrefix(`/`)"
- "traefik.http.routers.rabbitmq.entrypoints=web"
I am using the same certs to serve the management frontend with no problems.
ca_certificate.pem ->
CN=T-TeleSec GlobalRoot Class 2, OU=T-Systems Trust Center, O=T-Systems Enterprise Services GmbH, C=DE
Chain-Cert 1
Chain-Cert 2
server_certificate.pem ->
Wild-Card-Cert
server_key.pem ->
Key for Wild-Card-Cert
I think I tried almost every other configuration as well, for which part goes where.
RabbitMQ-Conf:
log.console.level = debug
listeners.ssl.default = 5671
listeners.tcp = none
ssl_options.cacertfile = /certs/ca_certificate.pem
ssl_options.certfile = /certs/server_certificate.pem
ssl_options.keyfile = /certs/server_key.pem
ssl_options.verify = verify_none
ssl_options.fail_if_no_peer_cert = false
ssl_options.versions.1 = tlsv1.3
ssl_options.versions.2 = tlsv1.2
ssl_options.honor_cipher_order = false
ssl_options.honor_ecc_order = false
ssl_handshake_timeout = 10000
ssl_options.ciphers.1 = ECDHE-ECDSA-AES256-GCM-SHA384
ssl_options.ciphers.2 = ECDHE-RSA-AES256-GCM-SHA384
ssl_options.ciphers.3 = ECDHE-ECDSA-AES128-GCM-SHA256
ssl_options.ciphers.4 = ECDHE-RSA-AES128-GCM-SHA256
ssl_options.ciphers.5 = ECDHE-ECDSA-CHACHA20-POLY1305
ssl_options.ciphers.6 = ECDHE-RSA-CHACHA20-POLY1305
ssl_options.ciphers.7 = DHE-RSA-AES128-GCM-SHA256
ssl_options.ciphers.8 = DHE-RSA-AES256-GCM-SHA384
Logs seem to be OK:
...
2022-09-06 14:42:22.024539+00:00 [info] <0.615.0> started TLS (SSL) listener on [::]:5671
...
Going through "Troubleshooting TLS-enabled Connections" I can check all the boxes I can find up until connecting with TLS:
Check Listeners:
rabbitmq#rabbitmq:/$ rabbitmq-diagnostics listeners
Asking node rabbit#rabbitmq to report its protocol listeners ...
Interface: [::], port: 15672, protocol: http, purpose: HTTP API
Interface: [::], port: 15692, protocol: http/prometheus, purpose: Prometheus exporter API over HTTP
Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Interface: [::], port: 5671, protocol: amqp/ssl, purpose: AMQP 0-9-1 and AMQP 1.0 over TLS
Check permissions:
rabbitmq#rabbitmq:/certs$ ls -ll
total 20
-rw-r--r-- 1 rabbitmq rabbitmq 1366 Sep 6 14:39 ca_certificate.pem
-rw-r--r-- 1 rabbitmq rabbitmq 6943 Sep 6 14:39 server_certificate.pem
-rw-r--r-- 1 rabbitmq rabbitmq 3275 Sep 5 13:20 server_key.pem
Check TLS-support in erlang:
rabbitmq#rabbitmq:/certs$ rabbitmq-diagnostics --silent tls_versions
tlsv1.3
tlsv1.2
tlsv1.1
tlsv1
Attempt TLS-Connection with openssl
rabbitmq#rabbitmq:/certs$ openssl s_client -connect localhost:5671
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 293 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
The debug logs don't even show any error when trying to connect.
Trying to connect with TLS to TCP throws an error, though. Somehow the listener does not accept connections at all.
Maybe someone experienced something similar already or I am just dumb overlooking the obvious. But for everything else the VM and the certs work just fine (e.g. Mosquitto-MQTTS).
Help would be really appreciated.
First time using Docker + Sail + Caddy. I copied the Caddy setup from here. I have a local Laravel dev project which uses 2x subdomains (sd1.project.local, sd2.project.local).
Everything looks fine on sail up. I can see the certs created in the expected local stores, one for each subdomain. I then import the two certs to Chrome.
I can get to the site, though Chrome does not trust it. I have restarted Chrome after the cert imports.
What have I missed / messed? Thanks!
The error of curl -vvv on sd1.project.local is:
$ curl -vvv https://sd1.project.local
* Trying 127.0.0.1:443...
* TCP_NODELAY set
* Connected to sd1.project.local (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
The output of the sail up command for caddy is:
project-caddy-1 | {"level":"info","ts":1651470513.5381901,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
project-caddy-1 | {"level":"warn","ts":1651470513.5383825,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-Host: the reverse proxy's default behavior is to pass headers to the upstream"}
project-caddy-1 | {"level":"warn","ts":1651470513.5384026,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream"}
project-caddy-1 | {"level":"warn","ts":1651470513.5386436,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-Host: the reverse proxy's default behavior is to pass headers to the upstream"}
project-caddy-1 | {"level":"warn","ts":1651470513.5386622,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream"}
project-caddy-1 | {"level":"warn","ts":1651470513.5391574,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
project-caddy-1 | {"level":"warn","ts":1651470513.5395052,"logger":"admin","msg":"admin endpoint disabled"}
project-caddy-1 | {"level":"info","ts":1651470513.5397356,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0001264d0"}
project-caddy-1 | {"level":"info","ts":1651470513.5441852,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
project-caddy-1 | {"level":"warn","ts":1651470513.5442154,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv1","http_port":80}
project-caddy-1 | {"level":"info","ts":1651470513.5448365,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}
project-caddy-1 | {"level":"info","ts":1651470513.544853,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["sd1.project.local","sd2.project.local"]}
project-caddy-1 | {"level":"info","ts":1651470513.545812,"logger":"tls.obtain","msg":"acquiring lock","identifier":"sd1.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.5458567,"logger":"tls.obtain","msg":"acquiring lock","identifier":"sd2.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.5476136,"logger":"tls.obtain","msg":"lock acquired","identifier":"sd1.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.5479813,"logger":"tls","msg":"finished cleaning storage units"}
project-caddy-1 | {"level":"info","ts":1651470513.5485632,"logger":"tls.obtain","msg":"lock acquired","identifier":"sd2.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.5563135,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"sd1.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.556336,"logger":"tls.obtain","msg":"releasing lock","identifier":"sd1.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.5574615,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"sd2.project.local"}
project-caddy-1 | {"level":"info","ts":1651470513.557494,"logger":"tls.obtain","msg":"releasing lock","identifier":"sd2.project.local"}
project-caddy-1 | {"level":"warn","ts":1651470513.5588439,"logger":"pki.ca.local","msg":"installing root certificate (you might be prompted for password)","path":"storage:pki/authorities/local/root.crt"}
project-caddy-1 | 2022/05/02 05:48:33 define JAVA_HOME environment variable to use the Java trust
project-caddy-1 | 2022/05/02 05:48:33 not NSS security databases found
project-caddy-1 | {"level":"warn","ts":1651470513.561766,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [sd1.project.local]: no OCSP server specified in certificate","identifiers":["sd1.project.local"]}
project-caddy-1 | {"level":"warn","ts":1651470513.5634398,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [sd2.project.local]: no OCSP server specified in certificate","identifiers":["sd2.project.local"]}
project-caddy-1 | 2022/05/02 05:48:33 certificate installed properly in linux trusts
project-caddy-1 | {"level":"info","ts":1651470513.5809436,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
project-caddy-1 | {"level":"info","ts":1651470513.580976,"msg":"serving initial configuration"}
My docker-compose is:
# For more information: https://laravel.com/docs/sail
version: '3'
services:
project.local:
build:
context: ./vendor/laravel/sail/runtimes/8.0
dockerfile: Dockerfile
args:
WWWGROUP: '${WWWGROUP}'
image: sail-8.0/app
extra_hosts:
- 'host.docker.internal:host-gateway'
# ports:
# - "${APP_PORT:-80}:80"
environment:
WWWUSER: '${WWWUSER}'
LARAVEL_SAIL: 1
XDEBUG_MODE: '${SAIL_XDEBUG_MODE:-off}'
XDEBUG_CONFIG: '${SAIL_XDEBUG_CONFIG:-client_host=host.docker.internal}'
GITHUB_TOKEN: '${GITHUB_TOKEN}'
FONTAWESOME_NPM_AUTH_TOKEN: '${FONTAWESOME_NPM_AUTH_TOKEN}'
volumes:
- '.:/var/www/html'
networks:
- sail
depends_on:
- redis
- mailhog
redis:
build:
context: "./Docker/Redis"
dockerfile: Dockerfile
privileged: true
command: sh -c "./init.sh"
ports:
- "${FORWARD_REDIS_PORT:-6379}:6379"
volumes:
- "sail-redis:/data"
networks:
- sail
healthcheck:
test: ["CMD", "redis-cli", "ping"]
retries: 3
timeout: 5s
mailhog:
image: "mailhog/mailhog:latest"
ports:
- "${FORWARD_MAILHOG_PORT:-1025}:1025"
- "${FORWARD_MAILHOG_DASHBOARD_PORT:-8025}:8025"
networks:
- sail
caddy:
build:
context: "./Docker/Caddy"
dockerfile: Dockerfile
args:
WWWGROUP: "${WWWGROUP}"
restart: unless-stopped
ports:
- "${APP_PORT:-80}:80"
- "${APP_SSL_PORT:-443}:443"
environment:
LARAVEL_SAIL: 1
HOST_DOMAIN: project.local
volumes:
- "./Docker/Caddy/file:/etc/caddy"
- ".:/srv:cache"
- "./Docker/Caddy/certificates:/data/caddy/certificates/local"
- "./Docker/Caddy/authorities:/data/caddy/pki/authorities/local"
- "sailcaddy:/data:cache"
- "sailcaddyconfig:/config:cache"
networks:
- sail
depends_on:
- project.local
networks:
sail:
driver: bridge
volumes:
sail-redis:
driver: local
sailcaddy:
external: true
sailcaddyconfig:
driver: local
My Caddifile is:
{
admin off
# debug
on_demand_tls {
ask http://project.local/caddy
}
local_certs
default_sni project
}
:80 {
reverse_proxy project.local {
header_up Host {host}
header_up X-Real-IP {remote}
header_up X-Forwarded-Host {host}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Port 443
# header_up X-Forwarded-Proto {scheme}
health_timeout 5s
}
}
:443 {
tls internal {
on_demand
}
reverse_proxy project.local {
header_up Host {host}
header_up X-Real-IP {remote}
header_up X-Forwarded-Host {host}
header_up X-Forwarded-For {remote}
header_up X-Forwarded-Port 443
# header_up X-Forwarded-Proto {scheme}
health_timeout 5s
}
}
sd1.project.local {
reverse_proxy project.local
}
sd2.project.local {
reverse_proxy project.local
}
My Dockerfile is:
FROM caddy:alpine
LABEL maintainer="Adrian Mejias"
ARG WWWGROUP
ENV DEBIAN_FRONTEND noninteractive
ENV TZ=UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apk add --no-cache bash \
&& apk add --no-cache nss-tools \
&& rm -rf /var/cache/apk/*
RUN addgroup -S $WWWGROUP
RUN adduser -G $WWWGROUP -u 1337 -S sail
COPY start-container /usr/local/bin/start-container
RUN chmod +x /usr/local/bin/start-container
ENTRYPOINT ["start-container"]
My start-container is:
#!/usr/bin/env sh
if [ ! -z "$WWWUSER" ]; then
addgroup $WWWUSER sail
fi
if [ $# -gt 0 ];
then
# #todo find alpine equivilent of below
# exec gosu $WWWUSER "$#"
else
/usr/bin/caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
fi
I've spend hours looking to solve this issue, however I'm unable to find any topics related to this issue, since all I find is custom registeries.
When running any of the docker commands that connect to docker hub, either through https://registry-1.docker.io/v2/ or https://index.docker.io/v1, all requests end up in "x509: certificate signed by unknown authority". However using curl to run query the same endpoints seem to function just fine.
I've reinstalled docker completely, purging all configuration files, however it does not seem to make a difference.
Anything I'm missing?
docker info:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 17.05.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.35-1-lts
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.34GiB
ID: 5Q4D:TLJF:3I3U:O522:VQMK:24BU:H5ND:UPOU:MWYS:WGTB:XFXR:BQES
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Ena
Using docker:
[user#hostname]$ docker search ubunut
Error response from daemon: Get https://index.docker.io/v1/search?q=ubunut&n=25: x509: certificate signed by unknown authority
Using curl:
[user#hostname]$ curl -v https://index.docker.io/v1/search?q=ubunut&n=25
[1] 2152
[user#hostname]$ * Trying 34.200.194.233...
* TCP_NODELAY set
* Connected to index.docker.io (34.200.194.233) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: OU=GT98568428; OU=See www.rapidssl.com/resources/cps (c)15; OU=Domain Control Validated - RapidSSL(R); CN=*.docker.io
* start date: Mar 19 17:34:32 2015 GMT
* expire date: Apr 21 01:51:52 2018 GMT
* subjectAltName: host "index.docker.io" matched cert's "*.docker.io"
* issuer: C=US; O=GeoTrust Inc.; CN=RapidSSL SHA256 CA - G3
* SSL certificate verify ok.
> GET /v1/search?q=ubunut HTTP/1.1
> Host: index.docker.io
> User-Agent: curl/7.54.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.6.2
< Date: Wed, 05 Jul 2017 12:10:22 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Vary: Cookie
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000
<
{"num_pages": 1, "num_results": 21, "results": [{"is_automated": true, "name": "han4wluc/try-docker-ubunut-node", "is_trusted": true, ... *truncated*
I solved the problem as follows:
I removed the file /etc/ssl/cert/ca-certificates.crt.
I ran the command sudo pacman -S ca-certificates-utils.
I restarted docker with the systemctl restart docker command.
I got this hint from this link:
https://unix.stackexchange.com/questions/339613/arch-linux-ca-certificates-crt-not-found/396169#396169