Docker images and SSL certificates - docker

As a neophyte to Docker and SSL, I can't get Docker to expose a secure (443) port no matter how hard I try, and it keeps giving me a certificate error...it wants one.
But search as I might on the internet I can't find any simple, plain language explanation of how to do this. Most of what I see assumes you already know Docker and SSL, and the rest are all edge cases - people trying to figure out how to do things outside the box.
Right now I'd just be happy with the box.
All I need to know is three things:
How to create a self signed certificate (I think I've done this, I just want to be sure I did it right).
How to get it into a Docker image at build (Dockerfile command?)
How to tell my service (inside the container) and my client (from outside the container) to use the secure SSL port (443) by way of the certificate when communicating accross the container boundary.
Can anyone give me a simple "...do this, then this..." explanation. I swear I'm not stupid, I can take it from there.

I'm not sure this is programming or development -- containerization has made the boundary very fuzzy. But no one has voted to close, or answered, so I'll try an answer, though frankly I'm skeptical it'll really help.
First to be clear, docker doesn't implement SSL/TLS for a container, or do anything with certificates. Docker does networking at the TCP/IP level only. An application in a container can do SSL/TLS the same -- or very nearly the same -- as when running outside. Your second formulation -- "my service (inside the container) ... to use SSL ..." -- is the correct one.
Second, 443 is the standard port not just for SSL/TLS but for HTTPS = HTTP over SSL/TLS. You aren't clear if that's what you want, or if you want any of the hundreds of other protocols that use SSL/TLS and could use 443, although most usually don't. I'll answer the former because the latter would be much more work, and if that's not what you wanted you should have been clearer.
So restating, you want a self-signed cert (and matching privatekey) and an HTTPS server on 443 in a container, and a client on the host.
There are many hundreds of ways to create a self-signed cert, and key, depending partly on what kind of server you want to run (especially as to the file format) and what kind(s) of client(s) you want to connect to it (especially as to some fields in the cert). Based on my choices below I want OpenSSL file format(s), and the easiest way is:
$ openssl req -newkey rsa:2048 -nodes -keyout key -x509 -sha256 -days 3650 -subj /CN=localhost -out crt
Now, the easiest trivial HTTPS server I can write is in nodejs, for which hub conveniently has images, so to run the server in a container I do:
$ echo "let rf=require('fs').readFileSync; require('https').createServer({key:rf('/app/key'),cert:rf('/app/crt')},(req,res)=>{res.end('yay!\n')}).listen(443); console.log('ready\n')" \
| sudo docker run -ip443:443 -v$PWD:/app node:lts-alpine
ready
-v$PWD:/app makes the key and crt files I created with openssl available to the nodejs server code. -p443:443 makes the port 443 created in the container by that code accessible as 443 on the host. Note there must not be any other server on the host using 443 (or any other container mapping to it).
Finally, the easiest client to use with a PEM-format certificate and an arbitrary response is curl in a different terminal on the same host:
$ curl -v --cacert crt https://localhost
* Rebuilt URL to: https://localhost/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=localhost
* start date: Feb 23 08:09:00 2022 GMT
* expire date: Feb 21 08:09:00 2032 GMT
* common name: localhost (matched)
* issuer: CN=localhost
* SSL certificate verify ok.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.58.0
> Accept: */*
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/1.1 200 OK
< Date: Wed, 23 Feb 2022 08:09:30 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Content-Length: 5
<
yay!
* Connection #0 to host localhost left intact
$
The yay! in the next to last line is the response from the server (to any request I make).

Related

Docker build failed to solve with frontend dockerfile.v0: failed to create LLB definition

My docker build is failing with the following error:
#3 [internal] load metadata for pdr.akamai.com/library/tomcat9-java11:platform-2.19.4-RELEASE
#3 sha256:fe12d438fd0a4596fbe4a8af247e39a88858ee59a23a35a954fc8676153423fd
#3 ERROR: failed to do request: Head "https://pdr.akamai.com/v2/library/tomcat9-java11/manifests/platform-2.19.4-RELEASE": dial tcp 127.10.0.28:443: connect: connection refused
------
> [internal] load metadata for pdr.akamai.com/library/tomcat9-java11:platform-2.19.4-RELEASE:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to do request: Head "https://pdr.akamai.com/v2/library/tomcat9-java11/manifests/platform-2.19.4-RELEASE":
dial tcp 127.10.0.28:443: connect: connection refused
It looked like the server was not reachable from my machine.
However when I tried curl command with verbose option, it's the same IP address that dns resolves to and I get a response.
Following is the output of curl command
❯ curl --verbose https://pdr.akamai.com/v2/library/tomcat9-java11/manifests/platform-2.19.4-RELEASE
* Trying 127.10.0.28:443...
* Connected to pdr.akamai.com (127.10.0.28) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=US; ST=Massachusetts; L=Cambridge; O=Akamai Technologies; CN=pdr.akamai.com
* start date: Oct 26 12:53:44 2022 GMT
* expire date: Apr 26 12:53:44 2023 GMT
* subjectAltName: host "pdr.akamai.com" matched cert's "pdr.akamai.com"
* issuer: C=US; ST=Massachusetts; L=Cambridge; O=Akamai Technologies; OU=KMI; CN=kdc_ca.2
* SSL certificate verify ok.
> GET /v2/library/tomcat9-java11/manifests/platform-2.19.4-RELEASE HTTP/1.1
> Host: pdr.akamai.com
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Tue, 10 Jan 2023 09:00:20 GMT
< Server: Apache
< Docker-Distribution-Api-Version: registry/2.0
< Content-Length: 34915
< Content-Type: application/vnd.docker.distribution.manifest.v1+prettyjws
< Docker-Content-Digest: sha256:98be2134cf221dc1bdb43a65cf7d7ce49a748ef02ad95e52fc6f999484f1cb91
< Docker-Distribution-Api-Version: registry/2.0
< Etag: "sha256:98be2134cf221dc1bdb43a65cf7d7ce49a748ef02ad95e52fc6f999484f1cb91"
< X-Content-Type-Options: nosniff
<
I tried flushing the DNS cache and still the issue remains. Is there some configuration I am missing?
There seemed to be some issue with docker. I uninstalled docker completely including all config files and reinstalled it again. It's working now.

How to fix DeviceTokenNotForTopic for iOS push notification?

I am getting DeviceTokenNotForTopic when trying to test push notification in sandbox environment.
#!/bin/bash
curl -v -d '{
"aps": {
"alert": {
"title": "Hello Mate",
"body": "Thanks"
},
"sound": "default",
"badge": 2
},
"aditional-info": "Test"
}' -H "apns-topic: ca.xlogix.test.ios.push.example" -H "apns-priority: 10" --http2 --cert aps-dev.pem https://api.sandbox.push.apple.com/3/device/7b97568a823edevicetokenhere7c4f5
I am following the example from https://gokulgovind26.medium.com/make-your-own-apns-pusher-40aeabe5034e
I have downloaded the Apple Sandbox Push Services certificate, imported to keychain, exported to p12 and converted to pem.
func pushRegistry(_ registry: PKPushRegistry, didUpdate credentials: PKPushCredentials, for type: PKPushType) {
print(credentials.token)
let deviceToken = credentials.token.map { String(format: "%02x", $0) }.joined()
print("pushRegistry -> deviceToken :\(deviceToken)")
}
How to fix this?
I also tried Pusher, but it is not connecting to the APNS server. Also using latest certificate, import fails saying no certificate found.
./push.sh
* Trying 17.188.168.149:443...
* Connected to api.sandbox.push.apple.com (17.188.168.149) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
* CApath: none
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Request CERT (13):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Certificate (11):
* (304) (OUT), TLS handshake, CERT verify (15):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=api.development.push.apple.com; OU=management:idms.group.533599; O=Apple Inc.; ST=California; C=US
* start date: Dec 10 00:29:46 2021 GMT
* expire date: Jan 9 00:29:45 2023 GMT
* subjectAltName: host "api.sandbox.push.apple.com" matched cert's "api.sandbox.push.apple.com"
* issuer: CN=Apple Public Server RSA CA 12 - G1; O=Apple Inc.; ST=California; C=US
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fbc8e80ca00)
> POST /3/device/7b97568a823edevicetokenherea4b072929b7e7c4f5 HTTP/2
> Host: api.sandbox.push.apple.com
> user-agent: curl/7.79.1
> accept: */*
> apns-topic: ca.xlogix.test.ios.push.example
> apns-priority: 10
> content-length: 146
> content-type: application/x-www-form-urlencoded
>
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS == 1000)!
< HTTP/2 400
< apns-id: BF2F2FE8-AC95-38D1-D96E-D542B819CDC0
<
* Connection #0 to host api.sandbox.push.apple.com left intact
{"reason":"DeviceTokenNotForTopic"}%

Azure Devops Server self hosted docker agent setup SSL errors

I'm trying to set up a proof of concept Azure Devops Server environment including a self hosted Docker agent on Ubuntu 20.04.5 LTS, following the documentation here.
When I attempt to start the container with this command it fails with an SSL error from curl:
myuser#ubuntu20-04dockertest:~$ sudo docker run -e AZP_URL=https://win-k58ocndvak6/DefaultCollection -e AZP_TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXXXXX -e AZP_AGENT_NAME=scs-docker-agent dockeragent:latest
[sudo] password for myuser:
1. Determining matching Azure Pipelines agent...
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
I have tried with a self signed certificate in the Azure Devops Server IIS bindings and by installing Active Directory Certificate Services and configuring the Azure Devops Server as a standalone root CA, issuing a certificate and applying it Azure Devops Server IIS bindings. Both the self signed certificate and the CA certificate were imported into the Ubuntu machines trusted certificate store. Neither of these configurations work but running curl against the Azure Devops Server URL seems fine:
myuser#ubuntu20-04dockertest:~$ curl -v https://win-k58ocndvak6/DefaultCollection
* Trying 192.168.122.204:443...
* TCP_NODELAY set
* Connected to win-k58ocndvak6 (192.168.122.204) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=AU; ST=WA; L=Perth; O=WIN-K58OCNDVAK6; OU=SCS-DEV; CN=WIN-K58OCNDVAK6
* start date: Sep 24 08:06:11 2020 GMT
* expire date: Sep 24 08:16:11 2021 GMT
* common name: WIN-K58OCNDVAK6 (matched)
* issuer: CN=WIN-K58OCNDVAK6-CA-1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55d60c17ddb0)
> GET /DefaultCollection HTTP/2
> Host: win-k58ocndvak6
> user-agent: curl/7.68.0
> accept: */*
Does anyone know what's going on here?
A way to get the "docker run" command to not care about SSL would also be fine, this is just a proof of concept Dev setup.
Thanks :)

How to debug for TLS/SSL connection

Ar first I made three files with these.
$ openssl genrsa 2048 > server.key
$ openssl req -new -key server.key > server.csr
$ openssl x509 -days 3650 -req -signkey server.key < server.csr > server.crt
Then,I made registry container by docker-compose which including server.key server.crt and port 5000 is open.
version: '3'
services:
registry:
container_name: registry
image: registry:2
restart: always
ports:
- '5000:5000'
volumes:
- /home/ubuntu/docker/data:/var/lib/registry
- /home/ubuntu/docker/certs:/certs
- /etc/localtime:/etc/localtime
environment:
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/server.crt
REGISTRY_HTTP_TLS_KEY: /certs/server.key
then in the localhost I rename server.crt to ca.crt and put the key /etc/docker/certs.d/docker.mysite.jp\:5000/ca.crt.
Then I try to curl but in vain.
$curl https://docker.mysite.jp:5000/v2/ --cacert /etc/docker/certs.d/docker.mysite.jp\:5000/ca.crt
/etc/docker/certs.d/docker.mysite.jp\:5000/ca.crt
curl: (60) SSL: unable to obtain common name from peer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
OK I see there is something wrong with tls/ssl
However how can I debug where to start??
$curl https://docker.mysite.jp:5000/v2/ --cacert /etc/docker/certs.d/docker.mysite.jp\:5000/ca.crt -vvv
here is the log
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x7f89b4800000)
* Connected to docker.mysite.jp (135.132.179.73) port 5000 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/docker/certs.d/docker.mysite.jp:5000/ca.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* start date: Mar 24 16:55:37 2020 GMT
* expire date: Feb 29 16:55:37 2120 GMT
* SSL: unable to obtain common name from peer certificate
* Closing connection 0
curl: (60) SSL: unable to obtain common name from peer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
I set the SQDN for crt file. then error message changed.
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x7fb4de806c00)
* Connected to docker.mysite.jp (135.132.179.73) port 5000 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/docker/certs.d/docker.mysite.jp:5000/ca.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
but ,Then I restart with docker-compose down & docker-compose up, it fixed!!!
execute curl with -vvv option to see all steps. Also, you can try
tcpdump
and
wireshark
to see every network action at including network level 4.

Resumable upload session uri request with Google Cloud Storage

I am trying to generate a resumable session uri using the Google Cloud Storage JSON API. Following the docs I have put together the following curl command to satisfy only the required arguments
curl \
-X POST \
-H "Content-Length: 1000000" \
-d "uploadType=resumable&name=cat.jpg" \
https://www.googleapis.com/upload/storage/v1/b/my-bucket/o
However, this times out and the server never responds. Note that Content-Type is only required if also sending the file metadata. I've tried also adding the metadata and associated data, but this also fails.
In the example request in the docs there is an Authorization: Bearer [YOUR_AUTH_TOKEN] header which isn't mentioned in the steps. I have also tried adding this, using the app's api key, but this also times out.
The ACL on the bucket has been set to all users writable. CORS is not configured.
Could anyone point out where I'm going wrong?
The curl verbose output is
* Hostname was NOT found in DNS cache
* Trying 216.58.208.138...
* Connected to www.googleapis.com (216.58.208.138) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-ECDSA-AES128-GCM-SHA256
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google Inc; CN=*.googleapis.com
* start date: 2017-10-24 08:38:00 GMT
* expire date: 2017-12-29 00:00:00 GMT
* subjectAltName: www.googleapis.com matched
* issuer: C=US; O=Google Inc; CN=Google Internet Authority G2
* SSL certificate verify ok.
> POST /upload/storage/v1/b/aits-resumables-test/o HTTP/1.1
> User-Agent: curl/7.35.0
> Host: www.googleapis.com
> Accept: */*
> Content-Length: 1000000
> Content-Type: application/json; charset=UTF-8
>
* upload completely sent off: 33 out of 33 bytes
So I have this working now. I added the object name in the JSON body and then added the uploadType=resumable to the url directly so that it looks like the following
curl \
-X POST \
-H "Content-Type: application/json; charset=UTF-8" \
-H "X-Upload-Content-Type: image/jpeg" \
-H "X-Upload-Content-Length: 2000000" \
-d '{"name": "cat.jpg"}' \
https://www.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=resumable

Resources