How to use a secure docker registry within a kind kubernetes cluster - docker

I am trying to create a secure docker registry to be used inside a development kind cluster. I am going to use a container for the registry and 3 other containers for kind workers. In order to be consistent with the production environment I want to use TLS, so I created a self signed certificate for the docker registry. I connected the containers using docker network. However, when I create a deployment based on an image from that registry, I get x509 certificate signed by unknown authority error.
I used this tutorial
containerdConfigPatches: # Enable a local image registry, placeholders automatically replaced in bootstrap script -- https://kind.sigs.k8s.io/docs/user/local-registry/
- |-
[plugins."io.containerd.grpc.v1.cri".registry.configs.my-registry.tls]
cert_file = "/etc/docker/certs.d/my-registry/domain.crt"
key_file = "/etc/docker/certs.d/my-registry/domain.key"
But it does not seem to work.
My kind version:
kind v0.17.0 go1.20 linux/amd64
The command I use to create the registry:
docker run -d \
--restart=always \
--name my-registry \
-v `pwd`/auth:/auth \
-v `pwd`/certs:/certs \
-v `pwd`/certs:/certs \
-e REGISTRY_AUTH=htpasswd \
-e REGISTRY_AUTH_HTPASSWD_REALM="Registry Realm" \
-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
-e REGISTRY_HTTP_ADDR=0.0.0.0:80 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
-p 7443:80 \
registry:2

You are using a self-signed certificate for your docker registry instead of a certificate issued by a trusted certificate authority (CA). The docker daemon does not trust the self-signed certificate, which is causing the x509 error.
This may occur due to the expiration of the current certificate, due to a changed hostname, and other changes.
Verify that the $HOME/.kube/config file contains a valid certificate, and regenerate a certificate if necessary. The certificates in a kubeconfig file are base64 encoded. The base64 --decode command can be used to decode the certificate and openssl x509 -text -noout can be used for viewing the certificate information.
Unset the KUBECONFIG environment variable using:
unset KUBECONFIG
Or set it to the default KUBECONFIG location:
export KUBECONFIG=/etc/kubernetes/admin.conf
Another workaround is to overwrite the existing kubeconfig for the "admin" user:
mv $HOME/.kube $HOME/.kube.bak
mkdir $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
For more information refer to the documentation

Related

How to run Confluent Schema Registry using Docker on AWS ec2

I want to run schema registry for my AWS MSK cluster on EC2 within the same VPC as my MSK cluster using confluentinc/cp-schema-registry.
But the container is exiting without any proper error message.
Here is my docker command:
docker run \
--net=host \
--name=schema-registry \
-e SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=<PLAINTEXT-ZOOKEEPER-CONNECTION-URL> \
-e SCHEMA_REGISTRY_HOST_NAME=localhost \
-e SCHEMA_REGISTRY_LISTENERS=http://localhost:8081 \
-p 8081:8081 \
confluentinc/cp-schema-registry
===== UPDATE ======
I have also tried by running confluent schema-registry as follows:
bin/schema-registry-start etc/schema-registry/schema-registry.properties
But getting the error:
java.lang.RuntimeException: Error initializing the ssl context for RestService
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
I have generated the signed certificate, added to keystore by following:
https://docs.aws.amazon.com/msk/latest/developerguide/msk-authentication.html
This keystore is working fine with console-producer and consumers but not working with schema-registry.
and here is my content of schema-registry.properties
listeners=http://0.0.0.0:8081
kafkastore.bootstrap.servers=<MY-MSK-BOOTSTRAP-SERVER>
kafkastore.topic=_schemas
debug=true
security.protocol=SSL
ssl.truststore.location=/tmp/kafka/kafka.client.truststore.jks
ssl.keystore.location=/tmp/kafka/kafka.client.keystore.jks
ssl.keystore.password=xxxx
ssl.key.password=xxxx

Grafana with https - Cannot find SSL cert_file

I'm running Grafana in a Docker container on my NAS. Everything is fine when using http.
However I fail to start the container when I setup Grafana for https, as the Certificate file can't be found according to the Docker log.
I create a self-certificate using OpenSSL in order to use Grafana with https.
I modified the docker script to overwrite the enviroment Server section for https and defined the path for the cert and key file.
INFO[12-08|12:28:50] Config overridden from Environment variable logger=settings var="GF_SERVER_PROTOCOL=https"
INFO[12-08|12:28:50] Config overridden from Environment variable logger=settings var="GF_SERVER_CERT_FILE=/share/CACHEDEV2_DATA/Container/grafana/config/ssl/grafana.crt"
INFO[12-08|12:28:50] Config overridden from Environment variable logger=settings var="GF_SERVER_CERT_KEY=/share/CACHEDEV2_DATA/Container/grafana/config/ssl/grafana.key"
As far as I can see, this seems to be fine, however for unknown reason the cert-file isn't found, even it is available in the defined path.
INFO[12-08|12:28:50] HTTP Server Listen logger=http.server address=0.0.0.0:3000 protocol=https subUrl= socket=
EROR[12-08|12:28:50] Stopped HTTPServer logger=server reason="Cannot find SSL cert_file at /share/CACHEDEV2_DATA/Container/grafana/config/ssl/grafana.crt"
When I check the path I see it is valid
[/share/CACHEDEV2_DATA/Container/grafana] # ls -l /share/CACHEDEV2_DATA/Container/grafana/config/ssl/grafana.crt
-rw-r--r-- 1 admin administrators 1228 2019-12-08 10:55 /share/CACHEDEV2_DATA/Container/grafana/config/ssl/grafana.crt
Any idea what could be the reason for this?
Could the Certificate be invalid and the error message is just misleading?
Many thanks for a hint :)
Stefan
Edit:
The script I use to start the Docker Container:
GRAFANA_DIR_CONF=$(readlink -f ./config)
GRAFANA_VER='latest'
docker run -it \
--name=grafana \
-v $GRAFANA_DIR_CONF:/var/lib/grafana \
-v /etc/localtime:/etc/localtime:ro \
-e "GF_SECURITY_ALLOW_EMBEDDING=true" \
-e "GF_USERS_ALLOW_SIGN_UP=false" \
-e "GF_AUTH_ANONYMOUS_ENABLED=true" \
-e "GF_AUTH_BASIC_ENABLED=false" \
-e "GF_SERVER_PROTOCOL=https" \
-e "GF_SERVER_CERT_FILE=$GRAFANA_DIR_CONF/ssl/grafana.crt" \
-e "GF_SERVER_CERT_KEY=$GRAFANA_DIR_CONF/ssl/grafana.key" \
-d \
--restart=always \
-p 3000:3000 \
grafana/grafana:$GRAFANA_VER
[/share/CACHEDEV2_DATA/Container/grafana/config/ssl] # ls -l
total 16
-rw-r--r-- 1 admin administrators 1228 2019-12-08 10:55 grafana.crt
-rw-r--r-- 1 admin administrators 1702 2019-12-08 10:44 grafana.key
[/share/CACHEDEV2_DATA/Container/grafana/config/ssl] #
You are using volume for the configuration folder, so correct path to the cert/key in the container is:
-e "GF_SERVER_CERT_FILE=/var/lib/grafana/ssl/grafana.crt" \
-e "GF_SERVER_CERT_KEY=/var/lib/grafana/ssl/grafana.key" \

Secure private docker-registry

How to you setup a private secure docker-registry.
I have installed via helm
Now how can I make it secure(TLS certs), so I can push and pull to the registry from docker and from kubernetes deployment?
I can see that there is a Helm configuration:
tlsSecretName Name of secret for TLS certs
Update - current status:
I was able to get cert-manager working and install with TLS:
helm install stable/docker-registry --set tlsSecretName=example-com-tls
I am not strong in certificates - but I am unclear about the following:
1.
Can I now create an Ingress(with a secret to cert) that will only accept incomming request with that certificate? I will look at the suggested link from #xzesstence tomorrow
2.
I guess I need to tell docker push where to find the certificate?
Maybe this(I will try this tomorrow): https://docs.docker.com/engine/security/certificates/
Check out the official Docker Tutorials
https://docs.docker.com/registry/deploying/
and especially the point
Get a certificate
So overall in short, you need to get a certificate and place it in /certs (or change the folder mount of the following docker run command -v /cert).
Also check the certificate name, either rename to domain.crt or change the filename in the docker run command
then run
docker run -d \
--restart=always \
--name registry \
-v "$(pwd)"/certs:/certs \
-e REGISTRY_HTTP_ADDR=0.0.0.0:443 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
-p 443:443 \
registry:2
If you don't have a certificate, you can use letsencrypt
https://letsencrypt.org/
Maybe you want to checkout this startscript with letsencrypt certs. (untested from my side)
The advantage of this is, that you have the letsencrypt service integrated which can renew the license automatically
https://gist.github.com/PieterScheffers/63e4c2fd5553af8a35101b5e868a811e
Edit:
Since you are using Docker on a Kubernetes Cluster checkout this great tutorial
https://medium.com/#jmarhee/in-cluster-docker-registry-with-tls-on-kubernetes-758eecfe8254

docker Run an externally-accessible registry with self signed certificate

Can i setup docker Run an externally-accessible registry with self signed certificate or i required CA certificate only after configuration of nginx with ssl self singed certificate i run below command and its given me the error so can somebody help me int that
i) # cd /etc/nginx
ii) # docker run -d \
--restart=always \
--name sogetiaws \
-v pwd/ssl:/ssl \
-e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/ssl/docker-reg.sogeti-aws.nl.crt \
-e REGISTRY_HTTP_TLS_KEY=/ssl/docker-reg.sogeti-aws.nl.key \
-p 5000:5000 \
registry:2
Domain Name : docker-reg.sogeti-aws.nl
ERROR
iii) # docker push docker-reg.sogeti-aws.nl/my-ubuntu
The push refers to a repository [docker-reg.sogeti-aws.nl/my-ubuntu]
Get https://docker-reg.sogeti-aws.nl/v1/_ping: x509: certificate signed by unknown authority
Add
--insecure-registry docker-reg.sogeti-aws.nl:5000
To your local daemon (the one you use to push the image)
Or
add
{
"insecure-registries" : [ "docker-reg.sogeti-aws.nl:5000" ]
}
to your /etc/docker/daemon.json config file.
Source : Add Insecure Registry to Docker

docker private registry user creation

I have created my private docker registry running on localhost:5000/v1 but it does not provide authentication, How to have username and password so that only authorized users can push an image to it.
I am also not able to list all the images present in private registry, all document says running below command will list it localhost:5000/v1/search but it gives a blank json response as:
{
"num_results": 0,
"query": "",
"results": []
}
How to resolve this?
Thanks,
Yash
An answer to your first question: You need to use something like nginx in front of the registry to do the actual password authentication. There are example nginx configuration files for pre-1.3.9 nginx and later versions in the Docker Registry Github repo for wrapping the registry with nginx; there is more information on authentication configuration on the nginx wiki.
You can use htpasswd to setup a login with dockers registry image. However, I don't believe they have implemented a search function in this image yet. To create a user, I have the following script:
#!/bin/sh
usage() { echo "$0 user"; exit 1; }
if [ $# -ne 1 ]; then
usage
fi
user=$1
cd `dirname $0`
if [ ! -d "auth" ]; then
mkdir -p auth
fi
chmod 666 auth/htpasswd
docker run --rm -it \
-v `pwd`/auth:/auth \
--entrypoint htpasswd registry:2 -B /auth/htpasswd $user
chmod 444 auth/htpasswd
Then to run the registry, I use the following script (from the same folder):
#!/bin/sh
cd `dirname $0`
docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/certs:/certs:ro \
-v `pwd`/auth/htpasswd:/auth/htpasswd:ro \
-v `pwd`/registry:/var/lib/registry \
-e "REGISTRY_HTTP_TLS_CERTIFICATE=/certs/host-cert.pem" \
-e "REGISTRY_HTTP_TLS_KEY=/certs/host-key.pem" \
-e "REGISTRY_AUTH=htpasswd" \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e "REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd" \
-e "REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry" \
registry:2
Note that I'm also using TLS certificates in the above under the certs directory. You can create these with openssl commands (same ones used for securing the docker daemon socket).

Resources