Insecure Docker registry and self-signed certificates - docker

Question

I have already setup a localized docker registry with self-signed certificates based on this reference: https://docs.docker.com/registry/insecure/#deploy-a-plain-http-registry.
I pussed an example image using:
sudo docker run hello-world
sudo docker tag hello-world registry.local.doc:5045/hello-world
sudo docker push registry.local.doc:5045/hello-world
I am able to see the "contents" of the registry on https://registry.local.doc:5045/v2/_catalog:
{"repositories":["hello-world"]}
1st Problem
I have used self-signed certificates using:
openssl req -newkey rsa:4096 -nodes -sha256 -keyout ./certs/tls.key -x509 -days 365 -subj "/C=GR/ST=./L=./O=./CN=registry.local.doc" -addext "subjectAltName = DNS:registry.local.doc" -out ./certs/tls.crt
However, in order to make it work I have to add in /etc/docker/daemon.json the following:
{ "insecure-registries" : ["registry.local.doc:5000"] }
which is confusing as I am using self-signed certificates as already mentioned.
2nd Problem
I have a k3s node and I want to pull images from inside a Pod.
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry-test
labels:
app: registry-test
spec:
replicas: 1
selector:
matchLabels:
app: registry-test
template:
metadata:
labels:
app: registry-test
spec:
containers:
- name: registry-test
image: registry.local.doc:5045/hello-world
It works but first I need to run:
sudo update-ca-certificates
sudo systemctl restart containerd
sudo systemctl restart k3s
I understand the reasons behind the execution of the first two commnads sudo update-ca-certificates & sudo systemctl restart containerd, however I have no clue why I have to run sudo systemctl restart k3s to make it works.
Is there a solution without restarting the cluster, as something like this is forbidden when we talk about a cluster in production?
And if I had a k8s cluster, what exactly would Ι have to restart?

Answer 1

1st Problem
registry is a server side, your docker is client side, the config insecure-registries tell your docker to skip server cert validation.
Without this settings, docker will not pull image because the cert is invalid.
2st Problem
you need to restart k3s only because you using insecure-registries config. docker need restart to reload this config.
When you using self-sign cert registry, you have two way to pull image from there.
Add { "insecure-registries" : ["registry.local.doc:5000"] } to /etc/docker/daemon.json and restart k3s
sudo systemctl restart containerd
sudo systemctl restart k3s
In production, normally you should have more then one worker and follow below step:
drain the worker, see doc here. This will move your pod to other worker.
add insecure-registries and restart k3s.
uncordon the worker, pod can move back to this worker.
repeat above step on every worker.
Add your self-sign cert to trust store on every woker. This way don't require restart k3s.
For ubuntu see below.
sudo cp tls.crt /usr/local/share/ca-certificates
sudo update-ca-certificates
add cert to docker certs folder
Instruct every Docker daemon to trust that certificate. The way to do this depends on your OS.
Linux: Copy the domain.crt file to /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt on every Docker host. You do not need to restart Docker.
sudo cp tls.crt /etc/docker/certs.d/registry.local.doc:5000/ca.crt