After installing k0s I generated and adjusted a containerd config as described here https://docs.k0sproject.io/v0.9.0/containerd_config/.
I added the following lines to not check TLS for my custom registry:
/etc/k0s/containerd.toml
[plugin."io.containerd.grpc.v1.cri".registry.configs."registry.myproject.localdev".tls]
insecure_skip_verify = true
However when I try to pull an image I still get a TLS error:
sudo k0s ctr images pull registry.myproject.localdev/vue-project:1.0.0
Does anyone know how to allow for insecure registries with containerd running under k0s?
Related
On Ubuntu 18, I installed Docker (19.03.12) from these instructions
https://docs.docker.com/engine/install/ubuntu/
And then went through these steps
manage docker as non-root user
https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user
start on boot using systemd
https://docs.docker.com/engine/install/linux-postinstall/#configure-docker-to-start-on-boot
and set up a private docker registry using this
docker run -d -p 5000:5000 -e REGISTRY_DELETE_ENABLED=true --restart=always --name registry registry:2
I also added this to the daemon.json file
{ "insecure-registries" : ["my.registrydomain.lan:5000"] }
And restarted the docker daemon
sudo /etc/init.d/docker restart
I checked docker info to make sure the setting for insecure registry was applied and I saw this at the end so it seems ok
Insecure Registries:
my.registrydomain.lan:5000
127.0.0.0/8
On the same machine I start minikube (1.12.3) with this command
minikube start --driver=docker --memory=3000 --insecure-registry=my.registrydomain.lan:5000
So everything is running and fine, and I proceed to apply my deployments using kubectl except when I get to the pod that needs to pull the container form the local registry I get an ErrImagePull status. Here is part of my deployment
spec:
containers:
- name: my-container
image: my.registrydomain.lan:5000/name:1.0.0.9
imagePullPolicy: IfNotPresent
When I describe the pod that failed using
kubectl describe pod mypod-8474577f6f-bpmp2
I see this message
Failed to pull image "my.registrydomain.lan:5000/name:1.0.0.9": rpc
error: code = Unknown desc = Error response from daemon: Get
https://my.registrydomain.lan:5000/v2/: http: server gave HTTP
response to HTTPS client
EDIT: I forgot to mention that I am able to PUSH my images into the registry without any issues from a separate machine over http (machine is Windows 10 and I set the insecure registry option in the daemon config)
I tried to reproduce your issue with exact same settings that you provided and this works just fine. Image is being pulled without any problem. I tested this with my debian 9 and fresh ubuntu installation with this settings:
minikube version: v1.12.3
docker version: v19.03.12
k8s version: v1.18.3
ubuntu version: v18
What I`ve done what is not described in the question is to place an entry in minikube container hosts file:
root#minikube:/# cat /etc/hosts
...
10.128.5.6 my.registrydomain.lan
...
And the tag/push commands:
docker tag 4e2eef94cd6b my.registrydomain.lan:5000/name:1.0.0.9
docker push my.registrydomain.lan:5000/name:1.0.0.9
Here`s the describe from the pod:
Normal Pulling 8m19s (x5 over 10m) kubelet, minikube Pulling image "my.registrydomain.lan:5000/name:1.0.0.9"
As suggested in the comments already you may want to check this github case. It goes thru couple of solution of your problem:
First is to check your hosts file and update it correctly if you hosting your repository on another node. Second solution is related to pushing images in to repository which turned for the user that both insecure-registries and docker push command are case sensitive. Third one is to use systemd to control docker daemon.
Lastly If those would not help I would try to clear all settings, uninstall docker, clear docker configuration and start again from scratch.
There is no requirement for secure registry. I just need to connect to a registry using http protocol.
The registry must be on a pod and not directly on the VM.
docker has registry image that is made just for this purpose but when I'm using it inside a pod, docker fails to communicate with it because it thinks its a secure registry:
> docker pull 192.168.64.3:31549/repo630444582240256/image1
Using default tag: latest
Error response from daemon: Get https://192.168.64.3:31549/v2/: http: server gave HTTP response to HTTPS client
I came across these solutions but each of them requires installing prerequisits in the VM or doesn't use a pod to setup a registry:
https://github.com/SeldonIO/k8s-local-docker-registry
https://github.com/alexellis/k8s-tls-registry
https://github.com/ContainerSolutions/trow
> set -x && curl -X GET 192.168.64.3:31549/v2/_catalog
+ curl -X GET 192.168.64.3:31549/v2/_catalog
{"repositories":[]}
I tried this on my local machine: https://github.com/SeldonIO/k8s-local-docker-registry and works like a charm. (I had to make a few changes to the K8s manifests so they support the latest K8s)
You can get to the registry using curl -X GET 192.168.64.3:31549/v2/_catalog which means there is no redirect to https.
I believe your docker client config doesn't have explicit Insecure Registry config for 192.168.x.x. You can check with:
$ docker info | grep -i -A5 'Insecure Registries'
Insecure Registries:
10.96.0.0/12
127.0.0.0/8
192.168.64.0/24 <== should have something like this
If not you can configure your 192.168.0.0/24 as an insecure registry in the daemon.json config:
{
"insecure-registries" : ["10.96.0.0/12", "127.0.0.0/8", "192.168.64.0/24" ]
}
just came across another path where you can add the insecure registries
$ vi /var/lib/boot2docker/profile
add the following
EXTRA_ARGS='
--label provider=virtualbox
--insecure-registry 127.0.0.0/8
--insecure-registry 192.168.99.0/24
'
restart docker daemon
$ /etc/init.d/docker restart
I have latest Docker version 18.06.0 installed on CentOS 7.
My server is in a corporate network so using a proxy server to access the registry.
I have added proxy settings as per docker documentation.
But finally adding proxy settings in worked partially. i.e. now docker is using proxy to pull the image before it runs.
But now it fails giving following error:
$ sudo docker run hello-world
Unable to find image ‘hello-world:latest’ locally
docker: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: net/http: TLS handshake timeout.
Also, the login fails:
$ sudo docker login --username=XXXX
Password:
Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: EOF
My proxy is pretty fast. When I’m using it to download anything from the internet using wget or curl I see 90 to 100Mbps speed.
Other applications utils on my machine such as yum are using this proxy perfectly and it works well.
Now I don’t know why only Docker has a problem in downloading the images.
The details of installation and configuration are as follow:
Version:
$ sudo docker version
Client:
Version: 18.06.0-ce
API version: 1.38
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:08:18 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.0-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:10:42 2018
OS/Arch: linux/amd64
Experimental: false
OS Version:
Description: CentOS Linux release 7.2.1511 (Core)
Release: 7.2.1511
Docker proxy config:
$ cat /etc/systemd/system/docker.service.d/https-proxy.conf
[Service]
Environment=“HTTP_PROXY=http://pqr.corp.xxx.com:8080”
Environment=“HTTPS_PROXY=https://pqr.corp.xxx.com:8080”
Environment=“NO_PROXY=localhost,127.0.0.1”
after adding this configuration I have already done
sudo systemctl daemon-reload
sudo systemctl restart docker
Looking for specific help.
I have gone through most of the posts on TLS Handshake Timeout issues but didn’t get any answer or suggestions working for me.
My proxy is fine and docker is using it. I don’t understand why it gets that nasty timeout.
The config file is:
[Service]
Environment=“HTTP_PROXY=http://pqr.corp.xxx.com:8080”
Environment=“HTTPS_PROXY=https://pqr.corp.xxx.com:8080”
Environment=“NO_PROXY=localhost,127.0.0.1”
Notice that 3rd line: HTTPS_PROXY=https
Is the proxy server support HTTPS? Or there should be an error.
Maybe HTTPS_PROXY=http is correct.
By the way, a similar error is Get https://registry-1.docker.io/v2/: proxyconnect tcp: EOF
Reason is usually related with your network settings. Especially, if your Docker is set up with Proxy, make it temporary with "No Proxy" option and it should work.
Cheers
I was also facing same issue behind firewall. Follow below steps:
$ sudo vim /etc/systemd/system/docker.service.d/http_proxy.conf
[Service]
Environment=“HTTP_PROXY=http://username:password#IP:port/”
Don’t use or remove https_prxoy.conf file.
reload and restart your docker
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:2557*********************************8
Status: Downloaded newer image for hello-world:latest
The answers before mine point towards this direction, but neither states it clearly: Removing all https proxy settings solves this problem. I had a https-proxy.conf file just like OP's and docker pull started to work after I deleted the HTTPS_PROXY line. I know that the https proxy I use works all right so it must be a problem on Docker's side.
I found this solution on serverfault.
I receive this error when I try to push and sign image in a private registry. Steps to reproduce:
Deploy registry:
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
Tag the image:
$ docker tag hello-world localhost:5000/hello-world:latest
Enable content trust:
$ export DOCKER_CONTENT_TRUST=1
Push on registry:
$ docker push localhost:5000/hello-world:latest
=> The push refers to a repository [localhost:5000/hello-world]
a02596fdd012: Pushed
latest: digest: sha256:a18ed77532f6d6781500db650194e0f9396ba5f05f8b50d4046b294ae5f83aa4 size: 524
Signing and pushing trust metadata
tls: oversized record received with length 20527
Your registry is not accepting https connections since it is on http. forget about the DOCKER_CONTENT_TRUST and add your registry as an insecure registry using --insecure-registry daemon option in your docker daemon config and then restart docker afterwards
I resolved this only when I set up the content trust server to point to running notary server.
export DOCKER_CONTENT_TRUST_SERVER=https://localhost:4443
In case you're using a proxy to pull down the images, take a look at proxy configuration. In particular, make sure https proxy URL does not contain "https" string, for example:
Environment="HTTPS_PROXY=https://proxy.url:8080/" => Environment="HTTPS_PROXY=http://proxy.url:8080/"
Also see here for a more complete explanation
I have setup the private registry using docker run -t -i -p 5000:5000 registry and it is accessible from 192.168.59.103:5000 (I am using boot2docker). Now I have already pulled an image tutum/tomcat and when I try to push the image (docker push 192.168.59.103:5000/tomcat) into the registry after tagging it as 192.168.59.103:5000/tomcat I face the following error as shown below --
FATA[0004] Error: v1 ping attempt failed with error: Get https://192.168.59.103:5000/v1/_ping: EOF. If this private regi
stry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry 192.168.59.103:5000` to
the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag
; simply place the CA certificate at /etc/docker/certs.d/192.168.59.103:5000/ca.crt
FROM OTHER POSTS --
I have read some some posts about this where they state that adding DOCKER_OPTS would solve the issue and that it should be added into /etc/default/docker but In my system /etc/default doesn't contain a docker folder as shown from the command sudo ls /etc/default/ which provides the result as a sungle folder aufs.
Another post states that this can be thrawted by simply placing the certificate att /etc/docker/certs.d/192.168.59.103:5000/ca.crt but I am not able to move into the docker directory even when I use sudo as seen below -- for cd /etc/docker
-sh: cd: can't cd to /etc/docker
I have repeated the same process using the registry ip as 0.0.0.0:5000 inside docker (with the assumption the that $boot2docker ip should be used in the hosts machine and not within docker) -- But that still didn't fix the issue that I am facing.
Please do help me out to sort out this problem. I have added the system specification below --
System Specification:
Microsoft Windows 7 via. boot2docker (CLI version 1.5)
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef
To use the --insecure-registry option, add it to the file /var/lib/boot2docker/profile inside the boot2docker VM. You can get into the VM with boot2docker ssh. The file contents should look like:
EXTRA_ARGS="--insecure-registry REGISTRY_IP:PORT"
You will then need to restart boot2docker (e.g. boot2docker restart).
I have no idea why you can't edit /etc/docker. The following works for me:
docker#boot2docker:~$ sudo ls /etc/docker
key.json