Docker registry mirror not used - docker

Question

When I try to pull an image from my local mirror, it works :
$ docker login -u docker -p mypassword nexus3.pleiade.mycomp.fr:5000
$ docker pull nexus3.pleiade.mycomp.fr:5000/hello-world
Using default tag: latest
latest: Pulling from **hello-world**
78445dd45222: Pull complete
Digest: sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7
Status: Downloaded newer image for **nexus3.pleiade.mycomp.fr:5000/hello-world:latest**
But then, when I want to use this registry as mirror, it is just ignored, images are always pulled from web Docker hub, not from my local mirror :
$ ps -ef | grep docker
/usr/bin/dockerd -H fd:// --storage-driver=overlay2 --registry-mirror=https://nexus3.pleiade.mycomp.fr:5000
$ docker info
Registry Mirrors:
https://nexus3.pleiade.mycomp.fr:5000/
$ docker rmi nexus3.pleiade.mycomp.fr:5000/hello-world
_
$ docker pull hello-world
Using default tag: latest
latest: Pulling from **library/hello-world**
78445dd45222: Pull complete
Digest: sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7
Status: Downloaded newer image for **hello-world:latest**
I know for sure it doesn't use my mirror, because when I unset the proxy settings, it cannot reach hello-world image.
Is it a Docker bug, or am I missing something ?
Docker info (short) :
Server Version: 1.13.1
Storage Driver: overlay2
(...)
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.8.0-37-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 15.67 GiB
(...)
Registry Mirrors:
https://nexus3.pleiade.edf.fr:5000/
UPDATE :
Doing "journalctl -xe", I can see some useful information :
level=error msg="Attempting next endpoint for pull after error: Get https://nexus3.pleiade.mycomp.fr:5000/v2/library/hello-world/manifests/latest: no basic auth credentials"
It looks related to : https://github.com/docker/docker/issues/20097, but the workaround is not working : when I replace --registry-mirror=https://nexus3.pleiade.mycomp.fr:5000 by --registry-mirror=https://docker:password#nexus3.pleiade.mycomp.fr:5000
I get exactly the same error.
If it matters, the nexus is using a self signed certificate which has been copied to /etc/docker/certs.d/nexus3.pleiade.mycomp.fr:5000/ca.crt and this allowed to login via "docker login".

Answer 1

It's a docker bug : https://github.com/docker/docker/issues/30880
The workaround is to set up a https reverse proxy setting a hard-coded authentication header.
Here is an example config from Felipe C. :
In nginx docker config, add :
proxy_set_header Authorization "Basic a2luZzppc25ha2Vk";
Full example:
server {
listen *:443 ssl http2;
server_name docker.domain.blah.net;
ssl on;
include ssl/domain.blah.net.conf;
# allow large uploads of files - refer to nginx documentation
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
proxy_pass http://127.0.0.1:8083/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization "Basic YWRtaW46YWRtaW4xMjM=";
#proxy_set_header X-Forwarded-Proto "https";
}
}
server {
listen *:80;
server_name docker.domain.blah.net;
return 301 https://$server_name$request_uri;
}

Answer 2

Another way is docker logout other servers.
And enable the registry config Allow anonymous docker pull ( Docker Bearer Token Realm required ).

Answer 3

It worked for me to add a /etc/docker/daemon.json:
{
"registry-mirrors": [ "nexus3.pleiade.mycomp.fr" ],
"max-concurrent-downloads": 20
}

Answer 4

I may be late to the party but i hope this helps someone. I was facing the same issue and getting the auth error in nexus logs.
It turns out I had to enable anonymous docker pull in my nexus repository settings
Also after doing so check under Security->Realms that Docker Bearer Token Realm is active and given high priority

Answer 5

You can add basic auth in URL and it works for me. Something like
https://username:password#nexus3.pleiade.mycomp.fr:5000