Docker private registry with mirror - docker

I created two Docker containers. The first one provides a private Docker registry and the second one is a mirror of the official Docker registry:
docker run -d --name registry -v /local/path/to/registry:/registry -e SETTINGS_FLAVOR=local -e STORAGE_PATH=/registry -p 5000:5000 registry
docker run -d --name mirror -v /local/path/to/mirror:/registry -e STORAGE_PATH=/registry -e STANDALONE=false -e MIRROR_SOURCE=https:/registry-1.docker.io -e MIRROR_SOURCE_INDEX=https://index.docker.io -p 5555:5000 registry
Now I would like to combine both. Whenever a user pulls images it should first query the private registry and then the mirror. And when images are pushed they should only be pushed to the private registry.
I do not have an idea about how this can be done. Any help is appreciated.

You cannot just force all docker push commands to push to your private registry. One reason is that you can have any number of those registers. You have to first tell docker where to push by tagging the image (see lower).
Here is how you can setup docker hosts to work with a running private registry and local mirror.
Client set-up
Lets assume that you are running both mirror and private registry on (resolvable) host called dockerstore. Mirror on port 5555, registry on 5000.
Then on client machine(s) you should pass extra options to docker daemon startup. In your case:
Add --registry-mirror=http://dockerstore:5555 to tell daemon to prefer using local mirror rather then dockerhub. source
Add --insecure-registry dockerstore:5000 to access the private registry without further configuration. See this answer
Restart docker daemon
Using the mirror
When you pull any image the first source will be the local mirror. You can confirm by running a docker pull, e.g.
docker pull debian
In the output there will be message that image is being pulled from your mirror - dockerstore:5000
Using local registry
In order to push to private registry first you have to tag the image to be pushed with full name of the registry. Make sure that you have a dot or colon in the first part of the tag, to tell docker that image should be pushed to private registry.
Docker looks for either a “.” (domain separator) or “:” (port separator) to learn that the first part of the repository name is a location and not a user name.
Example:
Tag 30d39e59ffe2 image as dockerstore:5000/myapp:stable
docker tag 30d39e59ffe2 dockerstore:5000/myapp:stable
Push it to private registry
docker push dockerstore:5000/myapp:stable
Then you can pull as well
docker pull dockerstore:5000/myapp:stable

If not present, create the file:
sudo nano /etc/docker/daemon.json
Then paste the following:
{
"registry-mirrors": [
"https://hub-mirror.c.163.com",
"https://mirror.baidubce.com"
]
}
Then retart Docker daemon
$ sudo systemctl restart docker
[Source]

Just to be clear, docker documentation confirms that:
It’s currently not possible to mirror another private registry. Only
the central Hub can be mirrored.

Repository names are intended to be global, that is the repository redis always refers to the official Redis image from the Docker Hub. If you want to use a private registry, you prefix the repository name with the name of the registry e.g. localhost.localdomain:5000/myimage:mytag.
So when you pull or push, it will automatically go to the relevant registry. The mirror should be easy to set up, you just pass the URL to the daemon with the --registry-mirror= argument.
This isn't perfect for enterprise users, hence this (closed) Docker issue.

Related

Pulling an image from local Docker registry

I installed a Docker registry to my server like below;
docker run -d -p 5000:5000 --name registry registry:2
So after that I pushed Alpine image to that registry.
docker pull alpine
docker image tag alpine localhost:5000/alpinetest
docker push localhost:5000/alpinetest
So the problem is I want to access this image from another server.
So I can run the command below from client to Docker registry's server;
user#clientserver ~
$ curl 10.10.2.18:5000/v2/_catalog
{"repositories":["alpinetest"]}
So how can I pull this "Alpinetest" image from another "clientserver"?
For example the command below is not working;
user#clientserver ~
$ docker pull 10.10.2.18:5000/alpinetest:latest
Using default tag: latest
Error response from daemon: Get "https://10.10.2.18:5000/v2/": http: server gave HTTP response to HTTPS client
Thanks!
On the machine that wants to pull the image, create or edit /etc/docker/daemon.json and enter this:
{
"insecure-registries": ["10.10.2.18:5000"]
}
and then run:
sudo systemctl restart docker
Just be aware that the registry is, just like it says, insecure. This setup shouldn't be used when the registry is accessed over the internet or in any other environment that you don't have full control over. But it's definitely nice for local tests.

How to access Docker Registry publicly from both sub network and outside world

I have just run a docker registry by:
$ docker run -d --name registry --restart always -p 5961:5000 registry:2.7.1
Now I can push to it by:
$ docker tag ubuntu:v2 localhost:5961/ubuntu:v2
$ docker push localhost:5961/ubuntu:v2
But not from outside. For example I can not push to it from another machine on the same network by executing:
$ docker tag ubuntu:v2 192.168.1.122:5961/ubuntu:v2
$ docker push 192.168.1.122:5961/ubuntu:v2
The error is:
The push refers to repository [192.168.1.122:5961/ubuntu]
Get https://192.168.1.122:5961/v2/: http: server gave HTTP response to HTTPS client
Why?
Also I don't know how to pull this image (192.168.1.122:5961/ubuntu:v2) from outside world. For example by:
$ docker pull <public-ip>:5961/ubuntu:v2
Note that I can port forward the port 5961 of the machine 192.168.1.122 to the same port of <public-ip>.
1 Regarding local network:
Your docker registry is insecure and is using HTTP, not HTTPS. So you need to define an insecure registry for the client daemon, by updating the /etc/docker/daemon.json file like so:
{
"insecure-registries" : ["192.168.1.122:5961"]
}
See: docs
2 Regarding pulling the image from the outside world:
It should work the way you described it docker pull <public-ip>:5961/ubuntu:v2 (as long as all clients defines the registry as insecure if it is)
But please DO NOT use an insecure registry open to the outside world, and unless you want everyone in the world to be able to pull your images, add some authentication mechanism in front of your registry service

How to verify locally built docker images?

In this question it turned out, that I cannot use the sha256 mechanism in the FROM line in a Dockerfile to verify I am using the correct locally built non-DockerHub image in another derived image.
Is there another way to verify locally built Docker images? Some best practice maybe?
From docs:
By default, docker pull pulls images from Docker Hub. It is also
possible to manually specify the path of a registry to pull from
You can start a private docker registry on you localhost with the following command:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Say your image name isubuntu Then push image to that specific registry with:
docker push localhost:5000/ubuntu
In your Dockerfile you can use:
From localhost:5000/ubuntu

Can I pull an image without Internet when the image exists in docker registry-mirror?

I create a registry mirror. Can I pull an image without Internet? I have created a mirror using this command:
docker run -d -p 5555:5000 -e STORAGE_PATH=/mirror -e STANDALONE=false -e MIRROR_SOURCE=https://registry-1.docker.io -e MIRROR_SOURCE_INDEX=https://index.docker.io -v /Users/v11/Documents/docker-mirror:/mirror --restart=always --name mirror registry
When I pull an image like hello-world:
docker pull image
I can find the image in local path what I set "/Users/v11/Documents/docker-mirror". Does it mean I succeed in creating mirror? But, I closed the Internet, and delete the hello-world:
docker rmi hello-world
and pull again, but it failed. I want to know whether I must use mirror with Internet? if not, what is wrong with me?
By the way, I have started docker daemon with this ENV:
docker --insecure-registry 192.168.59.103:5555 --registry-mirror=http://192.168.59.103:5555 -d &
and 192.168.59.103 is my boot2docker ip.
In your configuration you specified a non-standalone setup using a MIRROR_SOURCE_INDEX. So whenever you want to search for an image that index is queried. The other MIRROR_SOURCE gets queried for the image itself.
You might be able to retrieve an image offline with a pull request (that has already been pulled before while you were online). But you are not able to issue docker search commands when the index is not available.
If you want to be completely independent from the public docker registry then you would need to setup your own private registry.

Setting up a remote private Docker registry

I need some tips on setting up a 'remote private Docker registry'.
README.md on Docker-Registry mainly focus on private registry running on the same host, does not specify how other machines can access it remotely (or maybe too complex to understand).
So far I found these threads:
Docker: Issue with pulling from a private registry from another server
(Still an open thread, no solution offered. Further discussion on Github gives hint on proxy, but how does that work?)
Create a remote private registry
(Maybe closest to what I'm looking for, but what command do I need to access the registry from other machines?)
How to use your own registry (Again, this focuses on running registry on the same host. It did mention running on port 443 or 80 for other machines to access, but need more detail!)
Running out of clues, any input very appreciated!
I was able to set up a remote private registry by referring to this:
Remote access to a private docker-registry
Steps:
On registry host, run docker run -p 5000:5000 registry
On client host, start Docker service by docker -d --insecure-registry 10.11.12.0:5000 (replace 10.11.12.0 with your own registry ip, and you might want to daemonize the process so it'll continue running after shell closes.)
Edit: Alternatively, you can edit Docker's init script (/etc/sysconfig/docker for RHEL/CentOS, /var/lib/docker for Ubuntu/Debian). Add this line other_args="--insecure-registry 10.11.12.0:5000", then do a service docker restart. This is a recommended method as it daemonizes the Docker process.
Now, try if it works:
In client, download a busybox image docker pull busybox
Give it a new tag docker tag busybox 10.11.12.0:5000/busybox
Push it to registry docker push 10.11.12.0:5000/busybox
Verify the push docker search 10.11.12.0:5000/busybox
Remove all images and pull it from your registry docker rmi busybox 10.11.12.0:5000:busybox docker pull 10.11.12.0:5000:busybox
Run docker images should have the image you just pulled from your own remote private registry.
I use private registry in the next way:
It has FQDN: docker.mycompany.com
All images which I create have name: docker.mycompany.com/image1, docker.mycompany.com/image2, etc
After that all is working seamlessly:
Push image to registry:
docker push docker.mycompany.com/image1
Pull and run image:
docker run docker.mycompany.com/image2

Resources