I create a registry mirror. Can I pull an image without Internet? I have created a mirror using this command:
docker run -d -p 5555:5000 -e STORAGE_PATH=/mirror -e STANDALONE=false -e MIRROR_SOURCE=https://registry-1.docker.io -e MIRROR_SOURCE_INDEX=https://index.docker.io -v /Users/v11/Documents/docker-mirror:/mirror --restart=always --name mirror registry
When I pull an image like hello-world:
docker pull image
I can find the image in local path what I set "/Users/v11/Documents/docker-mirror". Does it mean I succeed in creating mirror? But, I closed the Internet, and delete the hello-world:
docker rmi hello-world
and pull again, but it failed. I want to know whether I must use mirror with Internet? if not, what is wrong with me?
By the way, I have started docker daemon with this ENV:
docker --insecure-registry 192.168.59.103:5555 --registry-mirror=http://192.168.59.103:5555 -d &
and 192.168.59.103 is my boot2docker ip.
In your configuration you specified a non-standalone setup using a MIRROR_SOURCE_INDEX. So whenever you want to search for an image that index is queried. The other MIRROR_SOURCE gets queried for the image itself.
You might be able to retrieve an image offline with a pull request (that has already been pulled before while you were online). But you are not able to issue docker search commands when the index is not available.
If you want to be completely independent from the public docker registry then you would need to setup your own private registry.
Related
I installed a Docker registry to my server like below;
docker run -d -p 5000:5000 --name registry registry:2
So after that I pushed Alpine image to that registry.
docker pull alpine
docker image tag alpine localhost:5000/alpinetest
docker push localhost:5000/alpinetest
So the problem is I want to access this image from another server.
So I can run the command below from client to Docker registry's server;
user#clientserver ~
$ curl 10.10.2.18:5000/v2/_catalog
{"repositories":["alpinetest"]}
So how can I pull this "Alpinetest" image from another "clientserver"?
For example the command below is not working;
user#clientserver ~
$ docker pull 10.10.2.18:5000/alpinetest:latest
Using default tag: latest
Error response from daemon: Get "https://10.10.2.18:5000/v2/": http: server gave HTTP response to HTTPS client
Thanks!
On the machine that wants to pull the image, create or edit /etc/docker/daemon.json and enter this:
{
"insecure-registries": ["10.10.2.18:5000"]
}
and then run:
sudo systemctl restart docker
Just be aware that the registry is, just like it says, insecure. This setup shouldn't be used when the registry is accessed over the internet or in any other environment that you don't have full control over. But it's definitely nice for local tests.
How to create a local-registry container that mounts a volume from the host machine and persist locally all the images that get pulled?
Local Docker registry with persisted images
It should be possible to have an ephemeral registry container (and its docker volume), allowing to not download images more than once, even after the registry (or the whole Docker VM) is being throw away and recreated.
This would allow to pull just once the images, having them available when internet connectivity isn't good (or available at all); would allow also to mount a docker volume with pre-downloaded images.
It would be more convenient than having to manually docker push/docker pull onto the local registry, or to docker save/docker load each image that need to be available there.
Notes:
destination of the mount should probably be /var/lib/registry/docker/registry.
it is possible to configure a local Docker registry as a pull-through cache.
my specific setup runs docker via minikube, on macOS; but the answer doesn't have to be specific to it.
I managed it, here are the step-by-step instructions. Hopefully will make life easier to somebody else!
Configuration
Define first your environment variables with the desired values. See env-vars in the code below (PROXIED_REGISTRY, REGISTRY_USERNAME, REGISTRY_PASSWORD, PATH_WHERE_TO_PERSIST_IMAGES, etc.)
On the host machine
Minikube
If using minikube, first bind to docker on its VM's
eval $(minikube docker-env)
or run the commands directly from inside the VM, via minikube ssh.
Create local registry
(note: some envs might be unnecessary; check Docker docs to see what you need)
The -v option mounts onto the local registry the path where you want to persist the registry data (repositories folders and image layers).
When you use Minikube, this latter will automatically mount the home folder from the host (/Users/, on macOS) onto the virtual machine where Docker is run.
docker run -d -p 5000:5000 \
-e STANDALONE=false \
-e "REGISTRY_LOG_LEVEL=debug" \
-e "REGISTRY_REDIRECT_DISABLE=true" \
-e MIRROR_SOURCE="https://${PROXIED_REGISTRY}" \
-e REGISTRY_PROXY_REMOTEURL="https://${PROXIED_REGISTRY}" \
-e REGISTRY_PROXY_USERNAME="${REGISTRY_USER}" \
-e REGISTRY_PROXY_PASSWORD="${REGISTRY_PASSWORD}" \
-v /Users/${MACOS_USERNAME}/${PATH_WHERE_TO_PERSIST_IMAGES}/docker/registry:/var/lib/registry \
--restart=always \
--name local-registry \
registry:2
Login to your local registry
echo -n "${REGISTRY_PASSWORD}" | docker login -u "${REGISTRY_USER}" --password-stdin "localhost:5000"
(optional) Verify that the persist directories are present
docker exec registry ls -la /var/lib/registry/docker/registry
ll /Users/${MACOS_USERNAME}/${PATH_WHERE_TO_PERSIST_IMAGES}/docker/registry/docker/registry
Try to pull one image from your private registry
(to see it proxied through the repository localhost:5000)
docker pull localhost:5000/${REPOSITORY}/${IMAGE}:${IMAGE_TAG}
(optional) Verify the image data has been synced on local host, where desired
docker exec registry ls -la /var/lib/registry/docker/registry
ll /Users/${MACOS_USERNAME}/${PATH_WHERE_TO_PERSIST_IMAGES}/docker/registry/docker/registry
If using Kubernetes
change the deployment spec container image to:
localhost:5000/${REPOSITORY}/${IMAGE}:${IMAGE_TAG}
Et voila!
You now can keep the images downloaded from your repository stored onto your host machine!
If internet is available, the local registry will ensure to have the most recent version of your pulled images, requesting it to the proxied registry (private, or the the Docker hub).
And you will have a last resort backup to run your container also when your internet connection is too slow for re-downloading everything you need, or is unavailable altogether!
(really useful with Minikube, when you need to destroy your docker virtual machine)
References:
https://docs.docker.com/registry/recipes/mirror/#run-a-registry-as-a-pull-through-cache
https://minikube.sigs.k8s.io/docs/handbook/mount/#driver-mounts
In this question it turned out, that I cannot use the sha256 mechanism in the FROM line in a Dockerfile to verify I am using the correct locally built non-DockerHub image in another derived image.
Is there another way to verify locally built Docker images? Some best practice maybe?
From docs:
By default, docker pull pulls images from Docker Hub. It is also
possible to manually specify the path of a registry to pull from
You can start a private docker registry on you localhost with the following command:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Say your image name isubuntu Then push image to that specific registry with:
docker push localhost:5000/ubuntu
In your Dockerfile you can use:
From localhost:5000/ubuntu
I found that every time I create a container in a new machine it is pulling image from docker hub. I have searched the web but found no well formed result. I have pulled the images from hub on host machine. now my question is how to make docker machine to check the host image location before pulling any image into virtualbox? I want machines to share and save images from host's default location so that I can share images among multiple machines without pulling it from registry for each machine
In a new machine, meaning a new VM, docker will indeed pull images from docker hub to the local VM docker image storage path:
/var/lib/docker/images
Each new machine has its own image cache.
If you want to share images amongst machine, you need one acting as your private registry, with its own certificate to enable https access.
See an example here, using gen_ssl_key_and_crt.sh for the key/certificate generation:
./gen_ssl_key_and_crt.sh
if [[ "$(docker inspect -f {{.State.Running}} registry 2> /dev/null)" == "" ]]; then
docker run -d -p 5000:5000 --restart=always --name registry \
-v $(pwd)/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/key \
registry:2
fi
Then your other machine can pull from that registry, which is much faster, and which will pull from docker hub only once, the first time it sees it does not have in its own registry the image.
For instance:
if [[ "$(docker images -q kv:5000/b2d/git:2.8.1 2> /dev/null)" == "" ]]; then
docker pull kv:5000/b2d/git:2.8.1
fi
For a simpler workaround, see:
"How to change the default docker registry from docker.io to my private registry"
"Run a local registry mirror"
I created two Docker containers. The first one provides a private Docker registry and the second one is a mirror of the official Docker registry:
docker run -d --name registry -v /local/path/to/registry:/registry -e SETTINGS_FLAVOR=local -e STORAGE_PATH=/registry -p 5000:5000 registry
docker run -d --name mirror -v /local/path/to/mirror:/registry -e STORAGE_PATH=/registry -e STANDALONE=false -e MIRROR_SOURCE=https:/registry-1.docker.io -e MIRROR_SOURCE_INDEX=https://index.docker.io -p 5555:5000 registry
Now I would like to combine both. Whenever a user pulls images it should first query the private registry and then the mirror. And when images are pushed they should only be pushed to the private registry.
I do not have an idea about how this can be done. Any help is appreciated.
You cannot just force all docker push commands to push to your private registry. One reason is that you can have any number of those registers. You have to first tell docker where to push by tagging the image (see lower).
Here is how you can setup docker hosts to work with a running private registry and local mirror.
Client set-up
Lets assume that you are running both mirror and private registry on (resolvable) host called dockerstore. Mirror on port 5555, registry on 5000.
Then on client machine(s) you should pass extra options to docker daemon startup. In your case:
Add --registry-mirror=http://dockerstore:5555 to tell daemon to prefer using local mirror rather then dockerhub. source
Add --insecure-registry dockerstore:5000 to access the private registry without further configuration. See this answer
Restart docker daemon
Using the mirror
When you pull any image the first source will be the local mirror. You can confirm by running a docker pull, e.g.
docker pull debian
In the output there will be message that image is being pulled from your mirror - dockerstore:5000
Using local registry
In order to push to private registry first you have to tag the image to be pushed with full name of the registry. Make sure that you have a dot or colon in the first part of the tag, to tell docker that image should be pushed to private registry.
Docker looks for either a “.” (domain separator) or “:” (port separator) to learn that the first part of the repository name is a location and not a user name.
Example:
Tag 30d39e59ffe2 image as dockerstore:5000/myapp:stable
docker tag 30d39e59ffe2 dockerstore:5000/myapp:stable
Push it to private registry
docker push dockerstore:5000/myapp:stable
Then you can pull as well
docker pull dockerstore:5000/myapp:stable
If not present, create the file:
sudo nano /etc/docker/daemon.json
Then paste the following:
{
"registry-mirrors": [
"https://hub-mirror.c.163.com",
"https://mirror.baidubce.com"
]
}
Then retart Docker daemon
$ sudo systemctl restart docker
[Source]
Just to be clear, docker documentation confirms that:
It’s currently not possible to mirror another private registry. Only
the central Hub can be mirrored.
Repository names are intended to be global, that is the repository redis always refers to the official Redis image from the Docker Hub. If you want to use a private registry, you prefix the repository name with the name of the registry e.g. localhost.localdomain:5000/myimage:mytag.
So when you pull or push, it will automatically go to the relevant registry. The mirror should be easy to set up, you just pass the URL to the daemon with the --registry-mirror= argument.
This isn't perfect for enterprise users, hence this (closed) Docker issue.