I'm trying to figure out how to publish a versioned Docker container to Quay.io and am not seeing how to do so in their docs. From the docs:
# Login to Quay.io
docker login quay.io (will now be prompted for username + password)
# Get your <containerID>
docker ps -l
# Tag that container by <containerId>
docker commit <containerId> quay.io/<myUsername>/<myRegistry>
# Now publish it to Quay.io
docker push quay.io/<myUsername>/<myRegistry>
However this leaves me very confused:
What if I want my containers tagged/versioned with, say, 1.0.0-SNAPSHOT, or 1.0-rc, or 2.0 or 3.1.6-G.A? Where do I specify the actual tag name/version?
When I run the docker push command, how does Docker know which (tagged/versioned) container to publish?
First, it's worth noting that you don't push containers to a registry. You push images. The docker commit command in your question creates new image based on an active container.
When assigning tags to images -- using either the docker tag command, which is more common [1], or when using docker commit -- the format of a tag is [<registry>/]<repository>[:<tag>], where [...] is used to represent an option component. So for example you might tag an image targeting your account on Docker Hub like this:
docker tag smeeb/webserver
This would be assigned the latest tag, since you haven't specified one explicitly. To explicitly assign the tag awesome instead of latest:
docker tag smeeb/webserver:awesome
When you're tagging an image for an alternate registry, you include the registry name as part of the tag, as in:
docker tag <imageId> quay.io/smeeb/webserver
Here, quay.io is the registry, smeeb/webserver is the repository, and you've implicitly assigned the latest tag. You can be explicit instead:
docker tag <imageId> quay.io/smeeb/webserver:1.0.0-SNAPSHOT
And so forth. You can use the same syntax when creating images with docker commit.
[1]: The reason I saw that using docker tag is more common is that use of docker commit is generally considered an anti-pattern. Use a Dockerfile to create your new image, which makes it much easier to reproduce the same configuration (and allows you to version control the configuration of your image).
Related
In my use case I always fetch the image tagged with "latest" tag. This "latest" tag gets updated regularly. So even if the latest tag image is updated on registry, the "docker run" command does not update it on local host. This is expected behavior as the "latest" image exists on local host.
But I want to make sure that if the "latest" image on local host and registry are different then it should pull the latest image from registry.
Is there any way to achieve this?
You can manually docker pull the image before you run it. This is fairly inexpensive, especially if the image hasn't changed. You can do it while the old container is still running to minimize downtime.
docker pull the-image
docker stop the-container
docker rm the-container
docker run -d ... --name the-container the-image
In an automated environment you might consider avoiding the latest tag and other similar fixed strings due to exactly this ambiguity. In Kubernetes, for example, the default behavior is to reuse a local image that has some name, which can result in different nodes running different latest images. If you label your images with a date stamp or source-control ID or something else such that every image has a unique tag, you can just use that tag.
Finding the tag value can be problematic outside the context of a continuous-deployment system; Docker doesn't have any built-in way to find the most recent tag for an image.
# docker pull the-image:20220704 # optional
docker stop the-container
docker rm the-container
docker run -d ... --name the-container the-image:20220704
docker rmi the-image:20220630
One notable advantage of this last approach is that it's very easy to go back to an earlier build if today's build happens to be broken; just switch the image tag back a build or two.
Can docker be connected to more than one registry at a time and how to figure out which registries it is currently connected too?
$ docker help | fgrep registr
login Log in to a Docker registry
logout Log out from a Docker registry
pull Pull an image or a repository from a registry
push Push an image or a repository to a registry
As you can see, there is no option to list the registries. I did find
a way by running:
$ docker system info | fgrep -i registr
Registry: https://index.docker.io/v1/
So... one regsitry at a time only? It is not like apt where one can point to more than one source? Anybody can point me to some good documentation about docker and registries?
Oddly, I search the web to no vail.
Aside from docker login, Docker isn't "connected to a registry" per se. Registry names are part of the image name, and Docker will connect to a registry server if it needs to pull an image.
As a specific example, the official Docker image for Elasticsearch is on a non-default registry run by Elastic. The example in that documentation is
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.0
# ^^^^^^^^^^^^^^^^^
# registry host name
You don't need to otherwise configure your system to connect to that registry, download an index, or anything else. In fact, you don't even need this docker pull command; if you directly docker run the image, Docker will download it if it doesn't have a copy locally.
The default registry is Docker Hub, docker.io, and this cannot be changed.
There are several alternate registries out there. The various public-cloud providers each have their own, and there are also several free-standing image registries. Each has its own instructions on how to set it up. You always need to include the registry name as part of the image name. The Google Container Registry has a simple name syntax, for example, so if you use GCR then you can
# build an image locally, labeled to be stored in GCR
# (this step does not contact or use GCR at all)
docker build gcr.io/my-name/my-image:tag
# authenticate to the registry
# (normally GCR has a Google-specific login sequence)
docker login https://gcr.io
# push the image
docker push gcr.io/my-name/my-image:tag
# run the image, pulling it if not present
docker run ... gcr.io/my-name/my-image:tag
There's a well-known approach to have docker images copied from one container registry to another. In case the original registry is dockerhub, the typical workflow would be something like this:
docker pull <image:tag>
docker tag <image:tag> <new-reg-url/uid/image:tag>
docker push <new-reg-url/uid/image:tag>
Now, how do you the above when dealing with images with multi-architecture layers?
As per the information in this link, you can rely on buildx to construct multi-arch images, and while doing that, you can also upload those to whichever repo you wish, but how do i do this without having to first build the images?
Looks like buildx cli has unnecessarily (?) coupled the uploading process with the building one. Any suggestions?
Thanks!
While the docker pull ...; docker tag ...; docker push ... syntax is the easy way to move images between registries, it has a couple drawbacks. First, as you've seen, is that it dereferences a multi-platform image to a single platform. And the second is that it pulls all layers to the docker engine even if the remote registry already has those layers, making it a bad method for ephemeral CI workers that would always need to pull every layer.
To do this, I prefer talking directly to the registry servers rather than the docker engine itself. You don't need the functionality from the engine to run the images, all you need is the registry API. Docker has documented the original registry API and OCI recently went 1.0 on the distribution-spec which should get us some standardization.
There's a variety of tooling based on those specs, from the docker engine itself and containerd, to skopeo, google's crane, and I've also been working on regclient. Doing this with regclient's regctl command looks like:
regctl image copy <source_image:tag> <target_image:tag>
And the result is the various layers, image config, manifests, and multi-platform manifest list will be copied between registries, but only for the layers that don't already exist on the target registry.
2022 (docker builtin) solution
It's posible to perform the copy using the not-well-documented built in command docker buildx imagetools create using --tag
# i.e.
OLD_TAG=registry.example.com/namespaced/repository/example-image:old-tag
NEW_TAG=registry.example.com/namespaced/repository/example-image:new-tag
# we can
docker buildx imagetools create --tag "$NEW_TAG" "$OLD_TAG"
Reference documentation
IMPORTANT NOTE: There is no support at the moment to perform this operation against different repositories. Given tags like
OLD_TAG=registry.example.com/namespaced/repository/example-image:latest
NEW_TAG=registry.example.com/other-repository/example-image:latest
You end with an error like
error: multiple repositories currently not supported
For this situation I'm going to test the actual accepted answer
As #laconbass has written, this can be done with docker buildx imagetools create. The ability to do this over multiple repos was added in this PR
docker buildx imagetools create -t <NEW-TAG> <OLD-TAG>
I setup my own docker registry using this image: https://hub.docker.com/_/registry
The first thing I noticed was that the latest tag was not automatically being set. When pushing an image to dockerHub it gets assigned automatically the latest tag so you can always access the newest image with that tag.
That does not seem to be the default behaviour for a self-hosted docker registry. Is there any way to configure it so I get a latest-tag automatically?
Even though Docker Hub behaves in that way, pushing latest tag automatically is not the default behavior for a Docker registry.
Usually, the action of pushing latest in addition to the tag that you are pushing is a custom logic that is a part of CI process set up for you app.
This is just one of many ways to achieve this
docker build -t myregistry.com/myapp:VARIABLE_TAG -t myregistry.com/myapp:latest .
docker push myregistry.com/myapp:VARIABLE_TAG
docker push myregistry.com/myapp:latest
I have some private Docker registry: http://some-registry-somewhere.com:5000.
When I need to run my compose configuration, I need to pull a target image.
$ docker pull some-registry-somewhere.com:5000/target/image:tag1
In docker-compose.yml file, I have to set the same full URL-path because there is pulled image some-registry-somewhere.com:5000/target/image:tag1.
To have an image with image name only we may tag it:
$ docker tag some-registry-somewhere.com:5000/target/image:tag1 target/image:tag1
But is there any way to automatically cut Docker registry URL through Docker?
There is no such way, because of API specification. The image name is not just the tag, it also identifies for docker engine, which registry should be used for pushes and pulls of this image.
While the first some-registry-somewhere.com:5000/target/image:tag1 is image target/image:tag1 which is located in some-registry-somewhere.com:5000.
The second one target/image:tag1 is, in other words, image, docker.io/target/image:tag1, which is located in official repository.
In fact, they can be different in most of the cases.
The one way, which is not good, actually, because can be confusing (see again about repositories), is to use &&:
docker pull some-registry-somewhere.com:5000/target/image:tag1 && docker tag some-registry-somewhere.com:5000/target/image:tag1 target/image:tag1