I'm stuck following our internal procedure that requires pinning a docker base image to a specific SHA. And, I'm stuck needing to build a mulitarch image for both x86_64 and ARM.
The base image that I'm looking at has a SHA for each architecture (which I would expect, makes total sense). So how can I specify a specific SHA in the FROM if the resulting image is also supposed to be multiarch? Am I stuck using a Dockerfile for each arch??
Docker Hub doesn't show it on the web page, but the manifest list for a multi-platform image has its own digest, and that is what you want to provide to tools. There are a variety of tools that can get this. My own tool is regclient with the regctl CLI, go-containerregistry from Google has crane, and Docker has been including an imagetools CLI under buildx:
$ regctl image digest bitnami/minideb
sha256:713d1fbd2edbc7adf0959721ad360400cb39d6b680057f0b50599cba3a4db09f
$ crane digest bitnami/minideb
sha256:713d1fbd2edbc7adf0959721ad360400cb39d6b680057f0b50599cba3a4db09f
$ docker buildx imagetools inspect bitnami/minideb
Name: docker.io/bitnami/minideb:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:713d1fbd2edbc7adf0959721ad360400cb39d6b680057f0b50599cba3a4db09f
Manifests:
Name: docker.io/bitnami/minideb:latest#sha256:2abaa4a8ba2c3ec9ec3cb16a55820db8d968919f41439e1e8c86faca81c8674a
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/bitnami/minideb:latest#sha256:3c44390903734b2657728fcad8fb33dcdf311bdeaafcc3b9f179d78bdf4da669
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
Related
This question already has answers here:
How to copy multi-arch docker images to a different container registry?
(3 answers)
Closed 2 months ago.
I'm trying to pull a Docker image from Dockerhub and copy it into my own Dockerhub registry. I want to do this for all os/architectures available under the tag I am copying. For example, node:latest has the following architectures:
linux/amd64
linux/arm/v7
linux/arm64/v8
If I run the following:
docker pull node:latest
docker tag node:latest myregistry/node:latest
docker push myregistry/node:latest
I only end up with linux/amd64 in my registry, because my laptop is Intel and only pulls/pushes the Intel architecture. I want to be able to pull/push the arm architectures as well, so M1 users can pull from my registry.
How can I do this? The images are already built, so I don't want to have to rebuild them, and I don't have an M1. I just want to "copy" the image that has already been built into my registry.
This seems to have done the trick:
docker pull node:latest
docker manifest inspect # <- note the digests of the architectures you want to include
# Create a tag for any digests you are interested in
docker tag node:latest#sha256:867c09f220095929f3ab4113e7530a6e38833f2eb4317cb8998307528026621f myregistry/node:latest-amd64
docker tag node:latest#sha256:55298bd901ba7d9b914842c0cbb1087571b50121791846a17b78fa02f904962b myregistry/node:latest-arm64
# Push to your registry
docker push myregistry/node:latest-arm64
docker push myregistry/node:latest-amd64
# Create and push the manifest
docker manifest create myregistry/node:latest --amend myregistry/node:latest-arm64 --amend myregistry/node:latest-amd64
docker manifest push myregistry/node:latest
Dockerhub will list both architectures under the "latest" tag:
I wanted to download docker images of oraclelinux for amd64 and arm64 architectures. But both are showing same sha256 digest. Why is that?
docker pull --platform=linux/amd64 oraclelinux:7-slim
Trying to pull repository docker.io/library/oraclelinux ...
7-slim: Pulling from docker.io/library/oraclelinux
Digest: sha256:7a46c0134e2cad2d15a98eac50c89f9e0f4640c0461b838672d41ea0710d75c5
Status: Downloaded newer image for oraclelinux:7-slim
oraclelinux:7-slim
docker pull --platform=linux/arm64 oraclelinux:7-slim
Trying to pull repository docker.io/library/oraclelinux ...
7-slim: Pulling from docker.io/library/oraclelinux
Digest: sha256:7a46c0134e2cad2d15a98eac50c89f9e0f4640c0461b838672d41ea0710d75c5
Status: Downloaded newer image for oraclelinux:7-slim
oraclelinux:7-slim
I wanted to use both separately in Gitlab CI-CD as mentioned in How to specify image platform in gitlab-ci.yml .
How can I do this?
The digest listed by docker is the manifest list. It will be dereferenced to the platform specific manifest as the image is pulled, but docker lists the manifest list for portability (the same digest can be used other multiple platforms).
$ regctl manifest get oraclelinux:7-slim
Name: oraclelinux:7-slim
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:7a46c0134e2cad2d15a98eac50c89f9e0f4640c0461b838672d41ea0710d75c5
Manifests:
Name: docker.io/library/oraclelinux:7-slim#sha256:eccab04a8a5299ea5ae6cc51ad697aa01012ff2732c999360c4d218dd9451440
Digest: sha256:eccab04a8a5299ea5ae6cc51ad697aa01012ff2732c999360c4d218dd9451440
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/library/oraclelinux:7-slim#sha256:fd4d966f65ddc0ac1727570766563e2f9d0dd8e2557234d179a017e244e67979
Digest: sha256:fd4d966f65ddc0ac1727570766563e2f9d0dd8e2557234d179a017e244e67979
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
As mentioned by #BMitch this sha256 is from the manifest list, which contains all architectures.
Since multi architecture/platform tags of a Docker image have different digests, You can pull a Docker image using its digest (instead of using tags) to pull the desired architecture/platform.
As we can see in oraclelinux -- The sha256 from arm64 is fd4d966f65ddc0ac1727570766563e2f9d0dd8e2557234d179a017e244e67979 So you can pull this image by running:
$ docker pull oraclelinux#sha256:fd4d966f65ddc0ac1727570766563e2f9d0dd8e2557234d179a017e244e67979
But it looks like the oraclelinux has another repository to store arm64 images, so maybe you can run:
$ docker pull oraclelinux:7-slim
$ docker pull arm64v8/oraclelinux:7-slim
Does it work for you?
Add before_script
export IMAGE_NAME
export IMAGE_TAG
Then
script
docker pull $IMAGE_NAME:$IMAGE_TAG
Adjust this to suit best for you.
I pulled both.
diff can help you.
diff <(docker inspect 6b9fd09833be) <(docker inspect 554de8d676bd)
I am running a Jenkins Alpine Linux AMD64 docker image, which I'm building myself and want to add linux/arm64 docker buildx support to it, in order to generate multi-platform images and I do not know how it supposed to work.
When I check the supported platform I get:
+ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
default * docker
default default running linux/amd64, linux/386
since I'm within an AMD64 image. I read that I need to install qemu for this, but I have no clue how buildx will recognize that.
The documentation is relatively bare on this at: https://docs.docker.com/buildx/working-with-buildx/
Anyone an idea how to add linux/arm64 build capability within a linux/amd64 image?
The only solution I see right now is to build an image on an actual arm64 system.
To use buildx, make sure your Docker runtime is at least version 19.03. buildx actually comes bundled with Docker by default, but needs to be enabled by setting the environment variable DOCKER_CLI_EXPERIMENTAL.
export DOCKER_CLI_EXPERIMENTAL=enabled
If you're on Linux, you need to set up binfmt_misc. This is pretty easy in most distributions, but is even easier now that you can just run a privileged Docker container to set it up for you.
docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d
Create a new builder which gives access to the new multi-architecture features:
docker buildx create --use --name multi-arch-builder
Then you'll be able to build the containers with:
docker buildx build --platform=[your target platform] ...
This is the setup I use on my Jenkins pipeline.
Relevant documentation:
https://docs.docker.com/desktop/multi-arch/
In-depth tutorial: https://medium.com/#artur.klauser/building-multi-architecture-docker-images-with-buildx-27d80f7e2408
When I docker pull hello-world, I got the image with a digest of f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e
$ docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
Digest: sha256:f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e
Status: Image is up to date for hello-world:latest
docker.io/library/hello-world:latest
I was using a Mac, but when I docker inspect hello-world:latest, I saw the os/arch is linux/amd64
...
"Architecture": "amd64",
"Os": "linux",
...
So I went to https://hub.docker.com/_/hello-world/?tab=tags and found strangely enough, the latest hello-world for linux/amd64 is at https://hub.docker.com/layers/hello-world/library/hello-world/latest/images/sha256-92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a?context=explore with a digest of 92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
So I pulled down this image as well
$ docker pull hello-world#sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a: Pulling from library/hello-world
Digest: sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
Status: Downloaded newer image for hello-world#sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
docker.io/library/hello-world#sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
Surprisingly, I ended up with two images with the same tag, same image ID, but different digests.
$ docker image ls --digests
REPOSITORY TAG DIGEST IMAGE ID CREATED SIZE
hello-world latest sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a fce289e99eb9 15 months ago 1.84kB
hello-world latest sha256:f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e fce289e99eb9 15 months ago 1.84kB
Are these two images the same one? How can I uniquely address an image if I want consistency across my team?
The hello-world image is a multi-platform image. Each platform contains its own manifest, and there is a manifest list that points to all of the platforms. Each of those manifests and the manifest list have their own digest. In your listing, docker is showing the digest for the manifest list, and your specific platform manifest's digest:
$ docker buildx imagetools inspect hello-world#sha256:f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e
Name: docker.io/library/hello-world#sha256:f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:f9dfddf63636d84ef479d645ab5885156ae030f611a56f3a7ac7f2fdd86d7e4e
Manifests:
Name: docker.io/library/hello-world#sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/library/hello-world#sha256:e5785cb0c62cebbed4965129bae371f0589cadd6d84798fb58c2c5f9e237efd9
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v5
Name: docker.io/library/hello-world#sha256:50b8560ad574c779908da71f7ce370c0a2471c098d44d1c8f6b513c5a55eeeb1
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7
Name: docker.io/library/hello-world#sha256:963612c5503f3f1674f315c67089dee577d8cc6afc18565e0b4183ae355fb343
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64/v8
Name: docker.io/library/hello-world#sha256:85dc5fbe16214366748ebe9d7cc73bc42d61d19d61fe05f01e317d278c2287ed
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/386
Name: docker.io/library/hello-world#sha256:8aaea2a718a29334caeaf225716284ce29dc17418edba98dbe6dafea5afcda16
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/ppc64le
Name: docker.io/library/hello-world#sha256:577ad4331d4fac91807308da99ecc107dcc6b2254bc4c7166325fd01113bea2a
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/s390x
Name: docker.io/library/hello-world#sha256:468a2702c410d84e275ed28dd0f46353d57d5a17f177aa7c27c2921e9ef9cd0e
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: windows/amd64
OSVersion: 10.0.17763.1098
-> This is how manifest works with docker. Config.digest file remains same for both images as they are pointing towards same configuration of layers.
->The digest used for docker pull represents the digest of image manifest which is stored in a registry. This digest is considered the root of a hash chain since the manifest itself contains the hash of the content which will be downloaded and imported into docker.
->See the schema 2 spec for a description of this manifest https://docs.docker.com/registry/spec/manifest-v2-2/. The image id used within docker can be found in this manifest as config.digest. This config represents the image configuration which will be used within docker.
->So you could say the manifest is the envelope and image is what is inside. The manifest digest will always be different than the image id BUT for any given manifest the same image id should always be produced.
->As it is a hash chain, we cannot guarantee that the manifest digest will always be the same for a given image id.
-> In most cases it should usually produce the same digest, we just cannot guarantee it but do a best effort. The possible difference in manifest digest is because we do not store gzipped blobs locally and exporting of layers may produce a different digest, even though the uncompressed content should remain the same.
->The image id itself verifies that uncompressed content is the same, this is what we mean when we say the image id is now a content addressable identifier.
So I'm no expert in Docker, I can't tell you why this happenned or if it becomes an issue, however I just discovered here how to clean it up:
https://docs.docker.com/engine/reference/commandline/rmi/
You can delete the second image using image ID, then all is well. To get the image ID use the command:
docker image list
Then once you have the ID, use the command:
docker image rmi <image ID> <-f if there are multiple images with the same ID>
You may end up having to delete all of your images and building the image you intended to use, however it will allow you to be confident that you are pushing the intended build.
I have a Docker image in dockerhub and this has been built several times because I need to update the PHP version to the newest. I need to use a previous version of that image and I think the way to go is by using the immutable identifier aka digest.
Here is the documentation in how to pull a given image by it's digest but I can't find a way to get all the digest from that image.
If you double click on a given build you obtain certain information like a build code, for example: berpxpunhmqe7bqh6lce5ub but I don't think that is such digest.
How do I find that digest for a given build?
Assuming you have a tag/identifier for the previous version and/or have a version in you local image cache, finding the digest to use with pull by digest can be done with a docker image inspect as follows:
$ docker image inspect --format "{{.RepoDigests}}" alpine:3.6
[alpine#sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab]
In this example I'm looking at the 3.6 tag of the alpine image, and the response is a string I can use with commands like docker pull:
$ docker pull alpine#sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab
sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab: Pulling from library/alpine
Digest: sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab
Status: Image is up to date for alpine#sha256:b40e202395eaec699f2d0c5e01e6d6cb8e6b57d77c0e0221600cf0b5940cf3ab
The potential problem with your specific image is that it looks like the latest tag has been used for all your builds, so unless you have a local cache of an older image, it may be quite difficult to find the older sha256 digest references to prior builds.
There are a few possible ways to find the digest of a prior image if local cached information hasn't been deleted via docker system prune or other cleanup utilities:
docker images -a | grep <image name> can be used to display all images, including those which have been untagged. A below example shows an updated ubuntu:latest where I still have access to the older image. Using that ID (which is not a digest), I can use the same docker image inspect --format '{{.RepoDigests}}' <image ID> to retrieve the actual digest of an older "build" of ubuntu.
If I had a container that is running or exited using a prior version of the image, I could find the digest of that image by first inspecting the container and finding the image ID, and then inspecting that image ID as above and retrieving the older image's digest. In this somewhat contrived example I have an exited container, 1edd.., which I inspect to find the image ID, which happens to still be validly tagged, but using it's id I can then use image inspect to get the digest, even if it is no longer tagged in my image cache.
Example 1:
$ docker images -a | grep ubuntu
ubuntu latest 747cb2d60bbe 3 weeks ago 122MB`
ubuntu <none> ebcd9d4fca80 5 months ago 118MB
$ docker image inspect --format '{{.RepoDigests}}' ebcd9
[ubuntu#sha256:382452f82a8bbd34443b2c727650af46aced0f94a44463c62a9848133ecb1aa8]
Example 2:
$ docker ps -aq
1edd14b528db
$ docker container inspect 1edd | grep Image
"Image": "sha256:76da55c8019d7a47c347c0dceb7a6591144d232a7dd616242a367b8bed18ecbc",
"Image": "alpine:3.6",
$ docker image inspect --format '{{.RepoDigests}}' 76da55
[alpine#sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d]