Adding annotation/label to OCI container-image without creating the container first - docker

For adding some annotations/labels to a image, we first have to create the container from the image, and then we can add some annotations/labels using buildah-config: buildah-config [options] containerID
But this requires first creating a container from image, then adding the required annotation/label, and then committing the changes back to an image.
But creating a container just for making some small change to the image configuration seems a unnecessary step which would take additional time and memory.
Is there a better way to add annotations/labels to a image?

For an image on a registry (or in an OCI Layout structure on the filesystem) there are various tools to modify that image. From go-containerregistry (by Google), there's crane mutate:
$ crane mutate --help
Modify image labels and annotations. The container must be pushed to a registry, and the manifest is updated there.
Usage:
crane mutate [flags]
Flags:
-a, --annotation stringToString New annotations to add (default [])
--append strings Path to tarball to append to image
--cmd strings New cmd to set
--entrypoint strings New entrypoint to set
-e, --env stringToString New envvar to add (default [])
-h, --help help for mutate
-l, --label stringToString New labels to add (default [])
-t, --tag string New tag to apply to mutated image. If not provided, push by digest to the original image repository.
Global Flags:
--insecure Allow image references to be fetched without TLS
--platform platform Specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). (default all)
-v, --verbose Enable debug logs
And from regclient (by myself), there's regctl image mod:
$ regctl image mod --help
EXPERIMENTAL: Applies requested modifications to an image
Usage:
regctl image mod <image_ref> [flags]
Flags:
--annotation stringArray set an annotation (name=value) (default )
--annotation-base stringArray set base image annotations (image/name:tag,sha256:digest) (default )
--buildarg-rm string delete a build arg (default "")
--buildarg-rm-regex string delete a build arg with a regex value (default "")
--config-time-max string max timestamp for a config (default "")
--create string Create tag
--data-max stringArray sets or removes descriptor data field (size in bytes) (default )
--expose-add stringArray add an exposed port (default )
--expose-rm stringArray delete an exposed port (default )
--external-urls-rm remove external url references from layers (first copy image with "--include-external") (default )
--file-tar-time-max stringArray max timestamp for contents of a tar file within a layer (default )
-h, --help help for mod
--label stringArray set an label (name=value) (default )
--label-to-annotation set annotations from labels (default )
--layer-rm-created-by string delete a layer based on history (created by string is a regex) (default "")
--layer-rm-index uint delete a layer from an image (index begins at 0) (default )
--layer-strip-file string delete a file or directory from all layers (default "")
--layer-time-max string max timestamp for a layer (default "")
--rebase rebase an image using OCI annotations (default )
--rebase-ref string rebase an image with base references (base:old,base:new) (default "")
--replace Replace tag (ignored when "create" is used)
--time-max string max timestamp for both the config and layers (default "")
--to-oci convert to OCI media types (default )
--volume-add stringArray add a volume definition (default )
--volume-rm stringArray delete a volume definition (default )
Global Flags:
--logopt stringArray Log options
--user-agent string Override user agent
-v, --verbosity string Log level (debug, info, warn, error, fatal, panic) (default "warning")
For example:
$ regctl manifest get localhost:5000/library/alpine:latest
Name: localhost:5000/library/alpine:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:bc41182d7ef5ffc53a40b044e725193bc10142a1243f395ee852a8d9730fc2ad
Manifests:
Name: localhost:5000/library/alpine:latest#sha256:1304f174557314a7ed9eddb4eab12fed12cb0cd9809e4c28f29af86979a3c870
Digest: sha256:1304f174557314a7ed9eddb4eab12fed12cb0cd9809e4c28f29af86979a3c870
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: localhost:5000/library/alpine:latest#sha256:5da989a9a3a08357bc7c00bd46c3ed782e1aeefc833e0049e6834ec1dcad8a42
Digest: sha256:5da989a9a3a08357bc7c00bd46c3ed782e1aeefc833e0049e6834ec1dcad8a42
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v6
Name: localhost:5000/library/alpine:latest#sha256:0c673ee68853a04014c0c623ba5ee21ee700e1d71f7ac1160ddb2e31c6fdbb18
Digest: sha256:0c673ee68853a04014c0c623ba5ee21ee700e1d71f7ac1160ddb2e31c6fdbb18
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7
Name: localhost:5000/library/alpine:latest#sha256:ed73e2bee79b3428995b16fce4221fc715a849152f364929cdccdc83db5f3d5c
Digest: sha256:ed73e2bee79b3428995b16fce4221fc715a849152f364929cdccdc83db5f3d5c
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
Name: localhost:5000/library/alpine:latest#sha256:1d96e60e5270815238e999aed0ae61d22ac6f5e5f976054b24796d0e0158b39c
Digest: sha256:1d96e60e5270815238e999aed0ae61d22ac6f5e5f976054b24796d0e0158b39c
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/386
Name: localhost:5000/library/alpine:latest#sha256:fa30af02cc8c339dd7ffecb0703cd4a3db175e56875c413464c5ba46821253a8
Digest: sha256:fa30af02cc8c339dd7ffecb0703cd4a3db175e56875c413464c5ba46821253a8
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/ppc64le
Name: localhost:5000/library/alpine:latest#sha256:c2046a6c3d2db4f75bfb8062607cc8ff47896f2d683b7f18fe6b6cf368af3c60
Digest: sha256:c2046a6c3d2db4f75bfb8062607cc8ff47896f2d683b7f18fe6b6cf368af3c60
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/s390x
$ regctl image config localhost:5000/library/alpine:latest
{
"created": "2022-08-09T17:19:53.47374331Z",
"architecture": "amd64",
"os": "linux",
"config": {
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
]
},
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:994393dc58e7931862558d06e46aa2bb17487044f670f310dffe1d24e4d1eec7"
]
},
"history": [
{
"created": "2022-08-09T17:19:53.274069586Z",
"created_by": "/bin/sh -c #(nop) ADD file:2a949686d9886ac7c10582a6c29116fd29d3077d02755e87e111870d63607725 in / "
},
{
"created": "2022-08-09T17:19:53.47374331Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
}
]
}
$ regctl image mod localhost:5000/library/alpine:latest \
--create mod --to-oci \
--annotation new.annotation=value \
--label some.label=1234
localhost:5000/library/alpine:mod
$ regctl manifest get localhost:5000/library/alpine:mod
Name: localhost:5000/library/alpine:mod
MediaType: application/vnd.oci.image.index.v1+json
Digest: sha256:5e1e3ed73857b08895477418bdd2b73cc9536886d84d98ef986eacca08e100ea
Annotations:
new.annotation: value
Manifests:
Name: localhost:5000/library/alpine:mod#sha256:f457cc17d630de3464ee3cd796fe83527a3daf3d1d28f710ed65b7162c29a632
Digest: sha256:f457cc17d630de3464ee3cd796fe83527a3daf3d1d28f710ed65b7162c29a632
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/amd64
Name: localhost:5000/library/alpine:mod#sha256:a442ff642289b8b9a4268451c29fc7ce8f3685ba08b32adb1fd9ddd36b4f46e4
Digest: sha256:a442ff642289b8b9a4268451c29fc7ce8f3685ba08b32adb1fd9ddd36b4f46e4
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/arm/v6
Name: localhost:5000/library/alpine:mod#sha256:003520fdc1749c2bd2d49781b24ae6937a219576541d20754124c24467e75ffc
Digest: sha256:003520fdc1749c2bd2d49781b24ae6937a219576541d20754124c24467e75ffc
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/arm/v7
Name: localhost:5000/library/alpine:mod#sha256:624a37e9b645a5286db52b2a8536f43bb64e0352fa10e847f4f049b4773046fc
Digest: sha256:624a37e9b645a5286db52b2a8536f43bb64e0352fa10e847f4f049b4773046fc
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/arm64
Name: localhost:5000/library/alpine:mod#sha256:9fbcc84ca534fa1d3b0047226860e39321da082615bdef03946edd5225952217
Digest: sha256:9fbcc84ca534fa1d3b0047226860e39321da082615bdef03946edd5225952217
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/386
Name: localhost:5000/library/alpine:mod#sha256:08246b80173b3ace4446aca1e26406d1cef1eaea29126ab3ebe964e0fbba4dd9
Digest: sha256:08246b80173b3ace4446aca1e26406d1cef1eaea29126ab3ebe964e0fbba4dd9
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/ppc64le
Name: localhost:5000/library/alpine:mod#sha256:945b2dab97c5518c276e53df551871ec089736f3abc7fe62003923b04ed9bcb5
Digest: sha256:945b2dab97c5518c276e53df551871ec089736f3abc7fe62003923b04ed9bcb5
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/s390x
$ regctl image config localhost:5000/library/alpine:mod
{
"created": "2022-08-09T17:19:53.47374331Z",
"architecture": "amd64",
"os": "linux",
"config": {
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"Labels": {
"some.label": "1234"
}
},
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:994393dc58e7931862558d06e46aa2bb17487044f670f310dffe1d24e4d1eec7"
]
},
"history": [
{
"created": "2022-08-09T17:19:53.274069586Z",
"created_by": "/bin/sh -c #(nop) ADD file:2a949686d9886ac7c10582a6c29116fd29d3077d02755e87e111870d63607725 in / "
},
{
"created": "2022-08-09T17:19:53.47374331Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
}
]
}

But if you modify an image in any way the image digest will change and thus you have a different image.

Related

vscode containerEnv not working in mounts

I'm using the vscode command Remote-contains: Open Folder in container...
I'm trying to mount bind a file into the docker container.
~/.config/dart/pub-tokens.json
The host file is under my HOME directory and I need it mounted in the same location within the container's HOME directory.
Here is my mount command from the vscode devcontainer.json
"mounts": [
"source=${localEnv:HOME}/.config/dart/pub-tokens.json,target=${containerEnv:HOME}/.config/dart/pub-tokens.json,type=bind,consistency=cached",
]
Note the 'containerEnv' in the target clause.
Launching the container via the vscode Remote-contains: Open Folder in container...
produces the following error: (for readability I've added some newlines)
Start: Run: docker run --sig-proxy=false -a STDOUT -a STDERR
--mount type=bind,source=/home/bsutton/git/onepub/onepub,target=/workspaces/onepub
--mount source=/home/bsutton/.config/dart/pub-tokens.json,target=${containerEnv:HOME}/.config/dart/pub-tokens.json,type=bind,consistency=cached
--mount source=/home/bsutton/.onepub/onepub.yaml,target=${containerEnv:HOME}/.onepub/onepub.yaml,type=bind,consistency=cached
--mount type=volume,src=vscode,dst=/vscode -l devcontainer.local_folder=/home/bsutton/git/onepub/onepub
--entrypoint /bin/sh vsc-onepub-7ff341664d5755895634c2f74983ff45-uid -c echo Container started
docker: Error response from daemon:
invalid mount config for type "bind": invalid mount path: '${containerEnv:HOME}/.config/dart/pub-tokens.json' mount path must be absolute.
It would appear that vscode isn't expanding the the containerEnv.
If I replace containerEnv it with localEnv it does get expanded (but the wrong path).
i.e. the following works:
"mounts": [
"source=${localEnv:HOME}/.config/dart/pub-tokens.json,target=${localEnv:HOME}/.config/dart/pub-tokens.json,type=bind,consistency=cached",
]
Here is the complete devcontainer.json
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.245.0/containers/ubuntu
{
"name": "Ubuntu",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick an Ubuntu version: jammy / ubuntu-22.04, focal / ubuntu-20.04, bionic /ubuntu-18.04
// Use ubuntu-22.04 or ubuntu-18.04 on local arm64/Apple Silicon.
"args": { "VARIANT": "ubuntu-22.04" }
},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "uname -a",
// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "vscode",
"features": {
"git": "latest",
"github-cli": "latest"
},
"mounts": [
"source=${localEnv:HOME}/.config/dart/pub-tokens.json,target=${containerEnv:HOME}/.config/dart/pub-tokens.json,type=bind,consistency=cached",
"source=${localEnv:HOME}/.onepub/onepub.yaml,target=${containerEnv:HOME}/.onepub/onepub.yaml,type=bind,consistency=cached"
]
}

How can I calculate a deterministic and reproducible checksum of a docker image, locally, without pinging any registry?

How can I calculate a deterministic and reproducible checksum of a docker image, locally, without pinging any registry?
The checksum should not depend on the image name or in which registry it lives. It should solely depend on the content of all layers.
For example, assume the following:
a given file a
a dockerfile with the content
FROM scratch
COPY a /a
Then building the image with docker build . --no-cache multiple times should always yield the same checksum.
The regular image ID does not cut it, as it somehow uses content from intermediate containers and hence always changes. I am also aware that since Docker 1.10, images have a "RepoDigest" attribute, which uniquely identifies images based on their layers' content. However, as far as I can tell, that digest is only calculated when pulling or pushing to a registry. Is there a way to get this field without contacting a registry? (and is it actually deterministic, regardless of image name, tag or repo?)
Basically, I'm looking for a way to run a good ol' sha256sum on a docker image. This would help me to achieve something similar to as what can be done with Bazel: a hermetic build environment, which in turn enables:
declaring dependencies between docker images, and have a CI system only rebuild what is needed without using docker's cache (assuming that I have a build tool which already manages caches)
allow me to "sign" images using the same approach as signing classic tarballs (that is, publish a checksum and somehow sign that)
the big one: enable reproducible builds!
This should be what Sigstore is for. It is made up of three projects:
Cosign, which signs software.
Fulcio, a certificate authority that lets anyone access short-lived certificates via OpenID Connect.
Rekor, a secure log of signing events that allows you to verify the provenance of software artifacts.
You can then follow "Keyless Sign and Verify Your Container Images With Cosign" (Chris Nesbitt-Smith)
Behind the scenes, cosign creates the keypair ephemerally (they last 20 minutes) and gets them signed by Fulcio using your authenticated OIDC identity.
That is OIDC: OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol.
OIDC allows:
Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server,
as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
COSIGN_EXPERIMENTAL=1 cosign sign image:tag
COSIGN_EXPERIMENTAL=1 cosign verify image:tag
But you would need to setup your own local OCI registry in order to keep the all toolchain local, since cosign stores signatures in an OCI registry, and uses a naming convention (tag based on the sha256 of what we're signing) for locating the signature index.
It looks like you're working on the same problem that I'm actively solving right now.
The big issue with the question is that container image builds with docker build are not deterministic or reproducible unless it happens to reuse the cache from a previous build. A container image build, even with the same filesystem layers, contains metadata on that build, and the metadata contains timestamps:
$ regctl manifest get localhost:5000/library/alpine --platform linux/amd64 --format body | jq .
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 1472,
"digest": "sha256:0ac33e5f5afa79e084075e8698a22d574816eea8d7b7d480586835657c3e1c8b"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 2814559,
"digest": "sha256:df9b9388f04ad6279a7410b85cedfdcb2208c0a003da7ab5613af71079148139"
}
]
}
$ regctl blob get localhost:5000/library/alpine sha256:0ac33e5f5afa79e084075e8698a22d574816eea8d7b7d480586835657c3e1c8b | jq .
{
"architecture": "amd64",
"config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"Image": "sha256:d49869997c508135352366cebd3509ee756bba1ceb8eef708a4c3ff0d481084a",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"container": "b714116bd3f3418e7b61a6d70dd7244382f0844e47a8d1d66dbf61cb1cb02b2b",
"container_config": {
"Hostname": "b714116bd3f3",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ",
"CMD [\"/bin/sh\"]"
],
"Image": "sha256:d49869997c508135352366cebd3509ee756bba1ceb8eef708a4c3ff0d481084a",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
"created": "2022-04-05T00:19:59.912662499Z",
"docker_version": "20.10.12",
"history": [
{
"created": "2022-04-05T00:19:59.790636867Z",
"created_by": "/bin/sh -c #(nop) ADD file:5d673d25da3a14ce1f6cf66e4c7fd4f4b85a3759a9d93efb3fd9ff852b5b56e4 in / "
},
{
"created": "2022-04-05T00:19:59.912662499Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
}
],
"os": "linux",
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:4fc242d58285699eca05db3cc7c7122a2b8e014d9481f323bd9277baacfa0628"
]
}
}
Both the "created" and "history" steps have timestamps that will be unique to the build. Changing those timestamps changes the digest of the config blob, which changes the digest of the image manifest.
The next issue you'll run into is that json serialization would need to be canonical. Some tools will use pretty formatting like jq, others will eliminate all unneeded whitespace for compactness, the order of listing multiple keys in a map doesn't need to be alphabetical, etc. So you need to ensure the same tool is always used for serialization and it has canonical output.
To build without pushing to a registry, you can have docker's buildkit output to an OCI layout tar file:
docker build --output type=oci,dest=/path/to/file.tar .
And in that tar, you will find an index.json with the digest of an image manifest as it was created by buildkit. I've been taking this a step further with regclient's image modification features, changing timestamps (in my case to the git commit time) and stripping other mutable values from the build. Then I verify the result matches a previous build.
Tools like cosign will allow you to sign an image using a digest rather than depending on the image in the registry, even before that image has been pushed.
The image mod feature in regclient is still very much a WIP, but you can see the current features here:
$ regctl image mod --help
EXPERIMENTAL: Applies requested modifications to an image
Usage:
regctl image mod <image_ref> [flags]
Flags:
--annotation stringArray set an annotation (name=value) (default )
--annotation-base stringArray set base image annotations (image/name:tag,sha256:digest) (default )
--buildarg-rm string delete a build arg (default "")
--buildarg-rm-regex string delete a build arg with a regex value (default "")
--config-time-max string max timestamp for a config (default "")
--create string Create tag
--data-max stringArray sets or removes descriptor data field (size in bytes) (default )
--expose-add stringArray add an exposed port (default )
--expose-rm stringArray delete an exposed port (default )
--external-urls-rm remove external url references from layers (first copy image with "--include-external") (default )
-h, --help help for mod
--label stringArray set an label (name=value) (default )
--label-to-annotation set annotations from labels (default )
--layer-rm-created-by string delete a layer based on history (created by string is a regex) (default "")
--layer-rm-index uint delete a layer from an image (index begins at 0) (default )
--layer-strip-file string delete a file or directory from all layers (default "")
--layer-time-max string max timestamp for a layer (default "")
--replace Replace tag (ignored when "create" is used)
--time-max string max timestamp for both the config and layers (default "")
--to-oci convert to OCI media types (default )
--volume-add stringArray add a volume definition (default )
--volume-rm stringArray delete a volume definition (default )
Global Flags:
--logopt stringArray Log options
--user-agent string Override user agent
-v, --verbosity string Log level (debug, info, warn, error, fatal, panic) (default "warning")
The other part of the puzzle is to make RUN steps reproducible. That's less trivial since not only do the files have timestamps, but the contents of the files being created could have timestamps or other mutable content, and the commands could pull from external mutable sources. Solving that part of the problem is still a work in progress for me.
For a docker image, named "hello-world":
docker save --output hello-world.tar hello-world
sha256sum hello-world.tar
It should give you the content sha of image.

Pull docker image for different architecture

I have an air gapped system that I'm running some docker containers on. I'm trying to get some images on it however the system is a different architecture than what I'm running. For a few images (like GCC) I was able to just say docker pull repo/gcc and that worked fine, however for some reason when I try to do docker pull repo/python I get:
Using default tag: latest
latest: Pulling from repo/python
no matching manifest for linux/amd64 in the manifest list entries
Is there a way to specify the architecture in my pull request?
With the latest docker, you can specify platform when you're pulling
docker pull --platform linux/arm64 alpine:latest
Docker from version 20.10.0+ (released on 2020-12-08) supports explicit definition of the platform via --platform tag, e.g.:
docker pull --platform linux/arm64 repo/python
Of course, source must contain an image for the requested platform.
Answer for Docker versions before 20.10.0:
To answer question from the title: you can pull image by digest.
Example: list all supported architectures (manifest):
$ docker manifest inspect ckulka/multi-arch-example
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 2200,
"digest": "sha256:6eaeab9bf8270ce32fc974c36a15d0bac4fb6f6cd11a0736137c4248091b3646",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 2413,
"digest": "sha256:f02e0fd2918a894ecd49d67b802c22082dc3c6424f6566e1753a83ba833b0993",
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v5"
}
},
...
And then pull by digest, e.g. arm arch (pulled on linux machine):
$ docker pull ckulka/multi-arch-example#sha256:f02e0fd2918a894ecd49d67b802c22082dc3c6424f6566e1753a83ba833b0993
But you can't run all architectures, so it can be useless when you pull image for different architecture.
If the image is not multi-arch, you cant unless you emulate your architecture to be of the target architecture of the manifest.

Why digests are different depend on registry?

AFAIK, image digest is a hash of image's manifest body.
When I pull busybox image from docker hub, and push it to my private registry, the digests get different.
$ docker pull busybox
...
Digest: sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4
Status: Downloaded newer image for busybox:latest
$ docker tag busybox myregistry/busybox
$ docker push myregistry/busybox
...
08c2295a7fa5: Pushed
latest: digest: sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac size: 527
$ docker images --digests
myregistry/busybox latest sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac efe10ee6727f 2 weeks ago 1.13MB
busybox latest sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4 efe10ee6727f 2 weeks ago 1.13MB
The images are not changed at all, and the image ids are same as each other.
But why image digests get different?
Updated:
Interestingly, the digest from another private registry is exactly same with the digest by my private registry.
$ docker image inspect efe10ee6727f
...
"RepoDigests": [
"myregistry/busybox#sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac",
"busybox#sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4",
"anotherregistry/busybox#sha256:8573b4a813d7b90ef3876c6bec33db1272c02f0f90c406b25a5f9729169548ac"
],
The digests you are looking at are registry digests, which are different from the image id digest. You can have an image id that has different registry references (and possibly digests) for all the places it has been pushed. You can see the two id's in the inspect output:
$ docker inspect busybox --format 'Id: {{.Id}}
Repo Digest: {{index .RepoDigests 0}}'
Id: sha256:efe10ee6727fe52d2db2eb5045518fe98d8e31fdad1cbdd5e1f737018c349ebb
Repo Digest: busybox#sha256:2605a2c4875ce5eb27a9f7403263190cd1af31e48a2044d400320548356251c4
If the registry is using an old v1 manifest, the repository name and tag are part of that manifest, which means it will change as it's moved between registries:
{
"name": <name>,
"tag": <tag>,
"fsLayers": [
{
"blobSum": "<digest>"
},
...
]
],
"history": <v1 images>,
"signature": <JWS>
}
However for OCI manifests and Docker's v2 manifests, this is no longer the case and you should see the same registry digest for the same image:
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 7023,
"digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 32654,
"digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 16724,
"digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 73109,
"digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736"
}
]
}
Digests themselves are a sha256 digest of the content, which you can also find in OCI's implementation. When you pull an image locally, some things change, including the layers being decompressed, and multi-platform images are dereferenced to your local platform. Because of those changes, the digest on the content will change and the image ID will not match the registry digest.
Therefore, to compare images between registries, make sure you specify you want a v2 schema with the accept header, otherwise the registry will convert the result back to a v1 schema. In curl, passing those headers looks like:
curl \
-H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
-H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" \
http://$registry/v2/$repo/manifests/$tag

Issue in building docker image using habitus

I am trying to build Docker image using Habitus, so that I can securely pass build time secrets (github ssh keys) to docker.
My Habitus build file:
build:
version: 2016-03-14
steps:
builder:
name: search/poirot
dockerfile: Dockerfile
secrets:
id_rsa:
type: file
value: _env(HOME)/.ssh/id_rsa
cleanup:
commands:
- rm -rf /root/.ssh/
I have the latest Docker for Mac installed on my machine. I am running this command to build:
sudo ./habitus
--certs=$HOME/.docker/machine/certs/ --host=192.168.99.100:59124
I am using the IP and port from "~/.docker/machine/machines/default/config.json"
Content:
"Driver": {
"IPAddress": "192.168.99.100",
"MachineName": "default",
"SSHUser": "docker",
"SSHPort": 59124,
"SSHKeyPath": "/Users/shiladityamandal/.docker/machine/machines/default/id_rsa",
"StorePath": "/Users/shiladityamandal/.docker/machine",
"SwarmMaster": false,
"SwarmHost": "tcp://0.0.0.0:3376",
"SwarmDiscovery": "",
"VBoxManager": {},
"HostInterfaces": {},
"CPU": 1,
"Memory": 2048,
"DiskSize": 20000,
"NatNicType": "82540EM",
"Boot2DockerURL": "",
"Boot2DockerImportVM": "",
"HostDNSResolver": false,
"HostOnlyCIDR": "192.168.99.1/24",
"HostOnlyNicType": "82540EM",
"HostOnlyPromiscMode": "deny",
"NoShare": false,
"DNSProxy": true,
"NoVTXCheck": false
},
I keep getting the following error during build:
Build for step test/test failed due to Post
https://192.168.99.100:59124/build?dockerfile=Dockerfile.generated&rm=1&t=search%2Fservice:
dial tcp 192.168.99.100:59124: i/o timeout
What am I doing wrong?
I was following this process- https://dzone.com/articles/using-ssh-private-keys-securely-in-docker-build
Solved it myself. Had to connect to 192.168.99.100:2376

Resources