Docker images can be built for multi architectures. This is usually done by creating a specific image per architecture, and then creating manifest as a proxy to the right image depending on the system that pulls the image.
That's great.
Now, with docker-compose, it's also possible to build images, but I don't see a way to build the image depending on the architecture, so it seems like the only way to have a single docker-compose definition for multiple architectures, is to have pushed multi-arch images to a registry and pull from there.
Does anyone know of a way to build local images for the right arch with the docker-compose build step?
I don't think you can use docker-compose build, but you could use docker buildx bake to build multi-arch using a docker-compose.yml.
docker buildx bake --push --set *.platform=linux/amd64,linux/arm64
You'll probably want to read more about docker buildx bake and building multi-platform images.
Related
I want to build Docker image for AMD and ARM Graviton2 processors. I already know about multi-arch CLI command docker buildx build --platform linux/amd64,linux/arm64, manifests and the fact that Docker will pull the right image variant matching architecture.
I wonder if I have to use in my Dockerfile for ARM as a parent arm64v8/ubuntu:20.04 or it's fine to use ubuntu:20.04 for both? Will it work the same way on both architectures? What's the purpose of this official arm64v8 dockerhub repo?
There is a significant difference in build times - 5min with FROM ubuntu:20.04 vs 30min with FROM arm64v8/ubuntu:20.04.
Ok so I figured it out that this ubuntu:20.04 and this arm64v8/ubuntu:20.04 two images has exactly the same SHA. So Ubuntu:20.04 is only the parent of all these per-arch images and if you run docker manifest inspect ubuntu you will see it all.
So it's clear that arm64v8/ubuntu:20.04 repo is only for the case you want to build ARM image on different architecture (if you don't want to use multibuild buildx command). It that case you have to start writing your Dockerfile with FROM arm64v8/ubuntu:20.04.
There's a well-known approach to have docker images copied from one container registry to another. In case the original registry is dockerhub, the typical workflow would be something like this:
docker pull <image:tag>
docker tag <image:tag> <new-reg-url/uid/image:tag>
docker push <new-reg-url/uid/image:tag>
Now, how do you the above when dealing with images with multi-architecture layers?
As per the information in this link, you can rely on buildx to construct multi-arch images, and while doing that, you can also upload those to whichever repo you wish, but how do i do this without having to first build the images?
Looks like buildx cli has unnecessarily (?) coupled the uploading process with the building one. Any suggestions?
Thanks!
While the docker pull ...; docker tag ...; docker push ... syntax is the easy way to move images between registries, it has a couple drawbacks. First, as you've seen, is that it dereferences a multi-platform image to a single platform. And the second is that it pulls all layers to the docker engine even if the remote registry already has those layers, making it a bad method for ephemeral CI workers that would always need to pull every layer.
To do this, I prefer talking directly to the registry servers rather than the docker engine itself. You don't need the functionality from the engine to run the images, all you need is the registry API. Docker has documented the original registry API and OCI recently went 1.0 on the distribution-spec which should get us some standardization.
There's a variety of tooling based on those specs, from the docker engine itself and containerd, to skopeo, google's crane, and I've also been working on regclient. Doing this with regclient's regctl command looks like:
regctl image copy <source_image:tag> <target_image:tag>
And the result is the various layers, image config, manifests, and multi-platform manifest list will be copied between registries, but only for the layers that don't already exist on the target registry.
2022 (docker builtin) solution
It's posible to perform the copy using the not-well-documented built in command docker buildx imagetools create using --tag
# i.e.
OLD_TAG=registry.example.com/namespaced/repository/example-image:old-tag
NEW_TAG=registry.example.com/namespaced/repository/example-image:new-tag
# we can
docker buildx imagetools create --tag "$NEW_TAG" "$OLD_TAG"
Reference documentation
IMPORTANT NOTE: There is no support at the moment to perform this operation against different repositories. Given tags like
OLD_TAG=registry.example.com/namespaced/repository/example-image:latest
NEW_TAG=registry.example.com/other-repository/example-image:latest
You end with an error like
error: multiple repositories currently not supported
For this situation I'm going to test the actual accepted answer
As #laconbass has written, this can be done with docker buildx imagetools create. The ability to do this over multiple repos was added in this PR
docker buildx imagetools create -t <NEW-TAG> <OLD-TAG>
I have a use case where I call docker build . on one of our build machines.
During the build, a volume mount from the host machine is used to persist intermediate artifacts.
I don't care about this image at all. I have been tagging it with -t during the build and calling docker rmi after it's been created, but I was wondering if there was a one-liner/flag that could do this.
The docker build steps don't seem to have an appropriate flag for this behavior, but it may be simply because build is the wrong term.
I am running a script to build a Docker image. I need to push this image to 2 different repositories with different tags. For which I need to capture and store the build ID of the image.
Docker build documentation talks about a way to store it in a file using --iidfile.
Do we have any examples around the same?
Thank you.
Actually, you don't need image ID. Just use docker tag existing image in your build pipeline to create other "cloned" images. Example:
docker build -t repo1/image:tag1 .
docker tag repo1/image:tag1 repo2/image:tag2
docker push repo1/image:tag1
docker push repo2/image:tag2
While building a docker image, image id is different if the image is built using docker-compose build vs docker build. The env has different hostname.
What else is different? Why images are different?
There are no differences between the actual image that gets build between docker-compose build and a "manual" docker build in terms of the contents of the image.
The difference is only in naming/tagging of the build result, which docker-compose does automatically for you.
Other than that the docker-compose build is no different behind the scenes and simply a wrapper for the normal docker build.
In addition to the naming difference, there could be difference in the resulting image if the parameters of the builds are different: the build: section of the docker-compose.yml and the command line parameters of the docker build command.
For example the --build-args can be different.