When I want to create multiarch builds with docker, I use the command:
docker buildx build --push --platform <list of archs> -t <tag1> -t <tag2> .
This works perfectly fine, but seems to be building the images concurrently. In most situations, it may be acceptable, but in some other cases, this seems to be too heavy, requires too much RAM and causes network failures (too many parallel connections).
Is there any way to build the images sequentially?
The only solution I found is to build for each arch separately and then use "docker manifest create" to assemble:
docker buildx build --push --platform <arch1> -t <tag-arch1> .
docker buildx build --push --platform <arch2> -t <tag-arch2> .
[...]
docker manifest create ... --amend ...
docker manifest push
This may be fine, but it seems that each image is supposed to be pushed to the registry for "docker manifest create" to be able to assemble. This is not very good, as it pollutes the list of images of tags I don't want.
Would it be possible to use "docker manifest create" on local images, without the need to upload each of them separately with a tag to the registry? Is there any better way to simply build the images sequentially?
Thanks!
At present, there's no way to limit the concurrency of a single build. This is currently an open issue with buildkit. There's also a similar issue to allow setting cgroup limits that would allow you to run the builds concurrently, but with less maximum CPU time, which would have a similar effect for some use cases.
Related
I'm in low cost project that we send to container registry (DigitalOcean) only latest image.
But all time, after running:
docker build .
Is generating the same digest, every time.
This is my script for build:
docker build .
docker tag {image}:latest registry.digitalocean.com/{company}/{image}:latest;
docker push registry.digitalocean.com/{company}/{image}
I tried:
BUILD_VERSION=`date '+%s'`;
docker build -t {image}:"$NOW" -t {image}:latest .
docker tag {image}:latest registry.digitalocean.com/{company}/{image}:latest;
docker push registry.digitalocean.com/{company}/{image}
but not worked.
Editing my answer, what David said is correct - the push with out the tag should pick up latest tag.
If you provide what you have in your local repository and the output of the above commands, it would shed more light to your problem.
Edit 2:
I think I have figured out on why:
Is generating the same digest, every time.
This means, although you are running your docker build - there has been no change to the underlying artifacts which are being packaged into the image and hence it results into the same digest.
Sometimes layers are cached but there are changes that aren't detected so you can delete the image or use 'docker system prune' to force clearing cache here
Suppose I am building an image using Docker Buildkit. My image is from a multistage Dockerfile, like so:
FROM node:12 AS some-expensive-base-image
...
FROM some-expensive-base-image AS my-app
...
I am now trying to build both images. Suppose that I push these to Docker Hub. If I were to use Docker Buildkit's external caching feature, then I would want to try to save build time on my CI pipeline by pulling in the remote some-expensive-base-image:latest image as the cache when building the some-expensive-base-image target. And, I would want to pull in both the just-built some-expensive-base-image image and the remote my-app:latest image as the caches for the latter image. I believe that I need both in order to prevent requiring the steps of some-expensive-base-image from needing to be rebuilt, since...well...they are expensive.
This is what my build script looks like:
export DOCKER_BUILDKIT=1
docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from some-expensive-base-image:latest --target some-expensive-base-image -t some-expensive-base-image:edge .
docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from some-expensive-base-image:edge --cache-from my-app:latest --target my-app -t my-app:edge .
My question: Does the order of the --cache-from arguments matter for the second docker build?
I have been getting inconsistent results on my CI pipeline for this build. There are cache misses that are happening when building that latter image, even though there hasn't been any code changes that would have caused cache busting. The Cache Minefest can be pulled without issue. There are times when the cache image is pulled, but other times when all steps of that latter target need to be rerun. I don't know why.
By chance, should I instead try to docker pull both images before running the docker build commands in my script?
Also, I know that I referred to Docker Hub in my example, but in real life, my application uses AWS ECR for its remote Docker repository. Would that matter for proper Buildkit functionality?
Yes, the order of --cache-from matters!
See the explanation on Github from the person who implemented the feature, quoting here:
When using multiple --cache-from they are checked for a cache hit in the order that user specified. If one of the images produces a cache hit for a command only that image is used for the rest of the build.
I've had similar problems in the past, you might find useful to check ths answer, where I've shared about using Docker cache in the CI.
I have a use case where I call docker build . on one of our build machines.
During the build, a volume mount from the host machine is used to persist intermediate artifacts.
I don't care about this image at all. I have been tagging it with -t during the build and calling docker rmi after it's been created, but I was wondering if there was a one-liner/flag that could do this.
The docker build steps don't seem to have an appropriate flag for this behavior, but it may be simply because build is the wrong term.
Docker images can be built for multi architectures. This is usually done by creating a specific image per architecture, and then creating manifest as a proxy to the right image depending on the system that pulls the image.
That's great.
Now, with docker-compose, it's also possible to build images, but I don't see a way to build the image depending on the architecture, so it seems like the only way to have a single docker-compose definition for multiple architectures, is to have pushed multi-arch images to a registry and pull from there.
Does anyone know of a way to build local images for the right arch with the docker-compose build step?
I don't think you can use docker-compose build, but you could use docker buildx bake to build multi-arch using a docker-compose.yml.
docker buildx bake --push --set *.platform=linux/amd64,linux/arm64
You'll probably want to read more about docker buildx bake and building multi-platform images.
I recently switched to multi-stage docker builds, and it doesn't appear that there's any caching on intermediate builds. I'm not sure if this is a docker limitation, something which just isn't available or whether I'm doing something wrong.
I am pulling down the final build and doing a --cache-from at the start of the new build, but it always runs the full build.
This appears to be a limitation of docker itself and is described under this issue - https://github.com/moby/moby/issues/34715
The workaround is to:
Build the intermediate stages with a --target
Push the intermediate images to the registry
Build the final image with a --target and use multiple --cache-from paths, listing all the intermediate images and the final image
Push the final image to the registry
For subsequent builds, pull the intermediate + final images down from the registry first
Since the previous answer was posted, there is now a solution using the BuildKit backend: https://docs.docker.com/engine/reference/commandline/build/#specifying-external-cache-sources
This involves passing the argument --build-arg BUILDKIT_INLINE_CACHE=1 to your docker build command. You will also need to ensure BuildKit is being used by setting the environment variable DOCKER_BUILDKIT=1 (on Linux; I think BuildKit might be the default backend on Windows when using recent versions of Docker Desktop). A complete command line solution for CI might look something like:
export DOCKER_BUILDKIT=1
# Use cache from remote repository, tag as latest, keep cache metadata
docker build -t yourname/yourapp:latest \
--cache-from yourname/yourapp:latest \
--build-arg BUILDKIT_INLINE_CACHE=1 .
# Push new build up to remote repository replacing latest
docker push yourname/yourapp:latest
Some of the other commenters are asking about docker-compose. It works for this too, although you need to additionally specify the environment variable COMPOSE_DOCKER_CLI_BUILD=1 to ensure docker-compose uses the docker CLI (with BuildKit thanks to DOCKER_BUILDKIT=1) and then you can set BUILDKIT_INLINE_CACHE: 1 in the args: section of the build: section of your YAML file to ensure the required --build-arg is set.
I'd like to add another important point to the answer
--build-arg BUILDKIT_INLINE_CACHE=1 caches only the last layer, and works in case nothing changed.
So, to enable the caching of layers for the whole build, this argument should be replaced by --cache-to type=inline,mode=max. See the documentation