I have a Dockerfile that pulls FROM hivemq/hivemq-ce. This works well on "standard" platforms but not on the Raspberry Pi. So I built the image for arm64 myself directly on the RasPi following the tutorial in the official HiveMQ repo and pushed it to my private docker registry. The Dockerfile works well on RasPi if I change the FROM line to FROM my-private-registry/hivemq-ce.
So now I have images that work on different platforms in different sources. But how can I make my Dockerfile work on all platforms? Is there any way to pull from different sources for different architectures?
As outlined here docker supports multiple cpu architectures and will select the correct image for the correct platform. So you could build a non arm64 image for frederikheld/hivemq-ce and push it to the same location without affecting the arm64 image.
You should be able to run docker manifest inspect frederikheld/hivemq-ce to see the available architectures for a given image.
I went with this approach:
start.sh:
...
if [ "$(uname -m)" = "aarch64" ]; then
docker-compose -f docker-compose.aarch64.yml up -d --build --force-recreate
else
docker-compose up -d --build --force-recreate
fi
...
This requires one standard docker-compose.yml and additional docker-compose.<architecture>.yml for each architecture that has different needs.
It's not great, but it works in my environment.
I'm still open for better solutions though!
Related
I'm trying use MacBook M1 with docker build Go project, this project depends on lib confluent-kafka-go, thus must use below ways.
Both works, but what's the difference?
What I can see, first one use multi-platform image, second one use a special amd64/golang image.
docker run --platform linux/amd64 --rm -v $PWD:/build -w /build -e GOOS=linux -e GOARCH=amd64 golang:1.17-buster go build
docker run --rm -v "$PWD":/build -w /build -e GOOS=linux -e GOARCH=amd64 amd64/golang:1.17 go build
That would be the difference between ""Shared" and "Simple" tags":
"Simple Tags" are instances of a "single" Linux or Windows image.
It is often a manifest list that can include the same image built for other architectures; for example, mongo:4.0-xenial currently has images for amd64 and arm64v8.
The Docker daemon is responsible for picking the appropriate image for the host architecture.
"Shared Tags" are tags that always point to a manifest list which includes some combination of potentially multiple versions of Windows and Linux images across all their respective images' architectures -- in the mongo example, the 4.0 tag is a shared tag consisting of (at the time of this writing) all of 4.0-xenial, 4.0-windowsservercore-ltsc2016, 4.0-windowsservercore-1709, and 4.0-windowsservercore-1803.
So:
The "Simple Tags" enable docker run mongo:4.0-xenial to "do the right thing" across architectures on a single platform (Linux in the case of mongo:4.0-xenial).
The "Shared Tags" enable docker run mongo:4.0 to roughly work on both Linux and as many of the various versions of Windows that are supported (such as Windows Server Core LTSC 2016, where the Docker daemon is again responsible for determining the appropriate image based on the host platform and version).
In your case, the Docker Golang image offers both tags
golang:1.17-buster is a shared tag, from which you are extracting the linux/amd64 through the --platform option.
If you did not specify the platform, would get a warning like:
The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
amd64/golang:1.17 is a simple tag, already tailored to linux/amd64
I want to build Docker image for AMD and ARM Graviton2 processors. I already know about multi-arch CLI command docker buildx build --platform linux/amd64,linux/arm64, manifests and the fact that Docker will pull the right image variant matching architecture.
I wonder if I have to use in my Dockerfile for ARM as a parent arm64v8/ubuntu:20.04 or it's fine to use ubuntu:20.04 for both? Will it work the same way on both architectures? What's the purpose of this official arm64v8 dockerhub repo?
There is a significant difference in build times - 5min with FROM ubuntu:20.04 vs 30min with FROM arm64v8/ubuntu:20.04.
Ok so I figured it out that this ubuntu:20.04 and this arm64v8/ubuntu:20.04 two images has exactly the same SHA. So Ubuntu:20.04 is only the parent of all these per-arch images and if you run docker manifest inspect ubuntu you will see it all.
So it's clear that arm64v8/ubuntu:20.04 repo is only for the case you want to build ARM image on different architecture (if you don't want to use multibuild buildx command). It that case you have to start writing your Dockerfile with FROM arm64v8/ubuntu:20.04.
Why are some docker images incompatible with platforms like Raspberry Pi (linux/arm/v7)?
Furthermore, can you modify a Dockerfile or another config file such that it is compatible?
Thanks for any suggestions!
So far I've installed docker and docker-compose then followed the puckel/docker-airflow readme, skipping the optional build, then tried to run the container by:
docker run -d -p 8080:8080 puckel/docker-airflow webserver
got this warning:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
found this issue and ran:
docker run -d -p 8080:8080 --platform linux/arm/v7 puckel/docker-airflow:latest webserver
then, this error:
docker: Error response from daemon: image with reference puckel/docker-airflow:latest was found but does not match the specified platform: wanted linux/arm/v7, actual: linux/amd64.
See 'docker run --help'.
Executable files, i.e. binary files, are dependent on the computer's architecture (amd64, arm...). Docker's image contains binary files. That is, the docker image is computer architecture dependent.
Therefore, if you look at the registry of docker, the OS and Architecture of the image are specified. Refer to the dockerhub/puckel/docker-airflow you used, linux/amd64 You can see it only supports. In other words, it doesn't work in arm architecture.
If you want to run this arm architecture there will be several ways, but the point is one. It is to build the result of building the origin code with arm, not amd64, as docker image.
In github.com/puckel/docker-airflow, guidelines for building are well stated.
First, if you look at the Dockerfile provided by the github, it starts from the image FROM python:3.7-slim-buster. For the corresponding python:3.7-slim-buster, it supports linux/arm/v5, linux/arm/v7, linux/arm/v5, linux/arm64/v8. dockerhub/python/3.7-slim-buster
In other words, you can build to arm architecture
I have experience creating images for multiple architectures through the docker buildx command. Of course, other methods exist, but I will only briefly introduce the commands below.
dockerhub/buildx
docker buildx is an experimental feature, and it is still recommended Experimental features must not be used in production environments.
docker buildx build --platform linux/arm/v5,linux/arm/v7 .
Docker images can be built for multi architectures. This is usually done by creating a specific image per architecture, and then creating manifest as a proxy to the right image depending on the system that pulls the image.
That's great.
Now, with docker-compose, it's also possible to build images, but I don't see a way to build the image depending on the architecture, so it seems like the only way to have a single docker-compose definition for multiple architectures, is to have pushed multi-arch images to a registry and pull from there.
Does anyone know of a way to build local images for the right arch with the docker-compose build step?
I don't think you can use docker-compose build, but you could use docker buildx bake to build multi-arch using a docker-compose.yml.
docker buildx bake --push --set *.platform=linux/amd64,linux/arm64
You'll probably want to read more about docker buildx bake and building multi-platform images.
I'm trying to build an ARM (arm32v7) container, but using an x86_64 host. While I know there are some pretty cool things like Resin using Qemu shenanigans, and Multiarch for doing crossbuilding of generic containers, I have a slight issue: The container I'm trying to build starts off as multiarch, and so Docker always chooses the x86 image in the FROM instruction.
I want to build an ARM container from a Multi-arch Rust image on an x86 host. The problem is, I can't find any documentation to explicitly say I want to start with the ARM container and build from that, not the x86 container. Additionally, the tags on the image don't disambiguate, so I can't use those to select the starting container.
I've tried editing the /etc/docker/daemon.json file to contain:
{
"labels": [ "os=linux", "arch=arm32v7" ],
"experimental": true
}
but that hasn't helped at all. docker pull still retrieves the x86 images. The purpose of all this is to boost compile times for containers ultimately running on Raspberry Pi; compile times are super slow as it stands.
Are there any ways to explicitly say that I want to build starting with the ARM image?
It is possible to build simple Docker containers for another architecture ("cross-compile") by using an appropriate base image for that architecture. By simple, I mean images that don't need a RUN command in their Dockerfile to be built. This is because Docker doesn't have the ability to actually run commands in a container for another architecture. While this sounds restrictive, it can be quite powerful when combined with multi-stage builds to cross-compile code.
Let's walk through this step-by-step. First off, let's enable experimental mode for our Docker client to enable docker manifest by adding the following option to ~/.docker/config.json:
{
"experimental": "enabled"
}
We can then use docker manifest inspect debian:stretch to show the fat manifest that contains a digest for the image in the architecture we want to build for. For example, the arm32v7 image has "architecture": "arm" and "variant": "v7" specified under the platform key. Using jq, we can extract the digest for this image programatically:
docker manifest inspect debian:stretch | jq -r '.manifests[] | select(.platform.architecture == "arm" and .platform.variant == "v7") | .digest'`
This digest can then be used in the FROM command in a Dockerfile:
FROM debian#sha256:d01d682bdbacb520a434490018bfd86d76521c740af8d8dbd02397c3415759b1
It is then possible to COPY cross-compiled binary into the image. This binary could come from a cross-compiler on your machine or from another container in a multi-stage build. To get rid of the hard-coded digest in the Dockerfile's FROM line, it's possible to externalise it through a Docker build argument (ARG).