Why are some docker images incompatible with platforms like Raspberry Pi (linux/arm/v7)?
Furthermore, can you modify a Dockerfile or another config file such that it is compatible?
Thanks for any suggestions!
So far I've installed docker and docker-compose then followed the puckel/docker-airflow readme, skipping the optional build, then tried to run the container by:
docker run -d -p 8080:8080 puckel/docker-airflow webserver
got this warning:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
found this issue and ran:
docker run -d -p 8080:8080 --platform linux/arm/v7 puckel/docker-airflow:latest webserver
then, this error:
docker: Error response from daemon: image with reference puckel/docker-airflow:latest was found but does not match the specified platform: wanted linux/arm/v7, actual: linux/amd64.
See 'docker run --help'.
Executable files, i.e. binary files, are dependent on the computer's architecture (amd64, arm...). Docker's image contains binary files. That is, the docker image is computer architecture dependent.
Therefore, if you look at the registry of docker, the OS and Architecture of the image are specified. Refer to the dockerhub/puckel/docker-airflow you used, linux/amd64 You can see it only supports. In other words, it doesn't work in arm architecture.
If you want to run this arm architecture there will be several ways, but the point is one. It is to build the result of building the origin code with arm, not amd64, as docker image.
In github.com/puckel/docker-airflow, guidelines for building are well stated.
First, if you look at the Dockerfile provided by the github, it starts from the image FROM python:3.7-slim-buster. For the corresponding python:3.7-slim-buster, it supports linux/arm/v5, linux/arm/v7, linux/arm/v5, linux/arm64/v8. dockerhub/python/3.7-slim-buster
In other words, you can build to arm architecture
I have experience creating images for multiple architectures through the docker buildx command. Of course, other methods exist, but I will only briefly introduce the commands below.
dockerhub/buildx
docker buildx is an experimental feature, and it is still recommended Experimental features must not be used in production environments.
docker buildx build --platform linux/arm/v5,linux/arm/v7 .
Related
I'm trying use MacBook M1 with docker build Go project, this project depends on lib confluent-kafka-go, thus must use below ways.
Both works, but what's the difference?
What I can see, first one use multi-platform image, second one use a special amd64/golang image.
docker run --platform linux/amd64 --rm -v $PWD:/build -w /build -e GOOS=linux -e GOARCH=amd64 golang:1.17-buster go build
docker run --rm -v "$PWD":/build -w /build -e GOOS=linux -e GOARCH=amd64 amd64/golang:1.17 go build
That would be the difference between ""Shared" and "Simple" tags":
"Simple Tags" are instances of a "single" Linux or Windows image.
It is often a manifest list that can include the same image built for other architectures; for example, mongo:4.0-xenial currently has images for amd64 and arm64v8.
The Docker daemon is responsible for picking the appropriate image for the host architecture.
"Shared Tags" are tags that always point to a manifest list which includes some combination of potentially multiple versions of Windows and Linux images across all their respective images' architectures -- in the mongo example, the 4.0 tag is a shared tag consisting of (at the time of this writing) all of 4.0-xenial, 4.0-windowsservercore-ltsc2016, 4.0-windowsservercore-1709, and 4.0-windowsservercore-1803.
So:
The "Simple Tags" enable docker run mongo:4.0-xenial to "do the right thing" across architectures on a single platform (Linux in the case of mongo:4.0-xenial).
The "Shared Tags" enable docker run mongo:4.0 to roughly work on both Linux and as many of the various versions of Windows that are supported (such as Windows Server Core LTSC 2016, where the Docker daemon is again responsible for determining the appropriate image based on the host platform and version).
In your case, the Docker Golang image offers both tags
golang:1.17-buster is a shared tag, from which you are extracting the linux/amd64 through the --platform option.
If you did not specify the platform, would get a warning like:
The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
amd64/golang:1.17 is a simple tag, already tailored to linux/amd64
Just got done compiling a buildx multi-arch docker container that includes amd64, arm64 and armv7. It took about two hours to build -- with the armv7 portion being particularly slow. It only takes about 15 minutes to build natively on a RaspberryPi3.
I've setup a Docker swarm with 4 nodes, including each of the architectures I want to build for, but I haven't been able to work out how to setup the buildx cluster to use these native nodes.
I'd appreciate some guidance from anyone that's been down this particular road!
Edit:
I've made some progress with this, and here's what I know so far:
buildx clusters have nothing to do with docker swarm -- it's a separate thing
ssh public key authentication needs to be setup between the ssh "client" and "server" with your platform-specific target build machines being the servers in this case, and here's a good link for doing that:
https://www.ssh.com/academy/ssh/copy-id
test that ssh authentication is working from the machine initiating your cluster, with:
docker -H ssh://username#hostname info
you should see docker client and server info returned
create a new builder on the machine you'll be running the buildx from in the form:
docker buildx create --name <your_cluster_name> --node <your_node_name> --platform <your_desired_build_platforms> --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=10000000 --driver-opt env.BUILDKIT_STEP_LOG_MAX_SPEED=10000000
create additional target build platforms just like the above, but add --append to the options
docker buildx ls
is used to confirm the builder, nodes and platforms (the ones with an * should be what you specified and will override defaults.
docker buildx use <your_cluster_name>
will change the default builder
docker buildx inspect --bootstrap
will initiate your cluster multi-arch multi-platform builder, including downloading the buildkit image on each platform.
I had issues here, with one of my platforms requiring multiple attempts with docker buildx inspect --bootstrap. Ultimately, for my modest armv7 machine, I downloaded the:
moby/buildkit:buildx-stable-1
image on that machine first, and then ran buildx inspect from the machine I'm using to initiate the build. With the image already present, the container was spun-up without further issue.
It all looks to be setup correctly:
bnhf_cluster * docker-container
bnhf_cluster0 unix:///var/run/docker.sock running linux/amd64*, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
rpi6 ssh://pi#raspberrypi6 running linux/armv7*, linux/arm/v7
rpi5 ssh://pi#raspberrypi5 running linux/arm64*, linux/arm/v7, linux/arm/v6
Next, I'll find something new to try out a mutli-arch multi-platform build, as my primary project is all cached by buildx now -- so wouldn't be a good test.
I found this post to be the most helpful in getting multi-arch multi-platform buildx configured:
https://medium.com/#spurin/using-docker-and-multiple-buildx-nodes-for-simultaneous-cross-platform-builds-cee0f797d939
If you use portainer, you should see something like this in your container list on each of your platform specific targets, after the cluster is created:
Edit #2:
Wow, did that work well! I decided to build my primary application with the --no-cache option. The same build that took over 2 hours (with some layers already cached) on an 8th Gen i7 (4 cores, 8 threads, 32GB RAM) running Debian 11 in a WSL2 instance, took 742 seconds.
Sending the arm64 and arm/v7 jobs even to lowly Raspberry Pis is a huge improvement over the emulation offered with Docker Desktop.
I am running a Jenkins Alpine Linux AMD64 docker image, which I'm building myself and want to add linux/arm64 docker buildx support to it, in order to generate multi-platform images and I do not know how it supposed to work.
When I check the supported platform I get:
+ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
default * docker
default default running linux/amd64, linux/386
since I'm within an AMD64 image. I read that I need to install qemu for this, but I have no clue how buildx will recognize that.
The documentation is relatively bare on this at: https://docs.docker.com/buildx/working-with-buildx/
Anyone an idea how to add linux/arm64 build capability within a linux/amd64 image?
The only solution I see right now is to build an image on an actual arm64 system.
To use buildx, make sure your Docker runtime is at least version 19.03. buildx actually comes bundled with Docker by default, but needs to be enabled by setting the environment variable DOCKER_CLI_EXPERIMENTAL.
export DOCKER_CLI_EXPERIMENTAL=enabled
If you're on Linux, you need to set up binfmt_misc. This is pretty easy in most distributions, but is even easier now that you can just run a privileged Docker container to set it up for you.
docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d
Create a new builder which gives access to the new multi-architecture features:
docker buildx create --use --name multi-arch-builder
Then you'll be able to build the containers with:
docker buildx build --platform=[your target platform] ...
This is the setup I use on my Jenkins pipeline.
Relevant documentation:
https://docs.docker.com/desktop/multi-arch/
In-depth tutorial: https://medium.com/#artur.klauser/building-multi-architecture-docker-images-with-buildx-27d80f7e2408
I want to build Docker image for AMD and ARM Graviton2 processors. I already know about multi-arch CLI command docker buildx build --platform linux/amd64,linux/arm64, manifests and the fact that Docker will pull the right image variant matching architecture.
I wonder if I have to use in my Dockerfile for ARM as a parent arm64v8/ubuntu:20.04 or it's fine to use ubuntu:20.04 for both? Will it work the same way on both architectures? What's the purpose of this official arm64v8 dockerhub repo?
There is a significant difference in build times - 5min with FROM ubuntu:20.04 vs 30min with FROM arm64v8/ubuntu:20.04.
Ok so I figured it out that this ubuntu:20.04 and this arm64v8/ubuntu:20.04 two images has exactly the same SHA. So Ubuntu:20.04 is only the parent of all these per-arch images and if you run docker manifest inspect ubuntu you will see it all.
So it's clear that arm64v8/ubuntu:20.04 repo is only for the case you want to build ARM image on different architecture (if you don't want to use multibuild buildx command). It that case you have to start writing your Dockerfile with FROM arm64v8/ubuntu:20.04.
I have a Dockerfile that pulls FROM hivemq/hivemq-ce. This works well on "standard" platforms but not on the Raspberry Pi. So I built the image for arm64 myself directly on the RasPi following the tutorial in the official HiveMQ repo and pushed it to my private docker registry. The Dockerfile works well on RasPi if I change the FROM line to FROM my-private-registry/hivemq-ce.
So now I have images that work on different platforms in different sources. But how can I make my Dockerfile work on all platforms? Is there any way to pull from different sources for different architectures?
As outlined here docker supports multiple cpu architectures and will select the correct image for the correct platform. So you could build a non arm64 image for frederikheld/hivemq-ce and push it to the same location without affecting the arm64 image.
You should be able to run docker manifest inspect frederikheld/hivemq-ce to see the available architectures for a given image.
I went with this approach:
start.sh:
...
if [ "$(uname -m)" = "aarch64" ]; then
docker-compose -f docker-compose.aarch64.yml up -d --build --force-recreate
else
docker-compose up -d --build --force-recreate
fi
...
This requires one standard docker-compose.yml and additional docker-compose.<architecture>.yml for each architecture that has different needs.
It's not great, but it works in my environment.
I'm still open for better solutions though!