I want to build Docker image for AMD and ARM Graviton2 processors. I already know about multi-arch CLI command docker buildx build --platform linux/amd64,linux/arm64, manifests and the fact that Docker will pull the right image variant matching architecture.
I wonder if I have to use in my Dockerfile for ARM as a parent arm64v8/ubuntu:20.04 or it's fine to use ubuntu:20.04 for both? Will it work the same way on both architectures? What's the purpose of this official arm64v8 dockerhub repo?
There is a significant difference in build times - 5min with FROM ubuntu:20.04 vs 30min with FROM arm64v8/ubuntu:20.04.
Ok so I figured it out that this ubuntu:20.04 and this arm64v8/ubuntu:20.04 two images has exactly the same SHA. So Ubuntu:20.04 is only the parent of all these per-arch images and if you run docker manifest inspect ubuntu you will see it all.
So it's clear that arm64v8/ubuntu:20.04 repo is only for the case you want to build ARM image on different architecture (if you don't want to use multibuild buildx command). It that case you have to start writing your Dockerfile with FROM arm64v8/ubuntu:20.04.
Related
I have setup a build pipeline on an ARM device that is building a .NET Core application. The last step of the build pipeline would be to store the compiled .NET Core app in a docker image.
Is it possible to store the app in the .NET Core runtime image for X86?
My hope is that the .NET Core app does not care about the system architecture as long as the .NET framework is deployed. And that docker does not need to start the X86 image to generate the new image:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
COPY /my-application/build/ /app/
EXPOSE 80/tcp
WORKDIR /app
ENTRYPOINT ["dotnet", "app.dll"]
If I understand your question correctly you have a ARM machine running the pipeline and you want it to compile both the ARM and the x86 image?
Buildx - for cross-platform image building
Sure you can. You can use buildx to manage the cross-compiling for you. So go ahead and install buildx.
After you have setup buildx and configured it. You can just run:
docker buildx build \
--platform linux/amd64,linux/386,linux/arm/v7 \
--push \
-t docker_user/docker_image:latest \
.
Since the base image this will work for every platform you want. You can change the platforms you want to build for.
What buildx does, it emulate the target platform and execute all the steps in your regular docker file as if running on that platform. Buildx also tags the image, -t parameter. And Pushes it to the docker registry of choice, if you specify --push.
Actually it pushes an image per platform and a manifest file joining those images. If an other docker client wants to run the image, the manifest is loaded and the needed platform is selected.
In docker compiling
For this to work, you'll need to compile the image in the docker pipeline. That is recommended anyway because compiling it locally and then copying it to the container will result in different images depending on the the installed software on the machine building the image.
Follow the instructions here to created a needed dockerfile.
Requirements
For this to work the base image has to also support multiple architectures. You can check this in the docker registry. It is the case for the dotnet core images. But if your base image isn't supporting the platform it probably won't work. However recompiling the entire image should work (as long al the base image is supporting that platform).
See in action
You also have a github action for installing buildx in a github runner. I use this for several of my libraries, see this workflow file or the result here
I am trying to build the docker-compose container from source. Ultimately I want two of these containers, one for x86_64 and one for an arm arch. I see that I can get the x86 compatible container with this script https://github.com/docker/compose/releases/download/1.25.5/run.sh, but I need to be able to build from source. I have cloned the compose repository and have tried two things, first just trying to build with the Dockerfile in the repo. The result:
>docker build -t docker-compose .
Sending build context to Docker daemon 1.872 MB
Step 1/39 : ARG DOCKER_VERSION=19.03.8
Please provide a source image with `from` prior to commit
I was then looking at the build scripts in script/build and tried executing the linux build script.
> script/build/linux
+ ./script/clean
++ script/build/write-git-sha
+ DOCKER_COMPOSE_GITSHA=9c5351cf
+ TAG=docker/compose:tmp-glibc-linux-binary-9c5351cf
+ docker build -t docker/compose:tmp-glibc-linux-binary-9c5351cf . --build-arg BUILD_PLATFORM=debian --build-arg GIT_COMMIT=9c5351cf
Sending build context to Docker daemon 1.872 MB
Step 1/39 : ARG DOCKER_VERSION=19.03.8
Please provide a source image with `from` prior to commit
I'm stuck here. My plan for the arm container is to use the linux build script but swap the ARCH to my arm architecture. Any help would be appreciated
I will answer this question if anyone happens upon it, because I have solve it. The issue was with my docker-version. You need an up to date docker version, and then clone the docker-compose repo and do docker build like shown above. To build an ARM version I had to make an ARM emulator with Qemu and do the same steps there.
Docker images can be built for multi architectures. This is usually done by creating a specific image per architecture, and then creating manifest as a proxy to the right image depending on the system that pulls the image.
That's great.
Now, with docker-compose, it's also possible to build images, but I don't see a way to build the image depending on the architecture, so it seems like the only way to have a single docker-compose definition for multiple architectures, is to have pushed multi-arch images to a registry and pull from there.
Does anyone know of a way to build local images for the right arch with the docker-compose build step?
I don't think you can use docker-compose build, but you could use docker buildx bake to build multi-arch using a docker-compose.yml.
docker buildx bake --push --set *.platform=linux/amd64,linux/arm64
You'll probably want to read more about docker buildx bake and building multi-platform images.
In Docker Hub website Is it possible to build an automated build for an Arm Image ?
I have modified my existing DockerFile to use an Arm base image, but it fails on the next line when it tries to run the apk command with exec format error. So it looks like Docker Hub is trying to build as Intel despite base image being built for Arm.
Is it possible to build Arm image with Docker Hub website or not ?
If not can anyone provide succint instructions on how to build Arm Image from my Dockerfile either by
Using my (Intel PC) from the docker command line
Using my (Intel Mac) from the docker command line
Using QNAP TS131P Container station, (since this is natively Arm maybe this is simpler)
Turned out to be relatively easy using the QNAP, alot simpler than it seemed from the posts I had read, I think my confusion was these posts were about building Arm version on an Intel machine, which i didnt need to do. And all the Arm specific instructions were for Raspberry Pi which had its own problems.
Created new empty repository in DockerHub
Uploaded my DockerFile to my webserver
ssh qnapserver
docker build DockerFileUrl
docker login DockerHubUsername DockerHubPassword
docker images (to get imageId of built image)
docker tag imageId DockerHubNameSpace/DockerHubRepository:latest
docker push DockerHubNameSpace/DockerHubRepository:latest
The push worked, and I was then able to use ContainerStation to get the image from Docker Hub and run in a container.
I'm trying to build an ARM (arm32v7) container, but using an x86_64 host. While I know there are some pretty cool things like Resin using Qemu shenanigans, and Multiarch for doing crossbuilding of generic containers, I have a slight issue: The container I'm trying to build starts off as multiarch, and so Docker always chooses the x86 image in the FROM instruction.
I want to build an ARM container from a Multi-arch Rust image on an x86 host. The problem is, I can't find any documentation to explicitly say I want to start with the ARM container and build from that, not the x86 container. Additionally, the tags on the image don't disambiguate, so I can't use those to select the starting container.
I've tried editing the /etc/docker/daemon.json file to contain:
{
"labels": [ "os=linux", "arch=arm32v7" ],
"experimental": true
}
but that hasn't helped at all. docker pull still retrieves the x86 images. The purpose of all this is to boost compile times for containers ultimately running on Raspberry Pi; compile times are super slow as it stands.
Are there any ways to explicitly say that I want to build starting with the ARM image?
It is possible to build simple Docker containers for another architecture ("cross-compile") by using an appropriate base image for that architecture. By simple, I mean images that don't need a RUN command in their Dockerfile to be built. This is because Docker doesn't have the ability to actually run commands in a container for another architecture. While this sounds restrictive, it can be quite powerful when combined with multi-stage builds to cross-compile code.
Let's walk through this step-by-step. First off, let's enable experimental mode for our Docker client to enable docker manifest by adding the following option to ~/.docker/config.json:
{
"experimental": "enabled"
}
We can then use docker manifest inspect debian:stretch to show the fat manifest that contains a digest for the image in the architecture we want to build for. For example, the arm32v7 image has "architecture": "arm" and "variant": "v7" specified under the platform key. Using jq, we can extract the digest for this image programatically:
docker manifest inspect debian:stretch | jq -r '.manifests[] | select(.platform.architecture == "arm" and .platform.variant == "v7") | .digest'`
This digest can then be used in the FROM command in a Dockerfile:
FROM debian#sha256:d01d682bdbacb520a434490018bfd86d76521c740af8d8dbd02397c3415759b1
It is then possible to COPY cross-compiled binary into the image. This binary could come from a cross-compiler on your machine or from another container in a multi-stage build. To get rid of the hard-coded digest in the Dockerfile's FROM line, it's possible to externalise it through a Docker build argument (ARG).