docker cross-compilation - missing platforms in buildx - docker

i'm trying to build a Dockerfile on Centos7 x86_64. the docker image should run on arm64 linux machine.
in order to prepare my machine for the cross-compilation i checked:
>docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
default * docker
default default running linux/amd64, linux/386
i am trying to add linux/arm64 to the platform list but nothing works
i followed: https://docs.docker.com/build/buildx/install/
i downloaded from arm https://github.com/docker/buildx/releases from to /usr/libexec/docker/cli-plugins/docker-buildx
the consequence was that i could run "docker buildx ls" at all. so i revered it.
i set docker to experimental true
i work on docker version 20.10.18
i gave full permission to /usr/libexec/docker/cli-plugins/docker-buildx
i called
docker run --privileged --rm tonistiigi/binfmt --install all
the result is:
installing: arm64 cannot register "/usr/bin/qemu-aarch64" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: arm cannot register "/usr/bin/qemu-arm" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: riscv64 cannot register "/usr/bin/qemu-riscv64" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: mips64le cannot register "/usr/bin/qemu-mips64el" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: mips64 cannot register "/usr/bin/qemu-mips64" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: s390x cannot register "/usr/bin/qemu-s390x" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument installing: ppc64le cannot register "/usr/bin/qemu-ppc64le" to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument { "supported": [ "linux/amd64", "linux/386" ], "emulators": [ "jexec", "kshcomp" ] }
After all that, it seems that with Docker Desktop the missing platforms appears.
Can i add arm64 without Docker Desktop?
BTW, i tried building Dockerfile like this:
FROM ubuntu
ARG TARGETARCH=arm64
ARG CPUARCH=aarch64
RUN mkdir -p config/cmap
and i ran the build command with this:
docker build --platform linux/arm64 .
it failed with this message:
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM ubuntu
latest: Pulling from library/ubuntu
00f50047d606: Already exists
Digest: sha256:20fa2d7bb4de7723f542be5923b06c4d704370f0390e4ae9e1c833c8785644c1
Status: Downloaded newer image for ubuntu:latest
---> 21735dab04ba
Step 2/4 : ARG TARGETARCH=arm64
---> [Warning] The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
---> Running in 7de7701f6b3a
Removing intermediate container 7de7701f6b3a
---> b4438f9c8791
Step 3/4 : ARG CPUARCH=aarch64
---> [Warning] The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
---> Running in b3aa3d8c24f7
Removing intermediate container b3aa3d8c24f7
---> 5be105ca2646
Step 4/4 : RUN mkdir -p config/cmap
---> [Warning] The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
---> Running in 4be5019859f8
standard_init_linux.go:228: exec user process caused: exec format error
The command '/bin/sh -c mkdir -p config/cmap' returned a non-zero code: 1
it seems that unless there's a way to set linux/arm64 in buildx platform no cross compilation of docker build could work

Kernel version should be newer than 4.8.
Centos 7 with kernel 3.10 doesn't support cross compilation on docker images

Related

WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) [duplicate]

I'm starting the "get-started" guide from official Docker website. At the Part 4 "Share the application", I'm facing this error message when I try to run my image on the docker hub from play-with-docker.com.
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
I built the image from my apple M1 laptop:
FROM node:12-alpine
# Adding build tools to make yarn install work on Apple silicon / arm64 machines
RUN apk add --no-cache python2 g++ make
WORKDIR /app
COPY . .
RUN yarn install --producti
CMD ["node", "src/index.js"]
If you want to run the image on a linux/amd64 platform, you need to build it for that platform. You can do that with docker buildx like this and specify both your platforms
docker buildx build --platform linux/amd64,linux/arm64 -t <tag> .
I had the same error as srevinu as I was also using the tutorial which points to using the docker playground.
This sequence will build and push to docker hub so that it can be run on docker playground.
On your osx arm based computer docker buildx build --platform linux/amd64,linux/arm64 -t <YOUR_DOCKERHUB_ID/getting-started --push .
(If it gives an error and suggestion about issuing a docker buildx create --use, enter the command verbatim.)
After this command in tags pane on docker hub, you should see two platforms listed for the image. One image for linux/amd64 and one for linux/arm64.
On docker playground in the instance, docker run -dp 3000:3000 --platform linux/amd64 johndavis940/getting-started
The image will run and the port icon will be functional.

How to use combination of ARG and ENV in Dockerfile to prevent unsupported modifier in substitution?

I am trying to use an example from Docker docs on ARG
FROM ubuntu
ARG TEST
ENV TEST=${TEST:foo}
RUN echo $TEST
Docker docs says:
The variable expansion technique in this example allows you to pass arguments from the
command line and persist them in the final image by leveraging the ENV instruction.
The build fails with the following error:
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM ubuntu
latest: Pulling from library/ubuntu
16ec32c2132b: Pull complete
Digest: sha256:82becede498899ec668628e7cb0ad87b6e1c371cb8a1e597d83a47fac21d6af3
Status: Downloaded newer image for ubuntu:latest
---> 1318b700e415
Step 2/4 : ARG TEST
---> Running in 4130b437fd93
Removing intermediate container 4130b437fd93
---> a31b0f8a81ff
Step 3/4 : ENV TEST=${TEST:foo}
failed to process "${TEST:foo}": unsupported modifier (f) in substitution
Why is this not working? The error sounds like a complete misunderstanding between me and my docker engine.
OS: Ubuntu 20.04.3 LTS
Engine: Docker version 20.10.8, build 3967b7d
You're missing -, it should be:
FROM ubuntu
ARG TEST
ENV TEST=${TEST:-foo}

Dockerfile FROM --platform option

Dockerfile documentation states that there is possibility to pass --platform option in FROM instruction like this:
FROM [--platform=<platform>] <image> [AS <name>]
In my dockerfile I have following statements:
ARG arch
FROM --platform linux/${arch} bounz/hgbe.base
where bounz/hgbe.base image has two os/arch variants: linux/amd64 and linux/arm/v7.
But trying to build an image using this dockerfile I get an error:
$ docker build -f hgbe.dockerfile --build-arg arch=amd64 -t bounz/hgbetest:amd64-0.1 .
Sending build context to Docker daemon 12.29kB
Error response from daemon: Dockerfile parse error line 2: FROM requires either one or three arguments
What am I doing wrong?
And is there a way to reference specific platform of the base image if it has been built using docker buildx?
update your docker file, you are missing =
ARG arch
FROM --platform=linux/${arch} bounz/hgbe.base

ERRO[0001] error waiting for container: context canceled

Getting error while running docker image. It seems to look like the problem is on my pc.
I'm using MacOS 10.13.6.
I have followed steps to create a docker image.
Sanjeet:server-api sanjeet$ docker build -t apicontainer .
Sending build context to Docker daemon 24.01MB
Step 1/2 : FROM alpine:3.6
---> da579b235e92
Step 2/2 : CMD ["/bin/bash"]
---> Running in f43fa95302d4
Removing intermediate container f43fa95302d4
---> 64d0b47af4df
Successfully built 64d0b47af4df
Successfully tagged apicontainer:latest
Sanjeet:server-api sanjeet$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
apicontainer latest 64d0b47af4df 3 minutes ago 4.03MB
alpine 3.6 da579b235e92 2 weeks ago 4.03MB
Sanjeet:server-api sanjeet$ docker run -it apicontainer
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown.
Sanjeet:server-api sanjeet$ ERRO[0001] error waiting for container: context canceled
Inside Dockerfile
FROM alpine:3.6
CMD ["/bin/bash"]
alpine does not include bash by default.
If you want to include bash, you should add RUN apk add --no-cache bash in your Dockerfile.
Issues while running/re-initialize Docker
ERRO[0000] error waiting for the container: context canceled
Steps to Stop the Running Docker Container Forcefully
docker ps
//copy the CONTAINER ID of the running process, ex: CONTAINER ID: 37146513b713
docker kill 37146513b713
docker rm 37146513b713
alpine does not provide glibc. alpine is that small because it uses a stripped down version of libstdc called musl.libc.org.
So we'll check statically linked dependencies using ldd command.
$ docker run -it <image name> /bin/sh
$ cd /go/bin
$ ldd scratch
Check the linked static files, do they exist on that version of alpine? If the do not from, the binaries perspective, it did not find the file-- and will report File not found.
The following step depends on what binaries were missing, you can look it up on the internet on how to install them.
Add RUN apk add --no-cache libc6-compat to your Dockerfile to add libstdc in some Golang alpine image based Dockerfiles.
In you case the solution is to either
disable CGO : use CGO_ENABLED=0 while building
or add
RUN apk add --no-cache libc6-compat
to your Dockerfile
or do not use golang:alpine
I get this error when the network doesn't exist.
docker: Error response from daemon: network xxx not found.
ERRO[0000] error waiting for container: context canceled
It's quite easy to miss the output line after a long docker command and before the red error message.

Dockerfile compile in local machine but fails in docker hub for automated build

I am trying to make an automated build docker image in docker hub by linking my github repo which contains a Dockerfile. This image is for Raspberry Pi 3(can be seen in Dockerfile). While the Dockerfile works fine in the Raspberry Pi it give error in x86_64 GNU/Linux machine and also in Docker hub it fails to compile. Here is the log from Docker hub. The final error is same in docker hub and x86_64 GNU/Linux machine.
Building in Docker Cloud's infrastructure...
Cloning into '.'...
KernelVersion: 4.4.0-79-generic
Arch: amd64
BuildTime: 2017-03-28T19:26:53.326478373+00:00
ApiVersion: 1.27
Version: 17.03.1-ee-2
MinAPIVersion: 1.12
GitCommit: ad495cb
Os: linux
GoVersion: go1.7.5
Starting build of index.docker.io/swapnil18/docker-swarm-arm-socat:latest...
Step 1/11 : FROM resin/raspberrypi3-alpine
---> ca84ecc9fee4
Step 2/11 : MAINTAINER Swapnil Das <dasswapnil96#gmail.com>
---> Running in 2a99be7f728e
---> e4c7fdd1bc5b
Removing intermediate container 2a99be7f728e
Step 3/11 : ARG "version=0.1.0-dev"
---> Running in 1019fe4333d4
---> d50c71235386
Removing intermediate container 1019fe4333d4
Step 4/11 : ARG "build_date=5/7/17"
---> Running in cbf271f27274
---> c9af4b356b3c
Removing intermediate container cbf271f27274
Step 5/11 : ARG "commit_hash=unknown"
---> Running in 1778288bf297
---> 35d52f36b527
Removing intermediate container 1778288bf297
Step 6/11 : ARG "vcs_url=unknown"
---> Running in db86bc110c3d
---> 9062830e7754
Removing intermediate container db86bc110c3d
Step 7/11 : ARG "vcs_branch=unknown"
---> Running in 919c08a0f584
---> 9532842acc1b
Removing intermediate container 919c08a0f584
Step 8/11 : LABEL org.label-schema.vendor "Personal" org.label-schema.name "Socat" org.label-schema.description "Exposes to a defined OUTPUT PORT the INPUT from a TCP endpoint." org.label-schema.usage "/src/README.md" org.label-schema.url "https://github.com/swapnil96/docker-swarm-arm-socat/blob/master/README.md" org.label-schema.vcs-url $vcs_url org.label-schema.vcs-branch $vcs_branch org.label-schema.vcs-ref $commit_hash org.label-schema.version $version org.label-schema.schema-version "1.0" org.label-schema.docker.cmd.devel "" org.label-schema.docker.params "IN=Input,OUT=Output" org.label-schema.build-date $build_date
---> Running in 641e40ae7219
---> 9c5d5da14397
Removing intermediate container 641e40ae7219
Step 9/11 : ENV "IN 172.18.0.1:4999" "OUT 4999"
---> Running in 866b338a1588
---> 3c046f12eb1e
Removing intermediate container 866b338a1588
Step 10/11 : RUN apk add --no-cache socat
---> Running in 86d0de61864f
[91mstandard_init_linux.go:178: exec user process caused "exec format error"
[0m
Removing intermediate container 86d0de61864f
The command '/bin/sh -c apk add --no-cache socat' returned a non-zero code: 1
As it can be seen in the last it fails by The command '/bin/sh -c apk add --no-cache socat' returned a non-zero code: 1. Is the Dockerfile having some architecture issue? or there is some problem with the base image which is resin/alpine. I have build other Dockerfiles for Raspberry Pi and all have compiled fine.
For working purposes I will just push the image that is built in Raspberry Pi. But having an automated build repository is much better. So please correct me if I am doing some obvious mistake.
Here is the link to my other builds in docker hub link.
PS: I have tried other options mentioned at stackoverflow. I tried to post the links of the other questions but due to less than 10 reputation I was only given to post 2 links.
This is because a Raspberry Pi runs on arm and docker hub does not support arm only x86, if you do wish to build the docker image on docker hub you will need to make edits to your docker image. I have done this before but it is pretty old but take a look at resin io's work or my own customisation based off of that over here. This method emulates arm architecture with qemu allowing the image to run natively on arm devices like a Raspberry Pi while still having the ability to be compiled by qemu on an x86 platform like docker hub.

Resources