I am building docker images locally on a M1 Macbook using docker-compose and run it on google compute engine.
The third image ...flask-nginx_1 works fine - probably as docker indicates it being a amd64 build.
The next_js_1 image however crashes saying 'standard_init_linux.go:228: exec user process caused: exec format error'
Is there a way to specify this in the dockerfile to create an amd64 image?
Most Docker commands support a --platform parameter, which sets the platform for the current invocation:
docker build --platform amd64 .
If you want a Dockerfile that always builds for a specific platform, you may specify the platform for an image in your Dockerfile using FROM --platform:
FROM --platform=amd64 debian
...
This overrides any command-line platform specification you might have.
In Compose, you can also specify the platform:
services:
my-svc:
platform: linux/amd64
image: my-svc-image
It's quite possible to mix images from different platforms in the same compose file.
Related
I'm trying use MacBook M1 with docker build Go project, this project depends on lib confluent-kafka-go, thus must use below ways.
Both works, but what's the difference?
What I can see, first one use multi-platform image, second one use a special amd64/golang image.
docker run --platform linux/amd64 --rm -v $PWD:/build -w /build -e GOOS=linux -e GOARCH=amd64 golang:1.17-buster go build
docker run --rm -v "$PWD":/build -w /build -e GOOS=linux -e GOARCH=amd64 amd64/golang:1.17 go build
That would be the difference between ""Shared" and "Simple" tags":
"Simple Tags" are instances of a "single" Linux or Windows image.
It is often a manifest list that can include the same image built for other architectures; for example, mongo:4.0-xenial currently has images for amd64 and arm64v8.
The Docker daemon is responsible for picking the appropriate image for the host architecture.
"Shared Tags" are tags that always point to a manifest list which includes some combination of potentially multiple versions of Windows and Linux images across all their respective images' architectures -- in the mongo example, the 4.0 tag is a shared tag consisting of (at the time of this writing) all of 4.0-xenial, 4.0-windowsservercore-ltsc2016, 4.0-windowsservercore-1709, and 4.0-windowsservercore-1803.
So:
The "Simple Tags" enable docker run mongo:4.0-xenial to "do the right thing" across architectures on a single platform (Linux in the case of mongo:4.0-xenial).
The "Shared Tags" enable docker run mongo:4.0 to roughly work on both Linux and as many of the various versions of Windows that are supported (such as Windows Server Core LTSC 2016, where the Docker daemon is again responsible for determining the appropriate image based on the host platform and version).
In your case, the Docker Golang image offers both tags
golang:1.17-buster is a shared tag, from which you are extracting the linux/amd64 through the --platform option.
If you did not specify the platform, would get a warning like:
The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
amd64/golang:1.17 is a simple tag, already tailored to linux/amd64
When loading an image from a docker hub in dockerFile command FROM, if the platform is not specified, is it loaded based on the current server architecture?
I mean, when I build dockerfile on amazon linux 2 x86, is docker engine get docker image which can running on linux/x86-64?
Thank you!!
The docker build defaults to pulling base images with the platform of the target platform you are building. And that target platform defaults to the platform of your docker host. So if your docker engine is running on linux/amd64, and you do not pass --platform to either docker build or the FROM line, it will attempt to pull linux/amd64 images from a multi platform base image.
You can check your docker host by running:
docker system info --format '{{.OSType}}/{{.Architecture}}'
I am running a Jenkins Alpine Linux AMD64 docker image, which I'm building myself and want to add linux/arm64 docker buildx support to it, in order to generate multi-platform images and I do not know how it supposed to work.
When I check the supported platform I get:
+ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
default * docker
default default running linux/amd64, linux/386
since I'm within an AMD64 image. I read that I need to install qemu for this, but I have no clue how buildx will recognize that.
The documentation is relatively bare on this at: https://docs.docker.com/buildx/working-with-buildx/
Anyone an idea how to add linux/arm64 build capability within a linux/amd64 image?
The only solution I see right now is to build an image on an actual arm64 system.
To use buildx, make sure your Docker runtime is at least version 19.03. buildx actually comes bundled with Docker by default, but needs to be enabled by setting the environment variable DOCKER_CLI_EXPERIMENTAL.
export DOCKER_CLI_EXPERIMENTAL=enabled
If you're on Linux, you need to set up binfmt_misc. This is pretty easy in most distributions, but is even easier now that you can just run a privileged Docker container to set it up for you.
docker run --rm --privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d
Create a new builder which gives access to the new multi-architecture features:
docker buildx create --use --name multi-arch-builder
Then you'll be able to build the containers with:
docker buildx build --platform=[your target platform] ...
This is the setup I use on my Jenkins pipeline.
Relevant documentation:
https://docs.docker.com/desktop/multi-arch/
In-depth tutorial: https://medium.com/#artur.klauser/building-multi-architecture-docker-images-with-buildx-27d80f7e2408
Why are some docker images incompatible with platforms like Raspberry Pi (linux/arm/v7)?
Furthermore, can you modify a Dockerfile or another config file such that it is compatible?
Thanks for any suggestions!
So far I've installed docker and docker-compose then followed the puckel/docker-airflow readme, skipping the optional build, then tried to run the container by:
docker run -d -p 8080:8080 puckel/docker-airflow webserver
got this warning:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
found this issue and ran:
docker run -d -p 8080:8080 --platform linux/arm/v7 puckel/docker-airflow:latest webserver
then, this error:
docker: Error response from daemon: image with reference puckel/docker-airflow:latest was found but does not match the specified platform: wanted linux/arm/v7, actual: linux/amd64.
See 'docker run --help'.
Executable files, i.e. binary files, are dependent on the computer's architecture (amd64, arm...). Docker's image contains binary files. That is, the docker image is computer architecture dependent.
Therefore, if you look at the registry of docker, the OS and Architecture of the image are specified. Refer to the dockerhub/puckel/docker-airflow you used, linux/amd64 You can see it only supports. In other words, it doesn't work in arm architecture.
If you want to run this arm architecture there will be several ways, but the point is one. It is to build the result of building the origin code with arm, not amd64, as docker image.
In github.com/puckel/docker-airflow, guidelines for building are well stated.
First, if you look at the Dockerfile provided by the github, it starts from the image FROM python:3.7-slim-buster. For the corresponding python:3.7-slim-buster, it supports linux/arm/v5, linux/arm/v7, linux/arm/v5, linux/arm64/v8. dockerhub/python/3.7-slim-buster
In other words, you can build to arm architecture
I have experience creating images for multiple architectures through the docker buildx command. Of course, other methods exist, but I will only briefly introduce the commands below.
dockerhub/buildx
docker buildx is an experimental feature, and it is still recommended Experimental features must not be used in production environments.
docker buildx build --platform linux/arm/v5,linux/arm/v7 .
I'm working on a dockerfile.
I just realised that I've been using FROM with indexed images all along.
So I wonder:
How can I use one of my local (custom) images as my base (FROM) image without pushing it to the index?
You can use it without doing anything special. If you have a local image called blah you can do FROM blah. If you do FROM blah in your Dockerfile, but don't have a local image called blah, then Docker will try to pull it from the registry.
In other words, if a Dockerfile does FROM ubuntu, but you have a local image called ubuntu different from the official one, your image will override it.
For anyone who faces this issue in the future, where you have the image in your local but docker build still tries to pull the image from docker hub, the problem might be that the architecture types are different.
You can check the architecture of the image using
docker inspect --format='{{.Os}}/{{.Architecture}}' IMAGE_NAME
Now in your Dockerfile change FROM IMAGE_NAME to something like FROM --platform=linux/amd64 IMAGE_NAME and docker would now use the local image.
Verified: it works well in Docker 1.7.0.
Don't specify --pull=true when running the docker build command
From this thread on reference locally-built image using FROM at dockerfile:
If you want use the local image as the base image, pass without the
option --pull=true
--pull=true will always attempt to pull a newer
version of the image.
You can have - characters in your images. Assume you have a local image (not a local registry) named centos-base-image with tag 7.3.1611.
docker version
Client:
Version: 1.12.6
API version: 1.24
Package version: docker-common-1.12.6-16.el7.centos.x86_64
Go version: go1.7.4
Server:
Version: 1.12.6
API version: 1.24
Package version: docker-common-1.12.6-16.el7.centos.x86_64
Go version: go1.7.4
docker images
REPOSITORY TAG
centos-base-image 7.3.1611
Dockerfile
FROM centos-base-image:7.3.1611
RUN yum -y install epel-release libaio bc flex
Result
Sending build context to Docker daemon 315.9 MB
Step 1 : FROM centos-base-image:7.3.1611
---> c4d84e86782e
Step 2 : RUN yum -y install epel-release libaio bc flex
---> Running in 36d8abd0dad9
...
In the example above FROM is fetching your local image, you can provide additional instructions to fetch an image from your custom registry (e.g. FROM localhost:5000/my-image:with.tag). See https://docs.docker.com/engine/reference/commandline/pull/#pull-from-a-different-registry and https://docs.docker.com/registry/#tldr
Finally, if your image is not being resolved when providing a name, try adding a tag to the image when you create it
This GitHub thread describes a similar issue of not finding local images by name.
By omitting a specific tag, docker will look for an image tagged "latest", so either create an image with the :latest tag, or change your FROM
Remember to put not only the tag but also the repository in which that tag is, this way:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
elixir 1.7-centos7_3 e15e6bf57262 20 hours ago 925MB
You should reference it this way:
elixir:1.7-centos7_3
I had to disable BUILDKIT on a mac with M1 to be able to use a local image.
You can do it by setting the DOCKER_BUILDKIT environment variable to 0.
DOCKER_BUILDKIT=0 docker build -t YOUR_TAG --pull=false .
For users with M1 chips, problem can be arisen when platform differs of local image differs from target Dockerfile. In my case, I've built base image with M1 support and then tried to use command FROM in image that was building with platform linux/amd64
Our /etc/docker/daemon.json had a line declaring
"disable-legacy-registry" : true,
With that line in, the local registry was refusing access.
With it removed, it's working.
I just ran into this on an M1 Mac with engine version 20.10.14. It wasn't obvious from docker build --help, but passing --pull=false worked for me.