I am building an image for a docker container running on a different architecture. As I don't have internet access all the time, I usually just pull the image when I have internet and docker uses the local image instead of pulling a new one. After I started to build the image with buildx, this does not seem to work anymore. Is there any way to tell docker to only use the local image? When I have connection, docker seems to check wherever there is a new version available but uses the local (or cached) image as I would expect it without internet connection.
$ docker image ls
ros galactic bac817d14f26 5 weeks ago 626MB
$ docker image inspect ros:galactic
...
"Architecture": "arm64",
"Variant": "v8",
"Os": "linux",
...
Example build command
$ docker buildx build . --platform linux/arm64
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 0.3s (3/3) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 72B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/ros:galactic 0.0s
------
> [internal] load metadata for docker.io/library/ros:galactic:
------
Dockerfile:1
--------------------
1 | >>> FROM ros:galactic
2 | RUN "echo hello"
3 |
--------------------
error: failed to solve: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fros%3Apull&service=registry.docker.io": proxyconnect tcp: dial tcp 127.0.0.1:3333: connect: connection refused
My workaround for this is to explicitly state the registry in the Dockerfile FROM sections, be it your own private registry or dockerhub.
For example, to use the dockerhub ubuntu:latest image, instead of just doing FROM ubuntu:latest I would write in the Dockerfile:
FROM docker.io/library/ubuntu:latest
To use myprivateregistry:5000 I would use:
FROM myprivateregistry:5000/ubuntu:latest
And also you must set --pull=false flag for the docker buildx build or DOCKER_BUILDKIT=1 docker build commands. When you have internet again you can use --pull=true again
Related
I have been using several locally built docker images that I am trying to migrate to building with docker buildx. Essentially I have a local container to build something from source, and then a prod container that references the local build container.
For example, I have two Dockerfiles in a directory, Dockerfile.builder and Dockerfile.prod
# Dockerfile.builder
FROM maven:3-eclipse-temurin-17
ARG VERSION
# clone git repository, do building things
# Dockerfile.prod
ARG BUILDER_TAG
FROM builder:$BUILDER_TAG as builder
# pull in build artifacts from builder container, do other things
Then from that working directory I would build the containers like so:
docker build --no-cache --build-arg VERSION=$BUILD_VERSION -t builder-container:${BUILD_VERSION} -f Dockerfile.builder .
docker build --no-cache --build-arg BUILDER_TAG=$BUILD_VERSION -t prod-container:${BUILD_VERSION} -f Dockerfile.prod .
I'm trying to adapt this to docker buildx but am struggling with the extra overhead and complexity.
I think this would be the closest to what I'm wanting to do:
docker buildx build --no-cache --build-arg VERSION=$BUILD_VERSION -t builder:${BUILD_VERSION} - < Dockerfile.builder
However, when I try that, I get the following:
[+] Building 4.3s (2/2) FINISHED
=> ERROR [internal] load .dockerignore 4.0s
=> => transferring context: 0.0s
=> ERROR [internal] load build definition from Dockerfile 4.3s
=> => transferring dockerfile: 30B 0.0s
------
> [internal] load .dockerignore:
------
------
> [internal] load build definition from Dockerfile:
------
ERROR: failed to solve: failed to read dockerfile: failed to remove: /var/lib/docker/zfs/graph/hiqgpytehhglat0nn1a06dop1/.zfs: unlinkat /var/lib/docker/zfs/graph/hiqgpytehhglat0nn1a06dop1/.zfs/snapshot: operation not permitted
So that is telling me that it's not reading from my Dockerfile that I'm trying to supply via STDIN, and the path /var/lib/docker/zfs/graph/hiqgpytehhglat0nn1a06dop1/.zfs/snapshot doesn't exist.
Am I invoking docker buildx correctly for my use case?
Do I need to start with a fresh graph directory in order to start building my own images with buildx, or is there something I need to do with docker buildx create first?
I'm finding Docker's documentation on buildx very lacking in terms of how it differs conceptually from the legacy docker build, and I think that's part of my problem.
buildx config:
docker buildx inspect
Name: default
Driver: docker
Last Activity: 2023-02-15 03:27:29 +0000 UTC
Nodes:
Name: default
Endpoint: default
Status: running
Buildkit: 23.0.1
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
The following commands do not show the output ubuntu1 image:
docker buildx build -f 1.dockerfile -t ubuntu1 .
docker image ls | grep ubuntu1
# no output
1.dockerfile:
FROM ubuntu:latest
RUN echo "my ubuntu"
Plus, I cannot use the image in FROM statements in other docker files (both builds are on my local Windows box):
2.dockerfile:
FROM ubuntu1
RUN echo "my ubuntu 2"
docker buildx build -f 2.dockerfile -t ubuntu2 .
#error:
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
[+] Building 1.8s (4/4) FINISHED
=> [internal] load build definition from 2.dockerfile 0.0s
=> => transferring dockerfile: 84B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/ubuntu1:latest 1.8s
=> [auth] library/ubuntu1:pull token for registry-1.docker.io 0.0s
------
> [internal] load metadata for docker.io/library/ubuntu1:latest:
------
2.dockerfile:1
--------------------
1 | >>> FROM ubuntu1:latest
2 | RUN echo "my ubuntu 2"
3 |
--------------------
error: failed to solve: ubuntu1:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed (did you mean ubuntu:latest?)
Any idea what's going on? How can I see what buildx prepared and reference one image in another dockerfile?
Ok found a partial solution, I need to add --output type=docker as per the docs. This puts the docker in the image list. But I still cannot use it in the second docker.
Total docker newbie here and I would appreciate any help I could get. I pulled an image from my ECR repository and tagged it as app:latest using this command:
docker tag xxxxxxxxxxxx.dkr.ecr.us-east-2.amazonaws.com/app app:latest. When I list my imaged with docker images, the image is there with the new tag.
REPOSITORY TAG IMAGE ID CREATED SIZE
xxxxxxxxxxxx.dkr.ecr.us-east-2.amazonaws.com/app latest b5c8c2b74272 4 weeks ago 660MB
app latest b5c8c2b74272 4 weeks ago 660MB
I want to use this app:latest image as the base image in my Dockerfile. I know docker's default behavior is to check locally for the image and pull from dockerhub if it's not stored locally. When I run docker build -t hello ., I get this error:
[+] Building 1.3s (4/4) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/app:latest 1.2s
=> [auth] library/app:pull token for registry-1.docker.io 0.0s
------
> [internal] load metadata for docker.io/library/app:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
Why is docker trying to pull from dockerhub when the app:latest image exists locally? Any insights would be greatly appreciated. Thank you!
I think this issue is related to me using an M1 computer. I ran these commands and I was able to successfully build my docker image from my Dockerfile
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0
I am tring to create a Dockerfile to run CentOS. One of the host systems needing to run this container is an ARM based (m1) Mac. These are the two files I have created so far.
# Dockerfile
FROM centos:7
# docker-compose.yml
version: "3.9"
services:
genesis:
build: .
When trying to run/build this cointainer I get the following error
Building genesis
[+] Building 0.9s (4/4) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 77B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/centos:7 0.8s
=> [auth] library/centos:pull token for registry-1.docker.io 0.0s
------
> [internal] load metadata for docker.io/library/centos:7:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to authorize: rpc error: code = Unknown desc = failed to fetch oauth token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fcentos%3Apull&service=registry.docker.io": read tcp 192.168.1.209:64469->3.228.155.36:443: read: software caused connection abort
ERROR: Service 'genesis' failed to build : Build failed
After some google searching and answers on stack overflow it looks like the issue is something to do with the architecture difference between the containers and the host. I have tried setting the dockerfile to
FROM --platform=aarch/arm centos:7
and
FROM --platform=linux/amd64 centos:7
but neither of these work, and they're returning the same error as before. I have also tried specifying the platform in the docker-compose file too but that didn't work either.
Interestingly I did seem to have it working when I use the command in the shell
$ docker run --rm -it --platform=linux/amd64 centos:7 sh
but I need to have it working in the dockerfile as I need to then have more setup in the dockefile
Docker isn't a virtual machine, so there are some limitations.
Your application is for Linux, but you appear to be trying to run it from macOS?
Does your FROM lines specify a version of Linux?
If so, you need a build a new container of binaries native to macOS and avoid using Linux containers in your FROM lines.
I am trying to cross-compile a rust app to run on my raspberry pi cluster. I saw buildx from docker was supposed to be able to make this possible. I have a minimal dockerfile right now, it is as follows:
FROM rust
RUN apt-get update
ENTRYPOINT ["echo", "hello world"]
I try to compile this by running the command: docker buildx build --platform=linux/arm/v7 some/repo:tag .
when I do I get the following error:
[+] Building 0.9s (5/5) FINISHED
=> [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 102B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/rust:latest 0.7s => CACHED [1/2] FROM docker.io/library/rust#sha256:65e254fff15478af71d342706b1e73b26fd883f3432813c129665a97a74e2278
0.0s => ERROR [2/2] RUN apt-get update 0.2s
------
> [2/2] RUN apt-get update:
#5 0.191 standard_init_linux.go:219: exec user process caused: exec format error
------ error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c apt-get update]: exit code: 1
I feel like I'm missing something pretty basic here, hoping for someone to tell me why such a simple thing isn't working for me.
I am running docker version 20.10.1 on a Ubuntu OS
Thanks in advance!
output of docker buildx inspect --bootstrap:
Name: default
Driver: docker
Nodes:
Name: default
Endpoint: default
Status: running
Platforms: linux/amd64, linux/386
output of ls -l /proc/sys/fs/binfmt_misc/:
total 0
--w------- 1 root root 0 Dec 19 07:29 register
-rw-r--r-- 1 root root 0 Dec 19 07:29 status
To cross-compile requires qemu-user-static and binfmt-support.
$ sudo apt install -y qemu-user-static binfmt-support
qemu-user-static for user mode emulation of QEMU, and binfmt_misc for switching to QEMU when read other executable binary. Then, tell docker to use them.
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
You must be afraid to run unknown image as privileged, but the content is safe. Next, create a user in docker for building images.
$ docker buildx create --name sofia # name as you like
$ docker buildx use sofia
$ docker buildx inspect --bootstrap
If you success, buildkit will be pulled:
[+] Building 9.4s (1/1) FINISHED
=> [internal] booting buildkit 9.4s
=> => pulling image moby/buildkit:buildx-stable-1 8.7s
=> => creating container buildx_buildkit_sofia0 0.7s
Name: sofia
Driver: docker-container
Nodes:
Name: sofia0
Endpoint: unix:///var/run/docker.sock
Status: running
Platforms: linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
Available targets expand!
Reference:
Building Multi-Architecture Docker Images With Buildx | by Artur Klauser | Medium