I am trying to build a common base image, that many of my other images will use. But Nerdctl/Containerd doesn't seem to check for the image locally first.
Reproduced with the simple example
# Dockerfile.base
FROM python:3
# Dockerfile.child
From mybase:local
OUTPUT:
PS C:\<REDACTED>\local_build_issue> nerdctl build -t mybase:local -f Dockerfile.base .
[+] Building 10.1s (5/5) FINISHED
=> [internal] load build definition from Dockerfile.base 0.1s
=> => transferring dockerfile: 55B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3 0.9s
=> CACHED [1/1] FROM docker.io/library/python:3#sha256:555f5affd32250ca74758b297f262fa8f421eb0102877596b48c0b8b464606ea 0.0s
=> => resolve docker.io/library/python:3#sha256:555f5affd32250ca74758b297f262fa8f421eb0102877596b48c0b8b464606ea 0.0s
=> exporting to oci image format 9.0s
=> => exporting layers 0.0s
=> => exporting manifest sha256:6bfdb5e41f7a1f0faf28f922e48cdd33e95b75cff958709cb7945291d34dffda 0.0s
=> => exporting config sha256:d449529a9f8347e81301101d04645624c6a52d71b0c6b85abfd8da6c65b19e31 0.0s
=> => sending tarball 9.0s
unpacking docker.io/library/mybase:local (sha256:6bfdb5e41f7a1f0faf28f922e48cdd33e95b75cff958709cb7945291d34dffda)...done
PS C:\<REDACTED>\local_build_issue> nerdctl images
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
mybase local 6bfdb5e41f7a About a minute ago linux/amd64 945.8 MiB 334.7 MiB
PS C:\<REDACTED>\local_build_issue> nerdctl build -t mychild:local -f Dockerfile.child .
[+] Building 0.5s (3/3) FINISHED
=> [internal] load build definition from Dockerfile.child 0.1s
=> => transferring dockerfile: 62B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> ERROR [internal] load metadata for docker.io/library/mybase:local 0.4s
------
> [internal] load metadata for docker.io/library/mybase:local:
------
Dockerfile.child:1
--------------------
1 | >>> FROM mybase:local
2 |
--------------------
error: failed to solve: mybase:local: pull access denied, repository does not exist or may require authorization: authorization status: 401: authorization failed
FATA[0000] unrecognized image format
I can't figure out how to get it to NOT look for the image on docker.io. All the recommendations for the docker cli do not seem apply or work.
it actually can't pull image locally first .
the nerdctl build image use the buildkitd tools.it has two types of backends.
you can find the answer from https://github.com/containerd/nerdctl/blob/master/docs/build.md
Related
This is a MWE where I try to expose a file called abc within RUN as a secret:
Dockerfile:
# syntax = docker/dockerfile:1.3
FROM alpine:latest
RUN --mount=type=secret,id=aaa,target=/tmp/aaa ls -l /tmp/aaa
Then I run echo 123 > aaa (that's the file that will be exposed) and docker build (or DOCKER_BUILDKIT=1 docker build if that is not the default), but the secret is not to be found. I will be grateful for some help with this (I am running the latest Docker release 20.10.21):
[+] Building 3.1s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 157B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1.3 0.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1.3#sha256:42399d4635eddd7a9b8a24be879d2f9a930d0ed040a61324cfdf59ef1357b3b2 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 1.5s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [stage-0 1/2] FROM docker.io/library/alpine:latest#sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4 0.5s
=> => resolve docker.io/library/alpine:latest#sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4 0.0s
=> => sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4 1.64kB / 1.64kB 0.0s
=> => sha256:c0d488a800e4127c334ad20d61d7bc21b4097540327217dfab52262adc02380c 528B / 528B 0.0s
=> => sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da 1.47kB / 1.47kB 0.0s
=> => sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 3.37MB / 3.37MB 0.3s
=> => extracting sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 0.1s
=> ERROR [stage-0 2/2] RUN --mount=type=secret,id=aaa,target=/tmp/aaa ls -l /tmp/aaa 0.4s
------
> [stage-0 2/2] RUN --mount=type=secret,id=aaa,target=/tmp/aaa ls -l /tmp/aaa:
#10 0.314 ls: /tmp/aaa: No such file or directory
------
executor failed running [/bin/sh -c ls -l /tmp/aaa]: exit code: 1
Running docker-compose up works for my colleagues on this project but not for me.
techn#MacBook-Pro ~/c/o (2022prod) [1]> docker-compose up (base)
[+] Running 0/1
⠿ web Warning 8.1s
[+] Building 29.2s (11/19)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1.1-experimental 27.6s
=> CACHED docker-image://docker.io/docker/dockerfile:1.1-experimental#sha256:de85b2f3a2f7 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> ERROR [internal] load metadata for ghcr.io/hac/ubuntu-22-od:latest 0.8s
=> [auth] hac/ubuntu-22-od:pull token for ghcr.io 0.0s
=> ERROR importing cache manifest from ghcr.io/hac/od-2022:latest 0.6s
=> ERROR [builder 1/4] FROM ghcr.io/hac/ubuntu-22-od:latest 0.3s
=> => resolve ghcr.io/hac/ubuntu-22-od:latest 0.3s
=> [internal] load build context 0.0s
=> [auth] hac/od-2022:pull token for ghcr.io 0.0s
------
> [internal] load metadata for ghcr.io/hac/ubuntu-22-od:latest:
------
------
> importing cache manifest from ghcr.io/hac/od-2022:latest:
------
------
> [builder 1/4] FROM ghcr.io/hac/ubuntu-22-od:latest:
------
failed to solve: failed to load cache key: unexpected status code [manifests latest]: 403 Forbidden
Doesn't seem to be a network thing:
[68]> ping ghcr.io (base)
PING ghcr.io (140.82.121.34): 56 data bytes
64 bytes from 140.82.121.34: icmp_seq=0 ttl=56 time=21.817 ms
Docker version 20.10.20, build 9fdeb9c
Any ideas?
Dockerfile has the following content,
FROM node:16.4.2-alpine3.14
WORKDIR /app
COPY package.json .
COPY . /app
And ran the following build command,
docker build -t app:0.1 .
It took 28.4 seconds and below is the terminal logs,
[+] Building 28.4s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 124B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 53B 0.0s
=> [internal] load metadata for docker.io/library/node:16.4.2-alpine3.14 17.1s
=> [auth] library/node:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 2.01MB 0.0s
=> [1/4] FROM docker.io/library/node:16.4.2-alpine3.14#sha256:fabfca5e7dcb339097f998d6ef11c53dd80a3f99ed5cecc005e93d0ff6d4bda9 9.9s
=> => resolve docker.io/library/node:16.4.2-alpine3.14#sha256:fabfca5e7dcb339097f998d6ef11c53dd80a3f99ed5cecc005e93d0ff6d4bda9 0.0s
=> => sha256:fabfca5e7dcb339097f998d6ef11c53dd80a3f99ed5cecc005e93d0ff6d4bda9 1.00kB / 1.00kB 0.0s
=> => sha256:75dec02064547a8ec570f2953e8d68a1674ad3f37730160f1570cce077be9ed0 1.16kB / 1.16kB 0.0s
=> => sha256:40cb916373b08a087466d2e72402d0b3a4587fd3e9135169498cf0db4ff42a88 6.53kB / 6.53kB 0.0s
=> => sha256:5843afab387455b37944e709ee8c78d7520df80f8d01cf7f861aae63beeddb6b 2.81MB / 2.81MB 0.8s
=> => sha256:c118dce16b0057d713fc98e31606a84e4348fa2c967eaf1bb5fd21ba42825956 35.55MB / 35.55MB 7.1s
=> => sha256:aef8e8137ac43c8199343c96874993063af6584260f22b15e99f735cce5de653 2.35MB / 2.35MB 2.6s
=> => extracting sha256:5843afab387455b37944e709ee8c78d7520df80f8d01cf7f861aae63beeddb6b 0.2s
=> => sha256:ad336e0e52b8dfc38c23599663deb060b1ac169d548dec8072ead94712f708be 281B / 281B 2.0s
=> => extracting sha256:c118dce16b0057d713fc98e31606a84e4348fa2c967eaf1bb5fd21ba42825956 2.0s
=> => extracting sha256:aef8e8137ac43c8199343c96874993063af6584260f22b15e99f735cce5de653 0.2s
=> => extracting sha256:ad336e0e52b8dfc38c23599663deb060b1ac169d548dec8072ead94712f708be 0.0s
=> [2/4] WORKDIR /app 0.6s
=> [3/4] COPY package.json . 0.1s
=> [4/4] COPY . /app 0.1s
=> exporting to image 0.2s
=> => exporting layers 0.2s
=> => writing image sha256:91d93eddff55cba6bd8b72144b7320e025de93e9865177ff584c75b94d1bafc1 0.0s
=> => naming to docker.io/library/app:0.1
When I run the same build command again, it is taking 14.6 seconds.
However if I pull the node:16.4.2-alpine3.14 using,
docker pull node:16.4.2-alpine3.14
and then run the build command, then build takes only 0.3 seconds
I think, when we build an image, dependencies also get downloaded and that is why the time taken reduces from 28.4 to 14.6 seconds. But why even 14.6 seconds? It should be as less as 0.3 seconds.
Why is this so? What am I missing?
The following could be the reasons it takes 18.4 seconds:
Docker client takes the entire build context to the docker daemon.The build context is the entire directory the Dockerfile.some files or folders can take up a lot of space e.g node_modules. A remedy to this is to add the file that is not required eg .git, node_module, log files to the .dockerignore file to get Docker to ignore some files.
DNS resolution. Check how long it takes to resolve the docker registry using the dig command.
Enabling the buildkit for your build will help with improving the build time. Setting the DOCKER_BUILDKIT=1 environment variable when invoking the docker build command such as:
DOCKER_BUILDKIT=1 docker build .
please look at this link for more information
Relatively new to Docker and Compose, but I have read every letter of the Docker Compose documentation, and unsuccessfully bounced around SO for hours, with no resolution to the above question.
I have an (example) directory with the following files:
./Dockerfile:
# syntax=docker/dockerfile:1
ARG CUSTOM_NODE_VERSION
FROM node:$CUSTOM_NODE_VERSION
ARG CUSTOM_NODE_VERSION
ARG HELLO
RUN echo "HELLO: -> $HELLO"
RUN echo "NODE_VERSION -> $NODE_VERSION"
RUN echo "CUSTOM_NODE_VERSION -> $CUSTOM_NODE_VERSION"
./docker-compose.yml:
version: "3.8"
services:
test:
build:
context: .
dockerfile: Dockerfile
args:
CUSTOM_NODE_VERSION: alpine
HELLO: 5
What I want is for docker compose build to use the args specified in the docker-compose.yml file, but it doesn't:
> docker compose build test
[+] Building 0.8s (4/4) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 0.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1#sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2 0.0s
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = failed to create LLB definition: failed to parse stage name "node:": invalid reference format
But run works fine:
docker compose run test
[+] Running 1/1
⠿ Network compose-args_default Created 4.3s
[+] Building 3.1s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 0.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1#sha256:e2a8561e419ab1ba6b2fe6cbdf49fd92b95912df1cf7d313c3e2 0.0s
=> [internal] load metadata for docker.io/library/node:alpine 1.0s
=> CACHED [1/4] FROM docker.io/library/node:alpine#sha256:f372a9ffcec27159dc9623bad29997a1b61eafbb145dbf4f7a64568b 0.0s
=> [2/4] RUN echo "HELLO: -> 5" 0.5s
=> [3/4] RUN echo "NODE_VERSION -> 16.3.0" 0.3s
=> [4/4] RUN echo "CUSTOM_NODE_VERSION -> alpine" 0.5s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:e61653277599e3555b67c1a50699dd83d5c1ed1a93fe8a1a16529c6ec20e3e31 0.0s
=> => naming to docker.io/library/compose-args_test
This is all the more baffling to me because, per the Compose docs:
args
Add build arguments, which are environment variables accessible only during the build process.
Any help is much appreciated 🙏🏻
Build args were only recently added to compose-cli. Most likely that change hasn't reached the version of docker compose you're running. You can use docker-compose build (with a -) until this feature reaches your install.
I'm trying to build an image from CentOS 6.9. Using this Dockerfile:
FROM centos:6.9
RUN ls
But it keeps failing with exit code 139 with the following output:
$ docker build -t centos-6.9 .
[+] Building 1.1s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 72B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/centos:6.9 0.6s
=> [internal] load build context 0.1s
=> => transferring context: 72B 0.0s
=> CACHED [1/3] FROM docker.io/library/centos:6.9#sha256:6fff0a9edc920968351eb357c5b84016000fec6956e6d745f695e5a34f18ecd2 0.0s
=> [2/3] COPY . . 0.0s
=> ERROR [3/3] RUN ls 0.3s
------
> [3/3] RUN ls:
------
executor failed running [/bin/sh -c ls]: exit code: 139
I'm running:
Windows 10 Enterprise Version 2004
Docker Desktop 3.0.0
This appears to be an issue with WSL 2 with older base images, not docker or the image itself.
Create %userprofile%\.wslconfig file.
Add the following:
[wsl2]
kernelCommandLine = vsyscall=emulate
Restart WSL. wsl --shutdown
Restart Docker Desktop.
References:
https://github.com/microsoft/WSL/issues/4694#issuecomment-556095344
https://github.com/docker/for-win/issues/7284#issuecomment-646910923
https://github.com/microsoft/WSL/issues/4694#issuecomment-558335829