I am running into a problem with buildkit and I cannot figure out which is the reason.
I have one Dockerfile using as base image sles OS and it tries to do some package installation via zypper. Everytime this step is executed, not cached, it takes years to complete.
This is a dummy Dockerfile for verification of this issue.
# syntax=docker/dockerfile:1.3
FROM registry.suse.com/suse/sles12sp4
RUN zypper search iproute2
This is execution when I enable Buildkit:
docker build --no-cache --progress=plain --pull -t test_zypper .
#1 [internal] load build definition from Dockerfile
#1 sha256:1e8bc50247fba08161184996db9e2b6bca36c339623376a360765244d9d3ed8b
#1 transferring dockerfile: 202B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:bfa4297d1f77b21d1d84347ff3f9c338cef560c9f5c8ef8f6843338b88a83178
#2 transferring context: 2B done
#2 DONE 0.0s
#3 resolve image config for docker.io/docker/dockerfile:1.3
#3 sha256:4fcd28d33487ad029eab28c03869fd56295f3902c713674c129a438f7a780653
#3 DONE 1.1s
#4 docker-image://docker.io/docker/dockerfile:1.3#sha256:42399d4635eddd7a9b8a24be879d2f9a930d0ed040a61324cfdf59ef1357b3b2
#4 sha256:7862c1373501a4a9cd96ccd04641bb1d96c86d034546e74fe74585e3dd12f952
#4 CACHED
#5 [internal] load build definition from Dockerfile
#5 sha256:adf8dd6b4b2604f820e4a4112252c8bfd5984ffa809d1fc7c5330e387575a53d
#5 DONE 0.0s
#6 [internal] load .dockerignore
#6 sha256:59c105584afe8ac8255febcea4650f6e8891b4b14fcdd7b93254039769df3828
#6 DONE 0.0s
#7 [internal] load metadata for registry.suse.com/suse/sles12sp4:latest
#7 sha256:30c143f62f5a593ad20fd34265d2933e13da97368f12f3e0c990b52851933dff
#7 DONE 0.5s
#8 [1/2] FROM registry.suse.com/suse/sles12sp4#sha256:06390bd3b9903f3d4bb1345deb7fc35e18af73de0263d0f4d5c619267bee2adf
#8 sha256:3d15a7aaf66ed6810de2347b0da9787e5a57b9c536d85ccc4b01e9eb5831bcc1
#8 CACHED
#9 [2/2] RUN zypper search iproute2
#9 sha256:17060fcd75740edd49881abc4d1b5a4f7de80f59cde5b2b6f32e97ff02bbc29d
#9 377.9 Refreshing service 'container-suseconnect-zypp'.
#9 556.7 Problem retrieving the repository index file for service 'container-suseconnect-zypp':
#9 556.7 [container-suseconnect-zypp|file:/usr/lib/zypp/plugins/services/container-suseconnect-zypp]
#9 556.7 Warning: Skipping service 'container-suseconnect-zypp' because of the above error.
#9 556.7 Loading repository data...
#9 556.7 Warning: No repositories defined. Operating only with the installed resolvables. Nothing can be installed.
#9 556.7 Reading installed packages...
#9 556.7 No matching items found.
#9 ERROR: executor failed running [/bin/sh -c zypper search iproute2]: exit code: 104
------
> [2/2] RUN zypper search iproute2:
------
executor failed running [/bin/sh -c zypper search iproute2]: exit code: 104
This is execution when I don't enable Buildkit:
time docker build --no-cache --progress=plain --pull -t test_zypper .
Sending build context to Docker daemon 678.5MB
Step 1/2 : FROM registry.suse.com/suse/sles12sp4
latest: Pulling from suse/sles12sp4
Digest: sha256:06390bd3b9903f3d4bb1345deb7fc35e18af73de0263d0f4d5c619267bee2adf
Status: Image is up to date for registry.suse.com/suse/sles12sp4:latest
---> 3126dff9c7fd
Step 2/2 : RUN zypper search iproute2
---> Running in 3efe8a741628
Refreshing service 'container-suseconnect-zypp'.
Problem retrieving the repository index file for service 'container-suseconnect-zypp':
[container-suseconnect-zypp|file:/usr/lib/zypp/plugins/services/container-suseconnect-zypp]
Warning: Skipping service 'container-suseconnect-zypp' because of the above error.
Loading repository data...
Warning: No repositories defined. Operating only with the installed resolvables. Nothing can be installed.
Reading installed packages...
No matching items found.
The command '/bin/sh -c zypper search iproute2' returned a non-zero code: 104
real 0m23.972s
user 0m1.987s
sys 0m2.161s
It is not a problem of not having repositories as in my original Dockerfile it is all defined and it eventually works, but taking 20min or more each zypper command.
Is something wrong in my way to use buildkit??
Thanks in advance!
Related
I'm building my docker images in GitLab CI and trying to cache the build so that if the app dependencies don't change, I don't need to re-build the entire image, but for some reason the Docker layer caching isn't working.
Dockerfile
FROM public.ecr.aws/lambda/python:3.9
RUN pip install poetry==1.3.2
WORKDIR /app
COPY ./pyproject.toml ./poetry.lock ./
RUN poetry config virtualenvs.create false && poetry install \
--no-interaction --no-ansi --without dev
ENV PYTHONPATH=/app
COPY . .
CMD ["app.handler"]
GitLab CI Log Output
$ docker build --cache-from "${CI_REGISTRY_IMAGE}:latest" -t $IMAGE_NAME .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 321B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 transferring context: 145B done
#2 DONE 0.0s
#3 [internal] load metadata for public.ecr.aws/lambda/python:3.9
#3 DONE 0.5s
#4 importing cache manifest from registry.gitlab.com/foo/bar:latest
#4 DONE 0.0s
#5 [1/6] FROM public.ecr.aws/lambda/python:3.9#sha256:7e3bcdc955c2c3cab1101bbbfd55849c0f56a6b5a21a149d50df91deacad6aac
#5 resolve public.ecr.aws/lambda/python:3.9#sha256:7e3bcdc955c2c3cab1101bbbfd55849c0f56a6b5a21a149d50df91deacad6aac 0.0s done
#5 sha256:ebf75d2390460a5a59d7c52cc4d6e7f6840c610b399c069b0ccf531792d77c7d 3.00kB / 3.00kB done
#5 sha256:7e3bcdc955c2c3cab1101bbbfd55849c0f56a6b5a21a149d50df91deacad6aac 772B / 772B done
#5 sha256:e3d5decbab4ddca757c7de36637b50826165efddecfe8b7ef76db761f51b790d 1.58kB / 1.58kB done
#5 DONE 0.3s
#6 [internal] load build context
#6 transferring context: 1.16MB 0.2s done
#6 DONE 0.3s
...
The pipeline is successfully picking up on the latest build as you can see from the importing cache manifest from registry.gitlab.com/foo/bar:latest line. However, it doesn't seem to be caching any of the build steps.
Can anyone see a reason why the cache might be missing in this case?
I am trying to build a docker image for my sample-go app.
I am running it from the sample-app folder itself and using the goland editor's terminal. But the build is failing and giving me certain errors.
My docker file looks like this:
FROM alpine:latest
RUN mkdir -p /src/build
WORKDIR /src/build
RUN apk add --no-cache tzdata ca-certificates
COPY ./configs /configs
COPY main /main
EXPOSE 8000
CMD ["/main"]
command for building:
docker build --no-cache --progress=plain - < Dockerfile
Error And Logs:
#1 [internal] load build definition from Dockerfile
#1 sha256:8bb9ee83603259cf748d90ce42602f12527fa720d7417da22799b2ad4e503497
#1 transferring dockerfile: 222B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:f93d938488588cd0e0a94d9d343fe69dcfd28d0cb1da95ad7aab00aac50235c3
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/alpine:latest
#3 sha256:13549c58a76bcb5dac9d52bc368a8fb6b5cf7659f94e3fa6294917b85546978d
#3 DONE 0.0s
#10 [1/6] FROM docker.io/library/alpine:latest
#10 sha256:d20daa00e252bfb345a1b4f53b6bb332aafe702d8de5e583a76fcd09ba7ea1c1
#10 CACHED
#7 [internal] load build context
#7 sha256:0f7a8a6082a837c139acc2855e1b745bba9f28cc96709d45cd0b7be42442c0e8
#7 transferring context: 2B done
#7 DONE 0.0s
#4 [2/6] RUN mkdir -p /src/build
#4 sha256:b9fa3007a44471d47414dd29b3ff07ead6af28ede820a2b4bae0ce84cf2c5a83
#4 CACHED
#5 [3/6] WORKDIR /src/build
#5 sha256:b2ec58a365fdd74c4f9030b0caff2e2225eea33617da306678ad037fce675388
#5 CACHED
#6 [4/6] RUN apk add --no-cache tzdata ca-certificates
#6 sha256:0966097abf956d5781bc2330d49cf715cd52c3807e8fedfff07dec50907ff03b
#6 CACHED
#9 [6/6] COPY main /main
#9 sha256:f4b81960427c014a020361bea0903728f289e1d796892fe0adc6409434f3ca76
#9 ERROR: "/main" not found: not found
#8 [5/6] COPY ./configs /configs
#8 sha256:630f272dd60dd307f40dbbdaef277ee0dfc24b71fa11e10a3b8efd64d3c05086
#8 ERROR: "/configs" not found: not found
#4 [2/6] RUN mkdir -p /src/build
#4 sha256:b9fa3007a44471d47414dd29b3ff07ead6af28ede820a2b4bae0ce84cf2c5a83
#4 DONE 0.2s
------
> [5/6] COPY ./configs /configs:
------
------
> [6/6] COPY main /main:
------
failed to compute cache key: "/main" not found: not found
PS: I am not able to find where is the problem? Help Please
The two folders /main and /configs does not exist.
The COPY command can't copy into this folders.
1. Solution
Create the folders on build
RUN mkdir -p /main
RUN mkdir -p /configs
And than use COPY
2. Solution
Try to build without COPY and CMD
Than run the the new image
exec into running container with bash or sh
Create the folders
Exit exec container
Create a new image of the running container with docker run commit
Stop the container and delete it
Build again with your new image and include COPY and CMD
This is a basic mistake.
COPY ./configs /configs: copy the folder configs from the host to the Docker image.
COPY main /main: copy the executable file main from the host to the Docker image.
The problems are:
The base Docker images do not have these folders /configs, /main. You must create them manually (Docker understood your command this way).
But I have some advice:
Create 2 Docker images for 2 purposes: build, production.
Copy the source code into Docker builder image which is use for building your app.
Copy necessary output files from the Docker builder image into the Docker production image.
In my case, the issue was the connected vpn/proxy network from my machine.
It worked after I disconnecting the vpn/proxy network.
In my case I missed the folder entry in .dockerignore file. Do something like that.
**/*
!docker-images
!configs
!main
I am trying to build a docker and it is giving me below exception. Any idea what could be wrong
First Day with the docker so my question could be bit out of sync
docker build .
[+] Building 1.2s (5/5) FINISHED
=> [internal] load build definition from Docker file 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.5s
=> CACHED [1/2] FROM docker.io/library/alpine#sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0 0.0s
=> ERROR [2/2] RUN apk add --update redis 0.6s
------
> [2/2] RUN apk add --update redis:
#5 0.171 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
#5 0.364 281472913525832:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1913:
#5 0.365 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.14/main: Permission denied
#5 0.365 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/main: No such file or directory
#5 0.366 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
#5 0.536 281472913525832:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1913:
#5 0.538 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.14/community: Permission denied
#5 0.538 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/community: No such file or directory
#5 0.538 ERROR: unable to select packages:
#5 0.541 redis (no such package):
#5 0.541 required by: world[redis]
# Use an exisiting docker image as a base
From alpine
# Download and install a dependency
Run apk add --update redis
#Tell the image what to do when it starts as a container
CMD ["redis-server"]
This question already has answers here:
When using BuildKit with Docker, how do I see the output of RUN commands?
(3 answers)
Closed 1 year ago.
I have this Dockerfile, where I run a command with RUN, and I want to see its output when running docker build:
FROM alpine:3.14
COPY . .
RUN echo "here are some numbers: $(seq 10)"
When I run docker build ., it doesn't show the output of the above command:
[+] Building 2.8s (8/8) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.14 1.2s
=> [internal] load build context 0.1s
=> => transferring context: 188.24kB 0.1s
=> CACHED [1/3] FROM docker.io/library/alpine:3.14#sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0 0.0s
=> [2/3] COPY . . 0.4s
=> [3/3] RUN echo "here are some numbers: $(seq 10)" 0.2s
=> exporting to image 0.8s
=> => exporting layers 0.8s
=> => writing image sha256:7e0bf9ff04a7b4a3a53395d430ddd950500be9a53733cce22a1da1929cae1a0a 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them.
How do I see what the output is?
Docker version:
$ docker --version
Docker version 20.10.6, build 370c289
Is that what are you looking for?
$ docker build --progress=plain .
Sending build context to Docker daemon 4.096kB
Step 1/3 : FROM alpine:3.14
3.14: Pulling from library/alpine
5843afab3874: Pull complete
Digest: sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0
Status: Downloaded newer image for alpine:3.14
---> d4ff818577bc
Step 2/3 : COPY . .
---> 106aa79185ae
Step 3/3 : RUN echo "here are some numbers: $(seq 10)"
---> Running in 30a81b6d5035
here are some numbers: 1
2
3
4
5
6
7
8
9
10
Removing intermediate container 30a81b6d5035
---> 3c059c9b6150
Successfully built 3c059c9b6150
$ docker --version
Docker version 19.03.8, build afacb8b
Docker is showing the output as it's generated, and then hiding it when the command is done. You can disable this interactive output by piping docker build to a file or pager. Make sure to pipe stderr as well as stdout. For example, in bash, the result of the command appears in item #7 here:
$ docker build . |& cat
#1 [internal] load build definition from Dockerfile
#1 sha256:0ce5eece4291db925ff5f4646c0f72c5c94a28a77431c2afd38ffa4c5a90ccab
#1 transferring dockerfile: 36B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:96c5d624b98a42c7926385cf683312588be6b5b47ce4512bbc5a0a59c4042367
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/alpine:3.14
#3 sha256:af035606328a1ce217c0e290353f888d2ee03ed437bef601e11d9cc421fbbb67
#3 DONE 0.9s
#4 [1/3] FROM docker.io/library/alpine:3.14#sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0
#4 sha256:2c66994e2bde1811eaa777244107d7b0d683d113e151d83a5f7aa09e37e1883f
#4 DONE 0.0s
#5 [internal] load build context
#5 sha256:c87370e17d42a78ca943cd74a0d2d983bc71dc609a92739af5013ed4373e31a6
#5 transferring context: 188.24kB 0.1s done
#5 DONE 0.1s
#4 [1/3] FROM docker.io/library/alpine:3.14#sha256:234cb88d3020898631af0ccbbcca9a66ae7306ecd30c9720690858c1b007d2a0
#4 sha256:2c66994e2bde1811eaa777244107d7b0d683d113e151d83a5f7aa09e37e1883f
#4 CACHED
#6 [2/3] COPY . .
#6 sha256:200618f800c74119b8db3b01ff18105a07477dcfae41f0528189443421f6ba31
#6 DONE 0.4s
#7 [3/3] RUN echo "here are some numbers: $(seq 10)"
#7 sha256:c094a42d846e25e5586053168f7231e17248fc7f09c75de1199642ad83688952
#7 0.241 here are some numbers: 1
#7 0.241 2
#7 0.241 3
#7 0.241 4
#7 0.241 5
#7 0.241 6
#7 0.241 7
#7 0.241 8
#7 0.241 9
#7 0.241 10
#7 DONE 0.3s
#8 exporting to image
#8 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#8 exporting layers
#8 exporting layers 0.8s done
#8 writing image sha256:7b4e8d44804bba048cf5aa47c58cd57ab701400143177fb63887ef51edc1a23b done
#8 DONE 0.8s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
I am trying to take advantage of the caching/pulling system of BUILDKIT for Docker for my CI/CD process. But it does not work as expected.
I created a dummy local example (but the same happens also in my CI system - AWS CodePipeline, and for both DockerHub and AWS ECR).
The Dockerfile:
# base image
FROM python:3.7-slim
# set working directory
WORKDIR /usr/src/app
# add and install requirements
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip $PIP_PROXY install --no-cache-dir --compile -r requirements.txt
RUN echo 123
# add app
COPY ./run_test.py /usr/src/app/run_test.py
# run server
CMD ["python", "run_test.py"]
run_test.py is actually not interesting, but here is the code just in case:
import requests
import time
while True:
time.sleep(1)
print(requests)
Also you need to create an empty requirements.txt file in the same folder.
In advance, I export two environment variables:
export DOCKER_BUILDKIT=1 # to activate buildkit
export DUMMY_IMAGE_URL=bi0max/test_docker
Then, to test I have the following command. First two commands remove local cache to resemble the CI environment, then build and push.
BE CAREFUL, CODE BELOW REMOVES LOCAL BUILD CACHE:
docker builder prune -a -f && \
(docker image rm $DUMMY_IMAGE_URL:latest || true) && \
docker build \
--cache-from $DUMMY_IMAGE_URL:latest \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--tag $DUMMY_IMAGE_URL:latest "." && \
docker push $DUMMY_IMAGE_URL:latest
As expected, the first run just builds everything from scratch:
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 434B done
#2 DONE 0.0s
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.1s
#3 [internal] load metadata for docker.io/library/python:3.7-slim
#3 DONE 0.0s
#12 [1/7] FROM docker.io/library/python:3.7-slim
#12 DONE 0.0s
#7 [internal] load build context
#7 DONE 0.0s
#4 importing cache manifest from bi0max/test_docker:latest
#4 ERROR: docker.io/bi0max/test_docker:latest not found
#12 [1/7] FROM docker.io/library/python:3.7-slim
#12 resolve docker.io/library/python:3.7-slim done
#12 DONE 0.0s
#7 [internal] load build context
#7 transferring context: 204B done
#7 DONE 0.1s
#5 [2/7] WORKDIR /usr/src/app
#5 DONE 0.0s
#6 [3/7] RUN pip install --upgrade pip
#6 1.951 Requirement already up-to-date: pip in /usr/local/lib/python3.7/site-packages (20.1.1)
#6 DONE 2.3s
#8 [4/7] COPY ./requirements.txt /usr/src/app/requirements.txt
#8 DONE 0.0s
#9 [5/7] RUN pip $PIP_PROXY install --no-cache-dir --compile -r requirement...
#9 0.750 Collecting requests==2.22.0
#9 0.848 Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
#9 0.932 Collecting idna<2.9,>=2.5
#9 0.948 Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
#9 0.995 Collecting chardet<3.1.0,>=3.0.2
#9 1.011 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
#9 1.135 Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
#9 1.153 Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
#9 1.264 Collecting certifi>=2017.4.17
#9 1.282 Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
#9 1.378 Installing collected packages: idna, chardet, urllib3, certifi, requests
#9 1.916 Successfully installed certifi-2020.4.5.1 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.9
#9 DONE 2.2s
#10 [6/7] RUN echo 123
#10 0.265 123
#10 DONE 0.3s
#11 [7/7] COPY ./run_test.py /usr/src/app/run_test.py
#11 DONE 0.0s
#13 exporting to image
#13 exporting layers done
#13 writing image sha256:f98327afae246096725f7e54742fe9b25079f1b779699b099e66c8def1e19052 done
#13 naming to docker.io/bi0max/test_docker:latest done
#13 DONE 0.0s
#14 exporting cache
#14 preparing build cache for export done
#14 DONE 0.0s
Then, I slightly adjust run_test.py file and the result is again as expected. All the layers until the last step ([7/7] COPY) are downloaded from repository and reused.
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 434B done
#1 DONE 0.1s
#3 [internal] load metadata for docker.io/library/python:3.7-slim
#3 DONE 0.0s
#8 [internal] load build context
#8 DONE 0.0s
#4 [1/7] FROM docker.io/library/python:3.7-slim
#4 DONE 0.0s
#5 importing cache manifest from bi0max/test_docker:latest
#5 DONE 1.2s
#8 [internal] load build context
#8 transferring context: 193B done
#8 DONE 0.0s
#6 [2/7] WORKDIR /usr/src/app
#6 CACHED
#7 [3/7] RUN pip install --upgrade pip
#7 CACHED
#9 [4/7] COPY ./requirements.txt /usr/src/app/requirements.txt
#9 CACHED
#10 [5/7] RUN pip $PIP_PROXY install --no-cache-dir --compile -r requirement...
#10 CACHED
#11 [6/7] RUN echo 123
#11 pulling sha256:79fc69c08b391d082b4d2617faed489d220444fa0cf06953cdff55c667866bed
#11 pulling sha256:071624272167ab4e35a30eb1640cb3f15ced19c6cd10fa1c9d49763372e81c23
#11 pulling sha256:04ed4ecd76e1a110f468eb1a3173bbfa578c6b4c85a6dc82bf4a489ed8b8c54d
#11 pulling sha256:79fc69c08b391d082b4d2617faed489d220444fa0cf06953cdff55c667866bed 0.2s done
#11 pulling sha256:d6406c1ce2dc5e841233ebce164ee469388102cb98f1473adaeca15455d6d797
#11 pulling sha256:071624272167ab4e35a30eb1640cb3f15ced19c6cd10fa1c9d49763372e81c23 0.5s done
#11 pulling sha256:04ed4ecd76e1a110f468eb1a3173bbfa578c6b4c85a6dc82bf4a489ed8b8c54d 0.5s done
#11 pulling sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
#11 pulling sha256:d6406c1ce2dc5e841233ebce164ee469388102cb98f1473adaeca15455d6d797 0.3s done
#11 pulling sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.2s done
#11 CACHED
#12 [7/7] COPY ./run_test.py /usr/src/app/run_test.py
#12 DONE 0.0s
#13 exporting to image
#13 exporting layers done
#13 writing image sha256:f37692114f10b9a3646203569a0849af20774651f4aa0f5dc8d6f133fb7ff062 done
#13 naming to docker.io/bi0max/test_docker:latest done
#13 DONE 0.0s
#14 exporting cache
#14 preparing build cache for export done
#14 DONE 0.0s
Now, I change run_test.py again and I would expect docker to do the same thing as last time. But I get the following result, where it build everything from scratch:
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 434B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/python:3.7-slim
#3 DONE 0.0s
#5 [1/7] FROM docker.io/library/python:3.7-slim
#5 DONE 0.0s
#8 [internal] load build context
#8 DONE 0.0s
#4 importing cache manifest from bi0max/test_docker:latest
#4 DONE 1.7s
#8 [internal] load build context
#8 transferring context: 182B done
#8 DONE 0.0s
#5 [1/7] FROM docker.io/library/python:3.7-slim
#5 resolve docker.io/library/python:3.7-slim done
#5 DONE 0.1s
#6 [2/7] WORKDIR /usr/src/app
#6 DONE 0.0s
#7 [3/7] RUN pip install --upgrade pip
#7 1.774 Requirement already up-to-date: pip in /usr/local/lib/python3.7/site-packages (20.1.1)
#7 DONE 2.1s
#9 [4/7] COPY ./requirements.txt /usr/src/app/requirements.txt
#9 DONE 0.0s
#10 [5/7] RUN pip $PIP_PROXY install --no-cache-dir --compile -r requirement...
#10 0.805 Collecting requests==2.22.0
#10 0.905 Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
#10 1.079 Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
#10 1.109 Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
#10 1.242 Collecting certifi>=2017.4.17
#10 1.259 Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
#10 1.336 Collecting idna<2.9,>=2.5
#10 1.353 Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
#10 1.410 Collecting chardet<3.1.0,>=3.0.2
#10 1.428 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
#10 1.545 Installing collected packages: urllib3, certifi, idna, chardet, requests
#10 2.102 Successfully installed certifi-2020.4.5.1 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.9
#10 DONE 2.4s
#11 [6/7] RUN echo 123
#11 0.259 123
#11 DONE 0.3s
#12 [7/7] COPY ./run_test.py /usr/src/app/run_test.py
#12 DONE 0.0s
#13 exporting to image
#13 exporting layers done
#13 writing image sha256:f4ffb0e84e334b4b35fe2504de11012e5dc1ca5978eace055932e9bbbe83c93e done
#13 naming to docker.io/bi0max/test_docker:latest done
#13 DONE 0.0s
#14 exporting cache
#14 preparing build cache for export done
#14 DONE 0.0s
But the strangest thing for me is, when I change run_test.py for the third time, it uses cached layers again. And it continues in the same way: fourth time - doesn't use, fifth time - uses, etc...
Do I miss something here?
If I pull the image each time before building, then it always uses cache, but it also works in the same way without the BUILDKIT.
This issue got fixed in newer docker versions, a simple upgrade resolves the issue.
Otherwise the solution described on GitHub can help to not rely on the systems docker version: https://github.com/moby/buildkit/issues/1981#issuecomment-785534131
I believe the inline cache image becomes invalid (or incomplete) if it was built while reusing the cache. It's either a limitation or a bug.
There is a workaround: you can tag a distinct cache image, that you'll only push to the registry when BuildKit has rebuilt the image. AFAIK there is no mean to know whether BuildKit used the cache or not, but we can see the log is filled with CACHED when it did, so we can reuse it. For example:
# enable buildkit:
$ export DOCKER_BUILDKIT=1
# build image trying to use cache image + build cache image:
$ docker build . \
--tag image:latest \
--tag image:build-cache \
--use-cache-from=image:build-cache \
--build-arg BUILDKIT_INLINE_CACHE = 1 \
| tee docker.log
# push new image to the registry:
docker push image:latest
# trick: only push cache image to the registry if it was rebuilt:
grep -q CACHED docker.log || docker push image:build-cache