I have a dockerfile looking like this
FROM debian:bullseye-slim
RUN apt-get update; \
apt-get install -y libssl-dev; \
apt-get clean
COPY ./runner /runner
CMD /runner
I build the image like this:
docker build -f RunnerDockerfile -t $(RUNNER_IMG) target/release
Even though my executable (runner) changes, it's not picked up by Docker, i.e. Docker uses the cache.
According to my understanding of the docs, if the hash of a file in a COPY statement is changed, it should be be picked up by Docker.
Example run
md5sum target/release/runner
32169761853677f5f7bc03acb7bbe19b target/release/runner
Then I make a change:
md5sum target/release/runner
0075f725ce4f2cd779706b2fb9f83218 target/release/runner
When rebuilding the Docker images I get this
docker build -f RunnerDockerfile -t "runner:latest" target/release
Sending build context to Docker daemon 513.4MB
Step 1/4 : FROM debian:bullseye-slim
---> f8000d381a2c
Step 2/4 : RUN apt-get update; apt-get install -y libssl-dev; apt-get clean
---> Using cache
---> fcd7212dd477
Step 3/4 : COPY ./runner /runner
---> Using cache
---> 1a79d547a9b5
Step 4/4 : CMD /runner
---> Using cache
---> a45a0fd57dd9
Successfully built a45a0fd57dd9
Successfully tagged runner:latest
I.e. it's using the cache for the COPY.
Am I doing something wrong or is this a bug?
Related
I have been given a project that is in a Docker container. I have managed to build the Docker container image and tag it, but when I run it I have problems.
bash-5.1$ docker build -t game:0.0.1 -t game:latest .
Sending build context to Docker daemon 2.584MB
Step 1/12 : FROM nvidia/cuda:10.2-base-ubuntu18.04
---> 84b82c2f5736
Step 2/12 : MAINTAINER me
---> Using cache
---> b8a86a8860d5
Step 3/12 : EXPOSE 5006
---> Using cache
---> fabdfc06768c
Step 4/12 : EXPOSE 8888
---> Using cache
---> a6f8585ce52d
Step 5/12 : ENV DEBIAN_FRONTEND noninteractive
---> Using cache
---> c4dd4de87fdc
Step 6/12 : ENV WD=/home/game/
---> Using cache
---> 871163f5db29
Step 7/12 : WORKDIR ${WD}
---> Using cache
---> 36678a12e551
Step 8/12 : RUN apt-get -y update && apt-get -y upgrade && apt-get -y install git ssh pkg-config python3-pip python3-opencv
---> Using cache
---> 4b83b4944484
Step 9/12 : COPY requirements.txt /requirements.txt
---> Using cache
---> 8e1db9206e80
Step 10/12 : RUN cd / && python3 -m pip install --upgrade pip && pip3 install -r requirements.txt
---> Using cache
---> e096029d458a
Step 11/12 : CMD ["start.py"]
---> Using cache
---> 795bb5a65bc8
Step 12/12 : ENTRYPOINT ["python3"]
---> Using cache
---> 59b472b693f2
Successfully built 59b472b693f2
Successfully tagged game:0.0.1
Successfully tagged game:latest
bash-5.1$ docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix game:latest
Unable to find image 'game:latest' locally
docker: Error response from daemon: pull access denied for game, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
bash-5.1$ sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix game:latest
It doesn't seem to find the game:latest image even though the output of the above command says it just created it.
I also try to do this after logging into my session.
I tried to run 59b472b693f2 (what is it, is it a container hash code?):
bash-5.1$ docker run 59b472b693f2
python3: can't open file 'start.py': [Errno 2] No such file or directory
bash-5.1$ ls
data_collection demonstrateur.ipynb demo.py Dockerfile examples README.md requirements.txt serious_game start.py test
bash-5.1$
Here is the list of available images:
bash-5.1$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
game 0.0.1 7e7ad7272cf0 15 minutes ago 1.77GB
game latest 7e7ad7272cf0 15 minutes ago 1.77GB
ubuntu latest ba6acccedd29 7 weeks ago 72.8MB
hello-world latest feb5d9fea6a5 2 months ago 13.3kB
nvidia/cuda 10.2-base-ubuntu18.04 84b82c2f5736 2 months ago 107MB
bash-5.1$ docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix game:latest
python3: can't open file 'start.py': [Errno 2] No such file or directory
bash-5.1$
I tried to add it in the Dockerfile but still got the same error:
Removing intermediate container 10f2d7506d17
---> 1b776923e5a9
Step 11/13 : COPY start.py /start.py
---> 172c81ff16e9
Step 12/13 : CMD ["start.py"]
---> Running in c7217e2e0f21
Removing intermediate container c7217e2e0f21
---> eaf947ffa0b1
Step 13/13 : ENTRYPOINT ["python3"]
---> Running in 77e2e7b90658
Removing intermediate container 77e2e7b90658
---> 924d8c473e36
Successfully built 924d8c473e36
Successfully tagged seriousgame:0.0.1
Successfully tagged seriousgame:latest
bash-5.1$ docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix seriousgame:latest
python3: can't open file 'start.py': [Errno 2] No such file or directory
Here is my Dockerfile:
#############################################################################################################
#
# Creation du container
#
##############################################################################################################
FROM nvidia/cuda:10.2-base-ubuntu18.04
MAINTAINER me
EXPOSE 5006
EXPOSE 8888
ENV DEBIAN_FRONTEND noninteractive
ENV WD=/home/game/
WORKDIR ${WD}
# Add git and ssh
RUN apt-get -y update && \
apt-get -y upgrade && \
apt-get -y install git ssh pkg-config python3-pip python3-opencv
# Dépendances python
COPY requirements.txt /requirements.txt
RUN cd / && \
python3 -m pip install --upgrade pip && \
pip3 install -r requirements.txt
COPY start.py /start.py
CMD ["start.py"]
ENTRYPOINT ["python3"]
Here are all the files within my project:
bash-5.1$ ls
data_collection demonstrateur.ipynb demo.py Dockerfile examples README.md requirements.txt serious_game start.py test
In the first block of code you posted it says Successfully tagged game:latest and Successfully tagged game:0.0.1, but in your docker images output you don't see those images. Looking at the output of your docker images I see that the last time you built the image named serious-game was 1 hour ago. I'm guessing so that you tried to rename the image, but the image ID didn't change.
You can try to remove the old image with docker image rm command (docs), and then try to build it again. The commands sequence to execute is the code block below. Data should be safe becouse I see that you're using volumes (I assume that you know what you're doing).
docker image rm 59b472b693f2
docker build -t game:0.0.1 -t game:latest .
The sequence 59b472b693f2 is the unique ID of the image in your Docker local environment (you can assume that it's an ID like the ones used in databases for indexing).
This Dockerfile:
FROM ubuntu:latest
COPY . /srv/keller
WORKDIR /srv/keller
RUN DEBIAN_FRONTEND=noninteractive apt-get update
results in
Sending build context to Docker daemon 1.167 MB
Step 0 : FROM ubuntu:latest
---> d9fad37da739
Step 1 : COPY . /srv/keller
---> 0691d53a9ddb
Removing intermediate container 76978e260250
Step 2 : WORKDIR /srv/keller
---> Running in 7d47ac19f397
---> 924513b02e82
Removing intermediate container 7d47ac19f397
Step 3 : RUN DEBIAN_FRONTEND=noninteractive apt-get update
---> Running in 97284e8842bc
exec format error
[8] System error: exec format error
on Raspbian GNU/Linux 9. What is the problem here? Note that this is not about entrypoint/command. This error occurs on apt update.
You are trying to run an intel x86 64 image: ubuntu:latest on a raspberry pi (an ARM64), it cannot work out of the box...
Either you can change your base image to be compatible with your host (ex: arm64v8/ubuntu).
Or you can install qemu:
apt install -y qemu qemu-system-x86 binfmt-support qemu-user-static
and register it in docker:
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
This happened to me while setting up pod in the cloud that runs Debian, while my machine is M1. I had to push some images to a private docker repo, but when I pushed them, I noticed the uploaded ones where not compatible with the architecture of the pod in the cluster.
My fix was pulling the correct image docker pull --platform=linux/amd64 python:3.9.16-buster and then using that one istead of the pulled by just doing docker pull --platform=linux/amd64 python:3.9.16-buster.
you are missing the export and a semi-colon, this should be
RUN export DEBIAN_FRONTEND=noninteractive; apt-get update
or you can use the && syntax
RUN export DEBIAN_FRONTEND=noninteractive && apt-get update
RAM is ramping up in my docker container and I aim to investigate the cause. I was recommended memory profiler to help me with this.
I was told the steps were as follows:
Install https://pypi.org/project/memory-profiler/ in the image
manually execute the program in the container using mprof run -o memory-profile.dat python3 -m vdx
mprof plot memory-profile.dat
Now, I'm having trouble realizing these steps, since I am new to docker, python and the memory profiler.
For the first step, I included RUN pip3 install memory-profiler in the Dockerfile below RUN pip3 install -r requirements.txt
Then, below where it says COPY src /vdx I've added RUN mprof run -o memory-profile.dat python3 -m vdp
Now, when I build and run the container everything seems to work as usual...
Container is being built:
Step 7/12 : RUN pip3 install -r requirements.txt
---> Using cache
---> 11ad28b8793a
Step 8/12 : RUN pip3 install memory-profiler
---> Using cache
---> 58885abbcd8b
Step 9/12 : COPY src /vdx
---> Using cache
---> 15e884cb94c5
Step 10/12 : WORKDIR /
---> Using cache
---> 93cdfcee7f4a
Step 11/12 : RUN mprof run -o memory-profile.dat python3 -m vdx
---> Using cache
---> de7b15fd74ef
Step 12/12 : ENTRYPOINT ["python3", "-m", "vdx"]
---> Using cache
---> 53675841da99
Successfully built 53675841da99
and the process runs smoothly ...
but no file memory-profile.dat is being created.
What am I doing wrong?
Notice there are two different stages in the life-cycle of a container: building and running. The Dockerfile applies to the building stage, whereas the ENTRYPOINT action in the Dockerfile applies to the running stage.
This means that RUN mprof run -o memory-profile.dat python3 -m vdx in the Dockerfile is probably not what you want, as you want to investigate the issue when the container is running and not when it's building, don't you?
Basically what you are doing is, you are running your app with the memory profiler at build time, and then you set an entrypoint without the memory profiler for runtime, so when you run the container, you actually don't have the output of the profiler because it's not being run.
You need to run the profiler at runtime, moving the instruction to the ENTRYPOINT. ENTRYPOINT ["mprof", "run", "-o", "memory-profile.dat", "python3", "-m", "vdx"].
I've been reading docs Best practices for writing Dockerfiles. I encountered small incorrectness (IMHO) for which meaning was clear after reading further:
Using apt-get update alone in a RUN statement causes caching issues
and subsequent apt-get install instructions fail.
Why fail I wondered. Later came explanation of what they meant by "fail":
Because the apt-get update is not run, your build can potentially get
an outdated version of the curl and nginx packages.
However, for the following I still cannot understand what they mean by "If not, the cache is invalidated.":
Starting with a parent image that is already in the cache, the next
instruction is compared against all child images derived from that
base image to see if one of them was built using the exact same
instruction. If not, the cache is invalidated.
That part is mentioned in some answers on SO e.g. How does Docker know when to use the cache during a build and when not? and as a whole the concept of cache invalidation is clear to me, I've read below:
When does Docker image cache invalidation occur?
Which algorithm Docker uses for invalidate cache?
But what is meaning of "if not"? At first I was sure the phrase meant if no such image is found. That would be overkill - to invalidate cache which maybe useful later for other builds. And indeed it is not invalidated if no image is found when I've tried below:
$ docker build -t alpine:test1 - <<HITTT
> FROM apline
> RUN echo "test1"
> RUN echo "test1-2"
> HITTT
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM apline
pull access denied for apline, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
(base) nb0408:docker a.martianov$ docker build -t alpine:test1 - <<HITTT
> FROM alpine
> RUN echo "test1"
> RUN echo "test1-2"
> HITTT
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM alpine
---> 965ea09ff2eb
Step 2/3 : RUN echo "test1"
---> Running in 928453d33c7c
test1
Removing intermediate container 928453d33c7c
---> 0e93df31058d
Step 3/3 : RUN echo "test1-2"
---> Running in b068bbaf8a75
test1-2
Removing intermediate container b068bbaf8a75
---> daeaef910f21
Successfully built daeaef910f21
Successfully tagged alpine:test1
$ docker build -t alpine:test1-1 - <<HITTT
> FROM alpine
> RUN echo "test1"
> RUN echo "test1-3"
> HITTT
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM alpine
---> 965ea09ff2eb
Step 2/3 : RUN echo "test1"
---> Using cache
---> 0e93df31058d
Step 3/3 : RUN echo "test1-3"
---> Running in 74aa60a78ae1
test1-3
Removing intermediate container 74aa60a78ae1
---> 266bcc6933a8
Successfully built 266bcc6933a8
Successfully tagged alpine:test1-1
$ docker build -t alpine:test1-2 - <<HITTT
> FROM alpine
> RUN "test2"
> RUN
(base) nb0408:docker a.martianov$ docker build -t alpine:test2 - <<HITTT
> FROM alpine
> RUN echo "test2"
> RUN echo "test1-3"
> HITTT
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM alpine
---> 965ea09ff2eb
Step 2/3 : RUN echo "test2"
---> Running in 1a058ddf901c
test2
Removing intermediate container 1a058ddf901c
---> cdc31ac27a45
Step 3/3 : RUN echo "test1-3"
---> Running in 96ddd5b0f3bf
test1-3
Removing intermediate container 96ddd5b0f3bf
---> 7d8b901f3939
Successfully built 7d8b901f3939
Successfully tagged alpine:test2
$ docker build -t alpine:test1-3 - <<HITTT
> FROM alpine
> RUN echo "test1"
> RUN echo "test1-3"
> HITTT
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM alpine
---> 965ea09ff2eb
Step 2/3 : RUN echo "test1"
---> Using cache
---> 0e93df31058d
Step 3/3 : RUN echo "test1-3"
---> Using cache
---> 266bcc6933a8
Successfully built 266bcc6933a8
Successfully tagged alpine:test1-3
Cache was again used for last build. What does docs mean by "if not"?
Let's focus on your original problem (regarding apt-get update) to make things easier. The following example is not based on any best practices. It just illustrates the point you are trying to understand.
Suppose you have the following Dockerfile:
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y nginx
You build a first image using docker build -t myimage:latest .
What happens is:
The ubuntu image is pulled if it does not exist
A layer is created and cached to run apt-get update
A layer is created an cached to run apt install -y nginx
Now suppose you modify your Docker file to be
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y nginx openssl
and you run a build again with the same command as before. What happens is:
There is already an ubuntu image locally so it will not be pulled (unless your force with --pull)
A layer was already created with command apt-get update against the existing local image so it uses the cached one
The next command has changed so a new layer is created to install nginx and openssl. Since apt database was created in the preceding layer and taken from cache, if a new nginx and/or openssl version was released since then, you will not see them and you will install the outdated ones.
Does this help you to grasp the concept of cached layers ?
In this particular example, the best handling is to do everything in a single layer making sure you cleanup after yourself:
FROM ubuntu:18.04
RUN apt-get update \
&& apt-get install -y nginx openssl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
The phrasing of the line would be better said:
If not, there is a cache miss and the cache is not used for this build step and any following build step of this stage of the Dockerfile.
That gets a bit verbose because a multi-stage Dockerfile can fail to find a cache match in one stage and then find a match in another stage. Different builds can all use the cache. The cache is "invalidated" for a specific build process, the cache itself is not removed from the docker host and it continues to be available for future builds.
I have build a Docker image from a Docker file using the below command.
$ docker build -t u12_core -f u12_core .
When I am trying to rebuild it with the same command, it's using the build cache like:
Step 1 : FROM ubuntu:12.04
---> eb965dfb09d2
Step 2 : MAINTAINER Pavan Gupta <pavan.gupta#gmail.com>
---> Using cache
---> 4354ccf9dcd8
Step 3 : RUN apt-get update
---> Using cache
---> bcbca2fcf204
Step 4 : RUN apt-get install -y openjdk-7-jdk
---> Using cache
---> 103f1a261d44
Step 5 : RUN apt-get install -y openssh-server
---> Using cache
---> dde41f8d0904
Step 6 : RUN apt-get install -y git-core
---> Using cache
---> 9be002f08b6a
Step 7 : RUN apt-get install -y build-essential
---> Using cache
---> a752fd73a698
Step 8 : RUN apt-get install -y logrotate
---> Using cache
---> 93bca09b509d
Step 9 : RUN apt-get install -y lsb-release
---> Using cache
---> fd4d10cf18bc
Step 10 : RUN mkdir /var/run/sshd
---> Using cache
---> 63b4ecc39ff0
Step 11 : RUN echo 'root:root' | chpasswd
---> Using cache
---> 9532e31518a6
Step 12 : RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
---> Using cache
---> 47d1660bd544
Step 13 : RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
---> Using cache
---> d1f97f1c52f7
Step 14 : RUN wget -O aerospike.tgz 'http://aerospike.com/download/server/latest/artifact/ubuntu12'
---> Using cache
---> bd7dde7a98b9
Step 15 : RUN tar -xvf aerospike.tgz
---> Using cache
---> 54adaa09921f
Step 16 : RUN dpkg -i aerospike-server-community-*/*.deb
---> Using cache
---> 11aba013eea5
Step 17 : EXPOSE 22 3000 3001 3002 3003
---> Using cache
---> e33aaa78a931
Step 18 : CMD /usr/sbin/sshd -D
---> Using cache
---> 25f5fe70fa84
Successfully built 25f5fe70fa84
The cache shows that aerospike is installed. However, I don't find it inside containers spawn from this image, so I want to rebuild this image without using the cache. How can I force Docker to rebuild a clean image without the cache?
There's a --no-cache option:
docker build --no-cache -t u12_core -f u12_core .
In older versions of Docker you needed to pass --no-cache=true, but this is no longer the case.
In some extreme cases, your only way around recurring build failures is by running:
docker system prune
The command will ask you for your confirmation:
WARNING! This will remove:
- all stopped containers
- all volumes not used by at least one container
- all networks not used by at least one container
- all images without at least one container associated to them
Are you sure you want to continue? [y/N]
This is of course not a direct answer to the question, but might save some lives... It did save mine.
To ensure that your build is completely rebuild, including checking the base image for updates, use the following options when building:
--no-cache - This will force rebuilding of layers already available
--pull - This will trigger a pull of the base image referenced using FROM ensuring you got the latest version.
The full command will therefore look like this:
docker build --pull --no-cache --tag myimage:version .
Same options are available for docker-compose:
docker-compose build --no-cache --pull
Note that if your docker-compose file references an image, the --pull option will not actually pull the image if there is one already.
To force docker-compose to re-pull this, you can run:
docker-compose pull
The command docker build --no-cache . solved our similar problem.
Our Dockerfile was:
RUN apt-get update
RUN apt-get -y install php5-fpm
But should have been:
RUN apt-get update && apt-get -y install php5-fpm
To prevent caching the update and install separately.
See: Best practices for writing Dockerfiles
Most of information here are correct.
Here a compilation of them and my way of using them.
The idea is to stick to the recommended approach (build specific and no impact on other stored docker objects) and to try the more radical approach (not build specific and with impact on other stored docker objects) when it is not enough.
Recommended approach :
1) Force the execution of each step/instruction in the Dockerfile :
docker build --no-cache
or with docker-compose build :
docker-compose build --no-cache
We could also combine that to the up sub-command that recreate all containers:
docker-compose build --no-cache &&
docker-compose up -d --force-recreate
These way don't use cache but for the docker builder and the base image referenced with the FROM instruction.
2) Wipe the docker builder cache (if we use Buildkit we very probably need that) :
docker builder prune -af
3) If we don't want to use the cache of the parent images, we may try to delete them such as :
docker image rm -f fooParentImage
In most of cases, these 3 things are perfectly enough to allow a clean build of our image.
So we should try to stick to that.
More radical approach :
In corner cases where it seems that some objects in the docker cache are still used during the build and that looks repeatable, we should try to understand the cause to be able to wipe the missing part very specifically.
If we really don't find a way to rebuild from scratch, there are other ways but it is important to remember that these generally delete much more than it is required. So we should use them with cautious overall when we are not in a local/dev environment.
1) Remove all images without at least one container associated to them :
docker image prune -a
2) Remove many more things :
docker system prune -a
That says :
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all images without at least one container associated to them
- all build cache
Using that super delete command may not be enough because it strongly depends on the state of containers (running or not).
When that command is not enough, I try to think carefully which docker containers could cause side effects to our docker build and to allow these containers to be exited in order to allow them to be removed with the command.
With docker-compose try docker-compose up -d --build --force-recreate
I would not recommend using --no-cache in your case.
You are running a couple of installations from step 3 to 9 (I would, by the way, prefer using a one liner) and if you don't want the overhead of re-running these steps each time you are building your image you can modify your Dockerfile with a temporary step prior to your wget instruction.
I use to do something like RUN ls . and change it to RUN ls ./ then RUN ls ./. and so on for each modification done on the tarball retrieved by wget
You can of course do something like RUN echo 'test1' > test && rm test increasing the number in 'test1 for each iteration.
It looks dirty, but as far as I know it's the most efficient way to continue benefiting from the cache system of Docker, which saves time when you have many layers...
You can manage the builder cache with docker builder
To clean all the cache with no prompt:
docker builder prune -af
GUI-driven approach: Open the docker desktop tool (that usually comes with Docker):
under "Containers / Apps" stop all running instances of that image
under "Images" remove the build image (hover over the box name to get a context menu), eventually also the underlying base image