Disable cache for Docker directly in Dockerfile - docker

Question

I use Gitpod as my online IDE. Gitpod builds a Docker container from a user-provided Dockerfile. The user doesn't have access to the terminal which runs the docker build command and thus no flags can be passed. At the moment, my Dockerfile fails build because Docker incorrectly caches instructions, including mkdir commands. Specifically, given the Dockerfile:
# Base image is one of Ubuntu's official distributions.
FROM ubuntu:20.04
# Install curl.
RUN apt-get update
RUN apt-get -y install sudo
RUN sudo apt-get install -y curl
RUN sudo apt-get install -y python3-pip
# Download Google Cloud CLI installation script.
RUN mkdir -p /tmp/google-cloud-download
RUN curl -sSL https://sdk.cloud.google.com > /tmp/google-cloud-download/install.sh
# Install Google Cloud CLI.
RUN mkdir -p /tmp/google-cloud-cli
RUN bash /tmp/gcloud.sh --install-dir=/tmp/google-cloud-cli --disable-prompts
# Move the content of /tmp/gcloud into the container.
COPY /tmp/google-cloud-cli /google-cloud-cli
The build fails with the following log:
#1 [internal] load .dockerignore
#1 transferring context: 114B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.43kB done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/ubuntu:20.04
#3 DONE 1.2s
#4 [ 1/13] FROM docker.io/library/ubuntu:20.04#sha256:af5efa9c28de78b754777af9b4d850112cad01899a5d37d2617bb94dc63a49aa
#4 resolve docker.io/library/ubuntu:20.04#sha256:af5efa9c28de78b754777af9b4d850112cad01899a5d37d2617bb94dc63a49aa done
#4 sha256:3b65ec22a9e96affe680712973e88355927506aa3f792ff03330f3a3eb601a98 0B / 28.57MB 0.1s
#4 ...
#5 [internal] load build context
#5 transferring context: 1.70MB 0.1s done
#5 DONE 0.1s
#6 [ 5/13] RUN sudo apt-get install -y python3-pip
#6 CACHED
#7 [ 9/13] RUN bash /tmp/gcloud.sh --install-dir=/tmp/google-cloud-cli --disable-prompts
#7 CACHED
#8 [ 4/13] RUN sudo apt-get install -y curl
#8 CACHED
#9 [ 7/13] RUN curl -sSL https://sdk.cloud.google.com > /tmp/google-cloud-download/install.sh
#9 CACHED
#10 [ 8/13] RUN mkdir -p /tmp/google-cloud-cli
#10 CACHED
#11 [ 3/13] RUN apt-get -y install sudo
#11 CACHED
#12 [ 6/13] RUN mkdir -p /tmp/google-cloud-download
#12 CACHED
#13 [10/13] COPY /tmp/google-cloud-cli /google-cloud-cli
#13 ERROR: failed to calculate checksum of ref j0t2zzxkw0572xeibprcp5ebn::w8exf03p6f5luerwcumrkxeii: "/tmp/google-cloud-cli": not found
#14 [ 2/13] RUN apt-get update
#14 CANCELED
------
> [10/13] COPY /tmp/google-cloud-cli /google-cloud-cli:
------
Dockerfile:22
--------------------
20 |
21 | # Move the content of /tmp/gcloud into the container.
22 | >>> COPY /tmp/google-cloud-cli /google-cloud-cli
23 |
24 | # Copy local code to the container image.
--------------------
error: failed to solve: failed to compute cache key: failed to calculate checksum of ref j0t2zzxkw0572xeibprcp5ebn::w8exf03p6f5luerwcumrkxeii: "/tmp/google-cloud-cli": not found
{"#type":"type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent","command":"build","error":"exit status 1","level":"error","message":"build failed","serviceContext":{"service":"bob","version":""},"severity":"ERROR","time":"2022-08-28T05:31:11Z"}
exit
headless task failed: exit status 1
Other than stop using Gitpod altogheter, which I'm considering, how could I solve this issue?

Answer 1

When you COPY /tmp/google-cloud-cli /google-cloud-cli, it tries to copy a file from outside of Docker space (the build context, the directory argument to docker build, frequently the same directory as the Dockerfile) into the image.
In your case, you already have the file inside the image, so you need to RUN cp or mv or another command to relocate the existing file.
RUN bash /tmp/gcloud.sh --install-dir=/tmp/google-cloud-cli --disable-prompts
RUN mv /tmp/google-cloud-cli /google-cloud-cli

Answer 2

A way to invalidate caches of docker layers in Gitpod is to put in an environment variable above all the layers you want to invalidate and change its value.
FROM gitpod/workspace-full
ENV INVALIDATE_CACHE=1
...
(If this doesn't help, please share a repository with the mentioned Dockerfile to reproduce)