Building Dockerfile fails when touching a file after a mkdir - docker

Question

I'm new to Docker and try to build an image with a simple Dockerfile:
FROM jenkins
USER root
RUN mkdir -pv /home/a/b
RUN touch /home/a/b/test.txt
RUN mkdir -pv /var/jenkins_home/a/b
RUN touch /var/jenkins_home/a/b/test.txt
USER jenkins
When I build it, it fails with the following output:
Step 0 : FROM jenkins
Step 1 : USER root
Step 2 : RUN mkdir -pv /home/a/b
mkdir: created directory '/home/a'
mkdir: created directory '/home/a/b'
Step 3 : RUN touch /home/a/b/test.txt
Step 4 : RUN mkdir -pv /var/jenkins_home/a/b
mkdir: created directory '/var/jenkins_home/a'
mkdir: created directory '/var/jenkins_home/a/b'
Step 5 : RUN touch /var/jenkins_home/a/b/test.txt
touch: cannot touch '/var/jenkins_home/a/b/test.txt': No such file or directory
Can anyone tell me, what I am missing here? Why does the first mkdir & touch combination work and the second does not?

Answer 1

Looking at https://registry.hub.docker.com/u/library/jenkins/, it seems that /var/jenkins_home is a volume. You can only create files there while the container is running, presumably with a volume mapping like
docker run ... -v /your/jenkins/home:/var/jenkins_home ...
The docker build process knows nothing about shared volumes.

Answer 2

This is currently investigated in docker/docker/issues/3639, and summarized in this comment:
Okay, I did little research and it seems that volume is non-mutable between Dockerfile instruction.
Here even smaller Dockerfile for testing:
FROM busybox
RUN mkdir /tmp/volume
RUN echo "hello" > /tmp/volume/hello
VOLUME ["/tmp/volume/"]
RUN [[ -f /tmp/volume/hello ]]
RUN rm /tmp/volume/hello
RUN [[ ! -e /tmp/volume/hello ]]
On each instruction we create new volume and copy content from original volume.
Update April 2019:
Use DOCKER_BUILDKIT=1
The new builder does not exhibit this behavior.
Example from dominikzalewski:
That's a very simple Dockerfile that I'm using:
FROM wordpress:latest
ARG UPLOAD_DIR=/var/www/html/wp-content/uploads
RUN mkdir -p $UPLOAD_DIR
RUN ls -lhd $UPLOAD_DIR
Cf. Build Enhancements for Docker
Docker Build enhancements for 18.09 release introduces a much-needed overhaul of the build architecture.
By integrating BuildKit, users should see an improvement on performance, storage management, feature functionality, and security.
Docker images created with buildkit can be pushed to Docker Hub and DTR just like Docker images created with legacy build
The Dockerfile format that works on legacy build will also work with buildkit builds
The new --secret command line option allows the user to pass secret information for building new images with a specified Dockerfile