I am using Docker 1.4.1 on Ubuntu 14.04.1 LTS with Kernel 3.13.0-4.
Consider the following Dockerfile
FROM debian:wheezy
VOLUME /var/myvol
# RUN mkdir /var/myvol
# copy content to volume
ADD foo /var/myvol/foo
# create user, and make it new owner of directory
RUN useradd nonroot \
&& chown -R nonroot:nonroot /var/myvol/ \
&& ls -al /var/myvol
# switch to new user
USER nonroot
# remove directory owned by user
RUN ls -al /var/myvol && rm /var/myvol/foo && ls -al /var/myvol
and build it with
touch foo
docker build -t test .
then the resulting output is
Step 0 : FROM debian:wheezy
---> c90d655b99b2
Step 1 : VOLUME /var/myvol
---> Running in d3bc83df9451
---> b860e18186d8
Removing intermediate container d3bc83df9451
Step 2 : ADD foo /var/myvol/foo
---> aded36dba841
Removing intermediate container db5dd1b08958
Step 3 : RUN useradd nonroot && chown -R nonroot:nonroot /var/myvol/ && ls -al /var/myvol
---> Running in 148941cb7858
total 8
drwxr-xr-x 2 nonroot nonroot 4096 Feb 6 09:55 .
drwxr-xr-x 13 root root 4096 Feb 6 09:55 ..
-rw-rw-r-- 1 nonroot nonroot 0 Feb 6 09:30 foo
---> 144e4ff90439
Removing intermediate container 148941cb7858
Step 4 : USER nonroot
---> Running in 924f317b6718
---> 345c1586c69f
Removing intermediate container 924f317b6718
Step 5 : RUN ls -al /var/myvol && rm /var/myvol/foo && ls -al /var/myvol
---> Running in 16c8c2349f27
total 8
drwxr-xr-x 2 root root 4096 Feb 6 09:55 .
drwxr-xr-x 13 root root 4096 Feb 6 09:55 ..
-rw-rw-r-- 1 root root 0 Feb 6 09:30 foo
rm: cannot remove `/var/myvol/foo': Permission denied
INFO[0000] The command [/bin/sh -c ls -al /var/myvol && rm /var/myvol/foo && ls -al /var/myvol] returned a non-zero code: 1
If I'd replace the VOLUME line with the commented one below, it works perfectly. What is really strange is the output of ls -al: While the first says the owner was nonroot, the second one outputs the owner as root, so the chown command seems to be somehow discarded or the permissions resetted after switching to the new user.
Am I understanding Docker volumes in a wrong way? Is only root allowed to work with them, or may this be a bug that I should report?
[Edit]
What I want to achieve is to use a volume as data-storage for a containerized service. This service isn't required to run as root (and so I would prefer to use a non-root user), but is required to delete directories and files that are no longer needed.
When you declare a directory as a VOLUME, you effectively can't use it in a Dockerfile any more. The basic reason is that volumes are set up when the container is run, not built.
In this case, you could simply move the VOLUME statement to the end of the Dockerfile. Any data in the image at that directory will be copied into the volume when the container is started.
Related
I guess you all know this error pretty good. In order to understand why I may encouter it, you may need some background info about the context as it's not really a common use (i think at least).
I have an UNRAID server
A VM running in this server
A bunch of services running in containers via compose in this VM (referenced later as my production VM)
[PROBLEM] I need to add a container that itself will run compose from this official image
This last container is maintained by myself and should run an API, web client, db, ect. When building it I first trigger the build of proxy, api, web and other project that I develop and once ot's done I build the compose one from the images I just built and some open source ones.
To test I created a dummy VM on my unraid server and created a compose environment similar to my production VM. (let's call it my test VM). I added a compose file with only my app and portainer.
The problem is when I run 'docker-compose up' on that test server, portainer start but my app fails because it cannot connect to it's docker daemon (see logs below)
What I tried:
running my app as root or a created user
adding the created user to docker group (but docker group does not exists so I'm creating it; seems odd to me...)
checking permission on /var/run/docker.sock: it returned a file not found error even as root
passing the socket from my test VM when running the parent compose via a volume (- "/var/run/docker.sock:/var/run/docker.sock")
Dockerfile:
FROM docker/compose
# Create plaxdmin user
RUN adduser plaxdmin --disabled-password
RUN addgroup docker
RUN addgroup plaxdmin docker
USER plaxdmin
# Final values
ARG PLAXDMIN_VERSION
ARG RELEASE_TYPE
ENV PLAXDMIN_VERSION=$PLAXDMIN_VERSION
ENV RELEASE_TYPE=$RELEASE_TYPE
# Default user defined values
ENV TIMEZONE=Europe/Paris
ENV PLAXDMIN_DNS="plaxdmin.default.org"
# Init folders and copy docker-compose api configuration files
WORKDIR /var/log/plaxdmin
WORKDIR /etc/plaxdmin
ADD ./resources/conf/* ./
WORKDIR /opt/plaxdmin/
ADD ./resources/docker-compose.yml ./
# Expose port
EXPOSE 80
# On run debug and start compose fleet
CMD docker -v \
&& docker-compose -v \
&& printenv \
&& ls -al /etc/plaxdmin \
&& ls -al /opt/plaxdmin/ \
&& ls -al /var/log/plaxdmin/ \
&& pwd \
&& whoami \
&& groups $user \
# && ls -la /var/run/docker.sock \
&& docker-compose up || true
docker build logs:
Step 1/18 : FROM docker/compose
latest: Pulling from docker/compose
aad63a933944: Pulling fs layer
b396cd7cbac4: Pulling fs layer
0426ec0ed60a: Pulling fs layer
9ac2a98ece5b: Pulling fs layer
9ac2a98ece5b: Waiting
b396cd7cbac4: Verifying Checksum
b396cd7cbac4: Download complete
aad63a933944: Verifying Checksum
aad63a933944: Download complete
aad63a933944: Pull complete
0426ec0ed60a: Verifying Checksum
0426ec0ed60a: Download complete
b396cd7cbac4: Pull complete
9ac2a98ece5b: Verifying Checksum
9ac2a98ece5b: Download complete
0426ec0ed60a: Pull complete
9ac2a98ece5b: Pull complete
Digest: sha256:b60a020c0f68047b353a4a747f27f5e5ddb17116b7b018762edfb6f7a6439a82
Status: Downloaded newer image for docker/compose:latest
---> c3e188a6b38f
Step 2/18 : RUN adduser plaxdmin --disabled-password
---> Running in 07aa9a297234
Removing intermediate container 07aa9a297234
---> 494c8a4291e0
Step 3/18 : RUN addgroup docker
---> Running in f64e5022e65d
Removing intermediate container f64e5022e65d
---> 84ee5fbf6dea
Step 4/18 : RUN addgroup plaxdmin docker
---> Running in 0efa66b73f4a
Removing intermediate container 0efa66b73f4a
---> eb647c03c118
Step 5/18 : USER plaxdmin
---> Running in 4529203341d1
Removing intermediate container 4529203341d1
---> 8501d9993307
Step 6/18 : ARG PLAXDMIN_VERSION
---> Running in 07d61186fadd
Removing intermediate container 07d61186fadd
---> ed6e9f9df0ab
Step 7/18 : ARG RELEASE_TYPE
---> Running in 0fa98c641843
Removing intermediate container 0fa98c641843
---> d0fe2f700e53
Step 8/18 : ENV TIMEZONE=Europe/Paris
---> Running in 5c5d383c6858
Removing intermediate container 5c5d383c6858
---> 48394a4e01b3
Step 9/18 : ENV PLAXDMIN_DNS="plaxdmin.default.org"
---> Running in 187304a8a1ed
Removing intermediate container 187304a8a1ed
---> 5827abebd0ff
Step 10/18 : ENV PLAXDMIN_VERSION=$PLAXDMIN_VERSION
---> Running in 54ff13db32e6
Removing intermediate container 54ff13db32e6
---> 9377ac82544e
Step 11/18 : ENV RELEASE_TYPE=$RELEASE_TYPE
---> Running in 2da68d0375ac
Removing intermediate container 2da68d0375ac
---> dd09ee57c867
Step 12/18 : WORKDIR /var/log/plaxdmin
---> Running in 9ac2fdb93c5e
Removing intermediate container 9ac2fdb93c5e
---> 252771ee5ff4
Step 13/18 : WORKDIR /etc/plaxdmin
---> Running in eb6c9a16b12f
Removing intermediate container eb6c9a16b12f
---> 6fd180adcb80
Step 14/18 : ADD ./resources/conf/* ./
---> 70e10c126b4f
Step 15/18 : WORKDIR /opt/plaxdmin/
---> Running in 0a6f15afc915
Removing intermediate container 0a6f15afc915
---> d8c321d31689
Step 16/18 : ADD ./resources/docker-compose.yml ./
---> 60847c38d0be
Step 17/18 : EXPOSE 80
---> Running in cbe2a4d7f8be
Removing intermediate container cbe2a4d7f8be
---> 56269d51e6d5
Step 18/18 : CMD docker -v && docker-compose -v && printenv && ls -al /etc/plaxdmin && ls -al /opt/plaxdmin/ && ls -al /var/log/plaxdmin/ && pwd && whoami && groups $user && docker-compose up || true
---> Running in 49d1a3505198
Removing intermediate container 49d1a3505198
---> beba0e2fd039
Successfully built beba0e2fd039
Successfully tagged plaxdmin/full:latest
Successfully tagged plaxdmin/full:unstable
Successfully tagged plaxdmin/full:v-202102010319
Successfully tagged plaxdmin/full:64ce4f02f88ac81219dd61ae0d8c2e4aa6e0403e
Successfully tagged plaxdmin/full:master
Start logs:
plaxdmin_1 | Docker version 19.03.8, build afacb8b7f0
plaxdmin_1 | docker-compose version 1.26.2, build eefe0d3
plaxdmin_1 | HOSTNAME=b3a358707bd6
plaxdmin_1 | SHLVL=2
plaxdmin_1 | HOME=/home/plaxdmin
plaxdmin_1 | PGID=1421
plaxdmin_1 | TIMEZONE=Europe/Paris
plaxdmin_1 | RELEASE_TYPE=unstable
plaxdmin_1 | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
plaxdmin_1 | PLAXDMIN_DNS=plaxdmin.vba.ovh
plaxdmin_1 | PUID=1421
plaxdmin_1 | PWD=/opt/plaxdmin
plaxdmin_1 | PLAXDMIN_VERSION=v-202102010319
plaxdmin_1 | total 20
plaxdmin_1 | drwxr-xr-x 1 root root 4096 Feb 1 15:57 .
plaxdmin_1 | drwxr-xr-x 1 root root 4096 Feb 1 15:59 ..
plaxdmin_1 | -rw-rw-rw- 1 root root 262 Jan 31 02:06 application.properties
plaxdmin_1 | -rw-rw-rw- 1 root root 690 Jan 31 02:06 log4j.properties
plaxdmin_1 | -rw-rw-rw- 1 root root 1518 Jan 31 19:31 nginx.conf
plaxdmin_1 | total 12
plaxdmin_1 | drwxr-xr-x 1 root root 4096 Feb 1 15:57 .
plaxdmin_1 | drwxr-xr-x 1 root root 4096 Feb 1 15:57 ..
plaxdmin_1 | -rw-rw-rw- 1 root root 2374 Feb 1 02:01 docker-compose.yml
plaxdmin_1 | total 8
plaxdmin_1 | drwxr-xr-x 2 root root 4096 Feb 1 15:57 .
plaxdmin_1 | drwxr-xr-x 1 root root 4096 Feb 1 15:57 ..
plaxdmin_1 | /opt/plaxdmin
plaxdmin_1 | plaxdmin
plaxdmin_1 | plaxdmin docker
plaxdmin_1 | Couldn't connect to Docker daemon at http+docker://localhost - is it running?
plaxdmin_1 |
plaxdmin_1 | If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
Since the goal of your container is to run Docker commands, it has permissions to take over the entire host system should it choose to. It's not any less safe to run it as USER root, which will also address the socket permission problem. Since your Dockerfile doesn't actually do anything switched to the alternate user (COPY makes files be owned by root by default and you do not RUN any commands) you can also delete the USER line and the alternate-user setup.
# This user and group will not be used; delete these lines
# RUN adduser plaxdmin --disabled-password
# RUN addgroup docker
# RUN addgroup plaxdmin docker
# Nothing is done as this user
# Stay as the default root user to be able to run `docker` commands
# USER plaxdmin
If the host's /var/run/docker.sock is mode 0660 and owned by a group docker (a typical setup) the container process must run as the same numeric group ID in order to be able to access the socket. This will intrinsically be host-specific and it's not something you can set in your Dockerfile.
When you launch the orchestration container, you can run it with an additional group to put it in the docker group
# If the container process isn't already running as root
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
--group-add docker \
...
Or, in Compose version 2 syntax (but not version 3) there is a group_add: option that can specify this
version: '2.4'
services:
orchestrator:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
group_add:
- docker
(The documentation says the group must exist in both contexts, so you may need to look up the numeric group ID and use that instead.)
I'm new to Docker so I might not have some of the terminology correct. Inside the container I'm getting a permission denied error on a directory shared with the host. They appear to have matching uid:gid and the permissions host side are 777. The container is not for running in the background.
I'm using the container to run a big series of untrusted programs one at a time each needing the same initial conditions. So I don't think it's feasible to copy stuff into the docker image at build time. I felt the optimal thing to do is copy the programs one at a time to a temp directory on the host and then share that directory with the fresh container for each run. I also need to collect the output from the container-run programs and keep them on the host so I can see how each program's output differs from the others.
I have looked at the following questions/answers:
Docker: Copying files from Docker container to host
How to fix docker: Got permission denied issue - successfully used to make docker run as someone other than root
How do I add a user when I'm using Alpine as a base image? and Setting up a new user - used to create the user and group
I am:
running docker as an ordinary user uid 1000, gid 1000, also belonging to the group docker
setting permissions on the shared directory host side to be 777 with uid:gid as 1000:1000 which is the same as the user
setting the uid and gid inside the container to match uid and gid from the host
using the Dockerfile to create a uid and gid each of 1000
I read here that If the first argument begins with a / or ~/, you’re creating a bindmount. Remove that, and you’re naming the volume. So I tried both. The bindmount version seems to have the correct uid:gid but is permission denied, the volume version comes out as root:root.
As a newbie it's hard to know what information to share so here's everything I think might be useful:
Docker command attempt 1
[osboxes#osboxes tmp]$ pwd
/var/tmp
osboxes#osboxes tmp]$ whoami
osboxes
[osboxes#osboxes tmp]$ grep osboxes /etc/passwd
osboxes:x:1000:1000:osboxes.org:/home/osboxes:/bin/bash
[osboxes#osboxes tmp]$ groups
osboxes wheel vboxsf docker
[osboxes#osboxes tmp]$ grep osboxes /etc/group
wheel:x:10:osboxes
osboxes:x:1000:osboxes
vboxsf:x:981:osboxes
docker:x:1001:osboxes
[osboxes#osboxes tmp]$ ls -al
total 2
drwxrwxrwt. 11 root root 4096 Dec 31 12:13 .
drwxr-xr-x. 21 root root 4096 Jul 5 05:00 ..
drwxr-xr-x. 2 abrt abrt 6 Jul 5 05:00 abrt
drwxrwxrwx. 2 osboxes osboxes 6 Dec 31 12:13 host
continues...
[osboxes#osboxes tmp]$ docker run --rm -v /var/tmp/host:/var/tmp/container:rw \
--user appuser:appgroup --workdir /var/tmp/container \
-it alpine_bash_jdk11 /bin/bash
bash-5.0$ pwd
/var/tmp/container
bash-5.0$ ls -al
ls: can't open '.': Permission denied
total 0
bash-5.0$ ls -al ..
total 0
drwxrwxrwt 1 root root 23 Dec 31 12:51 .
drwxr-xr-x 1 root root 17 Dec 16 10:31 ..
drwxrwxrwx 2 appuser appgroup 6 Dec 31 12:13 container
bash-5.0$ whoami
appuser
bash-5.0$ groups
appgroup
bash-5.0$ grep appuser /etc/passwd
appuser:x:1000:1000:Linux User,,,:/home/appuser:/sbin/nologin
bash-5.0$ grep appuser /etc/group
appgroup:x:1000:appuser
Docker command attempt 2
everything as before except
for removing the qualified path to the host's
/var/tmp/host directory
docker run --rm -v host:/var/tmp/container:rw \
--user appuser:appgroup --workdir /var/tmp/container \
-it alpine_bash_jdk11 /bin/bash
bash-5.0$ pwd
/var/tmp/container
bash-5.0$ ls -al
total 0
drwxr-xr-x 2 root root 6 Dec 31 12:13 .
drwxrwxrwt 1 root root 23 Dec 31 13:03 ..
bash-5.0$ ls -al ..
total 0
drwxrwxrwt 1 root root 23 Dec 31 13:03 .
drwxr-xr-x 1 root root 17 Dec 16 10:31 ..
drwxr-xr-x 2 root root 6 Dec 31 12:13 container
bash-5.0$ whoami
appuser
bash-5.0$ groups
appgroup
bash-5.0$ echo hello from contanier > container.msg.txt
bash: container.msg.txt: Permission denied
Docker build command
as user osboxes
docker build -t alpine_bash_jdk11 .
Dockerfile
FROM alpine:latest
RUN apk --no-cache update
RUN apk add --no-cache bash
RUN apk --no-cache add openjdk11 --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
ENV JAVA_HOME="/usr/lib/jvm/default-jvm"
ENV PATH=$PATH:${JAVA_HOME}/bin
RUN addgroup -g 1000 -S appgroup && adduser -S appuser -G appgroup -u 1000
USER appuser
I haven't used docker compose because I'm still getting my head round basic docker.
Virtual Machine which is the Docker Host
CentOS 7.2003 from osboxes.org, organization's decision, not mine
Linux osboxes 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
I did a yum update, then yum installed all the stuff needed to install VirtualBox guest additions which is working ok
Docker version 1.13.1, build 0be3e21/1.13.1
Physical Host
Windows 10 64-bit
VirtualBox 6.1.4r136177
both these are the organization's decisions
tl;dr: had old version of docker due to wrong install command
The answer: install docker-ce instead of docker. Depending on your system that might be
sudo apt-get install -y docker-ce
or
sudo yum -y install docker-ce
instead of sudo apt-get install -y docker
or
sudo yum -y install docker
Solution: update docker
Having found this article I could see that I had the wrong version of docker. I justifiably thought the correct command was
sudo yum install -y docker
but it should have been docker-ce
I had to yum erase -y docker docker-common
Now I have Docker version 20.10.1, build 831ebea
I am trying to launch an app, deployed using wildfly18 in a docker container, which internally connects to my host postgresql database installation. During the container creation process, I am also mapping my container's wildfly log directory to my local i.e "host" directory via a named volume, created using the docker volume create command.
The issue is, I get a "permission denied" error when the app runs and the container tries to create log files inside the mapped volume.
My Dockerfile contents are as below:
FROM jboss/base-jdk:8
ENV WILDFLY_VERSION 18.0.1.Final
ENV WILDFLY_SHA1=ef0372589a0f08c36b15360fe7291721a7e3f7d9
ENV JBOSS_HOME /opt/jboss/wildfly
USER root
RUN cd $HOME \
&& curl -O https://download.jboss.org/wildfly/$WILDFLY_VERSION/wildfly-$WILDFLY_VERSION.tar.gz \
&& sha1sum wildfly-$WILDFLY_VERSION.tar.gz | grep $WILDFLY_SHA1 \
&& tar xf wildfly-$WILDFLY_VERSION.tar.gz \
&& mv $HOME/wildfly-$WILDFLY_VERSION $JBOSS_HOME \
&& rm wildfly-$WILDFLY_VERSION.tar.gz
COPY ./bin $JBOSS_HOME/bin
COPY ./standalone/configuration/* $JBOSS_HOME/standalone/configuration/
COPY ./modules/com $JBOSS_HOME/modules/com
COPY ./modules/system/layers/base/org/ $JBOSS_HOME/modules/system/layers/base/org/
COPY ./standalone/waffle_resource $JBOSS_HOME/standalone/waffle_resource
COPY ./standalone/waffle_resource/waffle.ear $JBOSS_HOME/standalone/deployments/
COPY ./standalone/waffle_resource/waffle-war.ear $JBOSS_HOME/standalone/deployments/
RUN chown -R jboss:jboss ${JBOSS_HOME} && chmod -R g+rw ${JBOSS_HOME}
ENV LAUNCH_JBOSS_IN_BACKGROUND true
USER jboss
EXPOSE 8989 9990
WORKDIR $JBOSS_HOME/bin
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0"]
As you can see above, I am using user JBOSS inside the container to kick off wildfly.
The commands used to create an image and run a container and also to create a volume are as below:
docker image build -t viaduct/wildfly .
docker volume create viaduct-wildfly-logs
docker run -d -v viaduct-wildfly-logs:/opt/jboss/wildfly/standalone/log --network=host \
-e "DB_DBNAME=dbname" \
-e "DB_PORT=5432" \
-e "DB_USER=xyz" \
-e "DB_PASS=" \
-e "DB_HOST=127.0.0.1" \
--name petes viaduct/wildfly
I verified the permissions within the container and my local "host" directory created by docker volume create command. Also, it's worth noting,
I am running wildlfy as user JBOSS
.
The containers permissions are as below:
[jboss#localhost /]$ ll /opt/jboss/wildfly/standalone/
total 4
drwxrwxr-x 1 jboss jboss 62 Sep 18 00:24 configuration
drwxr-xr-x 6 jboss jboss 84 Sep 18 00:23 data
drwxrwxr-x 1 jboss jboss 64 Sep 18 00:24 deployments
drwxrwxr-x 1 jboss jboss 17 Nov 15 2019 lib
*drwxr-xr-x 2 root root 6 Sep 17 23:48 log*
drwxrwxr-x 1 jboss jboss 4096 Sep 18 00:24 tmp
drwxrwxr-x 1 jboss jboss 98 Sep 18 00:23 waffle_resource
[jboss#localhost /]$ exit
and the local volume permissions are as below:
[root#localhost xyz]# cd /var/lib/docker/volumes/
[root#localhost volumes]# ll
drwxrwsr-x 3 root root 19 Sep 18 11:48 viaduct-wildfly-logs
The docker volume create command creates directory in my local machine as below:
/var/lib/docker/volumes/viaduct-wildfly-logs/_data
and the permissions for each subdirectories by default are as follows, which definitely is for maintained for security reasons:
drwx--x--x 14 root root 182 Sep 14 09:32 docker
drwx------ 7 root root 285 Sep 18 11:48 volumes
drwxrwsr-x 3 root root 19 Sep 18 11:48 viaduct-wildfly-logs
To start with, please let me know whether my strategy is correct?
Secondly, let me know the best way to fix the permission issue?
You need to create a user with the same UID/GID and give the permission on the host folder for this volume.
The server is run as the jboss user which has the uid/gid set to 1000. doc
Running Docker version 17.06.0-ce, build 02c1d87, I have a dockerfile that looks like this:
FROM maven:3.5.2-jdk-8-alpine as builder
RUN chmod -R 777 /root/.m2 &&\
mkdir -p /root/.m2/repository/com/foo/bar &&\
echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt &&\
ls -R -a -l /root/.m2/repository/com/foo
RUN ls -R -a -l /root/.m2/repository/com/foo
The first RUN command successfully creates a file, but the second command can't find it:
Step 1/46 : FROM maven:3.5.2-jdk-8-alpine as builder
---> 293423a981a7
Step 2/46 : RUN chmod -R 777 /root/.m2 && mkdir -p /root/.m2/repository/com/foo/bar && echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt && ls -R -a -l /root/.m2/repository/com/foo
---> Running in a1c0fd142856
/root/.m2/repository/com/foo:
total 12
drwxr-xr-x 3 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
drwxr-xr-x 2 root root 4096 Nov 30 13:32 bar
/root/.m2/repository/com/foo/bar:
total 12
drwxr-xr-x 2 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
-rw-r--r-- 1 root root 5 Nov 30 13:32 baz.txt
---> b997ccbfd5b0
Step 3/46 : RUN ls -R -a -l /root/.m2/repository/com/foo
---> Running in 603671c87ecc
ls: /root/.m2/repository/com/foo: No such file or directory
The command '/bin/sh -c ls -R -a -l /root/.m2/repository/com/foo' returned a non-zero code: 1
What's going on? (NB. this is a toy example, but there is a real issue in that JARs installed into the Maven repository seem to disappear between layers.)
The upstream maven image defines this directory as a volume. Once an image does this, you cannot reliably make changes to that directory in the image.
From their Dockerfile:
ARG USER_HOME_DIR="/root"
...
VOLUME "$USER_HOME_DIR/.m2"
The Dockerfile documentation describes this behavior:
Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded.
Your options are to:
Use another directory for your build
Request that the upstream image removes this VOLUME definition
Build your own image without this definition (it's fairly easy to fork their repo and do your own build)
For more details, you can see an old blog post by me about this behavior and the problems it creates.
I am playing with a Dockerfile and I have this:
ARG PUID=1000
ARG PGID=1000
RUN groupadd -g $PGID docker-user && \
useradd -u $PUID -g docker-user -m docker-user && \
mkdir /home/docker-user/.composer
COPY container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer
RUN chown -R docker-user:docker-user /home/docker-user/.composer
USER docker-user
RUN composer global install
But when I try to build the image it ends with the following error:
Step 6 : COPY container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer
lstat container-files/home/docker-user/.composer/composer.json: no such file or directory
The file does exist on the host as per this output:
$ ls -la workspace/container-files/home/docker-user/.composer/
total 12
drwxrwxr-x 2 rperez rperez 4096 Oct 5 11:34 .
drwxrwxr-x 3 rperez rperez 4096 Oct 5 11:14 ..
-rw-rw-r-- 1 rperez rperez 208 Oct 5 11:20 composer.json
I have tried also this syntax:
COPY container-files /
But didn't work either. So I should ask: what's wrong? Why this keep failing once and once? What I am missing here?
The documentation addresses this with:
By default the docker build command will look for a Dockerfile at
the root of the build context. The -f, --file, option lets you
specify the path to an alternative file to use instead. This is useful
in cases where the same set of files are used for multiple builds. The
path must be to a file within the build context. If a relative path is specified then it is interpreted as relative to the root of the context.
In this case I think is
COPY workspace/container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer