I have a problem, that I cannot grasp at all. I'm running my Jenkins pipeline in a Docker container on the master node. Now I added another node and want to run the pipeline there as well.
However, using the same image I get different file permissions in the container:
### master
> docker image ls node:10.20.1-stretch
REPOSITORY TAG IMAGE ID CREATED SIZE
node 10.20.1-stretch c5f1efe092a0 13 days ago 912MB
> docker run --rm -ti -u 1000:1000 node:10.20.1-stretch ls -la /home/node
total 20
drwxr-xr-x 2 1000 1000 4096 May 15 20:31 .
drwxr-xr-x 3 0 0 4096 May 15 20:31 ..
-rw-r--r-- 1 1000 1000 220 May 15 2017 .bash_logout
-rw-r--r-- 1 1000 1000 3526 May 15 2017 .bashrc
-rw-r--r-- 1 1000 1000 675 May 15 2017 .profile
### node 1
> docker image ls node:10.20.1-stretch
REPOSITORY TAG IMAGE ID CREATED SIZE
node 10.20.1-stretch c5f1efe092a0 13 days ago 912MB
> docker run --rm -ti -u 1000:1000 node:10.20.1-stretch ls -la /home/node
total 20
drwxr-xr-x 2 0 0 4096 May 26 05:42 .
drwxr-xr-x 1 0 0 4096 May 26 05:42 ..
-rw-r--r-- 1 0 0 220 May 26 05:42 .bash_logout
-rw-r--r-- 1 0 0 3526 May 26 05:42 .bashrc
-rw-r--r-- 1 0 0 675 May 26 05:42 .profile
I observed a similar behavior for the /tmp directory, which has chmod 1777 on master and 1755 on node 1.
# master
> docker -v
Docker version 19.03.9, build 9d988398e7
> dockerd -v
Docker version 19.03.9, build 9d988398e7
# node 1
> docker -v
Docker version 19.03.10, build 9424aeaee9
> dockerd -v
Docker version 19.03.10, build 9424aeaee9
I assume the wrong behavior is on node 1, as the /home/node directory and all of its children are owned by root:root there, but the same directory is owned by node:node on the master. However, I already upgraded the Docker version on node 1 from 19.03.8 to 19.03.10 and nothing changed.
It there anything I don't understand about Docker containers? I have been working with them for a while, but never observed such a behavior.
I have change the storage driver from overlay2 to aufs. Now I have the correct permissions.
Related
Following is the Dockerfile for the image,
FROM jenkins/jenkins:lts-jdk11
USER jenkins
RUN jenkins-plugin-cli --plugins "blueocean:1.25.2 http_request" && ls -la /var/jenkins_home
When this is built using docker build -t ireshmm/jenkins:lts-jdk11 ., following is the output,
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM jenkins/jenkins:lts-jdk11
---> 9aee0d53624f
Step 2/3 : USER jenkins
---> Using cache
---> 49d657d24299
Step 3/3 : RUN jenkins-plugin-cli --plugins "blueocean:1.25.2 http_request" && ls -la /var/jenkins_home
---> Running in b459c4c48e3e
Done
total 20
drwxr-xr-x 3 jenkins jenkins 4096 Jan 22 16:49 .
drwxr-xr-x 1 root root 4096 Jan 12 15:46 ..
drwxr-xr-x 3 jenkins jenkins 4096 Jan 22 16:49 .cache
-rw-rw-r-- 1 jenkins root 7152 Jan 12 15:42 tini_pub.gpg
Removing intermediate container b459c4c48e3e
---> 5fd5ba428f1a
Successfully built 5fd5ba428f1a
Successfully tagged ireshmm/jenkins:lts-jdk11
When create a container and list files docker run -it --rm ireshmm/jenkins:lts-jdk11 ls -la /var/jenkins_home, following is the output:
total 40
drwxr-xr-x 3 jenkins jenkins 4096 Jan 22 16:51 .
drwxr-xr-x 1 root root 4096 Jan 12 15:46 ..
-rw-r--r-- 1 jenkins jenkins 4683 Jan 22 16:51 copy_reference_file.log
drwxr-xr-x 2 jenkins jenkins 16384 Jan 22 16:51 plugins
-rw-rw-r-- 1 jenkins root 7152 Jan 12 15:42 tini_pub.gpg
Question: Why do the contents of /var/jenkins_home differ while building the image and the inside the container created from it given that no command is run after listing the files while building image? How can that happen?
The jenkins/jenkins:lts-jdk11 has an ENTRYPOINT that runs /usr/local/bin/jenkins.sh, which among other things creates the copy_reference_file.log file:
$ grep -i copy_reference /usr/local/bin/jenkins.sh
: "${COPY_REFERENCE_FILE_LOG:="${JENKINS_HOME}/copy_reference_file.log"}"
touch "${COPY_REFERENCE_FILE_LOG}" || { echo "Can not write to ${COPY_REFERENCE_FILE_LOG}. Wrong volume permissions?"; exit 1; }
echo "--- Copying files at $(date)" >> "$COPY_REFERENCE_FILE_LOG"
find "${REF}" \( -type f -o -type l \) -exec bash -c '. /usr/local/bin/jenkins-support; for arg; do copy_reference_file "$arg"; done' _ {} +
The ENTRYPOINT scripts runs whenever you start a container from that image (before any command you've provided on the command line).
Using a Docker application, I want to run an app as Daemon:
docker run -v $(pwd)/:/src -dit --name DOCKER_NAME my-app
And then execute a Python script from the mounted drive:
docker exec -w /src DOCKER_NAME python my_script.py
This Python script generates some files and figures, that I would later want to use. However, I have an issue that the files generated from within the Docker app have different rights than my outer environment.
[2D] drwxrwxr-x 5 jenkins_slave jenkins_slave 4096 Mar 21 10:47 .
[2D] drwxrwxr-x 24 jenkins_slave jenkins_slave 4096 Mar 21 10:46 ..
[2D] drwxrwxr-x 2 jenkins_slave jenkins_slave 4096 Mar 21 10:46 my_script.py
[2D] -rw-r--r-- 1 root root 268607 Mar 21 10:46 spaider_2d_0_000.png
[2D] -rw-r--r-- 1 root root 271945 Mar 21 10:46 spaider_2d_0_001.png
[2D] -rw-r--r-- 1 root root 283299 Mar 21 10:46 spaider_2d_0_010.png
In the above example, the latter 3 files are generated from within the Docker mount.
Can I in any way specify that the Docker app should be run with same credentials as the outer environment, and/or the generated files should have certain permissions?
Use Docker's -u/--user instruction to set user and group to run the container.
For example, if I would like to run the container not by root but by myself, I can do the following:
user=$(id -u)
group=$(cut -d: -f3 < <(getent group $(whoami)))
docker run -it -u "$user:$group" <CONTAINER_NAME> <COMMAND>
Inside the container you will find the user ID has changed to the one as in the host.
$ whoami
whoami: unknown uid 1000
Yes the username becomes unknown, but I guess you will not bother with it. You are doing this to set the correct permissions, not to get a nicely displayed name, right?
P.S., Docs here: https://docs.docker.com/engine/reference/run/#user
I have a strange problem running a Docker container
It is working OK if I run:
docker run -it -v /home/drleo/pythonCourses:/home/pythonCurses /redpmorg/python-courses
But if I run container with publish option the Docker will create a new folder in my /home/drleo directory with the SAME name: pythonCourses, owned by root but obviously empty:
docker run -it -p 127.0.0.1:8080:8080 -v /home/drleo/pythonCourses:/home/pythonCurses /redpmorg/python-courses
-rw-r--r-- 1 drleo drleo 675 May 6 2016 .profile
drwxr-xr-x 2 drleo drleo 4096 May 6 2016 Public
drwxr-xr-x 2 root root 4096 Feb 16 13:08 pyhtonCourses
drwxrwxr-x 2 drleo drleo 4096 Feb 16 13:08 pythonCourses
-rwxrwxr-x 1 drleo drleo 71 Jan 20 22:35 reset-network
The question is why? Thanks!
You seem to have a type somewhere. python != pyhton.
Double check your command history.
I want my Dockerfile to mkdir .ssh/
But it does not, why not?
FROM jenkinsci/jnlp-slave
MAINTAINER Johnny5 isAlive <johnny5#hotmail.com>
USER root
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN apt-get update
RUN apt-get install unzip git curl vim -y
USER jenkins
RUN mkdir -p /home/jenkins/.ssh && touch /home/jenkins/.ssh/aFile
...building...
Looks fine?
Step 12 : RUN mkdir -p /home/jenkins/.ssh && touch /home/jenkins/.ssh/aFile
---> Running in ca19a679580d
---> 5980df7db482
Removing intermediate container ca19a679580d
Successfully built 5980df7db482
Running and looking around, the .ssh/ folder and aFile inside are not there ...
$ docker run -it -u 0 --entrypoint /bin/bash 5980df7db482
root#4aa40a18baf2:~# pwd
/home/jenkins
root#4aa40a18baf2:~# ls -al
total 24
drwxr-xr-x 3 jenkins jenkins 4096 Oct 17 23:17 .
drwxr-xr-x 4 root root 4096 Sep 14 08:50 ..
-rw-r--r-- 1 jenkins jenkins 220 Nov 12 2014 .bash_logout
-rw-r--r-- 1 jenkins jenkins 3515 Nov 12 2014 .bashrc
-rw-r--r-- 1 jenkins jenkins 675 Nov 12 2014 .profile
drwxr-xr-x 2 jenkins jenkins 4096 Sep 14 08:50 .tmp
root#4aa40a18baf2:~#
If I pull the parent image, jenkinsci/jnlp-slave, and inspect it with docker inspect jenkinsci/jnlp-slave, I can see that it already has a volume defined at /home/jenkins:
[
{
...
"ContainerConfig": {
...
"Volumes": {
"/home/jenkins": {}
},
...
}
]
This means that during each build step, any changes you make to that location won't be committed to your new layer.
Here's a simplified version of your Dockerfile to highlight what's going on:
FROM jenkinsci/jnlp-slave
RUN mkdir -p /home/jenkins/.ssh
Now, let's build with the following command: docker build --no-cache --rm=false -t jns .:
Sending build context to Docker daemon 2.56 kB
Step 1 : FROM jenkinsci/jnlp-slave
---> d7731d944ad7
Step 2 : RUN mkdir -p /home/jenkins/.ssh
---> Running in 520a8e2f7cae
---> 962189878d5e
Successfully built 962189878d5e
The --no-cache option makes the command easier to work with on repeat invocations. The --rm=false will cause the builder to not remove the containers created for each step.
In this case, the builder ran the Step 2 in 520a8e2f7cae on my system. I can now do a docker inspect 520a8e2f7cae and see the actual container used for this step. Specifically, I'm curious about the mounts location:
[
{
...
"Mounts": [
{
"Name": "e34fd82bd190f21dbd63b5cf70167a16674cd00d95fdc6159314c25c6d08e10e",
"Source": "/var/lib/docker/volumes/e34fd82bd190f21dbd63b5cf70167a16674cd00d95fdc6159314c25c6d08e10e/_data",
"Destination": "/home/jenkins",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
...
}
]
I see that there's an anonymous volume with id e34fd82bd190f21dbd63b5cf70167a16674cd00d95fdc6159314c25c6d08e10e for /home/jenkins.
I can inspect the contents of that volume like this:
$ docker run --rm -v e34fd82bd190f21dbd63b5cf70167a16674cd00d95fdc6159314c25c6d08e10e:/volume alpine ls -lah /volume
total 28
drwxr-xr-x 4 10000 10000 4.0K Oct 18 02:49 .
drwxr-xr-x 25 root root 4.0K Oct 18 02:55 ..
-rw-r--r-- 1 10000 10000 220 Nov 12 2014 .bash_logout
-rw-r--r-- 1 10000 10000 3.4K Nov 12 2014 .bashrc
-rw-r--r-- 1 10000 10000 675 Nov 12 2014 .profile
drwxr-xr-x 2 10000 10000 4.0K Oct 18 02:49 .ssh
drwxr-xr-x 2 10000 10000 4.0K Sep 14 08:50 .tmp
The .ssh directory created in the RUN step is in this volume. Since volumes aren't part of the container's write layer, it won't get committed. I can confirm this by doing a docker diff on this container:
docker diff 520a8e2f7cae
There is no output, showing no changes to the container's filesystem, which is why it doesn't come forward into this layer of the image.
The other contents at this location are files in the parent image that were committed before the VOLUME directive that made /home/jenkins into a volume.
First time docker user here, I'm using this image: https://github.com/dgraziotin/osx-docker-lamp
I want to make the apache in that container to use a configuration file from the host system. How do I do that?
I know I can use nsenter, but I think my changes will get deleted when the container is turned off.
Thank you
The best solution is using VOLUME.
docker pull dgraziotin/lamp
You need to copy /etc/apache2/ from container to current directory in host computer. Then you can do this:
cd ~
mkdir conf
docker run -i -t --rm -v ~/conf:/tmp/conf dgraziotin/lamp:latest /bin/bash
On container do:
ls /tmp/conf
cd /etc/apache2/
tar -cf /tmp/conf/apache-conf.tar *
exit
On host computer:
cd conf
tar -xf apache-conf.tar
cd ..
# alter your configuration in this file and save
vi conf/apache2.conf
# run your container : daemon mode
docker run -d -p 9180:80 --name web-01 -v ~/conf:/etc/apache2 dgraziotin/lamp:latest
docker ps
To list conf content on Container use:
docker exec web-01 ls -lAt /etc/apache2/
total 72
-rw-r--r-- 1 root root 1779 Jul 17 20:24 envvars
drwxr-xr-x 2 root root 4096 Apr 10 11:46 mods-enabled
drwxr-xr-x 2 root root 4096 Apr 10 11:45 sites-available
-rw-r--r-- 1 root root 7136 Apr 10 11:45 apache2.conf
drwxr-xr-x 2 root root 4096 Apr 10 11:45 mods-available
drwxr-xr-x 2 root root 4096 Apr 10 11:44 conf-enabled
drwxr-xr-x 2 root root 4096 Apr 10 11:44 sites-enabled
drwxr-xr-x 2 root root 4096 Apr 10 11:44 conf-available
-rw-r--r-- 1 root root 320 Jan 7 2014 ports.conf
-rw-r--r-- 1 root root 31063 Jan 3 2014 magic
Use docker exec web-01 cat /etc/apache2/apache2.conf to list content inside Container.
One the WEB page to test your environment.
I hope this help you.
You should use a Dockerfile to generate a new image containing your desired configuration. For example:
FROM dgraziotin/lamp
COPY my-config-file /some/configuration/file
This assumes that there is a file my-config-file located in the same directory as the Dockerfile. Then run:
docker build -t myimage
And once the build completes you will have an image named myimage available locally.