When do we need to add -u $(id -u):$(id -g) in docker run command?
I see that it is user id and group ip mapping but I want to understand this better.
One reason you'd want to run the container under the same UID and GID as your user is so that any files created by the container in the host file system will be owned by you.
Take for instance this command, that creates a file called test.txt in the current directory on the host
docker run --rm -v $(pwd):/app ubuntu touch /app/test.txt
In the host file system, that file will be owned by root.
By running the container with the same UID and GID as your user, the file will be owned by you instead
docker run --rm -v $(pwd):/app -u $(id -u):$(id -g) ubuntu touch /app/test2.txt
Brief docker background
Docker starts containers as a root user. The root user has almost full privileged access to the state of the container. Any processes running as that user inherit those permissions.
When do we need user and group?
It follows that if there’s a bug in one of those processes, it might damage the container. There are ways to limit the damage, but the most effective way to prevent these types of issues is not to use the root user. So we use the group and user.
RUN groupadd -r -g 2200 example && useradd -rM -g example -u 2200 example
Docker supports isolating the USR namespace. By default, user and group IDs inside a container are equivalent to the same IDs on the host machine. When the user namespace is enabled, user and group IDs in the container are remapped to IDs that do not exist on the host.
Hope this helps you!
Related
I'm trying to let a docker container access a letsencrypt certificate from the host file system.
I do not want to run the docker container as root, but rather as a user with very specific access rights.
Neither do I want to change the permissions of the certificate.
All I want, is for the given user, to have access to read the certificate inside the docker container.
The certificate has the following setup:
-rw-r----- 1 root cert-group
The user who's going to run the docker container, is in the cert-group:
uid=113(myuser) gid=117(myuser) groups=117(myuser),999(cert-group),998(docker)
This works as long as we're on the host - I am able to read the file as expected with the user "myuser".
Now I want to do this within a docker container with the certificate mounted as a volume.
I have done multiple test cases, but none with any luck.
A simple docker-compose file for testing:
version: '3.7'
services:
test:
image: alpine:latest
volumes:
- /etc/ssl/letsencrypt/cert.pem:/cert.pem:ro
command: >
sh -c 'ls -l / && cat /etc/passwd && cat /etc/group && cat /cert.pem'
user: "113:117"
restart: "no"
This ouputs a lot, but most important is:
test_1 | -rw-r----- 1 root ping 3998 Jul 15 09:51 cert.pem
test_1 | cat: can't open '/cert.pem': Permission denied
test_1 | ping:x:999:
Here I assume that "ping" is an internal group for docker alpine, however, im getting some mixed information about how this collaborates with the host.
From this article https://medium.com/#mccode/understanding-how-uid-and-gid-work-in-docker-containers-c37a01d01cf my takeaway is, that there's a single kernel handling all permissions (the host) and therefore if the same uid and gid is used, the permissions would inherit from the host. However, even though that the running user is 113:117, which on the host is part of the group 999 it still doesnt give me access to read the file.
Next I found this article https://medium.com/#nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca where especially this bullet point caught my attention:
The container OS enforces file permissions on all operations made in
the container runtime according to its own configuration. For example,
if a user A exists in both host and container, adding user A to group
B on the host will not allow user A to write to a directory owned by
group B inside the container unless group B is created inside the
container as well and user A is added to it.
This made me think, that maybe a custom Dockerfile was needed, to add the user inside docker, and make the user part of 999 (which is known as ping as earlier stated):
FROM alpine:latest
RUN adduser -S --uid 113 -G ping myuser
USER myuser
Running this gives me the exact same result, now with myuser appended to passwd though:
test_1 | myuser:x:113:999:Linux User,,,:/home/myuser:/sbin/nologin
This is just a couple of things that I've tried.
Another is syncing /etc/passwd and /etc/group with volumes found in some other blog
volumes:
- /etc/passwd:/etc/passwd
- /etc/group:/etc/group
This makes it visually look correct inside the container, but it doesnt change the end result - still permission denied.
Any help or pointers in the right direction would be really appreciated since I'm running out of ideas.
Docker containers do not know the uid/gid of the user running the container on the host. All requests to run containers go through the docker socket, and then to the docker engine that is often running as root, and no uid/gid's are passed in those API calls. The docker engine is just running the container as the user specified in the Dockerfile or as part of the container create command (in this case, from the docker-compose.yml).
Once inside the container, the mapping from uid/gid to names is done with the /etc/passwd and /etc/group file that is inside the container. Importantly, at the filesystem level, uid/gid values are not being mapped between the container and the host (with the exception of user namespaces, but if implemented properly, that would only make this problem worse). And all filesystem operations happen at the uid/gid level, not based on names. So when you do a host volume mount, the uid/gid's are passed directly through.
The issue you are encountering here is how you are telling the container to pick the uid/gid to run the container processes. By specifying user: "113:117" you have told the container to not only specify the uid (113), but also the gid (117) of the process. When that's done, none of the secondary groups from /etc/group are assigned to the user. To get those secondary groups assigned, you want to only specify the uid, user: "113", which will then lookup the group assignments from the /etc/passwd and /etc/group file inside the container. E.g.:
user: "113"
Unfortunately, the lookup for group membership is done by docker before any volumes are mounted, so you have the following scenario.
First, create an image with an example user assigned to a few groups:
$ cat df.users
FROM alpine:latest
RUN addgroup -g 4242 group1 \
&& addgroup -g 8888 group2 \
&& adduser -u 1000 -D -H test \
&& addgroup test group1 \
&& addgroup test group2
$ docker build -t test-users -f df.users .
...
Next, run that image, comparing the id on the host to the id inside the container:
$ id
uid=1000(bmitch) gid=1000(bmitch) groups=1000(bmitch),24(cdrom),25(floppy),...
$ docker run -it --rm -u bmitch -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro test-users:latest id
docker: Error response from daemon: unable to find user bmitch: no matching entries in passwd file.
Woops, docker doesn't see the entry from /etc/passwd, lets try with the test user we created in the image:
$ docker run -it --rm -u test -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro test-users:latest id
uid=1000(bmitch) gid=1000(bmitch) groups=4242,8888
That works, and assigns the groups from the /etc/group file in the image, not the one we mounted. We can also see that uid works too:
$ docker run -it --rm -u 1000 -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro test-users:latest id
uid=1000(bmitch) gid=1000(bmitch) groups=4242,8888
As soon as we specify the gid, the secondary groups are gone:
$ docker run -it --rm -u 1000:1000 -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro test-users:latest id
uid=1000(bmitch) gid=1000(bmitch)
And if we run without overriding the /etc/passwd and /etc/group file, we can see the correct permissions:
$ docker run -it --rm -u test test-users:latest id
uid=1000(test) gid=1000(test) groups=4242(group1),8888(group2)
Likely the best option is to add a container user with the group membership matching the uid/gid values from the host. For host volumes, I've also solved this problem with a base image that dynamically adjusts the user or group inside the container to match the uid/gid of the file mounted in a volume. This is done as root, and then gosu is used to drop permissions back to the user. You can see that at sudo-bmitch/docker-base on github, specifically the fix-perms script that I would run as part of an entrypoint.
Also, be aware that mounting the /etc/passwd and /etc/group can break file permissions of other files within the container filesystem, and this user may have access inside that container that is not appropriate (e.g. you may have special access to the ping command that gives the ability to modify files or run ping commands that a normal user wouldn't have access to). This is why I tend to adjust the container user/group rather than completely replace these files.
Actually your solution is not wrong. I did the same with few differences.
This is my Dockerfile:
FROM alpine:latest
RUN addgroup -S cert-group -g 117 \
&& adduser -S --uid 113 -G cert-group myuser
USER myuser
And my docker-compose.yml:
version: '3.7'
services:
test:
build:
dockerfile: ./Dockerfile
context: .
command: >
sh -c 'ls -l / && cat /etc/passwd && cat /etc/group && cat /cert.pem'
volumes:
- "/tmp/test.txt:/cert.pem:ro"
restart: "no"
My '/tmp/test.txt' is assigned to 113:117.
IMHO, I think the problem in your docker-compose.yml that doesn't use your image. You should remove the image: and add build:
I have gone through the same issue today and luckily, the below solution helped me.
"Add :Z to your volumes mounts"
Reference: https://github.com/moby/moby/issues/41202
Note: Unfortunately It's issue with only Centos, I didn't face any problem with Ubuntu.
I've setup a docker container that mounts a volume that is persistent on the host machine:
docker volume create --name work type=none --opt device=/home/username --opt o=bind
docker container run -it --rm --mount source=work,target=/work
I run the container and do stuff, so let's say:
echo "Hello world" > testfile.txt
When I try to delete whatever files (e.g. testfile.txt) the container writes to /work (which maps to /home/username on the host) from a host terminal, I am getting permissions error because it is written and owned by root.
Is there a way to setup Docker to run as the current user that I am logged on to, so this permissions issue doesn't arise? Also, is there a reason it runs as root (will other functionality break if I can somehow run as a user)? I know I can always chown by way through, but I'm wondering if there is a better approach. Thanks.
You can add a non root user to your container and set that as default:
RUN useradd -m myuser
USER myuser # every command below is executed by myuser
WORKDIR /home/myuser
Use USER root to switch to root user again. (docs)
To get permission to delete the files on the host the host user and the container user need to have the same user id.
To check the user ids you can use id -u <username>. If the ids don't match use the option -u <userid> to create your user:
RUN useradd -m -u <userid> <username>
See useradd for more information.
When start a docker container as user with name 'username1' in group 'usergroup1'.
And that container has files/folders on the local file system with volume:
eg.
$username1>docker run -v /homes/username1/output:output outputter
The files are created with root as owner.
What do i need to do in the Dockerfile or startup options to make sure the file rigths in the output folder are the same as the localuser:group, in this case username1:usergroup1?
As explained in this project:
By default, our docker containers run as the root user. Files created or modified by the container will thus become owned by the root user, even after quitting the container.
To avoid this problem, it is necessary to run the container using a non-root user.
If the host machine user has a UID other than 1000 (or 0, for root), the user should specify their UID when running docker, e.g.
docker run -d -p 8787:8787 -v $(pwd):/home/$USER/foo \
-e USER=$USER -e USERID=$UID rocker/rstudio
to avoid changing the permissions in the linked volume on the host
Here that works because that project Dockerfile, when starting the container, creates a user with the same uid (name is not important)
## (Docker cares only about uid, not username; diff users with same uid = confusion)
if [ "$USERID" -ne 1000 ]
## Configure user with a different USERID if requested.
then
echo "creating new $USER with UID $USERID"
useradd -m $USER -u $USERID
mkdir /home/$USER
chown -R $USER /home/$USER
You are going to have to wait for user namespace support, hopefully later this year.
I am new to docker, so if this is a fairly obvious process that I am missing, I do apologize for the dumb question up front.
I am setting up a continuous integration server using the jenkins docker image. I did a docker pull jenkins, and created a user jenkins to allow me to mount the /var/jenkins_home in the container to my host's /var/jenkins_home (also owned by jenkins:jenkins user).
the problem is that the container seems to define the jenkins user with uid 102, but my host has the jenkins user as 1002, so when I run it I get:
docker run --name jenkins -u jenkins -p 8080 -v /var/jenkins_home:/var/jenkins_home jenkins
/usr/local/bin/jenkins.sh: line 25: /var/jenkins_home/copy_reference_file.log: Permission denied
I would simply make the uid for the host's jenkins user be 102 in /etc/passwd, but that uid is already taken by sshd. I think the solution is to change the container to use uid 1002 instead, but I am not sure how.
Edit
Actually, user 102 on the host is messagebus, not sshd.
Please take a look at the docker file I just uploaded:
https://github.com/bdruemen/jenkins-docker-uid-from-volume/blob/master/Dockerfile .
Here the UID is extracted from a mounted volume (host directory), with
stat -c '%u' <VOLUME-PATH>
Then the UID of the container user is changed to the same value with
usermod -u <UID>
This has to be done as root, but then root privileges are dropped with
gosu <USERNAME> <COMMAND>
Everything is done in the ENTRYPOINT, so the real UID is unknown until you run
docker run -d -v <HOST-DIRECTORY>:<VOLUME-PATH> ...
Note that after changing the UID, there might be some other files no longer accessible for the process in the container, so you might need a
chown -R <USERNAME> <SOME-PATH>
before the gosu command.
You can also change the GID, see my answer here
Jenkins in docker with access to host docker
and maybe you want to change both to increase security.
You can simply change the UID in /etc/passwd, assuming that no other user has UID 1002.
You will then need to change the ownership of /var/jenkins_home on your host to UID 1002:
chown -R jenkins /var/jenkins_home
In fact, you don't even need a jenkins user on the host to do this; you can simply run:
chown -R 1002 /var/jenkins_home
This will work even if there is no user with UID 1002 available locally.
Another solution is to build your own docker image, based on the Jenkins image, that has an ENTRYPOINT script that looks something like:
#!/bin/sh
chown -R jenkins /var/jenkins_home
exec "$#"
This will (recursively) chown /var/jenkins_home inside the container to whatever UID is used by the jenkins user (this assumes that your Docker contains is starting as root, which is true unless there was a USER directive in the history of the image).
Update
You can create a new image, based on (FROM ...) the jenkins image, with a Dockerfile that performs the necessary edits to the /etc/passwd file. But that seems a lot of work for not much gain. It's not clear why you're creating jenkins user on the host or if you actually need access to the jenkins home directory on the host.
If all you're doing is providing data persistence, consider using a data volume container and --volumes-from rather than a host volume, because this will isolate the data volume from your host so that UID conflicts don't cause confusion.
I had the same error, I turned SELinux off (on CEntOS) and it works.
Otherwise, it woukd be better to tune SElinux with SEManage commands.
The ideal is to change the user UID in your Dockerfile used by jenkins with the same UID used by the Host (remember that it must be done for non-root users, if root create a new user and configure the service inside the container to that user).
Assuming the user's UID on the host is 1003 and the user is called jenkins (use $id to get the user and group id).
Add to your Dockerfile
# Modifies the user's UID and GID
RUN groupmod -g 1003 jenkins && usermod -u 1003 -g 1003 jenkins
# I use a group (docker) on my host to organize the privileges,
#if that's your # case add the user to this group inside the container.
RUN groupadd -g 998 docker && usermod -aG docker nginx
I have a docker container with a -v /home/dan:/home/dan:rw. When the container writes files to /home/dan, the files are owned by root in the host filesystem. Is there a way to make it so that files written from the container to the mounted volume are owned by some arbitrary user on the host filesystem?
As François Zaninotto has pointed out, the user id can be used.
This can be done by using the -u switch for the docker run command.
For example:
docker run -v /home/dan:/home/dan -u `id -u $USER` IMAGE
EDIT: this has changed since my original answer which said it couldn't be done. As per answer of Mandark:
This can be done by using the -u switch for the docker run command.
For example:
docker run -v /home/dan:/home/dan -u `id -u $USER` IMAGE
A follow up to mandark answer - I would say it's also good to include the user group otherwise you will end up with stuff belonging to user: USER and group: root. To achive user:user just pass in group id as well, for example:
docker run -v /home/dan:/home/dan -u `id -u $USER`:`id -g $USER` IMAGE
# if it's for the current user, then you can omit the $USER env var
docker run -v /home/dan:/home/dan -u `id -u`:`id -g` IMAGE
It's possible. It's hard to automate, but it's possible. Here is the process:
in the host, determine the current user id and group id
in the docker container, run a shell script to:
add a new group with the group id from the host
add a new user with the same user id from the host (and belonging to the group just created)
sudo as this new user
Now, each file generated inside the container will be using the right user id and group id, and the host will attach them to your user.
I've written a tool to automate that using make, it's called make-docker-command. Hope this helps.