I thought that docker doesn't namespace uids and gids, i.e. two users, one on the host and one in the container, with different names but the same uid are actually the same user as there's only a single kernel. so they must have identical permissions.
But when I run a container which has a line like:
RUN groupadd -g 1001 appuser && \
useradd -r -u 1001 -g appuser appuser
...in the Dockerfile, how come I can't see the uid or gid values of 1001 in the output of /etc/passwd or /etc/group resp. on the host?
Related
I'm trying to run a simple command with docker that read a file in my home directory and write an output file in the same directory on my local machine (/home/userxyz/projects/data). The data folder belongs to me (userxyz) and my group which has the same name (userxyz) with R/W permissions.
I built an image with a Dockerfile like so (following How to add users to Docker container?):
FROM osgeo/gdal:alpine-small-latest
RUN mkdir /usr/tmp
RUN addgroup -S userxyz \
&& adduser -S -D -h /usr/tmp userxyz userxyz \
&& chown -R userxyz:userxyz /usr/tmp
USER userxyz
WORKDIR /usr/tmp
and:
docker build -t userxyz/test .
I ran docker with:
docker run --rm -v /home/userxyz/projects/data:/usr/tmp userxyz/test gdal_translate -ot UInt32 /usr/tmp/input.tif /usr/tmp/output.tif
However, I still get:
ERROR 4: Attempt to create new tiff file `/usr/tmp/output.tif' failed: Permission denied
It works when I only read the file:
docker run --rm -v /home/userxyz/projects/data:/usr/tmp userxyz/test gdalinfo /usr/tmp/input.tif
Any idea what could be wrong with the writing permissions?
EDIT: when opening the data folder to read/write for "Others", the file is correctly created by user#500100, so I guess it's something with username attribution? How can I do so that the file is created by userxyz and not user#500100?
Although you set the same user & group in Dockerfile with userxyz:userxyz, but host rootfs & container rootfs are 2 different rootfs. This means the UID & GID for same username/group name still be different.
As a result, just copy username, groupname is not enough, you also need to copy UID, GID, something like next:
Dockerfile:
FROM ubuntu:16.04
ARG UID
ARG GID
RUN groupadd -g $GID userxyz
RUN useradd -m -u $UID -g $GID -s /bin/bash userxyz
Build command:
docker build -t abc:1 --build-arg UID=`id userxyz -u` --build-arg GID=`id userxyz -g` .
Explain:
id userxyz -u & id userxyz -g get the UID/GID on host, and pass them to build container, then docker build use them to set the same UID/GID with the host for userxyz. This will assure in container you have same permission with the host.
You must add the user in your group based on the group ID not on the group name. User and group names are isolated in the container, so it won't work.
You can find the group ID in the /etc/group file, using this command on the host machine:
cat /etc/group | grep userxyz
Once found, you add the user to the group in the Dockerfile:
FROM osgeo/gdal:alpine-small-latest
RUN mkdir /usr/tmp
RUN groupadd -g $GID userxyz \
&& adduser -S -D -h /usr/tmp userxyz userxyz \
&& chown -R userxyz:userxyz /usr/tmp
USER userxyz
WORKDIR /usr/tmp
where you replace $GID by the the group ID.
I'm not asking how to create a rootless container from scratch. Rather, I've been given some software deployed as pre-built Docker container images that run as root by default. I need to modify these containers so they can be deployed on Kubernetes, which means I need to make these containers rootless. To be clear, I DO NOT have the source to these containers so I can't simply rebuild them from scratch.
I've found plenty of articles about building rootless containers in general, but they all assume you're building your containers from scratch. I've spent hours searching but can't find anything about modifying an existing container to be rootless.
I realize this might be a very open question, but I don't know all the things I need to take into consideration. Two things I've been able to gather is adding a line such as USER 1000 to Dockerfile, and adjusting ownership and permissions on certain files and folders. Beyond that, I'm not sure what I need to do.
Create users in the container and switch users;
Add a new user, named user;
Let this user have root privileges;
Set its password to password;
After the Container is started, log in as user and go directly to the user's home directory;
Put the following code snippet in the Dockerfile.
RUN useradd --create-home --no-log-init --shell /bin/bash user \
&& RUN adduser user sudo \
&& RUN echo 'user:password' | chpasswd
USER user
WORKDIR /home/user
Use fixuid to modify the uid and gid of non-root users in the container;
After creating a non-root user with the above code, the user's uid and gid are generally 1000:1000.
Docker and the host share a set of kernel, and there is still only one set of uid and gid controlled by the kernel. In other words, we execute the process as a newly created docker user (uid 1000) in the container, and the host will think that the process is executed by a user with a uid of 1000 on the host, and this user may not necessarily be our account, which is equivalent to us A user who has replaced someone else with an impostor makes it difficult to trace back to the real user.
To solve this problem, you can specify the uid as the user's uid when adding the user, such as 1002;
RUN addgroup --gid 1002 docker && \
adduser --uid 1002 --ingroup docker --home /home/docker --shell /bin/sh --gecos "" docker
A better solution is to use fixuid to switch the uid when the container starts:
RUN useradd --create-home --no-log-init --shell /bin/bash user\
&& adduser user sudo \
&& echo 'user:password' | chpasswd
RUN USER=user && \
GROUP=docker && \
curl -SsL https://github.com/boxboat/fixuid/releases/download/v0.4.1/fixuid-0.4.1-linux-amd64.tar.gz | tar -C /usr/local/bin -xzf - && \
chown root:root /usr/local/bin/fixuid && \
chmod 4755 /usr/local/bin/fixuid && \
mkdir -p /etc/fixuid && \
printf "user: $USER\ngroup: $GROUP\n" > /etc/fixuid/config.yml
USER user:docker
ENTRYPOINT ["fixuid"]
At this time, you need to specify the uid and gid when starting the container. The command is as follows:
docker run --rm -it -u $(id -u):$(id -g) image-name bash
On my host my uid and gid are:
$ id ravi
uid=1000(ravi) gid=100(users)
i want to mount a folder host_test with uid:gid = 1000:100 into docker
I have the following docker and its id
$ docker run --rm -it python:3.7.9-buster /bin/bash
root#48d7d3122ffd:/# id root
uid=0(root) gid=0(root) groups=0(root)
root#48d7d3122ffd:/#
After going through various solutions, the best solution i see is to create a user and group same as host on docker image.
i.e
Dockerfile_mypc
FROM python:3.7.9-buster
RUN groupadd --gid 100 users \
&& useradd --uid 1000 --gid users --create-home ravi
WORKDIR /home/ravi
USER ravi
this now solves the problem of permissions
but when I want to use this Dockerfie on aws ubuntu instance, I see there the user ubuntu has 1000:1000 has uid and gid. So again i have to change the Dockerfile
Dockerfile_aws
FROM python:3.7.9-buster
RUN groupadd --gid 1000 users \
&& useradd --uid 1000 --gid users --create-home ubuntu
WORKDIR /home/ubuntu
USER ubuntu
I am just looking what is the best practise. Should i make Dockerfiles for each system where ever i am going to setup like
Dockerfile_mypc
Dockerfile_aws
or is there any way i can do this with single dockerfile
Because i have to mount the host folder from my pc to docker in development and also mount host folder from aws ubuntu instance to docker in production
I try to start a new container from ubuntu 18.04 docker image. I do as follows:
pull the docker image
docer pull ubuntu:18.04
create a new container
docker run -ti -v $(pwd):/home/shared --name ubuntu_test ubuntu:18.04
and then log out.
start the created container
docker start ubuntu_test
log in as root user, update OS and install vim
docker exec -ti ubuntu_test /bin/bash and apt update, apt install -y vim
then log out.
log in as non-root user
docker exec -ti -u daemon ubuntu_test /bin/bash
Then I found that I have no permission to create new files or new folders.
I do not want to log in as root user since there could be some problems with mpirun.
Is there any solution for this problem ? Thank you for any help.
It is not like you (non-root user) don't have permissions to write or read. It is that everything on the system (files/folders) belong to the root user and no other user can modify anything by default.
You can create a new user as well as home folder for that user when you are building the image and then the user will be able to modify stuff in its home (standard linux stuff).
Example Dockerfile
FROM ubuntu
RUN groupadd --gid 1000 someuser \
&& useradd --uid 1000 --gid someuser --shell /bin/bash --create-home someuser
test with
docker build -t utest .
docker container run -it -u someuser utest /bin/bash
cd /home/someuser
touch myfile
If you need to add some other folders under that user's administration other than its home, you can use chown -R someuser:someuser <folder> which will recursively change ownership of the specified folder and everything in it to that of the new user.
Example: changing ownership of /etc folder
FROM ubuntu
RUN groupadd --gid 1000 someuser \
&& useradd --uid 1000 --gid someuser --shell /bin/bash --create-home someuser
RUN chown -R someuser:someuser /etc
I have a docker with a php application on it
I have a share volume, for example
/home/me/dev/site <=> /var/www/site
I can write something in my host, it will be sync with the container
if I launch
sudo docker exec test touch /var/www/site/test.txt
It works
But if my server is trying to create a file as www-data this is not working because of the rights.
Is there a way to give access to my shared volumes to www-data ?
I am using boot2docker
(Bind-mounted) volumes in Docker will maintain the permissions that are set on
the Docker host itself. You can use this to set the permissions on those
files and directories before using them in the container.
Some background;
Permissions in Linux are based on user and group ids ('uid' / 'gid'). Even
though you see a user- and group name as owner, those names aren't actually
important in Linux, they are only there to make it easier for you to see who's the owner of a file (they are looked up from the /etc/passwd file).
You can set any uid/gid on a file; a user doesn't have to exist when setting those permissions. For example;
touch foobar && sudo chown 1234:5678 foobar && ls -la foobar
# UID and GID are set to 1234 / 5678, even though there's no such user
-rw-rw-r-- 1 1234 5678 0 Mar 25 19:14 foobar
Checking permissions (inside and outside a container)
As mentioned above, Docker maintains ownership of the host when using
a volume. This example shows that permissions and ownership in the volume are the
same outside and inside a container;
# (First create a dummy site)
mkdir -p volume-example/site && cd volume-example
echo "<html><body>hello world</body></html>" > site/index.html
# Permissions on the Docker host;
ls -n site
# total 4
# -rw-rw-r-- 1 1002 1002 38 Mar 25 19:15 index.html
# And, permissions inside a nginx container, using it as volume;
sudo docker run --rm -v $(pwd)/site:/var/www nginx ls -n /var/www
# total 4
# -rw-rw-r-- 1 1002 1002 38 Mar 25 19:15 index.html
Setting the permissions
As explained, a user doesn't have to exist in order to use them, so even if
we don't have a www-data user on the Docker host, we can still set the correct
permissions if we know the "uid" and "gid" of that user inside the container;
Let's see what the uid and gid of the www-data user is inside the container;
sudo docker run --rm nginx id www-data
# uid=33(www-data) gid=33(www-data) groups=33(www-data)
First check the state before changing the permissions. This time we
run the nginx container as user www-data;
sudo docker run \
--rm \
--volume $(pwd)/site:/var/www \
--user www-data nginx touch /var/www/can-i-write.txt
# touch: cannot touch `/var/www/can-i-write.txt': Permission denied
Next, set the permissions on the local directory, and see if we are able to write;
sudo chown -R 33:33 site
sudo docker run \
--rm \
--volume $(pwd)/site:/var/www \
--user www-data nginx touch /var/www/can-i-write.txt
Success!
Add the following lines to your dockerfile and rebuild your image
RUN usermod -u 1000 www-data
RUN usermod -G staff www-data