Even after going through lot of materials and SO answers still I'm not clear on docker uid/user usage or implementation.
I understand the below points:
An instance of an image is called a container.
uid/gid is maintained by the underlying kernel, not by Container.
Kernel understand uid/gid number not username/groupname and name is an alias and just for human readable.
All containers are processes maintained by docker daemon and will be visible as process in host machine (ps -ef)
root (id = 0) is the default user within a container and this can be changed either by USER instruction in Dockerfile or by passing -u flag in docker run
With the above all said, when I have the below command in my Dockerfile, I presume that a new user (my-user) will be created with incremented uid.
RUN addgroup my-group && adduser -D my-user -G my-group
What happens if I run the same image multiple times i.e multiple containers? Will the same uid be assigned to all processes?
What happens if I add same above command in another image and run that image as container? - will I get new uid or same uid as the previous one?
How the uid increment happens in Container in relation with the host machine.
Any pointers would be helpful.
Absent user namespace remapping, there are only two things that matter:
What the numeric user ID is; and
What's in the /etc/passwd file.
Remember that each container and the host have separate filesystems, so each of these things could have separate /etc/passwd files.
What happens if I run the same image multiple times i.e multiple containers? Will the same uid be assigned to all processes?
Yes, because each container gets a copy of the same /etc/passwd file from the image.
What happens if I add same above command in another image and run that image as container? - will I get new uid or same uid as the previous one?
It depends on what adduser actually does; it could be the same or different.
How the uid increment happens in Container in relation with the host machine.
They're completely and totally independent.
Also remember that you can docker push/docker pull a built image to run it on a different host. That will bring the image's /etc/passwd file along with it, but the host environment could be totally different. Correspondingly, it's not a best practice to try to match some specific host's uid mapping in a Dockerfile, because it will be wrong if you try to run the same image anywhere else.
When you try to add users in the RUN statement, it does not create a user on the host. If you do not specify an user with the USER statement in your Dockerfile or the -u flag while starting container (Assuming the parent Dockerfiles also do not include the USER statement), the container process on host will simple run as root user if you have started the docker daemon as root.
So if you create a user using RUN addgroup my-group && adduser -D my-user -G my-group it will simply create an user in the container i.e. the user is local to the container. So each instance (container) of that image you run will have the same uid of the user inside the container. Note: That user will not exist on the host.
If you want to run the container process on host as another user (which exists on host) then you have 3 options:
Add a USER statement in the Dockerfile
Use the -u flag while running the container
You can use docker's user namespace feature
I highly recommend understanding the user namespace and mappings by reading this documentation: Isolate containers with a user namespace
Related
I understand that it's considered a bad security practice to run Docker images as root, but I have a specific situation that I wanted to pass by the community to see if anyone can help.
We are currently using a pipeline on an Amazon Linux 2 instance with a single user called ec2-user. Unfortunately, a lot of the scripts we're using for our pipeline have hard-coded paths baked in (notably /home/ec2-user/) ... which may or may not reference the $HOME variable.
I've been talking to one of the engineers that is building a Docker image for our pipeline and suggested that he creates a new user entirely so root user isn't running our pipeline.
For example:
# add clip user
RUN groupadd-r clip && useradd -r -g clip clip
# disable root
RUN chsh -s /usr/sbin/nologin root
# set environment variables
ENV HOME /home/clip
ENV DEBIAN FRONTEND-noninteractive
However, the engineer mentioned that the clip user inside the container will have some uid that may or may not exist in the host machine. For example, if the clip user had uid 1001 in the container, but 1001 was john in the host, all the files created as the clip user inside the container would be owned by john on the outside.
Further, he is more concerned about the situation where the clip user has a uid in the container that doesn’t exist in the host’s passwd. In that case files created by the clip user in the container would be owned by a bare unassociated uid on the host.
If we decided to pass in ids from the host as the user/group to run the image. The kernel will be ok with it (same kernel as the host), and when all is said and done files created inside the container will then be owned by the user/group you pass in. However, the container wouldn’t know who that user/group are, so it’ll just use the raw ids, and stuff like $HOME or whoami won’t work.
With that said, we're curious if anyone else has experienced these problems and if anyone has found solutions?
Everything you say is totally normal. The container has its own /etc/passwd file, and so a given numeric user ID might map to different user names (or to not at all) in the host and in the container. Beyond some cosmetic issues around debug shells, it shouldn't usually matter if the current numeric uid is actually present in the container /etc/passwd, and there's no reason a container uid would need to be mapped in the host /etc/passwd.
Note that there are a couple of ways to directly assume another user ID in Docker, either using the docker run -u option or the Dockerfile USER directive. The RUN chsh command you propose doesn't really do anything and doesn't prevent becoming root inside a container.
clip user inside the container will have some uid that may or may not exist in the host machine.
True, totally normal.
For example, if the clip user had uid 1001 in the container, but 1001 was john in the host, all the files created as the clip user inside the container would be owned by john on the outside.
This is partially true, but only in the case where you've explicitly mapped a host directory into the container with a docker run -v option. Otherwise, the host user with uid 1001 won't be able to navigate to the /var/lib/docker/... directory that actually contains the container files, so it doesn't matter that they could hypothetically write them.
The more usual case around this is to explicitly supply a host uid so that the container process can save its state in a mapped host directory. Pass a numeric uid to the docker run -u option; there's no particular need for that uid to exist in the container's /etc/passwd.
docker run \
-u $(id -u) \
-v "$PWD/data:/data" \
...
the container wouldn’t know who that user/group are, so it’ll just use the raw ids, and stuff like $HOME or whoami won’t work.
Unless your application explicitly calls these things, they won't usually matter. "Home directory" is a pretty poorly defined concept in a Docker container since it's usually a wrapper around a single process.
I am running a docker image for Juypter and tensorboard. The data seem to get deleted everytime the VM instance is stopped is there away to stop this from happening i could find anything on the web that would allow me to do this?
TL;DR: You are not persisting your data.
Docker containers does not persist data out of the box, you need to explicity tell docker to persist any data created inside the container when the container is deleted.
You can read more at Use volumes page at Docker documentation.
If you want to persist data you need to do the next steps:
Create a local volume inside the VM where you want to persist data. This command should be executed on the GCE instance
mkdir -p /opt/data/jupyterdata
Set the correct ownership of the folder to the user id that the user inside your container uses. For example, let's imagine that your container lspvic/tensorboard-notebook run the application using the user tensorflow with the UID 1500. So you need to set the ownership of your folder to the UID 1500:
chown 1500:1500 /opt/data/jupyterdata -R
Modify your docker run command to mount the local directory as a volume inside the container. For example, lets imagine that inside your container you want to save the files at /var/lib/jupyter (this is an example), you will need to modify the docker run command as follows:
docker run -it --rm -p 8888:8888 \
-v /opt/data/jupyterdata:/var/lib/jupyter:Z \
lspvic/tensorboard-notebook
NOTE: the :Z parameter is needed to avoid SELINUX issues
With this steps now your data saved on folder /var/lib/jupyter inside the container will be saved on /opt/data/jupyterdata inside the VM so no more data loss.
Actually, I am trying to run the below following command
docker run -it --rm -v $(pwd):/var/www/html --user node node:12.13.1-alpine ash.
Expected result
The files inside the container (i.e /var/www/html ) should have user as node.
Actual result
But, the files inside the containers are showing the same user as of the host.
Also, can't create a directory inside the container.
It is working for my other colleagues. So, any help in this would be much appreciated.
Many thanks,
Alwin
Note:
Docker version 19.03.7, build 7141c199a2
Have added necessary permission to docker command so that it doesn't
need sudo for running it
Running docker run with --user does not change the permission of the original existing files. From Docker reference:
The developer can set a default user to run the first process with the Dockerfile USER instruction. When starting a container, the operator can override the USER instruction by passing the -u option.
It only overrides the user running Node.js inside the container. During mount, the original permission and owner of /var/www/html is unchanged. Verify this by ls -n and see if the UID of the owner of the folder is the same when mounted inside Docker. Make sure the UID is the same as node user you specified.
I don't know how it works in your colleagues computers though. That's why it's important to use UID/GID instead just using username. The same username in the container can have different UID with the same username in the host.
EDIT: I checked that node image that you use contains node user with UID 1000. The first user created in Linux usually also has UID 1000. So if the /var/www/html is owned by UID 1000, it will run. But UID 1000 could possibly belong to different usernames in Docker and in the host. Because you specified --user node, which is translated into UID 1000 inside the container as username node itself exists, it won't work if /var/www/html is owned by different UID in your host, which probably is your case.
You have to add USER into Dockerfile before building
# App is running normal user mode
USER node
so now when you run docker image it will run with normal node user mode
I currently run the official Tensorflow Docker Container (GPU) with Nvidia-Docker:
https://hub.docker.com/r/tensorflow/tensorflow/
https://gcr.io/tensorflow/tensorflow/
However, I can't find a way to set a default user for the container. The default user for this container is "root", which is dangerous in term of security and problematic because it gives root access to the shared folders.
Let's say my host machine run with the user "CNNareCute", is there any way to launch my containers with the same user ?
Docker containers by default run as root. You can override the user by passing --user <user> to docker run command. Note however this might be problematic in case the container process needs root access inside the container.
The security concern you mention is handled in docker using User Namespaces. Usernamespaces basically map users in the container to a different pool of users on the host. Thus you can map the root user inside the container to a normal user on the host and the security concern should be mitigated.
AFAIK, docker images run by default as root. This means that any Dockerfile using the image as a base, doesn't have to jump through hoops to modify it. You could carry out user modification in a Dockerfile - same way you would on any other linux box which would give you the configuration you need.
You won't be able to use users (dynamically) from your host in the containers without creating them in the container first - and they will be in effect separate users of the same name.
You can run commands and ssh into containers as a specific user provided it exists on the container. For example, a PHP application needing commands run that retain www-data privileges, would be run as follows:
docker exec --user www-data application_container_1 sh -c "php something"
So in short, you can set up whatever users you like and use them to run scripts but the default will be root and it will exist unless you remove it which may also have repercussions...
Docker kind of always had a USER command to run a process as a specific user, but in general a lot of things had to run as ROOT.
I have seen a lot of images that use an ENTRYPOINT with gosu to de-elevate the process to run.
I'm still a bit confused about the need for gosu. Shouldn't USER be enough?
I know quite a bit has changed in terms of security with Docker 1.10, but I'm still not clear about the recommended way to run a process in a docker container.
Can someone explain when I would use gosu vs. USER?
Thanks
EDIT:
The Docker best practice guide is not very clear: It says if the process can run without priviledges, use USER, if you need sudo, you might want to use gosu.
That is confusing because one can install all sorts of things as ROOT in the Dockerfile, then create a user and give it proper privileges, then finally switch to that user and run the CMD as that user.
So why would we need sudo or gosu then?
Dockerfiles are for creating images. I see gosu as more useful as part of a container initialization when you can no longer change users between run commands in your Dockerfile.
After the image is created, something like gosu allows you to drop root permissions at the end of your entrypoint inside of a container. You may initially need root access to do some initialization steps (fixing uid's, host mounted volume permissions, etc). Then once initialized, you run the final service without root privileges and as pid 1 to handle signals cleanly.
Edit:
Here's a simple example of using gosu in an image for docker and jenkins: https://github.com/bmitch3020/jenkins-docker
The entrypoint.sh looks up the gid of the /var/lib/docker.sock file and updates the gid of the docker user inside the container to match. This allows the image to be ported to other docker hosts where the gid on the host may differ. Changing the group requires root access inside the container. Had I used USER jenkins in the dockerfile, I would be stuck with the gid of the docker group as defined in the image which wouldn't work if it doesn't match that of the docker host it's running on. But root access can be dropped when running the app which is where gosu comes in.
At the end of the script, the exec call prevents the shell from forking gosu, and instead it replaces pid 1 with that process. Gosu in turn does the same, switching the uid and then exec'ing the jenkins process so that it takes over as pid 1. This allows signals to be handled correctly which would otherwise be ignored by a shell as pid 1.
I am using gosu and entrypoint.sh because I want the user in the container to have the same UID as the user that created the container.
Docker Volumes and Permissions.
The purpose of the container I am creating is for development. I need to build for linux but I still want all the connivence of local (OS X) editing, tools, etc. My keeping the UIDs the same inside and outside the container it keeps the file ownership a lot more sane and prevents some errors (container user cannot edit files in mounted volume, etc)
Advantage of using gosu is also signal handling. You may trap for instance SIGHUP for reloading the process as you would normally achieve via systemctl reload <process> or such.