Well I'm running a container which will create/modify some files that I need to persist outside the life-cycle of the container. As a result, I am voluming a folder into the container for that purpose:
docker run -v $(pwd)/data:/app/data my-image
On the first run, $(pwd)/data does not exist, so Docker will create it for me. However, it creates it as the user its own daemon process is being executed as, which is root. This is problematic since the user within the container obviously is not root.
To fix this permission issue, I have created a user group in the host machine with the same ID as the user within the container, and have given them access to the parent folder of $(pwd)/data, and added group permission settings on the folder (chmod g+s). So if I run the following command as root:
sudo mkdir whatver
the folder whatever will be modifiable by that group and by extension, by the container's user. however when Docker creates the folder $(pwd)/data, it still is created as belonging to the group root, and again the user within the container cannot modify data inside it.
Right now, I have worked around this by making the required folders before running the container. However this is a dirty work-around, and I am looking for a cleaner solution. Has anyone else faced this issue? Is it an issue with Docker not respecting the group permission settings on the parent folder or am I missing something here / doing something wrong?
I don't consider making the folders first a dirty workaround, that's a best practice to me. Docker is running a bind mount for you, and this will fail when the source directory does not exist. For user convenience, docker will create this directory for you when you make a host mount and the directory is missing, but there's no need to use this functionality and you'll find other types of mounts will not perform the directory creation for you (e.g. the --mount syntax often used by swarm mode).
You can start your container as root and fix the permissions on the directory with an entrypoint, before running something like an exec gosu app_user app_server at the end to drop from root to the user. I do this in the entrypoint in my docker-base images, which is useful for a developer workflow where developers often need to dynamically update the container to work with their host environment.
Otherwise, you can switch to named volumes. These will be initialized when they are new or empty using the directory contents and permissions from the image. They are a best practice when you need persistence but not direct filesystem access to the contents of the volume from users on the host. You would instead access the named volume from inside of containers that mount the volume.
Related
My situation is the following:
I am having a docker image/container in which I am compiling. I had to install some components to $HOME via the Dockerfile (so while creating the image).
Let's say one of those components is in ~/.config, but also other folders.
I would like to have the possibility to override the files in .config by mounting a home folder from the host on top of the one inside docker. Whenever you place a file in the mounted folder, it overrides the one which is already inside the container.
So in theory, this is exactly what an OverlayFS does, right? While the lower directory would be the one inside the Docker container, the upper directory would be the one on my Host.
Is there a way to accomplish that?
Until now I found the following related topics:
https://serverfault.com/questions/841238/how-to-use-overlayfs-with-docker-volumes
Drawback: The answer does only show how to use overlayfs on the host, but getting acccess to the lower container/image directory is not that self-explaining and also feels dirty.
Can I mount docker host directory as copy on write/overlay?
Drawback: Using mount -t overlay inside docker does not work on newer kernels because of the disabled overlay-on/over-overlay option
I also thought about manipulating the docker files on host directly, i.e. the directories where docker stores the files, but that feels a bit dirty.
To do so, I would declare VOLUME /home/user at the end of the Dockerfile. Then I would find my files of that directory in /var/lib/docker/volumes/user/_data. I could then create a overlayfs on my host, using that directory as lower, my other folder as upper. I could then remount that new directory using docker run --volume. Unfortunately this would involve su rights to access the /var/lib directory.
The other way around would be to bind-mount single files, but that's maybe a bit hackish too.
I have a docker-based build environment - in order to build my project, I run a docker container with the --volume parameter, so it can access my project directory and build it.
The problem is that the files created by the container cannot be deleted by the host machine. The only workaround I currently have is to start an interactive container with the directory mounted and delete it.
Bottom line question: It is possible to make docker write to the mounted area files with permissions such that the host can later delete them?
This has less to do with Docker and more to do with basic Unix file permissions. Your docker containers are running as root, which means any files created by the container are owned by root on your host. You fix this the way you fix any other file permission problem, by either (a) ensuring that that the files/directories are created with your user id or (b) ensuring that permissions allow you do delete the files even if they're not owned by you or (c) using elevated privileges (e.g., sudo rm ...) to delete the files.
Depending on what you're doing, option (a) may be easy. If you can run the contanier as a non-root user, e.g:
docker run -u $UID -v $HOME/output:/some/container/path ...
...then everything will Just Work, because the files will be created with your userid.
If the container must run as root initially, you may be able to take care of root actions in your ENTRYPOINT or CMD script, and then switch to another uid to run the main application. To do this, you would need to pass your user id into the container (e.g., as an environment variable), and then later use something like runuser to switch to the new userid:
exec runuser -u $TARGE_UID /some/command
If neither of the above is an option, then sudo rm -rf mydirectory should work just as well as spinning up an interactive container.
If you need your build artifacts just to put them to the docker image on the next stage then it is probably worth to use multi-stage build option.
I'm setting up docker-compose for my php project on my mac. I mount a shared volume with all the code from host machine into container.
Obviously, I do not want to run the php container as root user. But do I have another option?
I tried:
Changing owner of the project files in Dockerfile
Changing owner in entrypoint
Both methods work fine, until you create a file in IDE - in this case the file appears to be owner by root inside container and then you need to restart the container (which is horrible experience for development)
Changing UID for user www-data in container to UID from my host user.
It didn't work, since the files are owned by root
Do I miss any points here?
Lets say I have a container running with a non-root user and I want to bind-mount a volume directory from the host into that container. The container then will write to that directory. Say, the directory on the host is /tmp/container/data. If that path does not exist on the host, I observe that it gets created (by docker) with ownership root. As a consequence the container is not able to write anything into that directory (access denied) because my container is not running with user root.
Of course I can take care of creating the /tmp/container/data directory with correct permissions on the host side before starting the container, but this solution obviously does not scale - I will have to do it for each and every container where I want to use a bind volume from the host for which the directory does not exist.
So my question is, what's the best way to use bind-volumes from the host for directories that do not yet exist while still let a non-root container have write access to the volume.
You accurately described the normal behavior of docker, non-existent bind mounts from the docker engine will get initialized to an empty directory owned by root. Note that this doesn't happen with swarm mode, it will fail to schedule the container on the host instead.
Options to use to avoid this include:
Using named volumes. These get initialized to the directory permissions in the image at that location. This is as easy as changing the full path on the host to a short name of the volume.
Run the container as root, and make the entrypoint fix the permissions and drop to the user before launching the application. Something similar to this is done in a jenkins-docker project I threw out on github recently.
Include a script in the container with permissions setuid-root which performs the chown of the directory.
I'm working with a container that will be running ZooKeeper but I'm running into issues with permissions on the host volumes that I mount into my container.
This is my setup:
On the host machine (Ubuntu 14.04):
Created a "zookeeper" system user (id=106) and group (id=111).
Created the directory "/var/log/zookeeper" and set its ownership to zookeeper (ie. chown zookeeper:zookeeper). This is the directory that I will be mounting into my container.
Inside the container (Ubuntu 14.04):
Also created a "zookeeper" system user (id=102) and group (id=105) which I use as the user from which to execute the command in the ENTRYPOINT.
Create the same directory "/var/log/zookeeper" that will be mounted to and also set its ownership to zookeeper:zookeeper (although I don't think this matters).
Once I start up my container with the /var/log/zookeeper mount and I open up a shell inside of the container as the zookeeper user (that was created inside the container), I find that I get a "Permission Denied" error if I try to create a file in the mounted directory /var/log/zookeeper. When I do an "ls -l" to look at the ownership of this directory (still inside of the container) it looks something like this:
drwxr-xr-x 2 106 111 4096 Jun 30 17:18 zookeeper
The 106 and 111 in this case correspond to the zookeeper user and group ids of the host machine which I think is where the issue lies. I tried opening a shell inside of the container but this time I went in as the root user and the scenario that I described above worked perfectly fine just that root was the owner of the file that got created (which was expected).
From this I concluded that I need to either:
(a) Run the application inside of my container as the default root user instead of the zookeeper user that I create.
(b) Create a zookeeper user and group both on my host machine and inside the container whose id's are exactly the same.
Neither case is ideal because for (a), running the application as the root user could have potential security issues (from what I've read anyways), and for (b), it can be very difficult to get the id's to match due to the fact that they might already be taken by other users that were created (which you don't have any control over).
Has anyone ever dealt with something like this before? Are there other possible solutions that I might be overlooking?
To my knowledge, user ID and group ID inside the container and on the host machine should match, in order to let the host machine to grant you permissions to the share directory.
Very important to see the difference between running a production and a development container.
Afaik, there's no real issue if your Docker container runs as root, even on production. However you should never want or need to mount a volume of production. If you want to run it as a zookeeper feel free to do so.
// Edit: The more I read, the more I'm convinced there actually might be a security issue when running stuff as root, so you better not doing so on production.
The solution to try and match uid and gid is viable only for small/local project - it really does make it unportable. You can try and set an arbitrary high uid and gid and then do the same on each of your devs machines, but that doesn't mean it'll always be fine.
tl;dr: On development run chmod -R 0777 on existing files and then umask 0000 to setup permissions on files and directories created later. Then you can mount and edit your files as you please, no matter what user created it.