I have a Dockerfile which specifies a USER and executes a script as that user. The script requires an file mounted in that user's home directory. Assuming I don't know what that user is or what its home directory is (and want to keep it dynamic instead of using docker inspect and manually entering it), is it possible to mount a file in the container with the destination being relative to USER's home?
i.e. docker run -v $PWD/file:somedir/file <image>
where $PWD/file on the host ends up mounted at ~USER/somedir/file in the container.
This currently gives docker: Error response from daemon: invalid volume specification: '$PWD/file:somedir/file': invalid mount config for type "bind": invalid mount path: 'somedir/file' mount path must be absolute.
Docker images generally have a fixed filesystem layout. They don't typically have "multiple users", "home directories", or variable paths, the way you might on a standard server setup.
For the sort of setup you describe, I might suggest:
Install your application in some easy-to-find directory like /app. (Pick a path; don't have it be an environment variable.)
Have the application and its files be owned by root and not world-writeable.
Have the image install some non-root user. It doesn't matter what that user is, and it doesn't need to match any particular host user.
The application should expect some easy-to-find directory like /data.
When you run the application, specify both the host user ID and the content to mount on /data.
FROM ???
WORKDIR /app
COPY . . # will be owned by root (and that's okay)
RUN adduser user # can be any name and any non-0 uid
RUN mkdir /data && chown user /data
ENV DATA_DIR=/data # to tell the application where it is
USER user
CMD ["/app/the_application"]
sudo docker run --rm \
-v "$PWD/content":/data \
-u $(id -u) \
the-image
It's important to keep the application and data separate, so that the bind-mount doesn't overwrite the application data. You wouldn't usually pass any host-specific data into the build process so that the built image can be reused in different environments.
(If the application is really just a script, it's not a long-running process, and the single important thing it does is manipulate files in the user's home directory, you might find just distributing the script to be much easier than trying to run it via Docker.)
Related
I know you can specify user and group IDs with the docker run command and you can force the same IDs when building an image in order to have an easy life dealing with file permissions (e.g. as described here).
But what if you want to reuse an image containing a user definition across different machines / users with different user/group IDs?
E.g. on a machine with user/group IDs are 1000:1000 you build an image and use those values to create a user. Then you push the image to some registry. On another machine (or just another user) with IDs 1001:1000 you want to pull and use the image.
AFAIU you would either have to know the IDs to use and provide them to docker run and you might get trouble dealing with files created by the container. Using the local IDs will let you experience those issues inside the container.
Right?
What's the usual approach to this? I'd like to have some way to 'translate' those ID's, i.e. having UID 1001 outside and 1000 inside the container.
Currently the only ways I know of are: 1. ignoring the issue, 2. not sharing images or 3. entering the container with IDs of the user inside the image and rewriting permissions afterwards.
Do not build specific user IDs into your image. As you note, if you do depend on the runtime user ID matching host-directory permissions, this will be wrong if a different host user runs the container. You must specify this at docker run time.
If at all possible, avoid writing to local files in your container. Store data somewhere like a relational database instead. If you don't need to write files, then it doesn't actually matter what user ID the container runs as. This also makes it easier to scale the application and to run it in clustered environments like Kubernetes.
If your application does write to local files, then limit it to a single directory. Say your application code is in /app; maybe the data goes in /data. Only that one directory needs to be writable. The files in /app should stay owned by root; you do not want the application to be able to overwrite its source code or static assets while it's running.
In the Dockerfile, a good practice would be to create a non-root user, with any user ID, but only switch to it at the end of the Dockerfile. Your container should be operable without any special options.
FROM ...
# Create a non-root user, with an arbitrary user ID
RUN adduser --system --no-create-home appuser
# We are still root; do the normal build-and-install steps
# (Do not run `chown` on anything here, leave it all owned by root)
WORKDIR /app
COPY ...
RUN ...
# Create the empty data directory and give the arbitrary user
# permissions on it
RUN mkdir /data && chown appuser /data
ENV APP_DATA_DIR=/data # recognized by the application
# Normal metadata to run the application, as the arbitrary user
USER appuser
CMD ...
If you decide you want the data to be backed by a bind-mounted host directory, then when you run the container you need to provide the corresponding user ID.
docker run \
-u $(id -u):$(id -g) \ # as the host user
-v "$PWD:/data" \ # mounting the current directory on /data
...
You may need an entrypoint wrapper script to set up the /data directory on first use. Anything that is in the image will be hidden by the bind mount, so if the host directory is initially empty then the image will need to know how to put the required initial data there.
I use build-arg to ensure my container runs with the same UID as my host user:
--build-arg UID=$(UID)
I am mounting a volume with:
-v $(PWD)/packages:/mnt/packages
Because I want my container to produce some output into that directory.
The directory belongs to the host user.
But, not matter what I try, the mountpoint in the container belongs to root. I can correct it with sudo in the running container:
function correct-mountpoint-permissions () {
# Hack alert:
# When mounting a host volume, it belongs to root, no matter what I do in the host or in the Dockerfile
# Here we correct the situation by using sudo. It's not nice, but that's life
# Note: the same problem does NOT happen with *named* volumes, but that completely defeats the purpose of
# mounting the volume in the first place - which is to produce some output visible on the host.
# If I have to start copying from containers to host, mounting volumes offers no benefit
sudo chown "$USER:$USER" "$PACKAGES"
}
But this is just ugly. I should not even need to have sudo installed in the first place, or sudo rights at all in my container.
The alternative is to either mount a named volume, which is difficult to access from the host, or to avoid mounting volumes in the container at all, and just docker cp from it when my container has produced the expected output.
Is there a simple way of mounting a host volume with the right ownership into a container (which means, not root, but my selected UID)
I understand you must be setting a user in your Dockerfile.
Before the user is set , try changing the ownership of the user during the build of the image.
RUN chown -R myuser:myuser ${PACKAGES_DIR}
USER myuser
I've been reading today on the theory behind uid 1001 specifically in Docker where it is a best principle not to have your container running as the root user.
What I've been able to tell so far for a unix system is...
root user has the UID of 0 Most unix distributions reserve the first
100 New users are assigned UIDs starting from 500 or 1000
When you
create a new account, it will usually be give the next-highest unused
number of 1001 (not sure what this means in relation to the previous dot point. If someone could clarify please)
I've seen a lot of Dockerfile examples that will just try to use 1001.
USER 1001
Two questions
Is the theory 1001 is safe to use because it is above the uid allocations ranges for new users on the host i.e. dot point 2?
Is it best practice to specify user as 1001 or would it be best adding a new user?
RUN useradd -ms /bin/bash newuser
USER newuser
WORKDIR /home/newuser
thank you
If your container doesn't need to write data in a named volume or a directory bind-mounted from the host, it usually doesn't matter at all what user ID the container runs as. There are a couple of restrictions still (if you're trying to listen on a port number less than 1024 your user ID must be 0 or you must manually add a capability at startup time).
I would use your second form, except I would not switch users until the end of the Dockerfile.
FROM python:3.8 # arbitrary choice
# ... install OS packages ...
# Create the user. This doesn't need to be repeated if the
# application code changes. It is a "system" user without
# a home directory and a default shell. We don't care what
# its numeric user ID is.
RUN useradd -r newuser
# WORKDIR creates the directory. It does not need to be
# under /home. It should be owned by root.
WORKDIR /app
# Copy the application in and do its installation, still as root.
COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY . .
# Switch USER only at the end of the file.
USER newuser
CMD ["./app.py"]
"Home directory" isn't usually a well-defined concept in Docker; I've created the user without a specific home directory here, and the application directory isn't under the normal Linux /home directory. Similarly, I haven't gone out of my way to specify a shell for this user or to specifically use GNU bash.
One important security point of this setup is that root owns the application files, but newuser is running the code. If there is some sort of compromise, this gives you an additional layer of protection: the compromised application code can't overwrite the application code or its fixed static data.
I started this with the caveat that this works fine if you don't need to persist data in the filesystem. If it does it will need to be somewhat adaptable, which probably means starting up as root but then dropping privileges. I'd support two modes of running:
The container is started up initially with a totally empty directory, possibly owned by root (a named Docker volume; a Kubernetes named volume). In this case, create the data directory you need and make it owned by the user in the Dockerfile.
docker run -v somevolume:/data myimage
The container is started up with a bind-mounted host directory and also a -u option naming the host user ID to use.
docker run -u $(id -u) "$PWD/data:data" myimage
You would need to use an entrypoint wrapper script to detect which case you're in, create the initial storage structure, set its permissions, and switch to a non-root user if required. There are lighter-weight tools like gosu or su-exec that specifically support this case. The Docker Hub consul image's entrypoint script has an example of doing this at startup time.
I'm using docker.io/solr:8.2.0 image on Google Compute Engine container instance, and have successfully got it running with a Mount path of /var/solr/data pointing to a Host path of /home/app/data
However, I'm having to do an extra step in the beginning to make /home/app/data writable by the container. I having to run in the host
sudo chown 8983:8983 /home/app -R
after that it works, /var/solr/data is mapped correctly and, on first run, even copies the files that are in the original /var/solr/data over to the host mount path
Is there a way to set this permission up in a start configuration so I can bypass this step?
I would like to volume mount a directory from a Docker container to my work station, so when I edit the content in the volume mount from my work station it updated in the container as well. It would be very useful for testing and develop web applications in general.
However I get a permission denied in the container, because the UID's in the container and host isn't the same. Isn't the original purpose of Docker that it should make development faster and easier?
This answer works around the issue I am facing when volume mounting a Docker container to my work station. But by doing this, I make changes to the container that I won't want in production, and that defeats the purpose of using Docker during development.
The container is Alpine Linux, work station Fedora 29, and editor Atom.
Question
Is there another way, so both my work station and container can read/write the same files?
There are multiple ways to do this, but the central issue is that bind mounts do not include any UID mapping capability, the UID on the host is what appears inside the container and vice versa. If those two UID's do not match, you will read/write files with different UID's and likely experience permission issues.
Option 1: get a Mac or deploy docker inside of VirtualBox. Both of these environments have a filesystem integration that dynamically updates the UID's. For Mac, that is implemented with OSXFS. Be aware that this convenience comes with a performance penalty.
Option 2: Change your host. If the UID on the host matches the UID inside the container, you won't experience any issues. You'd just run a usermod on your user on the host to change your UID there, and things will happen to work, at least until you run a different image with a different UID inside the container.
Option 3: Change your image. Some will modify the image to a static UID that matches their environment, often to match a UID in production. Others will pass a build arg with something like --build-arg UID=$(id -u) as part of the build command, and then the Dockerfile with something like:
FROM alpine
ARG UID=1000
RUN adduser -u ${UID} app
The downside of this is each developer may need a different image, so they are either building locally on each workstation, or you centrally build multiple images, one for each UID that exists among your developers. Neither of these are ideal.
Option 4: Change the container UID. This can be done in the compose file, or on a one off container with something like docker run -u $(id -u) your_image. The container will now be running with the new UID, and files in the volume will be accessible. However, the username inside the container will not necessarily map to your UID which may look strange to any commands you run inside the container. More importantly, any files own by the user inside the container that you have not hidden with your volume will have the original UID and may not be accessible.
Option 5: Give up, run everything as root, or change permissions to 777 allowing everyone to access the directory with no restrictions. This won't map to how you should run things in production, and the container may still write new files with limited permissions making them inaccessible to you outside the container. This also creates security risks of running code as root or leaving filesystems open to both read and write from any user on the host.
Option 6: Setup an entrypoint that dynamically updates your container. Despite not wanting to change your image, this is my preferred solution for completeness. Your container does need to start as root, but only in development, and the app will still be run as the user, matching the production environment. However, the first step of that entrypoint will be to change the user's UID/GID inside the container to match your volume's UID/GID. This is similar to option 4, but now files inside the image that were not replaced by the volume have the right UID's, and the user inside the container will now show with the changed UID so commands like ls show the username inside the container, not a UID to may map to another user or no one at all. While this is a change to your image, the code only runs in development, and only as a brief entrypoint to setup the container for that developer, after which the process inside the container will look identical to that in a production environment.
To implement this I make the following changes. First the Dockerfile now includes a fix-perms script and gosu from a base image I've pushed to the hub (this is a Java example, but the changes are portable to other environments):
FROM openjdk:jdk as build
# add this copy to include fix-perms and gosu or install them directly
COPY --from=sudobmitch/base:scratch / /
RUN apt-get update \
&& apt-get install -y maven \
&& useradd -m app
COPY code /code
RUN mvn build
# add an entrypoint to call fix-perms
COPY entrypoint.sh /usr/bin/
ENTRYPOINT ["/usr/bin/entrypoint.sh"]
CMD ["java", "-jar", "/code/app.jar"]
USER app
The entrypoint.sh script calls fix-perms and then exec and gosu to drop from root to the app user:
#!/bin/sh
if [ "$(id -u)" = "0" ]; then
# running on a developer laptop as root
fix-perms -r -u app -g app /code
exec gosu app "$#"
else
# running in production as a user
exec "$#"
fi
The developer compose file mounts the volume and starts as root:
version: '3.7'
volumes:
m2:
services:
app:
build:
context: .
target: build
image: registry:5000/app/app:dev
command: "/bin/sh -c 'mvn build && java -jar /code/app.jar'"
user: "0:0"
volumes:
- m2:/home/app/.m2
- ./code:/code
This example is taken from my presentation available here: https://sudo-bmitch.github.io/presentations/dc2019/tips-and-tricks-of-the-captains.html#fix-perms
Code for fix-perms and other examples are available in my base image repo: https://github.com/sudo-bmitch/docker-base
Since the UID in your containers are baked into the container definition, you can safely assume that they are relatively static. In this case, you can create a user in your host system with the machine UID and GID. Change user to the new account, and then make your edits to the files. Your host OS will not complain since it thinks it's just the user accessing its own files, and your container OS will see the same.
Alternatively, you can consider editing these files as root.