Variables in Dockerfile don't seem to be recognized? - docker

I am building an image using Dockfile. I would like to set the Username of the container via the command line to avoid permission issues.
The Dockfile is shown below, I used the variables of USER_NAME, GROUP_ID. But when I build, the problem keeps appearing.
The error is: groupadd: option '--gid' requires an argument
I'm guessing that both ${GROUP_ID} and ${USER_NAME} are recognized as empty strings, but shouldn't they be assigned values ​​when the container is created?
I've googled a few examples and based on the examples, I don't quite see where the problem is?
Please help me!
Thanks!
FROM matthewfeickert/docker-python3-ubuntu:latest
ARG USER_NAME
ARG USER_ID
ARG GROUP_ID
RUN groupadd -r --gid ${GROUP_ID} ${USER_NAME}
RUN useradd --no-log-init -r -g ${GROUP_ID} -u ${USER_ID} ${USER_NAME}
USER ${USER_NAME}
WORKDIR /usr/local/src

When you run the container, you can specify an arbitrary user ID with the docker run -u option.
docker run -u 1003 ... my-image
This doesn't require any special setup in the image. The user ID won't exist in the container's /etc/passwd file but there aren't really any consequences to this, beyond some cosmetic issues with prompts in interactive debugging shells.
A typical use of this is to give your container access to a bind-mounted data directory:
docker run \
-e DATA_DIR=/data \
-v "$PWD/app-data:/data" \
-u $(id -u) \
... \
my-image
I'd generally recommend not passing a specific user ID into your image build. This would make the user ID "baked in", and if someone with a different host uid wanted to run the image, they'd have to rebuild it.
It's often a good practice to set up some non-root user, but it doesn't matter what its user ID is so long as it's not zero. In turn, it's also typically a good practice to leave most of your application source code owned by the root user so that the application can't accidentally overwrite itself.
FROM matthewfeickert/docker-python3-ubuntu:latest
# Create an arbitrary non-root user; we don't care about its uid
# or other properties
RUN useradd --system user
# Still as root, do the normal steps to install and build the application
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY ./ ./
# Still as root, make sure the data directory exists
ENV DATA_DIR=/data
RUN mkdir "$DATA_DIR" && chown user "$DATA_DIR"
# VOLUME ["/data"]
# Normal metadata to run the container, only switching users now
EXPOSE 5000
USER user
CMD ["./app.py"]
This setup will still work with the extended docker run command shown initially: the docker run -v option will cause the container's /data directory to take on its numeric uid owner from the host, which (hopefully) matches the docker run -u uid.

You can pass the build args as shown below.
docker build --build-arg USER_NAME=test --build-arg USER_ID=805 --build-arg GROUP_ID=805 -t tag1 .
Also, as a best practice consider adding default vales to the args. So if the user doesn't specify the args the default values will be picked.

Related

Building Docker image as non root user

New here, was wondering if someone had experience with building images as non root user?
I am building Kotlin project, (2 step build) and my goal is now to build it as non root user. Here is what my Dockerfile looks like. Any help would be appreciated:
# Build
FROM openjdk:11-jdk-slim as builder
# Compile application
WORKDIR /root
COPY . .
RUN ./gradlew build
FROM openjdk:11-jre-slim
# Add application
COPY --from=builder /root/build/libs/*.jar ./app.jar
# Set the build version
ARG build_version
ENV BUILD_VERSION=$build_version
COPY docker-entrypoint.sh /
RUN chmod 777 /docker-entrypoint.sh
CMD /docker-entrypoint.sh
In order to use Docker, you don't need to be a root user, you just need to be inside of the docker user group.
On Linux:
If there is not already a docker group, you can create one using the command sudo groupadd docker.
Add yourself and any other users you would like to be able to access docker to this group using the command sudo usermod -aG docker [username of user].
Relog, so that Linux can re-evaluate user groups.
If you are not trying to run the command as root, but rather want to run the container as non-root, you can use the following DOCKERFILE contents (insert after FROM but before anything else.)
# Add a new user "john" with user id 8877
RUN useradd -u 8877 john
# Change to non-root privilege
USER john

Dockerfile - Creating non-root user almost doubles the image size

Created an application image on top of ubuntu. The application requires a non-root user.
I am able to create the image and get it working but the RUN statement for creating new user significantly increases the size of this image.
Dockerfile snippet:
## create newuser and give correct permissions to home directory
RUN useradd newuser --create-home --shell /bin/bash && \
echo 'newuser:newpassword' | chpasswd && \
chown -R newuser:newuser /home/newuser && \
chmod 755 /home/newuser
USER newuser
WORKDIR /home/newuser
Is there a better way to create a new user?
Thinking about alternate approaches, wonder if one uses multi-stage build to create this new user and then use copy --from to get relevant files in the final build. Not sure what those files would be.
Don't chown the directory; leave root owning it. You also shouldn't need to set a shell, or a password, or create a home directory; none of these things will be used in normal operation.
I'd suggest creating the user towards the start of the Dockerfile (it is fairly fixed and so this step can be cached) but only switching USER at the very end of the file, when you're setting up the metadata for how to run the container.
A Node-based example:
FROM node:lts # debian-based
# Create the non-root user up front
RUN adduser --system --group --no-create-home newuser
# Copy and build the package as usual
WORKDIR /app
COPY package.json yarn.lock .
RUN yarn install
COPY . .
RUN yarn build
# Now the application is built
# And root owns all the files
# And that's fine
# Say how to run the container
EXPOSE 3000
USER newuser
CMD yarn start
Having root owning the files gives you a little extra protection in case something goes wrong. If there's a bug that allows files in the container to be overwritten, having a different user owning those files prevents the application code or static assets from being inadvertently modified.
If your application needs to read or write files then you could create a specific directory for that:
# Towards the end of the file, but before the USER
RUN mkdir data && chown newuser data
This will let the operator mount some storage over the otherwise-empty directory. This is the only thing that has the newly created user ID in it at all, so if the storage comes with its own owner it shouldn't be an operational problem; you need to also specify the matching user ID at container startup time.
docker run -u $(id -u) -v $PWD/data:/app/data ...

What is a clean way to add a user in Docker with sudo priviledges?

I am trying to understand how to properly add non-root users in docker and give them sudo privileges. Let's say my current Ubuntu 18.04 system has janedoe as a sudo user. I want to create a docker image where I want to add janedoe as a non-root user who can have sudo privileges when needed. Since I new to this Linux system as well as Docker, I typically would appreciate someone explaining through an example how to do this.
The thing that I understand is that whenever I issue the command "USER janedoe" in the Dockerfile, many commands after that line cannot be executed by janedoe's privileges. I would assume we have to add janedoe to a sudo "group" when building the container similar to what we do when an admin adds a new user to the system.
I have been trying to look for some demo Dockerfile explaining the example but couldn't find it.
Generally you should think of a Docker container as a wrapper around a single process. If you ask this question about other processes, it doesn't really make sense. (How do I add a user to my PostgreSQL server with sudo privileges? How do I add a user to my Web browser?)
In Docker you almost never need sudo, for three reasons: it's trivial to switch users in most contexts; you don't typically get interactive shells in containers (how do I get a directory listing from inside the cron daemon?); and if you can run any docker command at all you can very easily root the whole host. sudo is also hard to script, and it's very hard to usefully maintain a user password in Docker (writing a root-equivalent password in a plain-text file that can be easily retrieved isn't a security best practice).
In the context of your question, if you've already switched to some non-root user, and you need to run some administrative command, use USER to switch back to root.
USER janedoe
...
USER root
RUN apt-get update && apt-get install -y some-package
USER janedoe
Since your containers have some isolation from the host system, you don't generally need containers to have the same user names or user IDs as the host system. The exception is when sharing files with the host using bind mounts, but there it's better to specify this detail when you start the container.
The typical practice I'm used to works like this:
In your Dockerfile, create some non-root user. It can have any name. It does not need a password, login shell, home directory, or any other details. Treating it as a "system" user is fine.
FROM ubuntu:18.04
RUN adduser --system --group --no-create-home appuser
Still in your Dockerfile, do almost everything as root. This includes installing your application.
RUN apt-get update && apt-get install ...
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
When you describe the default way to run the container, only then switch to the non-root user.
EXPOSE 8000
USER appuser
CMD ["./main.py"]
Ideally that's the end of the story: your code is built into your image and it stores all of its data somewhere external like a database, so it doesn't care about the host user space at all (there by default shouldn't be docker run -v or Docker Compose volumes: options).
If file permissions really matter, you can specify the numeric host user ID to use when you launch the container. The user doesn't specifically need to exist in the container's /etc/passwd file.
docker run \
--name myapp \
-d \
-p 8000:8000 \
-v $PWD:/data \
-u $(id -u) \
myimage
I think you are looking for the answer in this question:
How to add users to a docker container
RUN useradd -ms /bin/bash janedoe <-- this command adds the user
usermod -aG sudo janedoe <-- this command tells the container to put the user janedoe inside the SUDO group
Then, if you want to switch to that user for the remainder of the script, use:
USER janedoe <-- all lines after this now use the janedoe user to execute them
WORKDIR /home/janedoe <-- this tells your script from this line on to use paths relative to janedoe's home folder
Since the container itself runs linux modules, most (if not all) linux commands should work inside your container as well. If you have static users (i.e. it's predictable which users), you should be able to create them inside the Dockerfile used to create the image. Now everytime you run a container from said image you should get the janedoe user in there.

Copying files with execute permissions in Docker Image

Seems like a basic issue but couldnt find any answers so far ..
When using ADD / COPY in Dockerfile and running the image on linux, the default file permission of the file copied in the image is 644. The onwner of this file seems to be as 'root'
However, when running the image, a non-root user starts the container and any file thus copied with 644 permission cannot execute this copied/added file and if the file is executed at ENTRYPOINT it fails to start with permission denied error.
I read in one of the posts that COPY/ADD after Docker 1.17.0+ allows chown but in my case i dont know who will be the non-root user starting so i cannot set the permission as that user.
I also saw another work around to ADD/COPY files to a different location and use RUN to copy them from the temp location to actual folder like what am doing below. But this approach doesnt work as the final image doesnt have the files in /otp/scm
#Installing Bitbucket and setting variables
WORKDIR /tmp
ADD atlassian-bitbucket-${BITBUCKET_VERSION}.tar.gz .
COPY bbconfigupdater.sh .
#Copying Entrypoint script which will get executed when container starts
WORKDIR /tmp
COPY entrypoint.sh .
RUN ls -lrth /tmp
WORKDIR /opt/scm
RUN pwd && cp /tmp/bbconfigupdater.sh /opt/scm \
&& cp /tmp/entrypoint.sh /opt/scm \
&& cp -r /tmp/atlassian-bitbucket-${BITBUCKET_VERSION} /opt/scm \
&& chgrp -R 0 /opt/ \
&& chmod -R 755 /opt/ \
&& chgrp -R 0 /scm/bitbucket \
&& chmod -R 755 /scm/bitbucket \
&& ls -lrth /opt/scm && ls -lrth /scmdata
Any help is appreciated to figure out how i can get my entrypoint script copied to the desired path with execute permissions set.
The default file permission is whatever the file permission is in your build context from where you copy the file. If you control the source, then it's best to fix the permissions there to avoid a copy-on-write operation. Otherwise, if you cannot guarantee the system building the image will have the execute bit set on the files, a chmod after the copy operation will fix the permission. E.g.
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
A better option with newer versions of docker (and which didn't exist when this answer was first posted) is to use the --chmod flag (the permissions must be specified in octal at last check):
COPY --chmod=0755 entrypoint.sh .
You do not need to know who will run the container. The user inside the container is typically configured by the image creator (using USER) and doesn't depend on the user running the container from the docker host. When the user runs the container, they send a request to the docker API which does not track the calling user id.
The only time I've seen the host user matter is if you have a host volume and want to avoid permission issues. If that's your scenario, I often start the entrypoint as root, run a script called fix-perms to align the container uid with the host volume uid, and then run gosu to switch from root back to the container user.
A --chmod flag was added to ADD and COPY instructions in Docker CE 20.10. So you can now do.
COPY --chmod=0755 entrypoint.sh .
To be able to use it you need to enable BuildKit.
# enable buildkit for docker
DOCKER_BUILDKIT=1
# enable buildkit for docker-compose
COMPOSE_DOCKER_CLI_BUILD=1
Note: It seems to not be documented at this time, see this issue.

A working how-to for data extraction of non-root named volume permissions working with linux and win

I'm trying a simple workflow without success and it take me a loooooot of time to test many solutions on SO and github. Permission for named folder and more generaly permissions volume in docker is a nightmare link1 link2 imho.
So i restart from scratch, trying to create a simple proof of concept for my use case.
I want this general workflow :
user on windows and/or linux build the Dockerfile
user run the container (if possible not as root)
the container launch a crontab which run a script writing in the data volume each minute
users (on linux or windows) get the results from the data volume (not root) because permissions are correctly mapped
I use supercronic because it runs crontab in container without root permission.
The Dockerfile :
FROM artemklevtsov/r-alpine:latest as baseImage
RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result
COPY . /usr/local/src/myscript/
WORKDIR /usr/local/src/myscript/
RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
CMD ["supercronic", "crontab"]
The crontab file :
* * * * * sh /usr/local/src/myscript/run.sh > /proc/1/fd/1 2>&1
The run.sh script
#!/bin/bash
name=$(date '+%Y-%m-%d-%s')
echo "some data for the file" >> ./result/fileName$name
The commands :
# create the volume for result, uid/gid option are not possible for windows
docker volume create --name myTestVolume
docker run --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test
docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data
When i do this the result folder local_backup and files it contains has root:root permissions, so user who launch this container cannot access the files.
Is there a solution which works, which permits windows/linux/mac users who launch the same script to access easily the files into volume without problem of permissions ?
EDIT 1 :
The strategy first described here only work with binded volume, and not named volume. We use an entrypoint.sh to chown uid/gid of folders of container based on information given by docker run.
I copy paste the modified Dockerfile :
FROM artemklevtsov/r-alpine:latest as baseImage
RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result
COPY . /usr/local/src/myscript/
ENTRYPOINT [ "/usr/local/src/myscript/entrypoint.sh" ]
WORKDIR /usr/local/src/myscript/
RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl su-exec
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
CMD ["supercronic", "crontab"]
The entrypoint.sh
#!/bin/sh
set -e
addgroup -g $GID scrap && adduser -s /bin/sh -D -G scrap -u $UID scrap
if [ "$(whoami)" == "root" ]; then
chown -R scrap:scrap /usr/local/src/myscript/
chown --dereference scrap "/proc/$$/fd/1" "/proc/$$/fd/2" || :
exec su-exec scrap "$#"
fi
The procedure to build,launch, export:
docker build . --tag=test
docker run -e UID=1000 -e GID=1000 --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test
docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data
EDIT 2 :
For Windows, using docker toolbox and binded volume, i found the answer on SO. I use the c:/Users/MyUsers folder for binding, it's more simple.
docker run --name test -d -e UID=1000 -e GID=1000 --mount type=bind,source=/c/Users/myusers/localbackup,destination=/usr/local/src/myscript/result dockertest --name rflightscraps
Result of investigation
crontab run with scrap user [OK]
UID/GID of local user are mapped to container user scrap [OK]
Exported data continue to be root [NOT OK].
Windows / Linux [HALF OK]
If i use bind volume and not a named volume, it works. But this is not the desired behavior, how can i use the named volume with correct permission on Win/Linux ...
Let me divide the answer into two parts Linux Part and Docker part. You need to understand both in order to solve this problem.
Linux Part
It is easy to run cronjobs as user other than root in Linux.
This can be achieved by creating a user in docker container with the same UID as of that in the host machine and copying the crontab file as /var/spool/cron/crontabs/user_name.
From man crontab
crontab is the program used to install, deinstall or list the
tables used to drive the cron(8) daemon in Vixie Cron. Each user can
have their own crontab, and though these are files in
/var/spool/cron/crontabs, they are not intended to be edited directly.
Since Linux identifies users by User Id, inside docker the UID will be bound to the newly created user whereas in host machine the same will be binded with host user.
So, You don't have any permission issue as the files is owned by the host_user. Now you would have understood why I mentioned creating user with same UID as of that in host machine.
Docker Part
Docker considers all the directories(or layers) to be UNION FILE SYSTEM. Whenever you build an image each instruction creates a layer and the layer is marked as read-only. This is the reason Docker containers doesn't persist data. So you have to explicitly tell docker that some directories need to persist data by using VOLUME keyword.
You can run containers without mentioning volume explicitly. If you do so, docker daemon considers them to be UFS and resets the permissions.
In order to preserve the changes to a file/directory including ownership. The respective file should be declared as Volume in Dockerfile.
From UNION FILE SYSTEM
Indeed, when a container has booted, it is moved into memory, and the boot filesystem is unmounted to free up the RAM used by the initrd disk image. So far this looks pretty much like a typical Linux virtualization stack. Indeed, Docker next layers a root filesystem, rootfs, on top of the boot filesystem. This rootfs can be one or more operating systems (e.g., a Debian or Ubuntu filesystem).
Docker calls each of these filesystems images. Images can be layered on top of one another. The image below is called the parent image and you can traverse each layer until you reach the bottom of the image stack where the final image is called the base image. Finally, when a container is launched from an image, Docker mounts a read-write filesystem on top of any layers below. This is where whatever processes we want our Docker container to run will execute. When Docker first starts a container, the initial read-write layer is empty. As changes occur, they are applied to this layer; for example, if you want to change a file, then that file will be copied from the read-only layer below into the read-write layer. The read-only version of the file will still exist but is now hidden underneath the copy.
Example:
Let us assume that we have a user called host_user. The UID of host_user is 1000. Now we are going to create a user called docker_user in Docker container. So I'll assign him UID as 1000. Now whatever files that are owned by docker_user in Docker container is also owned by host_user if those files are accessible by host_user from host(i.e through volumes).
Now you can share the binded directory with others without any permission issues. You can even give 777 permission on the corresponding directory which allows others to edit the data. Else, You can leave 755 permissions which allows others to copy but only the owner to edit the data.
I've declared the directory to persist changes as a volume. This preserves all changes. Be careful as once you declare a directory as volume further changes made to that directory while building the will be ignored as those changes will be in separate layers. Hence do all your changes in the directory and then declare it as volume.
Here is the Docker file.
FROM alpine:latest
ARG ID=1000
#UID as arg so we can also pass custom user_id
ARG CRON_USER=docker_user
#same goes for username
COPY crontab /var/spool/cron/crontabs/$CRON_USER
RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
mkdir /temp && \
chown -R $ID:$ID /temp && \
chmod 777 /temp
VOLUME /temp
#Specify the dir to be preserved as Volume else docker considers it as Union File System
ENTRYPOINT ["crond", "-f", "-l", "2"]
Here is the crontab
* * * * * /usr/bin/whoami >> /temp/cron.log
Building the image
docker build . -t test
Create new volume
docker volume create --name myTestVolume
Run with Data volume
docker run --rm --name test -d -v myTestVolume:/usr/local/src/myscript/result test:latest
Whenever you mount myTestVolume to other container you can see the
data under /usr/local/src/myscript/result is owned by UID 1000
if no user exist with that UID in that container or the username of
corresponding UID.
Run with Bind volume
docker run --rm --name test - -dv $PWD:/usr/local/src/myscript/result test:latest
When you do an ls -al /home/host_user/temp You will see that file called cron.log is created and is owned by **host_user**.
The same will be owned by docker_user in docker container when you do an ls -al /temp. The contents of cron.log will be docker_user.
So, Your effective Dockerfile should be
FROM artemklevtsov/r-alpine:latest as baseImage
ARG ID=1000
ARG CRON_USER=docker_user
RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories && \
apk --no-cache add busybox-suid curl && \
mkdir -p /usr/local/src/myscript/result && \
chown -R $ID:$ID /usr/local/src/myscript/result && \
chmod 777 /usr/local/src/myscript/result
COPY crontab /var/spool/cron/crontabs/$CRON_USER
COPY . /usr/local/src/myscript/
VOLUME /usr/local/src/myscript/result
#This preserves chown and chmod changes.
WORKDIR /usr/local/src/myscript/
ENTRYPOINT ["crond", "-f", "-l", "2"]
Now whenever you attach a Data/bind volume to /usr/local/src/myscript/result it will be owned by user having UID 1000 and the same is persistent across all the containers whichever has mounted the same volume with their corresponding user with 1000 as file owners.
Please Note: I've given 777 permissions in order to share with every one. You can skip that step in your Dockerfle based on your convinence.
References:
Crontab manual.
User identiier - Wiki.
User ID Definition.
About storage drivers.
UNION FILE SYSTEM.

Resources