Seems like a basic issue but couldnt find any answers so far ..
When using ADD / COPY in Dockerfile and running the image on linux, the default file permission of the file copied in the image is 644. The onwner of this file seems to be as 'root'
However, when running the image, a non-root user starts the container and any file thus copied with 644 permission cannot execute this copied/added file and if the file is executed at ENTRYPOINT it fails to start with permission denied error.
I read in one of the posts that COPY/ADD after Docker 1.17.0+ allows chown but in my case i dont know who will be the non-root user starting so i cannot set the permission as that user.
I also saw another work around to ADD/COPY files to a different location and use RUN to copy them from the temp location to actual folder like what am doing below. But this approach doesnt work as the final image doesnt have the files in /otp/scm
#Installing Bitbucket and setting variables
WORKDIR /tmp
ADD atlassian-bitbucket-${BITBUCKET_VERSION}.tar.gz .
COPY bbconfigupdater.sh .
#Copying Entrypoint script which will get executed when container starts
WORKDIR /tmp
COPY entrypoint.sh .
RUN ls -lrth /tmp
WORKDIR /opt/scm
RUN pwd && cp /tmp/bbconfigupdater.sh /opt/scm \
&& cp /tmp/entrypoint.sh /opt/scm \
&& cp -r /tmp/atlassian-bitbucket-${BITBUCKET_VERSION} /opt/scm \
&& chgrp -R 0 /opt/ \
&& chmod -R 755 /opt/ \
&& chgrp -R 0 /scm/bitbucket \
&& chmod -R 755 /scm/bitbucket \
&& ls -lrth /opt/scm && ls -lrth /scmdata
Any help is appreciated to figure out how i can get my entrypoint script copied to the desired path with execute permissions set.
The default file permission is whatever the file permission is in your build context from where you copy the file. If you control the source, then it's best to fix the permissions there to avoid a copy-on-write operation. Otherwise, if you cannot guarantee the system building the image will have the execute bit set on the files, a chmod after the copy operation will fix the permission. E.g.
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
A better option with newer versions of docker (and which didn't exist when this answer was first posted) is to use the --chmod flag (the permissions must be specified in octal at last check):
COPY --chmod=0755 entrypoint.sh .
You do not need to know who will run the container. The user inside the container is typically configured by the image creator (using USER) and doesn't depend on the user running the container from the docker host. When the user runs the container, they send a request to the docker API which does not track the calling user id.
The only time I've seen the host user matter is if you have a host volume and want to avoid permission issues. If that's your scenario, I often start the entrypoint as root, run a script called fix-perms to align the container uid with the host volume uid, and then run gosu to switch from root back to the container user.
A --chmod flag was added to ADD and COPY instructions in Docker CE 20.10. So you can now do.
COPY --chmod=0755 entrypoint.sh .
To be able to use it you need to enable BuildKit.
# enable buildkit for docker
DOCKER_BUILDKIT=1
# enable buildkit for docker-compose
COMPOSE_DOCKER_CLI_BUILD=1
Note: It seems to not be documented at this time, see this issue.
Related
I have this Dockerfile setup:
FROM node:14.5-buster-slim AS base
WORKDIR /app
FROM base AS production
ENV NODE_ENV=production
RUN chown -R node:node /app
RUN chmod 755 /app
USER node
... other copies
COPY ./scripts/startup-production.sh ./
COPY ./scripts/healthz.sh ./
CMD ["./startup-production.sh"]
The problem I'm facing is that I can't execute ./healthz.sh because it's only executable by the node user. When I commented out the two RUN and the USER commands, I could execute the file just fine. But I want to enforce the executable permissions only to the node for security reasons.
I need the ./healthz.sh to be externally executable by Kubernetes' liveness & rediness probes.
How can I make it so? Folder restructuring or stuff like that are fine with me.
In most cases, you probably want your code to be owned by root, but to be world-readable, and for scripts be world-executable. The Dockerfile COPY directive will copy in a file with its existing permissions from the host system (hidden in the list of bullet points at the end is a note that a file "is copied individually along with its metadata"). So the easiest way to approach this is to make sure the script has the right permissions on the host system:
# mode 0755 is readable and executable by everyone but only writable by owner
chmod 0755 healthz.sh
git commit -am 'make healthz script executable'
Then you can just COPY it in, without any special setup.
# Do not RUN chown or chmod; just
WORKDIR /app
COPY ./scripts/healthz.sh .
# Then when launching the container, specify
USER node
CMD ["./startup-production.sh"]
You should be able to verify this locally by running your container and manually invoking the health-check script
docker run -d --name app the-image
# possibly with a `docker exec -u` option to specify a different user
docker exec app /app/healthz.sh && echo OK
The important thing to check is that the file is world-executable. You can also double-check this by looking at the built container
docker run --rm the-image ls -l /app/healthz.sh
That should print out one line, starting with a permission string -rwxr-xr-x; the last three r-x are the important part. If you can't get the permissions right another way, you can also fix them up in your image build
COPY ./scripts/healthz.sh .
# If you can't make the permissions on the original file right:
RUN chmod 0755 *.sh
You need to modify user Dockerfile CMD command like this : ["sh", "./startup-production.sh"]
This will interpret the script as sh, but it can be dangerous if your script is using bash specific features like [[]] with #!/bin/bash as its first line.
Moreover I would say use ENTRYPOINT here instead of CMD if you want this to run whenever container is up
I am building a Docker image using the Node:12 base image. I am trying to switch it to use Node:12-alpine instead due to the smaller image size. I have installed bash and shadow in Alpine to be able to run chmod commands.
I am running into an error with one of the RUN commands RUN chmod +755. The error message: chmod: invalid mode '+755'
Note that this works using the Node:12 base image, so I think I just need to install another package in Alpine linux? Thanks!
FROM node:12.8-alpine
# Create working directory
RUN mkdir -p /home/node/app
# Set working directory
WORKDIR /home/node/app
# Install bash and shadow for permissions chmod commands
RUN apk add --no-cache bash && apk add shadow
# Add `/home/node/app/node_modules/.bin` to $PATH
ENV PATH /home/node/app/node_modules/.bin:$PATH
# Copy code
COPY --chown=node . /home/node/app
# Update umask
RUN chmod +755 /home/node/app/entrypoint.sh && \
echo 'umask 002' >> /home/node/.profile && \
echo 'umask 002' >> /home/node/.bashrc && \
npm install
ENTRYPOINT ["./entrypoint.sh"]
CMD [ "npm", "start" ]
The various Alpine-based Docker images use a minimal toolset called BusyBox, which tends to only implement the functionality required in standard utilities and no more. In particular, the POSIX.1 definition of chmod specifies (emphasis mine):
The mode operand shall be either a symbolic_mode expression or a non-negative octal integer.
So according to the standard, you can either use the +rwx form to add bits, or the octal 0755 form to specify a permission, but not combine the two.
In the context of a Docker image, you're usually dealing with a pretty fixed filesystem layout, and in any case you know what you want the permissions to be; you should be able to run
RUN chmod 0755 /home/node/app/entrypoint.sh
without installing any additional packages.
(Also note that shell dotfiles usually aren't read by Docker, so the modifications to .profile and .bashrc have no effect. Typically you do want your application to be owned by root but executed by a different user, for an additional layer of security to prevent the application files from being unintentionally modified.)
I am trying to add a directory to my docker image. I tried the below methods. During the build I dont see any errors, but once I run the container ( I am using docker-compose) and get into it docker exec -it 410e434a7304 /bin/sh I dont see the directory in the path I am copying it into nor do I see it as a volume when I do docker inspect.
Approach 1 : Classic mkdir
# Define working directory
WORKDIR /root
RUN cd /var \
mdkir www \\ no www directory created
COPY <fileDirectory> /var/www/<fileDirectory>
Approach 2 : Volume
FROM openjdk:8u171 as build
# Define working directory
WORKDIR /root
VOLUME["/var/www"]
COPY <fileDirectory> /var/www/<fileDirectory>
Your first approach is correct in principle, only that your RUN statement is faulty. Try:
RUN cd /var && mkdir www
Also, please note the fundamental difference between RUN mkdir and VOLUME: the former simply creates a directory on your container, while the latter is chiefly intended for mounting directories from your container to the host your container is running on.
This is how I made it work:
# Define working directory
WORKDIR /root
COPY <fileDirectory> /root/<fileDirectory>
RUN cd /var && mkdir www && cp -R /root/<fileDirectory> /var/www
RUN rm -rf /root/email-media
I had to copy the from my host machine to docker image's working directory /root and from /root to the desired destination. Later removed the directory from/root`
Not sure if thats the cleanest way, if I followed the approach 1 with the right syntax suggested by #Fritz it could never find the the path created and throw an error.
After running the RUN layer it would remove the container (as below) and in the COPY line it would not have the reference to the path created in the run line.
Step 16/22 : RUN cd /var && mkdir www && cp -R /root/<fileDirectory> /var/www
---> Running in a9c7df27116e
Removing intermediate container a9c7df27116e
I'm trying a simple workflow without success and it take me a loooooot of time to test many solutions on SO and github. Permission for named folder and more generaly permissions volume in docker is a nightmare link1 link2 imho.
So i restart from scratch, trying to create a simple proof of concept for my use case.
I want this general workflow :
user on windows and/or linux build the Dockerfile
user run the container (if possible not as root)
the container launch a crontab which run a script writing in the data volume each minute
users (on linux or windows) get the results from the data volume (not root) because permissions are correctly mapped
I use supercronic because it runs crontab in container without root permission.
The Dockerfile :
FROM artemklevtsov/r-alpine:latest as baseImage
RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result
COPY . /usr/local/src/myscript/
WORKDIR /usr/local/src/myscript/
RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
CMD ["supercronic", "crontab"]
The crontab file :
* * * * * sh /usr/local/src/myscript/run.sh > /proc/1/fd/1 2>&1
The run.sh script
#!/bin/bash
name=$(date '+%Y-%m-%d-%s')
echo "some data for the file" >> ./result/fileName$name
The commands :
# create the volume for result, uid/gid option are not possible for windows
docker volume create --name myTestVolume
docker run --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test
docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data
When i do this the result folder local_backup and files it contains has root:root permissions, so user who launch this container cannot access the files.
Is there a solution which works, which permits windows/linux/mac users who launch the same script to access easily the files into volume without problem of permissions ?
EDIT 1 :
The strategy first described here only work with binded volume, and not named volume. We use an entrypoint.sh to chown uid/gid of folders of container based on information given by docker run.
I copy paste the modified Dockerfile :
FROM artemklevtsov/r-alpine:latest as baseImage
RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result
COPY . /usr/local/src/myscript/
ENTRYPOINT [ "/usr/local/src/myscript/entrypoint.sh" ]
WORKDIR /usr/local/src/myscript/
RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl su-exec
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
SUPERCRONIC=supercronic-linux-amd64 \
SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
CMD ["supercronic", "crontab"]
The entrypoint.sh
#!/bin/sh
set -e
addgroup -g $GID scrap && adduser -s /bin/sh -D -G scrap -u $UID scrap
if [ "$(whoami)" == "root" ]; then
chown -R scrap:scrap /usr/local/src/myscript/
chown --dereference scrap "/proc/$$/fd/1" "/proc/$$/fd/2" || :
exec su-exec scrap "$#"
fi
The procedure to build,launch, export:
docker build . --tag=test
docker run -e UID=1000 -e GID=1000 --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test
docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data
EDIT 2 :
For Windows, using docker toolbox and binded volume, i found the answer on SO. I use the c:/Users/MyUsers folder for binding, it's more simple.
docker run --name test -d -e UID=1000 -e GID=1000 --mount type=bind,source=/c/Users/myusers/localbackup,destination=/usr/local/src/myscript/result dockertest --name rflightscraps
Result of investigation
crontab run with scrap user [OK]
UID/GID of local user are mapped to container user scrap [OK]
Exported data continue to be root [NOT OK].
Windows / Linux [HALF OK]
If i use bind volume and not a named volume, it works. But this is not the desired behavior, how can i use the named volume with correct permission on Win/Linux ...
Let me divide the answer into two parts Linux Part and Docker part. You need to understand both in order to solve this problem.
Linux Part
It is easy to run cronjobs as user other than root in Linux.
This can be achieved by creating a user in docker container with the same UID as of that in the host machine and copying the crontab file as /var/spool/cron/crontabs/user_name.
From man crontab
crontab is the program used to install, deinstall or list the
tables used to drive the cron(8) daemon in Vixie Cron. Each user can
have their own crontab, and though these are files in
/var/spool/cron/crontabs, they are not intended to be edited directly.
Since Linux identifies users by User Id, inside docker the UID will be bound to the newly created user whereas in host machine the same will be binded with host user.
So, You don't have any permission issue as the files is owned by the host_user. Now you would have understood why I mentioned creating user with same UID as of that in host machine.
Docker Part
Docker considers all the directories(or layers) to be UNION FILE SYSTEM. Whenever you build an image each instruction creates a layer and the layer is marked as read-only. This is the reason Docker containers doesn't persist data. So you have to explicitly tell docker that some directories need to persist data by using VOLUME keyword.
You can run containers without mentioning volume explicitly. If you do so, docker daemon considers them to be UFS and resets the permissions.
In order to preserve the changes to a file/directory including ownership. The respective file should be declared as Volume in Dockerfile.
From UNION FILE SYSTEM
Indeed, when a container has booted, it is moved into memory, and the boot filesystem is unmounted to free up the RAM used by the initrd disk image. So far this looks pretty much like a typical Linux virtualization stack. Indeed, Docker next layers a root filesystem, rootfs, on top of the boot filesystem. This rootfs can be one or more operating systems (e.g., a Debian or Ubuntu filesystem).
Docker calls each of these filesystems images. Images can be layered on top of one another. The image below is called the parent image and you can traverse each layer until you reach the bottom of the image stack where the final image is called the base image. Finally, when a container is launched from an image, Docker mounts a read-write filesystem on top of any layers below. This is where whatever processes we want our Docker container to run will execute. When Docker first starts a container, the initial read-write layer is empty. As changes occur, they are applied to this layer; for example, if you want to change a file, then that file will be copied from the read-only layer below into the read-write layer. The read-only version of the file will still exist but is now hidden underneath the copy.
Example:
Let us assume that we have a user called host_user. The UID of host_user is 1000. Now we are going to create a user called docker_user in Docker container. So I'll assign him UID as 1000. Now whatever files that are owned by docker_user in Docker container is also owned by host_user if those files are accessible by host_user from host(i.e through volumes).
Now you can share the binded directory with others without any permission issues. You can even give 777 permission on the corresponding directory which allows others to edit the data. Else, You can leave 755 permissions which allows others to copy but only the owner to edit the data.
I've declared the directory to persist changes as a volume. This preserves all changes. Be careful as once you declare a directory as volume further changes made to that directory while building the will be ignored as those changes will be in separate layers. Hence do all your changes in the directory and then declare it as volume.
Here is the Docker file.
FROM alpine:latest
ARG ID=1000
#UID as arg so we can also pass custom user_id
ARG CRON_USER=docker_user
#same goes for username
COPY crontab /var/spool/cron/crontabs/$CRON_USER
RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
mkdir /temp && \
chown -R $ID:$ID /temp && \
chmod 777 /temp
VOLUME /temp
#Specify the dir to be preserved as Volume else docker considers it as Union File System
ENTRYPOINT ["crond", "-f", "-l", "2"]
Here is the crontab
* * * * * /usr/bin/whoami >> /temp/cron.log
Building the image
docker build . -t test
Create new volume
docker volume create --name myTestVolume
Run with Data volume
docker run --rm --name test -d -v myTestVolume:/usr/local/src/myscript/result test:latest
Whenever you mount myTestVolume to other container you can see the
data under /usr/local/src/myscript/result is owned by UID 1000
if no user exist with that UID in that container or the username of
corresponding UID.
Run with Bind volume
docker run --rm --name test - -dv $PWD:/usr/local/src/myscript/result test:latest
When you do an ls -al /home/host_user/temp You will see that file called cron.log is created and is owned by **host_user**.
The same will be owned by docker_user in docker container when you do an ls -al /temp. The contents of cron.log will be docker_user.
So, Your effective Dockerfile should be
FROM artemklevtsov/r-alpine:latest as baseImage
ARG ID=1000
ARG CRON_USER=docker_user
RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories && \
apk --no-cache add busybox-suid curl && \
mkdir -p /usr/local/src/myscript/result && \
chown -R $ID:$ID /usr/local/src/myscript/result && \
chmod 777 /usr/local/src/myscript/result
COPY crontab /var/spool/cron/crontabs/$CRON_USER
COPY . /usr/local/src/myscript/
VOLUME /usr/local/src/myscript/result
#This preserves chown and chmod changes.
WORKDIR /usr/local/src/myscript/
ENTRYPOINT ["crond", "-f", "-l", "2"]
Now whenever you attach a Data/bind volume to /usr/local/src/myscript/result it will be owned by user having UID 1000 and the same is persistent across all the containers whichever has mounted the same volume with their corresponding user with 1000 as file owners.
Please Note: I've given 777 permissions in order to share with every one. You can skip that step in your Dockerfle based on your convinence.
References:
Crontab manual.
User identiier - Wiki.
User ID Definition.
About storage drivers.
UNION FILE SYSTEM.
While building a Docker image, how do I COPY a file into the image so that the resulting file is owned by a user other than root?
For versions v17.09.0-ce and newer
Use the optional flag --chown=<user>:<group> with either the ADD or COPY commands.
For example
COPY --chown=<user>:<group> <hostPath> <containerPath>
The documentation for the --chown flag is now live on the main Dockerfile Reference page.
Issue 34263 has been merged and is available in release v17.09.0-ce.
For versions older than v17.09.0-ce
Docker doesn't support COPY as a user other than root. You need to chown / chmod the file after the COPY command.
Example Dockerfile:
from centos:6
RUN groupadd -r myuser && adduser -r -g myuser myuser
USER myuser
#Install code, configure application, etc...
USER root
COPY run-my-app.sh /usr/local/bin/run-my-app.sh
RUN chown myuser:myuser /usr/local/bin/run-my-app.sh && \
chmod 744 /usr/local/bin/run-my-app.sh
USER myuser
ENTRYPOINT ["/usr/local/bin/run-my-app.sh"]
Previous to v17.09.0-ce, the Dockerfile Reference for the COPY command said:
All new files and directories are created with a UID and GID of 0.
History
This feature has been tracked through multiple GitHub issues: 6119, 9943, 13600, 27303, 28499, Issue 30110.
Issue 34263 is the issue that implemented the optional flag functionality and Issue 467 updated the documentation.