I am having a problem adding a file to an image and setting ownership via --chown flag. Specifically, here is a dockerfile adding a simple text file:
FROM fedora:24
ARG user_name=slave
ARG user_uid=1000
ARG user_home=/home/$user_name/
RUN useradd -l -u ${user_uid} -ms /bin/bash $user_name
WORKDIR ${user_home}
USER ${user_name}
ADD --chown=1397765041:1397765041 test.txt ./
CMD ls -l
This results in expected ownership of text.txt as can be seen:
$ docker run --rm -it bm/tmp:latest
total 4
-rw-r--r-- 1 some_user 1397765041 6 Oct 21 20:00 test.txt
Cool. Now if I change test.txt to a tar file (for example boost_1_57_0.tar.bz2), and rebuild, this is what I get:
$ docker run --rm -it bm/tmp:latest
total 4
drwx------ 8 501 root 4096 Oct 31 2014 boost_1_57_0
Here is how I am building (probably doesn't matter tho):
docker build -t bm/tmp --build-arg user_name=some_user --build-arg user_uid=1397765041 .
As we can see, ownership is NOT as expected in this case. It seems the behavior of --chown differs from the two cases shown above. I know that ADD automatically extracts tars. I don't know how the ownership is being set in the case where the file is a tar file. Anyone?
Unfortunately, ADD --chown only works for regular files. ADD with a tarball uses the ownership and permissions listed inside in tarball.
Workarounds:
Run tar yourself with --owner/--owner-map/--group/--group-map.
chown -R after the ADD.
Related
How can I control which user owns the files I copy in and out of a container?
The docker cp command says this about file ownership:
The cp command behaves like the Unix cp -a command in that directories are copied recursively with permissions preserved if possible. Ownership is set to the user and primary group at the destination. For example, files copied to a container are created with UID:GID of the root user. Files copied to the local machine are created with the UID:GID of the user which invoked the docker cp command. However, if you specify the -a option, docker cp sets the ownership to the user and primary group at the source.
It says that files copied to a container are created as the root user, but that's not what I see. I create two files owned by user id 1005 and 1006. Those owners are translated into the container's user namespace. The -a option seems to make no difference when I copy the file into a container.
$ sudo chown 1005:1005 test.txt
$ ls -l test.txt
-rw-r--r-- 1 1005 1005 29 Oct 6 12:43 test.txt
$ docker volume create sandbox1
sandbox1
$ docker run --name run1 -vsandbox1:/data alpine echo OK
OK
$ docker cp test.txt run1:/data/test1005.txt
$ docker cp -a test.txt run1:/data/test1005a.txt
$ sudo chown 1006:1006 test.txt
$ docker cp test.txt run1:/data/test1006.txt
$ docker cp -a test.txt run1:/data/test1006a.txt
$ docker run --rm -vsandbox1:/data alpine ls -l /data
total 16
-rw-r--r-- 1 1005 1005 29 Oct 6 19:43 test1005.txt
-rw-r--r-- 1 1005 1005 29 Oct 6 19:43 test1005a.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 19:43 test1006.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 19:43 test1006a.txt
When I copy files out of the container, they are always owned by me. Again, the -a option seems to do nothing.
$ docker run --rm -vsandbox1:/data alpine cp /data/test1006.txt /data/test1007.txt
$ docker run --rm -vsandbox1:/data alpine chown 1007:1007 /data/test1007.txt
$ docker cp run1:/data/test1006.txt .
$ docker cp run1:/data/test1007.txt .
$ docker cp -a run1:/data/test1006.txt test1006a.txt
$ docker cp -a run1:/data/test1007.txt test1007a.txt
$ ls -l test*.txt
-rw-r--r-- 1 don don 29 Oct 6 12:43 test1006a.txt
-rw-r--r-- 1 don don 29 Oct 6 12:43 test1006.txt
-rw-r--r-- 1 don don 29 Oct 6 12:47 test1007a.txt
-rw-r--r-- 1 don don 29 Oct 6 12:47 test1007.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 12:43 test.txt
$
You can also change the ownership by logging in as root user into the container :
docker exec -it --user root <container-id> /bin/bash
chown -R <username>:<groupname> <folder/file>
In addition to #Don Kirkby's answer, let me provide a similar example in bash/shell script for the case that you want to copy something into a container while applying different ownership and permissions than those of the original file.
Let's create a new container from a small image that will keep running by itself:
docker run -d --name nginx nginx:alpine
Now wel'll create a new file which is owned by the current user and has default permissions:
touch foo.bar
ls -ahl foo.bar
>> -rw-rw-r-- 1 my-user my-group 0 Sep 21 16:45 foo.bar
Copying this file into the container will set ownership and group to the UID of my user and preserve the permissions:
docker cp foo.bar nginx:/foo.bar
docker exec nginx sh -c 'ls -ahl /foo.bar'
>> -rw-rw-r-- 1 4098 4098 0 Sep 21 14:45 /foo.bar
Using a little tar work-around, however, I can change the ownership and permissions that are applied inside of the container.
tar -cf - foo.bar --mode u=+r,g=-rwx,o=-rwx --owner root --group root | docker cp - nginx:/
docker exec nginx sh -c 'ls -ahl /foo.bar'
>> -r-------- 1 root root 0 Sep 21 14:45 /foo.bar
tar options explained:
c creates a new archive instead of unpacking one.
f - will write to stdout instead of a file.
foo.bar is the input file to be packed.
--mode specifies the permissions for the target. Similar to chown, they can be given in symbolic notation or as an octal number.
--owner sets the new owner of the file.
--group sets the new group of the file.
docker cp - reads the file that is to be copied into the container from stdin.
This approach is useful when a file needs to be copied into a created container before it starts, such that docker exec is not an option (which can only operate on running containers).
Just a one-liner (similar to #ramu's answer), using root to make the call:
docker exec -u 0 -it <container-id> chown node:node /home/node/myfile
In order to get complete control of file ownership, I used the tar stream feature of docker cp:
If - is specified for either the SRC_PATH or DEST_PATH, you can also stream a tar archive from STDIN or to STDOUT.
I launch the docker cp process, then stream a tar file to or from the process. As the tar entries go past, I can adjust the ownership and permissions however I like.
Here's a simple example in Python that copies all the files from /outputs in the sandbox1 container to the current directory, excludes the current directory so its permissions don't get changed, and forces all the files to have read/write permissions for the user.
from subprocess import Popen, PIPE, CalledProcessError
import tarfile
def main():
export_args = ['sudo', 'docker', 'cp', 'sandbox1:/outputs/.', '-']
exporter = Popen(export_args, stdout=PIPE)
tar_file = tarfile.open(fileobj=exporter.stdout, mode='r|')
tar_file.extractall('.', members=exclude_root(tar_file))
exporter.wait()
if exporter.returncode:
raise CalledProcessError(exporter.returncode, export_args)
def exclude_root(tarinfos):
print('\nOutputs:')
for tarinfo in tarinfos:
if tarinfo.name != '.':
assert tarinfo.name.startswith('./'), tarinfo.name
print(tarinfo.name[2:])
tarinfo.mode |= 0o600
yield tarinfo
main()
I am playing with a Dockerfile and I have this:
ARG PUID=1000
ARG PGID=1000
RUN groupadd -g $PGID docker-user && \
useradd -u $PUID -g docker-user -m docker-user && \
mkdir /home/docker-user/.composer
COPY container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer
RUN chown -R docker-user:docker-user /home/docker-user/.composer
USER docker-user
RUN composer global install
But when I try to build the image it ends with the following error:
Step 6 : COPY container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer
lstat container-files/home/docker-user/.composer/composer.json: no such file or directory
The file does exist on the host as per this output:
$ ls -la workspace/container-files/home/docker-user/.composer/
total 12
drwxrwxr-x 2 rperez rperez 4096 Oct 5 11:34 .
drwxrwxr-x 3 rperez rperez 4096 Oct 5 11:14 ..
-rw-rw-r-- 1 rperez rperez 208 Oct 5 11:20 composer.json
I have tried also this syntax:
COPY container-files /
But didn't work either. So I should ask: what's wrong? Why this keep failing once and once? What I am missing here?
The documentation addresses this with:
By default the docker build command will look for a Dockerfile at
the root of the build context. The -f, --file, option lets you
specify the path to an alternative file to use instead. This is useful
in cases where the same set of files are used for multiple builds. The
path must be to a file within the build context. If a relative path is specified then it is interpreted as relative to the root of the context.
In this case I think is
COPY workspace/container-files/home/docker-user/.composer/composer.json /home/docker-user/.composer
I use supervisor to run cron and nginx, the problem is when i try to COPY or VOLUME mount my cron files, it does not run my cron files in /etc/cron.d
But when I exec -it <container_id> bash into the container and create the exact same cron file from inside, it is immediately recognized and runs as it should.
Dockerfile :
FROM phusion/baseimage:latest
ENV TERM xterm
ENV HOME /root
RUN apt-get update && apt-get install -y \
nginx \
supervisor \
curl \
nano \
net-tools
RUN rm -rf /etc/nginx/*
COPY nginx_conf /etc/nginx
COPY supervisor_conf /etc/supervisor/
RUN mkdir -p /var/log/supervisor
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
CMD /usr/bin/supervisord
The cron itself
* * * * * root curl --silent http://127.0.0.1/cronjob/cron_test_docker.php >> /var/www/html/log/docker_test.log 2>&1
cron and nginx run through supervisor
[supervisord]
nodaemon = true
[program:nginx]
command = /usr/sbin/nginx -g "daemon off;"
autostart = true
[program:cron]
command = /usr/sbin/cron -f
autostart = true
The logs inside /var/log/supervisor/ relating to cron for stdout and stderr are empty.
I also tried stripping out supervisor and running cron on its own through phusion and CMD cron -f but got the same issue of it not working when the source is external(COPY or VOLUME) and magically works when created inside the container.
Initially believed it to be a permissions issue and tried chmod 644 (as this was the permission a file created in the container had) on all files that were the result of COPY into.
RUN chmod 644 /etc/cron.d/
After which tried every possible combination of permissions with rwx to no avail.
Also, tried to append the line of the cronjob into /etc/crontab but it is not recognized in crontab -l.
COPY crontab /tmp/crontab
RUN cat crontab >> /etc/crontab
It would be really handy if it worked just when it was created through COPY or VOLUME as it is a hassle to create it manually in the container everytime.
Any help would be greatly appreciated!
Edit 1 :
Some additional information about the file permissions after COPY or VOLUME.
When I perform
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
Inside the container running ls -l inside /etc/cron.d/ shows
-rw-r--r-- 1 root root 118 Jul 20 11:03 wwwcron-cron-docker_test
When I mount the folder through my docker-compose through VOLUME
volumes:
- ./server/crontabs:/etc/cron.d
ls -l shows
-rwxrwxrwx 1 1000 staff 118 Jul 20 11:03 wwwcron-cron-docker_test
In addition if I manually create the cron file in the container it looks like this and this works
-rw-r--r-- 1 root root 118 Jul 22 15:50 wwwcron-cron-docker_test_inside_docker
Clearly there are very different permissions and ownership when making COPY or VOLUME. But making a COPY with exact permissions does not work but seems to work when created in the container.
Thanks to #BMitch was able to find the issue which was related to line endings since my host machine was windows and the cron file origin was windows as well there was a disparity in the line endings thereby cron did not pick it up automatically.
I added this line to my Dockerfile and it works like a charm
RUN find /etc/cron.d/ -type f -print0 | xargs -0 dos2unix
And iterating on that the size of the file is indeed 1 byte smaller when a dos2unix is run, so you can verify if this operation indeed occurred.
-rw-r--r-- 1 root root 117 Jul 25 08:33 wwwcron-cron-docker_test
Have you tried installing the crontab as a separate command in the Dockerfile?
i.e.
...
COPY crontabs /path/to/crontab.txt
RUN crontab -u myUser /path/to/crontab.txt
...
I am using a very simple .travis.yml to compile a cpp program via docker in Travis-CI. (My motivation is to experiment running docker in Travis CI.)
sudo: required
services:
- docker
before_install:
- docker pull glot/clang
script:
- sudo docker run --rm -v "$(pwd)":/app -w /app glot/clang g++ main.cpp
But the build is failing with following error:
/usr/bin/ld: cannot open output file a.out: Permission denied. This is regardless of whether I use sudo or not. Can someone help me out figuring out the root cause and help fix this? Thanks.
I would suggest you to set mounting path explicitly rather then doing it with $(pwd). Then you need to check the permissions from inside the container. Try something like that:
sudo docker run --rm -v "$(pwd)":/app -w /app glot/clang stat /app
This will show folder permissions. Probably noone is able to write into this folder.
Also you should avoid building your software using root permissions, it's not secure. Create non-priveleged user and use them when you running the compiler.
UPD:
I cannot reproduce this issue with docker 1.6.0, probably it's caused by some filesystem settings persisted by Travis-CI virtual machine. This is what I have on my localhost:
➜ /tmp mkdir /tmp/code
➜ /tmp echo "int main(){}" > /tmp/code/main.cpp
➜ /tmp echo "g++ main.cpp && ls -l" > /tmp/code/build.sh
➜ /tmp docker run --rm -v /tmp/code:/app -w /app glot/clang bash /app/build.sh
total 20
-rwxr-xr-x 1 glot glot 8462 Dec 30 10:19 a.out
-rwxrwxr-x 1 glot glot 22 Dec 30 10:17 build.sh
-rw-rw-r-- 1 glot glot 13 Dec 30 10:10 main.cpp
As you see, the resulting binary appears in /app folder
With docker I would like to offer a vm to each client to compile and execute a C program in only one file.
For that, I share a folder with the docker and the host thanks to a dockerfile and the command "ADD".
My folder is like that:
folder/id_user/script.sh
folder/id_user/code.c
In script.sh:
gcc ./compil/code.c -o ./compil/code && ./compil/code
My problem is in the doc we can read this for ADD:
All new files and directories are created with mode 0755, uid and gid 0.
But when I launch "ls" on the file I have:
ls -l compil/8f41dacd-8775-483e-8093-09a8712e82b1/
total 8
-rw-r--r-- 1 1000 1000 51 Feb 11 10:52 code.c
-rw-r--r-- 1 1000 1000 54 Feb 11 10:52 script.sh
So I can't execute the script.sh. Do you know why?
Maybe you wonder why proceed like that.
It's because if I do:
sudo docker run ubuntu/C pwd && pwd
result:
/
/srv/website
So we can see the first command is in the VM but not the second. I understand it might be normal for docker.
If you have any suggestion I'm pleased to listen it.
Thanks !
You can set up the correct mode by RUN command with chmod:
# Dockerfile
...
ADD script.sh /root/script.sh
RUN chmod +x /root/script.sh
...
The second question, you should use CMD command - && approach does work in Dockerfile, try to put this line at the end of your Dockerfile:
CMD pwd && pwd
then docker build . and you will see:
root#test:/home/test/# docker run <image>
/
/
Either that your you can do:
RUN /bin/sh /root/script.sh
to achieve the same result