I have the following setup:
selenium-chrome:
image: selenium/node-chrome-debug:3.141.59-neon
container_name: chrome-e2e
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
- SHM-SIZE=2g
- GRID_DEBUG=false
- NODE_MAX_SESSION=1
- NODE_MAX_INSTANCES=5
- TZ=Europe/Brussels
hostname: chrome-e2e
networks:
- build-network
ports:
- 5900:5900
volumes:
- ./target:/home/seluser/Downloads
Selenium tests are run inside the container, the actual test code is outside of the container. Using Maven we handle the lifecycle of the containers.
As you can see I mounted the Chrome download folder (inside the container) to the target-folder of my application. All is mounted well but when Chrome tries to download a file, permission is denied to write to /home/seluser/Downloads.
The UID and GID of /home/seluser/Downloads is set to 2100:2100 by Docker. Chrome itself is run via the seluseruser.
What do I need to do to give seluser the permission to write to a folder owned by 2100?
Thanks in advance.
Regards
Bind mounts in Linux do not perform any namespacing on the uid or gid, and host mounts are running a bind mount under the covers. So if the uid inside the container is different from the uid on the host, you'll get permission issues. I've worked around this in other containers with a fix-perms script. Implementing that looks like the following Dockerfile:
FROM selenium/node-chrome-debug:3.141.59-neon
COPY --from=sudobmitch/base:scratch /usr/bin/gosu /usr/bin/fix-perms /usr/bin/
COPY entrypoint.sh /entrypoint.sh
# use a chmod here if you cannot fix permissions outside of docker
RUN chmod 755 /entrypoint.sh
USER root
ENTRYPOINT [ "/entrypoint.sh" ]
The entrypoint.sh looks like:
#!/bin/sh
if [ "$(id -u)" = "0" -a -d "/home/seluser/Downloads" ]; then
fix-perms -r -u seluser /home/seluser/Downloads
exec gosu seluser /opt/bin/entry_point.sh "$#"
else
exec /opt/bin/entry_point.sh "$#"
fi
What's happening here is the container starts as root, and the fix-perms script adjust the seluser inside the container to match the uid of the /home/seluser/Downloads directory. The exec gosu then runs your container process as the seluser as the new pid 1.
You can see the code used to implement this at: https://github.com/sudo-bmitch/docker-base
I've discussed this method in several of my presentations, including: https://sudo-bmitch.github.io/presentations/dc2019/tips-and-tricks-of-the-captains.html#fix-perms
You can create your own Dockerfile:
FROM selenium/node-chrome-debug:3.141.59-neon
VOLUME /home/seluser/Downloads
and the docker-compose.yml should be:
selenium-chrome:
build: .
container_name: chrome-e2e
depends_on:
- selenium-hub
...
The VOLUME directive will create the directory with the current user (seluser in this case) so that Docker will not have to create it with a different user.
Related
I was trying to setup Rails console in my dockerized container. The entire application has multiple components and I have set up the orchestration using docker-compose. Here is the relevant service from my docker-compose.yml file.
app:
image: <image_name>
# mount the current directory (on the host) to /usr/src/app on the container, any changes in either would be reflected in both the host and the container
tty: true
volumes:
- .:/usr/src/app
# expose application on localhost:36081
ports:
- "36081:36081"
# application restarts if stops for any reason - required for the container to restart when the application fails to start due to the database containers not being ready
restart: always
depends_on:
- other-db
# the environment variables are used in docker/config/env_config.rb to connect to different database containers
container_name: application
environment:
- CONSOLE=$CONSOLE
My Dockerfile has the following command ENTRYPOINT /usr/src/app/docker-entrypoint.sh
And in the docker-entrypoint.sh
#!/bin/bash
echo "waiting for all db connections to be healthy... Sleeping..."
sleep 1m
mkdir -p /usr/src/app/tmp/pids/
mkdir -p /usr/src/app/tmp/sockets/
if [ "$CONSOLE" = "Y" ];
then
echo "Starting Padrino console"
bundle exec padrino console
fi
When I run
export CONSOLE=Y
docker-compose -f docker-compose.yml up -d && docker attach application
The console starts up and I see >> but I cannot type in it. Where am I going wrong?
Try starting your container with -i mode.
-i, --interactive Attach container's STDIN
something like
docker-compose -f docker-compose.yml up -i && docker attach application
you can also mix -d and -i as per need.
With help from this post, I figured that I was missing stdin_open: true in the docker-compose.yml. Adding it worked like a breeze.
Here's a minimal docker-compose.yml:
version: "3.6"
services:
foo:
user: "${UID}:${GID}"
image: node:latest
container_name: foo
working_dir: /var/www/foo
volumes:
- bar:/var/www/foo/bar
volumes:
bar:
After running docker-compose up the named volume is created at /var/lib/docker/volumes/project_bar, and the project_bar directory is owned by root - this is fine. Although the _data directory inside it is also own by root - I would like it to be own by a specific, non-root user.
For example:
docker-compose run --rm foo sh -c "mkdir bar/foo"
will fail with mkdir: cannot create directory 'bar/foo': Permission denied.
I was under the impression that specifying user: "${UID}:${GID}" (note: uid/gid are available to docker, exported in shell they are) will solve this. Obviously, I was wrong.
Is there a way to have /var/lib/docker/volumes/project_bar/_data being own by specific user? Without chown'ing anything if possible. And without involving Dockerfile.
EDIT: To clarify, I would like /var/www/foo/bar in the container and its equivalent in /var/lib/docker/volumes to be owned by a specific, not root user.
You are either involving the Dockerfile, or you need to setup the volume with a separate process. The named volume will be initialized with the contents of the image, including files, directories, owners, and permissions. This happens when the volume is empty or non-existent.
So your options include:
update the image to include the desired ownership of that directory. Note if the image defines this as a volume, your change must preced that definition.
create a volume in advance with the desired ownership and mount that in the container
modify the entrypoint of the container to run as root, fix the permissions, and then switch to the desired user (using something like gosu)
Expanding on the second option, you can create a volume with the desired ownership and content by running a temporary container to do the task:
docker container run --rm -v bar-data:/data busybox /bin/sh -c \
"touch /data/.initialized && chown -R ${UID}:${GID} /data"
Then in your compose file you can declare that volume as external:
version: "3.6"
services:
foo:
user: "${UID}:${GID}"
image: node:latest
container_name: foo
working_dir: /var/www/foo
volumes:
- bar:/var/www/foo/bar
volumes:
bar:
# tell compose this volume is externally created with a different name
external: true
name: bar-data
That will create a volume with a single file, needed to prevent docker from reinitializing an empty volume from the image contents. This also means you need to load any data you want in the volume since it will be otherwise empty. A common method to copy data into a volume involves using tar:
tar -cC source_dir . | \
docker run --rm -i -v foo-data:/target busybox tar -xC /target
Yes,
I get this when I try to run traefik with https. Problem is I mount the dir on my Win7 machine but I cant chmod the file.
The mount is working but file permissions are off.
looks like this:
volumes
- d:/docker/traefikcompose/acme/acme.json:/etc/traefik/acme/acme.json:rw
traefik | time="2018-09-04T12:57:11Z" level=error msg="Error
starting provider *acme.Provider: unable to get ACME account :
permissions 777 for /etc/traefik/acme/acme.json are too open, please
use 600"
If I remove the acme.json file I get this:
ERROR: for traefik Cannot start service traefik: b'OCI runtime create
failed: container_linux.go:348: starting container process caused
"process_linux.go:402: container init caused \"rootfs_linux.go:58:
mounting \\\"/d/docker/traefikcompose/acme/acme.json\\\" to
rootfs
\\\"/mnt/sda1/var/lib/docker/aufs/mnt/c84d8644252848bde8f0322bafba3d206513ceb8479eb95aeee0b4cafd4a7251\\\"
at
\\\"/mnt/sda1/var/lib/docker/aufs/mnt/c84d8644252848bde8f0322bafba3d206513ceb8479eb95aeee0b4cafd4a7251/etc/traefik/acme/acme.json\\\"
caused \\\"not a directory\\\"\"": unknown: Are you trying to
mount a directory onto a file (or vice-versa)? Check if the specified
host path exists and is the expected type'
I did finally find the solution thanks to Cooshals kind help,
we have to ssh into the virtualbox-machine and make the file there, and then point it out right from the docker-compose.yml, in this case I did like this:
docker-machine ssh default
touch /var/acme.json
chmod 600 /var/acme.json
Then in my docker-compose:
volumes:
- /var/:/var/acme.json
Finally in traefik.toml:
[acme]
storage = "acme.json"
In addition to the above answer, to automate the creation of the acme.json file and assign the required permissions, create a Dockerfile and call it in your docker.compose.yml
FROM traefik:2.2
RUN touch /acme.json \
&& chmod 600 /acme.json
I solved this problem with a named docker volume:
docker-compose.yml
(only showing the relevant parts of the file)
services:
traefik:
environment:
- TRAEFIK_CERTIFICATESRESOLVERS_LE_ACME_STORAGE=/acme/acme.json
volumes:
- acme:/acme
volumes:
acme:
This just solved it for me:
Have WSL2 installed in Windows 10
Use PowerShell and navigate to the directory where your acme.json file is
Type wsl, this wil open the same location but now from WSL2
Type chmod 600 acme.json
Done!
I have the same problem as you, wanted to have the acme.json file outside the container/volume, that is, on the host FS. This way I wanted to make backups easy since my tests would exceed the let's encrypt / ACME quota quite fast at times.
Docker Windows
Turns out on Docker Windows you get this permission inside traefik container:
-rwxrwxrwx 1 root root 0 Dec 22 15:21 acme.json and on Linux
Docker Linux (ubuntu 22.04)
If the traefik creates the file on the host side using something like:
docker run -v ./acme:/acme ... traefik
On Linux docker the container side looks different:
-rw------- 1 root root 15.7K Dec 22 15:14 acme.json
But on the host I also have this:
-rw------- 1 root root 15.7K Dec 22 15:14 acme.json
Which means that my normal user can't see/backup or modify that file.
I think there is currently no sufficient support in maintaining this file on the host FS side.
Recommendation
Store this file inside a docker volume and access it using 'docker cp':
Backup:
docker container cp traefik:/acme/acme.json .
Restore:
docker container cp acme.json traefik:/acme/
docker exec -it traefik -> chmod 0700 /acme/acme.json
docker container restart traefik
This can be solved using a Dockerfile / entrypoint.sh and works like this:
Dockerfile
FROM traefik:v2.9.4
COPY entrypoint.sh /
ENTRYPOINT [ "/entrypoint.sh" ]
CMD ["traefik"]
entrypoint.sh
#! /bin/sh
set -e
echo "Setting acme.json permissions 0600"
touch /works
touch /acme/acme.json
chmod 600 /acme/acme.json
chown root:root /acme
chown root:root /acme/acme.json
# first arg is `-f` or `--some-option`
if [ "${1#-}" != "$1" ]; then
set -- traefik "$#"
fi
# if our command is a valid Traefik subcommand, let's invoke it through Traefik instead
# (this allows for "docker run traefik version", etc)
if traefik "$1" --help >/dev/null 2>&1
then
set -- traefik "$#"
else
echo "= '$1' is not a Traefik command: assuming shell execution." 1>&2
fi
exec "$#"
In the docker-compose.yaml I had:
traefik:
#image: traefik:v2.9.4
build: traefik/
So a docker compose build && docker compose up -d updated the file permissions according to the script in the entrypoint.sh
Note: It is important to do the updates of the /acme/acme.json file from the entrypoint.sh as the volumes are mounted then already. This is not the case when only using a Dockerfile.
Note: I'm using docker compose but docker will also support this but with a different synatx on the commands.
Summary
I think this is also too much maintainance burden. In the docker community we should come up with a volume system which can set owners/modes on directories for the container and leave the files on the host be whatever owner/mode they have.
volumes:
"file:acme.json:/acme.json:root:root:0600"
Also if that file does not exist on the host, just created it. Linux docker does create it on the host while Docker Windows would fails to start the docker compose up -d command.
I'm running a container with jenkins using "docker outside of docker". My docker compose is:
---
version: '2'
services:
jenkins-master:
build:
context: .
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /dev/urandom:/dev/random
- /home/jj/jenkins/jenkins_home/:/var/jenkins_home
ports:
- "8080:8080"
So all containers launched from "jenkins container" are running in host machine.
But when I try to run docker-compose in "jenkins container" in a job thats needs a volume, it takes the path from host instead of jenkins. I mean, when I run docker-compose with
volumes:
- .:/app
It is mounted in /var/jenkins_home/workspace/JOB_NAME in the host but I want that it is mounted in /home/jj/jenkins/jenkins_home/workspace/JOB_NAME
Any idea for doing this with a "clean" mode?
P.D.: I did a workaround using environments variables.
Docker on the host will map the path as is from the request, and docker-compose will make the request with the path it sees inside the container. This leaves you with a few options:
Don't use host volumes in your builds. If you need volumes, you can use named volumes and use docker io to read in and out of those volumes. That would look like:
tar -cC data . | docker run -i --rm -v app-data:/target busybox /bin/sh -c "tar -xC /target". You'd reverse the docker/tar commands to pull data back out.
Make the path on the host match that of the container. On your host, if you have access to make a symlink in var, you can ln -s /home/jj/jenkins/jenkins_home /var/jenkins_home and then update your compose file to have the same path (you may need to specify /var/jenkins_home/. to follow the symlink).
Make the path of the container match that of the host. This may be the easiest option, but I'm not positive it would work (depends on where compose thinks it's running). Your Dockerfile for the jenkins master can include the following:
RUN mkdir -p /home/jj/jenkins \
&& ln -s /var/jenkins_home /home/jj/jenkins/jenkins_home
ENV JENKINS_HOME /home/jj/jenkins/jenkins_home
If the easy option doesn't work, you can rebuild the image from jenkins and change the JENKINS_HOME variable to match your environment.
Make your compose paths absolute. You can add some code to set a variable:
export CUR_DIR=$(pwd | sed 's#/var/jenkins_home#/home/jj/jenkins/jenkins_home#'). Then you can set your volume with that variable:
volumes:
- ${CUR_DIR:-.}:/app
I'm trying to mount a volume in docker-compose to apache image. The problem is, that apache in my docker is run under www-data:www-data but the mounted directory is created under root:root. How can I specify the user of the mounted directory?
I tried to run command setupApacheRights.sh. chown -R www-data:www-data /var/www but it says chown: changing ownership of '/var/www/somefile': Permission denied
services:
httpd:
image: apache-image
ports:
- "80:80"
volumes:
- "./:/var/www/app"
links:
- redis
command: /setupApacheRights.sh
I would prefer to be able to specify the user under which it will be mounted. Is there a way?
To achieve the desired behavior without changing owner / permissions on the host system do the following steps.
get the ID of the desired user and or group you want the permissions to match with executing the id command on your host system - this will show you the uid and gid of your current user and as well all IDs from all groups the user is in.
$ id
add the definition to your docker-compose.yml
user: "${UID}:${GID}"
so your file could look like this
php: # this is my service name
user: "${UID}:${GID}" # we added this line to get a specific user / group id
image: php:7.3-fpm-alpine # this is my image
# and so on
set the values in your .env file
UID=1000
GID=1001
3a. Alternatively you can extend your ~/.bashrc file with:
export UID GID
to define it globally rather than defining it in a .env file for each project.
If this does not work for you (like on my current distro, the GID is not set by this, use following two lines:
export UID=$(id -u)
export GID=$(id -g)
Thanks #SteenSchütt for the easy solution for defining the UID / GID globally.
Now your user in the container has the id 1000 and the group is 1001 and you can set that differently for every environment.
Note: Please replace the IDs I used with the user / group IDs you found on your host system. Since I cannot know which IDs your system is using I gave some example group and user IDs.
If you don't use docker-compose or want to know more different approaches to achieve this have a read through my source of information: https://dev.to/acro5piano/specifying-user-and-group-in-docker-i2e
If the volume mount folder does not exist on your machine, docker will create it (with root user), so please ensure that it already exists and is owned by the userid / groupid you want to use.
I add an example for a dokuwiki container to explain it better:
version: '3.5'
services:
dokuwiki:
user: "${UID}" # set a specific user id so the container can write in the data dir
image: bitnami/dokuwiki:latest
ports:
- '8080:8080'
volumes:
- '/home/manuel/docker/dokuwiki/data:/bitnami/dokuwiki/'
restart: unless-stopped
expose:
- "8080"
The dokuwiki container will only be able to initialize correctly if it has write access to the host directory /home/manuel/docker/dokuwiki/data.
If on startup this directory does not exist, docker will create it for us but it will have root:root as user & group. --> Therefor the container startup will fail.
If we create the folder before starting the container
mkdir -P /home/manuel/docker/dokuwiki/data
and then check with
ls -nla /home/manuel/docker/dokuwiki/data| grep ' \.$'
which uid and gid the folder has - and check that they match the ones we put in our .env file in step 3. above.
The bad news is there's no owner/group/permission settings for volume 😢. The good news is the following trick will let you bake it into your config, so it's fully automated 🎉.
In your Dockerfile, create an empty directory in the right location and with desired settings.
This way, the directory will already be present when docker-compose mounts to the location. When the server mounts during boot (based on docker-compose), the mounting action happily leaves those permissions alone.
Dockerfile:
# setup folder before switching to user
RUN mkdir /volume_data
RUN chown postgres:postgres /volume_data
USER postgres
docker-compose.yml
volumes:
- /home/me/postgres_data:/volume_data
source
First determine the uid of the www-data user:
$ docker exec DOCKER_CONTAINER_ID id
uid=100(www-data) gid=101(www-data) groups=101(www-data)
Then, on your docker host, change the owner of the mounted directory using the uid (100 in this example):
chown -R 100 ./
Dynamic Extension
If you are using docker-compose you may as well go for it like this:
$ docker-compose exec SERVICE_NAME id
uid=100(www-data) gid=101(www-data) groups=101(www-data)
$ chown -R 100 ./
You can put that in a one-liner:
$ chown -R $(docker-compose exec SERVICE_NAME id -u) ./
The -u flag will only print the uid to stdout.
Edit: fixed casing error of CLI flag. Thanks #jcalfee314!
Adding rw to the end of the volume mount worked for me:
services:
httpd:
image: apache-image
ports:
- "80:80"
volumes:
- "./:/var/www/app:rw"
links:
- redis
command: /setupApacheRights.sh
Set user www-data for this compose service
user: "www-data:www-data"
Example:
wordpress:
depends_on:
- db
image: wordpress:5.5.3-fpm-alpine
user: "www-data:www-data"
container_name: wordpress
restart: unless-stopped
env_file:
- .env
volumes:
- ./wordpress/wp-content:/var/www/html/wp-content
- ./wordpress/wp-config-local.php:/var/www/html/wp-config.php
If your volumes create ownership issue then you might need to find your volume mount path by
cmd: docker volume ls
After that identify your volume name then inspect your mount path
cmd: docker volume inspect <volume name>
check your mount point there and go on mount point on your docker host machine.
where check ownership of volume by
cmd: ls -l
if it's suggest root:root then change owneship here to your docker user.
cmd: chown docker_user_id:docker_group_id -R volume_path
Note: you can find your docker user id & user group id by entering into your docker bash & hit "id" cmd.
cmd: docker-compose run --rm <container_name> bash
cmd: id
output: uid=102(www-data) gid=102(www-data) groups=102(www-data)
Find similar thread here. https://www.hamaraweb.com/sms/407/docker-volume-ownership-issue-errno-13-permission-denied-bgb6ld/