Docker-compose set user and group on mounted volume - docker

I'm trying to mount a volume in docker-compose to apache image. The problem is, that apache in my docker is run under www-data:www-data but the mounted directory is created under root:root. How can I specify the user of the mounted directory?
I tried to run command setupApacheRights.sh. chown -R www-data:www-data /var/www but it says chown: changing ownership of '/var/www/somefile': Permission denied
services:
httpd:
image: apache-image
ports:
- "80:80"
volumes:
- "./:/var/www/app"
links:
- redis
command: /setupApacheRights.sh
I would prefer to be able to specify the user under which it will be mounted. Is there a way?

To achieve the desired behavior without changing owner / permissions on the host system do the following steps.
get the ID of the desired user and or group you want the permissions to match with executing the id command on your host system - this will show you the uid and gid of your current user and as well all IDs from all groups the user is in.
$ id
add the definition to your docker-compose.yml
user: "${UID}:${GID}"
so your file could look like this
php: # this is my service name
user: "${UID}:${GID}" # we added this line to get a specific user / group id
image: php:7.3-fpm-alpine # this is my image
# and so on
set the values in your .env file
UID=1000
GID=1001
3a. Alternatively you can extend your ~/.bashrc file with:
export UID GID
to define it globally rather than defining it in a .env file for each project.
If this does not work for you (like on my current distro, the GID is not set by this, use following two lines:
export UID=$(id -u)
export GID=$(id -g)
Thanks #SteenSchütt for the easy solution for defining the UID / GID globally.
Now your user in the container has the id 1000 and the group is 1001 and you can set that differently for every environment.
Note: Please replace the IDs I used with the user / group IDs you found on your host system. Since I cannot know which IDs your system is using I gave some example group and user IDs.
If you don't use docker-compose or want to know more different approaches to achieve this have a read through my source of information: https://dev.to/acro5piano/specifying-user-and-group-in-docker-i2e
If the volume mount folder does not exist on your machine, docker will create it (with root user), so please ensure that it already exists and is owned by the userid / groupid you want to use.
I add an example for a dokuwiki container to explain it better:
version: '3.5'
services:
dokuwiki:
user: "${UID}" # set a specific user id so the container can write in the data dir
image: bitnami/dokuwiki:latest
ports:
- '8080:8080'
volumes:
- '/home/manuel/docker/dokuwiki/data:/bitnami/dokuwiki/'
restart: unless-stopped
expose:
- "8080"
The dokuwiki container will only be able to initialize correctly if it has write access to the host directory /home/manuel/docker/dokuwiki/data.
If on startup this directory does not exist, docker will create it for us but it will have root:root as user & group. --> Therefor the container startup will fail.
If we create the folder before starting the container
mkdir -P /home/manuel/docker/dokuwiki/data
and then check with
ls -nla /home/manuel/docker/dokuwiki/data| grep ' \.$'
which uid and gid the folder has - and check that they match the ones we put in our .env file in step 3. above.

The bad news is there's no owner/group/permission settings for volume 😢. The good news is the following trick will let you bake it into your config, so it's fully automated 🎉.
In your Dockerfile, create an empty directory in the right location and with desired settings.
This way, the directory will already be present when docker-compose mounts to the location. When the server mounts during boot (based on docker-compose), the mounting action happily leaves those permissions alone.
Dockerfile:
# setup folder before switching to user
RUN mkdir /volume_data
RUN chown postgres:postgres /volume_data
USER postgres
docker-compose.yml
volumes:
- /home/me/postgres_data:/volume_data
source

First determine the uid of the www-data user:
$ docker exec DOCKER_CONTAINER_ID id
uid=100(www-data) gid=101(www-data) groups=101(www-data)
Then, on your docker host, change the owner of the mounted directory using the uid (100 in this example):
chown -R 100 ./
Dynamic Extension
If you are using docker-compose you may as well go for it like this:
$ docker-compose exec SERVICE_NAME id
uid=100(www-data) gid=101(www-data) groups=101(www-data)
$ chown -R 100 ./
You can put that in a one-liner:
$ chown -R $(docker-compose exec SERVICE_NAME id -u) ./
The -u flag will only print the uid to stdout.
Edit: fixed casing error of CLI flag. Thanks #jcalfee314!

Adding rw to the end of the volume mount worked for me:
services:
httpd:
image: apache-image
ports:
- "80:80"
volumes:
- "./:/var/www/app:rw"
links:
- redis
command: /setupApacheRights.sh

Set user www-data for this compose service
user: "www-data:www-data"
Example:
wordpress:
depends_on:
- db
image: wordpress:5.5.3-fpm-alpine
user: "www-data:www-data"
container_name: wordpress
restart: unless-stopped
env_file:
- .env
volumes:
- ./wordpress/wp-content:/var/www/html/wp-content
- ./wordpress/wp-config-local.php:/var/www/html/wp-config.php

If your volumes create ownership issue then you might need to find your volume mount path by
cmd: docker volume ls
After that identify your volume name then inspect your mount path
cmd: docker volume inspect <volume name>
check your mount point there and go on mount point on your docker host machine.
where check ownership of volume by
cmd: ls -l
if it's suggest root:root then change owneship here to your docker user.
cmd: chown docker_user_id:docker_group_id -R volume_path
Note: you can find your docker user id & user group id by entering into your docker bash & hit "id" cmd.
cmd: docker-compose run --rm <container_name> bash
cmd: id
output: uid=102(www-data) gid=102(www-data) groups=102(www-data)
Find similar thread here. https://www.hamaraweb.com/sms/407/docker-volume-ownership-issue-errno-13-permission-denied-bgb6ld/

Related

Undefined volume with Docker Compose

I wanted to translate this docker CLI command (from smallstep/step-ca) into a docker-compose.yml file to run with docker compose (version 2):
docker run -d -v step:/home/step \
-p 9000:9000 \
-e "DOCKER_STEPCA_INIT_NAME=Smallstep" \
-e "DOCKER_STEPCA_INIT_DNS_NAMES=localhost,$(hostname -f)" \
smallstep/step-ca
This command successfully starts the container.
Here is the compose file I "composed":
version: "3.9"
services:
ca:
image: smallstep/step-ca
volumes:
- "step:/home/step"
environment:
- DOCKER_STEPCA_INIT_NAME=Smallstep
- DOCKER_STEPCA_INIT_DNS_NAMES=localhost,ubuntu
ports:
- "9000:9000"
When I run docker compose up (again, using v2 here), I get this error:
service "ca" refers to undefined volume step: invalid compose project
Is this the right way to go about this? I'm thinking I missed an extra step with volume creation in docker compose projects, but I am not sure what that would be, or if this is even a valid use case.
The Compose file also has a top-level volumes: block and you need to declare volumes there.
version: '3.9'
services:
ca:
volumes:
- "step:/home/step"
et: cetera
volumes: # add this section
step: # does not need anything underneath this
There are additional options possible, but you do not usually need to specify these unless you need to reuse a preexisting Docker named volume or you need non-standard Linux mount options (the linked documentation gives an example of an NFS-mount volume, for example).
Citing the Compose specification:
To avoid ambiguities with named volumes, relative paths SHOULD always begin with . or ...
So it should be enough to make your VOLUME's host path relative:
services:
ca:
volumes:
- ./step:/home/step
If you don't intend to share the step volume with other containers, you don't need to define it in the top-level volumes key:
If the mount is a host path and only used by a single service, it MAY be declared as part of the service definition instead of the top-level volumes key.
it seems that docker-compose don't know the "volume" you created via command: sudo docker volume create my_xx_volume
so ,just manually mkdir to create a folder and chmod 777 <my_folder>, then your mysql docker will use it very well.
( in production env, don't use chmod but chown to change the file permission )

docker-compose: create named volume as specific user

Here's a minimal docker-compose.yml:
version: "3.6"
services:
foo:
user: "${UID}:${GID}"
image: node:latest
container_name: foo
working_dir: /var/www/foo
volumes:
- bar:/var/www/foo/bar
volumes:
bar:
After running docker-compose up the named volume is created at /var/lib/docker/volumes/project_bar, and the project_bar directory is owned by root - this is fine. Although the _data directory inside it is also own by root - I would like it to be own by a specific, non-root user.
For example:
docker-compose run --rm foo sh -c "mkdir bar/foo"
will fail with mkdir: cannot create directory 'bar/foo': Permission denied.
I was under the impression that specifying user: "${UID}:${GID}" (note: uid/gid are available to docker, exported in shell they are) will solve this. Obviously, I was wrong.
Is there a way to have /var/lib/docker/volumes/project_bar/_data being own by specific user? Without chown'ing anything if possible. And without involving Dockerfile.
EDIT: To clarify, I would like /var/www/foo/bar in the container and its equivalent in /var/lib/docker/volumes to be owned by a specific, not root user.
You are either involving the Dockerfile, or you need to setup the volume with a separate process. The named volume will be initialized with the contents of the image, including files, directories, owners, and permissions. This happens when the volume is empty or non-existent.
So your options include:
update the image to include the desired ownership of that directory. Note if the image defines this as a volume, your change must preced that definition.
create a volume in advance with the desired ownership and mount that in the container
modify the entrypoint of the container to run as root, fix the permissions, and then switch to the desired user (using something like gosu)
Expanding on the second option, you can create a volume with the desired ownership and content by running a temporary container to do the task:
docker container run --rm -v bar-data:/data busybox /bin/sh -c \
"touch /data/.initialized && chown -R ${UID}:${GID} /data"
Then in your compose file you can declare that volume as external:
version: "3.6"
services:
foo:
user: "${UID}:${GID}"
image: node:latest
container_name: foo
working_dir: /var/www/foo
volumes:
- bar:/var/www/foo/bar
volumes:
bar:
# tell compose this volume is externally created with a different name
external: true
name: bar-data
That will create a volume with a single file, needed to prevent docker from reinitializing an empty volume from the image contents. This also means you need to load any data you want in the volume since it will be otherwise empty. A common method to copy data into a volume involves using tar:
tar -cC source_dir . | \
docker run --rm -i -v foo-data:/target busybox tar -xC /target

Create a directory in docker-compose.yml file

I have a docker-compose yml that creates a sftp image on my docker. I'd like to write a script in the yml file as I want directories to be created automatically as soon as I run the docker-compose.yml.
Here's my yml file;
sftp:
image: atmoz/sftp
volumes:
- C:\tmp\sftp:/home/foo/upload
ports:
- "2222:22"
command: username:password:1001
Is there a way to write mkdir and chmod in this file?
You do not need to create and set mod manually, just pass the directory name to CMD and the entrypoint will create one for you. Here is the simplest example.
Define users in (1) command arguments, (2) SFTP_USERS environment
variable or (3) in file mounted as /etc/sftp/users.conf (syntax:
user:pass[:e][:uid[:gid[:dir1[,dir2]...]]] ..., see below for
examples)
using docker-compose
sftp:
image: atmoz/sftp
command: username:password:100:100:upload
it will create user name username and directory upload under /home/username
You can verify this using
docker exec -it --user username <container_id> bash -c "ls /home/username"
if you want to access upload files from host just add mounting in your docker-compose
sftp:
image: atmoz/sftp
command: username:password:100:100:upload
volumes:
- /host/upload:/home/username/upload
Examples
Simplest docker run example
docker run -p 22:22 -d atmoz/sftp foo:pass:::upload
User "foo" with password "pass" can login with sftp and upload files
to a folder called "upload". No mounted directories or custom UID/GID.
Later you can inspect the files and use --volumes-from to mount them
somewhere else (or see next example).
see the offical documentation

Docker user cannot write to mounted folder

I have the following setup:
selenium-chrome:
image: selenium/node-chrome-debug:3.141.59-neon
container_name: chrome-e2e
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
- SHM-SIZE=2g
- GRID_DEBUG=false
- NODE_MAX_SESSION=1
- NODE_MAX_INSTANCES=5
- TZ=Europe/Brussels
hostname: chrome-e2e
networks:
- build-network
ports:
- 5900:5900
volumes:
- ./target:/home/seluser/Downloads
Selenium tests are run inside the container, the actual test code is outside of the container. Using Maven we handle the lifecycle of the containers.
As you can see I mounted the Chrome download folder (inside the container) to the target-folder of my application. All is mounted well but when Chrome tries to download a file, permission is denied to write to /home/seluser/Downloads.
The UID and GID of /home/seluser/Downloads is set to 2100:2100 by Docker. Chrome itself is run via the seluseruser.
What do I need to do to give seluser the permission to write to a folder owned by 2100?
Thanks in advance.
Regards
Bind mounts in Linux do not perform any namespacing on the uid or gid, and host mounts are running a bind mount under the covers. So if the uid inside the container is different from the uid on the host, you'll get permission issues. I've worked around this in other containers with a fix-perms script. Implementing that looks like the following Dockerfile:
FROM selenium/node-chrome-debug:3.141.59-neon
COPY --from=sudobmitch/base:scratch /usr/bin/gosu /usr/bin/fix-perms /usr/bin/
COPY entrypoint.sh /entrypoint.sh
# use a chmod here if you cannot fix permissions outside of docker
RUN chmod 755 /entrypoint.sh
USER root
ENTRYPOINT [ "/entrypoint.sh" ]
The entrypoint.sh looks like:
#!/bin/sh
if [ "$(id -u)" = "0" -a -d "/home/seluser/Downloads" ]; then
fix-perms -r -u seluser /home/seluser/Downloads
exec gosu seluser /opt/bin/entry_point.sh "$#"
else
exec /opt/bin/entry_point.sh "$#"
fi
What's happening here is the container starts as root, and the fix-perms script adjust the seluser inside the container to match the uid of the /home/seluser/Downloads directory. The exec gosu then runs your container process as the seluser as the new pid 1.
You can see the code used to implement this at: https://github.com/sudo-bmitch/docker-base
I've discussed this method in several of my presentations, including: https://sudo-bmitch.github.io/presentations/dc2019/tips-and-tricks-of-the-captains.html#fix-perms
You can create your own Dockerfile:
FROM selenium/node-chrome-debug:3.141.59-neon
VOLUME /home/seluser/Downloads
and the docker-compose.yml should be:
selenium-chrome:
build: .
container_name: chrome-e2e
depends_on:
- selenium-hub
...
The VOLUME directive will create the directory with the current user (seluser in this case) so that Docker will not have to create it with a different user.

docker, MYSQL_ROOT_PASSWORD do not work

docker-compose:
mysql:
image: mysql:5.7.16
container_name: f_mysql
volumes:
- ./db:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: sheep
expose:
- '3306'
and I use docker exec input this container,
and I type echo $MYSQL_ROOT_PASSWORD, then I got sheep,
but the mysql root password still is '',
when I type 'mysql -uroot', I login mysql.
For me the issue was that I'd created the db volume with the random password option set, then disabled that, but hadn't cleared the volume. So no matter what changes I made to the docker-compose file, the old volume with the old login information was still there.
I had to docker volume ls to find the volume then docker volume rm <name> to remove it. After re-upping, everything worked.
Regarding other answers on this page, the format for specifying env variables is correct, you can use either
environment:
MYSQL_ROOT_PASSWORD: a_password
OR
environment:
- MYSQL_ROOT_PASSWORD=a_password
The image entrypoint script will never make changes to a database which is existing. If you mount an existing data directory into var/lib/mysql then MYSQL_ROOT_PASSWORD will have no effect.
Workaround
Remove all unused volumes: docker volume prune
Remove the volume from your database service: docker volume rm <db_data>
Down containers, remove volumes: docker-compose down --volumes
You need to fix your docker-compose file:
environment:
- MYSQL_ROOT_PASSWORD=sheep
The following is the full docker-compose that achieves what you want:
version: '2'
services:
mysql:
image: mysql:5.7.16
container_name: f_mysql
volumes:
- ./db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=sheep
expose:
- '3306'
Then with a docker exec -it f_mysql /bin/bash and inside the container mysql -u root -p, using sheep, as the password, will be the only way to connect to the mysql server.
This happens when your volume from a directory has wrong permission.
You can fix this letting docker to create directory itself.
In case you have an existent data, you can compare the new one with the previous one in order to apply correct chmod, because this depends on if docker/your-user is part of root group.
Please note that according to the official docker image: "none of those variables will have any effect if you start the container with a data directory that already contains a database". In fact, in this case, you have already a "mysql.user" table, and you should use the user info set there that there. The same thing happens when you try to restore a full dump.
This happened when the the mount directory has ea(extended attribute) on Mac.
It is better to delete the directory once and recreate it or check the permission with the xattr command.
$ ls -l ./db
$ xattr ls ./db
I had a same problem, and after a lot of try I found my solution. When I run first time the docker-composer, I left everything on the original settings like this:
environment:
MYSQL_ROOT_PASSWORD: MYSQL_ROOT_PASSWORD
Then I change the password, say "docker-compose up" but it was still MYSQL_ROOT_PASSWORD.
My solution was delete the "mysql" docker image from my disk. After that, the docker download everything again BUT also set my password for the root as well. Maybe this is not the best, but I am also a beginner in Docker.
So in nutshell the simple "docker-compose up" does not enough.

Resources