I want create docker-containers with volumes and custom group. But faced with mistake with permission inside container. All file is have for example 'custom-group' and work fine, but the Document folder is have by default root group. I think this due to volumes. How to Document folder set 'custom-group'. My code is below
volumes:
- /base/documents:/app/documents:rw
The uid/gid inside of the container is typically the same as the uid/gid outside of the container, on the host (user namespaces are off by default and wouldn't solve this problem, in fact they would make it worse). Note that this is uid/gid, and not user name and group name, since containers can have their own /etc/passwd and /etc/group files.
You need to run your container with the uid/gid matching that of the files on your host. That is done with the user section of a compose file, or -u option on the docker run command line, e.g.:
docker run -u "$(id -u):$(id -g)" -v /base/documents:/app/documents:rw ...
or in the compose file:
user: "1000:1000"
If your application must run as root, then there are a lot fewer options to handle this. My own preference is to dynamically handle the uid/gid from an entrypoint that starts up as root, fixes the uid/gid inside the container to match the host (looking at the volume owner), and then drops from root to the container user for running the application. For more details on how that's done, you can see my base image repo, including the nginx example and fix-perms script.
use root user in your docker-compose to get full permission
EX:-
node-app:
container_name: node-app
image: node
user: root
volumes:
- ./:/home/node/app
- ./node_modules:/home/node/app/node_modules
- ./.env.docker:/home/node/app/.env
NOTE:- user: root => gives you a full permission of your volumne
Related
I am checking the docker documentation on how to use named volumes to share data between containers.
In Populate a volume using a container it is specified that:
If you start a container which creates a new volume, as above, and the container has files or directories in the directory to be mounted (such as /app/ above), the directory’s contents are copied into the volume. The container then mounts and uses the volume, and other containers which use the volume also have access to the pre-populated content.
So I did a simple example where:
I start a container which creates the volume and mounts it to a directory with existing files
I start a second container on which I mount the volume and indeed I can see the first container's files.
So far so good.
However I wanted to see if it is possible to have pre-populated content from more than one containers.
What I did was
Create two simple images which have their respective configuration files in the same directory
FROM alpine:latest
WORKDIR /opt/test
RUN mkdir -p "/opt/test/conf" && \
echo "container from image 1" > /opt/test/conf/config_1.cfg
FROM alpine:latest
WORKDIR /opt/test
RUN mkdir -p "/opt/test/conf" && \
echo "container from image 2" > /opt/test/conf/config_2.cfg
Create a docker compose which defines a named volume which is mounted on both services
services:
test_container_1:
image:
test_image_1
volumes:
- test_volume:/opt/test/conf
tty: true
test_container_2:
image:
test_image_2
volumes:
- test_volume:/opt/test/conf
tty: true
volumes:
test_volume:
Started the services.
> docker-compose -p example up
Creating network "example_default" with the default driver
Creating volume "example_test_volume" with default driver
Creating example_test_container_2_1 ... done
Creating example_test_container_1_1 ... done
Attaching to example_test_container_1_1, example_test_container_2_1
According to the logs container_2 was created first and it pre-populated the volume. However, the volume was then mounted to container_1 and the only file available on the mount was apparently /opt/test/conf/config_2.cfg effectively removing config_1.
So my question is, if it is possible to have a volume populated with data from 2 or more containers.
The reason I want to explore this, is so that I can have additional app configuration loaded from different containers, to support a multi tenant scenario, without having to rework the app to read the tenant configuration from different folders.
Thank you in advance
Once there is any content in a named volume at all, Docker will never automatically copy content into it. It will not merge content from two different images, update the volume if one of the images changes, or anything else.
I'd advise you to ignore the paragraph you quote in the Docker documentation. Assume any volume you mount into the container is initially empty. This matches the behavior you'll get with Docker bind-mounts (host directories), Kubernetes persistent volumes, and basically any other kind of storage besides Docker named volumes proper. Don't mount a volume over the content in your image.
If you can, restructure your application to avoid sharing files at all. One common use of named volumes I see is trying to republish static assets to a reverse proxy, for example; rather than trying to use a named volume (which will never update itself) you can COPY the static assets into a dedicated Web server image. This avoids the various complexities around trying to use a volume here.
If you really don't have a choice in the matter, then you can approach this with dedicated code in both of the containers. The basic setup here is:
Have a data directory somewhere outside your application directory, and mount the volume there.
Include the original files in the image somewhere different.
In an entrypoint wrapper script, copy the original files into the data directory (the mounted volume).
Let's say for the sake of argument that you've installed the application into /opt/test, and the data directory will be /etc/test. The entrypoint wrapper script can be as little as
#!/bin/sh
# Copy config files from the application tree into the config tree
# (overwriting anything that's already there)
cp /opt/test/* "$TEST_CONFIG_DIR"
# Run the main container command
exec "$#"
In the Dockerfile, you need to make sure that directory exists (and if you'll use a non-root user, that user needs permission to write to it).
FROM alpine
WORKDIR /opt/test
COPY ./ ./
ENV TEST_CONFIG_DIR=/etc/test
RUN mkdir "$TEST_CONFIG_DIR"
ENTRYPOINT ["./entrypoint.sh"]
CMD ["./my_app"]
Finally, in the Compose setup, mount the volume on that data directory (you can't use the environment variable, but consider the filesystem path part of the image's API):
version: '3.8'
volumes:
test_config:
services:
one:
build: ./one
volumes:
- test_config:/etc/test
two:
build: ./two
volumes:
- test_config:/etc/test
You would be able to run, for example,
docker-compose run one ls /etc/test
docker-compose run two ls /etc/test
to see both sets of files appear there.
The entrypoint script is code you control. There's nothing especially magical about it beyond the final exec "$#" line to run the main container command. If you want to ignore files that already exist, for example, or if you have a way to merge in changes, then you can implement something more clever than a simple cp command.
I have the following docker-compose:
version: '3.7'
services:
db:
image: bitnami/mongodb:5.0.6
volumes:
- "/app/local-data:/data/db"
env_file: ./db/.env
The problem is data does not persist between docker-compose up/down and docker does not seem to use /app/local-data even though it creates it.
When I run docker-compose, container starts and works naturally. The directory /app/local-data is created by docker, however Mongodb does not populate it, and no r/w error is being shown on console. This makes me thing a temporary volume is assigned to container instead.. But if that is true then why docker still creates /app/local-data and not using it?
Any ideas how can I debug this?
Docker directives like volumes: don't know anything about what's actually running in the image. That directive creates the specified host and container paths if required, and bind-mounts the host path into the container path. It's up to the application code to use that directory (or not).
If you look at the bitnami/mongodb Docker Hub page under "Persisting your database", the database is configured to store data in the /bitnami/mongodb directory inside the container, and that directory needs to be the second volumes: path. Also note the requirement that the data directory needs to be writable by user ID 1001, which may or may not exist on your host (there's no specific requirement to create it).
volumes:
- "/app/local-data:/bitnami/mongodb"
# ^^^^^^^^^^^^^^^^
sudo chown -R 1001 /app/local-data
sudo docker-compose up -d
I'm having a bit of bother with the following docker compose setup:
version: '2.4'
services:
graph_dev:
image: neo4j:3.5.14-enterprise
container_name: amlgraph_dev
user: 100:101
ports:
- 7474:7474
- 7473:7473
- 7687:7687
volumes:
- neo4jbackup:/backup
- neo4jdata:/data
volumes:
neo4jbackup:
neo4jdata:
I would like to run the neo4j-admin command, which must be run as the user 100 (_apt). However, the volume I need to backup to neo4jbackup, is mounted as root and _apt can't write there.
How do I create a volume that _apt can write to? The user _apt:neo4j obviously does not exist on the host. There are no users for which I have root on the docker image.
I can think of two options,
run neo4j docker container as a valid LINUX user and group and give that user access to a backup folder. Here is what my script looks like (I don't use compose currently) to run neo4j in docker under the current user
docker run
--user "$(id -u):$(id -g)"
Here is an article that covers doing the same thing with compose
https://medium.com/faun/set-current-host-user-for-docker-container-4e521cef9ffc
(hacky?) but you could run neo4j-admin outside docker, or in another container in a process that does have access to the backup volume? (I hear you want to run it as root?)
but of course I'm wondering why the backup process or db backup would be owned by root (as opposed to owned by a db owner or backup account...) Personally I feel it is best practice to avoid using root account, whenever possible.
I ended up solving this problem by running the command as _apt as required (docker-compose run graph_dev) and the using docker exec -it -u neo4j:neo4j graph_dev /bin/bash to copy the file over to the backup directory. Not elegant but works.
Is there better way to avoid folder permission issues when a relative folder is being set in a docker compose file when using manjaro?
For instance, take the bitnami/elasticsearch:7.7.0 image as an example:
This image as an example will always throw the ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/bitnami/elasticsearch/data/nodes]; error.
I can get around in by:
create the data directory with sudo, followed by chmod 777
attaching a docker volume
But I am looking for a bit easier to manage solution, similar to the docker experience in Ubuntu and OSX which I do not have to first create a directory with root in order for folder mapping to work.
I have made sure that my user is in the docker group by following the post install instructions on docker docs. I have no permission issues when accessing docker info, or sock.
docker-compose.yml
version: '3.7'
services:
elasticsearch:
image: bitnami/elasticsearch:7.7.0
container_name: elasticsearch
ports:
- 9200:9200
networks:
- proxy
environment:
- ELASTICSEARCH_HEAP_SIZE=512m
volumes:
- ./data/:/bitnami/elasticsearch/data
- ./config/elasticsearch.yml:/opt/bitnami/elasticsearch/config/elasticsearch.yml
networks:
proxy:
external: true
I am hoping for a more seamless experience when using my compose files from git which works fine in other systems, but running into this permission issue on the data folder on manjaro.
I did check other posts on SO, some some are temporary, like disabling selinux, while other require running docker with the --privileged flag, but I am trying to do with from compose.
This has nothing to do with the Linux distribution but is a general problem with Docker and bind mounts. A bind mount is when you mount a directory of your host into a container. The problem is that the Docker daemon creates the directory under the user it runs with (root) and the UID/GIDs are mapped literally into the container.
Not that it is advisable to run as root, but depending on your requirements, the official Elasticsearch image (elasticsearch:7.7.0) runs as root and does not have this problem.
Another solution that would work for the bitnami image is to make the ./data directory owned by group root and group writable, since it appears the group of the Elasticsearch process is still root.
A third solution is to change the GID of the bitnami image to whatever group you had the data created with and make it group writable.
version: '2'
services:
web:
build:
context: ./
dockerfile: deploy/web.docker
volumes:
- ./:/var/www
ports:
- "8080:80"
links:
- app
How can I change permission (chmod) /var/www automatically when docker-compose up -d --build?
When bind-mounting a directory from the host in a container, files and directories maintain the permissions they have on the host. This is by design: when using a bind-mount, you're giving the container access to existing files from the host, and Docker won't make modifications to those files; doing so would be very dangerous (for example, bind-mounting your home-directory would change file permissions of your host's home directory, possibly leading to your machine no longer being usable).
To change permissions of those files, change their permissions on the host.
You can find more information on this in another answer I posted on StackOverflow: https://stackoverflow.com/a/29251160/1811501
you can add the permissions after an extra column like:
volumes:
- ./:/var/www:ro #read only
Actually this is a little more complicated, some better clarity would be required.
If we mount the host folder folder_a into guest folder folder_b and the folder_b was already created, "chown"-ed and "chmod"-ed when the container image was created (Dockerfile), then folder_b defined permissions must take precedence from the guest perspective. This is what is happening on my boxes. As the guest user is not supposed to exist in the host an answer like "add the permissions to the host" is not very beautiful, it would mean "give write permissions to everybody".
If folder_b was not already "prepared" for the mount upfront, but it is mounted "on the fly"; I have not found any way to change the permissions contextually. Trying to reason out loud: the docker daemon on the host does not know anything about the guest users. I am also curious if a chmod/chown on the fly with guest names is technically possible as an evolution of docker. My initial expectation is that the on the fly mount inherited the parent folder permissions ( if /mnt/a is owned by app_user, then mounting in /mnt/a/myMount would preserve app_user ownership to the child folder as well, however, the new mounted folder is owned by root in my tests )
From a security perspective, the guest operating the mounts should never mount its own secured/sensitive folders to the guests. It is generally assumed that if you give a volume to a guest, the guest must be able to execute any operation on it. However mounting "on the fly" produces a "root" owned folder inside the guest as root is the only user known to both guest and host.