I am starting six or seven containers via a docker-compose file. One container is causing a major problem! Here is the relevant section:
services:
...
main-app:
image: mycompany/sys:1.2.3
container_name: "main-app-container"
ports:
- "8080:8080"
- "8009"
volumes:
- db_data:/var/lib/home/data:rw
- /opt/mycompany/sys/config:/opt/mycompany/sys/config:rw
networks:
- systeminternal
hostname: "mylocalhost.company.com"
volumes:
db_data:
driver: local
networks:
systeminternal:
When the main-app-container is started via docker-compose up (as the root user) the file system privileges in many of the directories in the committed container are all changed to root! This is running on Ubuntu 14.04, Docker 1.12.x (not sure which x).
We have another system where we run everything as a local user. When we exec a shell into that container, all the file privileges are of our local user that was ownership as it was committed. From googling, I am pretty sure it has something to do with the volumes, but could not find anything definitive. Any help is welcome!
This is the expected behavior for host-mounts, that said, everything inside /opt/mycompany/sys/config will be having the same UID/GID the files have on the host - that is by design.
Either change the files to the uid/gid you need on the host: chown -R 123:321 /opt/mycompany/sys/config or setup your container to be happy to use the uid/gid of the host.
It has nothing to do with docker-compose, it would happen the same way when you use
docker run -v /opt/mycompany/sys/config:/opt/mycompany/sys/config mycompany/sys:1.2.3
Related
Let's say I have a docker-compose file with two containers:
version: "3"
services:
app:
image: someimage:fpm-alpine
volumes:
- myvolume:/var/www/html
web:
image: nginx:alpine
volumes:
- myvolume:/var/www/html
volumes:
myvolume:
The app container contains the application code in the /var/www/html directory which gets updated with each version of the image, so I don't want this directory to be persistent.
Yet I need to share the data with the nginx container. If I use a volume or a host bind the data is persistent and doesn't get updated with a new version. Maybe there is a way to automatically delete a volume whenever I pull a new image? Or a way to share an anonymous volume?
i think its better for you to use anonymous volume
volumes:
- ./:/var/www/html
You would have to be willing to drop back to docker-compose version 2 and use data containers with the volumes_from directive.
Which is equivalent to --volumes-from on a docker run command.
This should work fine. The problem isn't with docker. You can use volumes to communicate in this way. If you run docker-compose up in a directory with the following compose file:
version: "3"
services:
one:
image: ubuntu
command: sleep 100000
volumes:
- vol:/vol
two:
image: ubuntu
command: sleep 100000
volumes:
- vol:/vol
volumes:
vol:
Then, in a 2nd terminal docker exec -it so_one_1 bash (you might have to do a docker ps to find the exact name of the container, it can change). You'll find yourself in a bash container. Change to the /vol directory cd /vol and then echo "wobble" > wibble.txt", then exit` the shell (ctrl-d).
In the same terminal you can then type docker exec -it so_two_1 bash (again, check the names). Just like last time you can cd /vol and type ls -gAlFh you'll see the wibble.txt file we created in the other container. You can even cat wibble.txt to see the contents. It'll be there.
So if the problem isn't docker, what can it be? I think the problem is that nginx isn't seeing the changes on the filesystem. For that, I believe that setting expires -1; inside a location block in the config will actually disable caching completely and may solve the problem (dev only).
I am quite new to docker and I try to build a LAMP stack with docker-compose. I have found a nice tutorial over there. I think I understood the difference between volumes and bind mounts, however, I guess I am running into a problem at some point. I want to make one of my folders available to the LAMP stack (my sources, residing in a folder 'src'). However, the sources are not visible within the /var/www/html folder.
My docker-compose file looks like this:
version: "3.7"
services:
mariadb:
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "no"
MYSQL_DATABASE: testdb
MYSQL_PASSWORD: testpassword
MYSQL_ROOT_PASSWORD: rootpwd
MYSQL_USER: testuser
TZ: Europe/Rome
image: "mariadb:10.5.2"
volumes:
- "mariadb-volume:/var/lib/mysql"
php-httpd:
image: "php:7.3-apache"
ports:
- "80:80"
volumes:
- ./src/:/var/www/html/
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- "mariadb:db"
ports:
- "8081:80"
volumes:
mariadb-volume: ~
Phpmyadmin works just fine, also the docker-compose runs without any warnings. My compose command is
docker-compose up -d --force-recreate
Interestingly, when I change "./src/" for "./DocumentRoot", the folder DocumentRoot is created on my host machine. However, placing files in DocumentRoot on the host or in /var/www/html in docker does not show the files on the docker or host, respectively. Nevertheless, I can say for sure that I am in the right directory at least.
Is there some trick or parameter I need to pass along to let docker see the files on my host?
Hah... thanks again. Your question has triggered another thought. It's quite natural to me, so I didn't mention it: When I execute the docker-compose from Desktop, everything works fine. However, if I execute it from my usual working directory, it does not. My usual working directory is a mounted volume with VeryCrypt on Windows. Obviously there are issues sharing the directory in the latter case.
Just in case anybody is experiencing that error too in the future.
I want to make one of my folders available to the LAMP stack (my
sources, residing in a folder 'src'). However, the sources are not
visible within the /var/www/html folder.
I think that there is a confusion about how mounts work with docker.
When you specify a mount for a docker container such as :
php-httpd:
image: "php:7.3-apache"
ports:
- "80:80"
volumes:
- ./src/:/var/www/html/
Only the container php-httpd will be set with the mount, not the other containers of your LAMP stack.
If you need to set that mount on other containers, do it explicitly on them.
Interestingly, when I change "./src/" for "./DocumentRoot", the folder
DocumentRoot is created on my host machine. However, placing files in
DocumentRoot on the host or in /var/www/html in docker does not show
the files on the docker or host, respectively.
That is the way which works the mounts. When the folder exists on the host (here src) , docker uses it to mount its content from host to container. When the folder doesn't exit on the host, Docker creates it.
I have finally found a solution. I am splitting the docker-compose file and I do the php-httpd part in a separate dockerfile. There, I can copy my sources into the dockercontainer.
It is not the original solution, so I would still be grateful for input on the problem why the bind mount does not work, but this solution works for me.
I have a VM and inside it, I am running an Elasticsearch Docker container built through docker-compose. It was working pretty well. Then after the power suddenly went out, I tried running the container back again but discovered an error that wasn't present before:
Then the container kept on restarting. And when I checked the file permissions (within the small window of time before the container restarts), I found this:
Here's my docker-compose.yml:
version: '2.3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.8.0
hostname: elasticsearch
restart: always
user: root
ports:
- "9200:9200"
- "9300:9300"
volumes:
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
env_file:
- devopsfw-elk.env
What is actually happening here? I'm fairly new to Docker and Elasticsearch and I'm very confused as to the errors that are occuring.
The problem is that the file has been corrupted, delete it and restart the container.
rm -i ./*elasticsearch.yml*
If you have problems to delete this, read this:
https://superuser.com/questions/197605/delete-a-corrupt-file-in-linux
Looks like the file was owned by the root user and has been corrupted, in order to delete the file, you have to use the super user access aka sudo , so correct command would be
sudo rm -i ./*elasticsearch.yml*
And after that, create a file and restart the conatainer.
Is there better way to avoid folder permission issues when a relative folder is being set in a docker compose file when using manjaro?
For instance, take the bitnami/elasticsearch:7.7.0 image as an example:
This image as an example will always throw the ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/bitnami/elasticsearch/data/nodes]; error.
I can get around in by:
create the data directory with sudo, followed by chmod 777
attaching a docker volume
But I am looking for a bit easier to manage solution, similar to the docker experience in Ubuntu and OSX which I do not have to first create a directory with root in order for folder mapping to work.
I have made sure that my user is in the docker group by following the post install instructions on docker docs. I have no permission issues when accessing docker info, or sock.
docker-compose.yml
version: '3.7'
services:
elasticsearch:
image: bitnami/elasticsearch:7.7.0
container_name: elasticsearch
ports:
- 9200:9200
networks:
- proxy
environment:
- ELASTICSEARCH_HEAP_SIZE=512m
volumes:
- ./data/:/bitnami/elasticsearch/data
- ./config/elasticsearch.yml:/opt/bitnami/elasticsearch/config/elasticsearch.yml
networks:
proxy:
external: true
I am hoping for a more seamless experience when using my compose files from git which works fine in other systems, but running into this permission issue on the data folder on manjaro.
I did check other posts on SO, some some are temporary, like disabling selinux, while other require running docker with the --privileged flag, but I am trying to do with from compose.
This has nothing to do with the Linux distribution but is a general problem with Docker and bind mounts. A bind mount is when you mount a directory of your host into a container. The problem is that the Docker daemon creates the directory under the user it runs with (root) and the UID/GIDs are mapped literally into the container.
Not that it is advisable to run as root, but depending on your requirements, the official Elasticsearch image (elasticsearch:7.7.0) runs as root and does not have this problem.
Another solution that would work for the bitnami image is to make the ./data directory owned by group root and group writable, since it appears the group of the Elasticsearch process is still root.
A third solution is to change the GID of the bitnami image to whatever group you had the data created with and make it group writable.
I have a pretty simple docker-compose setup which is working on my colleague computer (*), but for some obscure reason it doesn't work on mine.
Here is my docker-compose.yml
version: '3.3'
services:
... there are other services that are starting successfully ...
reporting:
image: microsoft/dotnet:2.0-runtime
hostname: reporting
container_name: reporting
volumes:
- publish-output:/app
command: dotnet /app/MocksGenerator.dll -s ${MSNAME_R} -p ${MSPORT_R} -c http://${CHOST} -m http://${MBHOST}${MSNAME_R}:${MBPORT}
networks:
- consul
links:
- mbreporting
- consul
- fabio
depends_on:
- mbreporting
- consul
- fabio
networks:
consul:
volumes:
publish-output:
driver: local
driver_opts:
device: /mnt/d/Repositories/microservices.mocking/docker/PublishOutput
o: bind
What I recieve as error from docker-compose when I try to start it using "docker-compose up".
ERROR: for reporting Cannot start service reporting: error while mounting volume '/var/lib/docker/volumes/betsreporting_publish-output/_data': error while mounting volume with options: type='' device='/mnt/d/Repositories/microservices.mocking/docker/PublishOutput' o='bind': no such file or directory
Running ls -la /mnt/d/Repositories/microservices.mocking/docker yields
drwxrwxrwx 0 root root 4096 May 30 16:12 PublishOutput
So host directory exists for sure, but docker-compose can't seem to find it for some reason. Why?
(*) My colleague is using volume of type bind, I tried with that, also didn't work for the same reason so I've decided to change the volume type, but then it seems like the root problem is that docker-compose can't seem to find the host directory.
After reset of Docker daemon credentials sharing for device window prompted and then after re-sharing the disk it started working again, even tho it was previously shared as well. I suspect that sharing of disk to Docker does not apply to directories created AFTER sharing was done (thus re-sharing was needed) but I am not entirely sure, will check that with docker engine guys.
One more thing, I was trying also to run it from Linux subsystem on Windows and it didn't work, I suspect that again permissions of Linux subsystem and Windows might be not matching or docker engine might have a bug, cause even after re-sharing error persisted so I had to run it from Powershell instead.