I am quite new to docker and I try to build a LAMP stack with docker-compose. I have found a nice tutorial over there. I think I understood the difference between volumes and bind mounts, however, I guess I am running into a problem at some point. I want to make one of my folders available to the LAMP stack (my sources, residing in a folder 'src'). However, the sources are not visible within the /var/www/html folder.
My docker-compose file looks like this:
version: "3.7"
services:
mariadb:
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "no"
MYSQL_DATABASE: testdb
MYSQL_PASSWORD: testpassword
MYSQL_ROOT_PASSWORD: rootpwd
MYSQL_USER: testuser
TZ: Europe/Rome
image: "mariadb:10.5.2"
volumes:
- "mariadb-volume:/var/lib/mysql"
php-httpd:
image: "php:7.3-apache"
ports:
- "80:80"
volumes:
- ./src/:/var/www/html/
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- "mariadb:db"
ports:
- "8081:80"
volumes:
mariadb-volume: ~
Phpmyadmin works just fine, also the docker-compose runs without any warnings. My compose command is
docker-compose up -d --force-recreate
Interestingly, when I change "./src/" for "./DocumentRoot", the folder DocumentRoot is created on my host machine. However, placing files in DocumentRoot on the host or in /var/www/html in docker does not show the files on the docker or host, respectively. Nevertheless, I can say for sure that I am in the right directory at least.
Is there some trick or parameter I need to pass along to let docker see the files on my host?
Hah... thanks again. Your question has triggered another thought. It's quite natural to me, so I didn't mention it: When I execute the docker-compose from Desktop, everything works fine. However, if I execute it from my usual working directory, it does not. My usual working directory is a mounted volume with VeryCrypt on Windows. Obviously there are issues sharing the directory in the latter case.
Just in case anybody is experiencing that error too in the future.
I want to make one of my folders available to the LAMP stack (my
sources, residing in a folder 'src'). However, the sources are not
visible within the /var/www/html folder.
I think that there is a confusion about how mounts work with docker.
When you specify a mount for a docker container such as :
php-httpd:
image: "php:7.3-apache"
ports:
- "80:80"
volumes:
- ./src/:/var/www/html/
Only the container php-httpd will be set with the mount, not the other containers of your LAMP stack.
If you need to set that mount on other containers, do it explicitly on them.
Interestingly, when I change "./src/" for "./DocumentRoot", the folder
DocumentRoot is created on my host machine. However, placing files in
DocumentRoot on the host or in /var/www/html in docker does not show
the files on the docker or host, respectively.
That is the way which works the mounts. When the folder exists on the host (here src) , docker uses it to mount its content from host to container. When the folder doesn't exit on the host, Docker creates it.
I have finally found a solution. I am splitting the docker-compose file and I do the php-httpd part in a separate dockerfile. There, I can copy my sources into the dockercontainer.
It is not the original solution, so I would still be grateful for input on the problem why the bind mount does not work, but this solution works for me.
Related
I am writing a docker-compose.yaml file for my project. I have checked the volumes documentation here .
I also understand the concept of volume in docker that I can mount a volume e.g. -v my-data/:/var/lib/db where my-data/ is a directory on my host machine while /var/lib/db is the path inside database container.
My confuse is with the link I put above. There it has the following sample:
version: "3.9"
services:
db:
image: db
volumes:
- data-volume:/var/lib/db
backup:
image: backup-service
volumes:
- data-volume:/var/lib/backup/data
volumes:
data-volume:
I wonder does it mean that I have to create a directory named data-volume on my host machine? What if I have a directory on my machine with path temp/my-data/ and I want to mount that path to the database container /var/lib/db ? Should I do something like below?
version: "3.9"
services:
db:
image: db
volumes:
- temp/my-data/:/var/lib/db
volumes:
temp/my-data/:
My main confusion is the volumes: section at the bottom, I am not sure whether the volume name should be the path of my directory or should be just literally a name I give & if it is the latter case then how could the given name be mapped with temp/my-data/ on my machine? The sample doesn't indicate that & is ambiguous to clarify that.
Could someone please clarify it for me?
P.S. I tried with above docker-compose I guessed, ended up with the error:
ERROR: The Compose file './docker-compose.yaml' is invalid because:
volumes value 'temp/my-data/' does not match any of the regexes: '^[a-zA-Z0-9._-]+$'
Mapped volumes can either be files/directories on the host machine (sometimes called bind mounts in the documentation) or they can be docker volumes that can be managed using docker volume commands.
The volumes: section in a docker-compose file specify docker volumes, i.e. not files/directories. The first docker-compose in your post uses such a volume.
If you want to map a file or directory (like in your last docker-compose file), you don't need to specify anything in the volumes: section.
Docker volumes (the ones specified in the volumes: section or created using docker volume create) are of course also stored somewhere on your host computer, but docker manages that and you shouldn't normally need to know where or what the format is.
This part of the documentation is pretty good about explaining it, I think https://docs.docker.com/storage/volumes/
As #HansKilian mentions, you don't need both volumes and services.volumes. To use services.volumes, map the host directory to the container directory like this:
services:
db:
image: db
volumes:
- /host/path/lib/db:/container/path/lib/db
With that, the directory /host/path/lib/db on the host machine will be used by the container and available at /container/path/lib/db.
Now, if you're like me, I get really confused with fake examples, so let's say the real directory on your host machine is /var/lib/db and you just want to see it at /db when you run a shell in Docker (i.e., docker exec -it /bin/bash container-id).
docker-compose.yaml would look like this:
services:
db:
image: db
volumes:
- /var/lib/db:/db
Now when you run the shell, cd /logs and ls, you'll see the same results as if you'd cd /var/lib/db on the host.
If you want to use the volumes section to indicate a global volume to use, you first have to create that volume using docker volume create. The documentation Hans linked includes steps to do this. The syntax of /host/path:/container/path is replaced by volume-name:/container/path. Then, once defined, you'd alter your docker-compose.yaml to be more like this:
services:
db:
image: db
volumes:
- your-global-volume-name:/db
volumes:
your-global-volume-name:
external: true
Note that I have not tested or used the this configuration. I'm assuming it's correct based on the other method working and the few changes I can identify in the docs.
I have a problem that I just can't understand. I am using docker to run certain containers, but I have problems with at least one Volume, where I't like to ask if anybody can give me a hint what I am doing wrong. I am using Nifi-Ingestion as example, but it affects even more container volumes.
First, let's talk about the versions I use:
Docker version 19.03.8, build afacb8b7f0
docker-compose version 1.27.4, build 40524192
Ubuntu 20.04.1 LTS
Now, let's show the volume in my working docker-compose-file:
In my container, it is configured as followed:
volumes:
- nifi-ingestion-conf:/opt/nifi/nifi-current/conf
Below my docker-compose file it is defined as a normal named volume:
volumes:
nifi-ingestion-conf:
This is a snippet from the docker-compose that I'd like to get working
In my container, it is configured in this case as followed (having my STORAGE_VOLUME_PATH defined as /mnt/storage/docker_data):
volumes:
- ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf
On the bottom I guess there is something to do but I don't know what I could need to do here. In this case it is the same as in the working docker-compose:
volumes:
nifi-ingestion-conf:
So, now whats my problem?
I have two docker-compose files. One uses the normal named volumes, and one uses the volumes in my extra mount path. When I run the containers, the volumes seem to work different since files are written in the first style, but not in the second. My mount paths are generated in the second version so there is nothing wrong with my environment variables in the .env-file.
Hint: the /mnt/storage/docker_data is an NFS-mount but my machine has the full privileges on that share.
Here is my fstab-entry to mount that volume (maybe I have to set other options):
10.1.0.2:/docker/data /mnt/storage/docker_data nfs auto,rw
Bigger snippets
Here is a bigger snipped if the docker-compose (i need to cut and remove confident data, my problem is not that it does not work, it is only that the volume acts different. Everything for this one volume is in the code.):
version: "3"
services:
nifi-ingestion:
image: my image on my personal repo
container_name: nifi-ingestion
ports:
- 0000
labels:
- app-specivic
volumes:
- ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf
#working: - nifi-ingestion-conf:/opt/nifi/nifi-current/conf
environment:
- app-specivic
networks:
- cnetwork
volumes:
nifi-ingestion-conf:
networks:
cnetwork:
external: false
ipam:
driver: default
config:
- subnet: 192.168.1.0/24
And here of the env (only the value we are using)
STORAGE_VOLUME_PATH=/mnt/storage/docker_data
if i understand your question correctly, you wonder why the following docker-compose snippet works for you
version: "3"
services:
nifi-ingestion:
volumes:
- nifi-ingestion-conf:/opt/nifi/nifi-current/conf
volumes:
nifi-ingestion-conf:
and the following docker-compose snippet does not work for you
version: "3"
services:
nifi-ingestion:
volumes:
- ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf
what makes them different is how you use volumes. you need to differentiate between mount host paths and mount named volumes
You can mount a host path as part of a definition for a single service, and there is no need to define it in the top level volumes key.
But, if you want to reuse a volume across multiple services, then define a named volume in the top-level volumes key.
named volumes are managed by docker
If you start a container with a volume that does not yet exist, Docker creates the volume for you.
also, would advise you to read this answer
update:
you might also want to read about docker nfs volumes
I have a Docker Compose setup to run a Nginx serving pictures (jpg files). The port, protocol, etc. is not relevant. My Nginx works correctly and serve those files:
web:
image: myweb
restart: always
volumes:
- ./logs/nginx:/var/log/nginx
ports:
...
The image myweb inherits from nginx and add all the configuration needed by this server, basically, the web root is:
index index.html index.php;
root /var/www/myweb/public;
I'm also using php:7.0-fpm and fastcgi locations but it is not the issue here.
What I would want is to mount a host volume with the pictures (instead of copying them into the docker image) in order to have the possibility of update it externally (Dropbox sync or whatever).
web:
image: myweb
restart: always
volumes:
- ./webcode:/var/www
- ./logs/nginx:/var/log/nginx
- ./images-data/catalog-images:/var/www/myweb/public/catalog/images
ports:
...
The host path webcode (relative to docker-compose.yml location) is the static web, also mounted on /var/www.
The host path images-data/catalog-images (relative to docker-compose.yml location) is the pictures directory, mounted on /var/www/myweb/public/catalog/images.
I'm not sure it this is a good practice (mount a host directory on the same path where other host directory was already mounted). Anyway, I also tried keeping webcode as part of the myweb image having only one mounted volume (the pictures one).
Mounting webcode and including pictures on it, works. And without any volume mount, just including all the stuff (website and pictures), also works. But I would need to have an isolated volume for the pictures.
I also tried symlinking within the container /var/www/myweb/public/catalog/images towards the pictures volume. But Nginx does not render it, perhaps because it it a different volume than /var/www.
Should it belong to the root of nginx server configuration ?
The thing is that, those images are not loaded in the browser. Going into the container, I confirm that the ownership for the volume is correct (www-data:www-data).
From the point of view of the container, ownership and content is the same if I mount the volume or if I copy in the container filesystem. But Nginx only can see the files within the Docker container file system.
Is this the normal behaviour ? Am I missing any Nginx configuration to work normally with host volumes inside the container ?
I think I finally solved.
When I said
I'm also using php:7.0-fpm and fastcgi locations but it is not the
issue here.
I was totally wrong: it was the key: the php execution. The problem was that the pictures was loaded by an index.php. Then it was sent to the fpm service where the pictures were not available !!.
This is my docker-compose.yml definition for the php service:
fpm:
image: php7fpm:latest
restart: always
volumes:
- ./webcode:/var/www
The thing is that the pictures volume was not visible there. I just added such volume to fpm volume list, and problem is solved:
fpm:
image: php7fpm:latest
restart: always
volumes:
- ./webcode:/var/www
- ./images-data/catalog-images:/var/www/myweb/public/catalog/images
Sorry for the inconvenience.
I'm using Docker Toolbox on Windows 10
I can access the php part succesfully via http://192.168.99.100:8000, I have been working around with the mariadb part but still having several problems
I have an sql file as /mariadb/initdb/abc.sql so I should be copied into /docker-entrypoint-initdb.d, after the container is created I use docker-compose exec mariadb to access the container, there is the file as /docker-entrypoint-initdb.d/abc.sql but the file never get executed, I also have tested to import the sql file to the container manually, it was succesful so the sql file is valid
I don't quite understand about the data folder mapping, and what to do to get the folder sync with the container, I always get the warning when recreate the container using docker-compose up -d
WARNING: Service "mariadb" is using volume "/var/lib/mysql" from the previous container. Host mapping "/.../mariadb/data" has no effect. Remove the existing containers (with docker-compose rm mariadb) to use the Recreating db ... done
Questions
How to get the sql file in /docker-entrypoint-initdb.d to be executed ?
What is the right way to map the data folder with the mariadb container ?
Please guide
Thanks
This is my docker-compose.yml
version: "3.2"
services:
php:
image: php:7.1-apache
container_name: web
restart: always
volumes:
- /.../php:/var/www/html
ports:
- "8000:80"
mariadb:
image: mariadb:latest
container_name: db
restart: always
environment:
- MYSQL_ROOT_PASSWORD=12345
volumes:
- /.../mariadb/initdb:/docker-entrypoint-initdb.d
- /.../mariadb/data:/var/lib/mysql
ports:
- "3306:3306"
For me the issue was the fact that Docker didn't clean up my mounted volumes from previous runs.
Doing a:
docker volume ls
Will list any volumes, and if previous exist, then run 'rm' command on the volume to remove it.
As stated on docker mysql docks, scripts in the '/docker-entrypoint-initdb.d' folder is only evalutated the first time the container runs, and if a previous volume remains, it won't run the scripts.
As for the mapping, you simply need to mount your script folder to the '/docker-entrypoint-initdb.d' folder in the image:
volumes:
- ./db/:/docker-entrypoint-initdb.d
I have a single script file in a folder named db, relative to my docker-compose file.
In your Docker file for creating mariaDB, at the end add the abc.sql file to your docker entry point like so:
COPY abc.sql /docker-entrypoint-initdb.d/
Remove the - /.../mariadb/initdb:/docker-entrypoint-initdb.d mapping as any file copied into the entry point will be executed.
Note: Windows containers do not execute anything in docker-entrypoint-initdb.d/
I am starting six or seven containers via a docker-compose file. One container is causing a major problem! Here is the relevant section:
services:
...
main-app:
image: mycompany/sys:1.2.3
container_name: "main-app-container"
ports:
- "8080:8080"
- "8009"
volumes:
- db_data:/var/lib/home/data:rw
- /opt/mycompany/sys/config:/opt/mycompany/sys/config:rw
networks:
- systeminternal
hostname: "mylocalhost.company.com"
volumes:
db_data:
driver: local
networks:
systeminternal:
When the main-app-container is started via docker-compose up (as the root user) the file system privileges in many of the directories in the committed container are all changed to root! This is running on Ubuntu 14.04, Docker 1.12.x (not sure which x).
We have another system where we run everything as a local user. When we exec a shell into that container, all the file privileges are of our local user that was ownership as it was committed. From googling, I am pretty sure it has something to do with the volumes, but could not find anything definitive. Any help is welcome!
This is the expected behavior for host-mounts, that said, everything inside /opt/mycompany/sys/config will be having the same UID/GID the files have on the host - that is by design.
Either change the files to the uid/gid you need on the host: chown -R 123:321 /opt/mycompany/sys/config or setup your container to be happy to use the uid/gid of the host.
It has nothing to do with docker-compose, it would happen the same way when you use
docker run -v /opt/mycompany/sys/config:/opt/mycompany/sys/config mycompany/sys:1.2.3