A few lines from docker-compose.yml
volumes:
- ./yii-application:/app/yii-application
- /app/yii-application/common/config/
First line adds to the volume an entire application.
The second one makes some sort of exclude of config folder, so I do not need my config from host machine.
A few lines from Dockerfile
COPY ./config-${APP_ENV}/common /app/yii-application/common/config
Instead of COPY I tried
RUN cp -a /app/config-${APP_ENV}/common/. /app/yii-application/common/config
It does not work either.
I think there is an issue in the order of the commands that are being executed:
When you are building and image with Dockerfile, you are coping the code inside dir /app/yii-application/common/config.
Then, you are mounting a volume: volumes: /app/yii-application/common/config/
and overwriting the existing dir with an empty dir that serves as a volume.
You need to work around that issue.
I moved the config files out of volume directory.
So, now the volume looks like this
volumes:
- ./yii-application:/app/yii-application
Config files in
./
Related
Supposed I have a Docker container and a folder on my host /hostFolder. Now if I want to add this folder to the Docker container as a volume, then I can do this either by using ADD in the Dockerfile or mounting it as a volume.
So far, so good.
Now /hostFolder contains a sub-folder, /hostFolder/subFolder.
I want to mount /hostFolder into the Docker container (whether as read-write or read-only does not matter, works both for me), but I do NOT want to have it included /hostFolder/subFolder. I want to exclude this, and I also want the Docker container be able to make changes to this sub-folder, without the consequence of having it changed on the host as well.
Is this possible? If so, how?
Using docker-compose I'm able to use node_modules locally, but ignore it in the docker container using the following syntax in the docker-compose.yml
volumes:
- './angularApp:/opt/app'
- /opt/app/node_modules/
So everything in ./angularApp is mapped to /opt/app and then I create another mount volume /opt/app/node_modules/ which is now empty directory - even if in my local machine ./angularApp/node_modules is not empty.
If you want to have subdirectories ignored by docker-compose but persistent, you can do the following in docker-compose.yml:
volumes:
node_modules:
services:
server:
volumes:
- .:/app
- node_modules:/app/node_modules
This will mount your current directory as a shared volume, but mount a persistent docker volume in place of your local node_modules directory. This is similar to the answer by #kernix, but this will allow node_modules to persist between docker-compose up runs, which is likely the desired behavior.
For those trying to get a nice workflow going where node_modules isn't overridden by local this might help.
Change your docker-compose to mount an anonymous persistent volume to node_modules to prevent your local overriding it. This has been outlined in this thread a few times.
services:
server:
build: .
volumes:
- .:/app
- /app/node_modules
This is the important bit we were missing. When spinning up your stack use docker-compose -V. Without this if you added a new package and rebuilt your image it would be using the node_modules from your initial docker-compose launch.
-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving
data from the previous containers.
To exclude a file, use the following
volumes:
- /hostFolder:/folder
- /dev/null:/folder/fileToBeExcluded
With the docker command line:
docker run \
--mount type=bind,src=/hostFolder,dst=/containerFolder \
--mount type=volume,dst=/containerFolder/subFolder \
...other-args...
The -v option may also be used (credit to Bogdan Mart), but --mount is clearer and recommended.
First, using the ADD instruction in a Dockerfile is very different from using a volume (either via the -v argument to docker run or the VOLUME instruction in a Dockerfile). The ADD and COPY commands just take a copy of the files at the time docker build is run. These files are not updated until a fresh image is created with the docker build command. By contrast, using a volume is essentially saying "this directory should not be stored in the container image; instead use a directory on the host"; whenever a file inside a volume is changed, both the host and container will see it immediately.
I don't believe you can achieve what you want using volumes, you'll have to rethink your directory structure if you want to do this.
However, it's quite simple to achieve using COPY (which should be preferred to ADD). You can either use a .dockerignore file to exclude the subdirectory, or you could COPY all the files then do a RUN rm bla to remove the subdirectory.
Remember that any files you add to image with COPY or ADD must be inside the build context i.e. in or below the directory you run docker build from.
for the people who also had the issue that the node_modules folder would still overwrite from your local system and the other way around
volumes:
node_modules:
services:
server:
volumes:
- .:/app
- node_modules:/app/node_modules/
This is the solution, With the trailing / after the node_modules being the fix.
Looks like the old solution doesn't work anymore(at least for me).
Creating an empty folder and mapping target folder to it helped though.
volumes:
- ./angularApp:/opt/app
- .empty:/opt/app/node_modules/
I found this link which saved me: Working with docker bind mounts and node_modules.
This working solution will create a "exclude" named volume in docker volumes manager. The volume name "exclude" is arbitrary, so you can use a custom name for the volume intead exclude.
services:
node:
command: nodemon index.js
volumes:
- ./:/usr/local/app/
# the volume above prevents our host system's node_modules to be mounted
- exclude:/usr/local/app/node_modules/
volumes:
exclude:
You can see more infos about volumes in Official docs - Use a volume with docker compose
To exclude a mounted file contained in the volume of your machine, you will have to overwrite it by allocating a volume to this same file.
In your config file:
services:
server:
build : ./Dockerfile
volumes:
- .:/app
An example in you dockerfile:
# Image Location
FROM node:13.12.0-buster
VOLUME /app/you_overwrite_file
Pretty much the title says it all.
I know I can copy the file (from the host) into a docker container.
I also know I can copy the directory into a docker container.
But how to copy the contents of a directory (preserving all subdirectories) into a directory in a docker container?
On my host I have a directory called src. On the docker container I have a directory /var/www/html. That src has both files and directories. I need all of them to be copied (with the command) into the container; not bound, not mounted, but copied.
It sounds like a trivial operation, but I've tried so many ways and couldn't find anything online that works! Ideally, it would be best if that copy operation would work every time I run the docker-compose up -d command.
Thanks in advance!
I found the solution. There is a way of specifying the context directory explicitly; in that case the dockerfile also need to be specified explicitly too.
In the docker-compose.yml one should have the following structure:
services:
php:
build:
context: .
dockerfile: ./php/Dockerfile
In this case the src is "visible" because it is inside the context! Then in that Dockerfile the COPY command will work!
Update: There is another way to achieve this via the command as well. However for me it started to work when I've added the ./ at the end. So the full command is:
docker cp ./src/./ $(docker-compose ps|grep php|awk '{print $1}'):/var/www/html/
I have a custom image created from my own project. The Dockerfile is quite simple:
from wordpress
COPY test.html /var/www/html
Using docker composer, if I run it using a volume, it's works fine.
wordpress:
...
image: my_project_image
volumes: ['volumetest:/var/www/html']
...
volumes:
volumetest:
But if instead of creating a volume, if I map a local folder to the remote folder, the file test.html is not created, neither inside the wordpress container, neither inside the local folder:
wordpress:
...
image: my_project_image
volumes: ['./testdir:/var/www/html']
...
#volumes:
# volumetest:
Is there a way I can create the file just by using the docker-compose?
Thanks a lot. :)
At the build stage of the Docker image, you copy test.html to /var/www/html directory inside the Docker image.
When you run the image as a container, you map local ./testdir to /var/www/html directory inside the container. This means /var/www/html now points to your ./testdir directory. If your ./testdir does not contain your test.html file, then you're not going to see it inside the container as well.
I believe you misunderstood that when you use volumes, it will copy the file in the image to the local file system. What happens is, whichever files you have in your local file system directory will reflect (you can think of this as replace as well) as the files in the mapped directory inside the container.
I use docker-compose for a simple keycloak container and I've been trying to install a new theme for keycloak.
However I've been unable to copy even a single file to the container usnig a Dockerfile. The Dockerfile and docker-compose.yml are in the same directory
Neither of these commands work or cause any events or warnings in the logs.
COPY test /tmp/
COPY /home/adm/workspace/docker/keycloak-cluster/docker/kctheme/theme/login/. /opt/jboss/keycloak/themes/keycloak/login/
Copying manually with
sudo docker cp test docker_keycloak_1:/tmp
works without any issues.
Quick understanding on Docker.
docker build: Create an image from a Dockerfile
docker run: Create a container from an image.
(you can create yourself the image or use a existing image from docker hub )
Based on what you said, you have 2 options.
Create a new docker image based on the existing one and add the theme.
something like
# Dockerfile
FROM jboss/keycloak
COPY test /tmp/
COPY /home/adm/workspace/docker/keycloak-cluster/docker/kctheme/theme/login/. /opt/jboss/keycloak/themes/keycloak/login/
and then use docker build to create your new image
Volume the theme in the correct directory
using docker-compose volume
version: '3'
services:
keycloak:
image: quay.io/keycloak/keycloak:latest
volumes:
- "./docker/kctheme/theme/login:/opt/jboss/keycloak/themes/keycloak/login"
Files have to be in the same directory or a subdirectory of the directory with your Dockerfile build file if you use COPY, and have to be present at build time. No absolute paths.
/tmp as destination is also a bit tricky, because the startup process of the container might have a /tmp cleanout, which means that you would never see that file in a running container.
Is it possible to to mount a host directory in to a container but only allowing overwriting files that exist on the host?
Example github repo: https://github.com/UniBen/stackoverflow-59031249
E.g
Host:
src/
public/
index.php (200kb)
Container:
src/
public/
index.php (100kb)
vendor/
...
Desired output: (The container file system merged with the mounted host system files which exists. Note the size of the index.php file.)
src/
public/
index.php (200kb)
vendor/
...
Actual output:
src/
public/
index.php (200kb)
Example docker-compose.yml
version: '3.2'
services:
php:
image: php
volumes:
- ./src:/src
Edit: So it looks like overlayfs implemented by docker is only used for bulding docker images and can not be used for volumes what so ever. I still think it's possible to specify a custom driver but not sure how. As a temp fix I've done some fancy stuff with mapping stuff out of container, diffing it and putting it back in but not ideal.
Is it possible to to mount a host directory in to a container but ...
No. The only Docker mounting option is a straight-up "replace this directory in the container with the equivalent directory from the host". There is no way to modify this, selectively hide subdirectories, or implement your "only if it already exists" rule.
In the example you're showing, you probably don't want a volume at all. Files like index.php and a vendor directory are application source code, and in typical use you'd write a Dockerfile, COPY index.php . to move the file into the image, and then RUN composer ... to create the vendor tree. This would be isolated from your host environment, so the vendor directory in the image would be separate from whatever existed on your host system.
Actually, there are no options to control such behavior, e.g, how data between source and dest will be handled. But if David's answer is not really your case you could do something like this:
version: '3.2'
services:
example:
build:
context: .
volumes:
- data:/src
- ./src:/src/host
volumes:
data:
How docker documentation says:
If you start a container which creates a new volume, and the container
has files or directories in the directory to be mounted the
directory’s contents are copied into the volume.
So after that, let's investigate a little bit:
/src # ls
file.a.txt file.c.txt host
/src # cat host/file.a.txt
host
/src # cat file.a.txt
container
/src # cat file.c.txt
container
Data from from container is saved into data named volume. The data from the host machine live in the host folder. Now you can copy from the host folder to the src with cp or rsync with any rules that you want.
This is a quite fishy and artificial example, maybe it's a good idea to rethink current implementation.