Supposed I have a Docker container and a folder on my host /hostFolder. Now if I want to add this folder to the Docker container as a volume, then I can do this either by using ADD in the Dockerfile or mounting it as a volume.
So far, so good.
Now /hostFolder contains a sub-folder, /hostFolder/subFolder.
I want to mount /hostFolder into the Docker container (whether as read-write or read-only does not matter, works both for me), but I do NOT want to have it included /hostFolder/subFolder. I want to exclude this, and I also want the Docker container be able to make changes to this sub-folder, without the consequence of having it changed on the host as well.
Is this possible? If so, how?
Using docker-compose I'm able to use node_modules locally, but ignore it in the docker container using the following syntax in the docker-compose.yml
volumes:
- './angularApp:/opt/app'
- /opt/app/node_modules/
So everything in ./angularApp is mapped to /opt/app and then I create another mount volume /opt/app/node_modules/ which is now empty directory - even if in my local machine ./angularApp/node_modules is not empty.
If you want to have subdirectories ignored by docker-compose but persistent, you can do the following in docker-compose.yml:
volumes:
node_modules:
services:
server:
volumes:
- .:/app
- node_modules:/app/node_modules
This will mount your current directory as a shared volume, but mount a persistent docker volume in place of your local node_modules directory. This is similar to the answer by #kernix, but this will allow node_modules to persist between docker-compose up runs, which is likely the desired behavior.
For those trying to get a nice workflow going where node_modules isn't overridden by local this might help.
Change your docker-compose to mount an anonymous persistent volume to node_modules to prevent your local overriding it. This has been outlined in this thread a few times.
services:
server:
build: .
volumes:
- .:/app
- /app/node_modules
This is the important bit we were missing. When spinning up your stack use docker-compose -V. Without this if you added a new package and rebuilt your image it would be using the node_modules from your initial docker-compose launch.
-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving
data from the previous containers.
To exclude a file, use the following
volumes:
- /hostFolder:/folder
- /dev/null:/folder/fileToBeExcluded
With the docker command line:
docker run \
--mount type=bind,src=/hostFolder,dst=/containerFolder \
--mount type=volume,dst=/containerFolder/subFolder \
...other-args...
The -v option may also be used (credit to Bogdan Mart), but --mount is clearer and recommended.
First, using the ADD instruction in a Dockerfile is very different from using a volume (either via the -v argument to docker run or the VOLUME instruction in a Dockerfile). The ADD and COPY commands just take a copy of the files at the time docker build is run. These files are not updated until a fresh image is created with the docker build command. By contrast, using a volume is essentially saying "this directory should not be stored in the container image; instead use a directory on the host"; whenever a file inside a volume is changed, both the host and container will see it immediately.
I don't believe you can achieve what you want using volumes, you'll have to rethink your directory structure if you want to do this.
However, it's quite simple to achieve using COPY (which should be preferred to ADD). You can either use a .dockerignore file to exclude the subdirectory, or you could COPY all the files then do a RUN rm bla to remove the subdirectory.
Remember that any files you add to image with COPY or ADD must be inside the build context i.e. in or below the directory you run docker build from.
for the people who also had the issue that the node_modules folder would still overwrite from your local system and the other way around
volumes:
node_modules:
services:
server:
volumes:
- .:/app
- node_modules:/app/node_modules/
This is the solution, With the trailing / after the node_modules being the fix.
Looks like the old solution doesn't work anymore(at least for me).
Creating an empty folder and mapping target folder to it helped though.
volumes:
- ./angularApp:/opt/app
- .empty:/opt/app/node_modules/
I found this link which saved me: Working with docker bind mounts and node_modules.
This working solution will create a "exclude" named volume in docker volumes manager. The volume name "exclude" is arbitrary, so you can use a custom name for the volume intead exclude.
services:
node:
command: nodemon index.js
volumes:
- ./:/usr/local/app/
# the volume above prevents our host system's node_modules to be mounted
- exclude:/usr/local/app/node_modules/
volumes:
exclude:
You can see more infos about volumes in Official docs - Use a volume with docker compose
To exclude a mounted file contained in the volume of your machine, you will have to overwrite it by allocating a volume to this same file.
In your config file:
services:
server:
build : ./Dockerfile
volumes:
- .:/app
An example in you dockerfile:
# Image Location
FROM node:13.12.0-buster
VOLUME /app/you_overwrite_file
Let's say, for example, I have a python script that I want to run as task A. It is located in C:\Users\Name\Desktop\Folder1\Script.py
I also want to move some csv files to C:\Users\Name\Desktop\Folder\CSVs through python script.
How do I map those 2 folders through volume? My compose.yaml currently looks like this:
volumes:
- ./dags:/opt/airflow/dags
- ./logs:/opt/airflow/logs
- ./plugins:/opt/airflow/plugins
When I try to indicate the local folder path as is, it's giving me "no such directory" error.
Figured I have to indicate an absolute path to any local directory on Windows to map it on container.
For example if I need to use this folder C:\Users\Name\Desktop\Folder\CSVs, map it with the airflow container by modifying it in docker-compose.yaml, then compose up to apply changes
volumes:
/c/Users/Name/Desktop/Folder/CSVs:/opt/airflow/CSVs
I have a custom image created from my own project. The Dockerfile is quite simple:
from wordpress
COPY test.html /var/www/html
Using docker composer, if I run it using a volume, it's works fine.
wordpress:
...
image: my_project_image
volumes: ['volumetest:/var/www/html']
...
volumes:
volumetest:
But if instead of creating a volume, if I map a local folder to the remote folder, the file test.html is not created, neither inside the wordpress container, neither inside the local folder:
wordpress:
...
image: my_project_image
volumes: ['./testdir:/var/www/html']
...
#volumes:
# volumetest:
Is there a way I can create the file just by using the docker-compose?
Thanks a lot. :)
At the build stage of the Docker image, you copy test.html to /var/www/html directory inside the Docker image.
When you run the image as a container, you map local ./testdir to /var/www/html directory inside the container. This means /var/www/html now points to your ./testdir directory. If your ./testdir does not contain your test.html file, then you're not going to see it inside the container as well.
I believe you misunderstood that when you use volumes, it will copy the file in the image to the local file system. What happens is, whichever files you have in your local file system directory will reflect (you can think of this as replace as well) as the files in the mapped directory inside the container.
I am not sure if this is a right question to ask. In the tutorial of Docker compose, https://docs.docker.com/compose/gettingstarted/#step-5-edit-the-compose-file-to-add-a-bind-mount, there is a volume key in the docker-compose.yml:
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: "redis:alpine"
And according to the tutorial, the volume key mounts the local file to the remote, and therefore we can change the code on the fly without restarting the Docker. My question is what internet protocol is used behind to transfer the updated code file.
Furthermore, I guess there would be more framework having this feature. What are the common protocols behind and why?
The tutorial doesn't say "the volume key mounts the local file to the remote". It says:
...in your project directory to add a bind mount for the web service:
[...]
The new volumes key mounts the project directory (current directory) on the host to /code inside the container, allowing you to modify the code on the fly, without having to rebuild the image.
If you click on the bind mount link, it will take you to
documentation that should answer all of your questions.
Briefly, a bind mount is way of making one directory on your system
appear in another location. For example, if I were run:
mkdir /tmp/newetc
mount -o bind /etc /tmp/newetc
Then running ls /tmp/newetc would show the same contents as /etc.
Docker uses this feature to expose host directories inside your
containers.
A bind mount only works on the same host; it cannot be used to expose
files on your local system to a remote system. It is a kernel feature and there are no internet protocols involved.
I'm using the Docker and docker-compose to run my app inside a container. Here is my docker-compose.yml:
version: '2'
services:
omsevents:
build:
context: ./omsevents
volumes:
- "../oms-events:/usr/app/oms-events"
- "../oms-events/assets:/usr/app/oms-events/assets"
The thing is, my app is a server and one of the things it does is file upload. When I upload a file it is created inside a container (I can bash inside the container and check and it's there, in /usr/app/oms-events/assets), bit it's not created on the host filesystem (the ../oms-events/assets/ folder is empty).
What can be the issue?
According to the documentation of Docker Compose:
You can mount a relative path on the host, which will expand relative to the directory of the Compose configuration file being used. Relative paths should always begin with . or ..
So you can try moving the docker-compose file into the context directory and remove the context clause from the build section, leaving it just build: . (or wherever your Dokerfile is). Then just fix the relative paths of the volumes or try with absolute ones.