Currently I'm trying to mount two folders (./app + ./test/public) and one file (./test/test.py) into a shared folder in the container (type: bind), so I always have the current code in the container without restarting. The problem is that the content in /test is also mounted to /app in the host system. Can this be avoided?
Here is my example file:
volumes:
- "./app:/app"
- "./test/public:/app/test/public"
- "./test/test.py:/app/test.py"
I've searched the ninternet for about an hour now and read the docker-compose documentation, but i coudn't figure out how to solve this problem..
Hope you can help :)
edit: after docker-compose up the ./app on the host machine contains
./app/test/public and ./app/test.py too; So i simply want to mount and merge these folders without changing the host files.
I don't think you can avoid this behavior: Docker needs to create filesystem entries to which it can attach the bind mounts for your test/... mounts. If you're bind-mounting a file, a file must exist at the target location first; similarly for a directory.
That means that before performing your bind mounts, Docker first creates a new (empty) file or directory to provide the target for the mount. This is what you see inside your app directory.
Your options are either (a) just live with it, or (b) restructure your project so that you don't need to bind mount things into an existing bind mount.
Related
I am trying to setup a gitea container and while checking the official docs, for the volumes sections, the following is defined:
volumes:
- ./gitea:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
I know that the volumes section is used to configure DB in docker-compose but I could not find why this specific configuration is done here. Can someone explain to me what do we achieve with the lines added here in the volumes section?
To be more specific, what do we achieve with ./gitea:/data, /etc/timezone:/etc/timezone:ro and /etc/localtime:/etc/localtime:ro and why is this needed?
Thanks.
The volume section is a way to share files and directories between the host system and the container. With :ro the shared files can be made read-only to the container.
One must understand, that a container is just a snapshot of a current build from e.g. docker hub. Whenever you delete this container, all data are deleted too.
So volumes are also used to create a place for data which shall be persistent and not affected by the removal of the container.
So what happens here:
With /etc/timezone:/etc/timezone:ro the file /etc/timezone on the host system (where the docker daemon is running on) is made available under /etc/timezone (:ro means readonly) inside the container. And the same for /etc/localtime.
Those files define the timezone used on the host. By sharing it with the container it can be used inside to recognize the systems timezone.
Now about the line ./gitea:/data.
The same way you can share files you also can share directories. In your case it's expected, that in whatever directory you are currently in, there is a folder gitea (./ means >here<). And if your execute the docker command the folder ./getea on the host is mapped to /data inside the container.
So when you start the container, the apps inside the container will write the data to /data - and you would be also able to access those data on the host under ./gitea.
I need a way to configure docker-compose to create a volume if it's missing, or in case it exists, use it.
I need it to be persistent between versions, but I cannot assure it'll be configured upon initial configuration.
volumes:
my_volume:
external: true
I need to mount docker volume and not host directory.
something like:
-v my_volume:/my_files
what's the best solution for such use-case?
You can use volume for each application or services you set in docker-compose file. For instance, I set a volume for my nginx server as like.
volumes:
- ./web/public:/srv/www/static
- ./default.conf:/etc/nginx/conf.d/default.conf
The left side before colon are the path of the files or folder I want to store inside my docker image as volume whereas on right side I wrote the path where the files will be stored.
When you build the file for first time it will create volume if in case if doesn't exist or use existing volume if it exist
hope this helps.
I am trying to change some configuration files required for application setup but do not want to change the original config file in the source code.
The path to the original config file is /usr/src/app/env_configs/local_db_setup.rb
The way I try to achieve this is in my Dockerfile
cp <path of new config on host>/local_db_setup.rb /usr/src/app/env_configs/
However, I perceive that due to my volume mounted in docker-compose.yml, the Copy is not taking place or overridden.
volumes:
-.:/usr/src/app
How can I go about this?
If you bind-mount into a non-empty directory on the container, the directory’s existing contents are obscured by the bind mount. So any existing contents at /usr/src/app inside the container are shadowed.
If you want to copy the new config file during image build time as mentioned in your question, you can copy it to a different directory in the image(/tmp/config/) and move it to correct location (/usr/src/app/env_configs/) using an entrypoint script which does the move first and then starts the actual entrypoint.
Instead you can also directly mount the config file from host machine if that is okay.
In the Docker Compose documentation, here, you have the following example related to the volumes section of docker-compose.yml files:
volumes:
# (1) Just specify a path and let the Engine create a volume
- /var/lib/mysql
# (2) Specify an absolute path mapping
- /opt/data:/var/lib/mysql
# (3) Path on the host, relative to the Compose file
- ./cache:/tmp/cache
# (4) User-relative path
- ~/configs:/etc/configs/:ro
# (5) Named volume
- datavolume:/var/lib/mysql
Which syntaxes produce a bind mount and which produce a docker volume?
At some place of the documentation, the two concepts are strictly differentiated but at this place they are mixed together... so it is not clear to me.
Whenever you see "volume" in the comment, that will create a volume: so (1) and (5).
If there is not a volume in the comment, this is about a bind mount.
The documentation regarding volumes in docker-compose is here:
Mount host paths or named volumes, specified as sub-options to a service.
You can mount a host path as part of a definition for a single service, and there is no need to define it in the top level volumes key.
But, if you want to reuse a volume across multiple services, then define a named volume in the top-level volumes key.
The top-level volumes key defines a named volume and references it from each service’s volumes list. This replaces volumes_from in earlier versions of the Compose file format. See Use volumes and Volume Plugins for general information on volumes.
Those are two completely different concepts. A volume means that given directory will be persisted between container runs. Imagine MySQL database. You don’t want to lose your data. On the other hand there’s a bind mount where you attach your local directory to the directory in the container. If the container writes something there it will appear in your file system and vice versa (synchronization).
As a side note a volume is nothing more than a symlink to the directory on your machine :) (to a /var/lib/docker/volumes/... directory by default)
I have a docker-compose.yml file with the following:
volumes:
- .:/usr/app/
- /usr/app/node_modules
First option maps current host directory to /usr/app, but what does the second option do?
[Refreshing this answer since it seems others have similar questions]
There are three kinds of volumes in docker:
Host volumes: these map a path from the host into the container with a bind mount. They have the short syntax /path/on/host:/path/in/container. Whatever exists on the host is what will be visible in the container, there's no merging of files or initialization from the image, and uid/gid's do not get any special mapping so you need to take care to allow the container uid/gid read and write access to this location (an exception is Docker for Mac with OSXFS). If the path on the host does not exist, docker will create an empty directory as root, and if it is a file, you can mount a single file into the container this way.
Named volumes: these have a name, instead of a host path as the source. They have the short syntax name:/path/in/container and in a compose file, you also need to define the named volume used in containers at the top level. By default, these are also a bind mount, but to a docker specific directory under /var/lib/docker/volumes that should be considered internal. However these defaults can be changed to allow things like NFS mounts, mounting disks, or even your own bind mounts to other locations. Named volumes also have a feature in docker, when they are new or empty and first used, docker copies the contents from the image into named volume before mounting it. This includes files, directories, uid/gid owners, and permissions. After that, they behave identical to a host volume, whatever is inside the volume overlays the image location.
Anonymous volumes: these only have a path inside the container. They are in the form /path/in/container and docker will create a default named volume with a guid as the name. They share the behaviors of named volumes, storing files under /var/lib/docker/volumes, initializing with the contents of the image, except they have a randomly generated guid that gives you no indication of how or even if they are being used. You can mount the volume in another container and inspect the contents, or you can find the container using the volume by inspecting each container to find the guid. If you create a container with the --rm flag, anonymous volumes will also be deleted automatically.
tmpfs: Wait, I said 3, and this is 4? That's because tmpfs isn't considered a volume, the syntax to mount it is different. The result is a pointer to an empty in memory filesystem. This is useful if you have temporary files you don't wish to save, they are relatively small, and you either need speed or want to be sure they aren't saved to disk.
In the OP's case:
/usr/app is mounted from the host, commonly used for development
/usr/app/node_modules is an anonymous volume initialized from the image
Why do this? Likely because you do not want to modify the node_modules directory on the host, particularly if there's platform specific data and you're running on Docker desktop where it's Mac/Win on the host and Linux in the container. It's also possible there's data in the image you want to get access to within the directory structure of the other volume mount.
Are there downsides to anonymous volumes? Two that I can think of:
If there's anything in /usr/app/node_modules that you want to reuse in a future container, you're unlikely to find the old volume. I tend to consider any data written to these as likely lost.
You'll often find the volumes on the host full of guids over time, and it's unclear which are in use and which can be deleted. Unused anonymous volumes are one of several causes of excessive disk use in docker.
For more details on docker volumes, see: https://docs.docker.com/storage/
Original answer:
The second one creates an anonymous volume. It will be listed in docker volume ls with a long unique id rather than a name. Docker-compose will be able to reuse this if you update your image, but it's easy to lose track of which volume belongs to what with those names, so I recommend always giving your volume a name.
Just to complement the accepted answer, according to Docker's Knowledge Base there are three types of volumes: host, anonymous, and named:
A host volume lives on the Docker host's filesystem and can be
accessed from within the container. Example volume path:
/path/on/host:/path/in/container
An anonymous volume is useful for when you would rather have
Docker handle where the files are stored. It can be difficult,
however, to refer to the same volume over time when it is an
anonymous volumes. Example volume path:
/path/in/container
A named volume is similar to an anonymous volume. Docker manages
where on disk the volume is created, but you give it a volume name. Example volume path:
name:/path/in/container
The path used in your example is an anonymous volume.
I had the same question while I was going through this tutorial, and the answer to what those lines could actually be doing is this:
Without the anonymous volume ('/usr/src/app/node_modules'), the node_modules directory would essentially disappear by the mounting of the host directory at runtime:
Build - The node_modules directory is created.
Run - The current directory is copied into the container, overwriting the node_modules that were just installed when the container was built.
The docker-compose.yml file for this:
version: '3.5'
services:
something-clever:
container_name: something-clever
build:
context: .
dockerfile: Dockerfile
volumes:
- '.:/usr/src/app'
- '/usr/src/app/node_modules'
ports:
- '4200:4200'