I am new to docker, and stumbled upon a docker-compose file. I get the gist of all other properties but I have no idea what below line is doing:
volumes:
- ./data:/data/db
Can anyone please help me with this.
multiple volumes can be attached to your container ... each are defined as a pair
volumes:
- /parent/host/path01:/inside/container/path_one
- /parent/host/path02:/inside/container/path_another
of each pair the left side is a pre-existing volume reachable on host before container is created ... right side is what the freshly launched container views that left side as from inside the container
in your example, in same dir where you launch docker-compose from, there evidently exists a dir called data ... using ./data will reach it using a relative path ... the right side /data/db is what the code in your container calls that same dir
/full/path/to/reach/data:/data/db
is using the absolute path to reach that same ./data dir which lives on the parent host which docker-compose is executed on
This volume mapping allows permanent storage on parent host to become visible (read/writable) to the container ... since the container filesystem is ephemeral and so goes away when container exits this volume mapping gives the container access to permanent storage for specified paths which must appear in your yaml file ... especially important for database containers like mongo ... all files used in your container not mapped in the volumes yaml disappear once the container exists
Here is a typical yaml snippet for mongo where it gains access to permanent storage on parent host
loudmongo:
image: mongo
container_name: loud_mongo
restart: always
ports:
- 127.0.0.1:27017:27017
volumes:
- /cryptdata7/var/data/db:/data/db
The dash symbol is probably what is throwing you off, because it is poorly formatted YAML syntax for a YAML list element.
The volume syntax after the dash is just following the so-called "short" syntax for a host-to-container bind-mounted volume mapping.
Related
I have the following docker-compose:
version: '3.7'
services:
db:
image: bitnami/mongodb:5.0.6
volumes:
- "/app/local-data:/data/db"
env_file: ./db/.env
The problem is data does not persist between docker-compose up/down and docker does not seem to use /app/local-data even though it creates it.
When I run docker-compose, container starts and works naturally. The directory /app/local-data is created by docker, however Mongodb does not populate it, and no r/w error is being shown on console. This makes me thing a temporary volume is assigned to container instead.. But if that is true then why docker still creates /app/local-data and not using it?
Any ideas how can I debug this?
Docker directives like volumes: don't know anything about what's actually running in the image. That directive creates the specified host and container paths if required, and bind-mounts the host path into the container path. It's up to the application code to use that directory (or not).
If you look at the bitnami/mongodb Docker Hub page under "Persisting your database", the database is configured to store data in the /bitnami/mongodb directory inside the container, and that directory needs to be the second volumes: path. Also note the requirement that the data directory needs to be writable by user ID 1001, which may or may not exist on your host (there's no specific requirement to create it).
volumes:
- "/app/local-data:/bitnami/mongodb"
# ^^^^^^^^^^^^^^^^
sudo chown -R 1001 /app/local-data
sudo docker-compose up -d
I need a way to configure docker-compose to create a volume if it's missing, or in case it exists, use it.
I need it to be persistent between versions, but I cannot assure it'll be configured upon initial configuration.
volumes:
my_volume:
external: true
I need to mount docker volume and not host directory.
something like:
-v my_volume:/my_files
what's the best solution for such use-case?
You can use volume for each application or services you set in docker-compose file. For instance, I set a volume for my nginx server as like.
volumes:
- ./web/public:/srv/www/static
- ./default.conf:/etc/nginx/conf.d/default.conf
The left side before colon are the path of the files or folder I want to store inside my docker image as volume whereas on right side I wrote the path where the files will be stored.
When you build the file for first time it will create volume if in case if doesn't exist or use existing volume if it exist
hope this helps.
I am trying to make a neo4j container use host system directories for data and logs using docker-compose. My compose file looks like this
neo4j:
image: neo4j:3.5.6
ports:
- "127.0.0.1:7474:7474"
- "127.0.0.1:7473:7473"
- "127.0.0.1:7687:7687"
environment:
NEO4J_AUTH: "none"
volumes:
- "~/neo4j/data:/data"
- "~/neo4j/logs:/logs"
However, it only works for the logs directory, for the data directory, the container keeps its own volume. The binds section of docker inspect looks like this
"Binds": [
"/home/rbusche/neo4j/logs:/logs:rw",
"6f989b981c12a252776404343044b6678e0fac48f927e80964bcef409ab53eef:/data:rw"
],
Peculiar enough it works when I use docker run and specify the volume there. The neo4j Dockerfile declares both data and logs as container volumes. This there any way to force docker-compose to override those?
After removing the volume 6f989b981c12a252776404343044b6678e0fac48f927e80964bcef409ab53eef and the container associated with it, it works as expected. It seems like the container was clinging to a volume it created on a previous start.
What's the difference between declaring in the docker-compose.yml file a volume section and just using the volumes keyword under a service?
For example, I map a volume this way for a container:
services:
mysqldb:
volumes:
- ./data:/var/lib/mysql
This will map to the folder called data from my working directory.
But I could also map a volume by declaring a volume section and use its alias for the container:
services:
mysqldb:
volumes:
- data_volume:/var/lib/mysql
volumes:
data_volume:
driver: local
In this method, the actual location of where the mapped files are stored appears to be somewhat managed by docker compose.
What are the differences between these 2 methods or are they the same? Which one should I really use?
Are there any benefits of using one method over the other?
The difference between the methods you've described is that first method is a bind mount, and the other is a volume. These are more of Docker functions (rather than Docker Compose), and there are several benefits volumes provide over mounting a path from your host's filesystem. As described in the documentation, they:
are easier to back up or migrate
can be managed with docker volumes or the API (as opposed to the raw filesystem)
work on both Linux and Windows containers
can be safely shared among multiple containers
can have content pre-populated by a container (with bind mounts sometimes you have to copy data out, then restart the container)
Another massive benefit to using volumes are the volume drivers, which you'd specify in place of local. They allow you to store volumes remotely (i.e. cloud, etc) or add other features like encryption. This is core to the concept of containers, because if the running container is stateless and uses remote volumes, then you can move the container across hosts and it can be run without being reconfigured.
Therefore, the recommendation is to use Docker volumes. Another good example is the following:
services:
webserver_a:
volumes:
- ./serving/prod:/var/www
webserver_b:
volumes:
- ./serving/prod:/var/www
cache_server:
volumes:
- ./serving/prod:/cache_root
If you move the ./serving directory somewhere else, the bind mount breaks because it's a relative path. As you noted, volumes have aliases and have their path managed by Docker, so:
you wouldn't need to find and replace the path 3 times
the volume using local stores data somewhere else on your system and would continue mounting just fine
TL;DR: try and use volumes. They're portable, and encourage practices that reduce dependencies on your host machine.
I'm struggling to understand how exactly does the named volume work in the following example from docker docs:
version: "3"
services:
db:
image: db
volumes:
#1
- data-volume:/var/lib/db
backup:
image: backup-service
volumes:
#2
- data-volume:/var/lib/backup/data
volumes:
data-volume:
My guess is, that the first occurrence of the named volume (#1) defines what is contained inside the volume, while subsequent occurrences (#2) simply share the volume's content with whatever containers they are referenced from.
Is this guess correct?
Listing data-volume: under the top-level volumes: key creates a named volume on the host if it doesn't exist yet. This behaves the following way according to this source
If you create a named volume by running a new container from image by docker run -v my-precious-data:/data imageName, the data within the image/container under /data will be copied into the named volume.
If you create another container binds to an existing named volume, no files from the new image/container will be copied/overwritten, it will use the existing data inside the named volume.
They don’t have a docker command to backup / export a named volume. However you can find out the actual location of the file by “docker volume inspect [volume-name]”.
In case the volume is empty and both containers have data in the target directory the first container to be run will mount its data into the volume and the other container will see that data (and not its own). I don't know which container will run first (although I expect it executes from top to bottom) however you can force an order with depends_on as shown here
------------------- Update
The depends_on option is ignored when deploying a stack in swarm mode with a version 3 Compose file.
The way that I understand your guess, you are not completely correct.
Declaring and referencing a named volume in a docker-compose file will create an empty volume which may then be accessed and shared by the services saying so in their volumes section.
If you want to share a named volume, you have to declare this volume in the top-level volume section of your docker-compose file. Example (as in the docker docs already linked by yourself):
version: "3"
services:
db:
image: db
volumes:
#1 uses the named and shared volume 'data-volume' created with #3
- data-volume:/var/lib/db
backup:
image: backup-service
volumes:
#2 uses the named and shared volume 'data-volume' created with #3
- data-volume:/var/lib/backup/data
volumes:
#3 creates the named volume 'data-volume'
data-volume:
The volume will be empty on start (and therefore the folders in the containers where that volume is mounted to). Its content will be a result of the services acions on runtime.
Hope that made it a bit more clear.