I got docker compose:
version: '2'
services:
elasticsearch:
image: 'elasticsearch:7.9.1'
environment:
- discovery.type=single-node
ports:
- '9200:9200'
- '9300:9300'
volumes:
- /var/lib/docker/volumes/elastic_search_volume:/usr/share/elasticsearch/data:rw
When I run:
docker volume ls
I see no results. How to list unnamed volumes?
docker volume ls as you've shown it will list all of the volumes that exist.
However, in the docker-compose.yml file you show, you're not creating a named or anonymous volume. Instead, you're creating a bind mount to connect a host directory to the container filesystem space. These aren't considered "volumes" in a technical Docker sense, and a docker volume command won't show or manipulate those.
Reaching directly into /var/lib/docker usually isn't a best practice. It's better to ask Docker Compose to manage the named volume for you:
version: '2'
services:
elasticsearch:
volumes:
# No absolute host path, just the volume name
- elastic_search_volume:/usr/share/elasticsearch/data:rw
volumes:
elastic_search_volume:
# Without this line, Compose will create the volume for you.
# With this line, Compose expects it to already exist; you may
# need to manually `docker volume create elastic_search_volume`.
# external: true
Related
I have this docker-compose.yml
version: 2.4
services:
foo:
image: ubuntu:focal
container_name: oof
volumes:
- ./foo:/foo
bar:
image: ubuntu:focal
container_name: rab
volumes_from:
- container:oof
With compose 1.29.2 and docker 20.10.8. But it does not start.
I get an error
ERROR: Service "bar" mounts volumes from "oof", which is not the name of a service or container.
How do I properly reference volumes by container name in compose?
The best way is to explicitly spell out the volumes you're reusing. If the foo image has VOLUME directives, you'll also mount the automatically-created anonymous volumes; you also have no control over where the volumes will be mounted, and if the two containers aren't similar enough, there could be conflicts. (This is essentially the logic behind Compose file version 3 removing volumes_from:.)
version: 2.4
services:
foo:
volumes:
- ./foo:/foo
bar:
volumes:
- ./foo:/foo # or a different container path if that suits your needs
In general in Compose, you shouldn't need to manually specify container_name:. Compose will generate unique container names, within the docker-compose.yml file if you need to refer to things you can use the service name, and there are docker-compose wrapper commands that know how to find the right container. If you do need to use the older volumes_from: syntax use the service name instead:
version: '2.4' # does not work in version 3
services:
foo:
image: ...
volumes:
- ./foo:/foo
# no container_name:
bar:
image: ...
volumes_from: foo
What is the meaning of {} in volume definition?
For example
version: '2'
volumes:
dataelasticsearch: {}
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
volumes:
- ./dataelasticsearch:/usr/share/elasticsearch/data
It's just an empty mapping. It just means that no extra options were given to the named volume.
From the tests I have done, this is no different from leaving it blank like:
volumes:
dataelasticsearch:
The docker-compose docs do not give any more insights into this.
One more thing: you are defining a named volume at the top but then you are binding a mounted volume in the service:
volumes:
- ./dataelasticsearch:/usr/share/elasticsearch/data
Here ./dataelasticsearch basically creates a folder in your local directory which is mounted as a volume. If you want this feature, you do not need the named volume at all.
If I run this command the volume mounts and the container starts as expected with initialized state:
docker run --name gogs --net mk1net --ip 203.0.113.3 -v gogs-data:/data -d gogs/gogs
However if I run the corresponding docker-compose script the volume does not mount. The container still starts up, but without the state it reads on startup.
version: '3'
services:
gogs:
image: gogs/gogs
ports:
- "3000:3000"
volumes:
- gogs-data:/data
networks:
mk1net:
ipv4_address: 203.0.113.3
volumes:
gogs-data:
networks:
mk1net:
ipam:
config:
- subnet: 203.0.113.0/24
Any ideas?
Looking at your command, the gogs-data volume was defined outside the docker compose file, probably using something like:
docker volume create gogs-data
If so then you need to specify it as external inside your docker compose file like this:
volumes:
gogs-data:
external: true
You can also define a different name for your external volume and keep using current volume name inside your docker compose file to avoid naming conflicts, like for example, let's say your project is about selling cars so you want the external volume to be call selling-cars-gogs-data but want to keep it simple as gogs-data inside your docker compose file, then you can do this:
volumes:
gogs-data:
external:
name: selling-cars-gogs-data
Or even better using environment variable to set the volume name for a more dynamic docker compose design, like this:
volumes:
gogs-data:
external:
name: "${MY_GOGS_DATA_VOLUME}"
And then start your docker compose like this:
env MY_GOGS_DATA_VOLUME='selling-cars-gogs-data' docker-compose up
Hope this helps, here is also a link to the docker compose external volumes documentation in case you want to learn more: https://docs.docker.com/compose/compose-file/#external
You can make pretty much everything external, including container linking to connect to other docker compose containers.
What's the right way to mix named volumes with and without local host path in docker compose v3?
This way I'm getting YML error:
volumes:
/mnt/volume-part1:/volume
conf:
vhost:
html:
certs:
Then I'd like to refer to volume inside containers...
For named volumes, you need to declare the volume name under the dedicated volumes section in the compose file. For a mount, you don't declare it in that section:
Consider the following compose file:
version: "3"
services:
db:
image: db
volumes:
- data-volume:/var/lib/db
- /mnt/volume-part1:/volume
volumes:
data-volume:
As you can see the named volume data-volume needes to be declared in the volumes section before being assiged to the container.
Whereas the directory mount is directly mounted onto the container.
UPDATE
If you don't want to replicate the machine path on all the container, you can use a clever trick to specify where exactly the named volume will be created as such:
version: "3"
services:
db:
image: db
volumes:
- data-volume:/var/lib/db
- volume-part1:/volume
volumes:
data-volume:
volume-part1:
driver_opts:
type: none
device: /mnt/volume-part1
o: bind
As you can see above, we have created a named volume volume-part1 and specified where this volume will be backuped on the host machine.
I trying to upgrade docker-compose.yml from version 1 to version 3.
Main question about
volumes_from: To share a volume between services,
define it using the top-level volumes option and
reference it from each service that shares it using the
service-level volumes option.
Simplest example:
version "1"
data:
image: postgres:latest
volumes:
- ./pg_hba.conf/:/var/lib/postgresql/data/pg_hba.conf
postgres:
restart: always
image: postgres:latest
volumes_from:
- data
ports:
- "5432:5432"
If I have understood correctly, should be converted to
version: "3"
services:
db:
image: postgres:latest
restart: always
volumes:
- db-data:/var/lib/postgresql/data
ports:
- "5432:5432"
networks:
- appn
networks:
appn:
volumes:
db-data:?
Question: How now in top-level volumes option i can set relative path to folder "example_folder" from windows host to "db-data" ?
In this instance, you might consider not using volumes_from.
As mentioned in this docker 1.13 issue by Sebastiaan van Stijn (thaJeztah):
The volumes_from is basically a "lazy" way to copy volume definitions from one container to another, so;
docker run -d --name one -v myvolume:/foo image-one
docker run -d --volumes-from=one image-two
Is the same as running;
docker run -d --name one -v myvolume:/foo image-one
docker run -d --name two -v myvolume:/foo image-two
If you are deploying to AWS you should not use bind-mounts, but use named volumes instead (as in my example above), for example;
version: "3.0"
services:
db:
image: nginx
volumes:
- uploads-data:/usr/share/nginx/html/uploads/
volumes:
uploads-data:
Which you can run with docker-compose;
docker-compose up -d
Creating network "foo_default" with the default driver
Creating volume "foo_uploads-data" with default driver
Creating foo_db_1
Basically, it is not available in docker compose version 3:
There's a couple of reasons volumes_from is not ported to the compose-file "3";
In a swarm, there is no guarantee that the "from" container is running on the same node. Using volumes_from would not lead to the expected result.
This is especially the case with bind-mounts, which, in a swarm, have to exist on the host (are not automatically created)
There is still a "race" condition (as described earlier)
The "data" container has to use exactly the right paths for volumes as the "app" container that uses the volumes (i.e. if the "app" uses the volume in /some/path/in/container, then the data container also has to have the volume at /some/path/in/container). There are many cases where the volume may be shared by multiple services, and those may be consuming the volume in different paths.
But also, as mentioned in issue 19990:
The "regular" volume you're describing is a bind-mount, not a volume; you specify a path from the host, and it's mounted in the container. No data is copied from the container to that path, because the files from the host are used.
For a volume, you're asking docker to create a volume (persistent storage) to store data, and copy the data from the container to that volume.
Volumes are managed by docker (or through a plugin) and the storage path (or mechanism) is an implementation detail, as all you're asking is a storage, that's managed.
For your question, you would need to define a docker volume container and copy your host content in it:
services:
data:
image: "nginx:alpine"
volumes:
- ./pg_hba.conf/:/var/lib/postgresql/data/pg_hba.conf