If I run this command the volume mounts and the container starts as expected with initialized state:
docker run --name gogs --net mk1net --ip 203.0.113.3 -v gogs-data:/data -d gogs/gogs
However if I run the corresponding docker-compose script the volume does not mount. The container still starts up, but without the state it reads on startup.
version: '3'
services:
gogs:
image: gogs/gogs
ports:
- "3000:3000"
volumes:
- gogs-data:/data
networks:
mk1net:
ipv4_address: 203.0.113.3
volumes:
gogs-data:
networks:
mk1net:
ipam:
config:
- subnet: 203.0.113.0/24
Any ideas?
Looking at your command, the gogs-data volume was defined outside the docker compose file, probably using something like:
docker volume create gogs-data
If so then you need to specify it as external inside your docker compose file like this:
volumes:
gogs-data:
external: true
You can also define a different name for your external volume and keep using current volume name inside your docker compose file to avoid naming conflicts, like for example, let's say your project is about selling cars so you want the external volume to be call selling-cars-gogs-data but want to keep it simple as gogs-data inside your docker compose file, then you can do this:
volumes:
gogs-data:
external:
name: selling-cars-gogs-data
Or even better using environment variable to set the volume name for a more dynamic docker compose design, like this:
volumes:
gogs-data:
external:
name: "${MY_GOGS_DATA_VOLUME}"
And then start your docker compose like this:
env MY_GOGS_DATA_VOLUME='selling-cars-gogs-data' docker-compose up
Hope this helps, here is also a link to the docker compose external volumes documentation in case you want to learn more: https://docs.docker.com/compose/compose-file/#external
You can make pretty much everything external, including container linking to connect to other docker compose containers.
Related
I got docker compose:
version: '2'
services:
elasticsearch:
image: 'elasticsearch:7.9.1'
environment:
- discovery.type=single-node
ports:
- '9200:9200'
- '9300:9300'
volumes:
- /var/lib/docker/volumes/elastic_search_volume:/usr/share/elasticsearch/data:rw
When I run:
docker volume ls
I see no results. How to list unnamed volumes?
docker volume ls as you've shown it will list all of the volumes that exist.
However, in the docker-compose.yml file you show, you're not creating a named or anonymous volume. Instead, you're creating a bind mount to connect a host directory to the container filesystem space. These aren't considered "volumes" in a technical Docker sense, and a docker volume command won't show or manipulate those.
Reaching directly into /var/lib/docker usually isn't a best practice. It's better to ask Docker Compose to manage the named volume for you:
version: '2'
services:
elasticsearch:
volumes:
# No absolute host path, just the volume name
- elastic_search_volume:/usr/share/elasticsearch/data:rw
volumes:
elastic_search_volume:
# Without this line, Compose will create the volume for you.
# With this line, Compose expects it to already exist; you may
# need to manually `docker volume create elastic_search_volume`.
# external: true
I persist container's data to a volume (not a bind mount) and I wonder how I can inspect this data later. For example, let's say that I use something like this to run a WordPress site:
docker-compose.yml:
services:
wordpress:
volumes:
- wordpress-files:/var/www/html
volumes:
wordpress-files:
Is it possible to start another container (based on Alpine or something) that would mount the same volume and also expose it to my host OS (macOS – I'm using Docker for Mac)? Something like this (pseudocode):
services:
wordpress:
image: wordpress
volumes:
- wordpress-files:/var/www/html
wordpress-files-inspector:
volumes:
- wordpress-files:/tmp:host
volumes:
wordpress-files:
It's possible to exec into a temporary container but I'd like to make the files available to my local filesystem so that I can use my local tools to browse them. Note that primarily, the files need to live in a named volume (for performance and other reasons) so it cannot be a bind mount like ./my-local-path:/var/www/html.
Why don't you just use samba? Like that:
services:
wordpress:
image: wordpress
volumes:
- wordpress-files:/var/www/html
wordpress-files-inspector:
image: dperson/samba
command: sh -c "samba.sh -s \"mount;/mount\""
volumes:
- wordpress-files:/mount
volumes:
wordpress-files:
You can inspect IP address of the wordpress-files-inspector container later (or set the container with static ip) and mount it into your host OS.
I am trying to allow nginx to proxy between multiple containers while also accessing the static files from those containers.
To share volumes between containers created using docker compose, the following works correctly:
version: '3.6'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
image: webtest
command: ./start.sh
volumes:
- .:/code
- static-files:/static/teststaticfiles
nginx:
image: nginx:1.15.8-alpine
ports:
- "80:80"
volumes:
- ./nginx-config:/etc/nginx/conf.d
- static-files:/static/teststaticfiles
depends_on:
- web
volumes:
static-files:
However what I actually require is for the nginx compose file to be in a separate file and also in a completely different folder. In other words, the docker compose up commands would be run separately. I have tried the following:
First compose file:
version: '3.6'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
image: webtest
command: ./start.sh
volumes:
- .:/code
- static-files:/static/teststaticfiles
networks:
- directorylocation-nginx_mynetwork
volumes:
static-files:
networks:
directorylocation-nginx_mynetwork:
external: true
Second compose file (ie: nginx):
version: '3.6'
services:
nginx:
image: nginx:1.15.8-alpine
ports:
- "80:80"
volumes:
- ./nginx-config:/etc/nginx/conf.d
- static-files:/static/teststaticfiles
networks:
- mynetwork
volumes:
static-files:
networks:
mynetwork:
The above two files work correctly in the sense that the site can be viewed. The problem is that the static files are not available in the nginx container. The site therefore displays without any images etc.
One work around which works correctly found here is to change the nginx container static files volume to instead be as follows:
- /var/lib/docker/volumes/directory_static-files/_data:/static/teststaticfiles
The above works correctly, but it seems 'hacky' and brittle. Is there another way to share volumes between containers which are housed in different compose files without needing to map the /var/lib/docker/volumes directory.
By separating the 2 docker-compose.yml files as you did in your question, 2 different volumes are actually created; that's the reason you don't see data from web service inside volume of nginx service, because there are just 2 different volumes.
Example : let's say you have the following structure :
example/
|- web/
|- docker-compose.yml # your first docker compose file
|- nginx/
|- docker-compose.yml # your second docker compose file
Running docker-compose up from web folder (or docker-compose -f web/docker-compose.yml up from example directory) will actually create a volume named web_static-files (name of the volume defined in docker-compose.yml file, prefixed by the folder where this file is located).
So, running docker-compose up from nginx folder will actually create nginx_static-files instead of re-using web_static-files as you want.
You can use the volume created by web/docker-compose.yml by specifying in the 2nd docker compose file (nginx/docker-compose.yml) that this is an external volume, and its name :
volumes:
static-files:
external:
name: web_static-files
Note that if you don't want the volume (and all resources) to be prefixed by the folder name (default), but by something else, you can add -p option to docker-compose command :
docker-compose \
-f web/docker-compose.yml \
-p abcd \
up
This command will now create a volume named abcd_static-files (that you can use in the 2nd docker compose file).
You can also define the volumes creation on its own docker-compose file (like volumes/docker-compose.yml) :
version: '3.6'
volumes:
static-files:
And reference this volume as external, with name volumes_static-files, in web and nginx docker-compose.yml files :
volumes:
volumes_static-files:
external: true
Unfortunately, you cannot set the volume name in docker compose, it will be automatically prefixed. If this is really a problem, you can also create the volume manually (docker volume create static-files) before running any docker-compose up command (I do not recommand this solution though because it adds a manual step that can be forgotten if you reproduce your deployment on another environment).
Really basic one but this documentation only seems to mention the docker-compose file itself as opposed to the command line: https://docs.docker.com/compose/networking/#links
When I call:
docker-compose up -d service1 service2
I want those created containers to be on a new network with a custom name. How do I specify this network when using the docker-compose up command?
Even if I add network config to the docker-compose file:
networks:
mynetwork:
service1:
networks:
- mynetwork
The docker-compose command still creates the containers on a default network, rather than use "mynetwork".
Create a external network if it doesn't exist -
$ docker network create mynetwork || true
Define external network to compose file -
.........
ports:
- "8088:8088"
networks:
- mynetwork
networks:
mynetwork:
external: true
Similarly, you can also use the default network create by compose but it will prefix your current directory name to the network name defined. You can also use the host network mode but that's not suggested.
Above two use cases are old now, since compose 3.5 you can give custom names to your compose environment. This lets services in multiple compose files talk to each other without doing much of a configuration.
Preferred
Recently in docker compsoe 3.5, they launched custom name feature. So in case you can use compose 3.5, you can opt for giving a custom name to your docker compose network. Compose will create a new network in case it doesn't exist.(preferred)
https://docs.docker.com/compose/compose-file/#name-1
Ex -
version: '3.5'
.........
ports:
- "8088:8088"
networks:
- mynetwork
networks:
mynetwork:
external: true
name: mynetwork
I trying to upgrade docker-compose.yml from version 1 to version 3.
Main question about
volumes_from: To share a volume between services,
define it using the top-level volumes option and
reference it from each service that shares it using the
service-level volumes option.
Simplest example:
version "1"
data:
image: postgres:latest
volumes:
- ./pg_hba.conf/:/var/lib/postgresql/data/pg_hba.conf
postgres:
restart: always
image: postgres:latest
volumes_from:
- data
ports:
- "5432:5432"
If I have understood correctly, should be converted to
version: "3"
services:
db:
image: postgres:latest
restart: always
volumes:
- db-data:/var/lib/postgresql/data
ports:
- "5432:5432"
networks:
- appn
networks:
appn:
volumes:
db-data:?
Question: How now in top-level volumes option i can set relative path to folder "example_folder" from windows host to "db-data" ?
In this instance, you might consider not using volumes_from.
As mentioned in this docker 1.13 issue by Sebastiaan van Stijn (thaJeztah):
The volumes_from is basically a "lazy" way to copy volume definitions from one container to another, so;
docker run -d --name one -v myvolume:/foo image-one
docker run -d --volumes-from=one image-two
Is the same as running;
docker run -d --name one -v myvolume:/foo image-one
docker run -d --name two -v myvolume:/foo image-two
If you are deploying to AWS you should not use bind-mounts, but use named volumes instead (as in my example above), for example;
version: "3.0"
services:
db:
image: nginx
volumes:
- uploads-data:/usr/share/nginx/html/uploads/
volumes:
uploads-data:
Which you can run with docker-compose;
docker-compose up -d
Creating network "foo_default" with the default driver
Creating volume "foo_uploads-data" with default driver
Creating foo_db_1
Basically, it is not available in docker compose version 3:
There's a couple of reasons volumes_from is not ported to the compose-file "3";
In a swarm, there is no guarantee that the "from" container is running on the same node. Using volumes_from would not lead to the expected result.
This is especially the case with bind-mounts, which, in a swarm, have to exist on the host (are not automatically created)
There is still a "race" condition (as described earlier)
The "data" container has to use exactly the right paths for volumes as the "app" container that uses the volumes (i.e. if the "app" uses the volume in /some/path/in/container, then the data container also has to have the volume at /some/path/in/container). There are many cases where the volume may be shared by multiple services, and those may be consuming the volume in different paths.
But also, as mentioned in issue 19990:
The "regular" volume you're describing is a bind-mount, not a volume; you specify a path from the host, and it's mounted in the container. No data is copied from the container to that path, because the files from the host are used.
For a volume, you're asking docker to create a volume (persistent storage) to store data, and copy the data from the container to that volume.
Volumes are managed by docker (or through a plugin) and the storage path (or mechanism) is an implementation detail, as all you're asking is a storage, that's managed.
For your question, you would need to define a docker volume container and copy your host content in it:
services:
data:
image: "nginx:alpine"
volumes:
- ./pg_hba.conf/:/var/lib/postgresql/data/pg_hba.conf