I am currently learning Docker. I am stucked at the idea of volumes. I assume that they made to store the data whenever we restart the container etc., but i do not understand what happens if we don't provide the ":" for the source:target.
Example:
- "/usr/src/my-app/frontend/node_modules"
- "/usr/src/my-app/backend/node_modules"
What do we store inside the container if we use volumes like above?
The whole docker-compose
version: '3'
services:
nginx:
image: nginx
container_name: nginx
ports:
- 80:80
restart: always
volumes:
- "./nginx/default.conf:/etc/nginx/conf.d/default.conf"
backend:
build:
dockerfile: Dockerfile.dev
context: ./backend
container_name: backend
volumes:
- "/usr/src/my-app/backend/node_modules"
- "./backend:/usr/src/my-app/backend"
frontend:
build:
dockerfile: Dockerfile.dev
context: ./frontend
container_name: frontend
environment:
CHOKIDAR_USEPOLLING: "true"
volumes:
- "/usr/src/my-app/frontend/node_modules"
- "./frontend:/usr/src/my-app/frontend"
It is an anonymous volume. It is managed by docker like a named volume, but it doesn't have a real name, only a GUID. Likewise, it's similar to the one you get when you use the VOLUME instruction in your Dockerfile without mounting a named volume or bind mount to that path when running the container.
See this for example (emphasis mine):
-v or --volume: Consists of three fields, separated by colon characters (:). The fields must be in the correct order, and the meaning of each field is not immediately obvious.
In the case of named volumes, the first field is the name of the volume, and is unique on a given host machine. For anonymous volumes, the first field is omitted.
https://docs.docker.com/storage/volumes/#choose-the--v-or---mount-flag
As long as you keep the container and only restart it, the same volume will be used. If you delete the container / create a new container, it will use a new volume.
Related
I have this docker-compose.yml, and I have a Postgres database and Grafana running over it to make queries on data.
version: "3"
services:
db:
image: postgres
container_name: db
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD=my_secret_password
grafana:
image: grafana/grafana
container_name: grafana
depends_on:
- db
ports:
- "3000:3000"
I start this compose with the command docker-compose up, but then, if I want to not lose any data, I must run docker-compose stop instead of docker-compose down.
I also read about docker commit, but "the commit operation will not include any data contained in volumes mounted inside the container", so I guess it's no use for my needs.
What's the proper way to store the created volumes and reusing them with commands up/down, so even when recreating the containers? I must use some sort of backup methods provided by every image (so, for example, a DB export for Postgres, and some other type of export for Grafana), or there is a way to do this inside docker-compose.yml?
EDIT:
I also read about volumes, but is there a standard way to store everything?
In the link provided by #DannyB, setting volumes to ./postgres-data:/var/lib/postgresql instead of ./postgres-data:/var/lib/postgresql/data caused the container to not store the actual folder.
My question is: every image must follow a particular pattern like the one above? This path to data to store the volume underlying is present in every Docker image Readme? Or is there something like:
volumes:
- ./my_image_root:/
Docker provides for volumes as the way to persist volumes between container invocations and to share data between containers.
They are quite simple to declare and use in compose files:
volumes:
postgres:
grafana:
services:
db:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD=my_secret_password
volumes:
- postgres:/var/lib/postgresql/data
grafana:
image: grafana/grafana
depends_on:
- db
volumes:
- grafana:/var/lib/grafana
ports:
- "3000:3000"
Optionally, you can also set a local directory as your container volume
with the added convince of having the files easily accessible not only from inside the container. This is especially helpful for mounting specific config files to their location in the container, you can edit the file locally like any other file restart the container with the updated configuration (certificates and other similar files also make good use of this option). And you do that like so:
volumes:
- /home/myusername/postgres_data/:/var/lib/postgresql/data/
PS. I have omitted the container_name and version directives from this compose.yml because (as of docker 20.10), the docker compose spec determines version automatically, and docker compose exposes enough functionality that accessing the containers directly using short names isn't necessary usually.
I have docker-compose file, where I have specified volumes:
services:
portal:
env_file:
- ${ENV_FILE}
build:
context: .
dockerfile: ./Dockerfile
container_name: sensus-portal
image: sensus-portal
hostname: ${DOMAIN}
working_dir: /var/www/html
volumes:
- ../localhost-certificates:/var/certificates
- ./portal:/var/www/html
- ../wp-content:/var/www/html/wp-content
ports:
- "80:80"
- "443:443"
Problem is that when last volume added, it replaces previous stuffs which was in var/www/html/wp-content ... I want to assign volume /var/www/html and then into this add (merge) volume /var/www/html/wp-content.
Is there any chance how to do this?
No, there’s no way to merge the contents of two volumes or host directories. This works the same way as normal Linux mount(8): the directory or volume you’re mounting into the container hides whatever was there already, and there’s no way to get at the hidden content.
The Dockerfile COPY directive is a little more flexible this way and you might consider building this content into your image.
I don't understand the syntax of the docker-compose-file.
First of all
version: '3'
services:
bla:
command: /bin/bash
stdin_open: true
#tty: true
container_name: docker-gulp-template
#restart: always
build: .
ports:
- '80:3000'
volumes:
- ".:/usr/src/html/bla-source"
volumes:
volumes-xyz:
If I execute it with
docker-compose up
It does create a container with the name
docker-gulp-template_bla
But that sounds illogical to me, though. Shouldn't the container be called this way:
bla_docker-gulp-template ?
Why was it solved like this? Does any of you have an example?
And another point:
volumes:
- ".:/usr/src/html/bla-source"
volumes:
volumes-xyz:
Why do I need the second volumes command and how does docker know that the first volume path belongs to the other volumes name?
Thanks in advance
You don't need the volume section.
A volume can be a named volume, created under the top level volumes section, like
volumes:
volumes-xyz:
and mounted under a service with
volumes:
- "volumes-xyz:/usr/src/html/bla-source"
Named volumes are managed by docker (/var/lib/docker/volumes/ on Linux).
Volume can also be anonymous by
volumes:
- "/usr/src/html/bla-source"
- ".:/usr/src/html/bla-source", on the other hand, creates a "bind mount". It's very similar to volume but you can choose its path to create a two-way mapping between your container and the host.
I am trying to allow nginx to proxy between multiple containers while also accessing the static files from those containers.
To share volumes between containers created using docker compose, the following works correctly:
version: '3.6'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
image: webtest
command: ./start.sh
volumes:
- .:/code
- static-files:/static/teststaticfiles
nginx:
image: nginx:1.15.8-alpine
ports:
- "80:80"
volumes:
- ./nginx-config:/etc/nginx/conf.d
- static-files:/static/teststaticfiles
depends_on:
- web
volumes:
static-files:
However what I actually require is for the nginx compose file to be in a separate file and also in a completely different folder. In other words, the docker compose up commands would be run separately. I have tried the following:
First compose file:
version: '3.6'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
image: webtest
command: ./start.sh
volumes:
- .:/code
- static-files:/static/teststaticfiles
networks:
- directorylocation-nginx_mynetwork
volumes:
static-files:
networks:
directorylocation-nginx_mynetwork:
external: true
Second compose file (ie: nginx):
version: '3.6'
services:
nginx:
image: nginx:1.15.8-alpine
ports:
- "80:80"
volumes:
- ./nginx-config:/etc/nginx/conf.d
- static-files:/static/teststaticfiles
networks:
- mynetwork
volumes:
static-files:
networks:
mynetwork:
The above two files work correctly in the sense that the site can be viewed. The problem is that the static files are not available in the nginx container. The site therefore displays without any images etc.
One work around which works correctly found here is to change the nginx container static files volume to instead be as follows:
- /var/lib/docker/volumes/directory_static-files/_data:/static/teststaticfiles
The above works correctly, but it seems 'hacky' and brittle. Is there another way to share volumes between containers which are housed in different compose files without needing to map the /var/lib/docker/volumes directory.
By separating the 2 docker-compose.yml files as you did in your question, 2 different volumes are actually created; that's the reason you don't see data from web service inside volume of nginx service, because there are just 2 different volumes.
Example : let's say you have the following structure :
example/
|- web/
|- docker-compose.yml # your first docker compose file
|- nginx/
|- docker-compose.yml # your second docker compose file
Running docker-compose up from web folder (or docker-compose -f web/docker-compose.yml up from example directory) will actually create a volume named web_static-files (name of the volume defined in docker-compose.yml file, prefixed by the folder where this file is located).
So, running docker-compose up from nginx folder will actually create nginx_static-files instead of re-using web_static-files as you want.
You can use the volume created by web/docker-compose.yml by specifying in the 2nd docker compose file (nginx/docker-compose.yml) that this is an external volume, and its name :
volumes:
static-files:
external:
name: web_static-files
Note that if you don't want the volume (and all resources) to be prefixed by the folder name (default), but by something else, you can add -p option to docker-compose command :
docker-compose \
-f web/docker-compose.yml \
-p abcd \
up
This command will now create a volume named abcd_static-files (that you can use in the 2nd docker compose file).
You can also define the volumes creation on its own docker-compose file (like volumes/docker-compose.yml) :
version: '3.6'
volumes:
static-files:
And reference this volume as external, with name volumes_static-files, in web and nginx docker-compose.yml files :
volumes:
volumes_static-files:
external: true
Unfortunately, you cannot set the volume name in docker compose, it will be automatically prefixed. If this is really a problem, you can also create the volume manually (docker volume create static-files) before running any docker-compose up command (I do not recommand this solution though because it adds a manual step that can be forgotten if you reproduce your deployment on another environment).
I have a docker compose file in a local folder on my mac. I have also another folder /src which should act as the root element. The docker-compose file looks like this:
version: '2'
services:
fpm:
image: sbusso/php-fpm-ion
nginx:
image: nginx:stable
ports:
- "80:80"
links:
- fpm
- db
db:
image: orchardup/mysql
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: myproject
I understand what we are doing here, but I am missing the solution that /src is taken as the root and I think I need to set up an lsync service which syncs between local and my docker container. So I found this one, but it is not working properly - the root /src is not taken into account. I just want to type localhost in my browser and it should open the /src folder.
version: '2'
services:
fpm:
image: sbusso/php-fpm-ion
links:
- sync
volumes_from:
- sync
db:
image: orchardup/mysql
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: myproject
links:
- sync
volumes_from:
- sync
nginx:
image: nginx:stable
ports:
- "80:80"
links:
- sync
volumes_from:
- sync
sync:
image: zeroboh/lsyncd
volumes:
- /var/www/html
- ./src:/src:Z
- ./docker-config/nginx:/etc/nginx/conf.d
- /var/lib/php/session
- ./docker-config/lrsync/lrsync.lua:/etc/lrsync/lrsync.lua
- ./sync:/sync
What I do understand is that every image that is loaded links the sync service into it. What I do not understand is why every image needs a volumes_from and that the syntax in sync explicitly says - can somebody help me, setting this up correctly?
Thanks
volumes_from imports volumes from another container
By default, each container has no volumes. You can define local volumes using the volumes attribute, but the volumes are only used in that container. In order for other containers to make use of them, those containers must import the volumes using volumes_from, pointing to the name of one or more containers. All volumes in those named containers are then made available in the current container.
The Z volume label indicates a private volume
You are mounting the /src volume using this:
volumes:
- ./src:/src:Z
That's fine, except you are also using volumes_from, and your question indicates that you specifically wanted to share /src. But by using the Z label, you have told Docker to make this a private volume.
From the documentation:
Volume labels
Labeling systems like SELinux require that proper labels are placed on volume content mounted into a container. Without a label, the security system might prevent the processes running inside the container from using the content. By default, Docker does not change the labels set by the OS.
To change a label in the container context, you can add either of two suffixes :z or :Z to the volume mount. These suffixes tell Docker to relabel file objects on the shared volumes. The z option tells Docker that two containers share the volume content. As a result, Docker labels the content with a shared content label. Shared volume labels allow all containers to read/write content. The Z option tells Docker to label the content with a private unshared label. Only the current container can use a private volume.
In this case, "current container" is sync, so only that container may use the volume. The others may not use it.