Up to recent version of Docker (v1.10), we were thought that we can use DOC: data-only containers. So I would create such DOC (based on e.g. busybox) and use --volumes-from to link it to my container. You can still read about this in Docker documentation.
With new version of docker, it is said that instead of DOC we should use named volumes. Here is an example of docker-compose.yml:
version: '2'
services:
elasticsearch:
image: elasticsearch:2.2.0
command: elasticsearch -Des.network.host=0.0.0.0
ports:
- "9201:9200"
volumes:
- "es-data:/usr/share/elasticsearch/data"
volumes:
es-data:
Here we created and use named volume es-data.
There is still not much documentation on this new feature. I am asking:
Can we replace DOC with named containers? How long volume is persisted? What if I remove the container that is using it?
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Can we replace DOC with named containers?
In many cases, yes, named containers will be a better option.
How long volume is persisted? What if I remove the container that is using it?
If you remove the container, the volume will still be there. The only way to remove the volume is to use docker-compose down -v or docker volume rm <volume name>.
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Instead of --volumes-from, you can use --volume=<volume name>.
Note that volumes created by docker-compose are always prefixed with the project name, so if you use it with a docker command the full name is actually <project_name>_es-data.
Related
I'm running a docker-compose setup, and when I want to update files in my image I create a new docker image. Though the problem is; the file I'm editing is located in the persistent volume, meaning the Docker image itself will get the changes, but since I'm not deleting docker-compose volumes the volume will be used by the new image, hence the old file will be used by new image.
Running docker-compose down -v is not an options because I want to keep other existing files in the volume (logs etc.).
I want to know if it possible to do this without too much hacks, since I'm looking to automate this.
Example docker-compose.yml
version: '3.3'
services:
myService:
image: myImage
container_name: myContainer
volumes:
- data_volume:/var/data
volumes:
data_volume
NOTE: The process of doing change in my case:
docker-compose down
docker build -t myImage:t1 .
docker compose up -d
You could start a container, mount the volume and execute a command to delete single files. Something like
docker run -d --rm -v data_volume:/var/data myImage rm /var/data/[file to delete]
I have a docker-compose file which describes several services. All services have volumes attached to them, however only one has the volume named. When I run docker compose down I want to automatically delete the not named volumes while at the same time create all volumes that are missing.
services:
service1:
image: some/image:1
volumes:
- named-volume:/home/user1
service2:
image: some/image:2
#volumes: not declared volumes that are named automatically with a hash
volumes:
named-volume:
name: volume-for-service1
The first time I run docker compose up I want to automatically create all volumes (named and unnamed) and when I run docker compose down I want that unnamed volumes to be deleted while the named one (volume-for-service1) to be preserved. Next time I run docker compose up it should only create the unnamed volumes as the named one already exists.
I have tried:
docker compose down -v which removed no volume
docker compose down --remove-orphans which removed no volume
docker compose down --rmi local which removed no volume
docker-compose down -v which removed the named volume
docker-compose down --remove-orphans which removed no volume
docker-compose down --rmi local which removed no volume
OS: Windows 10 x64
I don't quite get it. What command should I run to achieve desired results?
Try using --renew-anon-volumes flag when bringing up the services
and use --volumes when bringing down the services
> docker-compose --renew-anon-volumes up
> docker-compose --volumes down
Refer the docker compose documentation
-V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving
data from the previous containers.
-v, --volumes Remove named volumes declared in the `volumes`
section of the Compose file and anonymous volumes
attached to containers.
https://docs.docker.com/compose/reference/down/
To prevent removing named volumes, you should define them as external in the config file:
volumes:
volume-for-service1:
name: volume-for-service1
external: true
But you have to initially create them outside the config file somewhere else, either through:
docker volume create volume-for-service-1
or in a separate config file.
Reference: https://docs.docker.com/compose/compose-file/#external-1
I'm not aware of a way to remove unnamed volumes automatically, but you can match its hash and remove it with a small script.
To reuse your docker-compose.yml example, first you get the container name given the service name with:
docker-compose ps service2 # this is the one with unnamed volume in your example
Output could be something like:
NAME COMMAND SERVICE STATUS
project-service2-1 "docker-entrypoint.s…" service2 exited (0)
Then using the container name you can find its unamed volume hash:
docker inspect -f '{{ (index .Mounts 0).Name }}' project-service2-1
Now before deleting the volume you need to bring the container down or the volume would be in use.
docker-compose down
docker volume rm $volume # replace the "volume" var with the inspect output
Now that we saw the steps, let's try to make it a little script (slightly adjusted):
service_name=service2 # set the variable accordingly
container_id=$(docker-compose ps $service_name --quiet)
volume_name=$(docker inspect -f '{{ (index .Mounts 0).Name }}' $container_id)
docker-compose down
docker volume rm -f $volume_name
I am using docker-compose syntax version 3 and want to use some volumes. The documentation on the long syntax for volumes states the following:
type: the mount type volume or bind
but never fully explains the difference. What is it?
bind is the simpler one to understand. It takes a host path, say /data and mounts it inside your container, say /opt/app/data. /data can be anything, probably mounted on NFS or it maybe a local host path.
docker run -v /data:/opt/app/data -d nginx
volume mount is where you can use a named volume.
You would normally use a volume driver for this, but you can get a host mounted path using the default local volume driver something like the below:
docker volume create data
docker run -d -v data:/opt/app/data nginx
The named volume can also be anonymous if you run just this:
docker run -d -v /opt/app/data nginx
If you run docker volume ls, docker would have create an autogenerated long name for the anonymous volume.
In docker-compose, you would just use it as below:
web:
image: nginx:latest
volumes:
/data:/opt/app/data
data:/opt/app/data1
volumes:
data:
Until now, I have used a local LAMP stack to develop my web projects and deploy them manually to the server. For the next project I want to use docker and docker-compose to create a mariaDB, NGINX and a project container for easy developing and deploying.
When developing I want my code directory on the host machine to be synchronised with the docker container. I know that could be achieved by running
docker run -dt --name containerName -v /path/on/host:/path/in/container
in the cli as stated here, but I want to do that within a docker-compose v2 file.
I am as far as having a docker-composer.yml file looking like this:
version: '2'
services:
db:
#[...]
myProj:
build: ./myProj
image: myProj
depends_on:
- db
volumes:
myCodeVolume:/var/www
volumes:
myCodeVolume:
How can I synchronise my /var/www directory in the container with my host machine (Ubuntu desktop, macos or Windows machine)?
Thank you for your help.
It is pretty much the same way, you do the host:container mapping directly under the services.myProj.volumes key in your compose file:
version: '2'
services:
...
myProj:
...
volumes:
/path/to/file/on/host:/var/www
Note that the top-level volumes key is removed.
This file could be translated into:
docker create --links db -v /path/to/file/on/host:/var/www myProj
When docker-compose finds the top-level volumes section it tries to docker volume create the keys under it first before creating any other container. Those volumes could be then used to hold the data you want to be persistent across containers.
So, if I take your file for an example, it would translate into something like this:
docker volume create myCodeVolume
docker create --links db -v myCodeVoume:/var/www myProj
When using docker compose, I find a lot of volume instances:
› docker volume ls
DRIVER VOLUME NAME
local 4a34b9a352a459171137aac4c046a83f61e6e325b1df4b67dc2ddda8439a6427
local 6ce3e52ea363441b2c9d4b04c26b283d8b4cf631a137987da88db812a9a2d223
local a7af289b29c833510f2201647266001e4746e206128dc63313fe894821fa044d
local fb09475f75fe943671a4e73d76c09c27a4f592b8ddf62224fc4b20afa0095809
I'd like to tag or name them, then reuse them if possible rather recreating them each time.
Is that possible?
Those are anonymous container volumes that happen when you define a volume without a name or bind it to a host folder. This may be with the VOLUME definition in your Dockerfile, a docker run -v /dir ... rather than name:/dir, or a volumes entry in your docker-compose.yml with only the directory. An example of a compose file that does a named mount is:
version: '2'
volumes:
my-vol:
driver: local
services:
my-container:
image: my-image
volumes:
- my-vol:/container/path
Once the anonymous volume has been created, there's no easy way to rename it. Easiest solution is to mount the anonymous volume along with the your target named volume and do a copy, e.g.:
docker run -v 123456789:/source -v my-vol:/target --rm \
busybox cp -av /source/. /target/
Where 123456789 is the long name of your anonymous volume.