docker volume prune: Dont delete named volumes - docker

I have created a volume
docker volume create postgresql_db
Now i am using it in my docker-compose.xml
services:
postgresql:
image: "postgres:11-alpine"
volumes:
- type: volume
source: postgresql_data
target: /var/lib/postgresql/data
volume:
nocopy: true
environment:
PGDATA: '/var/lib/postgresql/data/pgdata'
networks: # connect to the bridge
- postgresql_network
command: ["postgres", "-c", "log_statement=all","-c", "log_destination=stderr"]
volumes:
postgresql_data:
external: true
Sometimes i want to stop and clean the docker system.
So I try to run
docker stop $(docker ps -aq) # stop all containers
docker rm $(docker ps -a -q) # remove all containers
docker container prune
docker image prune
docker network prune
docker volume prune #<-- remove all dangling volumes also
this also delete postgresql_data i.e named volume
how to avoid that.

Well, according to Docker's documentation, you can simply filter your volumes (by label, for instance):
docker volume prune --filter label!=postgresql_db
then all volumes that are not (look at the !) named postgresql_db and are not attached to 1 or more containers will be removed.

There's no option to the docker volume prune command for this. There's also no metadata on the volume that indicates it's an anonymous volume, it's just a named volume with a random hex name. The best I do is list the dangling volumes and delete the ones with a long hex name. As long as your named volumes are not exactly 64 hex characters, they won't be included in this:
docker volume ls -qf dangling=true | egrep '^[a-z0-9]{64}$' | \
xargs --no-run-if-empty docker volume rm

It worked for me.
Delete all unnamed volumes:
docker volume prune --filter "label!=com.docker.compose.volume"

docker rm offers the flag -v which is deleting all anonymous volumes corresponding to this container.
If you want to prune the dangling ones as well (not associated to any container you're currently running) do the docker volume prune before stopping and removing the running ones.
Therefore:
docker volume prune #<-- remove all dangling volumes also
docker stop $(docker ps -aq) # stop all containers
docker rm -v $(docker ps -a -q) # remove all containers
docker container prune
docker image prune
docker network prune

Related

docker volume prune not deleting unused volumes

My understanding is docker volume prune should delete volumes which are no longer associated with containers.
Locally I have
$ docker volume ls | head -n 2
DRIVER VOLUME NAME
local 0d4cd922a4ed3e970726b1edb860c7dda3ae1e47f812585d9517d512ae70d5cf
I confirm it doesn't have an associated container via
$ docker ps -a --filter volume=0d4cd922a4ed3e970726b1edb860c7dda3ae1e47f812585d9517d512ae70d5cf
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
therefore I expect this volume to be deleted, but alas
$ docker volume prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B
$ docker volume ls | head -n 2
DRIVER VOLUME NAME
local 0d4cd922a4ed3e970726b1edb860c7dda3ae1e47f812585d9517d512ae70d5cf
what am I missing ?

Why some volumes are already created inside docker engine?

Whenever I run the below command:
docker volume ls
I can see some volumes already created in my docker engine.
DRIVER VOLUME NAME
local 5df9458932cd504e10b2b37856c434cbdf3876733684b100cbf390c965ac9581
local 6f7037bc33861a5e42a9f8bcd699f8184ff1916a297a718ccc4df5f369d07530
local 8a86c462020f35f1051b47c48555228a1df359251f2496c32ed45a9081bb1872
local 85ed838d2e081eddc672fd8ddb15bbb3eecc73adb270678c98b7c50a03ecb2fc
Why are those volume created ?
How can I find for what purpose they exists ?
If you started a Docker container with a volume that doesn't have a name or host mount point, Docker will create a unique name for them. These docs briefly mention anonymous volumes like this. Most likely, a Dockerfile had a VOLUME section and wasn't run with a corresponding --mount or -v flag to bind some local volume to the container's volume.
Also see this devops stack exchange answer.
Here's an example of when an anonymous volume is created:
Dockerfile with anonymous volumes:
FROM alpine:3.9
VOLUME ["/root", "/test"]
Building/running container without mounting or otherwise naming the /root, /test volumes:
$ docker volume ls
DRIVER VOLUME NAME
$ docker build -t test .
$ docker run -it --rm -d --name volume-test test:latest sh
$ docker volume ls
DRIVER VOLUME NAME
local 5b332abd25b77c1ac324a0e3c00dc9a554cfe80c996a20bd77ef10c35c8ef98a
local 05c903f47f3f3666e03ee06154ff54b23547a5cc65750ca18bb40be40ed4049c
local 6f595aada6ae7c9fb16831996c2bdd8d652bec55a7cedf96afef95aec8f4e6e1
local 7f54c9dbbec46acc5a843499c65a50e23a78baa884facd026704d0dcb0362c9e
local 47a791197d6164757b015df1e2aba48bac3999720ead6b5981820a3aaece4113
local 214155fe63200cc859c1eddd2b31aa990fd6eb7c8614aa02bd8b57690b0fe53e
Of course, you can always inspect the volumes to try to find out where they came from but this may or may not be useful for you:
docker inspect 5b332abd25b77c1ac324a0e3c00dc9a554cfe80c996a20bd77ef10c35c8ef98a

How to cleanup docker containers and images on linux machines

We have Linux redhat machine with docker and docker compose
Now we want to clean all containers and images - like we have scratch new docker
As I understand to get that , we need to perform the following procedure with this order:
Am I right with this procedure? , or I missing something?
1. docker stop <CONTAINER ID>
2. docker container rm <CONTAINER ID>
3. docker image rm <IMAGE ID>
example
first find - CONTAINER ID
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
654fa81f4439 confluentinc/cp-enterprise-control-center:5.0.0 "/etc/confluent/dockā€¦" 9 minutes ago Up 9 minutes 0.0.0.0:9021->9021/tcp control-center
1)
stop container
docker stop 654fa81f4439
654fa81f4439
2)
delete container
docker container rm 654fa81f4439
654fa81f4439
3)
find image ID
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
confluentinc/cp-enterprise-control-center 5.0.0 e0bd9a5edb95 15 months ago 617MB
delete image
ocker image rm e0bd9a5edb95
Untagged: confluentinc/cp-enterprise-control-center:5.0.0
Untagged: confluentinc/cp-enterprise-control-center#sha256:2e406ff8c6b1b8be6bf01ccdf68b14be0f0759db27c050dddce4b02ee0894127
Deleted: sha256:e0bd9a5edb9510a326934fa1a80a4875ab981c5007354de28f53bfb3e11bc34a
Deleted: sha256:c23255297f6d75f156baf963786d3ded1d045b726d74ed59c258dc8209bac078
Deleted: sha256:6cab492e72ca2578897b7ceecb196e728671158e262957f3c01e53fd42f6f8b4
In short, yes, it is the correct procedure to clear all the containers and images.
But you can do it more easily. For example:
Stop all containers at
once:
docker container stop $(docker container ls -aq)
Remove all stopped containers: docker container prune --force
Remove all unnused images: docker image prune --all --force
Sometimes Docker volumes are used for containers to persist data. You may want to clean them too (docker volume prune --force).
Others Docker resources may be left on your system (such as networks and build caches).
You can appeal to docker system prune to remove all unused data:
$ docker system prune --all --volumes
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all volumes not used by at least one container
- all images without at least one container associated to them
- all build cache
Are you sure you want to continue? [y/N] y

Find which container uses a Docker Volume

In running docker volume ls the response contains:
DRIVER VOLUME NAME
local 09033b4b22135832411ff485d5adc90e778316c1b9ba27bf8032cb65b8de557b
local 2ae85f4d2270716f936c4ef06e1a9408f90826258c7489a150125ff9d13ce79c
local 58497ac069495708d3bd17aab2b16b9a02badd245de4614a05a7133bdf0efb34
local eb958485be4ae9c8a71a01a4f03c7cbf9f76c3a8fb2622ded58cf96542373b0d
local efb5d345f272b32f409800e4804e22a93759c5668c8cb43e1887b5943ea217f9
How can I correlate those volume names to the containers using them?
I'm trying to understand where the volumes came from and if I can remove them.
Simple example starting a centos container using a named volume
docker run --rm -dit -v my-vol:/my-vol centos
You can make use of the --filter flag to see which container(s) the volume is being used by[1]:
docker ps -a --filter volume=my-vol
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0f70a651c472 centos "/bin/bash" 24 seconds ago Up 23 seconds distracted_swirles
If you want to remove volumes that are not being used by any containers, you can use docker volume prune
docker volume prune
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N]
[1] https://docs.docker.com/engine/reference/commandline/ps/#filtering
[2] https://docs.docker.com/engine/reference/commandline/volume_prune/

Kill docker-compose containers without docker-compose.yml file

I have some docker containers provisioned by docker-compose with restart: always flag. But I accidentally delete docker-compose.yml file.
How do I delete those containers so that they will not restart automatically?
Without the compose file, docker-compose can't guess the name of the containers that need to be killed. You need to do that manually using docker command:
docker container ls
docker rm -f <container-name-or-id>
You can use docker ps -a to list all the running containers. After you can use docker stop container-name to stop individually the containers you want and after you can use docker rm $(docker ps -a -q) to delete and remove all the stopped containers.
Also if you want to delete the docker images as well you can use docker images to list the existing images and after docker rmi image-name to delete the image.
I had a similar issue - I didn't want to fetch the .yml files just to remove the containers. As docker-compose adds labels to the containers, we can use these to filter the matching containers and then remove them.
To remove all the containers that docker-compose created for project ProjectName (normally the name of the folder where the docker-compose.yml was) in shell (bash or similar), execute the following:
docker ps --filter 'label=com.docker.compose.project=ProjectName' --format {{.ID}} | xarg -n 1 docker rm --force --volumes
docker ps with the filter provides the list of containers, the format is provided so that only container IDs are printed
xargs -n 1 - takes the list of container IDs and executes the target command on each of them providing the container ID as the argument to the target
docker rm removes existing containers
--force - to also stop running containers (otherwise running containers would not be stopped or removed)
--volumes - to also remove any anonymous volumes associated with the containers

Resources