I have many Docker containers both running and exited. I only want to keep the containers that were created before today/some specified time -- I would like to remove all containers created today. Easiest way to approach this?
Out of the box on all OS there is the possibility to remove all containers younger than a given one:
docker rm -f $(docker ps -a --filter 'since=<containername>' --format "{{.ID}}")
so the container given in since will be kept, but all newer ones not. Maybe that suits your use case.
If you really need a period of time there will be some bash magic doing that. But specify your needs exactly then..
In detail:
docker rm: removing one or more containers
-f: force running containers to stop
docker ps -a: listing all containers
--filter 'since=..' filtering containers created since given with all details
--format "{{.ID}}": filtering ID-column only
Related
I've just done my first ever Docker deployment, when i run this command to see the status of recent processes...
docker ps -a
I get this output
My question is; what are those name referring to?
Those are random names generated for each container you are running.
You can see those names at pkg/namesgenerator/names-generator.go.
// Docker, starting from 0.7.x, generates names from notable scientists and hackers.
// Please, for any amazing man that you add to the list, consider adding an equally amazing woman to it, and vice versa.
right = [...]string{
// Muhammad ibn Jābir al-Ḥarrānī al-Battānī was a founding father of astronomy. https://en.wikipedia.org/wiki/Mu%E1%B8%A5ammad_ibn_J%C4%81bir_al-%E1%B8%A4arr%C4%81n%C4%AB_al-Batt%C4%81n%C4%AB
"albattani",
// Frances E. Allen, became the first female IBM Fellow in 1989. In 2006, she became the first female recipient of the ACM's Turing Award. https://en.wikipedia.org/wiki/Frances_E._Allen
"allen",
...
You could have fixed name by adding --name to your docker run commands.
That practice was introduced in commit 0d29244 by Michael Crosby (crosbymichael) for docker 0.6.5 in Oct. 2013:
Add -name for docker run
Remove docker link
Do not add container id as default name
Create an auto generated container name if not specified at runtime.
Solomon Hykes (shykes) evolved that practice in docker 0.7 with commit 5d6ef317:
New collection of random names for 0.7:
mood + famous inventor.
Eg. 'sad-tesla' or 'naughty-turing'
As VonC already wrote, "those are random names generated for each container you are running". So why should you use custom names?
docker run [image-name] -[container-name]
docker run wildfly-8.1 -mywildfly
well if you want to stop/kill/run/inspect or do anything with your container, you can use your name to do so:
docker kill mydocker
docker run mydocker
Otherwise you have to do docker ps all the time and check the id (since you will not remember those made-up custom names).
One important sidenote, if you assign custom name to docker, it will be forever assigned to that specific container, which means even if you kill the container, it is not running anymore, you cannot reuse the name.
If you run
docker ps -a
you can see the old one is still there, but stopped. You can't re-use a container name until the previous container has been removed by docker rm.
To remove containers older than some custom time, use the command:
$ docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm
More on removing containers here.
Using docker under Kubuntu 18 I got out of free space on the device.
I run commands to clear space:
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker-compose down --remove-orphans
docker system prune --force --volumes
As I was still of free space opened /var/lib/docker/overlay2/ directory and
deleted a lot of subdirectories ubder it.
After that I got error :
$ docker-compose up -d --build
Creating network "master_default" with the default driver
ERROR: stat /var/lib/docker/overlay2/36af81b800ebb595a24b6c724318c1126932d2bfae61e2c98bfc65a203b2b928: no such file or directory
Looks like that is not a good way to free space. Which way isd good in my case?
If there is a way to reinit my docker apps?
Thanks!
As I was still of free space opened /var/lib/docker/overlay2/ directory and deleted a lot of subdirectories ubder it.
At this point, the docker filesystem has been corrupted. To repair, the best you can do is backup anything you want to save, particularly any volumes, stop the docker engine (systemctl stop docker), delete the entire docker filesystem (rm -rf /var/lib/docker), and restart docker (systemctl start docker).
At that point the engine will be completely empty without any images, containers, etc. You'll need to pull/rebuild your images and recreate the containers you were running. Hopefully that's as easy as a docker-compose up.
/var/lib/docker/overlay2/ is where docker stores the image layers.
Now, docker system prune -a removes all the unused images, stopped containers and the build cache if I'm not wrong.
One advice, since you are building images, check docker buildkit. I know docker-compose added support for it, but I don't know if your version supports that. To make it short, building your images will be way faster.
I have a basic Hyperledger Fabric network where the nodes uses docker containers.
Inside each node I have done some file creation and editing. However, I now want to restart the network with clean containers.
I have tried to shut down all containers, images and networks, then run the docker prune command.
I have also tried to delete all volumes.
However, once I re-create the Fabric network, when bashing into a container the old files that I custom created are still there. I never created those files on host machine, only inside that container. But I do not understand how it is possible that those files still exists. I even tried to delete the images.
System is Ubuntu 18.4
Can anybody spot the potential fix to this?
Delete the volumes of the containers after stopping and removing the containers by below commands,
docker kill $(docker ps -aq)
docker rm $(docker ps -aq)
Then remove the volumes of the containers by below command.
docker system prune --volumes -f
This removes all the unused networks and volumes .
Hope this helps.
We have a system where user may install some docker containers. We dont have a limit on what he can install. After some time, we need to clean up - delete all the images that are not in used in the swarm.
What would be the solution for that using docker remote API?
Our idea is to have background image-garbage-collector thread that:
lists all the images
try to delete some
if it fails, just ignore
Would this make sense? Would this affect swarm somehow?
Cleaner way to list and (try to) remove all images
The command docker rmi $(docker images -q) would do the same as the answer by #tpbowden but in a cleaner way. The -q|--quiet only list the images ID.
It may delete frequently used images (not running at the cleaning date)
If you do this, when the user will try to swarm run deleted-image it will:
Either pull the image (< insert network consumption warning here />)
Either just block as the pull action is not automatic in swarm if I remember it right (< insert frequent support request warning here about misunderstood Swarm behavior />).
"dangling=true" filter:
A useful option is the --filter "dangling=true". Executing swarm images -q --filter "dangling=true" will display not-currently-running images.
Though challenge
You issue reminds me the memory management in a computer. Your real issue is:
How to remove image that won't be used in the future?
Which is really hard and really depends on your policy. If your policy is old images are to be deleted the command that could help is: docker images --format='{{.CreatedSince}}:{{ .ID}}'. But then the hack starts... You may need to grep "months" and then cut -d ':' -f 2.
The whole command would result as:
docker rmi $(docker images --format='{{.CreatedSince}}:{{ .ID}}' G months | cut -d ':' -f 2)
Note that this command will need to be run on every Swarm agent as well as the Swarm manager, not only the Swarm manager.
Swarm and registry
Be aware than a swarm pull image:tag will not pull the image on Swarm agents! Each Swarm agent must pull the image itself. Thus deleting still used images will result in network load.
I hope this answer helps. At this time there is no mean to query "image not used since a month" AFAIK.
All you need is 'prune'
$ docker image prune --filter until=72h --force --all
docker images | tail -n+2 | awk '{print $3}' | xargs docker rmi
This will list all images, strip the top line with column headings, grab the 3rd column (image ID hash) and then attempt to remove them all. Docker will prevent you from removing any images that are currently used by running containers.
If you want to do this in a slightly less 'hacky' way, you could use Docker's API to get images which aren't being used and delete them that way.
I've just done my first ever Docker deployment, when i run this command to see the status of recent processes...
docker ps -a
I get this output
My question is; what are those name referring to?
Those are random names generated for each container you are running.
You can see those names at pkg/namesgenerator/names-generator.go.
// Docker, starting from 0.7.x, generates names from notable scientists and hackers.
// Please, for any amazing man that you add to the list, consider adding an equally amazing woman to it, and vice versa.
right = [...]string{
// Muhammad ibn Jābir al-Ḥarrānī al-Battānī was a founding father of astronomy. https://en.wikipedia.org/wiki/Mu%E1%B8%A5ammad_ibn_J%C4%81bir_al-%E1%B8%A4arr%C4%81n%C4%AB_al-Batt%C4%81n%C4%AB
"albattani",
// Frances E. Allen, became the first female IBM Fellow in 1989. In 2006, she became the first female recipient of the ACM's Turing Award. https://en.wikipedia.org/wiki/Frances_E._Allen
"allen",
...
You could have fixed name by adding --name to your docker run commands.
That practice was introduced in commit 0d29244 by Michael Crosby (crosbymichael) for docker 0.6.5 in Oct. 2013:
Add -name for docker run
Remove docker link
Do not add container id as default name
Create an auto generated container name if not specified at runtime.
Solomon Hykes (shykes) evolved that practice in docker 0.7 with commit 5d6ef317:
New collection of random names for 0.7:
mood + famous inventor.
Eg. 'sad-tesla' or 'naughty-turing'
As VonC already wrote, "those are random names generated for each container you are running". So why should you use custom names?
docker run [image-name] -[container-name]
docker run wildfly-8.1 -mywildfly
well if you want to stop/kill/run/inspect or do anything with your container, you can use your name to do so:
docker kill mydocker
docker run mydocker
Otherwise you have to do docker ps all the time and check the id (since you will not remember those made-up custom names).
One important sidenote, if you assign custom name to docker, it will be forever assigned to that specific container, which means even if you kill the container, it is not running anymore, you cannot reuse the name.
If you run
docker ps -a
you can see the old one is still there, but stopped. You can't re-use a container name until the previous container has been removed by docker rm.
To remove containers older than some custom time, use the command:
$ docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm
More on removing containers here.