I am running docker on windows and even though I do docker system prune it is using more and more space somewhere on my harddisk.
Often after restarting the laptop and running prune I can get rid of some more but its less than it actually takes.
I know that docker is using these space because space on my HDD decreases when building new images and running containers but always decreases by much less space.
It's eaten over 50gb of my 256 gb SSD.
I appreciate any help in how to find and efficiently locate all files docker leaves when building and running containers.
I tried many lines from here and most work but I always fail to reclaim all space and given that I have a very small SSD I really need all the space I can get back.
Many thanks in advance!
I suggest you to add to the command docker system prune the `--all' because of:
-a, --all : Remove all unused images not just dangling ones
I use this to free up all my no more needed disk space.
Related
I've seen this issue a number of times and usually use docker system prune to solve it temporarily, but i'm not understanding why it says there is no space on the device?
The main drive on my mac currently has 170gb free space, i also have a second drive with 900gb free, the images i'm building take up a total of 900mb when built, so what is docker talking about? I have plenty of storage space!
Since you specified that the platform is Mac, your docker runtime is running inside a VM, which has it's own resources allocated.
Assuming you are using Docker For Mac, you should increase the allocated disk space for the docker VM:
In case you don't want to increase the amount of docker engine storage as answered here, you can free some space by running:
docker image prune
I am using Docker Desktop for Windows on Windows 10.
I was experiencing issues with system SSD always being full and moved 'docker-desktop-data' distro (which is used to store docker images and other stuff) out of the system drive to drive D: which is HDD using this guide.
Finally, I was happy to have a lot of space on my SSD... but docker containers started to work slower. I guess this happens due to HDD write/read operations being slower than on SSD.
Is there a better way to solve the problem of the continuously growing size of Docker distro's without impacting how fast containers actually work and images are built?
Actually only be design. As you know, a docker container is layered. So it might be feasible to check if it is possible to create something like a "base-container" from which your actual image in derived.
Also it might be sensible to check if your base distro is small enough. I often have seen containers created from full blown Debian or Ubuntu distros. Thats not the best idea. Try to derive from an alpine version or check for even smaller approaches.
I have a Kubernetes-cluster with 1 master-node and 3 worker-nodes. All Nodes are running on CentOS 7 with Docker 19.06. I'm also running Longhorn for dynamic provisioning of volumes (if that's important).
My problem is that every few days one of the worker nodes grows the HDD-usage to 85% (43GB). This is not a linear increase, but happens over a few hours, sometimes quite rapidly. I can "solve" this problem for a few days by first restarting the docker service and then doing a docker system prune -a. If I don't restart the service first, the prune removes next to nothing (only a few MB).
I also tried to find out which container is taking up all that space, but docker system df says it doesn't use the space. I used df and du to crawl along the /var/lib/docker subdirectories too, and it seems none of the folders (alone or all together) takes up much space either. Continuing this all over the system, I can't find any other big directories either. There are 24GB that I just can't account for. What makes me think this is a docker problem nonetheless is that a restart and prune just solves it every time.
Googling around I found a lot of similar issues where most people just decided to increase disk space. I'm not keen on accepting this as the preferred solution, as it feels like kicking the can down the road.
Would you have any smart ideas on what to do instead of increasing disk space?
It seems like it is expected behavior, from Docker documentation you can read:
Docker takes a conservative approach to cleaning up unused objects
(often referred to as “garbage collection”), such as images,
containers, volumes, and networks: these objects are generally not
removed unless you explicitly ask Docker to do so. This can cause
Docker to use extra disk space. For each type of object, Docker
provides a prune command. In addition, you can use docker system prune to clean up multiple types of objects at once. This topic shows
how to use these prune commands.
So it seems like you have to clean it up manually using docker system/image/container prune. Other issue might be that those containers create too much logs and you might need to clean it up.
When I am trying to build the docker image I am getting out of disk space error and after investigating I find the following:
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 4G 3.8G 0 100% /
How do I fix this out of space error?
docker system prune
https://docs.docker.com/engine/reference/commandline/system_prune/
This will clean up all images, containers, networks, volumes not used. We generally try to clean up old images when creating a new one but you could also have this run as a scheduled task on your docker server every day.
use command - docker system prune -a
This will clean up total Reclaimable Size for Images, Network & Volume..... This will remove all images related reclaimable space which are not associated with any running container.....
Run docker system df command to view Reclaimable memory
In case there is some Reclaimable memory then if above command does not work in first go then run the same command twice then it should cleaned up....
I have been experiencing this behavior almost on daily basis.....
Planning to report this bug to Docker Community but before that want to reproduce this bug with new release to see if this has been fixed or not with latest one....
Open up the docker settings -> Resources -> Advanced and up the amount of Hard Drive space it can use under disk image size.
If you are using linux, then most probably docker is filling up the directory /var/lib/docker/containers, because it is writing container logs to <CONTAINER_ID>-json.log file under this directory. You can use the command cat /dev/null > <CONTAINER_ID>-json.log to clear this file or you can set the maximum log file size be editing /etc/sysconfig/docker. More information can be found in this RedHat documentation. In my case, I have created a crontab to clear the contents of the file every day at midnight. Hope this helps!
NB:
You can find the docker containers with ID using the following command
sudo docker ps --no-trunc
You can check the size of the file using the command
du -sh $(docker inspect --format='{{.LogPath}}' CONTAINER_ID_FOUND_IN_LAST_STEP)
Nothing works for me. I change the disk images max size in Docker Settings, and just after that it free huge size.
Going to leave this here since I couldn't find the answer.
Go to the Docker GUI -> Prefereces -> Reset -> Uninstall
Completely uninstall Docker.
Then install it fresh using this link
My docker was using 20GB of space when building an image, after fresh install, it uses 3-4GB max. Definitely helps!
Also, if you using a macbook, have look at ~/Library/Containers/docker*
This folder for me was 60 GB and was eating up all the space on my mac! Even though this may not be relevant to the question, I believe it is vital for me to leave this here.
I am on docker version 1.11.2. I am trying to docker save an image but i get
an error.
i did docker images to see the size of the image and the result is this
myimage 0.0.1-SNAPSHOT e0f04657b1e9 10 months ago 1.373 GB
The server I am on is low on space but it has 2.2 GB available but when I run docker save myimage:0.0.1-SNAPSHOT > img.tar i get
write /dev/stdout: no space left on device
I removed all exited containers and dangling volumes in hopes of making it work but nothing helped.
You have no enough space left on device. So free some more space or try gzip on the fly:
docker save myimage:0.0.1-SNAPSHOT | gzip > img.tar.gz
To restore it, docker automatically realizes that is gziped:
docker load < img.tar.gz
In such a situation where you can't free enough space locally you might want to use storage available over a network connection. A little bit more difficult to set up are NFS or Samba.
The easiest approach could be piping the output through netcat, but keep in mind that this is at least by default unencrypted.
But as long as your production server is that low on space you are vulnerable to a bunch of other problems.
Until you can provide more free space I wouldn't create files locally, zipped or not. You could bring important services down when you run out of free space.