I'm using docker and I got an error "No space on the drive" when I tried to use the docker. So, I searched online and found that I had to remove the the unused images from the docker. So, I used the command
docker system prune --all
It deleted about ~30GB of the images. But When I run the docker again I got the same "No space" issue and when I checked the disk image size it showed me that 64 GB used even though I deleted the unused containers.
I had to increase the size of the drive every time when I use docker.
docker images output
REPOSITOY TAG IMAGE ID CREATED SIZE
postgres latest aal783da97 2 days ago 6.32GB
redis latest aal783da98 2 days ago 6.32GB
mysql latest aal783da99 2 days ago 6.32GB
airflow_worker latest aal783da91 2 days ago 6.32GB
airflow_scheduler latest aal783da92 2 days ago 6.32GB
How do I delete all the 64.1 GB?
Related
I have a bunch of docker images I want to get rid of and it would be convenient if I could remove them by specifying the repository name as it appears when I run docker images. For example if docker images returns:
REPOSITORY TAG IMAGE ID CREATED SIZE
ui_test 191127_manual 41a7ca9824d6 24 hours ago 1.42GB
ui git-24fa8d1a cdd254eff918 24 hours ago 1.44GB
ui git-31a4b052 9b4740060a62 25 hours ago 1.45GB
ui_test 191122_manual ba9cb04ce2d8 6 days ago 1.39GB
ui git-68110e426 f26ef80abc25 6 days ago 1.38GB
what command would I use to remove all of the ui_test images?
You can pass image IDs you want to delete to docker rmi:
docker rmi $(docker images -q 'ui_test')
From the docs:
The docker images command takes an optional [REPOSITORY[:TAG]] argument that restricts the list to images that match the argument. If you specify REPOSITORYbut no TAG, the docker images command lists all images in the given repository.
I want to delete a Docker daemon; because it take more space on my pc
I did insert a code to install php:5.3-apache
I used this command docker build -t my-php-app .
I get this error
Sending build context to Docker daemon 7.039GB
Error response from daemon: unexpected error reading Dockerfile: read /var/lib/docker/tmp/docker-builder079515785/dockerfile: is a directory
so I want to delete that daemon because It take big space on my pc; I did tried 3 times to install it and every-time I get the same error so I have now more than 21GB that I want to clean.
docker images give me :
orsolin/docker-php-5.3-apache latest d8e72369c6e9 2 years ago 533MB
but space on my computer has been decreased from 30GB to 8GB
I don't see anything with 7GB of size
REPOSITORY TAG IMAGE ID CREATED SIZE
abv_web latest 2194cdd678e3 45 hours ago 5.09GB
<none> <none> 58b330f4aa2f 46 hours ago 623MB
<none> <none> c4b5c889111d 46 hours ago 623MB
<none> <none> cfe4161b5af4 46 hours ago 623MB
<none> <none> f37f2a95529a 46 hours ago 623MB
httpd latest 19459a872194 4 days ago 154MB
phpmyadmin/phpmyadmin latest d8d2c1fd1eb9 9 days ago 458MB
phpmyadmin/phpmyadmin edge 4b557b055a8c 9 days ago 458MB
pweb_joomla latest 22c6d70d575e 10 days ago 1.01GB
fweb_joomla latest 0e36548560af 11 days ago 2.47GB
joomla apache 73acf8852f1b 13 days ago 461MB
mysql 5.6 732765f8c7d2 4 weeks ago 257MB
joomla 3.9.5-apache 398227376f4a 4 months ago 415MB
alterway/php 5.3-apache 87058120bc90 8 months ago 623MB
orsolin/docker-php-5.3-apache latest d8e72369c6e9 2 years ago 533MB
airinuit_mysql latest fa73519d1891 2 years ago 304MB
vsamov/mysql-5.1.73 latest fa73519d1891 2 years ago 304MB
Docker internally runs on a client/server architecture. In particular, when you run docker build, it creates a tar file of the directory you specify, sends that tar file across a socket to the Docker daemon, and unpacks it there. (True even on a totally local system.)
If something goes wrong, that content can back up in /var/lib/docker/tmp. Usually you shouldn’t be poking around in /var/lib/docker at all, but cleaning out this directory should be safe. Consider stopping the daemon while you do it. Stopping the daemon, deleting all of /var/lib/docker, and restarting the daemon should also help (you will need to re-docker build and/or docker pull images and re-run containers, and this will lose anything in named volumes you haven’t backed up).
The 7.039 GB build context is very large. In addition to problems like this it will just result in the docker build step being very slow; if all of that content gets COPYEd into the image you will probably hit similar problems trying to docker push the built image. You can try cleaning up things like log files or heap dumps if those are left lying around; adding things like your .git directory or a local vendor tree to a .dockerignore file can also help reduce the size of the context.
You don't want to delete the daemon, the daemon is actually the piece of software running on your host machine that able you to run docker commands.
If you want to remove existing images, volumes, containers, networks, build caches, you can dig into the command
docker system prune
Be careful with it, it will delete a lot things that you possibly still might use.
If you want to identify what does takes space in your host, prior to rennung any clean up, you can also run
$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 5 2 16.43 MB 11.63 MB (70%)
Containers 2 0 212 B 212 B (100%)
Local Volumes 2 1 36 B 0 B (0%)
Documentation about it: https://docs.docker.com/engine/reference/commandline/system_df/
That could reclaim lots of space on your host machine, and will even give you a sum up of the amount of space it reclaims:
$ docker system prune
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all build cache
Are you sure you want to continue? [y/N] y
Deleted Containers:
f44f9b81948b3919590d5f79a680d8378f1139b41952e219830a33027c80c867
792776e68ac9d75bce4092bc1b5cc17b779bc926ab04f4185aec9bf1c0d4641f
Deleted Networks:
network1
network2
Deleted Images:
untagged: hello-world#sha256:f3b3b28a45160805bb16542c9531888519430e9e6d6ffc09d72261b0d26ff74f
deleted: sha256:1815c82652c03bfd8644afda26fb184f2ed891d921b20a0703b46768f9755c57
deleted: sha256:45761469c965421a92a69cc50e92c01e0cfa94fe026cdd1233445ea00e96289a
Total reclaimed space: 1.84kB
The most radical form of it being:
$ docker system prune --all --volumes
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all volumes not used by at least one container
- all images without at least one container associated to them
- all build cache
Are you sure you want to continue? [y/N] y
Deleted Containers:
0998aa37185a1a7036b0e12cf1ac1b6442dcfa30a5c9650a42ed5010046f195b
73958bfb884fa81fa4cc6baf61055667e940ea2357b4036acbbe25a60f442a4d
Deleted Networks:
my-network-a
my-network-b
Deleted Volumes:
named-vol
Deleted Images:
untagged: my-curl:latest
deleted: sha256:7d88582121f2a29031d92017754d62a0d1a215c97e8f0106c586546e7404447d
deleted: sha256:dd14a93d83593d4024152f85d7c63f76aaa4e73e228377ba1d130ef5149f4d8b
untagged: alpine:3.3
deleted: sha256:695f3d04125db3266d4ab7bbb3c6b23aa4293923e762aa2562c54f49a28f009f
untagged: alpine:latest
deleted: sha256:ee4603260daafe1a8c2f3b78fd760922918ab2441cbb2853ed5c439e59c52f96
deleted: sha256:9007f5987db353ec398a223bc5a135c5a9601798ba20a1abba537ea2f8ac765f
deleted: sha256:71fa90c8f04769c9721459d5aa0936db640b92c8c91c9b589b54abd412d120ab
deleted: sha256:bb1c3357b3c30ece26e6604aea7d2ec0ace4166ff34c3616701279c22444c0f3
untagged: my-jq:latest
deleted: sha256:6e66d724542af9bc4c4abf4a909791d7260b6d0110d8e220708b09e4ee1322e1
deleted: sha256:07b3fa89d4b17009eb3988dfc592c7d30ab3ba52d2007832dffcf6d40e3eda7f
deleted: sha256:3a88a5c81eb5c283e72db2dbc6d65cbfd8e80b6c89bb6e714cfaaa0eed99c548
Total reclaimed space: 13.5 MB
Documentation could be found here: https://docs.docker.com/engine/reference/commandline/system_prune/
I created two new images anubh_custom_build_image/ubuntu_bionic:version1 & ubuntu_bionic_mldev:version1 from the base image ubuntu:bionic. The purpose of creating the custom-built ubuntu-docker images was to use Linux system on windows platform. I have faced many issues in past one such is installing a new version of tensorflow library ! pip install -q tf-nightly, I can't find a substitute of ! to run this command on windows cmd-prompt/PowerShell. Moreover, I want to invest more time on my codebase rather than fixing the issues on different OS. So, I pull the latest Ubuntu image from docker, installed a bunch of libraries for my usage and committed using docker commit command:
docker commit 503130713dff ubuntu_bionic_MLdev:version1
I can see the images using :
PS C:\Users\anubh> docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu_bionic_mldev version1 e7d1b154b69f 21 hours ago 9.33GB
anubh_custom_build_image/ubuntu_bionic version1 3c98f8954731 22 hours ago 9.33GB
tensorflow/tensorflow latest 2c8d1fd8bde4 2 days ago 1.25GB
ubuntu bionic 735f80812f90 2 weeks ago 83.5MB
ubuntu latest 735f80812f90 2 weeks ago 83.5MB
floydhub/dl-docker cpu 0b9fc622f1b7 2 years ago 2.87GB
When I tried to spin-up the containers using these images, The following command ran without any error.
PS C:\Users\anubh> docker run anubh_custom_build_image/ubuntu_bionic:version1
PS C:\Users\anubh> docker run ubuntu_bionic_mldev:version1
EDIT:
The issue is that run command is executing but the containers aren't spinning up for the above two images. I apologize for attaching the wrong error message in the first post, I edited it now. The below two containers were spined-up using docker run -it -p 8888:8888 tensorflow/tensorflow & docker run ubuntu:bionic commands.
PS C:\Users\anubh> docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
94d59b217b70 tensorflow/tensorflow "/run_jupyter.sh --a…" 21 hours ago Up 21 hours 6006/tcp, 8888/tcp boring_clarke
503130713dff ubuntu:bionic "bash" 38 hours ago Up 38 hours awesome_bardeen
Could anyone suggest what I am missing for running these images anubh_custom_build_image/ubuntu_bionic:version1 & ubuntu_bionic_mldev:version1 from the base image ubuntu:bionic as
containers?
Also, I can't find the location of any of these images on my disk.
Could anyone also suggest where to look for inside Windows OS?
NOTE: I will write a dockerfile in future to build a custom image, but for now, I want to use the commit command to create new image & use them.
Your docker run command doesn't work because you don't have the :version1 tag at the end of it. (Your question claims you do, but the actual errors you quote don't have it.)
However: if this was anything more than a simple typo, the community would probably be unable to help you, because you have no documentation about what went into your image. (And if not "the community", then "you, six months later, when you discover your image has a critical security vulnerability".) Learn how the Dockerfile system works and use docker build to build your custom images. Best-practice workflows tend to not use docker commit at all.
The output of sudo docker images shows images that were created ages ago, I don't see any use for them, sudo docker system prune does not remove them either.
How can I easily delete old images? Do I have to write a script to conditionally delete based on date created within the output of docker images?
I don't think you can filter images by created date, but you can use another image as a reference of time.
The flag (-f or--filter) with before shows only images created before the image with given id or reference.
For example, having these images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
image1 latest eeae25ada2aa 4 minutes ago 188.3 MB
image2 latest dea752e4e117 9 minutes ago 188.3 MB
image3 latest 511136ea3c5a 25 minutes ago 188.3 MB
Filtering with before (image1) would give:
$ docker images --filter "before=eeae25ada2aa"
REPOSITORY TAG IMAGE ID CREATED SIZE
image2 latest dea752e4e117 9 minutes ago 188.3 MB
image3 latest 511136ea3c5a 25 minutes ago 188.3 MB
To Remove images before image1(eeae25ada2aa)
$ docker rmi $(docker images --filter "since=511136ea3c5a" -q)
dea752e4e117
511136ea3c5a
Asking this question out of curiosity:
Is it possible to map the docker images and Dockerfiles if they are available in the same machine/server ?
If not is there any way to tag an image which can refer to the Dockerfile ...something of this sort ?
My requirement is know how the images are formed from the docker files if I have images of following sort:
root#labadmin-VirtualBox:~/RAGHU/DOCKER# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
env_path 1.1 01502209b2fc 13 minutes ago 187.9 MB
env_path 1.0 3610243625cd 14 minutes ago 187.9 MB
env_path 1.2 3610243625cd 14 minutes ago 187.9 MB
env_path 2.0 c6d1eb2cff1b 14 minutes ago 187.9 MB
An approach to achieve what you requested, is using the LABEL field in your Dockerfile to save the path or an id of your dockerfiles. Then, you could use the docker inspect command in order to retrieve this information.
You can find more information about labels and how to manage them in this link.