I noticed that Docker appears to be using a large amount of disk space. I can see the directory /Users/me/.docker/machine/machines/default is 27.4GB
I recently cleaned up all of the images and containers I wasn't using. Now, when I run docker ps I see that no images are running.
~ $ docker ps
I can also see that I have 2 containers available.
~ $ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
42332s42d3 cds-abm "/bin/bash" 2 weeks ago Exited (130) 2 weeks ago evil_shockley
9ssd64ee41 tes-abm "/bin/bash" 2 weeks ago Exited (130) 2 weeks ago lonely_brattain
I can then see I have 3 images.
~ $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ghr/get latest 6708fffdd4dfa 2 weeks ago 2.428 GB
<none> <none> 96c5974ddse18 2 weeks ago 2.428 GB
fdbh/ere latest bf1da53766b1 2 weeks ago 2.225 GB
How can these be taking up nearly 30GB?
It is because you are not removing the volumes created by the containers when you stop the container. In the future, use -v when you remove the containers.
docker rm -v <container-id>
Regrading cleaning up the space, you have to ssh into docker-machine and remove all volumes created. To do so,
docker-machine ssh default
sudo -i # You may not get permission to enter in to volumes directory
cd /var/lib/docker/volumes
rm -rf *
Make sure none of your containers are currently running. Also, make sure that you don't need any of these volumes for later use (like DB containers).
Related
I run docker ps and it shows that 5 containers that have been running for three weeks.
I then run docker-compose down but when I run docker ps again, they are all still running.
I have tried the following command but none seems to work
kill
stop
down --rmi local
rm
down
How can I stop these? I tried just bringing up my new docker-compose.yml and ignoring the olde one but I get:
ERROR: for apache Cannot create container for service apache: Conflict. The container name "/apache" is already in use by container "70c570d60e1248292f279a37634fd8b4ce7e2535d2bfa14b2c6e4089652c0152". You have to remove (or rename) that container to be able to reuse that name.
What to try to stop the old container?
You can list containers:
(base) docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c788727f0f7b postgres:14 "docker-entrypoint.s…" 7 days ago Up 7 days 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp dev-db
88e8ddcb7d4e redis "docker-entrypoint.s…" 7 days ago Up 7 days 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp beautiful_neumann
Delete container:
(base) docker rm -f c788727f0f7b # container_id
c788727f0f7b
List containers:
(base) docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
88e8ddcb7d4e redis "docker-entrypoint.s…" 7 days ago Up 7 days 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp beautiful_neumann
As you can see the container got stopped(c788727f0f7b).
You can list stopped containers using:
docker container ls -f 'status=exited'
I made a goof while trying to rename an image by following the steps on this page that say to create a tag then delete the original
Docker how to change repository name or rename image?
Now when I list the images it doesn't show up anymore. However, when I list the containers the image still shows up.
PS C:\Users\Grail> docker images -a
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 157be28c0fe3 7 days ago 668MB
fedora latest a368cbcfa678 2 months ago 183MB
PS C:\Users\Grail> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS
1ea5ffd50852 157be28c0fe3 "/bin/bash" 7 hours ago Exited (0) 7 hours ago
fb81990e756c 0d120b6ccaa8 "/bin/bash" 10 hours ago Exited (0) 24 minutes ago
081641b3e600 a368cbcfa678 "/bin/bash" 11 hours ago Exited (0) 31 minutes ago
Not only that, the image (0d120b6ccaa8) still shows up in my Docker Dashboard (running on Windows) and I can start/stop it without any problems.
Clearly the image still exists. Can I restore it such that I can see it when I list the images?
Can it be restored from the container?
If it's in a weird state/unrecoverable, how do I actually delete it so it's not taking up space?
Update:
Thanks to #prashanna I went down a path where I exported the container and imported to get the image:
docker export -o mycontainer.tar fb81990e756c
docker import mycontainer.tar
**docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]**
is for creating a new image from a container, meaning when you update or add new config or install new software, thus creating a new template images.
ref:docker commit
I am trying to install additional python packages in AWX container awx_tasks so that the changes could enable the ansible modules like snow, ec2_elb_facts run (which have pre-requisites as Python modules). I have made the changes in the container using:
# docker exec -it 80ab6bf562a9 bash
where 80ab6bf562a9 is the container id for awx_task container.
and then installed the required packages inside the custom virtual environment (as mentioned in the AWX documentation). Post this, i have made the changes permanent by creating a new image with the container changes using:
# docker commit 80ab6bf562a9 ansible/awx_task:latest
Post this, ran the following command to map the new container with the newly created image with container changes.
# docker run --name awx_task -d 5290f9b3268c
Following are the containers post the above changes. Here, the newly created container which was mapped with the new image with changes in existing container is 968fb2a7da2f.
# docker container ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
968fb2a7da2f 5290f9b3268c "/tini -- /bin/sh -c…" 2 days ago Exited (143) 2 days ago awx_task
80ab6bf562a9 535bb2b8e1f3 "/tini -- /bin/sh -c…" 3 weeks ago Up 2 days 8052/tcp awx_task_OLD
aea2551951d5 b7c261b76010 "/tini -- /bin/sh -c…" 3 weeks ago Up 2 days 0.0.0.0:80->8052/tcp awx_web
e789a4a82a9e memcached:alpine "docker-entrypoint.s…" 3 weeks ago Up 2 days 11211/tcp memcached
a8c74584255c ansible/awx_rabbitmq:3.7.4 "docker-entrypoint.s…" 3 weeks ago Up 2 days 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq
25f6f6ca7766 postgres:9.6 "docker-entrypoint.s…" 3 weeks ago Up 2 days 5432/tcp postgres
Following are my images post above changes. Here, the newly created image (with changes) is 5290f9b3268c.
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ansible/awx_task latest 5290f9b3268c 2 days ago 1.48GB
postgres 9.6 106bdfb062df 8 weeks ago 235MB
ansible/awx_task <none> 535bb2b8e1f3 8 weeks ago 1.07GB
ansible/awx_web <none> b7c261b76010 8 weeks ago 1.04GB
hello-world latest 2cb0d9787c4d 2 months ago 1.85kB
memcached alpine b40e8fa7e3e5 2 months ago 8.69MB
ansible/awx_rabbitmq 3.7.4 e08fe791079e 6 months ago 85.6MB
The new container is properly mapped with the new image (which has got the changes i wanted). The issue now is that when i stop the old container and start the new container AWX doesn't work. I can just view the UI, if i run any tasks like executing templates, it just freezes. It appears like the new container/images are not talking with the other containers like awx_rabbitmq, postgres etc. I have been reading multiple posts regarding this however, i couldn't find any single post which highlights anything regarding this.
I basically want the changes in the awx_task container to work so that i could achieve the goal of making the custom modules work. Could anyone suggest what can be done so that the new awx_task container could take the role of the older awx_task and AWX could work normally?
Since i found the way to do this, i will share the steps to make the required changes.
The python package versions can be controlled from the requirements directory, AWX Task and AWX Web Images related changes can be applied in the Dockerfile.j2 in the roles directory. Once the required changes are applied, we can run the setup using ansible-playbook install.yml -i inventory.
You should use the install.yml to restart the awx_task container, since it ensures the right environment variables are set, the right volumes are mapped, etc. Same command as you've used to install AWX:
ansible-playbook install.yml -i inventory.
See here for a full list of arguments that are used.
I have 3 containers, but I have a lot of images as you can see in the image.
⚡️ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
sscportalsmb_smb-portal latest ad0854c799f6 12 days ago 17.6MB
bheng_web latest d5a0ea011c0a 2 weeks ago 182MB
<none> <none> 957c22ababec 2 weeks ago 182MB
docker_web latest 70b443ed0495 2 weeks ago 182MB
bheng_app latest 509d58a68224 2 weeks ago 756MB
docker_app latest 509d58a68224 2 weeks ago 756MB
mysql 5.6 96dc914914f5 2 weeks ago 299MB
sscportalapi_ssc-portal-api latest e8295f9cb5f1 4 weeks ago 160MB
sscportaladmin_admin-portal latest fd141ceba4d6 4 weeks ago 17.7MB
mysql latest 5fac85ee2c68 5 weeks ago 408MB
redis latest 1fb7b6c8c0d0 6 weeks ago 107MB
alpine 3.6 76da55c8019d 2 months ago 3.97MB
keymetrics/pm2-docker-alpine 6 4a09bfc067d6 4 months ago 75.3MB
andrewmclagan/nginx-hhvm latest ec6cc741eb0e 7 months ago 580MB
nginx 1.10 0346349a1a64 7 months ago 182MB
tutum/haproxy latest 33bc771bec1e 17 months ago 232MB
php 7.0.4-fpm 81d7a2fdc6dc 20 months ago 494MB
I am not sure if this is normal.
How do I check for images that I don't use and get rid of them accordingly?
Above answers help us find and remove the dangling images,, but not unused.
So to fetch all the unused docker images on the machine
Fetch all the images belonging to the running containers(which are not stopped or exited)
Fetch all the images on the machine
Then filter the images in step 1 from step 2
Below is the basic shell script which could help do that
runningImages=$(docker ps --format {{.Image}})
docker images --format {{.Repository}}:{{.Tag}} | grep -v "$runningImages"
Just be sure before removing unused images(not the dangling ones) just list them and then decide which one to delete manually.
You can find unused images using the command:
docker images -f dangling=true
and just a list of their IDs:
docker images -q -f dangling=true
In case you want to delete them:
docker rmi $(docker images -q -f dangling=true)
Docker added special commands for this not so long ago:
docker image prune -> https://docs.docker.com/engine/reference/commandline/image_prune/ for removing unused images and docker container prune -> https://docs.docker.com/engine/reference/commandline/container_prune/ for stopped containers.
Comparing the result of docker images and of docker ps you should be able to understand what images are used in a running container. The second column of the output of docker ps is indeed the image used.
On my remote server, some developers run the same docker images named "my_account/analysis". So, once detached from the docker process, it is struggling to know which is my own process.
The result of docker ps is like this:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6968e76b3746 my_account/analysis "bash" 44 hours ago Up 44 hours 6023/tcp, 6073/tcp, 6800/tcp, 8118/tcp, 8888/tcp, 9050/tcp, 0.0.0.0:8887->8887/tcp modest_jones
42d970206a29 my_account/analysis "bash" 7 days ago Up 7 days 6023/tcp, 6073/tcp, 6800/tcp, 8118/tcp, 8888/tcp, 9050/tcp, 0.0.0.0:32771->8885/tcp gallant_chandrasekhar
ac9f804b7fe0 my_account/analysis "bash" 11 days ago Up 11 days 6023/tcp, 6073/tcp, 6800/tcp, 8118/tcp, 8888/tcp, 9050/tcp, 0.0.0.0:8798->8798/tcp suspicious_mayer
e8e260aab4fb my_account/analysis "bash" 12 days ago Up 12 days 6023/tcp, 6073/tcp, 6800/tcp, 8118/tcp, 8888/tcp, 9050/tcp, 0.0.0.0:32770->8885/tcp nostalgic_euler
In this case, because I remember that I ran docker around 2 days ago, I attach my container by docker attach 6968e. However, usually we forgot this.
What is the best practice to detect the container ID of mine under the situation that there are a lot of containers with the same Image name?
The simple way is to name the containers
docker run --name my-special-container my_account/analysis
docker attach my-special-container
You can store the container ID in a file when it launches
docker run --cidfile ~/my-special-container my_account/analysis
docker attach $(cat ~/my-special-container)
You can add more detailed metadata with object labels, but they are not as easily accessible as names
docker run --label com.rkjt50r983.tag=special my_account/analysis
docker ps --filter 'label=com.rkjt50r983.tag=special'