I have been running a nvidia docker image since 13 days and it used to restart without any problems using docker start -i <containerid> command. But, today while I was downloading pytorch inside the container, download got stuck at 5% and gave no response for a while.
I couldn't exit the container either by ctrl+d or ctrl+c. So, I exited the terminal and in new terminal I ran this docker start -i <containerid> again. But ever since this particular container is not responding to any command. Be it start/restart/exec/commit ...nothing! any command with this container ID or name is just non-responsive and had to exit out of it only after ctrl+c
I cannot restart the docker service since it will kill all running docker containers.
Cannot even stop the container using this docker container stop <containerid>
Please help.
You can make use of docker RestartPolicy:
docker update --restart=always <container>
while mindful of caveats on the docker version you running.
or explore an answer by #Yale Huang from a similar question: How to add a restart policy to a container that was already created
I had to restart docker process to revive my container. There was nothing else I could do to solve it. used sudo service docker restart and then revived my container using docker run. I will try to build the dockerfile out of it in order to avoid future mishaps.
I installed docker CE version on an ubuntu 18.04 server. Then, I installed a new jenkins container and everything worked well for two weeks.
After two weeks, for some reason, when I run docker ps I receive an empty list although the jenkins container is running and functioning (it worked in the past). I also tried to run docker ps -a, docker images and again, everything is empty. Also tried restarting the server and still every time the list is empty.
I then uninstalled and reinstalled docker and right after the installation, when running docker ps I see the containers....I thought that the problem was fixed, but today it happened to me again and I still see an empty list when running docker ps. Any ideas ? it will be much appreciated.
Run the command sudo service docker stop
After that find the process dockerd
ps aux | grep "dockerd"
and kill the one by
sudo kill {paste_dockerd_pid_here} -9
Start docker service
sudo service docker start
I have a fresh install using boot2docker. (DockerToolbox was giving me the same error. After uninstalling DockerToolbox, I deleted ~/.docker and searched my whole filesystem for anything starting with "docker" and found no other configuration files where things might be hiding.)
This is the second command I did, after docker run hello-world:
bash-3.2$ docker run -it ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
6071b4945dcf: Verifying Checksum
5bff21ba5409: Pulling fs layer
e5855facec0b: Download complete
8251da35e7a7: Download complete
8251da35e7a7: Layer already being pulled by another client. Waiting.
And I'm stuck here indefinitely.
I promise I only have 1 docker process running. I just want to get past this. If it means nuking whatever cache is in place and doing a manual download, that's okay. I just want to stop being stuck here for hours.
You need to restart Docker service or just restart the OS.
Also in this issue #avramirez pointed out that you can do this using boot2docker:
boot2docker stop
boot2docker up
docker pull <repo>
quote from issue#15603 message:
Hello all! I believe this should by fixed on master by #15489 (and
will soon ship in a few weeks as part of Docker 1.9.0).
This is a bug in Docker.
Try out the following in order (Trying to avoid restarting the OS):
ps aux | grep docker-compose and find the PID of docker-compose processes running.
Kill them using kill <pid>
Restart Docker using service docker restart (linux)
2nd method should ideally solve the problem, if not, Restart the OS.
Hopefully, this issue will be solved in version 1.9
When trying to stop or restart a docker container I'm getting the following error message:
$ docker restart 5ba0a86f36ea
Error response from daemon: Cannot restart container 5ba0a86f36ea: [2] Container does not exist: container destroyed
Error: failed to restart containers: [5ba0a86f36ea]
But when I run
$ docker logs -f 5ba0a86f36ea
I can see the logs, so obviously the container does exist. Any ideas?
Edit:
sorry, I forgot to mention this:
When I run docker ps -a I see the container as up and running. However the application inside it is malfunctioning so I want to restart it, or just get a fresh version of that application online. But when I can't stop and remove the container, I also can't get a new application up and running, which would be listening to the same port.
I couldn't locate boot2docker in my machine. So, I came up with something that worked for me.
$ sudo systemctl restart docker.socket docker.service
$ docker rm -f <container id>
Check if it helps you as well.
All the docker:
start | restart | stop | rm --force | kill commands
may not work if the container is stuck. You can always restart the docker daemon. However, if you have other containers running, that may not be the option. What you can do is:
ps aux | grep <<container id>> | awk '{print $1 $2}'
The output contains:
<<user>><<process id>>
Then kill the process associated with the container like so:
sudo kill -9 <<process id from above command>>
That will kill the container and you can start a new container with the right image.
That looks like docker/docker/issues/12738, seen with docker 1.6 or 1.7:
Some container fail to stop properly, and the restart
We are seeing this issue a lot in our users hosts when they upgraded from 1.5.0 to 1.6.0.
After the upgrade, some containers cannot be stopped (giving 500 Server Error: Internal Server Error ("Cannot stop container xxxxx: [2] Container does not exist: container destroyed")) or forced destroyed (giving 500 Server Error: Internal Server Error ("Could not kill running container, cannot remove - [2] Container does not exist: container destroyed")).
The processes are still running on the host.
Sometimes, it works after restarting the docker daemon.
There are some workarounds:
I've tried all remote API calls for that unkillable container and here are results:
json, stats, changes, top, logs returned valid responses
stop, pause, wait, kill reported 404 (!)
After I finished with remote API, I double-checked docker ps (the container was still there), but then I retried docker kill and it worked! The container got killed and I could remove it.
Or:
What worked was to restart boot2docker on my host. Then docker rm -f
$ boot2docker stop
$ boot2docker start
$ docker rm -f 1f061139ba04
Worth knowing:
If you are running an ENTRYPOINT script ... the script will work with the shebang
#!/bin/bash -x
But will stop the container from stopping with
#!/bin/bash -xe
Enjoy
sudo aa-remove-unknown
This is what worked for me.
Check if there is any zombie process using "top" command.
docker ps | grep <<container name>>
Get the container id.
ps -ef | grep <<container id>>
ps -ef|grep defunct | grep java
And kill the container by Parent PID .
For anyone on a Mac who has Docker Desktop installed. I was able to just click the tray icon and say Restart Docker. Once it restarted was able to delete the containers.
If you're on a Mac and try this via Terminal: Use killall Docker to quit Docker.
Restart it in the Applications folder or with open /Applications/Docker.app.
Subsequently you can run a docker rm <id> for the concerned container.
I had the same problem on a windows host machine and none of the other options here worked for me. I ended up just needing to delete the physical container folder, which was located here:
C:\ProgramData\Docker\containers\[container guid]
I had stopped the docker service first just to be safe and when I restarted it, the broken containers were now gone and I was able to create new ones. I suspect the same will work on a linux host machine, but I do not know where the container folders are kept on that OS.
Ubuntu
Stop the container by using its system process ID.
Get the main process ID using:
docker inspect -f '{{.State.Pid}}' container-id
This will return an id as ´25430´.
Kill this with the command
sudo kill -9 25430
in my case, i couldn't delete container created with nomad jobs,
there's no output for the docker logs <ContainerID> and, in general, it looks like frozen.
until now the solution is: sudo service docker restart, may someone suggest better one?
i forgot that i had made the container start as a system service.
so if i stopped or killed the container, the service would bring it back.
if you are using systemctl, you can list all the running services with systemctl | grep running and find the name of the service.
then use
sudo systemctl disable <your_service_name> to stop it.
If you're on Ubuntu, make sure docker-compose isn't installed as a snap. This will cause all kinds of random issues, including the above.
Remove the snap:
sudo snap remove docker-compose
And install manually from the compose repository:
Docker compose installation instruction
Sometimes this is caused by problem of the docker daemon.
I solved the problem by restarting the docker service.
On Linux:
systemctl restart docker
In my case, docker rm $(docker ps -aq) works for me.
I'm trying to push a container that was interrupted previously by a network drop out. But I get this error:
Error: push rimian/ruby-node-npm is already in progress
But when I run docker ps I don't see anything running.
What shall I do?
Restart the docker service on which you are running the docker.
With ubuntu:
sudo service docker restart
Just wait.
I had this once, too, and the problem is that the push is still running in the background, hence you can't do another one.
So just wait, and the problem will disappear automatically after some time.
The fact that you do not see anything running with docker ps is that this command only shows docker containers, not internal docker processes. And pushing an image is not run by a container.