I am attempting to send SIGSTOP, and then later, SIGKILL to a container. This line leads me to believe that it will behave as I expect: https://github.com/docker/docker/issues/5948#issuecomment-43684471
However, it is going ahead and actually removing the containers. The commands are:
docker kill -s STOP container
docker kill -s CONT container
(Equivalent through the dockerode API I am using, but I just went to command line when that wasn't working). Is there some missing options I'm missing?
I think you're actually looking for the commands docker pause and docker unpause. Using the STOP signal is likely to be error-prone and dependent on how the process handles the signal.
I guess what's happening in this case is that Docker thinks the process has terminated and stops the container (it shouldn't be removed however, you can restart it with docker start).
Related
Having an issue with Docker at the moment; I'm using it to run an image that launches an ipython notebook on startup. I'm looking to make some edits to ipython notebook itself, so I need to close it after launch.
However, hitting CTRL+C in the terminal just inputs "^C" as a string. There seems to be no real way of using CTRL+C to actually close the ipython notebook instance.
Would anyone have any clues as to what can cause this, or know of any solutions for it?
Most likely the container image you use is not handling process signals properly.
If you are authoring the image then change it as Roland Webers' answer suggests.
Otherwise try to run it with --init.
docker run -it --init ....
This fixes Ctrl+C for me.
Source: https://docs.docker.com/v17.09/engine/reference/run/#specify-an-init-process
The problem is that Ctrl-C sends a signal to the top-level process inside the container, but that process doesn't necessarily react as you would expect. The top-level process has ID 1 inside the container, which means that it doesn't get the default signal handlers that processes usually have. If the top-level process is a shell, then it can receive the signal through its own handler, but doesn't forward it to the command that is executed within the shell. Details are explained here. In both cases, the docker container acts as if it simply ignores Ctrl-C.
If you're building your own images, the solution is to run a minimal init process, such as tini or dumb-init, as the top-level process inside the container.
This post proposes CTRL-Z as a workaround for sending the process to background and then killing the process by its process id:
Cannot kill Python script with Ctrl-C
Possible problems:
The program catches ctrl-c and does nothing, very unlikely.
There are background processes that are not managed correctly. Only the main process receives the signal and sub-processes hang. Very likely what's happening.
Proposed Solution:
Check the programs documentation on how it's properly started and stopped. ctrl-c seems not to be the proper way.
Wrap the program with a docker-entrypoint.sh bash script that blocks the container process and is able to catch ctrl-c. This bash example should help: https://rimuhosting.com/knowledgebase/linux/misc/trapping-ctrl-c-in-bash
After catching ctrl-c invoke the proper shutdown method for ipython notebook.
From this post on the Docker message boards:
Open a new shell and execute
$ docker ps # get the id of the running container
$ docker stop <container> # kill it (gracefully)
This worked well for me. CTRL-Z, CTRL-\, etc. only came up as strings, but this killed the Docker container and returned the tab to terminal input.
#maybeg's answer already explains very well why this might be happening.
Regarding stopping the unresponsive container, another solution is to simply issue a docker stop <container-id> in another terminal. As opposed to CTRL-C, docker stop does not send a SIGINT but a SIGTERM signal, to which the process might react differently.
Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]
Stop a running container by sending SIGTERM and then SIGKILL after a grace period
If that fails, use docker kill <container-id> which sends a SIGKILL immediately.
I have have been trying to make it so that I am able to shutdown a docker container from inside. I have read that using tini is the best way to do this.
I have added init: true to my docker-compose.yml and I can see that docker-init is running as PID 1. However the only command that lets me shutdown the container from my shell script is using kill 1, but I want to gracefully shutdown my container so that it can do some cleanup.
I have tried using commands like kill -SIGINT 1 which results in the error kill: Illegal option -S
or kill -INT 1 and kill -2 1 which both seem to do nothing at all.
I can't seem to figure out the command that I can use. If there is an alternative to init that would also be an option.
The application inside the container doesn't need any special setup in order to shut down; it can just run its own shutdown sequence and exit, and when it does exit, the container will exit as well. If you're trying to do this from a debugging shell you launched with docker exec, you can just use docker stop to send SIGTERM and then SIGKILL. (...and reserve the docker exec shell for debugging; it should not be the primary way you interact with your container.)
If you need to send a container a non-default signal, docker kill has that option:
docker kill --signal SIGINT container_name
In terms of using kill(1) in a debugging shell, the man page for the underlying kill(2) function notes:
The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers. This is done to assure the system is not brought down accidentally.
It looks like tini collects and forwards a pretty broad range of signals, everything except SIGFPE, SIGILL, SIGSEGV, SIGBUS, SIGABRT, SIGTRAP, SIGSYS, and the uncatchable signals (notably SIGKILL). Since it does register a signal handler, kill -INT 1 should forward that signal on to the actual container process. (Its pid is probably 2, so kill -INT 2 should also tell the process to stop.)
here my snippet:
docker restart -t 5 waitforit_
then docker ps returns immediately :
status => run since 1s
How it is possible?
any hint would be great,
thanks
I believe docker restart is equivalent to docker stop; docker start. The -t option isn’t a hard wait. Rather, it says that if the process doesn’t stop on its own after receiving SIGTERM, then send it SIGQUIT (kill -9) after that many seconds.
If your process is well-behaved and exits promptly when it receives SIGTERM, then docker restart will in fact be pretty quick, regardless of whatever value you pass as -t.
I would like to have a RUnit service to supervise a set of containers launched by docker-compose tool, here's my the runit script:
In /etc/sv/app/run
#!/bin/bash
exec 2>&1
APP_HOME=/home/myapp
source $APP_HOME/env.sh
exec docker-compose -f $APP_HOME/docker-compose.yml up
Here's what I have then:
sv start app - launches the docker-compose thing just fine
sv stop app - stops docker-compose process itself but for unknown reason it leaves the containers running
Is there any chance to have the stop command to stop containers as well? I thought that is what docker-compose should do when it gets stopped by RUnit.
I'm not familiar with docker (yet) but I have familiarity with runit.
When you issue sv stop app you are actually telling runsvdir to signal the runsv for your docker launch to tear down the process. If you need something to signal the container to shut down, it won't happen because runsv will haul off and kill any child processes that are attached. You may wish to read up on ./finish scripts, which are tasked with cleaning up things.
Will docker stop fail if processes running inside the container fail to stop?? If i use docker kill, can unsaved data inside the container be preserved. Is docker stop time consuming compared to docker kill?? I want to do a shutdown of the container but without loosing any data(without high latency to complete kill or stop process).
Line reference:
docker stop: Stop a running container (send SIGTERM, and then SIGKILL
after grace period) [...] The main process inside the container will
receive SIGTERM, and after a grace period, SIGKILL. [emphasis mine]
docker kill: Kill a running container (send SIGKILL, or specified
signal) [...] The main process inside the container will be sent
SIGKILL, or any signal specified with option --signal. [emphasis mine]
You can get more info from this post: https://superuser.com/questions/756999/whats-the-difference-between-docker-stop-and-docker-kill
Docker stop:
When you issue a docker stop command a hardware signal is sent to the process inside of that container. In the case of docker stop we send a sig term message which is short for terminate signal its a message that's going to be received by the process telling it essentially to shut down on its own time.
SIGTERM is used any time that you want to stop a process inside of your container and shut the container down, and you want to give that process inside there a little bit of time to shut itself down and do a little bit of clean up.
A lot of different programming languages have the ability for you to listen for these signals inside of your code base, and as soon as you get that signal you could attempt to do a little bit of cleanup or maybe save some file or emit some message or something like that.
On the other hand the docker kill command issue is a sig kill or kills signal to the primary running process inside the container, so kill it essentially means you have to shut down right now and you do not get to do any additional work.
So ideally we always stop a container with the docker stop command in order to get the running process inside of it a little bit of time to shut itself down, otherwise if it feels like the container has locked up and it's not responding to the docker stop command then we could issue docker kill instead.
One kind of little oddity or interesting thing about docker stop, when issue docker stop to a container and if the container dose not automatically stop in 10 seconds then docker is going to automatically fall back to issuing the docker kill command.
So essentially at docker stop is us being nice but it's only got 10 seconds to actually shut down.
A good example could be ping command.
sudo docker run busybox ping google.com
now if you want to stop the container if you use docker stop container_id, you will see it takes 10 seconds before getting shut down because ping command dose not properly respond to a SIGTERM message. In other words the ping command doesn't really have the ability to say oh yeah I understand you want me to shut down.
So after we waited those 10 seconds eventually the kill signal was sent to it telling it hey ping you are done and shut yourself down.
But if you use docker kill container_id you are going to see that's it instantly dead.
You should use docker stop since it stops the container gracefully - like shutting down your laptop, instead of killing them - like forcibly turn off the laptop from it's battery.
But, Docker will force shut down (kill the processes) by the time it takes 10 seconds to stop them gracefully.
docker stop will send SIGTERM (terminate signal) to the process and docker will have 10 seconds to clean up like saving files or emitting some messages.
Use docker kill when container is locked up, if it is not responding.