I have a docker container that internally starts a server. (I don't own this. I am just reusing it)
Once the server starts, I am running some curl commands that hit this server.
I am running the above steps in a script. Here's the issue:
the docker container starts but internally I think it is taking some time to actually start the server in it.
Before that server is up and running, it looks like the curl commands start executing and give an error that server could not be found. If I manually run this a few seconds later, it works fine though.
Please let me know if there is a way to solve this. I don't think using entry point or CMD will work for similar reasons.
Also, if that matters, the server I am using is kong.
thanks, Om.
The general answer to this is to perform some sort of health check; once you've verified that the server is healthy you can start making live requests to it. As you've noticed, the container existing or the server process running on its own isn't enough to guarantee that the container can handle requests.
A typical approach to this is to make some request to the server that will fail until the server is ready. You don't need to modify the container to do this. In some environments like Kubernetes, you can specify health checks or probes as part of the deployment configuration, but for a simple shell script, you can just run curl in a loop:
docker run -p 8080:8080 -d ...
RUNNING=false
for i in $(seq 30); do
# Try GET / and see if it succeeds
if curl -s http://localhost:8080/
then
echo Server is running
RUNNING=true
break
else
echo Server not running, waiting
sleep 1
fi
done
if [ "$RUNNING" = false ]; then
echo Server did not start within 30s
# docker stop ... && docker rm ...
exit 1
fi
If you just need to know the port is up, this simple script is very handy:
https://github.com/vishnubob/wait-for-it
Related
I have a web application in a docker container, and it processes requests via HTTP. But, when there are too many requests app stops working. I am busy with other tasks, so don't really have time to fix it. When it crashes, the container is still running, but the app responds with a 500 error. Are there any ways to track it and restart docker automatically, because I don't have an option to check it all the time?
I suggest you:
Create the container with the restart policy set to always or unless-stopped or on-failure.
Instrument Docker Health Check like HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://localhost/ || exit 1.
How would you run a daemon or background process in Docker? I've seen some suggestions, like this answer that launches supervisor from CMD.
However, I'm trying to test a server configuration tool that connects via SSH. So I need to launch the SSH daemon in the background, and then run my tool.sh to test connecting via SSH to its own container. I need to monitor my tool's output in order to verify it's working. What's the best way to accomplish this?
Is there any way to make a RUN command run in the background, like RUN /usr/sbin/sshd -D & or would I have to have some wrapper script launched from CMD that does something like this?
#!/bin/bash
/usr/sbin/sshd -D
tool.sh
You can run a daemon inside a docker container the same as you would on a bare metal linux machine. The only hard part is getting it to start without the nice runlevel scripts to help.
How about this:
#!/bin/sh
function run_script() {
ssh_pids=0
while [ ${ssh_pids} -lt 1 ]; do
sleep 5
ssh_pids=$(pgrep sshd | wc -l)
done
test.sh
}
run_script &
sshd -D > /dev/null 2>&1
I've used this trick before to do what you describe, and it's worked OK. It will just background the call to run_script and proceed to start SSHD in non-daemon mode, piping it to /dev/null. Meanwhile, run_script polls for sshd; when it finds it, it quits polling and runs test.sh, which should still have the terminal as it's stdout. you'll probably need to use some external kill signal to stop the whole thing, once test.sh is done.
If you don't like this tomfoolery, the other option would be to do as you described: write a wrapper script to use as the CMD/ENTRYPOINT, and have it start SSHD without the debug flag, and then start test.sh.
The advantage of doing it with the script I posted is that the container will stick around after test.sh is finished, so you can log in and poke around, while also making your script wait until the daemon is running.
I'm pretty green regarding docker and find myself facing the following problem:
I'm trying to create a dockerfile to generate an image with my companie software on it. During the installation of that software the install process check if ssh is running with the following command:
if [ $(pgrep sshd | wc -l) -eq 0 ]; then
I probably need to precise that I'm installing and starting open-ssh during that same process.
Can you at all check that a service is running during the image creation ?
I cannot ignore that step has it is executed as part of a self extracting mechanism.
Any clue toward the right direction would be appreciated.
An image cannot run services. You are just creating all the necessary things needed for your container to run, like installing databases, servers, or copying some config files etc in the Dockerfile. The last step in the Dockerfile is where you can give instructions on what to do when you issue a docker run command. A script or command can be specified using CMD or ENTRYPOINT in the Dockerfile.
To answer your question, during the image creation process, you cannot check whether a service is running or not. When the container is started, docker will execute the script or command that you can specify in the CMD or ENTRYPOINT. You can use that script to check if your services are running or not and take necessary action after that.
It is possible to run services during image creation. All processes are killed once a RUN command completes. A service will not keep running between RUN commands. However, each RUN command can start services and use them.
If an image creation command needs a service, start the service and then run the command that depends on the service, all in one RUN command.
RUN sudo service ssh start \
&& ssh localhost echo ok \
&& ./install
The first line starts the ssh server and succeeds with the server running.
The second line tests if the ssh server is up.
The third line is a placeholder: the 'install' command can use the localhost ssh server.
In case the service fails to start, the docker build command will fail.
I'm using the default HAProxy Docker image from https://github.com/dockerfile/haproxy
Unfortunately I can't get it to reload my config properly.
If I run
$ sudo docker exec haprox haproxy -f /etc/haproxy/haproxy.cfg -p '$(</var/run/haproxy.pid)' -st '$(</var/run/haproxy.pid)'
it just dumps out the help file. If I run
$ sudo docker exec haprox 'haproxy -f /etc/haproxy/haproxy.cfg -p $(</var/run/haproxy.pid) -st $(</var/run/haproxy.pid)'
I get
2014/12/30 00:03:23 docker-exec: failed to exec: exec: "haproxy -f /etc/haproxy/haproxy.cfg -p $(</var/run/haproxy.pid) -st $(</var/run/haproxy.pid)": stat haproxy -f /etc/haproxy/haproxy.cfg -p $(</var/run/haproxy.pid) -st $(</var/run/haproxy.pid): no such file or directory
Boo. None of those things are what I want. I can run docker exec haprox service haproxy reload - but this ends out spawning several haproxy processes, so when I connect via the unix socket I get one set of information from show stat but I get an entirely different set of information from the http stats page.
I'm trying to set it up so that I can do graceful redeploys of our legacy software, but it does very very bad things with Tomcat sessions, so my only option is to keep existing sessions alive and pinging the same server.
backend legacy
cookie SERVERID insert indirect preserve
server A 123.0.0.123:8080 cookie A check
server B 123.0.0.123:8080 cookie B check
does the trick. I can call up the socket and run set weight legacy/A 0 and it will drain connections from server A.
But (remember that legacy part?) I have to bop my server A/B containers on the head and bring up new ones. I've got my system setup where I'm generating the new config just fine, but when I reload... strange things happen.
As mentioned earlier, it ends out spawning several haproxy processes. I get different information from the stats page and the unix socket. It also appears that the pid file of the process that I'm communicating with in the browser vs. socat are different.
Worst of all, though, is that it will stop http connections with a 503 - and using ab to test will report some dropped connections. This part is not OK.
Old sessions MUST continue to function until the old server goes down/cookies are cleared. It seems like the rest of the Internet is able to do what I'm trying to do... What am I doing wrong here?
You can now reload the config:
docker kill -s HUP haproxy_container_name
More info: https://hub.docker.com/_/haproxy
I know this is an old question and does not help after 6 years! :) But maybe useful for someone!
if you run ps inside the container as follows you will see the container you have linked runs haproxy as pid 1 which cannot be killed without killing the container and also it is run in foreground so without a pid file. If you want to reload run haproxy in the background in your container and make some other process such as supervisor the primary process.
docker exec -it haproxy ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
haproxy 1 0.0 0.2 28988 4576 ? Ss 02:41 0:00 haproxy -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid
I would like to do "starting a long-running worker process" as in this article
https://docs.docker.com/articles/basics/
I don't understand why sleep 1? why not sleep 86400??? one day or one year?
# Start a very useful long-running process
$ JOB=$(sudo docker run -d ubuntu /bin/sh -c "while true; do echo Hello world; sleep 1; done")
# Collect the output of the job so far
$ sudo docker logs $JOB
# Kill the job
$ sudo docker kill $JOB
What's the "best" way to make it run as a background process for apache, nginx, mysql etc?
Why do you need to echo? is that necessary?
This job is to write Hello world to stdout every second I guess for demonstration purposes only. If you want to do something else at some different interval you have to change it accordingly.
The key thing is the -d flag makes Docker run it in background ('detatched'), and the docker logs lets you examine the logs as many times as you like after that point.
For Apache, get hold of an Apache image; for Nginx use an Nginx image, and so on.
I found this image with Nginx and PHP, and the official Docker mysql image worked for me. You'll need to do a bit more reading to see how to integrate your data, web content, config, etc.