TL;DR
"Reading/writing to stdin/stdout of Entrypoint executable process on running container (ex. Docker, podman) via socket/http"
My question is in regards to executable containers and how to spin/wrap them up as servers, and connect to them remotely (socket/curl/http).
I have an executable container that is not a server, just a standalone executable ENTRYPOINT command. (Example: wkhtmltopdf). I would love to use an image like that as a container, but not make it a part of my own container/Dockerfile (i.e. not to modify/maintain it).
As I understand if I run this image as a container, it will immediately die as there is no input/ on this process and it does not act as a server/listening for connections.
A lot of tutorials state that you can run containers via docker run ... command.
If I am on Windows inside WSL2 distro I am not necessarily going to have docker installed locally. But I believe that we can connect to other containers that are on the same network (default bridge network).
But how would one spin-up/expose executable container as a server and connect to it's stdin/stdout?
I want for container to listen on port (direct to executable stdin), process the request with said executable, and obtain the results from stdout/curl.
What I think can work:
Spinning process as a server (Golang example)
Do I need to wrap every executable that I want to expose BUT not make it as a part of my own Dockerfile or is there a common pattern/tool I can use for such a scenario.
P.S. wkhtmltopdf executable example I gave also comes with python and node included. (I am thinking about some sort of one-liner solution akin to docker run --rm -v ... X ... python -m SimpleHTTPServer ... )
This answer just describes the situation on Linux.
I haven't tested this but I think you could combine systemd socket activation
[Socket]
ListenStream=127.0.0.1:9999
Accept=yes
(see example https://unix.stackexchange.com/questions/551000/systemd-socket-activation-stdin)
with netcat (see example
https://unix.stackexchange.com/questions/332163/netcat-send-text-to-echo-service-read-reply-then-exit)
Use the -i flag, i.e., podman run -i ... or docker run -i to pass stdin in to the container.
Related
When running a Docker Compose project, it would be nice to be able to open an app with certain parameters on the host operating system (on which docker-compose up is being invoked). This would be legitimately useful when running web apps. For example, I would love to have Docker Compose automatically open a browser on the host with location of http://localhost:8080, when I run docker-compose run, rather than manually opening a browser and entering the http://localhost:8080. Just the way we see in Minikube (e.g when running minikube service web-deployment).
I am aware there are parameters to use in docker-compose.yml to pass commands to run in containers, like command and entrypoint, but I don't know if that is possible for applications on the host OS.
Compose can do a pretty limited set of things. It can build Docker images, without any ordering constraints, and it can start (presumably long-running) Docker containers, with very limited ordering constraints. It can create a couple of associated Docker objects like networks and named volumes. That's literally all it can do, though; it cannot do larger-scale orchestration task ("run this migration container to completion, then run this application") or launch non-Docker tasks.
You might use some host-based tool to manage this instead. Even a shell script would be enough; possibly something like
#!/bin/sh
# start the container stack
# (assumes the caller has permission to do this)
docker-compose up -d
# wait for the service to be ready
while ! curl --fail --silent --head http://localhost:8080; do
sleep 1
done
# open the browser window
open http://localhost:8080
I used to run a long training process on a remote server with GPU capabilities. Now my work schedule changes, so I can't have my computer connected to a network all the time till I finish the process. I found that nohup is the solution for me. But I don't know how to keep invoke the process correctly related my situation.
I use ssh to connect to the remote server.
I have to use docker to access to GPU.
Then I start the process in the docker.
If I start the process with nohup in docker, I can't really leave docker, right. So, do I use nohup at each step?
Edit:
I need the terminal output of the process at step 3, because I need that information to carry out the rest of the work. Consider, step 3 is training a neural network. So, the training log tells me the accuracy of different models at different iterations. I use that information to do the testing.
Following #David Maze's suggestion, I did this (a slightly different approach as I was not familiar with docker a whole lot)
Logged in to the remote server.
Configured the docker script to have remote workdir.
...
WORKDIR /workspace
...
After building the docker container, run docker with mount option to mount the local project to docker workdir. When running docker, I used nohup. Since I don't need interactive mode I ignored the -it flag.
nohup docker run --gpus all -v $(pwd)/path-to-project-root:/workspace/ docker-image:tag bash -c "command1; command2" > project.out 2>&1 &
To test this, I logged out from the server and see the content of project.out later. It contained the expected output.
I am creating a docker container with Ubuntu:16.04 image using python docker package. I am passing tty as True and detach as True to the client.containers.run() function. The container starts with /sbin/init process. The container is created successfully. But the problem is, the login prompt on my host machine is replaced with the container login prompt on my host machine console. As a result, I am not able to the login on the machine on the console. SSH connection to the machine work fine.
This happens even when I run my python script after connecting SSH to the machine. I tried different options like setting tty to False, setting stdout to False, setting the environment variable TERM to xterm in the container, but nothing help.
It would be really great if someone can suggest a solution for this problem.
My script is very simple:
import docker
client = docker.from_env()
container = client.containers.run('ubuntu:16.04', '/sbin/init', privileged=True,
detach=True, tty=True, stdin_open=True, stdout=False, stderr=False,
environment=['TERM=xterm'])
I am not using any dockerfile.
I have been able to figure out that this problem happens when I start container in privileged mode. If I do this, the /sbin/init process launches /sbin/agetty processes which causes /dev/tty to be attached to the container. I need to figure out a way to start /sbin/init in such a way that it does not create /sbin/agetty processes.
/sbin/init in Ubuntu is a service called systemd. If you look at the linked page it does a ton of things – configures various kernel parameters, mounts filesystems, configures the network, launches getty process, .... Many of these things require changing host-global settings, and if you launch a container with --privileged you're allowing systemd to do that.
I'd give two key recommendations on this command:
Don't run systemd in Docker. If you really need a multi-process init system, supervisord is popular, but prefer single-process containers. If you know you need some init(8) (process ID 1 has some responsibilities) then tini is another popular option.
Don't directly run bare Linux distribution images. Whatever software you're trying to run, it's almost assuredly not in an alpine or ubuntu image. Build a custom image that has the software you need and run that; you should set up its CMD correctly so that you can docker run the image without any manual setup.
Also remember that the ability to run any Docker command at all implies unrestricted root-level access over the host. You're seeing some of that here where a --privileged container is taking over the host's console; it's also very very easy to read and edit files like the host's /etc/shadow and /etc/sudoers. There's nothing technically wrong with the kind of script you're showing, but you need to be extremely careful with standard security concerns.
In my Dockerfile I am trying to install multiple services and want to have them all start up automatically when I launch the container.
One among the services is mysql and when I launch the container I don't see the mysql service starting up. When I try to start manually, I get the error:
Failed to get D-Bus connection: Operation not permitted
Dockerfile:
FROM centos:7
RUN yum -y install mariadb mariadb-server
COPY start.sh start.sh
CMD ["/bin/bash", "start.sh"]
My start.sh file:
service mariadb start
Docker build:
docker build --tag="pbellamk/mariadb" .
Docker run:
docker run -it -d --privileged=true pbellamk/mariadb bash
I have checked the centos:systemd image and that doesn't help too. How do I launch the container with the services started using systemctl/service commands.
When you do docker run with bash as the command, the init system (e.g. SystemD) doesn’t get started (nor does your start script, since the command you pass overrides the CMD in the Dockerfile). Try to change the command you use to /sbin/init, start the container in daemon mode with -d, and then look around in a shell using docker exec -it <container id> sh.
Docker is designed around the idea of a single service/process per container. Although it definitely supports running multiple processes in a container and in no way stops you from doing that, you will run into areas eventually where multiple services in a container doesn't quite map to what Docker or external tools expect. Things like moving to scaling of services, or using Docker swarm across hosts only support the concept of one service per container.
Docker Compose allows you to compose multiple containers into a single definition, which means you can use more of the standard, prebuilt containers (httpd, mariadb) rather than building your own. Compose definitions map to Docker Swarm services fairly easily. Also look at Kubernetes and Marathon/Mesos for managing groups of containers as a service.
Process management in Docker
It's possible to run systemd in a container but it requires --privileged access to the host and the /sys/fs/cgroup volume mounted so may not be the best fit for most use cases.
The s6-overlay project provides a more docker friendly process management system using s6.
It's fairly rare you actually need ssh access into a container, but if that's a hard requirement then you are going to be stuck building your own containers and using a process manager.
You can avoid running a systemd daemon inside a docker container altogether. You can even avoid to write a special start.sh script - that is another benefit when using the docker-systemctl-replacement script.
The docker systemctl.py can parse the normal *.service files to know how to start and stop services. You can register it as the CMD of an image in which case it will look for all the systemctl-enabled services - those will be started and stopped in the correct order.
The current testsuite includes testcases for the LAMP stack including centos, so it should run fine specifically in your setup.
I found this project:
https://github.com/defn/docker-systemd
which can be used to create an image based on the stock ubuntu image but with systemd and multiuser mode.
My use case is the first one mentioned in its Readme. I use it to test the installer script of my application that is installed as a systemd service. The installer creates a systemd service then enables and starts it. I need CI tests for the installer. The test should create the installer, install the application on an ubuntu, and connect to the service from outside.
Without systemd the installer would fail, and it would be much more difficult to write the test with vagrant. So, there are valid use cases for systemd in docker.
Docker Version 1.12,
I got a Dockerfile from Here
FROM nginx:latest
RUN touch /marker
ADD ./check_running.sh /check_running.sh
RUN chmod +x /check_running.sh
HEALTHCHECK --interval=5s --timeout=3s CMD ./check_running.sh
I'm able to roll the updates and health checks with check_running.sh shell script. Here, the check_running.sh script is copied to image, so the launched container has it.
Now, my question is there any way to Health Check from out side of the container and script also located outside.
I'm excepting a health check command to get the container performance(Depends on what we wrote in script), IF the container is not performing good it should roll-back to previous version ( Kind of a process that monitors the containers, if it is not good, it should roll-back to previous)
Thanks
is there any way to Health Check from out side of the container and
script also located outside.
Kind of a process that monitors the containers, if it is not good, it should roll-back to previous
You have several options:
From outside, you run a process inside the container to check its health with docker exec. This could be any sequence of shell commands. If you want to keep your scripts outside of the container, you might use something like cat script.sh | docker exec -it container sh -s.
You check the container health from outside the container, e.g. by looking for a process that should be running inside the container (try to set a security profile and use ps -Zax or try looking for children of the daemon), or you can give each container a specific user ID with --user 12345 and then look for that or e.g. connecting to its services. You'd have to make sure it's running inside the right container. You can access the containers' filesystem below /var/lib/docker/devicemapper/mnt/<hash>/rootfs.
You run a HEALTHCHECK inside the container and check its health with docker inspect --format='{{json .State.Health.Status}}' <containername> combined with e.g. a line in the Dockerfile:
HEALTHCHECK CMD wget -q -s http://some.host to check the container has internet access.
I'd recommend option 3, because it's likely to be more compatible with other tools in the future.
Just got comment from a blog!. He refered Docker documentation HealthCheck section. There is a health check "option" for docker command to "override" the dockerfile defaults. I have not checked yet!. But it seems good for me to get what I want. Will check and update the answer!
The Docker inspect command lets you view the output of commands that succeed or fail
docker inspect --format='{{json .State.Health}}' your-container-name
That's not available with the Dockerfile HEALTHCHECK option, all checks run inside the container. To me, this is a good thing since it avoids potentially untrusted code running directly on the host, and it allows you to include the dependencies for the health check inside your container.
If you need to monitor your container from outside, you'll need to use another tool or monitoring application, there are quite a few of them out there.
You can view the results of the health check by running docker inspect on a container.
Another approach depending on your application would be to expose a /healthz endpoint that the healthcheck also probes, this way it can be queried externally or internally as needed.