I'm pretty green regarding docker and find myself facing the following problem:
I'm trying to create a dockerfile to generate an image with my companie software on it. During the installation of that software the install process check if ssh is running with the following command:
if [ $(pgrep sshd | wc -l) -eq 0 ]; then
I probably need to precise that I'm installing and starting open-ssh during that same process.
Can you at all check that a service is running during the image creation ?
I cannot ignore that step has it is executed as part of a self extracting mechanism.
Any clue toward the right direction would be appreciated.
An image cannot run services. You are just creating all the necessary things needed for your container to run, like installing databases, servers, or copying some config files etc in the Dockerfile. The last step in the Dockerfile is where you can give instructions on what to do when you issue a docker run command. A script or command can be specified using CMD or ENTRYPOINT in the Dockerfile.
To answer your question, during the image creation process, you cannot check whether a service is running or not. When the container is started, docker will execute the script or command that you can specify in the CMD or ENTRYPOINT. You can use that script to check if your services are running or not and take necessary action after that.
It is possible to run services during image creation. All processes are killed once a RUN command completes. A service will not keep running between RUN commands. However, each RUN command can start services and use them.
If an image creation command needs a service, start the service and then run the command that depends on the service, all in one RUN command.
RUN sudo service ssh start \
&& ssh localhost echo ok \
&& ./install
The first line starts the ssh server and succeeds with the server running.
The second line tests if the ssh server is up.
The third line is a placeholder: the 'install' command can use the localhost ssh server.
In case the service fails to start, the docker build command will fail.
Related
I'm creating an application that will allow users to upload video files that will then be put through some processing.
I have two containers.
Nginx container that serves the website where users can upload their video files.
Video processing container that has FFmpeg and some other processing stuff installed.
What I want to achieve. I need container 1 to be able to run a bash script on container 2.
One possibility as far as I can see is to make them communicate over HTTP via an API. But then I would need to install a web server in container 2 and write an API which seems a bit overkill.
I just want to execute a bash script.
Any suggestions?
You have a few options, but the first 2 that come time mind are:
In container 1, install the Docker CLI and bind mount
/var/run/docker.sock (you need to specify the bind mount from the
host when you start the container). Then, inside the container, you
should be able to use docker commands against the bind mounted
socket as if you were executing them from the host (you might also
need to chmod the socket inside the container to allow a non-root
user to do this.
You could install SSHD on container 2, and then ssh in from container 1 and run your script. The advantage here is that you don't need to make any changes inside the containers to account for the fact that they are running in Docker and not bare metal. The down side is that you will need to add the SSHD setup to your Dockerfile or the startup scripts.
Most of the other ideas I can think of are just variants of option (2), with SSHD replaced by some other tool.
Also be aware that Docker networking is a little strange (at least on Mac hosts), so you need to make sure that the containers are using the same docker-network and are able to communicate over it.
Warning:
To be completely clear, do not use option 1 outside of a lab or very controlled dev environment. It is taking a secure socket that has full authority over the Docker runtime on the host, and granting unchecked access to it from a container. Doing that makes it trivially easy to break out of the Docker sandbox and compromise the host system. About the only place I would consider it acceptable is as part of a full stack integration test setup that will only be run adhoc by a developer. It's a hack that can be a useful shortcut in some very specific situations but the drawbacks cannot be overstated.
I wrote a python package especially for this use-case.
Flask-Shell2HTTP is a Flask-extension to convert a command line tool into a RESTful API with mere 5 lines of code.
Example Code:
from flask import Flask
from flask_executor import Executor
from flask_shell2http import Shell2HTTP
app = Flask(__name__)
executor = Executor(app)
shell2http = Shell2HTTP(app=app, executor=executor, base_url_prefix="/commands/")
shell2http.register_command(endpoint="saythis", command_name="echo")
shell2http.register_command(endpoint="run", command_name="./myscript")
can be called easily like,
$ curl -X POST -H 'Content-Type: application/json' -d '{"args": ["Hello", "World!"]}' http://localhost:4000/commands/saythis
You can use this to create RESTful micro-services that can execute pre-defined shell commands/scripts with dynamic arguments asynchronously and fetch result.
It supports file upload, callback fn, reactive programming and more. I recommend you to checkout the Examples.
Running a docker command from a container is not straightforward and not really a good idea (in my opinion), because :
You'll need to install docker on the container (and do docker in docker stuff)
You'll need to share the unix socket, which is not a good thing if you have no idea of what you're doing.
So, this leaves us two solutions :
Install ssh on you're container and execute the command through ssh
Share a volume and have a process that watch for something to trigger your batch
It was mentioned here before, but a reasonable, semi-hacky option is to install SSH in both containers and then use ssh to execute commands on the other container:
# install SSH, if you don't have it already
sudo apt install openssh-server
# start the ssh service
sudo service start ssh
# start the daemon
sudo /usr/sbin/sshd -D &
Assuming you don't want to always be root, you can add default user (in this case, 'foobob'):
useradd -m --no-log-init --system --uid 1000 foobob -s /bin/bash -g sudo -G root
#change password
echo 'foobob:foobob' | chpasswd
Do this on both the source and target containers. Now you can execute a command from container_1 to container_2.
# obtain container-id of target container using 'docker ps'
ssh foobob#<container-id> << "EOL"
echo 'hello bob from container 1' > message.txt
EOL
You can automate the password with ssh-agent, or you can use some bit of more hacky with sshpass (install it first using sudo apt install sshpass):
sshpass -p 'foobob' ssh foobob#<container-id>
I believe
docker exec -it <container_name> <command>
should work, even inside the container.
You could also try to mount to docker.sock in the container you try to execute the command from:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
I pulled centos6 image and made a container from it. I got its bash by:
$ docker run -i -t centos:centos6 /bin/bash
On the centos6 container, I could use "service" command without any problem. But when I pulled&used centos7 image:
$ docker run -i -t centos:centos7 /bin/bash
Both of "service" and "systemctl" didn't work. The error message is:
Failed to get D-Bus connection: Operation not permitted
My question is:
1. How are people developing without "service" and "systemctl" commands?
2. If I want to use, for example, httpd.service on the centos7 container, what should I do? Or maybe running services on a container is not recommended?
There is no process supervisor running inside either container. The service command in your CentOS 6 container works by virtue of the fact that it just runs a script from /etc/init.d, which by design ultimately launch a command in the background and return control to you.
CentOS 7 uses systemd, and systemd is not running inside your container, so there is nothing for systemctl to talk to.
In either situation, using the service or systemctl command is generally the wrong thing to do: you want to run a single application, and you want to run it in the foreground, so that your container continues to run (from Docker's perspective, a command that goes into the background has exited, and if that was pid 1 in the container, the container will exit).
How are people developing without "service" and "systemctl" commands?
They are starting their programs directly, by consulting the necessary documentation to figure out the appropriate command line.
If I want to use, for example, httpd.service on the centos7 container, what should I do? Or maybe running services on a container is recommended?
You would start the httpd binary using something like:
CMD ["httpd", "-DFOREGROUND"]
If you like to stick with service/sytemctl commands to start/stop services then you can do that in a centos7 container by using the docker-systemctl-replacement script.
I had some deployment scripts that were using th service start/stop commands on a real machine - and they work fine with a container. Without any further modification. When putting the systemctl.py script into the CMD then it will simply start all enabled services somewhat like the init-process on a real machine.
systemd is included but not enabled by default in CentOS 7 docker image. It is mentioned on the repository page along with steps to enable it.
https://hub.docker.com/_/centos/
I have a node docker container on which i'm running a dev server.
In my docker-compose.yml file, the entry command is :
...
command: start-dev-server
...
Where start-dev-server points to a script that starts the server after a vendor install :
// /usr/local/bin/start-dev-server
#!/usr/bin/env bash
# install node modules if missing
npm i
# start the dev server
npm run start
So when I start my container, the server will also start.
I know that I can access my container in bash via the following command :
docker exec -it my-container bash
But there I can't stop or restart my server.
Is there a way to access the ssh with the started command ? (to see the server logs for example, or to stop & restart it).
Maybe I take it by the wrong path here because the entry command isn't supposed to be stopped ? So in this case, would anyone has a solution that could allow me to start my server & control it in a more flexible way ?
The best practices says that you should see the container as your server. If you want to stop it, stop the container (docker stop my-container), if you want to restart it, restart the container (docker restart my-container). Your server should log to stdout, so you can see the logs using docker logs -f my-container. So, you're right, the command isn't supposed to be stopped, as it will stop the container.
I am trying to connect to docker containers using the default SSHManager.
These containers only have a running sshd, with public key authentication, and julia installed.
Here is my dockerfile:
FROM rastasheep/ubuntu-sshd
RUN apt-get update && apt-get install -y julia
RUN mkdir -p /root/.ssh
ADD id_rsa.pub /root/.ssh/authorized_keys
I am running the container using:
sudo docker run -d -p 3333:22 -it --name julia-sshd julia-sshd
And then in the host machine, using the julia repl, I get the following error:
julia> import Base:SSHManager
julia> addprocs(["root#localhost:3333"])
stdin: is not a tty
Worker 2 terminated.
ERROR (unhandled task failure): EOFError: read end of file
Master process (id 1) could not connect within 60.0 seconds.
exiting.
I have tested that I can connect to the container via ssh without password.
I have also tested that in julia repl I can add a regular machine with julia installed to the cluster and it works fine.
But I cannot get this two things working together. Any help or suggestions will be apreciated.
I recommend you to also deploy the Master in a Docker container. It makes your environment easily and fully reproducible.
I'm working on a way of deploying Workers in Docker containers on-demand. i.e., the Master deployed in a container can deploy further DockerizedJuliaWorkers. It is similar to https://github.com/gsd-ufal/Infra.jl but assuming that Master and Workers run on the same host, which makes things not so hard.
It is an on-going work and I plan to finish next weeks. In a nutshell:
1) You'll need a simple DockerBackend and a wrapper to transparently run containers, set up SSH, and call addprocs with all the low-level parameters (i.e., the DockerizedJuliaWorker.jl file):
https://github.com/NaelsonDouglas/DistributedMachineLearningThesis/tree/master/src/docker
2) Read here how to build the Docker image (Dockerfile is included):
https://github.com/NaelsonDouglas/DistributedMachineLearningThesis
Please tell me if you have any suggestion on how to improve it.
Best,
André Lage.
I am trying to run a simple docker container with my web application installed (Not using docker file).
During the testing I would always run a container using -t -i option and then start the tomcat service inside it by running a shell script.
How when I am moving to production I dont want to use the -t -i option any more and just need my Tomcat service to start and be the only primary service.
I trying pointing the entrypoint to the start up script for starting tomcat but the container terminates after that script finishes.
How do I run a container, start a service and keep that service as the single primary service of the container?
Note: I read some posts about supervisor but not sure if I would need to start building my image from scratch if I go that route? I would prefer not doing that.
Any suggestions?
If you have a Dockerfile that uses an entrypoint pattern, it will look something like this:
(Dockerfile)
FROM ubuntu
...Some configuration steps...
add start.sh /start.sh
ENTRYPOINT ["/start.sh"]
All you need to do is make sure your start.sh script 'hangs' in some way. Some people like to tail the syslogs, but tailing any file that exists will work.
(start.sh)
#!/bin/bash
service Your_Service_Or_Whatever start
tail -f /var/log/dmesg
A shorter version:
FROM ubuntu
...Some configuration steps...
ENTRYPOINT ["/bin/sh", "-c", "while true; do sleep 1; done"]
tested with Docker version 1.12.1, build 23cf638
Use docker --version to find out your version
Docker containers as default will run according to the configuration in the images Dockerfile. If you usually run a container with the -i flag, you leave STDIN open allowing you access to the containers entrypoint or it could be a bash shell. To achieve what you want, you can run the container in a detached state passing your commands into docker run directly.
docker run -d myapp /opt/catalina/bin/startup.sh
This will run the myapp container in a detached state and will run the command passed as the 3rd argument. If the command results in a long lived service, the container will stay active as long as the service is.
This is explained in detail in the docs.