I am new to docker, and have been playing with the docker image in hub.docker.com
I want to add ssh service to the container. I could apt-get install the package (after apt-get update), but I don't know how to start it.
Normally (not in this container), we could start the service via a regular command of service ssh start. But I don't know how to do this in the container without out interfering with the ENTRYPOINT and CMD mechanisms.
The dockerfile comes with a docker-entrypoint.sh (see source code here) that pretty much expects to have a line for CMD
CMD ["mysqld"]
I have read some related SO articles, such as these:
How to automatically start a service when running a docker container?
Start sshd automatically with docker container
but they are not directly applicable here due to the interplay of ENTRYPOINT and CMD in the docker file.
What you want to do is not best practice, what you should do is use a user-defined docker network (Here is the documentation).
In your case you would for example:
1. Create a network
docker network create --driver bridge database_network
2. Start the mariadb container
docker run --network=database_network -itd --name=mariadb mariadb
3. Start the ssh container
Here we are using krlmlr/debian-ssh for the example, feel free to use any other image
docker run --network=database_network -itd --name=ssh -p 22:22 -e SSH_KEY="$(cat ~/.ssh/id_rsa.pub)" krlmlr/debian-ssh:wheezy
Then when you connect via port 22 you can connect to the database using mariadb as the hostname, so instead of let's say localhost:3306 you will use mariadb:3306 within the ssh tunnel.
This will even allow you to setup multiple containers with different ssh keys across different ports if your server can handle the load of that many containers.
To answer the question on which would be more effecient between: A) Run one container with both mysqld and sshd in it; and B) Run two containers one for mysqld and one for sshd
The difference in resource usage would be minimal because running ssh within the official mariadb image would require using supervisor or s6 which would be one more process than running the two containers individually. Which means depending on the size of the ssh image, the amount of memory usage may as well be the same. In terms of CPU usage I'm of the opinion that the case would be the same and may actually favor scenario B.
Related
Suppose, I want to connect a container with another container, where both docker containers are running on a different machine. How do I do that? Hopefully, the attached picture will help to understand what I need. thanks.
This works exactly the same way as if neither process was running in Docker: connect to the other system's IP address and the port you published when you launched the container.
machine02$ docker run --name m2-c1 -p 12345:80 image1
machine01$ docker run --name m1-c5 \
> -e CONTAINER_1_URL=http://192.168.1.102:12345 \
> image5
If you find yourself doing this often, a clustered setup like Kubernetes or Docker Swarm is built for this sort of environment. They have a piece called an overlay network that would allow all 10 containers to share a single "network", so you can directly call c1 as a host name and reach either copy of it. A non-Docker service discovery system, like Hashicorp's Consul, can also help remember what service is running on which node.
I do the docker tutorial document at part 3. Because my computer is windows, I use the docker toolbox. Before part 3, I use the command docker run -p 8080:80 test, and it can connect to 192.168.99.100:8080, that's successful.
But when creates a swarm and deploies the docker-compose.yml, it was a success.
ID NAME MODE REPLICAS IMAGE PORTS
uskmy4zkflhf testswarm_web replicated 5/5 ***/get-started:test *:6666->80/tcp
However, when I used 192.168.99.100:6666 to connect, the page could not be displayed, and using ping, I could see that 192.168.99.100 could be connected.
When I uninstall the toolbox and then reinstall it, I deploy it only once, which means that the entire program sets the port only once and no containers occupy it. It doesn't work in this case either.
What's the problem with that?
The port publishing mechanism works differently when you use standalone or swarm mode. If you're using a compose file in swarm mode, you should not be using docker-compose up but docker stack deploy instead.
I would suggest taking it step-by-step, instead of using the stack deploy or compose approach, first learn to use the docker service create command, and take it one service at a time.
Try docker service create --name proxy --publish 8080:80 nginx and see if you can reach NGINX in 192.168.99.100:8080. Once you're there, try scaling it with docker service update --replicas=5 proxy.
Once you feel comfortable with this, you should be able to tell what's going on with more precision.
If you want to delve deeper into how por publishing works in swarm mode, I suggest this docs article.
Here's my scenario.
I have 2 Docker containers:
C1: is a container with Ruby (but it could be anything else) that prepares data files on which it must perform a calculation in Julia language
C2: is a container with Julia (or R, or Octave...), used to perform the calculation, so as to avoid installing Julia on the same system or container that run Ruby code
From the host, obviously, I have no problem doing the processing.
Usually when two containers are linked (or belong to the same network) they communicate with each other via a network exposing some door. In this case Julia does not expose any door.
Can I run a command on C2 from C1 similar to what is done between host and C2?
If so, how?
Thanks!
Technically yes, but that's probably not what you want to do.
The Docker CLI is just an interface to the Docker service, which listens at /var/run/docker.sock on the host. Anything that can be done via the CLI can be done by directly communicating with this server. You can mount this socket into a running container (C1) as a volume to allow that container to speak to its host's docker service. Docker has a few permissions that need to be set to allow this; older versions allow containers to run in "privileged" mode, in which case they're allowed to (amongst other things) speak to /var/run/docker.sock with the authority of the host. I believe newer versions of Docker split this permission system up a bit more, but you'd have to look into this. Making this work in swarm mode might be a little different as well. Using this API at a code level without installing the full Docker CLI within the container is certainly possible (using a library or coding up your own interaction). A working example of doing this is JupyterHub+DockerSpawner, which has one privileged Hub server that instantiates new Notebook containers for each logged in user.
I just saw that you explicitly state that the Julia container has no door/interface. Could you wrap that code in a larger container that gives it a server interface while managing the serverless Julia program as a "local" process within the same container?
I needed to solve the same problem. In my case, it all started when I needed to run some scripts located in another container via cron, I tried the following scenarios with no luck:
Forgetting about the two-containers scenario and place all the logic in one container, so inter-container execution is no longer needed: Turns out to be a bad idea since the whole Docker concept is to execute single tasks in each container. In any case, creating a dockerfile to build an image with both my main service (PHP in my case) and a cron daemon proved to be quite messy.
Communicate between containers via SSH: I then decided to try building an image that would take care of running the Cron daemon, that would be the "docker" approach to solve my problem, but the bad idea was to execute the commands from each cronjob by opening an SSH connection to the other container (in your case, C1 connecting via SSH to C2). It turns out it's quite clumsy to implement an inter-container SSH login, and I kept running into problems with permissions, passwordless logins and port routing. It worked at the end, but I'm sure this would add some potential security issues, and I didn't feel it was a clean solution.
Implement some sort of API that I could call via HTTP requests from one container to another, using something like Curl or Wget. This felt like a great solution, but it ultimately meant adding a secondary service to my container (an Nginx to attend HTTP connections), and dealing with HTTP requisites and timeouts just to execute a shell script felt too much of a hassle.
Finally, my solution was to run "docker exec" from within the container. The idea, as described by scnerd is to make sure the docker client interacts with the docker service in your host:
To do so, you must install docker into the container you want to execute your commands from (in your case, C1), by adding a line like this to your Dockerfile (for Debian):
RUN apt-get update && apt-get -y install docker.io
To let the docker client inside your container interact with the docker service on your host, you need to add /var/run/docker.sock as a volume to your container (C1). With Docker compose this is done by adding this to your docker service "volumes" section:
- /var/run/docker.sock:/var/run/docker.sock
Now when you build and run your docker image, you'll be able to execute "docker exec" from within the docker, with a command like this, and you'll be talking to the docker service on the host:
docker exec -u root C2 /path/your_shell_script
This worked well for me. Since, in my case, I wanted the Cron container to launch scripts in other containers, it was as simple as adding "docker exec" commands to the crontab.
This solution, as also presented by scnerd, might not be optimal and I agree with his comments about your structure: Considering your specific needs, this might not be what you need, but it should work.
I would love to hear any comments from someone with more experience with Docker than me!
I'm trying to do an automatic deploy, so...
I have a .sh script to automatically pull docker images, for example:
docker pull mongo
docker stop db
docker rm db
docker run --name db -d mongo
And I am waiting for a POST request to start it.
So I have a container (with nginx) to act as a server. But I have to call that script outside the container, because it can update any container.
Is that possible? If so, how?
It sounds to me like you are looking for the Docker UNIX socket. See some explanation here (might be best to scroll down to the 'The Solution' part of that page.
Basically, you would start your Nginx container with the mounted UNIX socket. This allows you to use the docker command from inside the Nginx container, on other sibling containers.
Important security note:
Using the UNIX socket is a definite security issue, especially if you are exposing it to the worldwide web. See [1] and [2]. Other alternatives might include using Docker-in-Docker, though I am not certain that's suitable for your case right here. Docker did publish a blogpost on how to secure the UNIX socket here, if that is the path you want to go.
I'm running Jenkins inside a Docker container. I wonder if it's ok for the Jenkins container to also be a Docker host? What I'm thinking about is to start a new docker container for each integration test build from inside Jenkins (to start databases, message brokers etc). The containers should thus be shutdown after the integration tests are completed. Is there a reason to avoid running docker containers from inside another docker container in this way?
Running Docker inside Docker (a.k.a. dind), while possible, should be avoided, if at all possible. (Source provided below.) Instead, you want to set up a way for your main container to produce and communicate with sibling containers.
Jérôme Petazzoni — the author of the feature that made it possible for Docker to run inside a Docker container — actually wrote a blog post saying not to do it. The use case he describes matches the OP's exact use case of a CI Docker container that needs to run jobs inside other Docker containers.
Petazzoni lists two reasons why dind is troublesome:
It does not cooperate well with Linux Security Modules (LSM).
It creates a mismatch in file systems that creates problems for the containers created inside parent containers.
From that blog post, he describes the following alternative,
[The] simplest way is to just expose the Docker socket to your CI container, by bind-mounting it with the -v flag.
Simply put, when you start your CI container (Jenkins or other), instead of hacking something together with Docker-in-Docker, start it with:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
Now this container will have access to the Docker socket, and will therefore be able to start containers. Except that instead of starting "child" containers, it will start "sibling" containers.
I answered a similar question before on how to run a Docker container inside Docker.
To run docker inside docker is definitely possible. The main thing is that you run the outer container with extra privileges (starting with --privileged=true) and then install docker in that container.
Check this blog post for more info: Docker-in-Docker.
One potential use case for this is described in this entry. The blog describes how to build docker containers within a Jenkins docker container.
However, Docker inside Docker it is not the recommended approach to solve this type of problems. Instead, the recommended approach is to create "sibling" containers as described in this post
So, running Docker inside Docker was by many considered as a good type of solution for this type of problems. Now, the trend is to use "sibling" containers instead. See the answer by #predmijat on this page for more info.
It's OK to run Docker-in-Docker (DinD) and in fact Docker (the company) has an official DinD image for this.
The caveat however is that it requires a privileged container, which depending on your security needs may not be a viable alternative.
The alternative solution of running Docker using sibling containers (aka Docker-out-of-Docker or DooD) does not require a privileged container, but has a few drawbacks that stem from the fact that you are launching the container from within a context that is different from that one in which it's running (i.e., you launch the container from within a container, yet it's running at the host's level, not inside the container).
I wrote a blog describing the pros/cons of DinD vs DooD here.
Having said this, Nestybox (a startup I just founded) is working on a solution that runs true Docker-in-Docker securely (without using privileged containers). You can check it out at www.nestybox.com.
Yes, we can run docker in docker, we'll need to attach the unix socket /var/run/docker.sock on which the docker daemon listens by default as volume to the parent docker using -v /var/run/docker.sock:/var/run/docker.sock.
Sometimes, permissions issues may arise for docker daemon socket for which you can write sudo chmod 757 /var/run/docker.sock.
And also it would require to run the docker in privileged mode, so the commands would be:
sudo chmod 757 /var/run/docker.sock
docker run --privileged=true -v /var/run/docker.sock:/var/run/docker.sock -it ...
I was trying my best to run containers within containers just like you for the past few days. Wasted many hours. So far most of the people advise me to do stuff like using the docker's DIND image which is not applicable for my case, as I need the main container to be Ubuntu OS, or to run some privilege command and map the daemon socket into container. (Which never ever works for me)
The solution I found was to use Nestybox on my Ubuntu 20.04 system and it works best. Its also extremely simple to execute, provided your local system is ubuntu (which they support best), as the container runtime are specifically deigned for such application. It also has the most flexible options. The free edition of Nestybox is perhaps the best method as of Nov 2022. Highly recommends you to try it without bothering all the tedious setup other people suggest. They have many pre-constructed solutions to address such specific needs with a simple command line.
The Nestybox provide special runtime environment for newly created docker container, they also provides some ubuntu/common OS images with docker and systemd in built.
Their goal is to make the main container function exactly the same as a virtual machine securely. You can literally ssh into your ubuntu main container as well without the ability to access anything in the main machine. From your main container you may create all kinds of containers like a normal local system does. That systemd is very important for you to setup docker conveniently inside the container.
One simple common command to execute sysbox:
dock run --runtime=sysbox-runc -it any_image
If you think thats what you are looking for, you can find out more at their github:
https://github.com/nestybox/sysbox
Quicklink to instruction on how to deploy a simple sysbox runtime environment container: https://github.com/nestybox/sysbox/blob/master/docs/quickstart/README.md