I have a Jenkins container running inside Docker and I want to use this Jenkins container to spin up other Docker containers when running integration tests etc.
So my plan was to install Docker in the container but this doesn't seem to work so well for me. My Dockerfile looks something like this:
FROM jenkins
MAINTAINER xxxx
# Switch user to root so that we can install apps
USER root
RUN apt-get update
# Install latest version of Docker
RUN apt-get install -y apt-transport-https
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
RUN sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
RUN apt-get update
RUN apt-get install -y lxc-docker
# Switch user back to Jenkins
USER jenkins
The jenkins image is based on Debian Jessie. When I start bash terminal inside container based on the generated image and do for example:
docker images
I get the following error message:
FATA[0000] Get http:///var/run/docker.sock/v1.16/images/json: dial unix /var/run/docker.sock: no such file or directory. Are you trying to connect to a TLS-enabled daemon without TLS?
I suspect that this could be because the docker service is not started. But my next problem arise when I try to start the service:
service docker start
This gives me the following error:
mount: permission denied
I've tracked the error in /etc/init.d/docker to this line:
mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
So my questions are:
How do I actually start a Docker host inside a container? Or is this
something that should be avoided?
Is there something special I need to do if I'm running Mac and boot2docker?
Perhaps I should instead link to the Docker on the host machine as described here?
Update: I've tried the container as user root and jenkins. sudo is not installed.
A simpler alternative is to mount the docker socket and create sibling containers. To do this, install docker on your image and run something like:
docker run -it -v /var/run/docker.sock:/var/run/docker.sock myimage
In the container you should now be able to run docker commands as if you were on the host. The advantage of this method is that you don't need --privileged and get to use the cache from the host. The disadvantage is that you can see all running containers, not just the ones the created from the container.
1.- The first container you start (the one you launch other one inside) must be run with the --privileged=true flag.
2.- I think there is not.
3.- Using the privileged flag you don't need to mount the docker socket as a volume.
Check this project to see an example of all this.
Related
Below is the code snippet of jenkins image taken from here:
# Install Docker Engine
RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D && \
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | tee /etc/apt/sources.list.d/docker.list && \
apt-get update -y && \
apt-get purge lxc-docker* -y && \
apt-get install docker-engine=${DOCKER_ENGINE:-1.10.2}-0~trusty -y && \
usermod -aG docker jenkins && \
usermod -aG users jenkins
that installs docker engine within jenkins image. My understanding is, var/run/docker.sock is created withing Jenkins container, due to installation of docker engine.
Below is the volume mapping syntax taken from here:
volumes:
- jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
that launches jenkins container(above) on EC2 host.
EC2 host also has docker daemon running.
So, there is docker daemon running in EC2 host. There is also a docker daemon running within docker container(Jenkins)
With this syntax(/var/run/docker.sock:/var/run/docker.sock) in docker-compose(above) for socket files,
Does docker daemon within Jenkins container override its own socket file with the socket file present in EC2 host? If yes... what is its implication?
/var/run/docker.sock in the container is the host's Docker socket, and nothing else. This is because:
A Docker container doesn't run any programs other than what's explicitly started in its entrypoint and/or command, and that's almost always just a single application program.
You're presumably not going out of your way to start the Docker daemon, so it's installed but not running.
Unix socket files won't get created until a daemon starts up and it bind(2) a socket to the specific file.
The docker run -v option will always "push" the host's content into the container, and this happens before any of the container processes run.
So in the scenario you're describing, it can't be anything other than the host Docker socket, because there isn't a second Docker daemon.
Let's say for the sake of argument that you actually are starting a second daemon this way.
The order of operations here is (1) Docker sets up the container filesystem, (2) Docker starts running the entrypoint, (3) the entrypoint starts the daemon, (4) the daemon tries to create the socket file. At the point where the daemon starts up, its socket file will already exist. I believe that will cause the bind(2) call to fail with EADDRINUSE, and the daemon won't start up. Hopefully this will cause your container startup to fail.
You could legitimately want to start a daemon in a container, that publishes a Unix socket, that you want to access from the host. To make this work you need to mount a directory into the container, and point the daemon at it. It probably can't be /var/run on either side (there's a lot of stuff in the host /var/run; mounting the directory hides the existing contents in the container and you could want the container's /var/run too). It must be a directory and not a socket filename since Docker will create an empty directory if it doesn't exist; something will exist in the container at that path and the bind will fail.
So if you wanted to start a hypothetical foo daemon inside a container, it would look roughly like
docker run \
--name foo \ # container name
-v $PWD/socket:/socket \ # bind mount a directory
foo \ # image name
food \ # command to run in the container
--foreground \ # don't daemonize; keep the container alive
--bind fd://socket/foo.sock # put the socket in the shared directory
On the host you'd need to set FOO_SOCKET_PATH=$PWD/socket/foo.sock or otherwise point to this specific file.
From the docs:
Docker-engine is a client-server application
Please note that when you install docker-engine you install docker-daemon (server) and docker cli (client).
It means that if a docker daemon isn't running you will still be able run docker cli commands:
docker info
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Jenkins image you shared doesn't have instructions to run docker engine. So i assume it's not running inside the container.
/var/run/docker.sock:/var/run/docker.sock volume maps docker host's docker engine socket to the container.
So docker cli commands run within the container control the docker-engine running on the docker host.
This makes sense if you do CI/CD on your host from within containerized Jenkins.
Jenkins pipelines may use docker, docker-compose and docker swarm commands to run tests, build artifacts and deploy new versions of applications.
Simular Questions
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
How to solve Docker permission error when trigger by Jenkins
https://github.com/jenkinsci/docker/issues/263
Dockerfile
FROM jenkins/jenkins:lts
USER root
RUN apt-get -qq update && apt-get -qq -y install --no-install-recommends curl
RUN curl -sSL https://get.docker.com/ | sh
RUN usermod -aG docker jenkins
USER jekins
Terminal command
docker run -p 8080:8080 -p 50000:50000 \
-v jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
-ti bluebrown/docker-in-jenkins-in-docker /bin/bash
Inside the container
docker image ls
Output
Got permission denied while trying to connect to the Docker daemon
socket at unix:///var/run/docker.sock: Get
http://%2Fvar%2Frun%2Fdocker.sock/v1.39/images/json: dial unix
/var/run/docker.sock: connect: permission denied
When I comment the last line of the dockerfile out, to run the instance as root user,
# USER jenkins
I can access the docker socket without issues, for obvious reasons. However, I think this is not a proper solution. That is why I want to ask if anyone managed to access the docker socket as non root user.
You've added the docker group to the Jenkins user inside the container. However, that will not necessarily work because the mapping of users and groups to uids and gids can be different between the host and container. That's normally not an issue, but with host volumes and other bind mounts into the container, the files are mapped with the same uid/gid along with the permissions. Therefore, inside the container, the docker group will not have access to the docker socket unless the gid happens to be identical between the two environments.
There are several solutions, including manually passing the host gid as the gid to use inside the container. Or you can get the gid of the host and build the image with that value hard coded in.
My preferred solution is to start an entrypoint as root, fix the docker group inside the container to match the gid of the mounted docker socket, and then switch to the Jenkins user to launch the app. This works especially well in development environments where control of uid/gids may be difficult. All the steps/scripts for this are in my repo: https://github.com/sudo-bmitch/jenkins-docker
For production in a controlled environment, I try to get standardized uid/gid values, both on the host and in containers, for anything that mounts host volumes. Then I can run the container without the root entrypoint steps.
In your dockerfile you are enabling docker access for the user jenkins but droping down to the user jekins not jenkins?
Is this just a typo on this page?
I use this approach as you've described and it works correctly.
I have airflow running on an EC2 instance, and I am scheduling some tasks that spin up a docker container. How do I do that? Do I need to install docker on my airflow container? And what is the next step after. I have a yaml file that I am using to spin up the container, and it is derived from the puckel/airflow Docker image
I got a simpler solution working which just requires a short Dockerfile to build a derived image:
FROM puckel/docker-airflow
USER root
RUN groupadd --gid 999 docker \
&& usermod -aG docker airflow
USER airflow
and then
docker build -t airflow_image .
docker run -v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /usr/bin/docker:/bin/docker:ro \
-v /usr/lib/x86_64-linux-gnu/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7:ro \
-d airflow_image
Finally resolved
My EC2 setup is running unbuntu Xenial 16.04 and using a modified the puckel/airflow docker image that is running airflow
Things you will need to change in the Dockerfile
Add USER root at the top of the Dockerfile
USER root
mounting docker bin was not working for me, so I had to install the
docker binary in my docker container
Install Docker from Docker Inc. repositories.
RUN curl -sSL https://get.docker.com/ | sh
search for wrapdocker file on the internet. Copy it into scripts directory in the folder where the Dockerfile is located. This starts the docker daemon inside airflow docker
Install the magic wrapper
ADD ./script/wrapdocker /usr/local/bin/wrapdocker
RUN chmod +x /usr/local/bin/wrapdocker
add airflow as a user to the docker group so the airflow can run docker jobs
RUN usermod -aG docker airflow
switch to airflow user
USER airflow
Docker compose file or command line arguments to docker run
Mount docker socket from docker airflow to the docker image just installed
- /var/run/docker.sock:/var/run/docker.sock
You should be good to go !
You can spin up docker containers from your airflow docker container by attaching volumes to your container.
Example:
docker run -v /var/run/docker.sock:/var/run/docker.sock:ro -v /path/to/bin/docker:/bin/docker:ro your_airflow_image
You may also need to attach some libraries required by docker. This depends on the system you are running Docker on. Just read the error messages you get when running a docker command inside the container, it will indicate you what you need to attach.
Your airflow container will then have full access to Docker running on the host.
So if you launch docker containers, they will run on the host running the airflow container.
Am trying to install and configure openstack (devstack) inside docker container. While installing am getting the following error
"Failed to get D-Bus connection: No connection to service manager."
Later, I checked and found that its because of systemd problem. When I tried executing the command systemd
$>systemd
Am getting the following output.
Trying to run as user instance, but the system has not been booted with systemd.
Following are the things which am used.
Host machine OS : Ubuntu 14.04,
Docker Version : Docker version 1.12.4, build 1564f02,
Docker Container OS : Ubuntu 14.04
Can anyone help in this. Thanks in advance.
First of all, systemd expects /sys/fs/cgroup to be mounted. Additionally, you must make the container privileged, or else this happens:
docker run -v /sys/fs/cgroup:/sys/fs/cgroup:ro --privileged -it --rm ubuntu
Then you can go ahead and run /bin/systemd --system --unit=basic.target from bash, and it should run normally (with some errors of course, because Docker does not virtualize an entire system, nor is the library:ubuntu image more than the minimum size required to run properly):
After you have systemd running (semi-)properly, you can simply use a docker stop to stop the container.
This post is based on my own research, a few weeks of it too, for a project I like to call initbuntu (originally I tried to get init running, but running systemd directly was my only solution after all my failed tries). The container will be available on Docker Hub as logandark/initbuntu, Soon™. For now, a broken copy (or not broken, I dunno) is available there at the time of posting.
Sources (kinda):
/sys/fs/cgroup: Here
systemd --system: A StackOverflow post I lost the link to.
Existing DevStack on Docker Project
First of all, you can get a preconfigured Dockerfile with DevStack Ocata/Pike on Docker here. The repository also contains further information on DevStack and containers.
Build Your Own Image
Running systemd in Docker is certainly possible and has been done before. I found Ubuntu 16.04 LTS is a good foundation for the Docker host as well as the base image.
Your systemd/DevStack Dockerfile needs this configuration, which also cleans up services you probably don't want inside a Docker container:
FROM ubuntu:16.04
#####################################################################
# Systemd workaround from solita/ubuntu-systemd and moby/moby#28614 #
#####################################################################
ENV container docker
# No need for graphical.target
RUN systemctl set-default multi-user.target
# Gracefully stop systemd
STOPSIGNAL SIGRTMIN+3
# Cleanup unneeded services
RUN find /etc/systemd/system \
/lib/systemd/system \
-path '*.wants/*' \
-not -name '*journald*' \
-not -name '*systemd-tmpfiles*' \
-not -name '*systemd-user-sessions*' \
-exec rm \{} \;
# Workaround for console output error moby/moby#27202, based on moby/moby#9212
CMD ["/bin/bash", "-c", "exec /sbin/init --log-target=journal 3>&1"]
If you intend to run OpenStack/DevStack inside said container, it might save you lots of trouble to start it privileged instead of defining separate security capabilities and volumes:
docker run \
--name devstack \
--privileged \
--detach \
image
To get a bash inside your new systemd container try this:
docker exec \
--tty \
--interactive \
devstack \
bash
Systemd should work inside properly configured container. You can run the container in privileged mood to run systemd.
"Systemd cannot run without SYS_ADMIN, less privileges than that won't work (see #2296 (comment)). Yes it's possible to make it "easier" (a tool that automatically sets these), but it'll still need certain privileges"
See this Github issue
After all docker is an application container and it runs the process which you specify at run time , after completing that process it will exit. May be you need an OS container or a virtual machine for your use case. See OS container vs Application Container here
In most cases the error messages comes up because an installer program has tried to run "systemctl start ". Unlike initscripts the systemctl command will not try execute the start script directly - instead it tries to contact the systemd daemon to execute the start sequence of the service. So all services have a common parent in the systemd daemon.
It can be quite overdone to run a systemd daemon inside a docker container just to start a service. You could use the systemctl-docker-replacement overwriting /usr/bin/systemctl in which case the target service is started without the help of a systemd daemon. It runs the ExecStart from the *.service file directly.
Despite Docker's Interactive tutorial and faq I lose my data when the container exits.
I have installed Docker as described here: http://docs.docker.io/en/latest/installation/ubuntulinux
without any problem on ubuntu 13.04.
But it loses all data when exits.
iman#test:~$ sudo docker version
Client version: 0.6.4
Go version (client): go1.1.2
Git commit (client): 2f74b1c
Server version: 0.6.4
Git commit (server): 2f74b1c
Go version (server): go1.1.2
Last stable version: 0.6.4
iman#test:~$ sudo docker run ubuntu ping
2013/10/25 08:05:47 Unable to locate ping
iman#test:~$ sudo docker run ubuntu apt-get install ping
Reading package lists...
Building dependency tree...
The following NEW packages will be installed:
iputils-ping
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 56.1 kB of archives.
After this operation, 143 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise/main iputils-ping amd64 3:20101006-1ubuntu1 [56.1 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 56.1 kB in 0s (195 kB/s)
Selecting previously unselected package iputils-ping.
(Reading database ... 7545 files and directories currently installed.)
Unpacking iputils-ping (from .../iputils-ping_3%3a20101006-1ubuntu1_amd64.deb) ...
Setting up iputils-ping (3:20101006-1ubuntu1) ...
iman#test:~$ sudo docker run ubuntu ping
2013/10/25 08:06:11 Unable to locate ping
iman#test:~$ sudo docker run ubuntu touch /home/test
iman#test:~$ sudo docker run ubuntu ls /home/test
ls: cannot access /home/test: No such file or directory
I also tested it with interactive sessions with the same result. Did I forget something?
EDIT: IMPORTANT FOR NEW DOCKER USERS
As #mohammed-noureldin and others said, actually this is NOT a container exiting. Every time it just creates a new container.
You need to commit the changes you make to the container and then run it. Try this:
sudo docker pull ubuntu
sudo docker run ubuntu apt-get install -y ping
Then get the container id using this command:
sudo docker ps -l
Commit changes to the container:
sudo docker commit <container_id> iman/ping
Then run the container:
sudo docker run iman/ping ping www.google.com
This should work.
When you use docker run to start a container, it actually creates a new container based on the image you have specified.
Besides the other useful answers here, note that you can restart an existing container after it exited and your changes are still there.
docker start f357e2faab77 # restart it in the background
docker attach f357e2faab77 # reattach the terminal & stdin
There are following ways to persist container data:
Docker volumes
Docker commit
a) create container from ubuntu image and run a bash terminal.
$ docker run -i -t ubuntu:14.04 /bin/bash
b) Inside the terminal install curl
# apt-get update
# apt-get install curl
c) Exit the container terminal
# exit
d) Take a note of your container id by executing following command :
$ docker ps -a
e) save container as new image
$ docker commit <container_id> new_image_name:tag_name(optional)
f) verify that you can see your new image with curl installed.
$ docker images
$ docker run -it new_image_name:tag_name bash
# which curl
/usr/bin/curl
In addition to Unferth's answer, it is recommended to create a Dockerfile.
In an empty directory, create a file called "Dockerfile" with the following contents.
FROM ubuntu
RUN apt-get install ping
ENTRYPOINT ["ping"]
Create an image using the Dockerfile. Let's use a tag so we don't need to remember the hexadecimal image number.
$ docker build -t iman/ping .
And then run the image in a container.
$ docker run iman/ping stackoverflow.com
There are really great answers above to the asked question. There might be no need for another answer but still I want to give my personal opinion on the topic in the simplest words possible.
Here are some points about containers & images that will help us for a conclusion:
A docker image can be:
created-from-a-given-container
deleted
used-to-create-any-number-of-containers
A docker container can be:
created-from-an-image
started
stopped
restarted
deleted
used-to-create-any-number-of-images
A docker run command does this:
Downloads an image or uses a cached image
Creates a new container out of it
Starts the container
When a Dockerfile is used to create an image:
It is already well known that the image will eventually be used to run a docker container.
After issuing docker build command, docker behind-the-scenes creates a running container with a base-file-system and follows steps inside the Dockerfile to configure that container as per the developers need.
After the container is configured with specs of the Dockerfile, it will be committed as an image.
The image gets ready to rock & roll!
Conclusion:
As we can see, a docker container is independent of a docker image.
A container can be restarted provided the unique ID of that container [use docker ps --all to get the id].
Any operation like making a new directory, creating files, installing tools, etc. can be done inside the container when it is running. Once the container is stopped, it persists all the changes. Container stopping and restarting is like rebooting a computer system.
An already created container is always available for a restart but when we issue docker run command, a new container is created out of an image and hence it is like a new computer system. The changes made inside the old container - as we can understand now - are not available in this new container.
A final note:
I guess it's now obvious why the data seems to be lost yet it is always there.. but in a different [old] container. So, take a good note of the difference in docker start & docker run command & never get confused in them.
I have got a much simpler answer to your question, run the following two commands
sudo docker run -t -d ubuntu --name mycontainername /bin/bash
sudo docker ps -a
the above ps -a command returns a list of all containers. Take the name of the container which references the image name - 'ubuntu' . docker auto generates names for the containers for example - 'lightlyxuyzx', that's if you don't use the --name option.
The -t and -d options are important, the created container is detached and can be reattached as given below with the -t option.
With --name option, you can name your container in my case 'mycontainername'.
sudo docker exec -ti mycontainername bash
and this above command helps you login to the container with bash shell. From this point on any changes you make in the container is automatically saved by docker.
For example - apt-get install curl inside the container
You can exit the container without any issues, docker auto saves the changes.
On the next usage, All you have to do is, run these two commands every time you want to work with this container.
This Below command will start the stopped container:
sudo docker start mycontainername
sudo docker exec -ti mycontainername bash
Another example with ports and a shared space given below:
docker run -t -d --name mycontainername -p 5000:5000 -v ~/PROJECTS/SPACE:/PROJECTSPACE 7efe2989e877 /bin/bash
In my case:
7efe2989e877 - is the imageid of a previous container running
which I obtained using
docker ps -a
You might want to look at docker volumes if you you want to persist the data in your container. Visit https://docs.docker.com/engine/tutorials/dockervolumes/. The docker documentation is a very good place to start
My suggestion is to manage docker, with docker compose. Is an easy to way to manage all the docker's containers for your project, you can map the versions and link different containers to work together.
The docs are very simple to understand, better than docker's docs.
Docker-Compose Docs
Best
the similar problem (and no way Dockerfile alone could fix it) brought me to this page.
stage 0:
for all, hoping Dockerfile could fix it: until --dns and --dns-search will appear in Dockerfile support - there is no way to integrate intranet based resources into.
stage 1:
after building image using Dockerfile (by the way it's a serious glitch Dockerfile must be in the current folder), having an image to deploy what's intranet based, by running docker run script. example:
docker run -d \
--dns=${DNSLOCAL} \
--dns=${DNSGLOBAL} \
--dns-search=intranet \
-t pack/bsp \
--name packbsp-cont \
bash -c " \
wget -r --no-parent http://intranet/intranet-content.tar.gz \
tar -xvf intranet-content.tar.gz \
sudo -u ${USERNAME} bash --norc"
stage 2:
applying docker run script in daemon mode providing local dns records to have ability to download and deploy local stuff.
important point: run script should be ending with something like /usr/bin/sudo -u ${USERNAME} bash --norc to keep container running even after the installation scripts finishes.
no, it's not possible to run container in interactive mode for the full automation matter as it will remain inside internal shall command prompt until CTRL-p CTRL-q being pressed.
no, if interacting bash will not be executed at the end of the installation script, the container will terminate immediately after finishes script execution, loosing all installation results.
stage 3:
container is still running in background but it's unclear whether container has ended installation procedure or not yet. using following block to determine execution procedure finishes:
while ! docker container top ${CONTNAME} | grep "00[[:space:]]\{12\}bash \--norc" -
do
echo "."
sleep 5
done
the script will proceed further only after completed installation. and this is the right moment to call: commit, providing current container id as well as destination image name (it may be the same as on the build/run procedure but appended with the local installation purposes tag. example: docker commit containerID pack/bsp:toolchained.
see this link on how to get proper containerID
stage 4: container has been updated with the local installs as well as it has been committed into newly assigned image (the one having purposes tag added). it's safe now to stop container running. example: docker stop packbsp-cont
stage5: any moment the container with local installs require to run, start it with the image previously saved.
example: docker run -d -t pack/bsp:toolchained
a brilliant answer here How to continue a docker which is exited from user kgs
docker start $(docker ps -a -q --filter "status=exited")
(or in this case just docker start $(docker ps -ql) 'cos you don't want to start all of them)
docker exec -it <container-id> /bin/bash
That second line is crucial. So exec is used in place of run, and not on an image but on a containerid. And you do it after the container has been started.
None of the answers address the point of this design choice. I think docker works this way to prevent these 2 errors:
Repeated restart
Partial error