This is my Dockerfile:
FROM golang
# RUN cat /etc/*release
RUN apt-get update
RUN apt-get -y install apt-transport-https ca-certificates curl gnupg2 software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
RUN apt-get update
RUN apt-get -y install docker-ce
RUN docker run hello-world
The golang Dockerfile is official, it bases on the
Debian GNU/Linux 8 (jessie)
So I wrote down this Dockerfile by checking the install steps from Docker Install Tutor(Debian)
But the output is
Step 8/8 : RUN docker run hello-world
---> Running in b183b8cc5d10
docker: Cannot connect to the Docker daemon at
unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
How to solve this problem? I want to establish docker containers in the host docker container.
I had a similar problem trying to install Docker inside a Bamboo Server image. To solve this:
first remove the line: RUN docker run hello-world from your Dockerfile
The simplest way is to just expose the Docker socket, by bind-mounting it with the -v flag or mounting a volume using Docker Compose:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
Use Docker-in-Docker for this task. They have already solved many of the problems for you.
In your .dockerfile add this line to install Docker:
RUN curl -fsSL https://get.docker.com | sh
After build is done, when running your container, add a volume mapping to the host Docker socket with the -v switch , e.g.:
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock my-container
Then, from within the container shell, check the connection by running:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8bf420851572 my-image "bash" 8 minutes ago Up 8 minutes my-container
The easiest way is to use the official Docker-in-Docker images from https://hub.docker.com/_/docker/ with the :dind tag (which is the successor of the project Hendrikvh already mentioned).
You definitely need to use the --priviledged flag also:
docker run --privileged --name yourDockerContainerNameHere -d docker:dind
With that your Docker-in-Docker experiments should work - but be aware of the many stumbleblocks that could be in your way: https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
//create container in privileged mode
sudo docker container run -it --name uob_20.04 --privileged=true <dockerhub-image> /bin/bash
//give access
sudo chmod ugo+rw /var/run/docker.sock
sudo nohup dockerd > /dev/null 2>&1 &
//check docker installation
docker images
Try with starting docker service before of executing any docker command.
Add this line
RUN bash service docker start
to your Dockerfile above of this line:
RUN docker run hello-world
Related
Installed a brand new Gitlab CE 13.9.1 on a Ubuntu Server 20.04.2.0.
This is the pipeline
image: node:latest
before_script:
- apt-get update -qq
stages:
- install
install:
stage: install
script:
- npm install --verbose
To run it I configure my Gitlab Runner using the same procedure as in my previous Gitlab CE 12:
I pull last Gitlab runner image:
docker pull gitlab/gitlab-runner:latest
First try:
Start GitLab Runner container mounting on local volume
docker run -d \
--name gitlab-runner \
--restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
And register runner
docker run --rm -t -i \
-v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
When registering runner, for executor I pick shell
Finally, when I push to Gitlab, on the pipeline, I see this error:
$ apt-get update -qq
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
ERROR: Job failed: exit status 1
Second try:
Start GitLab Runner container mounting on Docker volume
Create volume
docker volume create gitlab-runner-config
Start GitLab Runner container
docker run -d \
--name gitlab-runner \
--restart always \
-v gitlab-runner-config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
Register runner (picking shell again as executor)
docker run \
--rm -t -i \
-v gitlab-runner-config:/etc/gitlab-runner gitlab/gitlab-runner register
Same results.
$ apt-get update -qq
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
ERROR: Job failed: exit status 1
Third try:
Granting permissions to gitlab-runner
I ended up reading In gitlab CI the gitlab runner choose wrong executor and https://docs.gitlab.com/runner/executors/shell.html#running-as-unprivileged-user, which states these solutions:
move to docker
grant user gitlab-runner the permissions he needs to run specified commands. gitlab-runner may run apt-get without sudo, also he will need perms for npm install and npm run.
grant sudo nopasswd to user gitlab-runner. Add gitlab-runner ALL=(ALL) NOPASSWD: ALL (or similar) to /etc/sudoers on the machine gitlab-runner is installed and change the lines apt-get update to sudo apt-get update, which will execute them as privileged user (root).
I need to use shell
I already did that with sudo usermod -aG docker gitlab-runner
Tried as well with sudo nano /etc/sudoers, adding gitlab-runner ALL=(ALL) NOPASSWD: ALL, and using sudo apt-get update -qq in the pipeline, which results in bash: line 106: sudo: command not found
I'm pretty lost here now. Any idea will be welcome.
IMHO, using shell executor on a Docker runner with already mounted Docker socket on it is not a good idea. You'd better use docker executor, which will take care of everything and probably is how it's supposed to be run.
Edit
Alternatively, you can use a customized Docker image to allow using the shell executor with root permissions. First, you'll need to create a Dockerfile:
FROM gitlab/gitlab-runner:latest
# Change user to root
USER root
Then, you'll have to build the image (here, I tagged it as custom-gitlab-runner):
$ docker build -t custom-gitlab-runner .
Finally, you'll need to use this image:
docker run -d \
--name gitlab-runner \
--restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
custom-gitlab-runner:latest
I had a similar issue trying to use locally installed gitlab-runner on ubuntu with a shell executor (I had other issues using docker executor not being able to communicate between stages)
$ docker build -t myapp .
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=myapp&target=&ulimits=null&version=1": dial unix /var/run/docker.sock: connect: permission denied
ERROR: Job failed: exit status 1
I then printed what user was running the docker command within the gitlab-ci.yml file, which was gitlab-runner
...
build:
script:
- echo $USER
- docker build -t myapp .
...
I then added gitlab-runner to the docker group using
sudo usermod -aG docker gitlab-runner
which fixed my issue. No more docker permission errors.
I need to create a container for which I'm able to create new images.
My first guest was to run docker on docker but found that the right
way to do this was using the --privileged argument so the container
has access to the docker daemon.
For this I'm runnin the following comand:
docker run --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /home/user/container_data:/app/app -d -p 5100:5100 mcf2:latest
I'm using -v /home/user/container_data:/app/app because I'm creating the folder for the new images from
templates for flask apps and saving them on that directory.
One of the files I'm creating from the templates is 'create_image.sh' which has the docker build statement E.G.
'docker build -t new_container:latest .'
for that I'm running the following code inside the running container:
bash_path= 'app/classification_model/create_image.sh'
subprocess.call([bash_path],shell=True)
But I always get this error:
/bin/sh: 1: app/model/create_image.sh: docker: not found
But the file does exist, if do ls in the container 'app/' is in the list of folders
I have also checked the bind directory and
'/home/user/container_data/classification_model/create_image.sh'
Does exist.
I have tried changing bash_path to
bash_path= '/app/classification_model/create_image.sh'
and
bash_path= '/app/app/classification_model/create_image.sh'
But get the same error for all the cases
**EDIT: **
I have changed the Docker file to:
From docker:dind
FROM ubuntu:18.04
RUN apt-get update -y && \
apt-get install -y python3-pip python3-dev
...
...
And run again:
docker run --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /home/user/container_data:/app/app -d -p 5100:5100 mcf2:latest
I'm still getting the same error:
/bin/sh: 1: docker: not found
You are mixing two thing
Docker in Docker
Docker in Docker with host Docker Socket
In the both cases, Docker should be installed in the container, it does not mean by mounting -v /var/run/docker.sock:/var/run/docker.sock this any container will able to launch or run docker command.
In the first option, it will start containers as a child container.
In the second option, the container will have access to the Docker socket, and will, therefore, be able to start containers. Except that instead of starting “child” containers, it will start “sibling” containers.
updated:
Docker offical dind image is alpine based so you can install using apk instead of apt.
FROM docker:dind
RUN apk add --no-cache python3 python3-dev
https://pkgs.alpinelinux.org/packages
I am going to leave the question with answer to help anyone who has similar problem with configuring docker and systemd together.
How we can start and stop docker container with systemd?
First we will create docker image
FROM debian:stretch
RUN apt-get update \
&& apt-get install -y systemd \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN rm -f /lib/systemd/system/multi-user.target.wants/* \
/etc/systemd/system/*.wants/* \
/lib/systemd/system/local-fs.target.wants/* \
/lib/systemd/system/sockets.target.wants/*udev* \
/lib/systemd/system/sockets.target.wants/*initctl* \
/lib/systemd/system/sysinit.target.wants/systemd-tmpfiles-setup* \
/lib/systemd/system/systemd-update-utmp*
# systemd should be started with PID 1
CMD [ "/lib/systemd/systemd" ]
Let's build our docker image
docker build -t test_image path_to_docker_file
Now we can create and start new container
docker run --name test_container -it -d --privileged --stop-signal RTMIN+3 test_image
Or you can start existing container
docker start test_image
Now you can attach to running container to execute some bash command for example
docker exec -it test_container bash
To stop container
docker stop test_container
IMPORTANT!!!
Do not specify ENTRYPOINT when you start container it should be always SYSTEMD, if you want to execute something do it by attaching to container
docker exec -it test_container bash (or other any commands)
You cannot start systemd during Docker image build.
Solution was tested for Docker version 2.0.0.3 on macOS
Here is exactly what I need. I already have a project which is starting up a particular set of docker images and it works completely fine.
But I want to create another image, which is particularly to build this project from the scratch having all the dependencies inside. So, the problem is, when building, to create docker images, we need to access the docker daemon running on the host machine from the building container.
Is there any way of doing this?
If you need to access docker on the host from inside a container, you can simply expose the Docker socket inside the container using a host mount (-v /host/path:/container/path on the docker run command line).
For example, if I start a new fedora container exposing the docker socket on my host:
$ docker run -it -v /var/run/docker.sock:/var/run/docker.sock fedora bash
Then install docker inside the container:
[root#d28650013548 /]# yum -y install docker
...many lines elided...
I can now talk to docker on my host:
[root#d28650013548 /]# docker info
Containers: 6
Running: 1
Paused: 0
Stopped: 5
Images: 530
Server Version: 17.05.0-ce
...
You can let the container access to the host's docker daemon through the docker socket and "tricking" it to have the docker executable inside the container without installing docker inside it. Just on this way (with an Ubuntu-Xenial container for the example):
docker run --name dockerInsideContainer -ti -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker ubuntu:xenial
Inside this, you can launch any docker command like for example docker images to check it's working.
If you see an error like this: docker: error while loading shared libraries: libltdl.so.7: cannot open shared object file: No such file or directory you should install inside the container a package called libltdl7. So for example you can create a Dockerfile for the container or installing it directly on run:
FROM ubuntu:xenial
apt update
apt install -y libltdl7
or
docker run --name dockerInsideContainer -ti -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker ubuntu:xenial bash -c "apt update && apt install libltdl7 && bash"
Hope it helps
I want to install docker inside a running docker container.
docker run -it centos:centos7
My base container is using centos, I can login to running container using docker exec. But when I try to install docker inside it using yum install -y docker it installs.
But somehow I can't start the docker service with docker -d &, it gives me error as:
INFO[0000] Option DefaultNetwork: bridge
WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: , error: exit status 1
FATA[0000] Error starting daemon: Error initializing network controller: Error initializing bridge driver: Setup IP forwarding failed: open /proc/sys/net/ipv4/ip_forward: read-only file system
Is there a way I can install docker inside docker container or build image already having running docker? I have already seen these examples but none works for me.
The output of uname -r on the host machine:
[fedora# ~]$ uname -r
4.2.6-200.fc22.x86_64
Any help would be appreciated.
Thanks in advance
Update
Thanks to https://stackoverflow.com/a/38016704/372019 I want to show another approach.
Instead of mounting the host's docker binary, you should copy or install a container specific release of the docker binary. Since you're only using it in a client mode, you won't need to install it as a system service. You still need to mount the Docker socket into the container so that you can easily communicate with the host's Docker engine.
Assuming that you got a base image with a working Docker binary (e.g. the official docker image), the example now looks like this:
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
docker:1.12 docker info
Without actually answering your question I'd suggest you to read Using Docker-in-Docker for your CI or testing environment? Think twice.
It explains why running docker-in-docker should be replaced with a setup where Docker containers run as siblings of the "outer" or "base" container. The article also links to the original https://github.com/jpetazzo/dind project where you can find working examples how to run Docker in Docker - in case you still want to have docker-in-docker.
An example how to enable a container to access the host's Docker daemon look like this:
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v /usr/bin/docker:/usr/bin/docker\
busybox:latest /usr/bin/docker info
If you are on Mac with Docker toolbox.
The below command WON’T WORK
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v /usr/bin/docker:/usr/bin/docker\
busybox:latest /usr/bin/docker info
Because /var/run/docker.sock will not be on your OSX filesystem
the Docker daemon is running inside the boot2docker VM - and that's where the unix socket is.
So you have to run the container from boot2docker VM
$ docker-machine ssh default
$ docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v $(which docker):/usr/bin/docker\
busybox:latest /usr/bin/docker info
$ exit
This looks like Docker-in-Docker, feels like Docker-in-Docker, but it’s not Docker-in-Docker, when this container will create more containers, those containers will be created in the top-level Docker.
You need the --privileged parameter.
By default, Docker containers are “unprivileged” and cannot, for
example, run a Docker daemon inside a Docker container.
Source
Run your base image with the command docker run --privileged -it centos:centos7 bash. Then you may install and run another docker container inside that container.
I`ve a similar problems in my vms.
I`ve solve the problem with change the storage file system from image to vfs(in daemon.json file)
like the image bellow
For image works first create a base image, in my case with centos7
FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
with this image builded (in my case i called local/c7-systemd) create a second image, installing docker and moving daemon.json to inside.
FROM local/c7-systemd
RUN yum install -y yum-utils
RUN yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
RUN yum install -y docker-ce docker-ce-cli containerd.io
RUN curl -L "https://github.com/docker/compose/releases/download/1.28.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod +x /usr/local/bin/docker-compose
RUN ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
COPY daemon.json /etc/docker/daemon.json
RUN yum install -y nano
RUN systemctl enable docker
EXPOSE 80
EXPOSE 8080
EXPOSE 8161
EXPOSE 6379
EXPOSE 8761
CMD ["/usr/sbin/init"]
enjoy!