I am trying to run a process in gitlab ci that mimics the clients use case to make sure our modifications do not disrupt their use case. This is the specific job that is failing.
docker-source:
stage: build
image: carlallen/docker:buildx
services:
- name: docker:dind
command: ["dockerd", "--host=tcp://0.0.0.0:2375"]
alias: 'docker'
script:
- echo "Building..."
- docker --version
- docker buildx
- docker buildx create --use --config buildkit.toml --driver-opt network=host --buildkitd-flags '--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host' --name test_name
- docker run -d -p 5000:5000 --restart=always --name registry registry:2
- ./build-docker.sh
$ docker --version
Docker version 19.03.14, build 5eb3275
$ docker buildx
Usage: docker buildx [OPTIONS] COMMAND
Build with BuildKit
Options:
--builder string Override the configured builder instance
Management Commands:
imagetools Commands to work on images in registry
Commands:
bake Build from a file
build Start a build
create Create a new builder instance
du Disk usage
inspect Inspect current builder instance
ls List builder instances
prune Remove build cache
rm Remove a builder instance
stop Stop builder instance
use Set the current builder instance
version Show buildx version information
Run 'docker buildx COMMAND --help' for more information on a command.
$ docker buildx create --use --config buildkit.toml --driver-opt network=host --buildkitd-flags '--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host' --name test_name
test_name
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
docker: error during connect: Post http://docker:2375/v1.40/containers/create?name=registry: dial tcp: lookup docker on XXX.XX.X.X:53: no such host.
See 'docker run --help'.
Thank you for the help!
Do not override the command or entrypoint for the docker:dind image. Use environment variables to control the behavior.
variables:
DOCKER_HOST: 'docker'
DOCKER_TLS_CERTDIR: "" # disable tls, force use of port 2375
services:
- docker:dind
script:
- docker info # verify connection/server details
If this doesn't work, then you are probably using a self-hosted runner that is not configured correctly for use with docker-in-docker. You should follow the docker in docker guide and make sure you runner is setup according to the documentation.
Related
For CI/CD purposes, I need docker in docker:
docker network create some-network
docker volume create some-docker-certs-ca
docker volume create some-docker-certs-client
docker run --privileged --name some-dind -d --network some-network --network-alias docker -e DOCKER_TLS_CERTDIR=/certs -v some-docker-certs-ca:/certs/ca -v some-docker-certs-client:/certs/client docker:dind
Now if I run the docker:latest image with -it option, I can use docker in the container as expected:
docker run --rm -it --network some-network -e DOCKER_TLS_CERTDIR=/certs -v some-docker-certs-client:/certs/client:ro docker:latest sh
/ # docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
/ #
But I need to customize docker:latest image in a Dockerfile:
FROM docker:latest
# install package1
# install package2
# install package3
...
# install package4
And build the customized docker image:
docker build -t customized-docker .
But when I run the customized-docker image with -it options, It cannot connect to docker daemon:
docker run --rm -it --network some-network -e DOCKER_TLS_CERTDIR=/certs -v some-docker-certs-client:/certs/client:ro customized-docker:latest sh
/ # docker ps
Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
/ #
What is the problem? Isn't there any way I can use customized docker image?
Edit:
I found a better way to do my CI/CD without "docker in docker". As David Maze mentioned, most of the times, there is no need to use "docker in docker". But my question still remains: Why can the docker:latest container connect to docker daemon but a customized docker container cannot.
I'm setting up a new appveyor server and trying to build docker images with using the docker build feature. But when I try to run docker commands in my custom build container, got the error that shown below.
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I think the appveyor server should run our custom build containers with the volume option that point the docker.sock.
sudo docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker name-of-the-custom-image bash
You can modify Docker cloud settings under Account -> Build environment and put the following into Custom Docker command arguments:
-v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker
https://help.appveyor.com/discussions/problems/24364-how-to-run-docker-command-in-docker-container-for-appveyor-server-docker-builds
I have a simple project setup in Gitlab CI/CD using Docker to serve the site on a Container following this guide. But I get "Container already in use..." error whenever there is a new job running on a push event. How do I "push" the new code to my already running website without taking it down or killing the container?
# .gitlab-ci.yml
stages:
- build
job 1:
stage: build
tags:
- windows-test
script:
- docker build -t vuejs-cookbook/dockerize-vuejs-app .
- docker run -p 8080:80 --rm --name dockerize-vuejs-app-1 vuejs-cookbook/dockerize-vuejs-app
The container name is the same every time. Stop and remove the old container first.
Run docker stop dockerize-vuejs-app-1 and docker rm dockerize-vuejs-app-1 after docker build.
Beside that I would suggest to run your container detached (-d) with --restart always (docs).
docker build -t vuejs-cookbook/dockerize-vuejs-app .
docker stop dockerize-vuejs-app-1
docker rm dockerize-vuejs-app-1
docker run -p 8080:80 -d --restart always --name dockerize-vuejs-app-1 vuejs-cookbook/dockerize-vuejs-app
I'm testing gitlab-ci and trying to generate an image on the registry from the Dockerfile.
I have the same code just to test:
#gitlab-ci
image: docker:latest
tages:
- build
- deploy
build_application:
stage: build
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . -f Dockerfile
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA-test
output:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
docker is running the image is being pulled but I can not execute docker commands.
In my local environment if a run:
docker run -it docker:latest
I stay inside the container and run docker info i have the same problem. I had to fix it by running the container on this way:
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:latest
but I do not know how to fix it on gitlab-ci. I configured my runner so:
docker run -d --name gitlab-runner --restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
Maybe someone can put me in the right direction.
thanks
By default it is not possible to run docker-in-docker (DIND) (as a security measure).
This section in the Gitlab docs is your solution. You must use Docker-in-Docker.
After configuring your runner to use DIND your .gitlab-ci.yml will look like this:
#gitlab-ci
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
services:
- docker:dind
before_script:
- docker info
stages:
- build
- deploy
build_application:
stage: build
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA . -f Dockerfile
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA-test
The dind (Docker IN Docker) pulled from DockerHub works fine, but I cannot build the dind from scratch.
I tried to build as follows.
My Docker version is 1.1.
$ git clone https://github.com/docker-library/docker
$ cd docker/
$ cd 1.1/
$ docker build -t docker:dind .
I could create a "dind" Docker image. After that I tried to run.
$ docker run -it --privileged --name test -d docker:dind
52e590b6636b3726bbe9774627f4424c2b9f8958a745d57c27d04cbec77a2d7b
$ docker run -it --rm --link test:docker docker run -it ubuntu bash
docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
See 'docker run --help'.
The dind did not work well.
So, I tried to customize from the dind pulled from Docker Hub.
$ docker pull docker:dind
$ docker run -it docker:dind ash
/ # exit
$ docker commit d508c2fd7131 docker:dind
sha256:f20e0314f996fe9f66806df47c1bdff956c84d11a6bfe2ff66279bee968323ec
$ docker run -it --privileged --name test -d docker:dind
d877c1993275fd4039b749f52d60a3095d40d52e13255c4fd88a319ca7ec306a
$ docker run -it --rm --link test:docker docker run -it ubuntu bash
docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
See 'docker run --help'.
It also had same problem. I just run the "dind" on Docker and exit immediately.
I cannot understand why I cannot customize the "dind" image.
Please tell me how to fix this problem.
Try this.
docker run --privileged -d --name test docker:dind
docker exec -it test docker version
docker commit test mydind
You can use mydind image