I am trying to build a super-simple CI/CD pipeline using GitLab CI.
Upon running it I get presented with the error:
Server:
ERROR: Cannot connect to the Docker daemon at tcp://docker:2375.
Is the docker daemon running?
My .gitlab-ci.yml is :
image: docker:latest
variables:
DOCKER_HOST: tcp://docker:2375
services:
- name: docker:dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
before_script:
- docker --version
docker_build:
stage: build
image: docker:latest
services:
- docker:dind
script:
- docker build -t arieltar/hubsec:1.1 .
- docker push arieltar/hubsec:1.1
Based on the error message I would ask, does the gitlab-runner user belong to the docker group?
You will need to decide if you want to use Docker-in-Docker with, or without TLS. This requires changing /etc/gitlab-runner/config.toml settings, and assigning the DOCKER_TLS_CERTDIR in your .gitlab-ci.yml file. See the Docker-in-docker section of the GitLab docs.
Please check below things as prelim.
Whether docker is running or not
Login with gitlab-user if you are running pipeline with gitlab user and check if that user can access or run docker ps without sudo :).
add below entry if pt1. and pt2 satisfied.
services:
name: docker:dind
entrypoint: ["dockerd-entrypoint.sh", "--tls=false"]
script:
export DOCKER_HOST=tcp://127.0.0.1:2375 && docker build -t arieltar/hubsec:1.1 .
Related
i have this pipeline to execute :
stages:
- build-gitlab
- deploy-uat
build:
image: node:14-alpine
stage: build-gitlab
services:
- docker
before_script:
- docker login $CI_REGISTRY_URL -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD
script:
- docker build --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
- docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_FRONTEND_REGISTRY_URL
- docker push $CI_FRONTEND_REGISTRY_URL
deploy:
image:
name: bitnami/kubectl:latest
stage: deploy-uat
before_script:
- kubectl config set-cluster deploy-cluster --server="$K8S_SERVER" --insecure-skip-tls-verify
- kubectl config set-credentials gitlab --token=$(echo $K8S_TOKEN | base64 -d)
- kubectl config set-context deploy-cluster --cluster=deploy-cluster --namespace=ns-frontend-dev --user=gitlab
- kubectl config use-context deploy-cluster
script:
- envsubst < deploy.tmpl > deploy.yaml
- kubectl apply -f deploy.yaml
Initially i defined a runner for my gitlab with shell executor. Docker is installed in my runner that is why the build stage executed itself successfully. But if i would like to use multiple docker images as you can see in my gitlab-ci.yaml file, the shell executor is not the appropriate one.
I saw this documentation about gitlab executors
but it is not explicit enough.
i register a new runner with docker executor, then i got this result :
Preparing the "docker" executor
Using Docker executor with image node:14-alpine ...
Starting service docker:latest ...
Pulling docker image docker:latest ...
Using docker image sha256:0f8d12a73562adf6588be88e37974abd42168017f375a1e160ba08a7ee3ffaa9 for docker:latest with digest docker#sha256:75026b00c823579421c1850c00def301a6126b3f3f684594e51114c997f76467 ...
Waiting for services to be up and running (timeout 30 seconds)...
*** WARNING: Service runner-jdn9pn3z-project-33-concurrent-0-0e760484a3d3cab3-docker-0 probably didn't start properly.
Health check error:
service "runner-jdn9pn3z-project-33-concurrent-0-0e760484a3d3cab3-docker-0-wait-for-service" health check: exit code 1
Health check container logs:
2023-01-18T15:50:31.037166246Z FATAL: No HOST or PORT found
and the deploy part did not succeed. What is the right executor to choose between :
docker, shell, ssh, kubernetes, custom, parallels, virtualbox, docker+machine, docker-ssh+machine, instance, docker-ssh
And how to use it
These are the steps I followed to setup a custom Gitlab runner:
Install Gitlab runner by following the instructions here: https://docs.gitlab.com/runner/install/linux-repository.html
Register a runner with docker exectuor and docker:19 image
Write gitlab-ci.yml as follow:
image: docker:19.03.1
services:
- name: docker:19.03.1-dind
alias: docker
stages:
- build
build:
stage: build
variables:
IMAGE_TAG: repo.azurecr.io/some-repo
DOCKER_HOST: tcp://172.17.0.1:2375
DOCKER_TLS_CERTDIR: ''
script:
- docker login someacr.azurecr.io -u "$SERVICE_PRINCIPAL_USER" -p "$SERVICE_PRINCIPAL_PASSWORD"
- if [[ "$CI_COMMIT_REF_NAME" == "develop" ]]; then docker build -t $IMAGE_TAG .; fi
Running the pipeline throws this error:
Login succeeded
if [[ "$CI_COMMIT_REF_NAME" == "develop" ]]; then docker build -t $IMAGE_TAG .; fi
Cannot connect to the Docker daemon at tcp://172.17.0.1:2375. Is the docker daemon running?
Cleaning up project directory and file based variables
00:02
ERROR: Job failed: exit code 1
This is what I have tried so far:
privileged: true in /etc/gitlab-runner/config.toml
dind:19 docker image in gitlab-ci.yml
adding volumes = ["/var/run/docker.sock /var/run/docker.sock" "/cache"] in /etc/gitlab-runner/config.toml
used DOCKER_TLS_CERTDIR: '' in gitlab-ci.yml as variable
populating DOCKER_HOST variable with DOCKER_HOST: tcp://172.17.0.1:2375
much more...
All these changes result in either of the 2 issues described in Gitlab Runner Troubleshooting here: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#troubleshooting
docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
Docker no such host error.
Your DOCKER_HOST variable value is not correct.
Use DOCKER_HOST: "tcp://docker:2375"
adding volumes = ["/var/run/docker.sock /var/run/docker.sock" "/cache"] in /etc/gitlab-runner/config.toml
I would recommend removing the volume mount of /var/run/docker.sock from the runner configuration here. This is not necessary when leveraging the docker:dind service container method. Though, it technically won't cause issues.
privileged: true in /etc/gitlab-runner/config.toml
[...]
used DOCKER_TLS_CERTDIR: '' in gitlab-ci.yml as variable
These changes are good, keep these changes.
i have two project on gitlab with same CI config file and ci variables. When i try to build dockerfile, one project passed, but second say:
Error: Cannot perform an interactive login from a non TTY device
config:
image: docker:latest
services:
- docker:dind
stages:
- build
variables:
CONTAINER_IMAGE: sleezy/go-hello-world:${CI_COMMIT_SHORT_SHA}
build:
stage: build
script:
- docker login -u ${DOCKER_USER} -p ${DOCKER_PASSWORD}
- docker build -t ${CONTAINER_IMAGE} .
- docker tag ${CONTAINER_IMAGE} ${CONTAINER_IMAGE}
- docker tag ${CONTAINER_IMAGE} sleezy/go-hello-world:latest
- docker push ${CONTAINER_IMAGE}
How i said, everything is same, variables, dockerhub account - username, password, config, even gitlab runner version, so i really dont know why? Any help, thanks.
I am trying to build my docker image within the gitlab ci pipeline.
However it is not able to find the docker command.
/bin/bash: line 69: docker: command not found ERROR: Job failed: error
executing remote command: command terminated with non-zero exit code:
Error executing in Docker Container: 1
.gitlab-ci.yml
stages:
- quality
- test
- build
- deploy
image: node:8.11.3
services:
- mongo
- docker:dind
before_script:
- npm install
quality:
stage: quality
script:
- npm run-script lint
test:
stage: test
script:
- npm run-script test
build:
stage: build
script:
- docker build -t server .
deploy:
stage: deploy
script:
- echo "TODO deploy push docker image"
you need to choose an image including docker binaries
image: gitlab/dind
services:
- docker:dind
You have 2 options to fix this. You will need to edit your config.toml file (located wherever you installed your gitlab runner).
OPTION 1
in config.toml:
privileged = true
in .gitlab-ci.yml:
myjob:
stage: myjob
image: docker:latest
services:
- docker:18.09.7-dind # older version that does not need demand TLS (see below)
OPTION 2
in config.toml:
privileged = true
volumes = ["/certs/client", "/cache"]
in .gitlab-ci.yml:
myjob:
stage: myjob
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay2 # not sure if this is needed
DOCKER_TLS_CERTDIR: "/certs"
IMPORTANT: ONCE YOU HAVE MADE THE CHANGES TO config.toml YOU WILL PROBABLY NEED TO RESTART THE GITLAB RUNNER (which may vary depending on OS) - I DID RESTART MINE, NOT SURE WHAT WOULD HAPPEN IF YOU DID NOT RESTART IT!
Instructions for restarting gitlab runner are here ... https://docs.gitlab.com/runner/commands/ ... basically gitlab-runner restart but on Windows I had to use Windows "Services" to restart it
Why this problem?
priviledged=true gets rid of the docker: command not found problem
However, docker:dind now requires TLS certs (whatever they are). If you are happy with an older docker version then you can use OPTION 1. If you want the latest you need to setup Gitlab CLI to use them which is OPTION 2. J.E.S.U.S loves you :)
For more info ... https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03
Problem here is that node docker image does not embed docker binaries.
Two possibilities :
split stages to two jobs. One using node images for quality and test, one using docker image for building and deploying. See jobs documentation.
build a custom docker image that embed both node and docker and use this image to build your repo.
Note that in both case you will have to enable docker inside your agent. See documentation.
im setting up my first GitLab Ci Pipeline including docker to run my project. I wanted to divide my pipeline into several stages so i created "build", "test", and "clean-build".
In this scenario everything works fine:
stages:
- build
- test
- clean
image: docker:latest
services:
- docker:dind
before_script:
- export RELEASE=${CI_BUILD_REF_NAME}
- docker version
build:
stage: build
tags:
- sendis-dind
script:
- echo "Hallo in Build Stage"
test:
stage: test
tags:
- sendis-dind
script:
- echo "Hallo in TEST Stage"
clean-build:
stage: clean
tags:
- sendis-dind
script:
- echo "Hallo beim Clean Up"
when: always
All 3 stages are run successfully
but this fails:
stages:
- build
- test
- clean
image: docker:latest
services:
- docker:dind
before_script:
- export RELEASE=${CI_BUILD_REF_NAME}
- docker version
build:
stage: build
tags:
- sendis-dind
script:
- apk add --update py-pip
- pip install docker-compose
- docker --version
- docker-compose --version
- docker-compose -p ${RELEASE} build
- docker-compose -p ${RELEASE} up -d
test:
stage: test
tags:
- sendis-dind
script:
- docker exec ${RELEASE}_phpfpm_1 bash -c "cd /app; composer install; make runTests"
clean-build:
stage: clean
tags:
- sendis-dind
script:
- docker-compose -p ${RELEASE} down --volumes
when: always
with the following message from second stage
Running with gitlab-ci-multi-runner 9.1.1 (6104325)
on sendis-dind-runner (8b9eca1e)
Using Docker executor with image docker:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image docker:dind ID=sha256:559dd16b4e0a64d9de2447d3de234743046443f770bf5226f45f9b7f9c68887b for docker service...
ERROR: Preparation failed: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Will be retried in 3s ...
Using Docker executor with image docker:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image docker:dind ID=sha256:559dd16b4e0a64d9de2447d3de234743046443f770bf5226f45f9b7f9c68887b for docker service...
ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-8b9eca1e-project-140-concurrent-0-docker" is already in use by container "db166f7894856c245c6a4f5318326c5f3b6ab82d82157961d18b079444153113". You have to remove (or rename) that container to be able to reuse that name.
Will be retried in 3s ...
Using Docker executor with image docker:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image docker:dind ID=sha256:559dd16b4e0a64d9de2447d3de234743046443f770bf5226f45f9b7f9c68887b for docker service...
ERROR: Preparation failed: Error response from daemon: Conflict. The container name "/runner-8b9eca1e-project-140-concurrent-0-docker" is already in use by container "db166f7894856c245c6a4f5318326c5f3b6ab82d82157961d18b079444153113". You have to remove (or rename) that container to be able to reuse that name.
Will be retried in 3s ...
ERROR: Job failed (system failure): Error response from daemon: Conflict. The container name "/runner-8b9eca1e-project-140-concurrent-0-docker" is already in use by container "db166f7894856c245c6a4f5318326c5f3b6ab82d82157961d18b079444153113". You have to remove (or rename) that container to be able to reuse that name.
Different stages only share artifacts with each other, but they're separate docker containers. That means that if you run docker-compose up -d in your build stage, the containers are not running in the test stage.
Combining dind with gitlab-ci is only necessary in very specific use-cases. In your case, you don't need dind at all. You can simply use the php-fpm image in your test step, since gitlab-ci is already running on docker.
test:
stage: test
image: <your php-fpm image here>
script:
- cd /app
- composer install
- make runTests