I have GitLab server and a separate docker-registry. I have created a testuser with a testpassword.
My .gitlab-ci.yml
image: docker:latest
services:
- docker:18.09.7-dind
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
# See https://github.com/docker-library/docker/pull/166
DOCKER_TLS_CERTDIR: ""
REGISTRY: myregistry:5000/golang-test
stages:
- build
# - test
before_script:
- docker login -u testuser -p testpassword myregistry:5000
build_project:
stage: build
tags:
- docker-ci
script:
- docker ps
- docker build -t $REGISTRY .
- docker push $REGISTRY
When CI pipeline runs, I get following error.
WARNING! Using --password via the CLI is insecure. Use
--password-stdin. Error response from daemon: Get https://myregistry:5000/v2/: dial tcp: lookup worker1 on
192.168.123.456:53: no such host ERROR: Job failed: exit code 1
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-ci"
url = "https://gitlab.sample.com:8000/"
token = "89MfxRTby83ny3eTkTf2"
tls-ca-file = "/etc/gitlab-runner/certs/gitlab.sample.com.crt"
executor = "docker"
pre_build_script = "export DOCKER_HOST=tcp://docker:2375"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.docker]
tls_verify = false
image = "ubuntu:18.04"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
This was after suggested modifications here.
I can successfully login to my docker-registry using docker login -u testuser -p testpassword https://myregistry:5000 from gitlab running server. refered
After uwe-plonuscomment, I have added following part, but still same error.
before_script:
- echo "192.168.10.219 worker1" >> /etc/hosts
- docker login -u testuser -p testpassword myregistry:5000
Related
How to configure gitlab-runner config.toml and .gitlab-ci.yml to run 'mvn test' with the need of Docker daemon running in the same stage ? I need this to make testcontainer work in CI.
So far I have solved this by creating an image called 'ci-server' with both maven and Docker. I want to solve this following the approach described in Testcontainers website, in this link, but none of these examples worked for me.
gitlab-ci.yml (example)
variables:
DOCKER_HOST: "tcp://docker:2375"
DOCKER_TLS_CERTDIR: ""
services:
- docker:dind
stages:
- clean-build-test
maven-build-test:
image: ci-server:1.0
stage: clean-build-test
tags:
- xxx
script:
- mvn $MAVEN_CLI_OPTS clean package -Dspring.profiles.active=test
gitlab-runner configuration:
[[runners]]
name = "xxx"
url = "xxx"
id = 99
token = "xxx"
token_obtained_at = 2023-01-18T21:24:45Z
token_expires_at = 0001-01-01T00:00:00Z
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
MaxUploadedArchiveSize = 0
[runners.cache.s3]
[runners.cache.gcs]
[runners.docker]
tls_verify = false
image = "ci-server:1.0"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
pull_policy = "if-not-present"
Been trying to build a simple gitlab CI pipeline which builds an image and pushes it to Google container repository. I am running through this error -
ERROR: error during connect: Get "http://docker:2375/v1.24/info": dial
tcp: lookup docker on 169.254.169.254:53: no such host
I have tried all the solutions posted across gitlab issues threads but no help. I am using public runners, it's a pretty simple ci script.
image: docker:latest
variables:
GCR_IMAGE: <GCR_IMAGE>
services:
- docker:dind
build:
stage: build
before_script:
- docker info
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Relevant issue thread: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4794
Using gitlab-runner 15.7.1
A few weeks ago I encountered this problem and was able to solve it with this method:
image:
name: docker:20.10.16
services:
- name: docker:20.10.16-dind
variables:
DOCKER_HOST: tcp://docker:2376/
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
before_script:
- until docker info; do sleep 1; done
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Also add this configuration to runner
[[runners]]
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "ubuntu:20.04"
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
I am using docker executor on gitlab runner as an image I am using docker:stable
I am getting this error:
$ docker login gitlab.mydomain.com:5050 -u myusername-p mytoken
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on 67.207.67.3:53: no such host
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
Here is my config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "dev-env"
url = "https://gitlab.mydomain.com/"
token = "REDACTED"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker:stable"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
Here is my ci file
deploy-dev:
stage: deploy
before_script:
- apk add make
script:
- docker login gitlab.mydomain.com:5050 -u myusnerma -p mytoken
only:
- mybranch
tags:
- dev
You must add the docker:dind service to your job configuration:
deploy-dev:
variables:
# these values may need to be different if using TLS, k8s, etc.
# You can alternatively set defaults in your runner config
DOCKER_TLS_CERTDIR: ""
DOCKER_HOST: "tcp://docker:2375"
services:
- docker:dind
# ...
For the dind container to work, your runner must allow privileged containers:
[runners.docker]
privileged = true
# ...
Beside privileged = true you should also map /var/run/docker.sock inside docker container to /var/run/docker.sock on host. something like this:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "my-runner"
url = "https://gitlab.mydomain.com/"
token = "GITLAB_RUNNER_TOKEN"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker:stable"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
shm_size = 0
More info from techoverflow
Ive an issue with gitlab-runner executor docker. After I ran my gitlab-ci.yml file , pipeline fail on step docker info during before_script with:
Running with gitlab-runner 13.10.0 (54944146)
on docker-runner N2_yEgUD
Preparing the "docker" executor 00:07
Using Docker executor with image docker:19.03.0 ...
Starting service docker:19.03.0-dind ...
Pulling docker image docker:19.03.0-dind ...
Using docker image sha256:fd0c64832f7e46b63a180e6000dbba7ad7a63542c5764841cba73429ba74a39e for docker:19.03.0-dind with digest docker#sha256:442ac4b31375cbe617f31759b5199d240f11d5f430e54946575b274b2fb6f096 ...
Waiting for services to be up and running...
.............................................................................................
$ docker info
Client:
Debug Mode: false
Server:
ERROR: error during connect: Get https://docker:2375/v1.40/info: dial tcp: lookup docker on 127.0.0.53:53: server misbehaving
errors pretty printing info
Cleaning up file based variables 00:01
ERROR: Job failed: exit code 1
I did a research on stack and official gitlab forum but none of the answers fix my issue:
add to .toml -> volume: ['/certs/client']
run against old: docker:18.x.x / docker:18.x.x -dind | docker:stable / docker:dind
run with: DOCKER_TLS_CERTDIR:""
run with/without:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
add endpoint to service:
services:
- name: docker:dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
Content of gitlab-runner toml
concurrent = 1
check_interval = 0
log_level = "debug"
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner"
url = "xxxxxxxx"
token = "xxxxxxx"
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/certs/client"]
network_mode = "host"
Content of gitlab-ci.yml
image: docker:19.03.0
services:
- docker:19.03.0-dind
stages:
- build
- test_framework
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
CONTAINER_TEST_IMAGE: xxxx
CONTAINER_RELEASE_IMAGE: xxxx
before_script:
- docker info
- docker login -u xxxx -p $CI_JOB_TOKEN xxxx
build:
stage: build
tags:
- adm-docker
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
image: $CONTAINER_TEST_IMAGE
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
Only one thing fix issue (workaround issue). When I add to .toml
volume: ["/var/run/docker.sock:/var/run/docker.sock"]
But after that Iam loosing DIND possibility to run my gitlab-ci.yml with different image for test stage (without using under script: -docker run MY_IMAGE python3....).
Which is not what I want
gitlab-runner under Ubuntu20 / Docker version 20.10.5, build 55c4c88
Ive worked with very similar gitlab-ci.yml around 1Yr ago and there was no issue with docker executor
Any ideas/suggestions ?
I was able to fix issue by changing flow of my gitlab-ci.yml
image: docker:19.03.5
services:
- docker:19.03.5-dind
stages:
- build
- test_framework
- release
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
CONTAINER_TEST_IMAGE: xxxxx
CONTAINER_RELEASE_IMAGE: xxxxx
build:
stage: build
tags:
- adm-docker
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
image: $CONTAINER_TEST_IMAGE
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
and toml
[[runners]]
name = "docker-runner"
url = xxxxx
token = xxxxx
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
network_mode = "host"
issue was fixed by volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
and issue with test stage was cased by:
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
in root structure of .yml file. I had to move it to build stage
I hope that will help ppl in the future
I have /etc/gitlab-runner/config.toml
[[runners]]
name = "My Docker Runner"
url = "###"
token = "###"
executor = "docker"
pre_clone_script = "git config --global http.sslVerify false"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "tiangolo/docker-with-compose"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
shm_size = 0
[runners.cache]
Insecure = false
[runners.cache.s3]
[runners.cache.gcs]
.gitlab-ci.yml
stages:
- test
before_script:
- apk update
- apk upgrade
- apk add make
test:
stage: test
tags: [test]
script:
- make prod-up # docker-compose up -d --build
- make composer-init-dev
- make test
When I run the script in gitlab, I get the error:
$make prod-up
docker-compose -f docker-compose.prod.yml up -d --build
Couldn't connect to Docker daemon at http://docker:2375 - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
make: *** [Makefile:16: docker-prod-up] Error 1
ERROR: Job failed: exit code 2
Please help, what am I doing wrong?
You're missing Docker service (and it's best to specify image explicitly):
stages:
- test
before_script:
- apk update
- apk upgrade
- apk add make
test:
stage: test
image: tiangolo/docker-with-compose:latest
services:
- docker:dind
tags: [test]
script:
- make prod-up # docker-compose up -d --build
- make composer-init-dev
- make test
Try mounting docker.sock:
test:
stage: test
tags: [test]
script:
- make prod-up # docker-compose up -d --build
- make composer-init-dev
- make test
volumes:
- /var/run/docker.sock:/var/run/docker.sock