Gitlab pipeline docker build stage - docker

Question

I'm trying to implement a docker build stage into my gitlab pipeline. Ideally I'd be able to build the image first and then run later stages against a container. However, when I use the build step in the gitlab-ci.yaml below I sometimes get the error:
error during connect: Get http://docker:2375/v1.37/info: dial tcp: lookup docker on 10.51.240.10:53: no such host
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1
On the occasions when it doesn't work I can only guess that the docker deamon isn't ready for connections. Does anyone have any ideas on how get this going?
gitlab-ci.yaml:
stages:
- build
build:
image: docker:18
services:
- docker:18-dind
variables:
DOCKER_DRIVER: overlay
DOCKER_HOST: tcp://docker:2375/
before_script:
- docker info
script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
- docker pull ${CI_REGISTRY_IMAGE}:latest || echo Could not fetch ${CI_REGISTRY_IMAGE}:latest from registry
- docker pull ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG} || echo Could not fetch ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG} from registry
- docker build -t ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG} --cache-from ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG} .
- docker push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG}
- if [[ ${CI_COMMIT_REF_SLUG} == "master" ]] ; then docker tag ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_SLUG} ${CI_REGISTRY_IMAGE}:latest ; fi
- if [[ ${CI_COMMIT_REF_SLUG} == "master" ]] ; then docker push ${CI_REGISTRY_IMAGE}:latest ; fi
stage: build
UPDATE: I am using a kubernetes executor.

Answer 1

Are you using gitlab-runner to run jobs? If so, are you setting --executor docker registering your runner?
Are you passing /var/run/docker.sock as a volume to the runner?
I have docker images building in GitLab CI working without any problems. I'm not passing any DOCKER_DRIVER or DOCKER_HOST variables.

Answer 2

We've seen this EXACT same error with nearly all the same conditions (we specify localhost instead of docker for DOCKER_HOST, and we use overlay2 instead of plain
overlay). We're seeing it on the Docker executor mainly; for now we're just chalking it up to the executor being flakey, because we retry the job and it succeeds. We've checked and there was no previous contention for the runner, so we couldn't find any explanation.

Answer 3

In the end we self hosted a runner on kubernetes that was deployed with the helm chart and set it going in privileged mode.
We also switched to overlay2

Answer 4

In https://docs.gitlab.com/ce/ci/docker/using_docker_build.html#use-docker-in-docker-executor they say that kubernetes should have tcp://localhost:2375 as DOCKER_HOST:
# Note that if you're using Kubernetes executor, the variable should be set to
# tcp://localhost:2375 because of how Kubernetes executor connects services
# to the job container
DOCKER_HOST: tcp://docker:2375/