I'm trying to create a Windows Docker container using Kaniko/Gitlab.
Here is the Error I see:
Resolving secrets
00:00
Preparing the "docker-windows" executor
Using Docker executor with image gcr.io/kaniko-project/executor:v1.6.0-debug ...
Pulling docker image gcr.io/kaniko-project/executor:v1.6.0-debug ...
WARNING: Failed to pull image with policy "always": no matching manifest for windows/amd64 10.0.17763 in the manifest list entries (docker.go:147:0s)
ERROR: Preparation failed: failed to pull image "gcr.io/kaniko-project/executor:v1.6.0-debug" with specified policies [always]: no matching manifest for windows/amd64 10.0.17763 in the manifest list entries (docker.go:147:0s)
For .gitlab-ci.yml file:
image:
name: microsoft/iis:latest
entrypoint: [""]
.build_variables: &build_variables
TAG: "docker-base-windows-2019-std-core"
AWS_ACCOUNT: "XXXXXXXXXX"
AWS_REGION: "XXXXXXX"
REGISTRY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
.build_script: &build_script
script:
- echo "{\"credsStore\":\"ecr-login\"}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $REGISTRY:$TAG
stages:
- build-docker-image
build_image_dev:
variables:
<<: *build_variables
stage: build-docker-image
image:
name: gcr.io/kaniko-project/executor:v1.6.0-debug
entrypoint: [""]
tags: ['XXXXX']
<<: *build_script
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == "main"'
- if: $CI_COMMIT_TAG
This is normal text Code for Docker file:
FROM Microsoft/iis:latest
CMD [ "cmd" ]
You have the error:
no matching manifest for windows/amd64
which means that particular image could not be found. It happens if you develop on windows and your server is a linux for instance.
This error implies your host machine's OS is not compatible with the OS docker image you are trying to pull.
Related
Been trying to build a simple gitlab CI pipeline which builds an image and pushes it to Google container repository. I am running through this error -
ERROR: error during connect: Get "http://docker:2375/v1.24/info": dial
tcp: lookup docker on 169.254.169.254:53: no such host
I have tried all the solutions posted across gitlab issues threads but no help. I am using public runners, it's a pretty simple ci script.
image: docker:latest
variables:
GCR_IMAGE: <GCR_IMAGE>
services:
- docker:dind
build:
stage: build
before_script:
- docker info
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Relevant issue thread: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4794
Using gitlab-runner 15.7.1
A few weeks ago I encountered this problem and was able to solve it with this method:
image:
name: docker:20.10.16
services:
- name: docker:20.10.16-dind
variables:
DOCKER_HOST: tcp://docker:2376/
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
before_script:
- until docker info; do sleep 1; done
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Also add this configuration to runner
[[runners]]
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "ubuntu:20.04"
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
On self hosted Gitlab on GCP installed by helm, I use Gitlab-runner.
On gitlab-runner I need to use docker so using dind, but I got error
tcp://docker:2375. Is the docker daemon running?
gitlab-runner deployment
...
spec:
containers:
- command:
- /bin/bash
- /scripts/entrypoint
env:
- name: CI_SERVER_URL
value: https://my-gitlab.com
- name: CLONE_URL
- name: RUNNER_REQUEST_CONCURRENCY
value: "1"
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
- name: KUBERNETES_IMAGE
- name: KUBERNETES_PRIVILEGED
value: "true" # <= set privileged true to use dind
...
gitlab-ci.yaml
services:
- docker:20.10.4-dind
stages:
- build
variables:
GIT_SSL_NO_VERIFY: "1"
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ''
DOCKER_HOST: tcp://docker:2375
image:
name: google/cloud-sdk:latest
before_script:
- docker version
build:
stage: build
script:
- echo hello
gitlab-runner log
Executing "step_script" stage of the job script
00:00
$ docker version
Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
Client: Docker Engine - Community
Version: 19.03.11
API version: 1.40
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:09:53 2020
OS/Arch: linux/amd64
Experimental: false
Cleaning up file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
troubleshooting says that it's because of TLS. So I set DOCKER_TLS_CERTDIR: '' , the way written in another document.
Also, this problem didn't happen when I used docker:19.03.0-dind. From 19.03.0-dind, TLS is automatically. So disable TLS configuration must be worked correctly.
(docker:19.3.13-dind also worked well.)
I don't know why from docker:20 this error showed up. Has anyone already tried gitlab-runner with grater than docker:20 ?
I figured out that I should follow https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-enabled-in-kubernetes
toml
runners:
config: |
[[runners]]
[runners.kubernetes]
image = "ubuntu:20.04"
privileged = true
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
gitlab-ci.yaml
services:
- docker:20.10.4-dind
stages:
- build
variables:
GIT_SSL_NO_VERIFY: "1"
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
image:
name: google/cloud-sdk:latest
before_script:
- docker version
build:
stage: build
script:
- echo hello
I am trying to build and push a docker image to ACR using Azure DevOps pipelines. I have to build it with a docker-compose.yml file to be able to use openvpn in the container.
When I run the pipeline I get the following error. Does anyone have an idea of how to solve this?
Starting: DockerCompose
==============================================================================
Task : Docker Compose
Description : Build, push or run multi-container Docker applications. Task can be used with Docker or Azure Container registry.
Version : 0.183.0
Author : Microsoft Corporation
Help : https://aka.ms/azpipes-docker-compose-tsg
==============================================================================
/usr/local/bin/docker-compose -f /home/vsts/work/1/s/src/docker-compose.yml -f /home/vsts/agents/2.188.2/.docker-compose.1624362077551.yml -p Compose up -d
Creating network "composeproject_default" with the default driver
Pulling getstatus (***/getstatus:)...
Head https://***/v2/getstatus/manifests/latest: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.
##[error]Creating network "composeproject_default" with the default driver
##[error]Pulling getstatus (***/getstatus:)...
##[error]Head https://***/v2/getstatus/manifests/latest: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.
##[error]The process '/usr/local/bin/docker-compose' failed with exit code 1
Finishing: DockerCompose
My azure-pipelines.yml look like this:
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- main
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: '*****************************'
imageRepository: 'getstatus'
containerRegistry: 'composeproject.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- task: DockerCompose#0
inputs:
containerregistrytype: 'Azure Container Registry'
dockerComposeFile: '**/docker-compose.yml'
action: 'Run a Docker Compose command'
dockerComposeCommand: 'up -d'
And the docker-compose.yml like this:
version: "3.3"
services:
getstatus:
image: composeproject.azurecr.io/getstatus
restart: always
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun
volumes:
- /etc/timezone:/etc/timezone:ro
I think your docker compose task is missing a couple of parameters
try adding azureContainerRegistry: composeproject.azurecr.io
and azureSubscriptionEndpoint: $(dockerRegistryServiceConnection)
Not sure why the credentials supplied in the Docker#2 task don't persist since they're in the same stage but then I could fill an encyclopedia with what I'm not sure on when it comes to Azure pipelines
Ive an issue with gitlab-runner executor docker. After I ran my gitlab-ci.yml file , pipeline fail on step docker info during before_script with:
Running with gitlab-runner 13.10.0 (54944146)
on docker-runner N2_yEgUD
Preparing the "docker" executor 00:07
Using Docker executor with image docker:19.03.0 ...
Starting service docker:19.03.0-dind ...
Pulling docker image docker:19.03.0-dind ...
Using docker image sha256:fd0c64832f7e46b63a180e6000dbba7ad7a63542c5764841cba73429ba74a39e for docker:19.03.0-dind with digest docker#sha256:442ac4b31375cbe617f31759b5199d240f11d5f430e54946575b274b2fb6f096 ...
Waiting for services to be up and running...
.............................................................................................
$ docker info
Client:
Debug Mode: false
Server:
ERROR: error during connect: Get https://docker:2375/v1.40/info: dial tcp: lookup docker on 127.0.0.53:53: server misbehaving
errors pretty printing info
Cleaning up file based variables 00:01
ERROR: Job failed: exit code 1
I did a research on stack and official gitlab forum but none of the answers fix my issue:
add to .toml -> volume: ['/certs/client']
run against old: docker:18.x.x / docker:18.x.x -dind | docker:stable / docker:dind
run with: DOCKER_TLS_CERTDIR:""
run with/without:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
add endpoint to service:
services:
- name: docker:dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
Content of gitlab-runner toml
concurrent = 1
check_interval = 0
log_level = "debug"
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner"
url = "xxxxxxxx"
token = "xxxxxxx"
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/certs/client"]
network_mode = "host"
Content of gitlab-ci.yml
image: docker:19.03.0
services:
- docker:19.03.0-dind
stages:
- build
- test_framework
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
CONTAINER_TEST_IMAGE: xxxx
CONTAINER_RELEASE_IMAGE: xxxx
before_script:
- docker info
- docker login -u xxxx -p $CI_JOB_TOKEN xxxx
build:
stage: build
tags:
- adm-docker
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
image: $CONTAINER_TEST_IMAGE
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
Only one thing fix issue (workaround issue). When I add to .toml
volume: ["/var/run/docker.sock:/var/run/docker.sock"]
But after that Iam loosing DIND possibility to run my gitlab-ci.yml with different image for test stage (without using under script: -docker run MY_IMAGE python3....).
Which is not what I want
gitlab-runner under Ubuntu20 / Docker version 20.10.5, build 55c4c88
Ive worked with very similar gitlab-ci.yml around 1Yr ago and there was no issue with docker executor
Any ideas/suggestions ?
I was able to fix issue by changing flow of my gitlab-ci.yml
image: docker:19.03.5
services:
- docker:19.03.5-dind
stages:
- build
- test_framework
- release
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
CONTAINER_TEST_IMAGE: xxxxx
CONTAINER_RELEASE_IMAGE: xxxxx
build:
stage: build
tags:
- adm-docker
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
image: $CONTAINER_TEST_IMAGE
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
and toml
[[runners]]
name = "docker-runner"
url = xxxxx
token = xxxxx
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
network_mode = "host"
issue was fixed by volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
and issue with test stage was cased by:
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
in root structure of .yml file. I had to move it to build stage
I hope that will help ppl in the future
I am trying to build CI with gitlab, I go from a docker image of docker, and i didn't have any problem with my front repository, but now with the back withe the same gitlab-ci configuration file, i have this daemon error.
Here is the output of the build :
[0KRunning with gitlab-ci-multi-runner 1.10.4 (b32125f)[0;m
[0;m[0KUsing Docker executor with image docker:1.13.1 ...
[0;m[0KPulling docker image docker:1.13.1 ...
[0;mRunning on runner-4e4528ca-project-1649638-concurrent-0 via runner-4e4528ca-machine-1487688057-7c0f1e46-digital-ocean-4gb...
[32;1mCloning repository...[0;m
Cloning into '/builds/***/formation-back'...
[32;1mChecking out af7cbcae as docker...[0;m
[32;1mSkipping Git submodules setup[0;m
[32;1m$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com[0;m
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[31;1mERROR: Build failed: exit code 1
[0;m
Here is my .gitlab-ci.yml :
image: docker:1.13.1
stages:
- build
- test
- deploy
variables:
BUILD_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF
TEST_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF_NAME
RELEASE_IMG: $CI_REGISTRY_IMAGE:latest
AWS_STAGING_ENV: "***"
AWS_PROD_ENV: "***"
DOCKERRUN: Dockerrun.aws.json
DEPLOY_ARCHIVE: ${AWS_APP}-${CI_BUILD_REF}.zip
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- .ci/before_script
build:
stage: build
script:
- docker build --pull -t $BUILD_IMG .
- docker push $BUILD_IMG
test:
stage: test
script:
- docker pull $BUILD_IMG
- docker run --rm $BUILD_IMG npm run test
- docker tag $BUILD_IMG $TEST_IMG
- docker push $TEST_IMG
deploy:staging:
stage: deploy
environment: Staging
variables:
DOCKER_IMG: ${CI_REGISTRY_IMAGE}:${CI_BUILD_REF}
script:
- ./.ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
- ./.ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_STAGING_ENV
artifacts:
paths:
- $DEPLOY_ARCHIVE
except:
- production
deploy:production:
stage: deploy
environment: Production
variables:
DOCKER_IMG: ${CI_REGISTRY_IMAGE}:latest
script:
- .ci/push-new-image $TEST_IMG $RELEASE_IMG
- .ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
- .ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_PROD_ENV
artifacts:
paths:
- $DEPLOY_ARCHIVE
only:
- production
when: manual
Here is my config.toml file :
concurrent = 1
check_interval = 0
[[runners]]
name = "***"
url = "https://gitlab.com/ci"
token = "750c63cba1c269d789bdb33c42b726"
executor = "docker"
[runners.docker]
tls_verify = false
image = "alpine:3.5"
privileged = true
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
[runners.cache]
Here is docker info :
DEBU[0771] Calling GET /v1.24/info
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.12.6
Storage Driver: devicemapper
Pool Name: docker-202:1-395267-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 519 MB
Data Space Total: 107.4 GB
Data Space Available: 6.569 GB
Metadata Space Used: 1.397 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.146 GB
Thin Pool Minimum Free Space: 10.74 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.93-RHEL7 (2015-01-28)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge overlay null host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 4.4.44-39.55.amzn1.x86_64
Operating System: Amazon Linux AMI 2016.09
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.2 MiB
Name: ip-172-31-30-143
ID: D6DU:OBWL:R3HK:DSZK:EOYC:5EHS:NU4I:4M3T:H5PL:JWLH:CIPD:I7VW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 20
Goroutines: 27
System Time: 2017-02-22T11:16:19.042666914Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Insecure Registries:
127.0.0.0/8
You need to add
services:
- docker:dind
to your .gitlab-ci.yml. This tells the runner to start a second container (docker:dind), which is an image of a working docker daemon. It needs to be in a second image in order to run.
For more Information, see the docker example project: https://gitlab.com/gitlab-examples/docker/blob/master/.gitlab-ci.yml
It doesn't work without the service because there is no running docker daemon and you can't run your build inside the docker:dind container because the run command would replace the docker daemon.
What worked for me was to disable TLS by adding this in your runners section
environment, as per https://about.gitlab.com/releases/2019/07/31/docker-in-docker-with-docker-19-dot-03/
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
The image docker:dind failed, but docker:18.09.7-dind worked for me.
services:
- name: docker:18.09.7-dind
Here is Gitlab issue details: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2623#note_206835782