Getting Error when creating a Windows Docker Container on Kaniko/Gitlab - docker

I'm trying to create a Windows Docker container using Kaniko/Gitlab.
Here is the Error I see:
Resolving secrets
00:00
Preparing the "docker-windows" executor
Using Docker executor with image gcr.io/kaniko-project/executor:v1.6.0-debug ...
Pulling docker image gcr.io/kaniko-project/executor:v1.6.0-debug ...
WARNING: Failed to pull image with policy "always": no matching manifest for windows/amd64 10.0.17763 in the manifest list entries (docker.go:147:0s)
ERROR: Preparation failed: failed to pull image "gcr.io/kaniko-project/executor:v1.6.0-debug" with specified policies [always]: no matching manifest for windows/amd64 10.0.17763 in the manifest list entries (docker.go:147:0s)
For .gitlab-ci.yml file:
image:
name: microsoft/iis:latest
entrypoint: [""]
.build_variables: &build_variables
TAG: "docker-base-windows-2019-std-core"
AWS_ACCOUNT: "XXXXXXXXXX"
AWS_REGION: "XXXXXXX"
REGISTRY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
.build_script: &build_script
script:
- echo "{\"credsStore\":\"ecr-login\"}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $REGISTRY:$TAG
stages:
- build-docker-image
build_image_dev:
variables:
<<: *build_variables
stage: build-docker-image
image:
name: gcr.io/kaniko-project/executor:v1.6.0-debug
entrypoint: [""]
tags: ['XXXXX']
<<: *build_script
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == "main"'
- if: $CI_COMMIT_TAG
This is normal text Code for Docker file:
FROM Microsoft/iis:latest
CMD [ "cmd" ]

You have the error:
no matching manifest for windows/amd64
which means that particular image could not be found. It happens if you develop on windows and your server is a linux for instance.
This error implies your host machine's OS is not compatible with the OS docker image you are trying to pull.

Related

Not able to build docker image in gitlab CI

Been trying to build a simple gitlab CI pipeline which builds an image and pushes it to Google container repository. I am running through this error -
ERROR: error during connect: Get "http://docker:2375/v1.24/info": dial
tcp: lookup docker on 169.254.169.254:53: no such host
I have tried all the solutions posted across gitlab issues threads but no help. I am using public runners, it's a pretty simple ci script.
image: docker:latest
variables:
GCR_IMAGE: <GCR_IMAGE>
services:
- docker:dind
build:
stage: build
before_script:
- docker info
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Relevant issue thread: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4794
Using gitlab-runner 15.7.1
A few weeks ago I encountered this problem and was able to solve it with this method:
image:
name: docker:20.10.16
services:
- name: docker:20.10.16-dind
variables:
DOCKER_HOST: tcp://docker:2376/
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
before_script:
- until docker info; do sleep 1; done
- echo $GOOGLE_CLOUD_ACCOUNT | docker login -u _json_key --password-stdin https://us.gcr.io
script:
- docker build -t $GCR_IMAGE:latest .
- docker push $GCR_IMAGE:$CI_COMMIT_SHA
Also add this configuration to runner
[[runners]]
[runners.kubernetes]
namespace = "{{.Release.Namespace}}"
image = "ubuntu:20.04"
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"

Gitlab-runner, docker:20.10.2-dind shows error, Cannot connect to the Docker daemon at tcp://docker:2375

On self hosted Gitlab on GCP installed by helm, I use Gitlab-runner.
On gitlab-runner I need to use docker so using dind, but I got error
tcp://docker:2375. Is the docker daemon running?
gitlab-runner deployment
...
spec:
containers:
- command:
- /bin/bash
- /scripts/entrypoint
env:
- name: CI_SERVER_URL
value: https://my-gitlab.com
- name: CLONE_URL
- name: RUNNER_REQUEST_CONCURRENCY
value: "1"
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
- name: KUBERNETES_IMAGE
- name: KUBERNETES_PRIVILEGED
value: "true" # <= set privileged true to use dind
...
gitlab-ci.yaml
services:
- docker:20.10.4-dind
stages:
- build
variables:
GIT_SSL_NO_VERIFY: "1"
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ''
DOCKER_HOST: tcp://docker:2375
image:
name: google/cloud-sdk:latest
before_script:
- docker version
build:
stage: build
script:
- echo hello
gitlab-runner log
Executing "step_script" stage of the job script
00:00
$ docker version
Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
Client: Docker Engine - Community
Version: 19.03.11
API version: 1.40
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:09:53 2020
OS/Arch: linux/amd64
Experimental: false
Cleaning up file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
troubleshooting says that it's because of TLS. So I set DOCKER_TLS_CERTDIR: '' , the way written in another document.
Also, this problem didn't happen when I used docker:19.03.0-dind. From 19.03.0-dind, TLS is automatically. So disable TLS configuration must be worked correctly.
(docker:19.3.13-dind also worked well.)
I don't know why from docker:20 this error showed up. Has anyone already tried gitlab-runner with grater than docker:20 ?
I figured out that I should follow https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-enabled-in-kubernetes
toml
runners:
config: |
[[runners]]
[runners.kubernetes]
image = "ubuntu:20.04"
privileged = true
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
gitlab-ci.yaml
services:
- docker:20.10.4-dind
stages:
- build
variables:
GIT_SSL_NO_VERIFY: "1"
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
image:
name: google/cloud-sdk:latest
before_script:
- docker version
build:
stage: build
script:
- echo hello

Authentication Error when Building and Pushing docker image to ACR using Azure DevOps Pipelines and docker-compose

I am trying to build and push a docker image to ACR using Azure DevOps pipelines. I have to build it with a docker-compose.yml file to be able to use openvpn in the container.
When I run the pipeline I get the following error. Does anyone have an idea of how to solve this?
Starting: DockerCompose
==============================================================================
Task : Docker Compose
Description : Build, push or run multi-container Docker applications. Task can be used with Docker or Azure Container registry.
Version : 0.183.0
Author : Microsoft Corporation
Help : https://aka.ms/azpipes-docker-compose-tsg
==============================================================================
/usr/local/bin/docker-compose -f /home/vsts/work/1/s/src/docker-compose.yml -f /home/vsts/agents/2.188.2/.docker-compose.1624362077551.yml -p Compose up -d
Creating network "composeproject_default" with the default driver
Pulling getstatus (***/getstatus:)...
Head https://***/v2/getstatus/manifests/latest: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.
##[error]Creating network "composeproject_default" with the default driver
##[error]Pulling getstatus (***/getstatus:)...
##[error]Head https://***/v2/getstatus/manifests/latest: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.
##[error]The process '/usr/local/bin/docker-compose' failed with exit code 1
Finishing: DockerCompose
My azure-pipelines.yml look like this:
# Docker
# Build and push an image to Azure Container Registry
# https://learn.microsoft.com/azure/devops/pipelines/languages/docker
trigger:
- main
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: '*****************************'
imageRepository: 'getstatus'
containerRegistry: 'composeproject.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- task: DockerCompose#0
inputs:
containerregistrytype: 'Azure Container Registry'
dockerComposeFile: '**/docker-compose.yml'
action: 'Run a Docker Compose command'
dockerComposeCommand: 'up -d'
And the docker-compose.yml like this:
version: "3.3"
services:
getstatus:
image: composeproject.azurecr.io/getstatus
restart: always
sysctls:
- net.ipv6.conf.all.disable_ipv6=0
cap_add:
- NET_ADMIN
devices:
- /dev/net/tun
volumes:
- /etc/timezone:/etc/timezone:ro
I think your docker compose task is missing a couple of parameters
try adding azureContainerRegistry: composeproject.azurecr.io
and azureSubscriptionEndpoint: $(dockerRegistryServiceConnection)
Not sure why the credentials supplied in the Docker#2 task don't persist since they're in the same stage but then I could fill an encyclopedia with what I'm not sure on when it comes to Azure pipelines

gitlab-runner docker exeuctor (dind) - Error https://docker:2375/v1.40/info dial tcp: lookup docker on

Ive an issue with gitlab-runner executor docker. After I ran my gitlab-ci.yml file , pipeline fail on step docker info during before_script with:
Running with gitlab-runner 13.10.0 (54944146)
on docker-runner N2_yEgUD
Preparing the "docker" executor 00:07
Using Docker executor with image docker:19.03.0 ...
Starting service docker:19.03.0-dind ...
Pulling docker image docker:19.03.0-dind ...
Using docker image sha256:fd0c64832f7e46b63a180e6000dbba7ad7a63542c5764841cba73429ba74a39e for docker:19.03.0-dind with digest docker#sha256:442ac4b31375cbe617f31759b5199d240f11d5f430e54946575b274b2fb6f096 ...
Waiting for services to be up and running...
.............................................................................................
$ docker info
Client:
Debug Mode: false
Server:
ERROR: error during connect: Get https://docker:2375/v1.40/info: dial tcp: lookup docker on 127.0.0.53:53: server misbehaving
errors pretty printing info
Cleaning up file based variables 00:01
ERROR: Job failed: exit code 1
I did a research on stack and official gitlab forum but none of the answers fix my issue:
add to .toml -> volume: ['/certs/client']
run against old: docker:18.x.x / docker:18.x.x -dind | docker:stable / docker:dind
run with: DOCKER_TLS_CERTDIR:""
run with/without:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
add endpoint to service:
services:
- name: docker:dind
entrypoint: ["env", "-u", "DOCKER_HOST"]
command: ["dockerd-entrypoint.sh"]
Content of gitlab-runner toml
concurrent = 1
check_interval = 0
log_level = "debug"
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner"
url = "xxxxxxxx"
token = "xxxxxxx"
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/certs/client"]
network_mode = "host"
Content of gitlab-ci.yml
image: docker:19.03.0
services:
- docker:19.03.0-dind
stages:
- build
- test_framework
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
CONTAINER_TEST_IMAGE: xxxx
CONTAINER_RELEASE_IMAGE: xxxx
before_script:
- docker info
- docker login -u xxxx -p $CI_JOB_TOKEN xxxx
build:
stage: build
tags:
- adm-docker
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
image: $CONTAINER_TEST_IMAGE
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
Only one thing fix issue (workaround issue). When I add to .toml
volume: ["/var/run/docker.sock:/var/run/docker.sock"]
But after that Iam loosing DIND possibility to run my gitlab-ci.yml with different image for test stage (without using under script: -docker run MY_IMAGE python3....).
Which is not what I want
gitlab-runner under Ubuntu20 / Docker version 20.10.5, build 55c4c88
Ive worked with very similar gitlab-ci.yml around 1Yr ago and there was no issue with docker executor
Any ideas/suggestions ?
I was able to fix issue by changing flow of my gitlab-ci.yml
image: docker:19.03.5
services:
- docker:19.03.5-dind
stages:
- build
- test_framework
- release
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
CONTAINER_TEST_IMAGE: xxxxx
CONTAINER_RELEASE_IMAGE: xxxxx
build:
stage: build
tags:
- adm-docker
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
script:
- docker pull $CONTAINER_RELEASE_IMAGE || true
- docker build -t $CONTAINER_TEST_IMAGE --cache-from $CONTAINER_RELEASE_IMAGE .
- docker push $CONTAINER_TEST_IMAGE
timeout: 1 hours
.test_commit: &test_commit
stage: test_framework
tags:
- adm-docker
timeout: 1 hours
artifacts:
reports:
junit: 'results/xunit.xml'
expire_in: 1 day
except:
- master
test-unit:
<<: *test_commit
image: $CONTAINER_TEST_IMAGE
script:
- python3 -m pytest --junitxml=results/xunit.xml test_unit/
and toml
[[runners]]
name = "docker-runner"
url = xxxxx
token = xxxxx
executor = "docker"
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
privileged = true
image = "docker:19.03.12"
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
network_mode = "host"
issue was fixed by volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
and issue with test stage was cased by:
before_script:
- docker info
- docker login -u xxxxx -p $CI_JOB_TOKEN xxxxx
in root structure of .yml file. I had to move it to build stage
I hope that will help ppl in the future

Cannot connect to the daemon docker in a docker image of docker

I am trying to build CI with gitlab, I go from a docker image of docker, and i didn't have any problem with my front repository, but now with the back withe the same gitlab-ci configuration file, i have this daemon error.
Here is the output of the build :
[0KRunning with gitlab-ci-multi-runner 1.10.4 (b32125f)[0;m
[0;m[0KUsing Docker executor with image docker:1.13.1 ...
[0;m[0KPulling docker image docker:1.13.1 ...
[0;mRunning on runner-4e4528ca-project-1649638-concurrent-0 via runner-4e4528ca-machine-1487688057-7c0f1e46-digital-ocean-4gb...
[32;1mCloning repository...[0;m
Cloning into '/builds/***/formation-back'...
[32;1mChecking out af7cbcae as docker...[0;m
[32;1mSkipping Git submodules setup[0;m
[32;1m$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com[0;m
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[31;1mERROR: Build failed: exit code 1
[0;m
Here is my .gitlab-ci.yml :
image: docker:1.13.1
stages:
- build
- test
- deploy
variables:
BUILD_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF
TEST_IMG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF_NAME
RELEASE_IMG: $CI_REGISTRY_IMAGE:latest
AWS_STAGING_ENV: "***"
AWS_PROD_ENV: "***"
DOCKERRUN: Dockerrun.aws.json
DEPLOY_ARCHIVE: ${AWS_APP}-${CI_BUILD_REF}.zip
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- .ci/before_script
build:
stage: build
script:
- docker build --pull -t $BUILD_IMG .
- docker push $BUILD_IMG
test:
stage: test
script:
- docker pull $BUILD_IMG
- docker run --rm $BUILD_IMG npm run test
- docker tag $BUILD_IMG $TEST_IMG
- docker push $TEST_IMG
deploy:staging:
stage: deploy
environment: Staging
variables:
DOCKER_IMG: ${CI_REGISTRY_IMAGE}:${CI_BUILD_REF}
script:
- ./.ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
- ./.ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_STAGING_ENV
artifacts:
paths:
- $DEPLOY_ARCHIVE
except:
- production
deploy:production:
stage: deploy
environment: Production
variables:
DOCKER_IMG: ${CI_REGISTRY_IMAGE}:latest
script:
- .ci/push-new-image $TEST_IMG $RELEASE_IMG
- .ci/create-deploy-archive $DOCKER_IMG $AWS_BUCKET $DOCKERRUN $DEPLOY_ARCHIVE
- .ci/aws-deploy $DEPLOY_ARCHIVE $CI_BUILD_REF $AWS_PROD_ENV
artifacts:
paths:
- $DEPLOY_ARCHIVE
only:
- production
when: manual
Here is my config.toml file :
concurrent = 1
check_interval = 0
[[runners]]
name = "***"
url = "https://gitlab.com/ci"
token = "750c63cba1c269d789bdb33c42b726"
executor = "docker"
[runners.docker]
tls_verify = false
image = "alpine:3.5"
privileged = true
disable_cache = false
volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
[runners.cache]
Here is docker info :
DEBU[0771] Calling GET /v1.24/info
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 1
Server Version: 1.12.6
Storage Driver: devicemapper
Pool Name: docker-202:1-395267-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 519 MB
Data Space Total: 107.4 GB
Data Space Available: 6.569 GB
Metadata Space Used: 1.397 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.146 GB
Thin Pool Minimum Free Space: 10.74 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.93-RHEL7 (2015-01-28)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge overlay null host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options:
Kernel Version: 4.4.44-39.55.amzn1.x86_64
Operating System: Amazon Linux AMI 2016.09
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.2 MiB
Name: ip-172-31-30-143
ID: D6DU:OBWL:R3HK:DSZK:EOYC:5EHS:NU4I:4M3T:H5PL:JWLH:CIPD:I7VW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 20
Goroutines: 27
System Time: 2017-02-22T11:16:19.042666914Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Insecure Registries:
127.0.0.0/8
You need to add
services:
- docker:dind
to your .gitlab-ci.yml. This tells the runner to start a second container (docker:dind), which is an image of a working docker daemon. It needs to be in a second image in order to run.
For more Information, see the docker example project: https://gitlab.com/gitlab-examples/docker/blob/master/.gitlab-ci.yml
It doesn't work without the service because there is no running docker daemon and you can't run your build inside the docker:dind container because the run command would replace the docker daemon.
What worked for me was to disable TLS by adding this in your runners section
environment, as per https://about.gitlab.com/releases/2019/07/31/docker-in-docker-with-docker-19-dot-03/
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
The image docker:dind failed, but docker:18.09.7-dind worked for me.
services:
- name: docker:18.09.7-dind
Here is Gitlab issue details: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2623#note_206835782

Resources