Jenkins Docker Container can't access docker.sock in GKE Autopilot - docker

Question

I have installed Jenkins using helm v3 in GKE Autopilot Clustor using default chart value. I am trying to create a Docker image from Jenkins but getting permission issue (autogke-no-write-mode-hostpath).
Jenkinsfile
pipeline {
agent {
kubernetes {
defaultContainer 'jnlp'
yaml '''
apiVersion: v1
kind: Pod
spec:
containers:
- name: nodejs
image: node:16
command:
- cat
tty: true
resources:
requests:
memory: "4Gi"
cpu: "1000m"
- name: gcloud-sdk
image: google/cloud-sdk:latest
command:
- cat
tty: true
- name: docker
image: docker:latest
command:
- cat
tty: true
volumeMounts:
- mountPath: /var/run/docker.sock
name: docker-sock
volumes:
- name: docker-sock
hostPath:
path: /var/run/docker.sock
'''
}
}
environment {
// GKE
PROJECT_ID = 'example-0000'
// Docker
GCR_HOSTNAME = 'us.gcr.io'
DOCKER_IMG = "${env.GCR_HOSTNAME}/${env.PROJECT_ID}/my-app"
DOCKER_IMG_TAG = "${env.DOCKER_IMG}:${env.BRANCH_NAME}-${env.BUILD_NUMBER}"
NODE_ENV = 'production'
HOME = "${WORKSPACE}"
NPM_CONFIG_CACHE = "${WORKSPACE}/.npm"
}
options {
disableConcurrentBuilds(abortPrevious: true)
parallelsAlwaysFailFast()
}
stages {
stage('Build: Docker Image') {
when {
beforeAgent true
anyOf { branch 'master'; branch 'sandbox' }
}
steps {
container('docker') {
sh "sed -i 's#__NODE_ENV__#${NODE_ENV}#' ./Dockerfile"
sh "docker build -t ${env.DOCKER_IMG_TAG} ."
withCredentials([file(credentialsId: 'my-gcr-cred', variable: 'GCR_MANAGER_KEY')]) {
sh 'chmod 600 $GCR_MANAGER_KEY'
sh('cat $GCR_MANAGER_KEY | docker login -u _json_key --password-stdin https://' + "${env.GCR_HOSTNAME}")
sh "docker push ${DOCKER_IMG_TAG}"
sh "docker logout https://${env.GCR_HOSTNAME}"
}
}
}
post {
always {
sh "docker rmi ${env.DOCKER_IMG_TAG}"
}
}
}
}
}
Error that I am getting
ERROR: Unable to create pod kubernetes jenkins/pro-7-bmvlz-n2z9s-bqxmd.
Failure executing: POST at: https://10.100.108.3/api/v1/namespaces/jenkins/pods. Message: admission webhook "gkepolicy.common-webhooks.networking.gke.io" denied the request: GKE Policy Controller rejected the request because it violates one or more policies: {"[denied by autogke-no-write-mode-hostpath]":["hostPath volume docker-sock in container docker is accessed in write mode; disallowed in Autopilot. Requested by user: 'system:serviceaccount:jenkins:jenkinsv2', groups: 'system:serviceaccounts,system:serviceaccounts:jenkins,system:authenticated'."]}. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "gkepolicy.common-webhooks.networking.gke.io" denied the request: GKE Policy Controller rejected the request because it violates one or more policies: {"[denied by autogke-no-write-mode-hostpath]":["hostPath volume docker-sock in container docker is accessed in write mode; disallowed in Autopilot. Requested by user: 'system:serviceaccount:jenkins:jenkinsv2', groups: 'system:serviceaccounts,system:serviceaccounts:jenkins,system:authenticated'."]}, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=[denied by autogke-no-write-mode-hostpath], status=Failure, additionalProperties={}).

Answer 1

What you are trying to access is at the /var/run and Docker in Jenkin
However, if we see GKE auto pilot is managed K8s service and you don't have access to underlying the infrastructure.
Autopilot GKE does not allow you to use the Hostpath method or mount the folder with written permission.
You are only allowed to perform the read operation.
HostPort and hostNetwork are not permitted because node management is handled by GKE. Using hostPath volumes in write mode is prohibited while using hostPath volumes in read mode is allowed only for /var/log/ path prefixes.
Read more at : https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview