Can't build docker image with kubernetes-plugin for Jenkins - docker

Question

I'm using kubernetes-plugin and I have some issue to build docker images on top of K8S pod:
I'm creating POD with:
podTemplate(containers: [
containerTemplate(
name: 'docker-build',
image: 'docker',
command: 'cat',
ttyEnabled: true
)
],
volumes: [
hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
])
{
node(POD_LABEL) {
stage("Checkout") {
dir("${env.WORKSPACE}/code") {
script {
// Checkout - Works
}
}
}
stage ('Build docker images') {
container('docker-build') {
dir("${env.WORKSPACE}/code") {
sh """
./build-images
"""
}
}
}
}
}
But it fails on the docker build step:
Err:1 http://deb.debian.org/debian stretch InRelease
Temporary failure resolving 'deb.debian.org'
Err:2 http://security.debian.org/debian-security stretch/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:3 http://deb.debian.org/debian stretch-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists...
[91mW: Failed to fetch http://deb.debian.org/debian/dists/stretch/InRelease Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/InRelease Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/InRelease Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
[0mReading package lists...
Building dependency tree...
Reading state information...
[91mE: Unable to locate package libpam-dev
E: Unable to locate package libpcap0.8-dev
E: Couldn't find any package by glob 'libpcap0.8-dev'
E: Couldn't find any package by regex 'libpcap0.8-dev'
E: Unable to locate package libpq5[0m[91m
E: Unable to locate package libtins-dev
E: Unable to locate package openjdk-8-jdk-headless
E: Unable to locate package python3
When accessing the POD with kubectl exec and trying to build, it fails on the same error:
docker build -t my_test .
When trying to build the same with --network=host it works:
docker build --network=host -t my_test .
I'm trying to understand why it requires --network=host in order to work.
BTW - when I'm in the Jenkins slave pod and I'm trying to download any packages or access to the internet it all works just fine, it happens only when I'm trying to build docker image and trying to download packages during this process.
I suspect that the docker build is failing because of some network misconfiguration or maybe docker network is getting a bad state during this docker build on top of docker ..
So far I have tried:
Create the pod with hostNetwork: true and it didn't help.
Create the pod with privileged: true and it didn't help.
Many other hacks to make the pod running on different network and it did't help as well.
Please assist.

Answer 1

The latest versions of the AWS EKS-optimized AMI disable the docker bridge network by default. To enable it, add the bootstrap_extra_args parameter to your worker group template. Source.