How to run Jenkins with Docker on Kubernetes? - docker

Question

I’m attempting to execute a Jenkins & Docker CLI container on Kubernetes. Here are my steps:
I create the pod using:
kubectl --kubeconfig my-kubeconfig.yml run my-jenkins-pod --image=trion/jenkins-docker-client --restart=Never
Which creates a pod-based on the image https://hub.docker.com/r/trion/jenkins-docker-client
I create the deployment using:
kubectl --kubeconfig my-kubeconfig.yml apply -f /kub/kube
/kub/kube contains jenkins-deployment-yaml which I have configured as:
apiVersion: v1
kind: Service
metadata:
name: my-jenkins-pod
spec:
ports:
- protocol: "TCP"
port: 50000
targetPort: 5001
selector:
app: my-jenkins-pod
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-jenkins-pod
spec:
selector:
matchLabels:
app: my-jenkins-pod
replicas: 1
template:
metadata:
labels:
app: my-jenkins-pod
spec:
containers:
- name: ml-services
image: trion/jenkins-docker-client
ports:
- containerPort: 5001
To access the Jenkins container I expose the IP using:
kubectl --kubeconfig my-kubeconfig.yml expose deployment my-jenkins-pod --type=LoadBalancer --name=my-jenkins-pod-public
To return the IP of the Jenkins and Docker image I use :
kubectl --kubeconfig my-kubeconfig.yml get services my-jenkins-pod-public
Which returns:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-jenkins-pod-public LoadBalancer 9.5.52.28 161.61.222.16 5001:30878/TCP 10m
To test I open the URL at location:
http://161.61.222.16:5001/
Which returns:
This page isn’t working
161.61.222.16 didn’t send any data.
ERR_EMPTY_RESPONSE
It seems the service has started but the port mappings are incorrect?
The log of the pod my-jenkins-pod contains:
Running from: /usr/share/jenkins/jenkins.war webroot:
EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:42.899+0000
[id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging
initialized #274ms to org.eclipse.jetty.util.log.JavaUtilLog
2021-04-03 11:15:43.012+0000 [id=1] INFO winstone.Logger#logInternal:
Beginning extraction from war file 2021-04-03 11:15:44.369+0000
[id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty
contextPath 2021-04-03 11:15:44.416+0000
[id=1] INFO org.eclipse.jetty.server.Server#doStart:
jetty-9.4.39.v20210325; built: 2021-03-25T14:42:11.471Z; git:
9fc7ca5a922f2a37b84ec9dbc26a5168cee7e667; jvm 1.8.0_282-b08 2021-04-03
11:15:44.653+0000
[id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP
Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2021-04-03 11:15:44.695+0000
[id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart:
DefaultSessionIdManager workerName=node0 2021-04-03 11:15:44.695+0000
[id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No
SessionScavenger set, using defaults 2021-04-03 11:15:44.696+0000
[id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0
Scavenging every 660000ms 2021-04-03 11:15:45.081+0000
[id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home
directory: /var/jenkins_home found at:
EnvVars.masterEnvVars.get("JENKINS_HOME") 2021-04-03 11:15:45.203+0000
[id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started
w.#24f43aa3{Jenkins
v2.286,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
2021-04-03 11:15:45.241+0000
[id=1] INFO o.e.j.server.AbstractConnector#doStart: Started
ServerConnector#4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2021-04-03 11:15:45.241+0000
[id=1] INFO org.eclipse.jetty.server.Server#doStart: Started #2616ms
2021-04-03 11:15:45.245+0000 [id=21] INFO winstone.Logger#logInternal:
Winstone Servlet Engine running: controlPort=disabled 2021-04-03
11:15:46.479+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained:
Started initialization 2021-04-03 11:15:46.507+0000
[id=26] INFO jenkins.InitReactorRunner$1#onAttained: Listed all
plugins 2021-04-03 11:15:47.654+0000
[id=27] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all
plugins 2021-04-03 11:15:47.660+0000
[id=26] INFO jenkins.InitReactorRunner$1#onAttained: Started all
plugins 2021-04-03 11:15:47.680+0000
[id=27] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all
extensions 2021-04-03 11:15:48.620+0000
[id=26] INFO jenkins.InitReactorRunner$1#onAttained: System config
loaded 2021-04-03 11:15:48.621+0000
[id=26] INFO jenkins.InitReactorRunner$1#onAttained: System config
adapted 2021-04-03 11:15:48.621+0000
[id=27] INFO jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
2021-04-03 11:15:48.622+0000
[id=27] INFO jenkins.InitReactorRunner$1#onAttained: Configuration for
all jobs updated 2021-04-03 11:15:48.704+0000
[id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Started
Download metadata 2021-04-03 11:15:48.722+0000
[id=40] INFO hudson.util.Retrier#start: Attempt #1 to do the action
check updates server 2021-04-03 11:15:49.340+0000
[id=26] INFO jenkins.install.SetupWizard#init:
************************************************************* Jenkins initial setup is required. An admin user has been created and a
password generated. Please use the following password to proceed to
installation: ab5dbf74145c405fb5a33456d4b97436 This may also be found
at: /var/jenkins_home/secrets/initialAdminPassword
************************************************************* 2021-04-03 11:16:08.107+0000
[id=27] INFO jenkins.InitReactorRunner$1#onAttained: Completed
initialization 2021-04-03 11:16:08.115+0000
[id=20] INFO hudson.WebAppMain$3#run: Jenkins is fully up and running
2021-04-03 11:16:08.331+0000
[id=40] INFO h.m.DownloadService$Downloadable#load: Obtained the
updated data file for hudson.tasks.Maven.MavenInstaller 2021-04-03
11:16:08.332+0000 [id=40] INFO hudson.util.Retrier#start: Performed
the action check updates server successfully at the attempt #1
2021-04-03 11:16:08.334+0000
[id=40] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished
Download metadata. 19,626 ms
Is Jenkins server is started at port 8080? because of this log message:
11:15:45.241+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart:
Started ServerConnector#4f0f2942{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2021-04-03
I tried changing jenkins-deployment-yaml to point at port 8080 instead of 50000, resulting in the updated jenkins-deployment-yaml :
apiVersion: v1
kind: Service
metadata:
name: my-jenkins-pod
spec:
ports:
- protocol: "TCP"
port: 8080
But the same error is returned when I attempt to access http://161.61.222.16:5001/
Are my port mappings incorrect? Is this a correct method of adding an existing docker container that is available on the docker hub to a Kubernetes cluster?
Update:
The result of command kubectl describe services my-jenkins-pod-public is :
Name: my-jenkins-pod-public
Namespace: default
Labels: <none>
Annotations: kubernetes.digitalocean.com/load-balancer-id: d46ae9ae-6e8a-4fd8-aa58-43c08310059a
Selector: app=my-jenkins-pod
Type: LoadBalancer
IP Families: <none>
IP: 10.245.152.228
IPs: 10.245.152.228
LoadBalancer Ingress: 161.61.222.16
Port: <unset> 5001/TCP
TargetPort: 5001/TCP
NodePort: <unset> 30878/TCP
Endpoints: 10.214.12.12:5001
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
Trying to access http://161.61.222.16:30878/ via browser returns:
This site can’t be reached159.65.211.46 refused to connect. Try:
Checking the connection Checking the proxy and the firewall
ERR_CONNECTION_REFUSED
Trying to access http://161.61.222.16:5001/ via browser returns:
This page isn’t working
161.61.222.16 didn’t send any data.
ERR_EMPTY_RESPONSE
Seems the port 5001 is exposed/accessible but is not sending any data.
I also tried accessing 10.214.12.12 on ports 5001 & 30878 but both requests time out.

Answer 1

You need to use http://161.61.222.16:30878/ from outside of the host which is running containers on. Port 5001 is just accessible inside the cluster with internal IP (9.5.52.28 is in your case). Whenever you expose your deployment, automatically (also you can define manually) one of the NodePort (by default between 30000 - 32767)assign to the service for external request.
For service details, you need to run the below command. The command output will give you NodePort and another details.
kubectl describe services my-service
Please check related kubernetes documentation
Also you have configured service with port 5001 but Jenkins working with 8080 as far as I see in logs. Try to change target port of service to 8080 from 5001