Jenkins+AWSLoadBalancerController example - jenkins

Question

I'm building a jenkins cluster in Amazon EKS and am trying to register Jenkins with the AWS Load Balancer Controller. I could use a bit of advice from some more experienced folks.
Here is my values for Jenkins helm3 install (I'm still a bit new at helm):
clusterZone: "cluster.local"
renderHelmLabels: true
controller:
componentName: "jenkins-controller"
image: "jenkins/jenkins"
tag: "2.263.3"
imagePullPolicy: "Always"
adminUser: "admin"
adminPassword: "admin"
jenkinsHome: "/var/jenkins_home"
jenkinsWar: "/usr/share/jenkins/jenkins.war"
resources:
requests:
cpu: "50m"
memory: "256Mi"
limits:
cpu: "2000m"
memory: "4096Mi"
usePodSecurityContext: true
runAsUser: 1000
fsGroup: 1000
servicePort: 8080
targetPort: 8080
serviceType: NodePort
serviceAnnotations:
alb.ingress.kubernetes.io/healthcheck-path: '{{ default "" .Values.controller.jenkinsUriPrefix }}/login'
alb.ingress.kubernetes.io/group.name: "jenkins-ingress"
healthProbes: true
probes:
startupProbe:
httpGet:
path: '{{ default "" .Values.controller.jenkinsUriPrefix }}/login'
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
failureThreshold: 5
httpGet:
path: '{{ default "" .Values.controller.jenkinsUriPrefix }}/login'
port: http
periodSeconds: 10
timeoutSeconds: 5
readinessProbe:
failureThreshold: 3
httpGet:
path: '{{ default "" .Values.controller.jenkinsUriPrefix }}/login'
port: http
periodSeconds: 10
timeoutSeconds: 5
agentListenerPort: 50000
agentListenerHostPort:
disabledAgentProtocols:
- JNLP-connect
- JNLP2-connect
csrf:
defaultCrumbIssuer:
enabled: true
proxyCompatability: true
agentListenerServiceType: "ClusterIP"
installPlugins:
- kubernetes:1.29.0
- workflow-aggregator:2.6
- git:4.5.2
- configuration-as-code:1.47
JCasC:
defaultConfig: true
securityRealm: |-
local:
allowsSignup: false
enableCaptcha: false
users:
- id: "${chart-admin-username}"
name: "Jenkins Admin"
password: "${chart-admin-password}"
authorizationStrategy: |-
loggedInUsersCanDoAnything:
allowAnonymousRead: false
sidecars:
configAutoReload:
enabled: true
image: kiwigrid/k8s-sidecar:0.1.275
imagePullPolicy: IfNotPresent
reqRetryConnect: 10
sshTcpPort: 1044
folder: "/var/jenkins_home/casc_configs"
ingress:
enabled: true
paths:
- backend:
serviceName: >-
{{ template "jenkins.fullname" . }}
servicePort: 8080
# path: "/jenkins"
apiVersion: "extensions/v1beta1"
annotations:
alb.ingress.kubernetes.io/group.name: "jenkins-ingress"
kubernetes.io/ingress.class: "alb"
persistence:
enabled: true
existingClaim: jenkins-0-claim
rbac:
create: true
readSecrets: false
serviceAccount:
create: true
name: "jenkins"
Here is the contents of my ingress.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
alb.ingress.kubernetes.io/group.name: jenkins-ingress
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}, {"HTTP":
8080}, {"HTTPS": 8443}]'
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/tags: Environment=dev,Team=test
kubernetes.io/ingress.class: alb
name: app-ingress
namespace: default
spec:
rules:
- http:
paths:
- backend:
serviceName: app1-nginx-nodeport-service
servicePort: 80
path: /app1/*
- backend:
serviceName: app2-nginx-nodeport-service
servicePort: 80
path: /app2/*
- backend:
serviceName: app3-nginx-nodeport-service
servicePort: 80
path: /app3/*
- backend:
serviceName: jenkins
servicePort: 8080
path: /jenkins/*
Here is the error, I suspect it is due to the namespace. Jenkins is in it's own namespace:
❯ kubectl describe ingress app-ingress
Name: app-ingress
Namespace: default
Address: internal-k8s-jenkinsingress-9f4e69d9f1-2066345703.us-west-2.elb.amazonaws.com
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
*
/app1/* app1-nginx-nodeport-service:80 (10.216.66.254:80)
/app2/* app2-nginx-nodeport-service:80 (10.216.66.248:80)
/app3/* app3-nginx-nodeport-service:80 (10.216.66.174:80)
/jenkins/* jenkins:8080 (<error: endpoints "jenkins" not found>)
Annotations: alb.ingress.kubernetes.io/group.name: jenkins-ingress
alb.ingress.kubernetes.io/listen-ports: [{"HTTP": 80}, {"HTTPS": 443}, {"HTTP": 8080}, {"HTTPS": 8443}]
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/tags: Environment=dev,Team=test
kubernetes.io/ingress.class: alb
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedDeployModel 35m (x16 over 37m) ingress Failed deploy model due to InvalidParameter: 1 validation error(s) found.
- minimum field value of 1, CreateTargetGroupInput.Port.
Warning FailedBuildModel 7m2s (x15 over 34m) ingress Failed build model due to ingress: default/app-ingress: Service "jenkins" not found

Answer 1

I was able to resolve my issue. Turns out I was defining the the jenkins path in too many places. I removed it from the primary ingress definition and altered my jenkins helm values.
I also set service type to NodePort instead of ClusterIP
Removed this from app-ingress.yaml:
- backend:
serviceName: jenkins
servicePort: 8080
path: /jenkins/*
Removed path value from jenkins helm ingress definition and set the jenkinsUriPrefix to "/jenkins".