Kubernetes deployment status in Jenkins - jenkins

Question

I am writing a Jenkins Global pipeline library where I have a stage to deploy my docker image to K8s cluster.
So after building my docker image during CI process I am promoting(deploying) the image to multiple environments(sequentially lower to higher).
So, to get the correct status of deployment after running
kubectl apply -f Application-k8s-file.yaml
I used following command in a shell step.
kubectl rollout status deployment deployment_name
Things goes well if my deployment does not have error but if my deployment has some error(might be some code bug, application does not start) then this command kubectl rollout status deployment <deployment name> runs infinitely(as k8s retries again and again to redeploy) and my Jenkins job runs infinitely(till the Job timeout).
So to find a hack I tried a logic to put the timeout on this command and calculations are something like this:
timeout = (number of pods * liveness probe time + number of pods* 10) seconds
Not sure if this calculation is correct or not.
My code snippet looks like this
sh(returnStdout: true,script:"#!/bin/sh +e\n timeout --preserve-status ${timeout_value} kubectl rollout status deployment ${deploymentName} --kubeconfig='/opt/kubernetes-secrets/${env}/kubeconfig' 2>tmpfile; echo \$? > tmpfile1")
def readThisFile = readFile "tmpfile1.txt"
def var=readThisFile.toInteger()
if(var==0)
{
echo "deployment successful"
}
else{"do something else"}
This works well initially but later I find that k8s "kubectl rollout status deployment " command doesn't give exit code 0 until all the pods get scheduled and old get terminated completely which sometimes take time.
What I basically want is a minimal calculated timeout value.
My K8s file have parameters like this:
spec:
minReadySeconds: 30
livenessProbe:
httpGet:
path: /ping
port: 80
initialDelaySeconds: 45
periodSeconds: 5
timeoutSeconds: 60
name: test-dummyservice
ports:
- containerPort: 80
readinessProbe:
httpGet:
path: /health
port: 80
initialDelaySeconds: 60
periodSeconds: 120
timeoutSeconds: 60
I did not find anything specific related to this in K8s documentation. Anyone facing same challenge?

Answer 1

You should take a look at progressDeadlineSeconds. Once this exceeds the deadline the rollout status will exit out.
kubectl rollout status deployment ng
Waiting for rollout to finish: 2 out of 7 new replicas have been updated...
error: deployment "ng" exceeded its progress deadline
https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#progress-deadline-seconds

Answer 2

You can add the timeout flag like below, here is the doc https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands
kubectl rollout status deployment deployment_name --watch --timeout=5m

Answer 3

If you don't want to wait for the rollout to finish then you can use --watch=false.
kubectl rollout status deployment deployment_name --watch=false
Now, you can check with this command for a duration with specific interval.