I have a Jenkins pipeline with some jobs running in parallel. I want to manage the timing out of one of them and keep the pipeline running. I have followed this very well-done guide, but I can't avoid that when my job times out; all the other parallel jobs and the whole process fail too.
Following, how I have implemented the pipeline:
pipeline {
agent any;
stages {
stage('Stage-1') {
steps {
sh 'job-1.sh'
}
}
stage('Stage-2') {
failFast false
parallel {
stage('Stage-2-1') {
steps {
sh 'job-2-1.sh'
}
}
stage('Stage2-2') {
stages {
stage('job-2-2-1') {
options {
timeout(time: 10, unit: "SECONDS")
}
steps {
script {
Exception caughtException = null
catchError(buildResult: 'SUCCESS', stageResult: 'ABORTED') {
try {
sh 'job-2-2-1.sh'
}
catch (org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e) {
echo "Time out"
TIMEOUT="1"
} catch (Throwable e) {
caughtException = e
}
}
if (caughtException) {
error caughtException.message
}
}
}
}
stage('job-2-2-2') {
when {
expression { TIMEOUT=="0" }
}
steps {
sh 'job-2-2-2.sh'
}
}
}
}
stage('Stage-2-3') {
steps {
sh 'job-2-3.sh'
}
}
}
}
stage('Stage-3') {
steps {
sh 'job-3.sh'
}
}
}
}
I tried to set failFast to false because I don't know if it has a default value.
How can I keep the other jobs run and get the successful final build?
Thanks
Related
I have declarative pipeline as below and it runs 2* and 3* stages in parallel, pasted the blue ocean diagram below.
pipeline {
agent { label 'my_node' }
options {
timestamps()
parallelsAlwaysFailFast()
}
stages {
stage('1') {
steps {
script {
step([$class: 'WsCleanup'])
}
}
}
stage('2') {
parallel {
stage("2.1") {
steps {
script {
sh 'echo hi 2.1'
}
}
}
stage("2p") {
steps {
script {
sh 'echo hi 2p'
}
}
}
}
}
stage('3') {
parallel {
stage('3.1') {
steps {
script {
sh """
echo hi 3.1
"""
}
}
}
stage('3.2') {
steps {
script {
sh """
echo "hi 3.2"
"""
}
}
}
}
}
stage('4') {
steps {
script {
sh "echo end"
}
}
}
}
}
But I am looking to run 2p in parallel to 2* and 3*, like shown below, is there a way?
I tried to use paralle under parallel, to start 2p in parallel to 2 and 3, and nested parallel to run 3.1. and 3.2 underneath, but declarative pipeline is not allowing nested parallel.
You can't do this only with Declarative syntax. But you can achieve this with a combination of Scripted and Declarative syntax. One thing to note is, AFAIK there is no visualization support for nested parallel stages as of now. There is a feature request for this here.
Following is a sample pipeline you can use as a reference for your use case.
pipeline {
agent any
options {
timestamps()
parallelsAlwaysFailFast()
}
stages {
stage('1') {
steps {
script {
step([$class: 'WsCleanup'])
}
}
}
stage('2 AND 3') {
steps {
script {
parallel getWrappedStages()
}
}
}
stage('4') {
steps {
script {
sh "echo end"
}
}
}
}
}
def getWrappedStages() {
stages = [:]
stages["Step2.1"] = { stage('2.1') {
sh """
echo hi 2.1
"""
}
parallel parallel3xstages()
}
stages["Step2.p"] = { stage('2.p') {
sh """
echo hi 2.p
"""
}
}
return stages
}
def parallel3xstages() {
stages = [:]
stages["Step3.1"] = { stage('3.1') {
sh """
echo hi 3.1
"""
}
}
stages["Step3.2"] = { stage('3.2') {
sh """
echo hi 3.2
"""
}
}
return stages
}
I have an Jenkinsfile to run some stages that I want, but after each commit, I would like to see all stages info in a clean way in my Gitlab server.
Here is the Jenkinsfile:
pipeline {
agent any
options {
gitLabConnection('gitlab connection')
}
stages {
stage('Build') {
steps {
gitlabCommitStatus(name: 'Build') {
(...)
}
}
}
stage('Static Code') {
parallel {
stage('SonarQube') {
steps {
gitlabCommitStatus(name: 'SonarQube') {
(...)
}
}
}
stage('FindBugs') {
steps {
gitlabCommitStatus(name: 'FindBugs') {
(...)
}
}
}
}
}
stage('Unit Tests') {
steps {
gitlabCommitStatus(name: 'Unit Tests') {
(...)
}
}
}
}
post {
changed {
gitlabCommitStatus(name: 'Teams Notification') {
(...)
}
}
}
}
When I run that pipeline I receive the stages info in a vertical way without any type of info on which stages run in parallel or not. All my stages show in vertical mode.
Is there anything that I can do to show something like this?
Thank you so much :)
I've a pipeline where multiple stages run in parallel but if any of the stages fails, I want to get its name to show failed stage. With following code, even if it fails in first stage ie Checking Redmine access, it always show last stage as failed i.e. Checking Reviewers list. This is because it runs in parallel and latest assigned value is picked up.
pipeline {
agent {
label {
label "<machine-ip>"
customWorkspace "workspace/RedmineAndReviewboardProject/SVNCheckoutTest"
}
}
stages {
stage('Verify inputs') {
parallel {
stage('Checking Redmine access') {
steps {
script {
FAILED_STAGE = env.STAGE_NAME
echo "Redmine"
sh'''hello'''
}
}
}
stage('Checking SVN access') {
steps {
script {
FAILED_STAGE = env.STAGE_NAME
echo "SVN"
}
}
}
stage('Checking Reviewers list') {
steps {
script {
FAILED_STAGE = env.STAGE_NAME
echo "Reviewer"
}
}
}
}
}
}
post {
failure {
script {
echo "Failed stage is " + FAILED_STAGE
}
}
}
}
Is there any way I can get exactly failed stage of parallel running stages? or it will be also ok with me if parent stage name is returned as failed stage.
I believe you can use a post { failure { block for each stage : see https://www.jenkins.io/doc/book/pipeline/syntax/#post
pipeline {
agent {
label {
label "<machine-ip>"
customWorkspace "workspace/RedmineAndReviewboardProject/SVNCheckoutTest"
}
}
stages {
stage('Verify inputs') {
parallel {
stage('Checking Redmine access') {
steps {
script {
echo "Redmine"
sh'''hello'''
}
}
post {
failure {
script {
echo "Failed stage is ${STAGE_NAME}"
}
}
}
}
stage('Checking SVN access') {
steps {
script {
FAILED_STAGE = env.STAGE_NAME
echo "SVN"
}
}
post {
failure {
script {
echo "Failed stage is ${STAGE_NAME}"
}
}
}
}
stage('Checking Reviewers list') {
steps {
script {
FAILED_STAGE = env.STAGE_NAME
echo "Reviewer"
}
}
post {
failure {
script {
echo "Failed stage is ${STAGE_NAME}"
}
}
}
}
}
}
}
}
There is a way to execute steps/post-actions on different nodes in parallel described in this article: https://jenkins.io/blog/2017/09/25/declarative-1/
stage('Run Tests') {
parallel {
stage('Test On Windows') {
agent {
label "windows"
}
steps {
bat "run-tests.bat"
}
post {
always {
junit "**/TEST-*.xml" // DUPLICATE CODE
}
}
}
stage('Test On Linux') {
agent {
label "linux"
}
steps {
sh "run-tests.sh"
}
post {
always {
junit "**/TEST-*.xml" // DUPLICATE CODE
}
}
}
}
}
Is there any possibility to execute same stages on multiple nodes without duplicating code?
Something like this:
stage('Run Tests') {
parallel {
stage("Test On ${NODE_NAME}") {
agents {
label "windows"
label "linux"
}
steps {
// do test steps
}
post {
always {
junit "**/TEST-*.xml"
}
}
}
}
}
You can create dynamic stages, but for your case not needed
pipeline {
agent any
stages {
stage ("Test") {
steps {
script {
testStages = ["Windows", "Linux"].collectEntries {
["${it}" : runTests(it)]
}
parallel testStages
}
}
}
}
}
def runTests(def name){
return {
node(name) {
stage("Run on ${name}") {
script {
command = "run-tests"
try {
switch(name.toLowerCase()) {
case "windows":
command += ".bat"
break;
case "linux":
command += ".sh"
break;
}
echo command
} catch (Exception ex) {
echo ex
} finally {
echo "post ${name}"
}
}
}
}
}
}
Declarative Matrix worked best for me:
pipeline {
agent none
stages {
stage('BuildAndTest') {
matrix {
agent {
label "${PLATFORM}-agent"
}
axes {
axis {
name 'PLATFORM'
values 'linux', 'windows'
}
}
stages {
stage('Test') {
steps {
echo "Do Test for ${PLATFORM}"
}
}
}
post {
always {
junit "**/TEST-*.xml"
}
}
}
}
}
}
This pipeline will execute the defined stages incl. post build actions on both platforms without any code duplication.
Quote from a Jenkins blog post about declartive matrix:
An equivalent pipeline created without matrix would easily be several
times larger, and much harder to understand and maintain.
In a declarative Jenkinsfile how can you determine if a job exists?
I want to conditionally run a stage if another job exists, like so:
pipeline {
agent { label 'mylabel' }
// Run the the stages:
stages {
stage('Pre-Build') {
steps {
timestamps { ansiColor('xterm') { sh 'build/jenkins_prebuild.sh' } }
}
}
stage('Pre-Build') {
steps {
timestamps { ansiColor('xterm') { sh 'build/jenkins_build.sh' } }
}
}
stage('Trigger Remote If Exists'){
when { JOB_EXISTS }
steps {
timestamps {
ansiColor('xterm') {
sh 'build/pre-trigger.sh'
build job: "../myjob/foo", wait: false, parameters: [booleanParam(name: 'FOO', value: true)]
}
}
}
}
}
}
Basically, what should I do for JOB_EXISTS?
if (jenkins.model.Jenkins.instance.getItem("YOUR_JOB_NAME") != null) {
}
You may not be able to use the above snippet in your declarative pipeline. But you should be able to use this in a global shared lib.
UPDATE:
An administrator needs to approve this snippet in "Manage Jenkins -> In-process Script Approval" section.
There is also this option: https://stackoverflow.com/a/52076776/3384609
try {
println("Preparing to build the ${jobName}...")
build job:"${jobName}", propagate:false, wait:false
} catch (hudson.AbortException e) {
println("Not building the job ${jobName} as it doesn't exist")
}```