Jenkins: Ignore failure in pipeline build step - jenkins

With jenkins build flow plugin this was possible:
ignore(FAILURE){
build( "system-check-flow" )
}
How to do this with Declarative Pipeline syntax?

To ignore a failed step in declarative pipeline you basically have two options:
Use script step and try-catch block (similar to previous proposition by R_K but in declarative style)
stage('someStage') {
steps {
script {
try {
build job: 'system-check-flow'
} catch (err) {
echo err.getMessage()
}
}
echo currentBuild.result
}
}
Use catchError
stage('someStage') {
steps {
catchError {
build job: 'system-check-flow'
}
echo currentBuild.result
}
}
In both cases the build won't be aborted upon exception in build job: 'system-check-flow'. In both cases the echo step (and any other following) will be executed.
But there's one important difference between these two options. In first case if the try section raises an exception the overall build status won't be changed (so echo currentBuild.result => SUCCESS). In the second case you overall build will fail (so echo currentBuild.result => FAILURE).
This is important, because you can always fail the overall build in first case (by setting currentBuild.result = 'FAILURE') but you can't repair build in second option (currentBuild.result = 'SUCCESS' won't work).

In addition to simply making the stage pass, it is now also possible to fail the stage, but continue the pipeline and pass the build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this feature is only available since "Pipeline: Basic Steps" 2.16 (May 14, 2019). Before that, catchError is still available but without parameters:
steps {
catchError {
sh "exit 1"
}
}

I was looking for an answer for a long time and I found a hack for it! I put the try/catch block on the whole stage:
try {
stage('some-stage') {
//do something
}
} catch (Exception e) {
echo "Stage failed, but we continue"
}
try {
stage("some-other-stage") { // do something }
} catch (Exception e) {
echo "Stage failed, but we still continue"
}
As result you will get something like this:
This is still not ideal, but it gives the necessary results.

In recent versions it's possible to pass propogate=false option to build step.
link:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
example:
build job:"jobName", propagate:false

Try this example:
stage('StageName1')
{
steps
{
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
SomeCodeThatCanBeErrored
}
}
}
stage('StageName2')
{
steps
{
ContinueOtherCode
}
}

For my decalartive pipeline I have found another solution:
stage('Deploy test')
{
steps
{
bat returnStatus: true, script: 'sc stop Tomcat9'
// The return value of the step will be the status code!
// evaluate return status yourself, or ignore it
}
}
The same works for the sh command to execute scripts on Unix platforms.
The example ignores the return status, because the tomcat might be already stopped, because of a previously failed pipeline run.

In the new pipeline, you can use try-catch to achieve this.
node{
try{
build job: 'system-check-flow'
}
catch (err){
echo "system-check-flow failed"
}
try{
build job: 'job2'
}
catch (err){
echo "job2 failed"
}
}
Here it will build the 'system-check-flow' job. If it fails it will catch the error, ignore, and then move on to build 'job2'

See this post for a full discussion.

pipeline {
agent any
stages {
stage('Stage') {
steps{
script{
jobresult = build(job: './failing-job',propagate:false).result
if(jobresult != 'SUCCESS'){
catchError(stageResult: jobresult, buildResult: 'UNSTABLE'){
error("Downstream job failing-job failed.")
}
}
}
}
}
}
}
For all those that are wondering about how to set the result of a downstream job to the stage/build) Not the most graceful solution, but it gets the job done. Funny thing is that if this stageResult variable was available as a global variable or as a variable outside the catchError block these kinds of solutions would not be needed. Sadly it isn't, and the only way to set the stage result in a pipeline that I thought of is this way. The error() block is needed, otherwise catchError will not set the stageResult/buildResult(the catchError block requires an error, ofcourse).

Complementing the existing working solutions that use catchError as a step or in script, you can also use catchError as a stage option.
This is useful if you have multiple sub stages that you want to catch errors for in the parent stage:
pipeline {
agent any
stages {
stage('Tests') {
options {
catchError(message: "Test failed", stageResult: 'UNSTABLE', buildResult: 'UNSTABLE')
}
stages {
stage('Test 1') {
echo 'test 1 succeeded'
}
stage('Test 2') {
error 'test 2 failed'
}
}
}
}
}
This isn't explicitly documented, but there is a hint that you may use steps as options (emphasis mine):
However, the stage-level options can only contain steps like retry,
timeout, or timestamps, or Declarative options that are relevant to a
stage, like skipDefaultCheckout.
It names a few steps as examples, but not as the only possible steps to be used as options. Also, if you enter an invalid option, Jenkins lists all available options in the error message, which includes catchError.

The cleanest and latest way would be:
stage('Integration Tests') {
steps {
script {
warnError(message: "${STAGE_NAME} stage was unstable.", catchInterruptions: false) {
// your scripts
}
}
}
}
Reference: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#warnerror-catch-error-and-set-build-and-stage-result-to-unstable

you could put the step script inside "post" step, if if it's a teardown like step
code as below:
post {
always {
script {
try{
echo 'put your alway need run scripts here....if it's a teardown like step'
}catch (err) {
echo 'here failed'
}
script{
emailext (
xxxx
)
}
}

Related

How to mark whole Jenkins pipeline build as SUCCESS, after a stage fails and the remaining stages don't run?

My first stage runs a shell script. Exit 0 marks it as success and exit 1 marks it as fail. How can I read this result into the pipeline and get the desired behavior:
Run stage 1
If stage 1 fails, don't run the remaining stages, but mark the whole pipeline as a success
If stage 1 succeeds, run the remaining stages
If any of them fail, mark the pipeline as a fail
If they all succeed, mark the pipeline as a success
I am doing this in a declarative pipeline, how can I enforce this behavior?
You can use something like this, catch the error and then change the currentBuild result :
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
script {
try {
// do something that fails
sh "exit 1"
} catch (Exception err) {
currentBuild.result = 'SUCCESS'
}
}
}
}
stage('Stage 2') {
steps {
echo "Stage 2"
}
}
stage('Stage 3') {
steps {
echo "Stage 3"
}
}
}
}
If you need to change a specific stage result, have a look to this link wich explain how to perform it.

How to execute next stage in sequential stages inspite of previous stage failure in Jenkins pipeline

At the moment I have two stages defined in Jenkins because they both need different agents.
Here is my Proof of concept code
stages {
stage("Run Tests"){
agent docker
stage("Do setup"){}
stage("Testing") {}
post{
always{
echo "always"
}
failure {
echo "failure"
}
}
}
stage("Generate Reports"){
agent node-label
stage("Generate"){
}
}
}
I need "Generate reports" on different agent since certain binaries are on the node and not inside docker container. Tests run inside docker share volume on node so I get all artifacts that are needed to generate report on the node
(I have tried to run "Generate reports" in post stage, but it seems to run inside docker container somehow.)
Now if "Run Tests" fails, "Generate reports" is skipped due to previous stage failure. Any idea how I can force "Generate reports" stage to be run even on failure of previous stage.
Below is the pipeline.
pipeline {
agent none
stages {
stage("Run Tests"){
agent { label "agent1" }
stages {
stage("Do setup"){
steps{
sh 'exit 0'
}
}
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
} //catchError
} //steps
post{
always{
echo "always"
}
failure {
echo "failure"
}
} // post
} // Testing
} // stages
} // Run Test
stage("Generate Reports"){
agent { label "agent2" }
steps {
sh 'exit 0'
}
} // Reports
}
}
The pipeline is successful, but stage Testing is showed as failed, you can choose the status of buildResult and stageResult in case you want it to be unstable or fail:
If you want to run "Generate Reports" stage always then you could mark your earlier stages as unstable if there are failures. By this way Jenkins will execute all stages and will not stop on errors at a particular stage.
Example:
stages {
stage("Run Tests"){
agent docker
stage("Do setup"){}
stage("Testing") {
steps {
script {
// To show an example, Execute "set 1" which will return failure
def result = bat label: 'Check bat......', returnStatus: true, script: "set 1"
if (result !=0) {
// If the result status is not 0 then mark this stage as unstable to continue ahead
// and all later stages will be executed
unstable ('Testing failed')
}
}
}
}
}
}
stage("Generate Reports"){
agent node-label
stage("Generate"){
}
}
}
Option 2 , if you dont want to handle via return status , you can use try and catch block
stage("Testing") {
steps {
script {
try {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
}
catch (e){
unstable('Testing failed!')
}
}
}
}
Option 3: You can change the return status as success of complete build irrespective of stage failure.
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE'){
script {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
}
}
}
NOTE: Option 3 has a drawback it will not execute further steps in the same stage if at all there were any errors.
Example:
In this example, the print message "Testing stage" will not be executed as there were issues in bat set 1 command
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE'){
script {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
// Note : Below step will not be executed since there was failure in prev step
println "Testing stage"
}
}
}
Option 4 : which you already tried out to keep Generate Reports stage in the Post always section of the build so that is executed always irrespective of any failure.

How to mark build success when one of the stages is aborted?

I have a pipeline with stages where one of the stages, intermittently takes longer than expected and hence using timeout to abort it. But if the stage is aborted, build also marked as aborted. Following is the code for the pipeline:
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 10, unit: 'SECONDS')
}
steps {
script {
catchError(buildResult: 'SUCCESS') {
echo "Executing stage I"
sleep 12
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
Even though the stage is marked as Aborted, I want to mark build as Success. Can you please help how I can achieve this?
I would suggest one improvement to Michael's answer (which is correct btw). You can use catchError to mark stage ABORTED (or UNSTABLE) and mark the build SUCCESS, but you need to wrap the code that may timeout with try-catch block to control the error. Consider the following example:
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 3, unit: 'SECONDS')
}
steps {
script {
catchError(buildResult: 'SUCCESS', stageResult: 'ABORTED') {
try {
echo "Executing stage I"
sleep 4
} catch(org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e) {
error "Stage interrupted with ${e.toString()}"
}
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
When you run this pipeline, the stage that timed out is marked as ABORTED, but the pipeline continues and if there is no failure in the remaining stages, it is marked as SUCCESS.
And here is what the UNSTABLE stage status looks like.
Michael's solution works as well, but it produces a slightly different result - the stage that times out is marked as SUCCESS, and this might be less intuitive. You need to click on the stage to check if it timed out or not.
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 3, unit: 'SECONDS')
}
steps {
script {
try {
echo "Executing stage I"
sleep 4
} catch(Exception e) {
currentBuild.result = "SUCCESS"
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
Your catchError() won't work in your case. The documantation (Source) tells the following:
buildResult (optional)
If an error is caught, the overall build result
will be set to this value. Note that the build result can only get
worse, so you cannot change the result to SUCCESS if the current
result is UNSTABLE or worse. Use SUCCESS or null to keep the build
result from being set when an error is caught.
The build status is set with currentBuild.currentResult which can have three values: SUCCESS, UNSTABLE, or FAILURE.
If you want to mark the build as SUCCESS on abortion the post-option (Source) aborted can be used:
pipeline {
agent any
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
post {
aborted {
// Executed only if stage is aborted
currentBuild.result = 'SUCCESS'
}
}
}

Abort current build from pipeline in Jenkins

I have a Jenkins pipeline which has multiple stages, for example:
node("nodename") {
stage("Checkout") {
git ....
}
stage("Check Preconditions") {
...
if(!continueBuild) {
// What do I put here? currentBuild.xxx ?
}
}
stage("Do a lot of work") {
....
}
}
I want to be able to cancel (not fail) the build if certain preconditions are not met and there is no actual work to be done. How can I do this? I know the currentBuild variable is available, but I can't find the documentation for it.
You can mark the build as ABORTED, and then use the error step to cause the build to stop:
if (!continueBuild) {
currentBuild.result = 'ABORTED'
error('Stopping early…')
}
In the Stage View, this will show that the build stopped at this stage, but the build overall will be marked as aborted, rather than failed (see the grey icon for build #9):
After some testing I came up with the following solution:
def autoCancelled = false
try {
stage('checkout') {
...
if (your condition) {
autoCancelled = true
error('Aborting the build to prevent a loop.')
}
}
} catch (e) {
if (autoCancelled) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
This will result into following stage view:
failed stage
If you don't like the failed stage, you have to use return. But be aware you have to skip each stage or wrapper.
def autoCancelled = false
try {
stage('checkout') {
...
if (your condition) {
autoCancelled = true
return
}
}
if (autoCancelled) {
error('Aborting the build to prevent a loop.')
// return would be also possible but you have to be sure to quit all stages and wrapper properly
// return
}
} catch (e) {
if (autoCancelled) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
The result:
custom error as indicator
You can also use a custom message instead of a local variable:
final autoCancelledError = 'autoCancelled'
try {
stage('checkout') {
...
if (your condition) {
echo('Aborting the build to prevent a loop.')
error(autoCancelledError)
}
}
} catch (e) {
if (e.message == autoCancelledError) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
Following this documentation from Jenkins, you should be able to generate an error to stop the build and set the build result like this:
currentBuild.result = 'ABORTED'
Hope that helps.
The thing that we use is:
try {
input 'Do you want to abort?'
} catch (Exception err) {
currentBuild.result = 'ABORTED';
return;
}
The "return" at the end makes sure that no further code is executed.
I handled in a declarative way as shown below:
Based on catchError block it will execute post block.
If post result falls under failure category, the error block will be executed to stop upcoming stages like Production, PreProd etc.
pipeline {
agent any
stages {
stage('Build') {
steps {
catchError {
sh '/bin/bash path/To/Filename.sh'
}
}
post {
success {
echo 'Build stage successful'
}
failure {
echo 'Compile stage failed'
error('Build is aborted due to failure of build stage')
}
}
}
stage('Production') {
steps {
sh '/bin/bash path/To/Filename.sh'
}
}
}
}
Inspired by all the answers I have put all the stuff together into one Scripted Pipeline. Keep in mind this is not a Declarative Pipeline.
To get this example working you will need:
QuickFIX form this answer Jenkins CI Pipeline Scripts not permitted to use method groovy.lang.GroovyObject
discord notifier plugin - https://plugins.jenkins.io/discord-notifier/
Discord channel webhook url filled in the code
The idea I had was to abort the pipeline if it is "replayed" vs started by "run button"(in branches tab of Jenskins BlueOcean):
def isBuildAReplay() {
// https://stackoverflow.com/questions/51555910/how-to-know-inside-jenkinsfile-script-that-current-build-is-an-replay/52302879#52302879
def replyClassName = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replyClassName) }
}
node {
try {
stage('check replay') {
if (isBuildAReplay()) {
currentBuild.result = 'ABORTED'
error 'Biuld REPLAYED going to EXIT (please use RUN button)'
} else {
echo 'NOT replay'
}
}
stage('simple stage') {
echo 'hello from simple stage'
}
stage('error stage') {
//error 'hello from simple error'
}
stage('unstable stage') {
unstable 'hello from simple unstable'
}
stage('Notify sucess') {
//Handle SUCCESS|UNSTABLE
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}", footer: 'No-Code', unstable: true, link: env.BUILD_URL, result: "${currentBuild.currentResult}", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
}
} catch (e) {
echo 'This will run only if failed'
if(currentBuild.result == 'ABORTED'){
//Handle ABORTED
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}\n\nERROR.toString():\n"+e.toString()+"\nERROR.printStackTrace():\n"+e.printStackTrace()+" ", footer: 'No-Code', unstable: true, link: env.BUILD_URL, result: "ABORTED", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
throw e
}else{
//Handle FAILURE
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}\n\nERROR.toString():\n"+e.toString()+"\nERROR.printStackTrace():\n"+e.printStackTrace()+" ", footer: 'No-Code', link: env.BUILD_URL, result: "FAILURE", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
throw e
}
} finally {
echo 'I will always say Hello again!'
}
}
Main trick was the order of lines to achive abort state:
currentBuild.result = 'ABORTED'
error 'Biuld REPLAYED going to EXIT (please use RUN button)'
First set the state then throw an exception.
In the catch block both work:
currentBuild.result
currentBuild.currentResult
If you're able to approve the constructor for FlowInterruptedException, then you can do the following:
throw new FlowInterruptedException(Result.ABORTED, new UserInterruption(getCurrentUserId()))
You can add to your shared library repo a file var/abortError.groovy:
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
import jenkins.model.CauseOfInterruption.UserInterruption
def call(message)
{
currentBuild.displayName = "#${env.BUILD_NUMBER} $message"
echo message
currentBuild.result = 'ABORTED'
throw new FlowInterruptedException(Result.ABORTED, new UserInterruption(env.BUILD_USER_ID))
}
Then you can use it this way (after importing library):
abortError("some message")
Note that if you se following error in console logs:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException hudson.model.Result jenkins.model.CauseOfInterruption[]
You need follow the link form log and approve security exception.
You can go to the script console of Jenkins and run the following to abort a hung / any Jenkins job build/run:
Jenkins .instance.getItemByFullName("JobName")
.getBuildByNumber(JobNumber)
.finish(hudson.model.Result.ABORTED, new java.io.IOException("Aborting build"));
This is the way to abort the currently running build pipeline in Jenkins UI(in Build History there is a cancel button), for capture:
The Executor.interrupt(Result) method is the cleanest, most direct way I could find to stop a build prematurely and choose the result.
script {
currentBuild.getRawBuild().getExecutor().interrupt(Result.ABORTED)
sleep(1) // Interrupt is not blocking and does not take effect immediately.
}
Pros:
Works in a declarative pipeline just as well as a scripted one.
No try/catch or exceptions to handle.
Marks the calling stage and any successive stages as green/passing in the UI.
Cons:
Requires a number of in-process script approvals, including one that is considered insecure. Approve and use with caution.
Taken from my answer on devops.stackexchange.com.
As for currentBuild, have a look at the docs for the RunWrapper class.

How do I assure that a Jenkins pipeline stage is always executed, even if a previous one failed?

I am looking for a Jenkinsfile example of having a step that is always executed, even if a previous step failed.
I want to assure that I archive some builds results in case of failure and I need to be able to have an always-running step at the end.
How can I achieve this?
We switched to using Jenkinsfile Declarative Pipelines, which lets us do things like this:
pipeline {
agent any
stages {
stage('Test') {
steps {
sh './gradlew check'
}
}
}
post {
always {
junit 'build/reports/**/*.xml'
}
}
}
References:
Tests and Artifacts
Jenkins Pipeline Syntax
try {
sh "false"
} finally {
stage 'finalize'
echo "I will always run!"
}
Another possibility is to use a parallel section in combination with a lock. For example:
pipeline {
stages {
parallel {
stage('Stage 1') {
steps {
lock('MY_LOCK') {
echo 'do stuff 1'
}
}
}
stage('Stage 2') {
steps {
lock('MY_LOCK') {
echo 'do stuff 2'
}
}
}
}
}
}
Parallel stages in a parallel section only abort other stages in the same parallel section if the fail fast option for the parallel section is set. See the docs.

Resources