I have a Jenkins pipeline script that for the most part works fine and I surround most things that will fire a fatal error with try catches. However from time to time really unexpected things happen and I'd like to be able to have a safe catch-all available to do some final reporting before failing the build.
Is there no final default 'stage' I can define that runs whenever an error isn't caught?
Although already been answered for a scripted pipeline I would like to point out that for a declarative pipeline this is done with a post section:
pipeline {
agent any
stages {
stage('No-op') {
steps {
sh 'ls'
}
}
}
post {
always {
echo 'One way or another, I have finished'
deleteDir() /* clean up our workspace */
}
success {
echo 'I succeeeded!'
}
unstable {
echo 'I am unstable :/'
}
failure {
echo 'I failed :('
}
changed {
echo 'Things were different before...'
}
}
}
Each stage can also have it's own section when required.
You can do it by wrapping all your build stages in a big try/catch/finally {} block, for example:
node('yournode') {
try {
stage('stage1') {
// build steps here...
}
stage('stage2') {
// ....
}
} catch (e) {
// error handling, if needed
// throw the exception to jenkins
throw e
} finally {
// some common final reporting in all cases (success or failure)
}
}
Related
I am trying to fail a build step in Jenkinsfile with failed results = failure. Once the step is failed it triggers my rollback job. Tried many different things, but had no luck. Any help would be greatly appreciated.
pipeline {
agent any
stages {
stage('Git Checkout') {
steps {
script {
git 'somegit-repo'
sh'''
mvn package
'''
echo currentBuild.result
catchError {
build 'rollback'
}
}
}
}
}
One way is to use a shell script and with exit 1 statement
e.g.
sh "exit 1"
Or you can use error step
error('Failing build because...')
See https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#error-error-signal
Use a try catch block
node {
stage("Run scripts") {
try {
<some command/script>
} catch (error) {
<rollback command/script>
}
}
}
Thank you so much. This seems to work!
stages {
stage("some test") {
steps{
script {
git 'mygitrepo.git'
try {
sh''' mvn test '''
} catch (error) {
script {
def job = build job: 'rollback-job'
}
}
}
}
}
If you check the cleaning and notifications page
You can do a post step and get rid of all the try/catch stuff and get a cleaner Jenkinsfile
pipeline {
agent any
stages {
stage('No-op') {
steps {
sh 'ls'
}
}
}
post {
always {
echo 'One way or another, I have finished'
deleteDir() /* clean up our workspace */
}
success {
echo 'I succeeeded!'
}
unstable {
echo 'I am unstable :/'
}
failure {
echo 'I failed :('
}
changed {
echo 'Things were different before...'
}
}
}
With jenkins build flow plugin this was possible:
ignore(FAILURE){
build( "system-check-flow" )
}
How to do this with Declarative Pipeline syntax?
To ignore a failed step in declarative pipeline you basically have two options:
Use script step and try-catch block (similar to previous proposition by R_K but in declarative style)
stage('someStage') {
steps {
script {
try {
build job: 'system-check-flow'
} catch (err) {
echo err.getMessage()
}
}
echo currentBuild.result
}
}
Use catchError
stage('someStage') {
steps {
catchError {
build job: 'system-check-flow'
}
echo currentBuild.result
}
}
In both cases the build won't be aborted upon exception in build job: 'system-check-flow'. In both cases the echo step (and any other following) will be executed.
But there's one important difference between these two options. In first case if the try section raises an exception the overall build status won't be changed (so echo currentBuild.result => SUCCESS). In the second case you overall build will fail (so echo currentBuild.result => FAILURE).
This is important, because you can always fail the overall build in first case (by setting currentBuild.result = 'FAILURE') but you can't repair build in second option (currentBuild.result = 'SUCCESS' won't work).
In addition to simply making the stage pass, it is now also possible to fail the stage, but continue the pipeline and pass the build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this feature is only available since "Pipeline: Basic Steps" 2.16 (May 14, 2019). Before that, catchError is still available but without parameters:
steps {
catchError {
sh "exit 1"
}
}
I was looking for an answer for a long time and I found a hack for it! I put the try/catch block on the whole stage:
try {
stage('some-stage') {
//do something
}
} catch (Exception e) {
echo "Stage failed, but we continue"
}
try {
stage("some-other-stage") { // do something }
} catch (Exception e) {
echo "Stage failed, but we still continue"
}
As result you will get something like this:
This is still not ideal, but it gives the necessary results.
In recent versions it's possible to pass propogate=false option to build step.
link:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
example:
build job:"jobName", propagate:false
Try this example:
stage('StageName1')
{
steps
{
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
SomeCodeThatCanBeErrored
}
}
}
stage('StageName2')
{
steps
{
ContinueOtherCode
}
}
For my decalartive pipeline I have found another solution:
stage('Deploy test')
{
steps
{
bat returnStatus: true, script: 'sc stop Tomcat9'
// The return value of the step will be the status code!
// evaluate return status yourself, or ignore it
}
}
The same works for the sh command to execute scripts on Unix platforms.
The example ignores the return status, because the tomcat might be already stopped, because of a previously failed pipeline run.
In the new pipeline, you can use try-catch to achieve this.
node{
try{
build job: 'system-check-flow'
}
catch (err){
echo "system-check-flow failed"
}
try{
build job: 'job2'
}
catch (err){
echo "job2 failed"
}
}
Here it will build the 'system-check-flow' job. If it fails it will catch the error, ignore, and then move on to build 'job2'
See this post for a full discussion.
pipeline {
agent any
stages {
stage('Stage') {
steps{
script{
jobresult = build(job: './failing-job',propagate:false).result
if(jobresult != 'SUCCESS'){
catchError(stageResult: jobresult, buildResult: 'UNSTABLE'){
error("Downstream job failing-job failed.")
}
}
}
}
}
}
}
For all those that are wondering about how to set the result of a downstream job to the stage/build) Not the most graceful solution, but it gets the job done. Funny thing is that if this stageResult variable was available as a global variable or as a variable outside the catchError block these kinds of solutions would not be needed. Sadly it isn't, and the only way to set the stage result in a pipeline that I thought of is this way. The error() block is needed, otherwise catchError will not set the stageResult/buildResult(the catchError block requires an error, ofcourse).
Complementing the existing working solutions that use catchError as a step or in script, you can also use catchError as a stage option.
This is useful if you have multiple sub stages that you want to catch errors for in the parent stage:
pipeline {
agent any
stages {
stage('Tests') {
options {
catchError(message: "Test failed", stageResult: 'UNSTABLE', buildResult: 'UNSTABLE')
}
stages {
stage('Test 1') {
echo 'test 1 succeeded'
}
stage('Test 2') {
error 'test 2 failed'
}
}
}
}
}
This isn't explicitly documented, but there is a hint that you may use steps as options (emphasis mine):
However, the stage-level options can only contain steps like retry,
timeout, or timestamps, or Declarative options that are relevant to a
stage, like skipDefaultCheckout.
It names a few steps as examples, but not as the only possible steps to be used as options. Also, if you enter an invalid option, Jenkins lists all available options in the error message, which includes catchError.
The cleanest and latest way would be:
stage('Integration Tests') {
steps {
script {
warnError(message: "${STAGE_NAME} stage was unstable.", catchInterruptions: false) {
// your scripts
}
}
}
}
Reference: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#warnerror-catch-error-and-set-build-and-stage-result-to-unstable
you could put the step script inside "post" step, if if it's a teardown like step
code as below:
post {
always {
script {
try{
echo 'put your alway need run scripts here....if it's a teardown like step'
}catch (err) {
echo 'here failed'
}
script{
emailext (
xxxx
)
}
}
I've tried to find documentation about how in a Jenkinsfile pipeline catching the error that occurs when a user cancels a job in jenkins web UI.
I haven't got the postor try/catch/finally approaches to work, they only work when something fails within the build.
This causes resources not to be free'd up when someone cancels a job.
What I have today, is a script within a declarative pipeline, like so:
pipeline {
stage("test") {
steps {
parallell (
unit: {
node("main-builder") {
script {
try { sh "<build stuff>" } catch (ex) { report } finally { cleanup }
}
}
}
)
}
}
}
So, everything within catch(ex) and finally blocks is ignored when a job is manually cancelled from the UI.
Non-declarative approach:
When you abort pipeline script build, exception of type org.jenkinsci.plugins.workflow.steps.FlowInterruptedException is thrown. Release resources in catch block and re-throw the exception.
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
def releaseResources() {
echo "Releasing resources"
sleep 10
}
node {
try {
echo "Doing steps..."
} catch (FlowInterruptedException interruptEx) {
releaseResources()
throw interruptEx
}
}
Declarative approach (UPDATED 11/2019):
According to Jenkins Declarative Pipeline docs, under post section:
cleanup
Run the steps in this post condition after every other post condition has been evaluated, regardless of the Pipeline or stage’s status.
So that should be good place to free resources, no matter whether the pipeline was aborted or not.
def releaseResources() {
echo "Releasing resources"
sleep 10
}
pipeline {
agent none
stages {
stage("test") {
steps {
parallel (
unit: {
node("main-builder") {
script {
echo "Doing steps..."
sleep 20
}
}
}
)
}
post {
cleanup {
releaseResources()
}
}
}
}
}
You can add a post trigger "cleanup" to the stage:
post {
cleanup {
script { ... }
sh "remove lock"
}
}
I'm trying a post failure action with a parallel step but it never works.
This is my Jenkinsfile:
pipeline {
agent any
stages {
stage("test") {
steps {
withMaven(
maven: 'maven3', // Maven installation declared in the Jenkins "Global Tool Configuration"
mavenSettingsConfig: 'maven_id', // Maven settings.xml file defined with the Jenkins Config File Provider Plugin
mavenLocalRepo: '.repository')
{
// Run the maven build
sh "mvn --batch-mode release:prepare -Dmaven.deploy.skip=true" --> it will always fail
}
}
}
stage("testing") {
steps {
parallel (
phase1: { sh 'echo phase1' },
phase2: { sh "echo phase2" }
)
}
}
}
post {
failure {
echo "FAIL"
}
}
}
But the post failure action here is a bit useles... I don´t see it any place.
Thanks to all!
Regards
I've found the issue, after several hours of searching. What you are missing (and I was missing too) is the catchError section.
pipeline {
agent any
stages {
stage('Compile') {
steps {
catchError {
sh './gradlew compileJava --stacktrace'
}
}
post {
success {
echo 'Compile stage successful'
}
failure {
echo 'Compile stage failed'
}
}
}
/* ... other stages ... */
}
post {
success {
echo 'whole pipeline successful'
}
failure {
echo 'pipeline failed, at least one step failed'
}
}
You should wrap every step that can potentially fail into a catchError function. What this does is:
If an error occurs...
... set build.result to FAILURE...
... and continue the build
The last point is important: your post{ } blocks did not get called because your entire pipeline was aborted before they even had a chance to execute.
Just in case someone else also made the same stupid mistake I did, don't forget the post block needs to be inside the pipeline block.
i.e. This is apparently valid, but (obviously) won't work:
pipeline {
agent { ... }
stages { ... }
}
// WRONG!
post {
always { ... }
}
This is what's correct:
pipeline {
agent { ... }
stages { ... }
post {
always { ... }
}
}
I have a Jenkins pipeline which has multiple stages, for example:
node("nodename") {
stage("Checkout") {
git ....
}
stage("Check Preconditions") {
...
if(!continueBuild) {
// What do I put here? currentBuild.xxx ?
}
}
stage("Do a lot of work") {
....
}
}
I want to be able to cancel (not fail) the build if certain preconditions are not met and there is no actual work to be done. How can I do this? I know the currentBuild variable is available, but I can't find the documentation for it.
You can mark the build as ABORTED, and then use the error step to cause the build to stop:
if (!continueBuild) {
currentBuild.result = 'ABORTED'
error('Stopping early…')
}
In the Stage View, this will show that the build stopped at this stage, but the build overall will be marked as aborted, rather than failed (see the grey icon for build #9):
After some testing I came up with the following solution:
def autoCancelled = false
try {
stage('checkout') {
...
if (your condition) {
autoCancelled = true
error('Aborting the build to prevent a loop.')
}
}
} catch (e) {
if (autoCancelled) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
This will result into following stage view:
failed stage
If you don't like the failed stage, you have to use return. But be aware you have to skip each stage or wrapper.
def autoCancelled = false
try {
stage('checkout') {
...
if (your condition) {
autoCancelled = true
return
}
}
if (autoCancelled) {
error('Aborting the build to prevent a loop.')
// return would be also possible but you have to be sure to quit all stages and wrapper properly
// return
}
} catch (e) {
if (autoCancelled) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
The result:
custom error as indicator
You can also use a custom message instead of a local variable:
final autoCancelledError = 'autoCancelled'
try {
stage('checkout') {
...
if (your condition) {
echo('Aborting the build to prevent a loop.')
error(autoCancelledError)
}
}
} catch (e) {
if (e.message == autoCancelledError) {
currentBuild.result = 'ABORTED'
echo('Skipping mail notification')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
Following this documentation from Jenkins, you should be able to generate an error to stop the build and set the build result like this:
currentBuild.result = 'ABORTED'
Hope that helps.
The thing that we use is:
try {
input 'Do you want to abort?'
} catch (Exception err) {
currentBuild.result = 'ABORTED';
return;
}
The "return" at the end makes sure that no further code is executed.
I handled in a declarative way as shown below:
Based on catchError block it will execute post block.
If post result falls under failure category, the error block will be executed to stop upcoming stages like Production, PreProd etc.
pipeline {
agent any
stages {
stage('Build') {
steps {
catchError {
sh '/bin/bash path/To/Filename.sh'
}
}
post {
success {
echo 'Build stage successful'
}
failure {
echo 'Compile stage failed'
error('Build is aborted due to failure of build stage')
}
}
}
stage('Production') {
steps {
sh '/bin/bash path/To/Filename.sh'
}
}
}
}
Inspired by all the answers I have put all the stuff together into one Scripted Pipeline. Keep in mind this is not a Declarative Pipeline.
To get this example working you will need:
QuickFIX form this answer Jenkins CI Pipeline Scripts not permitted to use method groovy.lang.GroovyObject
discord notifier plugin - https://plugins.jenkins.io/discord-notifier/
Discord channel webhook url filled in the code
The idea I had was to abort the pipeline if it is "replayed" vs started by "run button"(in branches tab of Jenskins BlueOcean):
def isBuildAReplay() {
// https://stackoverflow.com/questions/51555910/how-to-know-inside-jenkinsfile-script-that-current-build-is-an-replay/52302879#52302879
def replyClassName = "org.jenkinsci.plugins.workflow.cps.replay.ReplayCause"
currentBuild.rawBuild.getCauses().any{ cause -> cause.toString().contains(replyClassName) }
}
node {
try {
stage('check replay') {
if (isBuildAReplay()) {
currentBuild.result = 'ABORTED'
error 'Biuld REPLAYED going to EXIT (please use RUN button)'
} else {
echo 'NOT replay'
}
}
stage('simple stage') {
echo 'hello from simple stage'
}
stage('error stage') {
//error 'hello from simple error'
}
stage('unstable stage') {
unstable 'hello from simple unstable'
}
stage('Notify sucess') {
//Handle SUCCESS|UNSTABLE
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}", footer: 'No-Code', unstable: true, link: env.BUILD_URL, result: "${currentBuild.currentResult}", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
}
} catch (e) {
echo 'This will run only if failed'
if(currentBuild.result == 'ABORTED'){
//Handle ABORTED
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}\n\nERROR.toString():\n"+e.toString()+"\nERROR.printStackTrace():\n"+e.printStackTrace()+" ", footer: 'No-Code', unstable: true, link: env.BUILD_URL, result: "ABORTED", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
throw e
}else{
//Handle FAILURE
discordSend(description: "${currentBuild.currentResult}: Job ${env.JOB_NAME} \nBuild: ${env.BUILD_NUMBER} \nMore info at: \n${env.BUILD_URL}\n\nERROR.toString():\n"+e.toString()+"\nERROR.printStackTrace():\n"+e.printStackTrace()+" ", footer: 'No-Code', link: env.BUILD_URL, result: "FAILURE", title: "${JOB_NAME} << CLICK", webhookURL: 'https://discordapp.com/api/webhooks/')
throw e
}
} finally {
echo 'I will always say Hello again!'
}
}
Main trick was the order of lines to achive abort state:
currentBuild.result = 'ABORTED'
error 'Biuld REPLAYED going to EXIT (please use RUN button)'
First set the state then throw an exception.
In the catch block both work:
currentBuild.result
currentBuild.currentResult
If you're able to approve the constructor for FlowInterruptedException, then you can do the following:
throw new FlowInterruptedException(Result.ABORTED, new UserInterruption(getCurrentUserId()))
You can add to your shared library repo a file var/abortError.groovy:
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
import jenkins.model.CauseOfInterruption.UserInterruption
def call(message)
{
currentBuild.displayName = "#${env.BUILD_NUMBER} $message"
echo message
currentBuild.result = 'ABORTED'
throw new FlowInterruptedException(Result.ABORTED, new UserInterruption(env.BUILD_USER_ID))
}
Then you can use it this way (after importing library):
abortError("some message")
Note that if you se following error in console logs:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException hudson.model.Result jenkins.model.CauseOfInterruption[]
You need follow the link form log and approve security exception.
You can go to the script console of Jenkins and run the following to abort a hung / any Jenkins job build/run:
Jenkins .instance.getItemByFullName("JobName")
.getBuildByNumber(JobNumber)
.finish(hudson.model.Result.ABORTED, new java.io.IOException("Aborting build"));
This is the way to abort the currently running build pipeline in Jenkins UI(in Build History there is a cancel button), for capture:
The Executor.interrupt(Result) method is the cleanest, most direct way I could find to stop a build prematurely and choose the result.
script {
currentBuild.getRawBuild().getExecutor().interrupt(Result.ABORTED)
sleep(1) // Interrupt is not blocking and does not take effect immediately.
}
Pros:
Works in a declarative pipeline just as well as a scripted one.
No try/catch or exceptions to handle.
Marks the calling stage and any successive stages as green/passing in the UI.
Cons:
Requires a number of in-process script approvals, including one that is considered insecure. Approve and use with caution.
Taken from my answer on devops.stackexchange.com.
As for currentBuild, have a look at the docs for the RunWrapper class.