I have a stage in my pipeline job that requires user input to proceed to the next stage. The problem is sometime I forget to click the proceed button after N minutes of waiting. I want to send a Slack message to notify me that the stage has been paused for N minutes. Is there a way to achieve it?
Below is the sample of my pipeline script:
pipeline {
agent any
stages {
stage('A') {
steps {
echo 'Starting Stage A'
input message: 'Continue to the next stage?'
// send Slack message after 15 minutes user didn't click Proceed/Abort button
// but still wait for user input (don't mark it as failed even after 15 minutes)
}
}
stage('B') {
steps {
echo 'Starting Stage B'
}
}
}
}
I have tried using Jenkins timeout feature using the pipeline script below
But, if the timeout is reached, it will continue to the next stage automatically. What I want is even after the timeout is reached, still wait for user input on Stage A (don't continue to the Stage B immediately)
pipeline {
agent any
stages {
stage('A') {
steps {
script {
try {
echo 'Starting Stage A'
timeout(time: 15, unit: 'MINUTES') {
input message: 'Continue to the next stage?'
}
} catch (err) {
def user = err.getCauses()[0].getUser()
if ('SYSTEM' == user.toString()) { // failed because of timeout
// send Slack message
// how to still wait for user input on this stage even after timeout is reached?
}
}
}
}
}
stage('B') {
steps {
echo 'Starting Stage B'
}
}
}
}
Thanks
I figured it out using the script below:
pipeline {
agent any
stages {
stage('A') {
steps {
script {
echo 'Starting Stage A'
def showInputAgain = false
def proceed = true
try {
timeout(time: 15, unit: 'SECONDS') {
input message: 'Continue?'
}
} catch (err) {
def user = err.getCauses()[0].getUser()
if ('SYSTEM' == user.toString()) {
showInputAgain = true
echo 'Stage A failed. Reach timeout. Sending message to Slack'
} else {
proceed = false
echo 'Stage A failed. User abort'
}
}
if (proceed && showInputAgain) {
input message: 'Continue?'
}
}
}
}
stage('B') {
steps {
echo 'Starting Stage B'
}
}
}
}
But any other inputs/answers will be appreciated
Related
I have a pipeline with stages where one of the stages, intermittently takes longer than expected and hence using timeout to abort it. But if the stage is aborted, build also marked as aborted. Following is the code for the pipeline:
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 10, unit: 'SECONDS')
}
steps {
script {
catchError(buildResult: 'SUCCESS') {
echo "Executing stage I"
sleep 12
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
Even though the stage is marked as Aborted, I want to mark build as Success. Can you please help how I can achieve this?
I would suggest one improvement to Michael's answer (which is correct btw). You can use catchError to mark stage ABORTED (or UNSTABLE) and mark the build SUCCESS, but you need to wrap the code that may timeout with try-catch block to control the error. Consider the following example:
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 3, unit: 'SECONDS')
}
steps {
script {
catchError(buildResult: 'SUCCESS', stageResult: 'ABORTED') {
try {
echo "Executing stage I"
sleep 4
} catch(org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e) {
error "Stage interrupted with ${e.toString()}"
}
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
When you run this pipeline, the stage that timed out is marked as ABORTED, but the pipeline continues and if there is no failure in the remaining stages, it is marked as SUCCESS.
And here is what the UNSTABLE stage status looks like.
Michael's solution works as well, but it produces a slightly different result - the stage that times out is marked as SUCCESS, and this might be less intuitive. You need to click on the stage to check if it timed out or not.
pipeline {
agent any
stages {
stage('First') {
options {
timeout(time: 3, unit: 'SECONDS')
}
steps {
script {
try {
echo "Executing stage I"
sleep 4
} catch(Exception e) {
currentBuild.result = "SUCCESS"
}
}
}
}
stage('Second') {
steps {
script {
echo "Executing stage II"
}
}
}
}
}
Your catchError() won't work in your case. The documantation (Source) tells the following:
buildResult (optional)
If an error is caught, the overall build result
will be set to this value. Note that the build result can only get
worse, so you cannot change the result to SUCCESS if the current
result is UNSTABLE or worse. Use SUCCESS or null to keep the build
result from being set when an error is caught.
The build status is set with currentBuild.currentResult which can have three values: SUCCESS, UNSTABLE, or FAILURE.
If you want to mark the build as SUCCESS on abortion the post-option (Source) aborted can be used:
pipeline {
agent any
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
post {
aborted {
// Executed only if stage is aborted
currentBuild.result = 'SUCCESS'
}
}
}
I have a stage which runs one shell script at remote node. If the script takes very long time to execute, the stage should wait for sometime and should move to next stage without aborting the subsequent stages.
Could you please let me know the required syntax to achieve this.
in a declarative pipeline, add this stage:
stage ("do-and-skip-for-timeout") {
steps {
script {
try {
timeout (time: 10, unit: 'MINUTES') {
echo "do something here, that can take some time" // replace this line with your code
}
}
catch (error) {
def user = error.getCauses()[0].getUser()
if('SYSTEM' == user.toString()) { // SYSTEM means timeout.
echo "Timeout reached, continue to next stage"
}
else {
throw new Exception("[ERROR] stage failed!")
}
}
}
}
I want when some of the stages in the Jenkins pipeline get error then to have some options like asking for input, or print some message.
For example in this pipeline error is in the stage 2 by executing not existing 'shit' (sh 'asdasd')
node('master') {
try {
stage('stage1') {
echo 'stage 1 succeeeded'
}
stage('stage2') {
echo '1st task of stage 2 is cool'
// this is the ERROR
sh 'asdasd'
}
} finally {
echo 'SUCCESS'
}
}
My question is how ask for input IF error popup if some of the stages?
There are several posts that show how to capture errors using try - catch blocks inside anything, stages in this case: see e.g. this or this
What you want additionally is define an user input in the "catch" case. In general, that is a bad idea, see this article describing why user inputs are dangerous in pipelines. But the article also offers a solution including timeouts.
If really needed, code can be something like...
node('master') {
try {
stage('stage1') {
echo 'stage 1 succeeeded'
}
stage('stage2') {
try {
echo '1st task of stage 2 is cool'
// this is the ERROR
sh 'asdasd'
} catch (errScript) {
try {
timeout(time: 60, unit: 'SECONDS') { // set desired timeout
userInput = input( id: 'userInput',
message: 'Press OK to continue',
)
}
} catch (err) { // timeout reached or user input
def user = err.getCauses()[0].getUser()
if('SYSTEM' == user.toString()) { // SYSTEM means timeout.
// Do your timeout thing
} else {
// Evaluate your user input action
}
}
} // catch (errScript)
}
} finally {
echo 'SUCCESS'
}
I'm creating a declarative Jenkins pipeline, that looks like this:
pipeline {
agent {
label 'mylabel'
}
stages {
stage('Install dependencies') {
milestone()
steps {
sh "yarn install"
}
}
stage('Lint files') {
steps {
sh "eslint src"
}
}
stage('Create bundle') {
steps {
sh "yarn run build:server"
sh "yarn run build:client"
}
}
stage('Publish') {
steps {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
// deployment steps
}
}
}
}
It works great, however, if the timeout step fails (because we don't want to deploy this build, or nobody is there to push the button, and so on), the build is marked with status "aborted". Unfortunately this means that for example Github marks our pull requests as "checks failing".
Is there a way to declare the build with the status that it had before the timeout() step? Eg. if the build was a success up until the timeout step, it should be marked as success, even if the timeout happens.
Since all you want is to let the build abort without marking it as failed, you can just add a simple try/catch to your code.
stage('Publish') {
steps {
script {
def proceed = true
try {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
} catch (err) {
proceed = false
}
if(proceed) {
// deployment steps
}
}
}
}
If a user aborts the build or it times out, the error is suppressed, the build is still a success, and the deployment steps won't execute.
We have a situation where we don't want to start a build if there are no user commits. Because of a bug in the scm trigger prevention on message or user.
What we then do is fail the build with a NOT_BUILT result.
Maybe this will also work for your situation
Try the following in your script
try {
stage ('wait') {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
}
} catch (err) {
def user = err.getCauses()[0].getUser()
if('SYSTEM' == user.toString()) { //timeout
currentBuild.result = "SUCCESS"
}
}
If you don't want the job to be marked as "aborted" or "failed":
For a Declarative pipeline (not a scripted) you would do something like this
stage('Foo') {
steps {
script {
env.PROCEED_TO_DEPLOY = 1
try {
timeout(time: 2, unit: 'MINUTES') {
// ...
}
} catch (err) {
env.PROCEED_TO_DEPLOY = 0
}
}
}
}
stage('Bar') {
when {
expression {
env.PROCEED_TO_DEPLOY == 1
}
}
//rest of your pipeline
stage "Bar" would be skipped but as for the rest of the stages the job will be marked as passed (assuming nothing wrong happened before)
You could probably adjust currentBuild.result in post section of declarative pipeline.
E.g.
pipeline {
stages {
...
}
post {
aborted {
script {
currentBuild.result = 'SUCCESS'
}
}
}
}
Here's the code I'm playing with
node {
stage 'build'
echo 'build'
stage 'tests'
echo 'tests'
stage 'end-to-end-tests'
def e2e = build job:'end-to-end-tests', propagate: false
result = e2e.result
if (result.equals("SUCCESS")) {
stage 'deploy'
build 'deploy'
} else {
?????? I want to just fail this stage
}
}
Is there any way for me to mark the 'end-to-end-tests' stage as failed without failing the whole job? Propagate false just always marks the stage as true, which is not what I want, but Propagate true marks the job as failed which I also don't want.
This is now possible, even with declarative pipelines:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this is a fairly new feature.
Stage takes a block now, so wrap the stage in try-catch. Try-catch inside the stage makes it succeed.
The new feature mentioned earlier will be more powerful. In the meantime:
try {
stage('end-to-end-tests') {
node {
def e2e = build job:'end-to-end-tests', propagate: false
result = e2e.result
if (result.equals("SUCCESS")) {
} else {
sh "exit 1" // this fails the stage
}
}
}
} catch (e) {
result = "FAIL" // make sure other exceptions are recorded as failure too
}
stage('deploy') {
if (result.equals("SUCCESS")) {
build 'deploy'
} else {
echo "Cannot deploy without successful build" // it is important to have a deploy stage even here for the current visualization
}
}
Sounds like JENKINS-26522. Currently the best you can do is set an overall result:
if (result.equals("SUCCESS")) {
stage 'deploy'
build 'deploy'
} else {
currentBuild.result = e2e.result
// but continue
}
I recently tried to use vaza's answer
Show a Jenkins pipeline stage as failed without failing the whole job as template for writing a function that excutes a job in an own stage named like the job name. Surprisingly it worked, but maybe some groovy experts have a look at it :)
Here is how it looks like if one of the jobs is aborted:
def BuildJob(projectName) {
try {
stage(projectName) {
node {
def e2e = build job:projectName, propagate: false
result = e2e.result
if (result.equals("SUCCESS")) {
} else {
error 'FAIL' //sh "exit 1" // this fails the stage
}
}
}
} catch (e) {
currentBuild.result = 'UNSTABLE'
result = "FAIL" // make sure other exceptions are recorded as failure too
}
}
node {
BuildJob('job1')
BuildJob('job2')
}
In order to show a successful build with a failed stage when a downstream job fails AND support a user being able to cancel a build (including all subsequent stages), I had to use a combination of various solutions, specifically when, try/catch, throw and catchError().
env.GLOBAL_BUILD_ABORTED = false // Set if the user aborts the build
pipeline {
agent any
stages {
stage('First Stage') {
when { expression { env.GLOBAL_BUILD_ABORTED.toBoolean() == false } }
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
myLocalBuildMethod('Stage #1, build #1')
myLocalBuildMethod('Stage #1, build #2')
}
}
}
stage('Second Stage') {
when { expression { env.GLOBAL_BUILD_ABORTED.toBoolean() == false } }
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
myLocalBuildMethod('Stage #2, build #1')
myLocalBuildMethod('Stage #2, build #2')
myLocalBuildMethod('Stage #2, build #3')
}
}
}
}
}
def myLocalBuildMethod(myString) {
/* Dummy method to show User Aborts vs Build Failures */
echo "My Local Build Method: " + myString
try {
build (
job: "Dummy_Downstream_Job"
)
} catch (e) {
/* Build Aborted by user - Stop All Test Executions */
if (e.getMessage().contains("was cancelled") || e.getMessage().contains("ABORTED")) {
env.GLOBAL_BUILD_ABORTED = true
}
/* Throw the execiption to be caught by catchError() to mark the stage failed. */
throw (e)
}
// Do other stuff...
}
You could add a explicit fail task, such as 'sh "not exist command"' in the stage.
if (result.equals("SUCCESS")) {
stage 'deploy'
build 'deploy'
} else {
try {
sh "not exist command"
}catch(e) {
}
}
Solution steps
You must emit an error in a stage to mark it as an error
Outside the scope of the stage, handle the exception and choose the build status
This makes the effect desired by a couple of users here, including myself, #user3768904, #Sviatlana
Success with failed Step Example
node("node-name") {
try {
stage("Process") {
error("This will fail")
}
} catch(Exception error) {
currentBuild.result = 'SUCCESS'
return
}
stage("Skipped") {
// This stage will never run
}
}
Aborted with failure Step Example
node("node-name") {
try {
stage("Process") {
error("This will fail")
}
} catch(Exception error) {
currentBuild.result = 'ABORTED'
return
}
stage("Skipped") {
// This stage will never run
}
}
You can use the following code in your else statement:
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
error "some err msg"
}
This could be a general pattern showing how to customize the stage result with nice messages using the built-in functions and propagate the sub-job's result to the stage result. That the overall build is marked unstable if a sub-job is not successful is just a implementation choice for this example.
def run_sub_job() {
def jobBuild = build(job: 'foo', wait: true, propagate: false)
def result = jobBuild.getResult()
def msg = 'sub-job: ' + result
if ('SUCCESS' == result) {
println(msg)
} else if ('UNSTABLE' == result) {
unstable(msg) // will also set buildResult to UNSTABLE
} else { // anything else (FAILURE, ABORTED ...) is considered an error
catchError(
buildResult: 'UNSTABLE',
stageResult: result // propagate sub-job result
) {
error(msg)
}
}
}