I'm trying the new declarative pipeline syntax.
I wonder, how can I abort all the stages and steps of a pipeline, when for example a parameter has an invalid value.
I could add a when clause to every stage, but this isn't optimal for me. Is there a better way to do so?
This should work fine with a when directive, if you make use of the error step.
For example, you could do an up-front check and abort the build if the given parameter value is not acceptable — preventing subsequent stages from running:
pipeline {
agent any
parameters {
string(name: 'targetEnv',
defaultValue: 'dev',
description: 'Must be "dev", "qa", or "staging"')
}
stages {
stage('Validate parameters') {
when {
expression {
// Only run this stage if the targetEnv is invalid
!['dev', 'qa', 'staging'].contains(params.targetEnv)
}
}
steps {
// Abort the build, skipping subsequent stages
error("Invalid target environment: ${params.targetEnv}")
}
}
stage('Checkout') {
steps {
echo 'Checking out source code...'
}
}
stage('Build') {
steps {
echo 'Building...'
}
}
}
}
You could use the FlowInterruptedException, e.g.:
import hudson.model.Result
import org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
pipeline {
...
steps {
script {
throw new FlowInterruptedException(Result.ABORTED)
}
...
}
This will stop execution immediately like the error step, but with more control over the result.
Please note that it requires you to approve a signature:
new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException hudson.model.Result
Apart from Result.ABORTED there's also: Result.SUCCESS, Result.UNSTABLE, Result.FAILURE and Result.NOT_BUILT.
Disclaimer: it's a bit of a hack.
Related
def jobs = [
'subjob1': true,
'subjob2': false,
'subjob3': true
]
pipeline
{
agent { label "ag1" }
stages
{
stage('stage1')
{
steps
{
script
{
jobs.each
{
if ("$it.value".toBoolean())
{
stage("Stage $it.key")
{
build([job:"$it.key", wait:true, propagate:true])
}
}
}
}
}
}
}
}
This Jenkins job triggers other sub-jobs (via pipeline build step): subjob1, subjob2, subjob3. If any of the sub-jobs fail, this job immediately fails (propagate:true).
However, what I'd like to do is continue executing all jobs. And mark this one as failed if one or more sub-jobs fail. How would I do that?
Here is how you can do it. You can simply use a catchError block for this.
def jobs = [
'subjob1': true,
'subjob2': false,
'subjob3': true
]
pipeline
{
agent any
stages
{
stage('stage1')
{
steps
{
script
{
jobs.each
{
if ("$it.value".toBoolean())
{
stage("Stage $it.key")
{
catchError(buildResult: 'FAILURE', stageResult: 'FAILURE')
{
echo "Building"
build([job:"$it.key", wait:true, propagate:true])
}
}
}
}
}
}
}
}
}
Instead of executing all the jobs one by one, you can execute them in parallel. This way, all the jobs will be executed independently of each other and stage1 will fail only if one or more jobs fails.
According to the documentation
The parallel directive takes a map from branch names to closures and an optional argument failFast which will terminate all branches upon a failure in any other branch.
So, we have to transform the jobs to a Map of stage names to Closures that will execute in parallel. We will use jobs.collectEntries() to build the mapping and pass it as the argument to the parallel directive:
stage('Parallel') {
steps {
script {
parallel(jobs.collectEntries {
[(it.key): {
if (it.value) {
build(job: it.key)
} else {
echo "Skipping job execution: ${it.key}"
// This is required to mark the parallel stage as skipped - it is not required for the solution to work
org.jenkinsci.plugins.pipeline.modeldefinition.Utils.markStageSkippedForConditional(it.key)
}
}]
})
}
}
}
We can omit the wait and propagate flags in the build step because they are set by default.
In the provided solution the Parallel stage (and the resulting build) will fail only if one or more jobs fails. Additionally, if you have Blue Ocean plugin installed you will see a nice view graph of Parallel stage along with all the parallel children:
I have a parallel stage setup, and would like to know if it's possible to run a script prior to the nested stages, so something like this:
stage('E2E-PR-CYPRESS') {
when {
allOf {
expression {
return fileExists("cypress.json")
}
branch "PR-*"
}
}
steps {
script {
stash name: 'cypress-dir', includes: 'cypress/**/*'
}
}
parallel {
stage('Cypress Tests 1') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
stage('Cypress Tests 2') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
}
post {
always {
e2eAfterCypressRun(this, true)
}
}
}
I know the above is wrong, I get the error Only one of "matrix", "parallel", "stages", or "steps" allowed for stage "E2E-PR-CYPRESS"
I already have the stash script in a setup stage at the beginning of my pipeline, but I'd like to be able to restart from this stage above on Jenkins, and so need the stash part in this stage as the parallel stages need to unstash the contents.
Updated Answer:
After playing a bit with the Restart from a Stage option there is seems to be a nice feature designed exactly for your needs called Preserving stashes for Use with Restarted Stages:
Normally, when you run the stash step in your Pipeline, the resulting
stash of artifacts is cleared when the Pipeline completes, regardless
of the result of the Pipeline. Since stash artifacts aren’t accessible
outside of the Pipeline run that created them, this has not created
any limitations on usage. But with Declarative stage restarting, you
may want to be able to unstash artifacts from a stage which ran before
the stage you’re restarting from.
To enable this, there is a job property that allows you to configure a
maximum number of completed runs whose stash artifacts should be
preserved for reuse in a restarted run. You can specify anywhere from
1 to 50 as the number of runs to preserve.
This job property can be configured in your Declarative Pipeline’s options section, as below:
options {
preserveStashes()
// or
preserveStashes(buildCount: 5)
}
This built in feature is exactly what you need to solve your issue without any special modifications to your code, as it will allow you to rerun the pipeline from any stage and still use the existing file that were previously stashed.
Original Answer:
You can actually achieve this quite simply using the scripted syntax for the parallel command, and it will also allow you to avoid the duplicate code in the parallel stages.
parallel: Execute in parallel
Takes a map from branch names to closures and an optional argument failFast which will terminate all branches upon a failure in any other branch:
parallel firstBranch: {
// do something
}, secondBranch: {
// do something else
},
failFast: true|false
In your case it can look like:
stage('E2E-PR-CYPRESS') {
when {
allOf {
expression {
return fileExists("cypress.json")
}
branch "PR-*"
}
}
steps {
script {
stash name: 'cypress-dir', includes: 'cypress/**/*'
// Define the parallel execution stages
def stages = ['Cypress Tests 1', 'Cypress Tests 2']
// Create the parallel executions and run them
parallel stages.collectEntries {
["Running ${it}": {
node('aws_micro_slave_e2e') {
skipDefaultCheckout()
runE2eTests()
}
}]
}
}
}
post {
always {
e2eAfterCypressRun(this, true)
}
}
}
This way you can easily add more parallel steps by updating the stages list, or even receive it as an input parameter. In addition you can create the parallel executions by different labels or tests suits, instead of the stage name.
You can add a Prepare stage at the top like this:
stages{
stage('Preperation'){
when {
allOf {
expression {
return fileExists("cypress.json")
}
branch "PR-*"
}
}
steps {
script {
stash name: 'cypress-dir', includes: 'cypress/**/*'
}
}
}
stage('E2E-PR-CYPRESS') {
parallel {
stage('Cypress Tests 1') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
stage('Cypress Tests 2') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
}
}
}
post {
always {
e2eAfterCypressRun(this, true)
}
}
An out of the box concept
Propose splitting the job into 2 parts taking the following into consideration:
Currently use an EC2 plugin, as the current agents are EC2
Running the parallel stages with the same stashed content ready to unstash
Create jenkins pipeline job 1:
This job will checkout the workspace with any type of agent
Create a packer json to create a customised AMI for the EC2
The customised AMI will stash the contents and move to a directory that will appear on the EC2 when the agent is built
Output the AMI ID, run a groovy job to update the EC2 plugin AMI ID with the customised AMI ID to temporarily set the AMI in memory on Jenkins
pipeline {
agent {
docker {
test-container
}
}
options {
buildDiscarder(
logRotator(
numToKeepStr: '10',
artifactNumToKeepStr: '10'
)
)
ansiColor('xterm')
gitConnection("git")
}
stages {
stage('Run Stash Cypress Functional Test') {
steps {
dir('functional-test') {
// develop branch is canary build, all other branches are stable builds
script {
sh """
# script to stash cypress tests
"""
}
}
}
}
stage('Functional Test AMI Build') {
steps {
dir('functional-test/packer') {
withAWS(role: 'PackerBuild', roleAccount: '123456789012', roleSessionName: 'Jenkins-Workflow-FunctionalTest-Packer') {
script {
sh """
# packer json script will require to copy contents from workspace, run the script to stash content
# packer json script will require to capture new AMI ID
# https://discuss.devopscube.com/t/how-to-get-the-ami-id-after-a-packer-build/36
# https://www.packer.io/docs/post-processors/manifest
packer validate FunctionalTestPacker.json
packer build -debug FunctionalTestPacker.json
# grab AMI ID and export as jenkins env variable
"""
}
}
}
}
}
stage('run groovy script to update AMI ID on EC2 plugin') {
steps {
dir(groovy job dir) {
script {
sh """
# run groovy job to update AMI on Jenkins EC2 plugin
# https://gist.github.com/vrivellino/97954495938e38421ba4504049fd44ea
"""
}
}
}
}
stage('Kickoff Functional Test Deploy') {
// pipeline checkbox parameter, when ticked it will automatically kick off the functional test pipeline
when {
expression {params.RUN_TESTS.toBoolean()}
}
steps {
script{
env.branch = params.BRANCH
sh """
echo "Branch is ${branch}"
"""
}
build job: 'workflow/CypressFunctionaTestDeployAndRun',
parameters: [
string(name: 'BRANCH', value: env.branch)
],
wait : false
}
}
}
post {
always {
cleanWs()
}
}
}
Create jenkins pipeline job 2:
This job will create the EC2 agents via the plugin from the customised AMI from pipeline job 1
This means your agents will have the same workspace ready to unstash - so you can execute a parallel run
Also you could move a lot of your user data script that is in the EC2 plugin as part of the customised AMI build, thus cut down the time for each EC2 agent to get ready to carry out execution
pipeline {
stages {
stage('E2E-PR-CYPRESS') {
when {
allOf {
expression {
return fileExists("cypress.json")
}
branch "PR-*"
}
}
}
parallel {
stage('Cypress Tests 1') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
stage('Cypress Tests 2') {
agent { label 'aws_micro_slave_e2e' }
options { skipDefaultCheckout() }
steps {
runE2eTests()
}
}
}
}
post {
always {
e2eAfterCypressRun(this, true)
}
}
}
So I have created a shared library in jenkins with a listener that gets triggered each time the pipelines reads a FlowNode so I can run groovy code before and after each stage, step, etc...
I'm able to call the shared library in a step phase like this:
pipeline {
agent any
stages {
stage('prepare') {
steps{
prepareStepsWrapper()
}
}
stage('step1') {
steps {
echo 'step1'
}
}
stage('step2') {
steps {
echo 'step2'
}
}
stage('step3') {
steps {
echo 'step3'
// fail on purpose
sh 'notfoundexecutablelol'
}
}
stage('step4') {
steps {
echo 'step4'
}
}
}
post{
always{
println env.getEnvironment()
}
}
}
And works pretty great!
With this approach the 'prepare' stage needs to be filtered out so I've switched to the options directive:
pipeline {
agent any
options {
prepareStepsWrapper()
}
stages {
stage('step1') {
steps {
echo 'step1'
}
}
...
}
}
But the pipeline fails with
WorkflowScript: 4: Invalid option type "prepareStepsWrapper"
tl;dr; How can I load a shared library within the options directive?
What does the option-stage do?
The options directive allows configuring Pipeline-specific options
from within the Pipeline itself.
You can't call the shared-library in the options-stage. This stage should not be used for execute any logic, rather it sets configurations for the pipeline. All availables options and the documentation can be found here.
You could try to create a stage that simply calls your prepareStepsWrapper() and use locks to avoid that other stages are executed before this stage.
With jenkins build flow plugin this was possible:
ignore(FAILURE){
build( "system-check-flow" )
}
How to do this with Declarative Pipeline syntax?
To ignore a failed step in declarative pipeline you basically have two options:
Use script step and try-catch block (similar to previous proposition by R_K but in declarative style)
stage('someStage') {
steps {
script {
try {
build job: 'system-check-flow'
} catch (err) {
echo err.getMessage()
}
}
echo currentBuild.result
}
}
Use catchError
stage('someStage') {
steps {
catchError {
build job: 'system-check-flow'
}
echo currentBuild.result
}
}
In both cases the build won't be aborted upon exception in build job: 'system-check-flow'. In both cases the echo step (and any other following) will be executed.
But there's one important difference between these two options. In first case if the try section raises an exception the overall build status won't be changed (so echo currentBuild.result => SUCCESS). In the second case you overall build will fail (so echo currentBuild.result => FAILURE).
This is important, because you can always fail the overall build in first case (by setting currentBuild.result = 'FAILURE') but you can't repair build in second option (currentBuild.result = 'SUCCESS' won't work).
In addition to simply making the stage pass, it is now also possible to fail the stage, but continue the pipeline and pass the build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this feature is only available since "Pipeline: Basic Steps" 2.16 (May 14, 2019). Before that, catchError is still available but without parameters:
steps {
catchError {
sh "exit 1"
}
}
I was looking for an answer for a long time and I found a hack for it! I put the try/catch block on the whole stage:
try {
stage('some-stage') {
//do something
}
} catch (Exception e) {
echo "Stage failed, but we continue"
}
try {
stage("some-other-stage") { // do something }
} catch (Exception e) {
echo "Stage failed, but we still continue"
}
As result you will get something like this:
This is still not ideal, but it gives the necessary results.
In recent versions it's possible to pass propogate=false option to build step.
link:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
example:
build job:"jobName", propagate:false
Try this example:
stage('StageName1')
{
steps
{
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
SomeCodeThatCanBeErrored
}
}
}
stage('StageName2')
{
steps
{
ContinueOtherCode
}
}
For my decalartive pipeline I have found another solution:
stage('Deploy test')
{
steps
{
bat returnStatus: true, script: 'sc stop Tomcat9'
// The return value of the step will be the status code!
// evaluate return status yourself, or ignore it
}
}
The same works for the sh command to execute scripts on Unix platforms.
The example ignores the return status, because the tomcat might be already stopped, because of a previously failed pipeline run.
In the new pipeline, you can use try-catch to achieve this.
node{
try{
build job: 'system-check-flow'
}
catch (err){
echo "system-check-flow failed"
}
try{
build job: 'job2'
}
catch (err){
echo "job2 failed"
}
}
Here it will build the 'system-check-flow' job. If it fails it will catch the error, ignore, and then move on to build 'job2'
See this post for a full discussion.
pipeline {
agent any
stages {
stage('Stage') {
steps{
script{
jobresult = build(job: './failing-job',propagate:false).result
if(jobresult != 'SUCCESS'){
catchError(stageResult: jobresult, buildResult: 'UNSTABLE'){
error("Downstream job failing-job failed.")
}
}
}
}
}
}
}
For all those that are wondering about how to set the result of a downstream job to the stage/build) Not the most graceful solution, but it gets the job done. Funny thing is that if this stageResult variable was available as a global variable or as a variable outside the catchError block these kinds of solutions would not be needed. Sadly it isn't, and the only way to set the stage result in a pipeline that I thought of is this way. The error() block is needed, otherwise catchError will not set the stageResult/buildResult(the catchError block requires an error, ofcourse).
Complementing the existing working solutions that use catchError as a step or in script, you can also use catchError as a stage option.
This is useful if you have multiple sub stages that you want to catch errors for in the parent stage:
pipeline {
agent any
stages {
stage('Tests') {
options {
catchError(message: "Test failed", stageResult: 'UNSTABLE', buildResult: 'UNSTABLE')
}
stages {
stage('Test 1') {
echo 'test 1 succeeded'
}
stage('Test 2') {
error 'test 2 failed'
}
}
}
}
}
This isn't explicitly documented, but there is a hint that you may use steps as options (emphasis mine):
However, the stage-level options can only contain steps like retry,
timeout, or timestamps, or Declarative options that are relevant to a
stage, like skipDefaultCheckout.
It names a few steps as examples, but not as the only possible steps to be used as options. Also, if you enter an invalid option, Jenkins lists all available options in the error message, which includes catchError.
The cleanest and latest way would be:
stage('Integration Tests') {
steps {
script {
warnError(message: "${STAGE_NAME} stage was unstable.", catchInterruptions: false) {
// your scripts
}
}
}
}
Reference: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#warnerror-catch-error-and-set-build-and-stage-result-to-unstable
you could put the step script inside "post" step, if if it's a teardown like step
code as below:
post {
always {
script {
try{
echo 'put your alway need run scripts here....if it's a teardown like step'
}catch (err) {
echo 'here failed'
}
script{
emailext (
xxxx
)
}
}
I am looking for a Jenkinsfile example of having a step that is always executed, even if a previous step failed.
I want to assure that I archive some builds results in case of failure and I need to be able to have an always-running step at the end.
How can I achieve this?
We switched to using Jenkinsfile Declarative Pipelines, which lets us do things like this:
pipeline {
agent any
stages {
stage('Test') {
steps {
sh './gradlew check'
}
}
}
post {
always {
junit 'build/reports/**/*.xml'
}
}
}
References:
Tests and Artifacts
Jenkins Pipeline Syntax
try {
sh "false"
} finally {
stage 'finalize'
echo "I will always run!"
}
Another possibility is to use a parallel section in combination with a lock. For example:
pipeline {
stages {
parallel {
stage('Stage 1') {
steps {
lock('MY_LOCK') {
echo 'do stuff 1'
}
}
}
stage('Stage 2') {
steps {
lock('MY_LOCK') {
echo 'do stuff 2'
}
}
}
}
}
}
Parallel stages in a parallel section only abort other stages in the same parallel section if the fail fast option for the parallel section is set. See the docs.