We have a situation where JDK1.7 is used to compile a project and JDK1.8 is used in sonar pipeline job on the aforementioned compiled code.
Is it possible to use 2 JDKs in a single scripted job. If sonar stage is reached, then JDK 1.8 must be used.
Currently this is our pipeline.
node('jenkins_uat_sdg_1g'){
env.JAVA_HOME="${tool 'JDK1.7_110'}"
env.PATH="${env.JAVA_HOME}/bin:${env.PATH}"
deleteDir()
try{
try{
stage('SCM Code Checkout'){
echo "Checking out source code from SVN..."
echo "successfully checked out from SVN"
}
} catch(error){
println("Unable to checkout...there were some errors!")
currentBuild.result = "FAILURE"
error()
}
try{
stage('Compile & Package Generation'){
echo "Begining to compile the code"
bat label: 'build-maven', script: 'mvn -f pom.xml clean compile install'
echo "Successfully compiled"
}
}catch(error){
println("Unable to compile...there were some errors!")
currentBuild.result = "FAILURE"
error()
}
}
Different pipeline script is used for sonar analysis.
node('jenkins_uat_sdg_1g'){
env.JAVA_HOME="${tool 'JDK1.8'}"
env.PATH="${env.JAVA_HOME}/bin:${env.PATH}"
stage('SCM Code Checkout'){
echo "Checking out source code from SVN..."
echo "successfully checked out from SVN"
}
stage('sonarqube analysis'){
withSonarQubeEnv('SonarServer'){
bat label: 'sonar-analyis', script: '"D:/Apache Build Tools/apache-maven-3.6.1-bin/apache-maven-3.6.1/bin/mvn" org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar -f parent/pom.xml -Dsonar.host.url=http://10.xx.xx.xx:9500 -Dsonar.login=xxxxxxxxxxxxxxxxxxxxxa4b4cf180c6'
}
}
}
stage('Quality Gate'){
timeout(time: 5, unit: 'MINUTES'){
def qg = waitForQualityGate()
if(qg.status != 'OK'){
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Is it possible to have a single script which utilizes two JDKs in a single job.
Regards
You can make use of withEnv step to set one or more environment variables within a block. These are available to any external processes spawned within that scope
The following code works for me where i am accessing different Java versions in different stages:
node('slave1') {
deleteDir()
try {
try {
stage('During Build') {
withEnv(['JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin']) {
sh '${JAVA_HOME}/java -version'
println("or whatever command you want to run in this block...")
}
}
} catch(error) {
println("Unable to find Java 7!")
currentBuild.result = "FAILURE"
error()
}
try {
stage('During Sonar Analysis') {
withEnv(['JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin']) {
sh '${JAVA_HOME}/java -version'
println("or whatever command you want to run in this block...")
}
}
} catch(error) {
println("Unable to find Java 8!")
currentBuild.result = "FAILURE"
error()
}
} catch(error) {
println("Last catch block!")
error()
}
}
Note: As you would have noticed above, we are using single quotes in Groovy, so the variable expansion is being done by the Bourne shell, not Jenkins
Output:
Related
I have a series of steps in a stage that I want to run even if the first one fails. I want the stage result to fail and the build to get aborted, but only after all steps have run. For example,
pipeline {
agent any
stages {
stage('Run Test') {
steps {
sh "echo running unit-tests"
sh "echo running linting && false" // failure
sh "echo generating report" // This should still run (It currently doesn't)
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This should still run (It currently doesn't)
junit 'unit-test.xml' // This should still run (It currently doesn't)
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
The result should be a failed build where the "Run Test" stage failed and the "Deploy" stage did not run. Is this possible?
P.S.
I am NOT asking for the same behavior as in Continue Jenkins pipeline past failed stage. I want to run the steps following the failure, but not any of the stages afterwards. I tried to enclose each of the test steps with catchError (buildResult: 'FAILURE', stageResult: 'FAILURE'), but the "Deploy" stage still runs.
EDIT:
I cannot combine all the steps into one big sh step and capture its return code because some of the steps are not shell commands, but instead jenkins steps like junit and publishCoverage.
A script witha non-zero exit code will always cause a jenkins step to fail. You can use returnStatus as true so that jenkins does not fails the step.
Additionally considering your use case, you could use a post always execution, so that the steps are always carried out.
Please see below reference example:
stage('Run Test') {
steps {
def unit_test_result= sh returnStatus: true, script: 'echo "running unit-tests"'
def lint_result= sh returnStatus: true, script: 'echo "running linting"'
if (unit_test_result!=0 || lint_result!=0 ) {
// If the unit_test_result or lint_result status is not 0 then mark this stage as unstable to continue ahead
// and all later stages will be executed
unstable ('Testing failed')
// You can also mark as failed as below and it will not conintue other stages:
// error ('Testing failed')
}
}
post {
always {
// This block would always be executed inspite of failure
sh "echo generating report"
publishCoverage adapters: [coberturaAdapter("coverage.xml")]
junit 'unit-test.xml'
}
}
}
I found a slightly hacky way to get the behavior I want. The other answers didn't work for me, either because they need all the steps to be sh steps, or they don't stop the deploy stage from running. I used catchError to set the build and stage result. But to prevent the next stage from running, I needed to an explicit call to error if the stage failed.
pipeline {
agent any
stages {
stage('Run Test') {
steps {
script {
// catchError sets the stageResult to FAILED, but does not stop next stages from running
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running unit-tests"
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running linting && false" // failure
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo generating report" // This still runs
}
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This still runs
junit 'unit-test.xml' // This still runs
if (currentBuild.result == "FAILURE") { // This is needed to stop the next stage from running
error("Stage Failed")
}
}
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
Theoretically you should be able to use sh "<command>||true" It would ignore the error on command and continue. However, Jenkins will not fail as it would ignore the error.
If you don't want Jenkins to ignore the error and want it to stop at the end of the stage, you can do something like: sh "<command>||$error=true" then fail the build based on the $error variable. (sh "$error" might be enough but I am not sure, may require an if statement at the end.) It will be only set to true iff command fails.
Another option is to wrap your build steps in a try-catch block! if there's an exception, i.e. return code of build is not 0 you can catch it, mark the build as unstable and then the rest of the pipeline continues on.
here's an example `
pipeline {
agent {
node {
label 'linux'
}
}
options {
timestamps()
disableConcurrentBuilds()
buildDiscarder(logRotator(numToKeepStr: '3'))
}
tools {
maven 'Maven 3.6.3'
jdk 'jdk11'
}
stages {
stage('CleanWS') {
steps {
cleanWs()
}
}
stage('Build') {
steps {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml clean install -DskipTests -Pregression-test -Dmaven.javadoc.skip=true"
}
}
}
stage('Test') {
steps {
script {
try {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -B verify -Dmaven.source.skip=true -Dmaven.javadoc.skip=true"
}
} catch (exc) {
currentBuild.result = 'UNSTABLE'
}
}
}
post {
always {
script {
junit "**/surefire-reports/*.xml"
}
}
}
}
stage('Sonar Analyse') {
steps {
script {
withMaven(options: [artifactsPublisher(disabled: true)]) {
withSonarQubeEnv("SonarQube") {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn sonar:sonar"
}
}
}
}
}
stage('Deploy to Nexus') {
steps {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml -B clean deploy -DdeployAtEnd=true -DskipTests"
}
}
}
post {
failure {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
unstable {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Build Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
}
}`
At the moment I have two stages defined in Jenkins because they both need different agents.
Here is my Proof of concept code
stages {
stage("Run Tests"){
agent docker
stage("Do setup"){}
stage("Testing") {}
post{
always{
echo "always"
}
failure {
echo "failure"
}
}
}
stage("Generate Reports"){
agent node-label
stage("Generate"){
}
}
}
I need "Generate reports" on different agent since certain binaries are on the node and not inside docker container. Tests run inside docker share volume on node so I get all artifacts that are needed to generate report on the node
(I have tried to run "Generate reports" in post stage, but it seems to run inside docker container somehow.)
Now if "Run Tests" fails, "Generate reports" is skipped due to previous stage failure. Any idea how I can force "Generate reports" stage to be run even on failure of previous stage.
Below is the pipeline.
pipeline {
agent none
stages {
stage("Run Tests"){
agent { label "agent1" }
stages {
stage("Do setup"){
steps{
sh 'exit 0'
}
}
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
} //catchError
} //steps
post{
always{
echo "always"
}
failure {
echo "failure"
}
} // post
} // Testing
} // stages
} // Run Test
stage("Generate Reports"){
agent { label "agent2" }
steps {
sh 'exit 0'
}
} // Reports
}
}
The pipeline is successful, but stage Testing is showed as failed, you can choose the status of buildResult and stageResult in case you want it to be unstable or fail:
If you want to run "Generate Reports" stage always then you could mark your earlier stages as unstable if there are failures. By this way Jenkins will execute all stages and will not stop on errors at a particular stage.
Example:
stages {
stage("Run Tests"){
agent docker
stage("Do setup"){}
stage("Testing") {
steps {
script {
// To show an example, Execute "set 1" which will return failure
def result = bat label: 'Check bat......', returnStatus: true, script: "set 1"
if (result !=0) {
// If the result status is not 0 then mark this stage as unstable to continue ahead
// and all later stages will be executed
unstable ('Testing failed')
}
}
}
}
}
}
stage("Generate Reports"){
agent node-label
stage("Generate"){
}
}
}
Option 2 , if you dont want to handle via return status , you can use try and catch block
stage("Testing") {
steps {
script {
try {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
}
catch (e){
unstable('Testing failed!')
}
}
}
}
Option 3: You can change the return status as success of complete build irrespective of stage failure.
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE'){
script {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
}
}
}
NOTE: Option 3 has a drawback it will not execute further steps in the same stage if at all there were any errors.
Example:
In this example, the print message "Testing stage" will not be executed as there were issues in bat set 1 command
stage("Testing") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE'){
script {
// To show an example, Execute "set 1" which will return failure
bat "set 1"
// Note : Below step will not be executed since there was failure in prev step
println "Testing stage"
}
}
}
Option 4 : which you already tried out to keep Generate Reports stage in the Post always section of the build so that is executed always irrespective of any failure.
I'm implementing a try catch block on most of my stages inside my jenkins pipeline to skip all the following stages when the current stage fails however, one of my stages is returning an error but still continues to execute the next stages.
I've tried using sh 'exit 1', currentStage.result = 'FAILED', if else clause to check the stage result but to no avail.
pipeline {
agent none
stages {
stage ('one') {
steps {
echo 'Next stage should be skipped if this stage fails'
script {
try {
sh '''#!/bin/bash -l
source ~/.nvm/nvm.sh
nvm use node
node somefile.js'''
}
catch (e) {
currentBuild.result = 'FAILURE';
throw e
}
}
}
}
stage ('two') {
steps {
echo 'This stage should be skipped if prior stage throws an erorr'
}
}
}
}
I expect stage two to be skipped as my somefile.js is throwing an error.
You can use the when-clause that Jenkins provides (Source).
stage ('two') {
// Skip this stage if pipeline has already failed
when { not { equals expected: 'FAILURE', actual: currentBuild.result } }
steps {
echo 'This stage should be skipped if prior stage throws an erorr'
}
}
Using Jenkins now, this is my post build Pipeline:
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Hello this is stage 1'
}
}
stage('Stage 2') {
steps {
echo 'Still here? Well this is stage 2'
}
}
stage('Stage 3') {
steps {
echo 'Almost there ... stage 3'
}
}
stage('Stage 4 - Python') {
steps {
bat 'py atom_python_test.py'
}
}
stage('Stage 5 - Compile C++') {
steps {
catchError {
bat '''
cd C:\Program Files (x86)//Microsoft Visual Studio 12.0//VC
call vcvarsall
cd C://Users//praktikant3//Documents//Visual Studio 2013//Projects//FirstProjectBuiltOnJenkins//FirstProjectBuiltOnJenkins
dir
MSBuild.exe
echo 'Compile Stage successful'
'''
}
}
}
stage('Stage 6 - Build C++') {
steps {
bat '''
cd C://Program Files (x86)//Microsoft Visual Studio 12.0//VC
call vcvarsall
cd C://Users//praktikant3//Documents//Visual Studio 2013//Projects//FirstProjectBuiltOnJenkins//FirstProjectBuiltOnJenkins
dir
MSBuild.exe
'''
}
}
}
post {
success {
mail to: 'SomeEMAIL#company.com',
subject: "Succeeding Pipeline: ${currentBuild.fullDisplayName}",
body: "All is good with ${env.BUILD_URL}"
}
failure {
mail to: 'SomeEMAIL#company.com',
subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
body: "Something is wrong with ${env.BUILD_URL}"
}
}
}
The build C++ part works because I the syntax is correct, however in previous attempt I had the directory change using "\" which was not working. I added a "Compile stage" using catchError and one of the cd with a "\" to see if I get notification of failure. From my understanding:
"You should wrap every step that can potentially fail into a catchError function. What this does is:
If an error occurs...
... set build.result to FAILURE...
... andcontinue the build "
but when I intentionally left the back slash the pipeline failed and stopped and the post block did not execute.
With jenkins build flow plugin this was possible:
ignore(FAILURE){
build( "system-check-flow" )
}
How to do this with Declarative Pipeline syntax?
To ignore a failed step in declarative pipeline you basically have two options:
Use script step and try-catch block (similar to previous proposition by R_K but in declarative style)
stage('someStage') {
steps {
script {
try {
build job: 'system-check-flow'
} catch (err) {
echo err.getMessage()
}
}
echo currentBuild.result
}
}
Use catchError
stage('someStage') {
steps {
catchError {
build job: 'system-check-flow'
}
echo currentBuild.result
}
}
In both cases the build won't be aborted upon exception in build job: 'system-check-flow'. In both cases the echo step (and any other following) will be executed.
But there's one important difference between these two options. In first case if the try section raises an exception the overall build status won't be changed (so echo currentBuild.result => SUCCESS). In the second case you overall build will fail (so echo currentBuild.result => FAILURE).
This is important, because you can always fail the overall build in first case (by setting currentBuild.result = 'FAILURE') but you can't repair build in second option (currentBuild.result = 'SUCCESS' won't work).
In addition to simply making the stage pass, it is now also possible to fail the stage, but continue the pipeline and pass the build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this feature is only available since "Pipeline: Basic Steps" 2.16 (May 14, 2019). Before that, catchError is still available but without parameters:
steps {
catchError {
sh "exit 1"
}
}
I was looking for an answer for a long time and I found a hack for it! I put the try/catch block on the whole stage:
try {
stage('some-stage') {
//do something
}
} catch (Exception e) {
echo "Stage failed, but we continue"
}
try {
stage("some-other-stage") { // do something }
} catch (Exception e) {
echo "Stage failed, but we still continue"
}
As result you will get something like this:
This is still not ideal, but it gives the necessary results.
In recent versions it's possible to pass propogate=false option to build step.
link:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
example:
build job:"jobName", propagate:false
Try this example:
stage('StageName1')
{
steps
{
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE')
{
SomeCodeThatCanBeErrored
}
}
}
stage('StageName2')
{
steps
{
ContinueOtherCode
}
}
For my decalartive pipeline I have found another solution:
stage('Deploy test')
{
steps
{
bat returnStatus: true, script: 'sc stop Tomcat9'
// The return value of the step will be the status code!
// evaluate return status yourself, or ignore it
}
}
The same works for the sh command to execute scripts on Unix platforms.
The example ignores the return status, because the tomcat might be already stopped, because of a previously failed pipeline run.
In the new pipeline, you can use try-catch to achieve this.
node{
try{
build job: 'system-check-flow'
}
catch (err){
echo "system-check-flow failed"
}
try{
build job: 'job2'
}
catch (err){
echo "job2 failed"
}
}
Here it will build the 'system-check-flow' job. If it fails it will catch the error, ignore, and then move on to build 'job2'
See this post for a full discussion.
pipeline {
agent any
stages {
stage('Stage') {
steps{
script{
jobresult = build(job: './failing-job',propagate:false).result
if(jobresult != 'SUCCESS'){
catchError(stageResult: jobresult, buildResult: 'UNSTABLE'){
error("Downstream job failing-job failed.")
}
}
}
}
}
}
}
For all those that are wondering about how to set the result of a downstream job to the stage/build) Not the most graceful solution, but it gets the job done. Funny thing is that if this stageResult variable was available as a global variable or as a variable outside the catchError block these kinds of solutions would not be needed. Sadly it isn't, and the only way to set the stage result in a pipeline that I thought of is this way. The error() block is needed, otherwise catchError will not set the stageResult/buildResult(the catchError block requires an error, ofcourse).
Complementing the existing working solutions that use catchError as a step or in script, you can also use catchError as a stage option.
This is useful if you have multiple sub stages that you want to catch errors for in the parent stage:
pipeline {
agent any
stages {
stage('Tests') {
options {
catchError(message: "Test failed", stageResult: 'UNSTABLE', buildResult: 'UNSTABLE')
}
stages {
stage('Test 1') {
echo 'test 1 succeeded'
}
stage('Test 2') {
error 'test 2 failed'
}
}
}
}
}
This isn't explicitly documented, but there is a hint that you may use steps as options (emphasis mine):
However, the stage-level options can only contain steps like retry,
timeout, or timestamps, or Declarative options that are relevant to a
stage, like skipDefaultCheckout.
It names a few steps as examples, but not as the only possible steps to be used as options. Also, if you enter an invalid option, Jenkins lists all available options in the error message, which includes catchError.
The cleanest and latest way would be:
stage('Integration Tests') {
steps {
script {
warnError(message: "${STAGE_NAME} stage was unstable.", catchInterruptions: false) {
// your scripts
}
}
}
}
Reference: https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#warnerror-catch-error-and-set-build-and-stage-result-to-unstable
you could put the step script inside "post" step, if if it's a teardown like step
code as below:
post {
always {
script {
try{
echo 'put your alway need run scripts here....if it's a teardown like step'
}catch (err) {
echo 'here failed'
}
script{
emailext (
xxxx
)
}
}