jenkins pipeline setting up a parallel build and stopping tasks - jenkins

Need help setting up a pipeline on jenkins.
It is necessary to run tests and collect logs in parallel, it worked out, but now there is another problem, the collection of logs is not completed. Maybe there is some method how to stop a task after another task is completed?
stage('Smoke Run') {
steps {
parallel(
first: {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh '$PYTHON -m pytest --testit android_tv/tests/smoke_run/ --clean-alluredir --alluredir=/Users/jenkins/allure-report/android-tv'
}
},
second: {
sh "$ADB logcat -c"
sh "$ADB logcat -> ~/jenkins/workspace/Android_TV_Smoke_Run/android_tv/tests/smoke_run/logs_tv/log.log"
}
)
}
}

found a solution
steps {
script {
stop = false
parallel(
first: {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh '''
set +e
$PYTHON -m pytest --testit android_tv/tests/smoke_run/ --clean-alluredir --alluredir=/Users/jenkins/allure-report/android-tv
set -e
'''.stripIndent()
stop = true
}
},
second: {
while (!stop){
sleep 10
}
sh '''pgrep adb logcat | xargs kill'''
sh '''echo "Finish writing logs"'''
}
)
}
}
}```

Related

Jenkins pipeline: Run all steps in stage, even if the first one fails

I have a series of steps in a stage that I want to run even if the first one fails. I want the stage result to fail and the build to get aborted, but only after all steps have run. For example,
pipeline {
agent any
stages {
stage('Run Test') {
steps {
sh "echo running unit-tests"
sh "echo running linting && false" // failure
sh "echo generating report" // This should still run (It currently doesn't)
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This should still run (It currently doesn't)
junit 'unit-test.xml' // This should still run (It currently doesn't)
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
The result should be a failed build where the "Run Test" stage failed and the "Deploy" stage did not run. Is this possible?
P.S.
I am NOT asking for the same behavior as in Continue Jenkins pipeline past failed stage. I want to run the steps following the failure, but not any of the stages afterwards. I tried to enclose each of the test steps with catchError (buildResult: 'FAILURE', stageResult: 'FAILURE'), but the "Deploy" stage still runs.
EDIT:
I cannot combine all the steps into one big sh step and capture its return code because some of the steps are not shell commands, but instead jenkins steps like junit and publishCoverage.
A script witha non-zero exit code will always cause a jenkins step to fail. You can use returnStatus as true so that jenkins does not fails the step.
Additionally considering your use case, you could use a post always execution, so that the steps are always carried out.
Please see below reference example:
stage('Run Test') {
steps {
def unit_test_result= sh returnStatus: true, script: 'echo "running unit-tests"'
def lint_result= sh returnStatus: true, script: 'echo "running linting"'
if (unit_test_result!=0 || lint_result!=0 ) {
// If the unit_test_result or lint_result status is not 0 then mark this stage as unstable to continue ahead
// and all later stages will be executed
unstable ('Testing failed')
// You can also mark as failed as below and it will not conintue other stages:
// error ('Testing failed')
}
}
post {
always {
// This block would always be executed inspite of failure
sh "echo generating report"
publishCoverage adapters: [coberturaAdapter("coverage.xml")]
junit 'unit-test.xml'
}
}
}
I found a slightly hacky way to get the behavior I want. The other answers didn't work for me, either because they need all the steps to be sh steps, or they don't stop the deploy stage from running. I used catchError to set the build and stage result. But to prevent the next stage from running, I needed to an explicit call to error if the stage failed.
pipeline {
agent any
stages {
stage('Run Test') {
steps {
script {
// catchError sets the stageResult to FAILED, but does not stop next stages from running
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running unit-tests"
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running linting && false" // failure
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo generating report" // This still runs
}
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This still runs
junit 'unit-test.xml' // This still runs
if (currentBuild.result == "FAILURE") { // This is needed to stop the next stage from running
error("Stage Failed")
}
}
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
Theoretically you should be able to use sh "<command>||true" It would ignore the error on command and continue. However, Jenkins will not fail as it would ignore the error.
If you don't want Jenkins to ignore the error and want it to stop at the end of the stage, you can do something like: sh "<command>||$error=true" then fail the build based on the $error variable. (sh "$error" might be enough but I am not sure, may require an if statement at the end.) It will be only set to true iff command fails.
Another option is to wrap your build steps in a try-catch block! if there's an exception, i.e. return code of build is not 0 you can catch it, mark the build as unstable and then the rest of the pipeline continues on.
here's an example `
pipeline {
agent {
node {
label 'linux'
}
}
options {
timestamps()
disableConcurrentBuilds()
buildDiscarder(logRotator(numToKeepStr: '3'))
}
tools {
maven 'Maven 3.6.3'
jdk 'jdk11'
}
stages {
stage('CleanWS') {
steps {
cleanWs()
}
}
stage('Build') {
steps {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml clean install -DskipTests -Pregression-test -Dmaven.javadoc.skip=true"
}
}
}
stage('Test') {
steps {
script {
try {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -B verify -Dmaven.source.skip=true -Dmaven.javadoc.skip=true"
}
} catch (exc) {
currentBuild.result = 'UNSTABLE'
}
}
}
post {
always {
script {
junit "**/surefire-reports/*.xml"
}
}
}
}
stage('Sonar Analyse') {
steps {
script {
withMaven(options: [artifactsPublisher(disabled: true)]) {
withSonarQubeEnv("SonarQube") {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn sonar:sonar"
}
}
}
}
}
stage('Deploy to Nexus') {
steps {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml -B clean deploy -DdeployAtEnd=true -DskipTests"
}
}
}
post {
failure {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
unstable {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Build Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
}
}`

Running multiple steps in sequenze in one parallel block In Jenkinsfile

I'm trying to optimize my pipeline. I'm using the pipeline to generate and deploy some docs. At the end I clear my document root and write the newly generated docs into the document root. I'm doing this for several stages in parallel.
o-----o-----o--+--o--+---+--o--+-----o
| | | |
+--o--+ +--o--+
| | | |
+--o--+ +--o--+
this is the pipeline exerpt for the parallel stages
stage("clear nfs directory") {
steps {
parallel(
test: {
sh "rm -rf /mnt/nfs/test/docs/$pipelineParams.groupname"
sh "mkdir /mnt/nfs/test/docs/$pipelineParams.groupname"
},
rele: {
sh "rm -rf /mnt/nfs/rele/docs/$pipelineParams.groupname"
sh "mkdir /mnt/nfs/rele/docs/$pipelineParams.groupname"
},
prod: {
sh "rm -rf /mnt/nfs/prod/docs/$pipelineParams.groupname"
sh "mkdir /mnt/nfs/prod/docs/$pipelineParams.groupname"
}
)
}
}
stage("copy generated docs to nfs directory") {
steps {
parallel(
test: {
dir("target/public") {
sh "cp -r * /mnt/nfs/test/docs/$pipelineParams.groupname"
}
},
rele: {
dir("target/public") {
sh "cp -r * /mnt/nfs/rele/docs/$pipelineParams.groupname"
}
},
prod: {
dir("target/public") {
sh "cp -r * /mnt/nfs/prod/docs/$pipelineParams.groupname"
}
}
)
}
}
Since clear and write should depend on each other I would like to refactor the pipeline into a more sequential design (running multiple steps in sequence in less parallel steps)
o-----o-----o--+--o---o--+-----o
| |
+--o---o--+
| |
+--o---o--+
I'm not sure how to run multiple steps in the same parallel block ... can anyone give me a hint? Thanks guys
Please see below reference which will allow you to run multiple steps in same parallel block.
You would need to use sequential stages which will give below output :
o-----o-----o--+--o---o--+-----o
| |
+--o---o--+
| |
+--o---o--+
pipeline {
agent { label 'master' }
stages {
stage('Build and Test') {
parallel {
stage("Build and Test Linux") {
stages {
stage("Build (Linux)") {
agent any
steps {
echo "Inside for loop 1"
}
}
stage("Test (Linux)") {
agent any
steps {
echo "Inside for loop 2"
}
}
}
}
stage("Build and Test Windows") {
stages {
stage("Build (Windows)") {
agent any
steps {
echo "Inside for loop 3"
}
}
stage("Test (Windows)") {
agent any
steps {
echo "Inside for loop 4"
}
}
}
}
}
}
}
}
For more info see:-
https://www.jenkins.io/blog/2018/07/02/whats-new-declarative-piepline-13x-sequential-stages/
Below link gives reference example:
https://issues.jenkins.io/browse/JENKINS-55438

Jenkins piepleine with dir correct syntax

Hi guys I'm trying to write a jenkins pipeline with change dir, I can't get the syntax right, can anyone help me fix this?
the documentation is kinda unclear on where should I put this
pipeline {
agent {
docker {
image 'node'
args '-p 5000:5000'
}
}
stages {
dir("Backend") {
stage('build') {
steps {
sh 'pwd'
sh 'echo start build'
sh 'ls'
sh 'npm install'
}
}
}
dir("Backend") {
stage('Test') {
steps {
sh 'cd Backend'
sh 'echo start test'
sh 'ls'
sh 'npm run test'
}
}
}
}
}
dir is a step, therefore it must be contained within steps block:
stage('build') {
steps {
dir("Backend") {
sh 'pwd'
sh 'echo start build'
sh 'ls'
sh 'npm install'
}
}
}

JenkinsFile for Sonar Quality Gates doesn't work

I have read the documentation of Jenkins and Sonar to write and configure my pipeline and everything works except the Sonar Quality Gate which remains for the response from the Sonar without success. This is my code :
pipeline {
agent master
tools {
jdk 'JAVA'
maven 'mvn_3_5_2'
}
stages {
stage('test Java & Maven installation') {
steps {
sh 'java -version'
sh 'which java'
sh 'mvn -version'
sh 'which mvn'
}
}
stage('Clean stage') {
steps {
sh 'mvn -f project/pom.xml clean'
}
}
stage("build & SonarQube analysis") {
node {
withSonarQubeEnv('Sonar') {
sh 'mvn -f project/pom.xml org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar'
}
}
}
stage("Quality Gate"){
timeout(time: 2, unit: 'MINUTES') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
stage('Test stage') {
steps {
sh 'mvn -f project/pom.xml test'
}
}
stage('Build stage') {
steps {
sh 'mvn -f project/pom.xml install'
}
}
stage('Deploy stage') {
steps {
echo 'Deployment...'
// sh 'mvn -f project/pom.xml appengine:deploy -Dapp.deploy.project=acn-shelf-check -Dapp.deploy.version=v1'
}
}
}
}
In the stage "Quality Gate" I never received the result. I have configured the webhook on the sonar with correct URL of Jenkins.
Do I miss any step?

How to test sh script return status with jenkins declarative pipeline new syntax

Using new jenkins declarative pipeline syntax, I'd like to test the return status of a sh script execution. Is it possible without using script step?
Script pipeline (working) :
...
stage ('Check url') {
node {
timeout(15) {
waitUntil {
sleep 20
def r = sh script: "wget -q ${CHECK_URL} -O /dev/null", returnStatus: true
return (r == 0);
}
}
}
}
Declarative pipeline (try) :
...
stage('Check url'){
steps {
timeout(15) {
waitUntil {
sleep 20
sh script: "wget -q ${CHECK_URL} -O /dev/null", returnStatus: true == 0
}
}
}
}
log : java.lang.ClassCastException: body return value null is not boolean
Since it's not possible without script block, we get something like :
...
stage('Check url'){
steps {
script {
timeout(15) {
waitUntil {
sleep 20
def r = sh script: "wget -q ${CHECK_URL} -O /dev/null", returnStatus: true
return r == 0
}
}
}
}
}

Resources