I am currently starting to convert our builds into a Jenkins build pipeline. At a certain point it is necessary for us to wait for the startup of a web application within a docker container.
My idea was to use something like this:
timeout(120) {
waitUntil {
sh 'wget -q http://server:8080/app/welcome.jsf -O /dev/null'
}
}
Unfortunately this makes the pipeline build fail:
ERROR: script returned exit code 4
Is there any simple way to make this work?
Edit:
I managed to make it work using the following code, but the stage is still marked as failed (although the build continues and is marked green in the end).
timeout(120) {
waitUntil {
try {
sh 'wget -q http://server:8080/app/welcome.jsf -O /dev/null'
return true
} catch (exception) {
return false
}
}
}
They just released a new version of the Pipeline Nodes and Processes Plugin which adds support for returning the exit status.
This seems to do the job now:
timeout(5) {
waitUntil {
script {
def r = sh script: 'wget -q http://remoterhoste/welcome.jsf -O /dev/null', returnStdout: true
return (r == 0);
}
}
}
You can use wget options to achieve that:
waitUntil {
sh 'wget --retry-connrefused --tries=120 --waitretry=1 -q http://server:8080/app/welcome.jsf -O /dev/null'
}
120 tries, plus wait 1s between retries, retry even in case of connection refused, this might be slightly more seconds. So to make sure it is only 120s, then you can use timeout from shell:
waitUntil {
sh 'timeout 120 wget --retry-connrefused --tries=120 --waitretry=1 -q http://server:8080/app/welcome.jsf -O /dev/null'
}
If you don't have wget on the jenkins node (e.g. the default docker image) you can also install and use the HTTPRequest Plugin like this.
timeout(5) {
waitUntil {
script {
try {
def response = httpRequest 'http://server:8080/app/welcome.jsf'
return (response.status == 200)
}
catch (exception) {
return false
}
}
}
}
Related
I am working with Jenkins pipelines and I have this code:
stages {
stage('Stage1') {
options {
timeout(time: 1, unit: "MINUTES")
}
steps {
script {
sh'''
#!/bin/bash
set -eux pipefail
ssh user#server.com "
ssh -p 50 user#localhost'\
docker run --rm --name name\
-e user=...\
-e passwd=...\
-v /location:/location2\
-w location2\
server2.com:6000/my-x-y:1.1\
python script.py\
'\
"
'''
}
}
}
}
When the connection inside the script is not being made the job will timeout but it will still go on and will be still marked as succeeded.
I get this message:
17:10:53 Cancelling nested steps due to timeout
17:10:53 Sending interrupt signal to process
After that the jobs moves to the next stage and the status is success.
So even though I am getting timeout the job is being marked as success.
I'd like to send notifications when this stage is not properly executed (I already have a notification.sh script for it).
Anyway I can get this job to be aborted when it gets the timeout?
Or any other way to go around this in order to warn users that this stage was not properly executed?
Try something like below.
try {
timeout (time: 10, unit: 'SECONDS') {
sh'''
#!/bin/bash
set -eux pipefail
ssh user#server.com "
ssh -p 50 user#localhost'\
docker run --rm --name name\
-e user=...\
-e passwd=...\
-v /location:/location2\
-w location2\
server2.com:6000/my-x-y:1.1\
python script.py\
'\
"
'''
}
}
catch (error) {
echo "Error: $error"
def cause = error.getCauses()[0].getClass().toString()
if(cause.contains("ExceededTimeout")) { // If you want handle timeout as a special case
echo "This was a Timeout"
// Do whatever you want
}
}
Full Sample Pipeline
pipeline {
agent any
stages {
stage('TimerTest') {
steps {
script {
try {
timeout (time: 10, unit: 'SECONDS') {
echo "In timer"
sleep 15
}
}
catch (error) {
echo "XXXX: $error"
def cause = error.getCauses()[0].getClass().toString()
println "$cause"
if(cause.contains("ExceededTimeout")) {
echo "This was a Timeout"
// Do what ever you want
}
}
}
}
}
}
}
I am trying to populate a global variable with the exit code status and use that in the catch block to avoid certain exceptions. Not able to store the exit code value to the variable in groovy docker file. Should I take a different approach here? Please suggest.
Scenario: I'm looking to make sure if docker command fails, it should not fail the build, but if the python command inside the sh""" inside docker.image.inside fails as shown below, only then it should fail the build. That's why I started looking for exit code of python script to just fail the build when python cmd is failed and not otherwise.
def scriptStatus = 0
stages {
stage('Call Update) {
steps {
script {
try {
def image = docker.image("${repo}:${tag}")
image.pull()
image.inside("--env-file ${PWD}/cc.env -v /work/user.ssh") {
scriptStatus = sh(script: """
python -u /config/env-python/abc.py
""", returnStdout: true)
}
} catch(Exception e) {
echo "scriptStatus = ${scriptStatus}" --> not showing any result
if (scriptStatus == 0){
currentBuild.result = 'SUCCESS'
} else {
currentBuild.result = 'FAILURE'
}
}
}
}
}
}
I have tried couple options here:
returnStatus: true -> but it doesn't export any result outside the try block, hence I can't check whether the returned value is 0 or non-zero.
then I also tried exitCode=$? -> this also doesn't get stored in the global variable which can be further used in the catch block for if/else condition.
Use returnStatus: true instead returnStdout: true
scriptStatus = sh(script: """
python -u /config/env-python/abc.py
""", returnStatus: true)
returnStatus : returns the status code either 0 or 1.
returnStdout: returns the output.
I have a series of steps in a stage that I want to run even if the first one fails. I want the stage result to fail and the build to get aborted, but only after all steps have run. For example,
pipeline {
agent any
stages {
stage('Run Test') {
steps {
sh "echo running unit-tests"
sh "echo running linting && false" // failure
sh "echo generating report" // This should still run (It currently doesn't)
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This should still run (It currently doesn't)
junit 'unit-test.xml' // This should still run (It currently doesn't)
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
The result should be a failed build where the "Run Test" stage failed and the "Deploy" stage did not run. Is this possible?
P.S.
I am NOT asking for the same behavior as in Continue Jenkins pipeline past failed stage. I want to run the steps following the failure, but not any of the stages afterwards. I tried to enclose each of the test steps with catchError (buildResult: 'FAILURE', stageResult: 'FAILURE'), but the "Deploy" stage still runs.
EDIT:
I cannot combine all the steps into one big sh step and capture its return code because some of the steps are not shell commands, but instead jenkins steps like junit and publishCoverage.
A script witha non-zero exit code will always cause a jenkins step to fail. You can use returnStatus as true so that jenkins does not fails the step.
Additionally considering your use case, you could use a post always execution, so that the steps are always carried out.
Please see below reference example:
stage('Run Test') {
steps {
def unit_test_result= sh returnStatus: true, script: 'echo "running unit-tests"'
def lint_result= sh returnStatus: true, script: 'echo "running linting"'
if (unit_test_result!=0 || lint_result!=0 ) {
// If the unit_test_result or lint_result status is not 0 then mark this stage as unstable to continue ahead
// and all later stages will be executed
unstable ('Testing failed')
// You can also mark as failed as below and it will not conintue other stages:
// error ('Testing failed')
}
}
post {
always {
// This block would always be executed inspite of failure
sh "echo generating report"
publishCoverage adapters: [coberturaAdapter("coverage.xml")]
junit 'unit-test.xml'
}
}
}
I found a slightly hacky way to get the behavior I want. The other answers didn't work for me, either because they need all the steps to be sh steps, or they don't stop the deploy stage from running. I used catchError to set the build and stage result. But to prevent the next stage from running, I needed to an explicit call to error if the stage failed.
pipeline {
agent any
stages {
stage('Run Test') {
steps {
script {
// catchError sets the stageResult to FAILED, but does not stop next stages from running
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running unit-tests"
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo running linting && false" // failure
}
catchError (buildResult: 'FAILURE', stageResult: 'FAILURE') {
sh "echo generating report" // This still runs
}
publishCoverage adapters: [coberturaAdapter("coverage.xml")] // This still runs
junit 'unit-test.xml' // This still runs
if (currentBuild.result == "FAILURE") { // This is needed to stop the next stage from running
error("Stage Failed")
}
}
}
}
stage('Deploy') {
steps {
echo "deploying" // This should NOT run
}
}
}
}
Theoretically you should be able to use sh "<command>||true" It would ignore the error on command and continue. However, Jenkins will not fail as it would ignore the error.
If you don't want Jenkins to ignore the error and want it to stop at the end of the stage, you can do something like: sh "<command>||$error=true" then fail the build based on the $error variable. (sh "$error" might be enough but I am not sure, may require an if statement at the end.) It will be only set to true iff command fails.
Another option is to wrap your build steps in a try-catch block! if there's an exception, i.e. return code of build is not 0 you can catch it, mark the build as unstable and then the rest of the pipeline continues on.
here's an example `
pipeline {
agent {
node {
label 'linux'
}
}
options {
timestamps()
disableConcurrentBuilds()
buildDiscarder(logRotator(numToKeepStr: '3'))
}
tools {
maven 'Maven 3.6.3'
jdk 'jdk11'
}
stages {
stage('CleanWS') {
steps {
cleanWs()
}
}
stage('Build') {
steps {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml clean install -DskipTests -Pregression-test -Dmaven.javadoc.skip=true"
}
}
}
stage('Test') {
steps {
script {
try {
withMaven(options: [artifactsPublisher(disabled: true)]) {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -B verify -Dmaven.source.skip=true -Dmaven.javadoc.skip=true"
}
} catch (exc) {
currentBuild.result = 'UNSTABLE'
}
}
}
post {
always {
script {
junit "**/surefire-reports/*.xml"
}
}
}
}
stage('Sonar Analyse') {
steps {
script {
withMaven(options: [artifactsPublisher(disabled: true)]) {
withSonarQubeEnv("SonarQube") {
sh "export MAVEN_OPTS=\"-Xmx2048m\" && export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn sonar:sonar"
}
}
}
}
}
stage('Deploy to Nexus') {
steps {
sh "export NLS_LANG=GERMAN_GERMANY.WE8ISO8859P1 && mvn -f pom.xml -B clean deploy -DdeployAtEnd=true -DskipTests"
}
}
}
post {
failure {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
unstable {
script {
emailext(
body: "Please go to ${env.BUILD_URL}/console for more details.",
to: emailextrecipients([developers(), requestor()]),
subject: "Nightly-Build-Pipeline Build Status is ${currentBuild.result}. ${env.BUILD_URL}"
)
}
}
}
}`
I have a spring boot application on centos server and use a shell file to restart it.
jenkins version: docker run -dp 8080:8080 --name jenkins jenkinsci/blueocean
start-service.sh
#!/bin/bash
sudo systemctl restart sb
In my Jenkinsfile i upload jar file to server and execute the start-service.sh, but jenkins seem dosen't know my java application restart success or fail.
Jenkinsfile
pipeline {
agent none
stages {
stage('Build') {
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
}
steps {
sh 'mvn -B -DskipTests clean package'
sh 'mvn help:evaluate -Dexpression=project.name | grep "^[^\\[]" > project-name'
sh 'mvn help:evaluate -Dexpression=project.version | grep "^[^\\[]" > project-ver'
}
}
stage('Deploy') {
agent any
environment {
HOST = "${HEHU_HOST}"
USER = "yunwei"
DIR = "/www/java/sb-demo"
VERSION_FILE = "${DIR}/version"
CMD_SERVICE = "${DIR}/start-service.sh"
}
steps {
sshagent (credentials: ['hehu']) {
sh '''
name=$(cat project-name)
ver=$(cat project-ver)
jarFile=${name}-${ver}.jar
scp target/${jarFile} ${USER}#${HOST}:${DIR}/${jarFile}
scp project-ver ${USER}#${HOST}:${VERSION_FILE}
ssh -o StrictHostKeyChecking=no -l ${USER} ${HOST} -a ${CMD_SERVICE}
'''
}
}
}
}
}
I deliberately let Java application go wrong, and systemctl restart is fail but jenkins stage is success.
#SpringBootApplication
#RestController
public class Application {
public static void main(String[] args) {
throw new RuntimeException("Test error");
// SpringApplication.run(Application.class, args);
}
#GetMapping("/test")
String test() {
return "furukawa nagisa\n";
}
}
Try Daniel Taub solution get syntax error.
pipeline {
agent none
stages {
stage('Hello') {
agent any
steps {
sshagent (credentials: ['hehu']) {
SH_SUCCESS = sh(
script: '''
ssh -o StrictHostKeyChecking=no -l yunwei ${HEHU_HOST} -a /www/java/sb-demo/start-service.sh
''',
returnStatus: true
) == 0
echo '${SH_SUCCESS}'
}
}
}
}
}
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 9: Expected a step # line 9, column 21.
SH_SUCCESS = sh(
You can get the sh returned exit status code and fail the build manually if its error status.
For checking your exit code, Jenkins support it that way
returnStatus (optional)
Normally, a script which exits with a nonzero status code will cause the step to fail with an exception. If this option is checked, the return value of the step will instead be the status code. You may then compare it to zero, for example.
script {
SH_SUCCESS = sh (
script: "your command",
returnStatus: true
) == 0
}
To manually fail the build you have couple of options:
error('Fail my build!')
Or alternatively
currentBuild.result = 'FAILURE'
return
I've a Jenkinsfile, which has a two different stages: Pre-Build and Build. The Pre-Build is executing pylint and uses the warnings-ng-plugin to report that back to Jenkins.
Something like that:
stages {
stage('Pre-build') {
steps {
script {
sh """#!/usr/bin/env bash
pip install .
pylint --exit-zero --output-format=parseable --reports=n myProject > reports/pylint.log
"""
}
}
post {
always {
recordIssues(
enabledForFailure: true,
tool: pyLint(pattern: '**/pylint.log'),
unstableTotalAll: 20,
failedTotalAll: 30,
)
}
failure {
cleanWs()
}
}
}
stage('Build') {
steps {
script {
sh """#!/usr/bin/env bash
set -e
echo 'I AM STAGE TWO AND I SHOULD NOT BE EXECUTED'
"""
}
}
post {
always {
cleanWs()
}
}
}
}
I'm running into a couple of issues here. Currently I'm setting pylint to --exit-zero, as I want the warnings-ng plugin decide if it is good to go or not, based on the report.
Currently this is set to fail at a total of 30 issues. Now, myProject has 45 issues and I want to prevent that the next stage, Build is entered. But currently I can't seem to be able to prevent this behaviour, as it always continuous to the Build stage.
The build is flagged as failure, because of the results determined within recordIssues, but it doesn't abort the job.
I've found a ticket on https://issues.jenkins-ci.org (Ticket), but I can't seem to make sense out of all of this.
I've found a solution to your problem and I think that this is a bug in the pipeline workflow. The warnings-ng correctly sets build status to failed, but next stages are started despite the status in ${currentBuild.currentResult} variable.
You can use when { expression { return currentBuild.currentResult == "SUCCESS" } } to skip later stages, or throwing an error. But I think this should be default behaviour. Your file then should like:
stages {
stage('Pre-build') {
steps {
script {
sh """#!/usr/bin/env bash
pip install .
pylint --exit-zero --output-format=parseable --reports=n myProject > reports/pylint.log
"""
}
}
post {
always {
recordIssues(
enabledForFailure: true,
tool: pyLint(pattern: '**/pylint.log'),
unstableTotalAll: 20,
failedTotalAll: 30,
)
}
}
}
stage('Build') {
when { expression { return currentBuild.currentResult == "SUCCESS" } }
steps {
script {
echo "currentResult: ${currentBuild.currentResult}"
sh """#!/usr/bin/env bash
set -e
echo 'I AM STAGE TWO AND I SHOULD NOT BE EXECUTED'
"""
}
}
}
post {
always {
cleanWs()
}
}
}
I've created an issue in their Jira.
My environment:
Jenkins ver.: 2.222.1
warnings-ng ver.: 8.1
worfklow-api ver.: 2.40
You have used post 2 times which is wrong implementation as post is designed to get executed only once after all stages are done. It should be written after all the stages just before end of pipeline.
To stop or skip the execution of 2nd Build stage, you can create global varaible at the top, capture the output of pylint in that and use if or when condition at start of stage. Something similar to --
pipeline {
def result
stages {
stage('Pre-build') {
steps {
script {
sh """#!/usr/bin/env bash
pip install .
pylint --exit-zero --output-format=parseable --reports=n myProject > reports/pylint.log
"""
}
}
}
}
stage('Pylint result') { // Not sure how recordIssue works. This just an example.
result = recordIssues(
enabledForFailure: true,
tool: pyLint(pattern: '**/pylint.log'),
unstableTotalAll: 20,
failedTotalAll: 30,
)
}
stage('Build') {
if ( result == "pass") {
steps {
script {
sh """#!/usr/bin/env bash
set -e
echo 'I AM STAGE TWO AND I SHOULD NOT BE EXECUTED'
"""
}
}
}
}
}
post { // this should be used after stages
always {
cleanWs()
}
failure {
cleanWs()
}
}
Also, stages are designed in such a way that if they fail, next stage will not be executed so it's a good idea to have the pylint to be executed inside a stage instead of post condition.
Note: The code above is just an example. Please modify it according to your need.
One option that you may consider is to fail the build explicitly with the following code:
post {
always {
recordIssues(
enabledForFailure: true,
tool: pyLint(pattern: '**/pylint.log'),
unstableTotalAll: 20,
failedTotalAll: 30
)
script {
if (currentBuild.currentResult == 'FAILURE') {
error('Ensure that the build fails if the quality gates fail')
}
}
}
}
Here, after you record the issues, you also check if the value of currentBuild.currentResult is FAILURE and in that case you explicitly call the error() function which fails the build correctly.