I'm using Cucumber reports plugin in my declarative pipeline like that:
cucumber '**/cucumber.json'
I'm able to check if some tests fail through link on the sidebar, but do I need to do something to mark the stage containing cucumber.json check as failed if some cucumber reports are failed? Because the problem is the build and stage are both green and successful despite there are some failed cucumber reports.
Jenkins version is 2.176.3
Cucumber reports version is 4.10.0
Cucumber command you are using just generates the report regardless the test result.
So yes, you have to make your pipeline fail somehow as the problem you are facing is that your test command is not returning making your pipeline fail.
The way to go is to make that the command that runs the tests returns non-zero exit code (exit 1) if something went wrong on your tests. That would make your pipeline stage to go red.
In case you run your tests using Maven this would be automatically managed on 'mvn test' (or whatever).
Otherwise, if you cannot do that, you will have to manage to make something like for example an sh script
that returns the exit code (0 pass / 1 fail) or a groovy function inside 'script' tag that sets the pipeline currentBuild.result value:
def checkTestResult() {
// Check some file to see if tests went fine or not
return 'SUCCESS' // or 'FAILURE'
}
...
stage {
script {
currentBuild.result = checkTestResult()
if (currentBuild.result == 'FAILURE') {
sh "exit 1" // Force pipeline exit with build result failed
}
}
}
...
I recommend you to use cucumber command on a 'always' post build action of your declarative pipeline
as it is a step that you will likely execute every time at the end of the pipeline either if it passes or fails. See the following example:
pipeline {
stages {
stage('Get code') {
// Whatever
}
stage('Run tests') {
steps {
sh "mvn test" // run_tests.sh or groovy code
}
}
}
post {
always {
cucumber '**/cucumber.json'
}
}
}
It is possible to set BuildStatus : 'FAILURE' to mark build as failed if a report marked as failed.
cucumber fileIncludePattern: '**/cucumber.json', buildStatus: 'FAILURE'
Related
I have a site in production. And I have a simple playwright test that browses to the site and does some basic checks to make sure that it's up.
I'd like to have this job running in Jenkins every 5 minutes, and if the tests fail I want to run a script that will restart the production server. If the tests pass, I don't want to do anything.
What's the easiest way of doing this?
I have the MultiJob plugin that I thought I could use, and have the restart triggered on the failed test step, but it doesn't seem to have the ability to trigger specifically on fail.
Something like the following will do the Job for you. I'm assuming you have a second Job that will take care of the restart.
pipeline {
agent any
triggers{
cron('*/5 * * * *')
}
stages {
stage("Run the Test") {
steps{
echo "Running the Test"
// I'm returning exit code 1 so jenkins will think this failed
sh '''
echo "RUN SOMETHING"
exit 1
'''
}
}
}
post {
success {
echo "Success: Do nothing"
}
failure {
echo 'I failed :(, Execute restart Job'
// Executing the restart Job.
build job: 'RestartJob'
}
}
}
I have a jenkins job.
It is pretty simple: pull from git, and run the build.
The build is just one step:
Execute window command batch
In my use case, I will need to run some python scripts.
Some will fail, some others will not.
python a.py
python b.py
What does determine the final status of the build?
It seems I can edit that by:
echo #STABLE > build.proprieties
but how are the STABLE/UNSTABLE status assigned if not specified by the user?
What happens if b.py raise an error and fails?
Jenkins interprets a pipeline as failed if a command returns an exit code unequal zero.
Internally the build status is set with currentBuild.currentResult which can have three values: SUCCESS, UNSTABLE, or FAILURE.
If you want to control the failure / success of your pipeline yourself you can catch exceptions / exit codes and manually set the value for currentBuild.currentResult. Plugins also use this attribute to change the result of the pipeline.
For example:
stage {
steps {
script {
try {
sh "exit 1" // will fail the pipeline
sh "exit 0" // would be marked as passed
currentBuild.currentResult = 'SUCCESS'
} catch (Exception e) {
currentBuild.currentResult = 'FAILURE'
// or currentBuild.currentResult = 'UNSTABLE'
}
}
}}
My jenkinsfile does not compile anymore when trying to add a POST action. This last one should be displayed to the jenkins console output at the end of build.
Part I is about my jenkinsfile code for which builds are done well.
Part II is the patch added to part I for which any builds fail.
I want to integrate part I and part II to get the expected output described hereafter but integration fails whatever how insertion is made.
I have tried a lot of thing and i'm stucked now, so any help will be appreciate.
// Part I : my base code
node {
def mvnHome
stage('Preparation') {
git 'https://github.com/jglick/simple-maven-project-with- tests.git'
// Get the Maven tool.
// ** NOTE: This 'M3' Maven tool must be configured
// ** in the global configuration.
mvnHome = tool 'M3'
}
stage('Build') {
// Run the maven build
if (isUnix()) {
sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
} else {
bat(/"${mvnHome}\bin\mvn" -Dmaven.test.failure.ignore clean package/)
}
}
stage('Results') {
junit '**/target/surefire-reports/TEST-*.xml'
archiveArtifacts 'target/*.jar'
}
}
// Part II : code to add to the previous code
post {
always {
echo 'I have finished and deleting workspace'
// deleteDir()
}
success {
echo 'Job succeeeded!
}
unstable {
echo 'I am unstable :/'
}
failure {
echo 'I failed :('
}
changed {
echo 'Things were different before...'
}
}
output expected in the console output : 'Job succeeeded! or I am unstable :/ or 'I failed :(' ... depending on the jenkins build status and always clean the workspace before each new build
Actual result is the error message from the console output :
java.lang.NoSuchMethodError: No such DSL method 'post' found among steps [archive, bat, build, catchError, checkout, deleteDir, dir ......
You are mixing up scripted and declarative pipeline syntax. post is part of declarative, but you use the scripted variant (no pipeline, but node steps).
You have to use try/catch.
See the documentation.
I am working on a project with a Jenkinsfile setup. This project runs on a number of integration tests, some of which are expected to fail. We are fixing the tests ( or the implementation ) one by one, but in the meantime the jobs are marked as failed.
The relevant state snippet is
stage ('Run ITs') {
steps {
sh 'SHOW_LOGS=0 ./compose/scripts/up-testing.sh'
sh 'sleep 60'
timeout (720) {
sh './testing/scripts/run-its.sh'
}
}
post {
always {
sh './testing/scripts/summarize-it-results.sh'
junit 'testing/failsafe-resports/*.xml'
sh './compose/scripts/killall.sh'
}
}
I'd like to set a threshold (T) on the number of failures + errors (F+E) and mark the build as unstable if we get F+E <= T and failed otherwise.
How can I do this with the Jenkins pipeline plugin?
I think it is currently not possible with JUnit plugin "out-of-the-box". Here is the corresponding issue in Jenkins issue tracker.
I'm trying to replicate the equivalent of a conditional stage in Jenkins pipeline using a try / catch around a preceding stage, which then sets a success variable, which is used to trigger the conditional stage.
It appears that a try catch block is the way to go, setting a success var to SUCCESS or FAILED, which is used as part of a when statement later (as part of the conditional stage).
The code I am using is as follows:
pipeline {
agent any
stages {
try{
stage("Run unit tests"){
steps{
sh '''
# Run unit tests without capturing stdout or logs, generates cobetura reports
cd ./python
nosetests3 --with-xcoverage --nocapture --with-xunit --nologcapture --cover-package=application
cd ..
'''
currentBuild.result = 'SUCCESS'
}
}
} catch(Exception e) {
// Do something with the exception
currentBuild.result = 'SUCCESS'
}
stage ('Speak') {
when {
expression { currentBuild.result == 'SUCCESS' }
}
steps{
echo "Hello, CONDITIONAL"
}
}
}
}
The latest syntax error I am receiving is as follows:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
failed:
WorkflowScript: 4: Expected a stage # line 4, column 9.
try{
I've also tried lots of variations.
Am I taking the wrong approach here? This seems like a fairly common requirement.
Thanks.
This might solve your problem depending on what you are going for. Stages are only run when the preceding stages succeed, so if you actually have two stages like in your example, and if you want the second to only run when the first succeeds, you want to ensure that the first stage fails appropriately when tests fail. Catching will prevent the (desirable) failure. Finally will preserve the failure, and can also still be used to grab your test results.
So here, the second stage will only run when the tests pass, and the test results will be recorded regardless:
pipeline {
agent any
stages {
stage("Run unit tests"){
steps {
script {
try {
sh '''
# Run unit tests without capturing stdout or logs, generates cobetura reports
cd ./python
nosetests3 --with-xcoverage --nocapture --with-xunit --nologcapture --cover-package=application
cd ..
'''
} finally {
junit 'nosetests.xml'
}
}
}
}
stage ('Speak') {
steps{
echo "Hello, CONDITIONAL"
}
}
}
}
Note that I'm actually using try in a declarative pipeline, but like StephenKing says, you can't just use try directly (you have to wrap arbitrary groovy code in the script step).