I'm using Cucumber reports plugin in my declarative pipeline like that:
cucumber '**/cucumber.json'
I'm able to check if some tests fail through link on the sidebar, but do I need to do something to mark the stage containing cucumber.json check as failed if some cucumber reports are failed? Because the problem is the build and stage are both green and successful despite there are some failed cucumber reports.
Jenkins version is 2.176.3
Cucumber reports version is 4.10.0
Cucumber command you are using just generates the report regardless the test result.
So yes, you have to make your pipeline fail somehow as the problem you are facing is that your test command is not returning making your pipeline fail.
The way to go is to make that the command that runs the tests returns non-zero exit code (exit 1) if something went wrong on your tests. That would make your pipeline stage to go red.
In case you run your tests using Maven this would be automatically managed on 'mvn test' (or whatever).
Otherwise, if you cannot do that, you will have to manage to make something like for example an sh script
that returns the exit code (0 pass / 1 fail) or a groovy function inside 'script' tag that sets the pipeline currentBuild.result value:
def checkTestResult() {
// Check some file to see if tests went fine or not
return 'SUCCESS' // or 'FAILURE'
}
...
stage {
script {
currentBuild.result = checkTestResult()
if (currentBuild.result == 'FAILURE') {
sh "exit 1" // Force pipeline exit with build result failed
}
}
}
...
I recommend you to use cucumber command on a 'always' post build action of your declarative pipeline
as it is a step that you will likely execute every time at the end of the pipeline either if it passes or fails. See the following example:
pipeline {
stages {
stage('Get code') {
// Whatever
}
stage('Run tests') {
steps {
sh "mvn test" // run_tests.sh or groovy code
}
}
}
post {
always {
cucumber '**/cucumber.json'
}
}
}
It is possible to set BuildStatus : 'FAILURE' to mark build as failed if a report marked as failed.
cucumber fileIncludePattern: '**/cucumber.json', buildStatus: 'FAILURE'
In Jenkins Pipeline, how can I copy the artifacts from a previous build to the current build?
I want to do this even if the previous build failed.
Stuart Rowe also recommended to me on the Pipeline Authoring Sig Gitter channel that I look at the Copy Artifact Plugin, but also gave me some sample Jenkins Pipeline syntax to use.
Based on the advice that he gave, I came up with this fuller Pipeline example
which copies the artifacts from the previous build into the current build,
whether the previous build succeeded or failed.
pipeline {
agent any;
stages {
stage("Zeroth stage") {
steps {
script {
if (currentBuild.previousBuild) {
try {
copyArtifacts(projectName: currentBuild.projectName,
selector: specific("${currentBuild.previousBuild.number}"))
def previousFile = readFile(file: "usefulfile.txt")
echo("The current build is ${currentBuild.number}")
echo("The previous build artifact was: ${previousFile}")
} catch(err) {
// ignore error
}
}
}
}
}
stage("First stage") {
steps {
echo("Hello")
writeFile(file: "usefulfile.txt", text: "This file ${env.BUILD_NUMBER} is useful, need to archive it.")
archiveArtifacts(artifacts: 'usefulfile.txt')
}
}
stage("Error") {
steps {
error("Failed")
}
}
}
}
Suppose you want a single file to from previous build, you can even use curl to place file in workspace before mvn invocation.
stage('Copy csv') {
steps {
sh "mkdir -p ${env.WORKSPACE}/dump"
sh "curl http://<jenkins-url>:<port>/job/<job-folder>/job/<job-name>/job/<release>/lastSuccessfulBuild/artifact/dump/sample.csv/*view*/ -o ${env.WORKSPACE}/dump/sample.csv"
}
}
Thanks,
Ashish
You Can Use Copy Artifact Plugin
For configuration visit https://wiki.jenkins.io/display/JENKINS/Copy+Artifact+Plugin
How to define post build actions for Jenkins multi pipeline project?
There is a separate option available when you have a simple project but not for multipipeline.
To add post build steps to a Multibranch Pipeline, you need to code these steps into the finally block, an example is below:
node {
try {
stage("Checkout") {
// checkout scm
}
stage("Build & test") {
// build & Unit test
}
} catch (e) {
// fail the build if an exception is thrown
currentBuild.result = "FAILED"
throw e
} finally {
// Post build steps here
/* Success or failure, always run post build steps */
// send email
// publish test results etc etc
}
}
For most of the post-build steps you would want there are online examples of them on how to write in pipeline format. If you have any specific one please list it here
When you write a pipeline, you describe the whole flow yourself, which gives you great flexibility to do whatever you want, including running post-build steps.
You can see an example of using post-build steps in a pipeline I wrote:
https://github.com/geek-kb/Android_Pipeline/blob/master/Jenkinsfile
Example from that code:
run_in_stage('Post steps', {
sh """
# Add libCore.so files to symbols.zip
find ${cwd}/Product-CoreSDK/obj/local -name libCore.so | zip -r ${cwd}/Product/build/outputs/symbols.zip -#
# Remove unaligned apk's
rm -f ${cwd}/Product/build/outputs/apk/*-unaligned.apk
"""
})
The following SonarQube (6.3) analysis stage in a declarative pipeline in Jenkins 2.50 is failing with this error in the console log: http://pastebin.com/t2ja23vC. More specifically:
SonarQube installation defined in this job (SonarGate) does not match any configured installation. Number of installations that can be configured: 1.
Update: after changing "SonarQube" to "SonarGate" in the Jenkins settings (under SonarQube servers, so it'll match the Jenkinsfile), I get a different error: http://pastebin.com/HZZ6fY6V
java.lang.IllegalStateException: Unable to get SonarQube task id and/or server name. Please use the 'withSonarQubeEnv' wrapper to run your analysis.
The stage is a modification of the example from the SonarQube docs: https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Jenkins#AnalyzingwithSonarQubeScannerforJenkins-AnalyzinginaJenkinspipeline
stage ("SonarQube analysis") {
steps {
script {
STAGE_NAME = "SonarQube analysis"
if (BRANCH_NAME == "develop") {
echo "In 'develop' branch, don't analyze."
}
else { // this is a PR build, run sonar analysis
withSonarQubeEnv("SonarGate") {
sh "../../../sonar-scanner-2.9.0.670/bin/sonar-scanner"
}
}
}
}
}
stage ("SonarQube Gatekeeper") {
steps {
script {
STAGE_NAME = "SonarQube Gatekeeper"
if (BRANCH_NAME == "develop") {
echo "In 'develop' branch, skip."
}
else { // this is a PR build, fail on threshold spill
def qualitygate = waitForQualityGate()
if (qualitygate.status != "OK") {
error "Pipeline aborted due to quality gate coverage failure: ${qualitygate.status}"
}
}
}
}
}
I also created a webhook, sonarqube-webhook, with the URL http://****/sonarqube-webhook/. Should it be like that, or http://****/sonarqube/sonarqube-webhook? To access the server dashboard I use http://****/sonarqube.
In SonarQube's Quality Gates section I created a new quality gate:
I am not sure if the setting in SonarGate is correct. I do use jenkins-mocha to generate an lcov.info file that is used in Sonar to generate the coverage data.
Perhaps the quality gate setting is the wrong setting to do? The end result is to fail the job in Jenkins if coverage % is not met.
Finally, I am not sure if the following configurations in the Jenkins system configuration are at all required:
And
(It's 9000 not 900... cut text in the screen shot)
The SonarQube Jenkins plugin scans the build output for two specific lines, which it uses to get the SonarQube report task properties and project URL. If your invocation of sonar-scanner does not output these lines, the waitForQualityGate() call won't have the task ID to look them up. So you will have to figure out the correct settings to make it more verbose.
See the extractSonarProjectURLFromLogs and extractReportTask methods in the SonarUtils class of the plugin to understand how they work:
ANALYSIS SUCCESSFUL, you can browse <project URL> is used to add a link to the badge (in the build history)
Working dir: <dir with report-task.txt> is used to pass the task ID to the waitForQualityGate step
This was discovered to be a bug in the SonarQube scanner for Jenkins, when using a Jenkins slave for jobs (if the job is run on the master, it'd work). You can read more here: https://jira.sonarsource.com/browse/SONARJNKNS-282
I have tested this using a test build of v2.61 of the scanner plug-in and found it working.
The solution is to upgrade to v2.61 when released.
This stage will then work:
stage ("SonarQube analysis") {
steps {
withSonarQubeEnv('SonarQube') {
sh "../../../sonar-scanner-2.9.0.670/bin/sonar-scanner"
}
def qualitygate = waitForQualityGate()
if (qualitygate.status != "OK") {
error "Pipeline aborted due to quality gate coverage failure: ${qualitygate.status}"
}
}
}
If you're running SonarCube in a docker container check that the memory isn't exhausted. We were maxing out. Which seemed to be the issue.
We are running Jenkins 2.x and love the new Pipeline plugin. However, with so many branches in a repository, disk space fills up quickly.
Is there any plugin that's compatible with Pipeline that I can wipe out the workspace on a successful build?
Like #gotgenes pointed out with Jenkins Version. 2.74, the below works, not sure since when, maybe if some one can edit and add the version above
cleanWs()
With, Jenkins Version 2.16 and the Workspace Cleanup Plugin, that I have, I use
step([$class: 'WsCleanup'])
to delete the workspace.
You can view it by going to
JENKINS_URL/job/<any Pipeline project>/pipeline-syntax
Then selecting "step: General Build Step" from Sample step and then selecting "Delete workspace when build is done" from Build step
The mentioned solutions deleteDir() and cleanWs() (if using the workspace cleanup plugin) both work, but the recommendation to use it in an extra build step is usually not the desired solution. If the build fails and the pipeline is aborted, this cleanup-stage is never reached and therefore the workspace is not cleaned on failed builds.
=> In most cases you should probably put it in a post-built-step condition like always:
pipeline {
agent any
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
post {
always {
cleanWs()
}
}
}
You can use deleteDir() as the last step of the pipeline Jenkinsfile (assuming you didn't change the working directory).
In fact the deleteDir function recursively deletes the current directory and its contents. Symbolic links and junctions will not be followed but will be removed.
To delete a specific directory of a workspace wrap the deleteDir step in a dir step.
dir('directoryToDelete') {
deleteDir()
}
Using the following pipeline script:
pipeline {
agent { label "master" }
options { skipDefaultCheckout() }
stages {
stage('CleanWorkspace') {
steps {
cleanWs()
}
}
}
}
Follow these steps:
Navigate to the latest build of the pipeline job you would like to clean the workspace of.
Click the Replay link in the LHS menu.
Paste the above script in the text box and click Run
I used deleteDir() as follows:
post {
always {
deleteDir() /* clean up our workspace */
}
}
However, I then had to also run a Success or Failure AFTER always but you cannot order the post conditions.
The current order is always, changed, aborted, failure, success and then unstable.
However, there is a very useful post condition, cleanup which always runs last, see https://jenkins.io/doc/book/pipeline/syntax/
So in the end my post was as follows :
post {
always {
}
success{
}
failure {
}
cleanup{
deleteDir()
}
}
Hopefully this may be helpful for some corner cases
If you have used custom workspace in Jenkins then deleteDir() will not delete #tmp folder.
So to delete #tmp along with workspace use following
pipeline {
agent {
node {
customWorkspace "/home/jenkins/jenkins_workspace/${JOB_NAME}_${BUILD_NUMBER}"
}
}
post {
cleanup {
/* clean up our workspace */
deleteDir()
/* clean up tmp directory */
dir("${workspace}#tmp") {
deleteDir()
}
/* clean up script directory */
dir("${workspace}#script") {
deleteDir()
}
}
}
}
This snippet will work for default workspace also.
Using the 'WipeWorkspace' extension seems to work as well. It requires the longer form:
checkout([
$class: 'GitSCM',
branches: scm.branches,
extensions: scm.extensions + [[$class: 'WipeWorkspace']],
userRemoteConfigs: scm.userRemoteConfigs
])
More details here: https://support.cloudbees.com/hc/en-us/articles/226122247-How-to-Customize-Checkout-for-Pipeline-Multibranch-
Available GitSCM extensions here: https://github.com/jenkinsci/git-plugin/tree/master/src/main/java/hudson/plugins/git/extensions/impl
For Jenkins 2.190.1 this works for sure:
post {
always {
cleanWs deleteDirs: true, notFailBuild: true
}
}
pipeline {
agent any
tools {nodejs "node"}
environment {
}
parameters {
string(name: 'FOLDER', defaultValue: 'ABC', description: 'FOLDER', trim: true)
}
stages {
stage('1') {
steps{
}
}
stage("2") {
steps {
}
}
}
post {
always {
echo "Release finished do cleanup and send mails"
deleteDir()
}
success {
echo "Release Success"
}
failure {
echo "Release Failed"
}
cleanup {
echo "Clean up in post work space"
cleanWs()
}
}
}
We make sure we are working with a clean workspace by using a feature of the git plugin. You can add additional behaviors like 'Clean before checkout'. We use this as well for 'Prune stale remote-tracking branches'.
In my case, I want to clear out old files at the beginning of the build, but this is problematic since the source code has been checked out.
My solution is to ask git to clean out any files (from the last build) that it doesn't know about:
sh "git clean -x -f"
That way I can start the build out clean, and if it fails, the workspace isn't cleaned out and therefore easily debuggable.
Cleaning up : Since the post section of a Pipeline is guaranteed to run at the end of a Pipeline’s execution, we can add some notification or other steps to perform finalization, notification, or other end-of-Pipeline tasks.
pipeline {
agent any
stages {
stage('No-op') {
steps {
sh 'ls'
}
}
}
post {
cleanup {
echo 'One way or another, I have finished'
deleteDir() /* clean up our workspace */
}
}
}
Currently both deletedir() and cleanWs() do not work properly when using Jenkins kubernetes plugin, the pod workspace is deleted but the master workspace persists
it should not be a problem for persistant branches, when you have a step to clean the workspace prior to checkout scam. It will basically reuse the same workspace over and over again: but when using multibranch pipelines the master keeps the whole workspace and git directory
I believe this should be an issue with Jenkins,
any enlightenment here?
I usually use this:
post {
success {
cleanWs()
}
}