Custom changelog in Jenkins Pipelines - jenkins

I was wondering if it is possible to have a custom changelog appear for Jenkins Pipelines. Ideally, I'd like to propagate the downstream changelogs, but failing that I've tried to create a custom changelog derived from the downstream builds. However, it doesn't appear to work (with no option for viewing the pipeline's workspace either).
I was wondering if this is something that I'm just getting wrong or whether it's actually supported or not.
This is the sample code I'm testing with
node('master')
{
stage('Source')
{
build 'SourceBuild'
def rootDir = currentBuild.rawBuild.getRootDir().toString()
echo rootDir
def changelog = new File(rootDir, "changelog.xml")
PrintWriter writer = new PrintWriter(new FileWriter(changelog));
writer.println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
writer.println("<changelog>");
writer.println(String.format("\t\t<user>%s</user>", 'User'));
writer.println(String.format("\t\t<comment>Change</comment>", 'Comment'));
writer.println("\t</changeset>");
writer.println("</changelog>");
writer.close();
}
}
Many thanks

In Jenkins pipeline, I noticed that there is a global variable named currentBuild. It has a readable property called changeSets. I would rather take this approach with pipeline instead of playing around with changelog.xml
stage('some name') {
def gitChangeSetList = currentBuild.changeSets
formatGitChangeLog(gitChangeSetList)
}
def formatGitChangeLog(GitChangeSetList changeSetList) {
def formatStr = ""
for (setItem in changeSetList) {
for (change in setItem.getLogs()) {
formatStr += "${change.getAuthor().getDisplayName()}: ${change.getMsg()}\n"
}
}
return formatStr
}
currentBuild.changeSets is of type GitChangeSetList. From the javadoc, we can derive various methods involved in GitChangeSet.

Related

XmlSlurper() in jenkins pipeline. how to avoid java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild

I'm trying to read properties from my pom.xml file. I tried the following and it worked:
steps {
script {
def xmlfile = readFile "pom.xml"
def xml = new XmlSlurper().parseText(xmlfile)
def version = "${xml.version}"
echo version
}
}
When I tried to do something like this:
steps {
script {
def xmlfile = readFile "pom.xml"
def xml = new XmlSlurper().parseText(xmlfile)
def version = "${xml.version}"
def mystring = "blabhalbhab-${version}"
echo mystring
}
}
the pipeline suddenly fails with the error:
Caused: java.io.NotSerializableException: groovy.util.slurpersupport.NodeChild
What might be the problem here?
EDIT: just adding this for others finding it with the same use case. My specific question was about how to avoid the CPS related error with XmlSlurper(). BUT for anyone else trying to parse POMs, afraid of that PR that supposedly will deprecate readMavenPom, the safest most maveny way of doing this is probably something like:
def version = sh script: "mvn help:evaluate -f 'pom.xml' -Dexpression=project.version -q -DforceStdout", returnStdout: true trim()
This way your using maven itself to tell you what the version is and not grepping or sedding all over the damn place. How to get Maven project version to the bash command line
In general, using groovy.util.slurpersupport.NodeChild (the type of your xml variable) or groovy.util.slurpersupport.NodeChildren (the type of xml.version) inside CPS pipeline is a bad idea. Both classes are not serializable, so you can't predicate their behavior in the Groovy CPS. For instance, I run successfully your second example in my Jenkins Pipeline. Most probably because the example you gave is not complete or something like that.
groovy:000> xml = new XmlSlurper().parseText("<tag></tag>")
===>
groovy:000> xml instanceof Serializable
===> false
groovy:000> xml.tag instanceof Serializable
===> false
groovy:000> xml.dump()
===> <groovy.util.slurpersupport.NodeChild#0 node=groovy.util.slurpersupport.Node#5b1f29fa parent= name=tag namespacePrefix=* namespaceMap=[xml:http://www.w3.org/XML/1998/namespace] namespaceTagHints=[xml:http://www.w3.org/XML/1998/namespace]>
groovy:000> xml.tag.dump()
===> <groovy.util.slurpersupport.NodeChildren#0 size=-1 parent= name=tag namespacePrefix=* namespaceMap=[xml:http://www.w3.org/XML/1998/namespace] namespaceTagHints=[xml:http://www.w3.org/XML/1998/namespace]>
groovy:000>
If you want to read pom.xml file, use the readMavenPom pipeline step. It is dedicated to read pom files and what is most important - it is safe to do it without applying any workarounds. This step comes with the pipeline-utility-steps plugin.
However, if you want to use XmlSlurper for some reason, you need to use it inside the method that is annotated with #NonCPS. That way you can access "pure" Groovy and avoid problems you have faced. (Yet still using readMavenPom is the safest way to achieve what you are trying to do.) The point here is to use any non-serializable objects inside a #NonCPS scope so the pipeline does not try to serialize it.
Below you can find a simple example of the pipeline that shows both approaches.
pipeline {
agent any
stages {
stage("Using readMavenPom") {
steps {
script {
def xmlfile = readMavenPom file: "pom.xml"
def version = xmlfile.version
echo "version = ${version}"
}
}
}
stage("Using XmlSlurper") {
steps {
script {
def xmlfile = readFile "pom.xml"
def version = extractFromXml(xmlfile) { xml -> xml.version }
echo "version = ${version}"
}
}
}
}
}
#NonCPS
String extractFromXml(String xml, Closure closure) {
def node = new XmlSlurper().parseText(xml)
return closure.call(node)?.text()
}
PS: not to mention that using XmlSlurper requires at least script 3 approvals before you can start using it.

How to get the current Jenkins pipeline StepContext

I have a step in a pipeline that pulls objects from the context and uses them. However, I need to access those objects outside of the steps to feed into different steps, and the second step doesn't expose it.
stage() {
steps {
script {
def status = waitForQualityGate()
// Use the taskId
}
}
}
}
The waitForQualityGate() call only returns a boolean, so I can't access it there.
I could instead manually initialize the step, like so:
script {
def qualityGate = new WaitForQualityGateStep()
def taskId = qualityGate.getTaskId()
}
but the taskId is null. If I try to run the start methods manually on the step:
script {
def qualityGate = new WaitForQualityGateStep()
qualityGate.start().start()
def taskId = qualityGate.getTaskId()
}
It fails with the message:
java.lang.IllegalStateException: you must either pass in a StepContext to the StepExecution constructor, or have the StepExecution be created automatically
The WaitForQualityGateStep has the info I need, but I can't initialize it without having a StepContext (which is an Abstract class). How can I get one from the pipeline?
You can define the variable before the pipeline and in the step just set its value. This way the variable is visible across the pipeline.
I still have no idea how to manually get a step context to manually execute a step, but in case anyone else finds this by trying to get information out of the Sonar plugin, this is how I got the task ID that I needed.
def output = sh(script: "mvn sonar:sonar", returnStdout: true)
echo output // The capture prevents printing to console
def taskUri = output.find(~'/api/ce/task\\?id=[\\w-]*')

Groovy Missing Property Exception

I have a jenkins build that needs to get the filenames for all files checked in within a changeset.
I have installed groovy on the slave computer and configured Jenkins to use it. I am running the below script that should return the names (or so I assume as this may be wrong as well) and print to the console screen however I am getting this error:
groovy.lang.MissingPropertyException: No such property: paths for class: hudson.plugins.tfs.model.ChangeSet
Here is the Groovy System Script:
import hudson.plugins.tfs.model.ChangeSet
// work with current build
def build = Thread.currentThread()?.executable
// get ChangesSets with all changed items
def changeSet= build.getChangeSet()
def items = changeSet.getItems()
def affectedFiles = items.collect { it.paths }
// get file names
def fileNames = affectedFiles.flatten().findResults
fileNames.each {
println "Item: $it" // `it` is an implicit parameter corresponding to the current element
}
I am very new to Groovy and Jenkins so if its syntax issue or if I'm missing a step please let me know.
I don't know the version of jenkins you are using but according to the sourcecode of ChangeSet that you can find here I suggest you to replace line 9 with:
def affectedFiles = items.collect { it.getAffectedPaths() }
// or with the equivalent more groovy-idiomatic version
def affectedFiles = items.collect { it.affectedPaths }
Feel free to comment the answer if there will be more issues.

How to differentiate build triggers in Jenkins Pipeline

I'm hoping to add a conditional stage to my Jenkinsfile that runs depending on how the build was triggered. Currently we are set up such that builds are either triggered by:
changes to our git repo that are picked up on branch indexing
a user manually triggering the build using the 'build now' button in the UI.
Is there any way to run different pipeline steps depending on which of these actions triggered the build?
The following code should works to determine if a user has started the pipeline or a timer/other trigger:
def isStartedByUser = currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) != null
In Jenkins Pipeline without currentBuild.rawBuild access the build causes could be retrieved in the following way:
// started by commit
currentBuild.getBuildCauses('jenkins.branch.BranchEventCause')
// started by timer
currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')
// started by user
currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
You can get a boolean value with:
isTriggeredByTimer = !currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').isEmpty()
Or, as getBuildCauses() returns an array, the array's size will work correctly with Groovy truthy semantics:
if (currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')) {
The ability to get causes for a workflow run was released in version 2.22 (2018 Nov 02) to the Pipeline Supporting APIs Plugin. The feature was requested in JENKINS-41272.
A couple methods were added to the currentBuild global variable with that release:
getBuildCauses
Returns a JSON array of build causes for the current build
EXPERIMENTAL - MAY CHANGE getBuildCauses(String causeClass)
Takes a string representing the fully qualified Cause class and returns a JSON array of build causes filtered by that type for the current build, or an empty JSON array if no causes of the specified type apply to the current build
And an example from me submitting:
echo "${currentBuild.buildCauses}" // same as currentBuild.getBuildCauses()
echo "${currentBuild.getBuildCauses('hudson.model.Cause$UserCause')}"
echo "${currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')}"
And the output:
[Pipeline] echo
[[_class:hudson.model.Cause$UserIdCause, shortDescription:Started by user anonymous, userId:null, userName:anonymous], [_class:org.jenkinsci.plugins.workflow.cps.replay.ReplayCause, shortDescription:Replayed #12]]
[Pipeline] echo
[]
[Pipeline] echo
[]
[Pipeline] End of Pipeline
Finished: SUCCESS
NOTE
There appears to be an issue with the currentBuild.getBuildCauses(type) when the type is a type of Cause contributed by a plugin. For example, currentBuild.getBuildCauses('org.jenkinsci.plugins.workflow.cps.replay.ReplayCause') fails with a java.lang.ClassNotFoundException. This was reported in JENKINS-54673 for the 2.22 version of the Pipeline: Supporting APIs (workflow-support) plugin. It is reportedly fixed in the 2.24 version.
I might be missing something, but you can achieve what you want easily by making use of the when directive:
pipeline {
agent any
stages {
stage('Always') {
steps {
echo "I am always executed"
}
}
stage('ManualTimed') {
steps {
echo "I am only executed when triggered manually or timed"
}
when {
beforeAgent true
anyOf {
triggeredBy 'TimerTrigger'
triggeredBy cause: 'UserIdCause'
}
}
}
stage('GitLabWebHookCause') {
steps {
echo "I am only executed when triggered by SCM push"
}
when {
beforeAgent true
triggeredBy 'GitLabWebHookCause'
}
}
}
}
You will find many similar useful examples for various use cases in the documentation of the when directive.
Edit:
thanks to Jean-Francois Larvoire's answer, I was able to figure out 'my trigger' GitLabWebHookCause I required for my use case.
#vitalii-blagodir:
Your answer works for detecting builds triggered by users and timers, but not by commits.
Instead, I found this to work in my case:
def isTriggeredByIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
def isTriggeredByCommit = currentBuild.getBuildCauses('com.cloudbees.jenkins.GitHubPushCause').size()
def isTriggeredByUser = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').size()
def isTriggeredByTimer = currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').size()
The .size() suffix returns 0 if the object is missing, or 1 if it's present. This makes the result usable as a boolean.
For finding the object name to use, I found it convenient to display this in the log:
echo "# Build causes"
def buildCauses = currentBuild.buildCauses
def numCause = 0
for (cause in buildCauses) {
echo "${numCause++}: ${cause.shortDescription}" // Display a human-readable index and description
echo "${cause}" // Display the object class name. This allows knowing what names to use in getBuildCauses(name) calls below.
}
Finally, if the goal is to abort a pipeline build in specific cases, then the test must be done before the beginning of the pipeline.
For example, we had a problem with the branch indexing triggering extra useless builds. This was fixed by adding this before the pipeline:
// Avoid useless buils: The branch indexing should only trigger the initial build of a new branch.
def isTriggeredByBranchIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
if (isTriggeredByBranchIndexing && currentBuild.previousBuild) { // Then it's not the initial build.
echo "# Reindexing a branch already built. It is useless to rebuild it now. Aborting."
currentBuild.result = 'SUCCESS' // Make sure the build is not displayed in red in the Jenkins UI.
return // Abort before the pipeline even starts. (Inside the pipeline, this would only abort one stage.)
}
I think that the answers here are incomplete and do not provide an actual ready to use answer. Here's my code to get it working:
import com.cloudbees.groovy.cps.NonCPS
#NonCPS
def isStartedByTimer() {
def buildCauses = currentBuild.rawBuild.getCauses()
echo buildCauses
boolean isStartedByTimer = false
for (buildCause in buildCauses) {
if ("${buildCause}".contains("hudson.triggers.TimerTrigger\$TimerTriggerCause")) {
isStartedByTimer = true
}
}
echo isStartedByTimer
return isStartedByTimer
}
// [...]
// Other pipeline stuff
script {
isStartedByTimer()
}
When started by user:
00:00:01.353 [hudson.model.Cause$UserIdCause#fa5cb22a]
[Pipeline] echo
00:00:01.358 false
When started by timer:
00:00:01.585 [hudson.triggers.TimerTrigger$TimerTriggerCause#5]
[Pipeline] echo
00:00:01.590 true
Note: the NonCPS decorator is needed because otherwise the next non-script step will throw.
Assuming the two different build causes are "timer" and "push" (to a git repo), you can add the following stage to your Jenkinsfile (in a declarative Jenkins pipeline) to make use of getBuildCauses():
pipeline {
stages {
stage('preparation') {
steps {
script {
// get build cause (time triggered vs. SCM change)
def buildCause = currentBuild.getBuildCauses()[0].shortDescription
echo "Current build was caused by: ${buildCause}\n"
// e.g. "Current build was caused by: Started by GitHub push by mirekphd"
// vs. "Started by timer"
}
}
}
}
}
Then I can decide whether to perform certain stages conditionally (depending on the build cause). For example, pulling a docker base image and inspecting for changes in system libraries (likely security updates) should be done periodically, regardless of whether there was a source code change or not.
We can use "BUILD_CAUSE" variable for getting the information about who initiated the run
for [jenkins-pipeline] you may use
currentBuild.rawBuild.getCauses()
(see github.com/jenkinsci/pipeline-examples/blob/master/… for more details)
There was a similar requirement, where user detail who triggered the build should be there in success / failure notification. The job was already had time based triggered, hence could not use wrap([$class: 'BuildUser']) directly.
I used below step, which print username if the job is triggered manually or timer triggered. So, I used this:
pipeline {
agent any
stages {
stage('Test') {
steps {
script{
env.buildCauses = currentBuild.rawBuild.getCauses()
if (buildCauses.contains("hudson.triggers.TimerTrigger")){
env.builduser = "TimerTrigger"
} else {
wrap([$class: 'BuildUser']) {
env.builduser = "${BUILD_USER}"
}
}
}
echo "Initiated by: ${env.builduser}"
}
}
}
}

Can a Job DSL script be tested

Ideally I'd like to be able to invoke the script with some kind of unit test before I have it execute on a Jenkins.
Is there any way to test a Job DSL script other than having jenkins run it?
Besides the examples in job-dsl-gradle-example, you can also go a step further and write tests for individual files or jobs. For example let's assume you have a job configuration file located in jobs/deployJob.groovy
import javaposse.jobdsl.dsl.DslScriptLoader
import javaposse.jobdsl.dsl.MemoryJobManagement
import javaposse.jobdsl.dsl.ScriptRequest
import spock.lang.Specification
class TestDeployJobs extends Specification {
def 'test basic job configuration'() {
given:
URL scriptURL = new File('jobs').toURI().toURL()
ScriptRequest scriptRequest = new ScriptRequest('deployJob.groovy', null, scriptURL)
MemoryJobManagement jobManagement = new MemoryJobManagement()
when:
DslScriptLoader.runDslEngine(scriptRequest, jobManagement)
then:
jobManagement.savedConfigs.each { String name, String xml ->
with(new XmlParser().parse(new StringReader(xml))) {
// Make sure jobs only run manually
triggers.'hudson.triggers.TimerTrigger'.spec.text().isEmpty()
// only deploy every environment once at a time
concurrentBuild.text().equals('false')
// do a workspace cleanup
buildWrappers.'hudson.plugins.ws__cleanup.PreBuildCleanup'
// make sure masked passwords are active
!buildWrappers.'com.michelin.cio.hudson.plugins.maskpasswords.MaskPasswordsBuildWrapper'.isEmpty()
}
}
}
}
This way you are able to go through every XML node you want to make sure to have all the right values set.
Have a look at the job-dsl-gradle-example. The repo contains a test for DSL scripts.
Doing it in the same way as crasp but using Jenkins test harness as explained in Jenkins Unit Test page, which is slower but would work with auto-generated DSL giving syntax errors as explained here.
After setting the code as explained here, you can just do a test like this one:
#Unroll
void 'check descriptions #file.name'(File file) {
given:
JobManagement jobManagement = new JenkinsJobManagement(System.out, [:], new File('.'))
Jenkins jenkins = jenkinsRule.jenkins
when:
GeneratedItems items = new DslScriptLoader(jobManagement).runScript(file.text)
then:
if (!items.jobs.isEmpty()) {
items.jobs.each { GeneratedJob generatedJob ->
String text = getItemXml(generatedJob, jenkins)
with(new XmlParser().parse(new StringReader(text))) {
// Has some description
!description.text().isEmpty()
}
}
}
where:
file << TestUtil.getJobFiles()
}

Resources