Jenkinsfile - get all changes between builds - jenkins

With reference to this question is there a way to get the equivalent information from when using the mult-branch pipeline? Specifically - the list of commits since the last successful build.
Currently we use the following
def scmAction = build?.actions.find { action ->
action instanceof jenkins.scm.api.SCMRevisionAction
}
return scmAction?.revision?.hash
but this only returns the last commit that triggered the build if multiple commits were pushed. I accept that the very first build of a new branch might be ambiguious but getting a list of commits that triggered the build when possible would be very useful.

I have found a solution that seems to work for us. It revolves around getting the currentBuild commit hash and then the lastSuccessfulBuild commit hash. First we wrote a utility method for getting a commit hash of a given Jenkins build object:
def commitHashForBuild(build) {
def scmAction = build?.actions.find { action -> action instanceof jenkins.scm.api.SCMRevisionAction }
return scmAction?.revision?.hash
}
then use that to get the lastSuccessfulBuild's hash:
def getLastSuccessfulCommit() {
def lastSuccessfulHash = null
def lastSuccessfulBuild = currentBuild.rawBuild.getPreviousSuccessfulBuild()
if ( lastSuccessfulBuild ) {
lastSuccessfulHash = commitHashForBuild(lastSuccessfulBuild)
}
return lastSuccessfulHash
}
finally combine those two in a sh function to get the list of commits
def lastSuccessfulCommit = getLastSuccessfulCommit()
def currentCommit = commitHashForBuild(currentBuild.rawBuild)
if (lastSuccessfulCommit) {
commits = sh(
script: "git rev-list $currentCommit \"^$lastSuccessfulCommit\"",
returnStdout: true
).split('\n')
println "Commits are: $commits"
}
you can then use the commits array to query various things in Git as your build requires. E.g. you can use this data to get a list of all changed files since the last successful build.
I have put this into a complete example Jenkinsfile Gist to show how it fits together in context.
A possible improvement would be to use a Java/Groovy native Git library instead of shelling out to a sh step.

I think Jenkins Last changes plugin can provide the information you need, take a look here:
https://plugins.jenkins.io/last-changes, following is an example:
node {
stage("checkout") {
git url: 'https://github.com/jenkinsci/last-changes-plugin.git'
}
stage("last-changes") {
def publisher = LastChanges.getLastChangesPublisher "LAST_SUCCESSFUL_BUILD", "SIDE", "LINE", true, true, "", "", "", "", ""
publisher.publishLastChanges()
def changes = publisher.getLastChanges()
println(changes.getEscapedDiff())
for (commit in changes.getCommits()) {
println(commit)
def commitInfo = commit.getCommitInfo()
println(commitInfo)
println(commitInfo.getCommitMessage())
println(commit.getChanges())
}
}
}
Note that by default (without the need for groovy scripting) the plugin makes the list of commits available for browsing on the jenkins UI, see here.
I hope it helps.

I was looking for how to get the git hash of the last successful build to do some git diff-tree change detection. I was following Friedrich's answer to programmatically get the SCM actions and revision hash out of the build returned from currentBuild.rawBuild.getPreviousSuccessfulBuild().
I was struggling to get this working with the current Jenkins API, and then realized that the git Jenkins plugin now exports the env var GIT_PREVIOUS_SUCCESSFUL_COMMIT. I now just refer to ${env.GIT_COMMIT} and ${env.GIT_PREVIOUS_SUCCESSFUL_COMMIT} to do a git diff-tree in my pipelines.

Related

Jenkins: Parameters disappear from pipeline job after running the job

I've been trying to construct multiple jobs from a list and everything seems to be working as expected. But as soon as I execute the first build (which works correctly) the parameters in the job disappears. This is how I've constructed the pipelineJob for the project.
import javaposse.jobdsl.dsl.DslFactory
def repositories = [
[
id : 'jenkins-test',
name : 'jenkins-test',
displayName: 'Jenkins Test',
repo : 'ssh://<JENKINS_BASE_URL>/<PROJECT_SLUG>/jenkins-test.git'
]
]
DslFactory dslFactory = this as DslFactory
repositories.each { repository ->
pipelineJob(repository.name) {
parameters {
stringParam("BRANCH", "master", "")
}
logRotator{
numToKeep(30)
}
authenticationToken('<TOKEN_MATCHES_WITH_THE_BITBUCKET_POST_RECEIVE_HOOK>')
displayName(repository.displayName)
description("Builds deploy pipelines for ${repository.displayName}")
definition {
cpsScm {
scm {
git {
branch('${BRANCH}')
remote {
url(repository.repo)
credentials('<CREDENTIAL_NAME>')
}
extensions {
localBranch('${BRANCH}')
wipeOutWorkspace()
cloneOptions {
noTags(false)
}
}
}
scriptPath('Jenkinsfile)
}
}
}
}
}
After running the above script, all the required jobs are created successfully. But then once I build any job, the parameters disappear.
After that when I run the seed job again, the job starts showing the parameter. I'm having a hard time figuring out where the problem is.
I've tried many things but nothing works. Would appreciate any help. Thanks.
This comment helped me to figure out similar issue with my .groovy file:
I called parameters property twice (one at the node start and then tried to set other parameters in if block), so the latter has overwritten the initial parameters.
BTW, as per the comments in the linked ticket, it is an issue with both scripted and declarative pipelines.
Fixed by providing all job parameters in each parameters call - for the case with ifs.
Though I don't see repeated calls in the code you've provided, please check the full groovy files for your jobs and add all parameters to all parameters {} blocks.

Show custom form then trigger a job

I want to show a dynamically generated form (for example with a list of currently existing branches) then trigger a job with the selected parameters.
There is Pipeline Input Step but:
You can optionally request information back, hence the name of the step. The parameter entry screen can be accessed via a link at the bottom of the build console log or via link in the sidebar for a build.
I need after clicking "Build" to show the custom form immediately.
A solution I see is to have a third party web-server which generates the form then remotely triggers the job, but I would rather use something available inside Jenkins, so that I would have access to Jenkins internals when rendering the form template (using Groovy).
Is there a solution?
Have you tried the git Parameter Plugin ??
https://wiki.jenkins.io/display/JENKINS/Git+Parameter+Plugin
The final result will populate the list of branches from your repo and you can select the branch you would like to build
Configuration
It's straight forward and it will build the branch that you have selected :)
This will build only 1 branch i.e. no multiple option
IF you want to do the multiple option then it gets tricky....
You have to get the Active Choices Plug-in
https://wiki.jenkins.io/display/JENKINS/Active+Choices+Plugin
Create a shell script(provide permissions)-- get_git_branches.sh
add the following in the shell
#!/bin/bash
GIT_URL=$1
git ls-remote --heads --tags ${GIT_URL} | awk -F" " '{print $NF}'
Configuration
Add the following
tags = []
text = "get_git_branches.sh https://user:pass#bitbucket.org/project/repo_name.git".execute().text
text.eachLine { tags.push(it) }
return tags
Pipeline
node {
echo States
def values = Branches.split(',')
for(value in values)
println value //build them 'mvn build value'
}
I have given the example based on your "e.g. requirement" but if you wish to do something different like you mentioned creating a form Active choice plugin is the best bet as you can create your custom script with choice,textbox,multiple select (basically do a javascript page rendering) the wiki which I have posted above has great examples.
Hope it helps :)
You can write your own Jenkinsfile in groovy syntax and generate your parameters for the build. You can even create your own HTML files with that. It is part of the Pipeline Multibranch Plugin
You can get a view like this:
With following Jenkinsfile:
#!/usr/bin/env groovy
def buildInfo
def gitEnv
pipeline {
agent any
options {
timeout(time: 1, unit: 'HOURS')
}
tools {
nodejs 'NodeJS LTS 8.9.4'
}
parameters {
choice(
choices: 'BUILD_AND_DEPLOY\nBUILD\nDEPLOY_ONLY',
description: '''
Sets the desired build and deployment level
<table>
<thead>
<tr>
<th>Level</th>
<th>Effect</th>
</tr>
</thead>
<tbody>
<tr>
<td>BUILD_AND_DEPLOY</td>
<td>BUILD and DEPLOY afterwards</td>
</tr>
<tr>
<td>BUILD</td>
<td>Builds the project and uploads the resulting artifact(s)</td>
</tr>
<tr>
<td>DEPLOY_ONLY</td>
<td>Fetches the latest service artifact and deploys it. Replaces existing service on the agent if present.</td>
</tr>
</tbody>
''',
name: 'deploymentLevel'
)
}
stages {
stage ('Preparation') {
steps {
script {
// maybe do something special with a tag
gitEnv = checkout scm
}
}
}
stage ('Install Dependencies') {
when {
anyOf {
expression { return params.deploymentLevel == 'BUILD' }
expression { return params.deploymentLevel == 'BUILD_AND_DEPLOY' }
}
}
steps {
bat 'npm install'
}
}
stage ('and more ...')
{
// build and ...
}
}
There is also a nice tutorial how to generate the choices from a json: Jenkins dynamic parameters using Extended Choice Parameter Plugin and Groovy
As far as I know ,there is no direct solution for this . What you can try (if it is extremely required ) is you can break this step into two jobs :
A
B
In job A you can hit build and then send a confirmation mail or slack with a form and you can fill the variables from the build . Then in that mail you have to give a url on submit which also passes the required parameters as a GET or POST request .
This url will point to a server side script where with the help of these parameters you can build the job B .If you decide to go with this and get stuck somewhere I can further elaborate .
I see in JENKINS-46971 there is a choice parameter:
def environmentChoices = ['blue', 'green'].join('\n')
def stagingEnvironmentInpit = input( message: "apistaging.evolution-software.com is currently pointed to ${currentEnvironment}. Where do you want to promote to?", ok: 'Deploy', parameters: [choice(choices: environmentChoices, name: 'RELEASE_ENVIRONMENT')] )
echo env.RELEASE_ENVIRONMENT
You could adapt that to your case, defining a branches variable with the list of Git branches:
def branches = sh(returnStdout: true, script: "git for-each-ref --format='%(refname:short)' refs/heads/")
And use that in your input step.

How to differentiate build triggers in Jenkins Pipeline

I'm hoping to add a conditional stage to my Jenkinsfile that runs depending on how the build was triggered. Currently we are set up such that builds are either triggered by:
changes to our git repo that are picked up on branch indexing
a user manually triggering the build using the 'build now' button in the UI.
Is there any way to run different pipeline steps depending on which of these actions triggered the build?
The following code should works to determine if a user has started the pipeline or a timer/other trigger:
def isStartedByUser = currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) != null
In Jenkins Pipeline without currentBuild.rawBuild access the build causes could be retrieved in the following way:
// started by commit
currentBuild.getBuildCauses('jenkins.branch.BranchEventCause')
// started by timer
currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')
// started by user
currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
You can get a boolean value with:
isTriggeredByTimer = !currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').isEmpty()
Or, as getBuildCauses() returns an array, the array's size will work correctly with Groovy truthy semantics:
if (currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')) {
The ability to get causes for a workflow run was released in version 2.22 (2018 Nov 02) to the Pipeline Supporting APIs Plugin. The feature was requested in JENKINS-41272.
A couple methods were added to the currentBuild global variable with that release:
getBuildCauses
Returns a JSON array of build causes for the current build
EXPERIMENTAL - MAY CHANGE getBuildCauses(String causeClass)
Takes a string representing the fully qualified Cause class and returns a JSON array of build causes filtered by that type for the current build, or an empty JSON array if no causes of the specified type apply to the current build
And an example from me submitting:
echo "${currentBuild.buildCauses}" // same as currentBuild.getBuildCauses()
echo "${currentBuild.getBuildCauses('hudson.model.Cause$UserCause')}"
echo "${currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')}"
And the output:
[Pipeline] echo
[[_class:hudson.model.Cause$UserIdCause, shortDescription:Started by user anonymous, userId:null, userName:anonymous], [_class:org.jenkinsci.plugins.workflow.cps.replay.ReplayCause, shortDescription:Replayed #12]]
[Pipeline] echo
[]
[Pipeline] echo
[]
[Pipeline] End of Pipeline
Finished: SUCCESS
NOTE
There appears to be an issue with the currentBuild.getBuildCauses(type) when the type is a type of Cause contributed by a plugin. For example, currentBuild.getBuildCauses('org.jenkinsci.plugins.workflow.cps.replay.ReplayCause') fails with a java.lang.ClassNotFoundException. This was reported in JENKINS-54673 for the 2.22 version of the Pipeline: Supporting APIs (workflow-support) plugin. It is reportedly fixed in the 2.24 version.
I might be missing something, but you can achieve what you want easily by making use of the when directive:
pipeline {
agent any
stages {
stage('Always') {
steps {
echo "I am always executed"
}
}
stage('ManualTimed') {
steps {
echo "I am only executed when triggered manually or timed"
}
when {
beforeAgent true
anyOf {
triggeredBy 'TimerTrigger'
triggeredBy cause: 'UserIdCause'
}
}
}
stage('GitLabWebHookCause') {
steps {
echo "I am only executed when triggered by SCM push"
}
when {
beforeAgent true
triggeredBy 'GitLabWebHookCause'
}
}
}
}
You will find many similar useful examples for various use cases in the documentation of the when directive.
Edit:
thanks to Jean-Francois Larvoire's answer, I was able to figure out 'my trigger' GitLabWebHookCause I required for my use case.
#vitalii-blagodir:
Your answer works for detecting builds triggered by users and timers, but not by commits.
Instead, I found this to work in my case:
def isTriggeredByIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
def isTriggeredByCommit = currentBuild.getBuildCauses('com.cloudbees.jenkins.GitHubPushCause').size()
def isTriggeredByUser = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').size()
def isTriggeredByTimer = currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').size()
The .size() suffix returns 0 if the object is missing, or 1 if it's present. This makes the result usable as a boolean.
For finding the object name to use, I found it convenient to display this in the log:
echo "# Build causes"
def buildCauses = currentBuild.buildCauses
def numCause = 0
for (cause in buildCauses) {
echo "${numCause++}: ${cause.shortDescription}" // Display a human-readable index and description
echo "${cause}" // Display the object class name. This allows knowing what names to use in getBuildCauses(name) calls below.
}
Finally, if the goal is to abort a pipeline build in specific cases, then the test must be done before the beginning of the pipeline.
For example, we had a problem with the branch indexing triggering extra useless builds. This was fixed by adding this before the pipeline:
// Avoid useless buils: The branch indexing should only trigger the initial build of a new branch.
def isTriggeredByBranchIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
if (isTriggeredByBranchIndexing && currentBuild.previousBuild) { // Then it's not the initial build.
echo "# Reindexing a branch already built. It is useless to rebuild it now. Aborting."
currentBuild.result = 'SUCCESS' // Make sure the build is not displayed in red in the Jenkins UI.
return // Abort before the pipeline even starts. (Inside the pipeline, this would only abort one stage.)
}
I think that the answers here are incomplete and do not provide an actual ready to use answer. Here's my code to get it working:
import com.cloudbees.groovy.cps.NonCPS
#NonCPS
def isStartedByTimer() {
def buildCauses = currentBuild.rawBuild.getCauses()
echo buildCauses
boolean isStartedByTimer = false
for (buildCause in buildCauses) {
if ("${buildCause}".contains("hudson.triggers.TimerTrigger\$TimerTriggerCause")) {
isStartedByTimer = true
}
}
echo isStartedByTimer
return isStartedByTimer
}
// [...]
// Other pipeline stuff
script {
isStartedByTimer()
}
When started by user:
00:00:01.353 [hudson.model.Cause$UserIdCause#fa5cb22a]
[Pipeline] echo
00:00:01.358 false
When started by timer:
00:00:01.585 [hudson.triggers.TimerTrigger$TimerTriggerCause#5]
[Pipeline] echo
00:00:01.590 true
Note: the NonCPS decorator is needed because otherwise the next non-script step will throw.
Assuming the two different build causes are "timer" and "push" (to a git repo), you can add the following stage to your Jenkinsfile (in a declarative Jenkins pipeline) to make use of getBuildCauses():
pipeline {
stages {
stage('preparation') {
steps {
script {
// get build cause (time triggered vs. SCM change)
def buildCause = currentBuild.getBuildCauses()[0].shortDescription
echo "Current build was caused by: ${buildCause}\n"
// e.g. "Current build was caused by: Started by GitHub push by mirekphd"
// vs. "Started by timer"
}
}
}
}
}
Then I can decide whether to perform certain stages conditionally (depending on the build cause). For example, pulling a docker base image and inspecting for changes in system libraries (likely security updates) should be done periodically, regardless of whether there was a source code change or not.
We can use "BUILD_CAUSE" variable for getting the information about who initiated the run
for [jenkins-pipeline] you may use
currentBuild.rawBuild.getCauses()
(see github.com/jenkinsci/pipeline-examples/blob/master/… for more details)
There was a similar requirement, where user detail who triggered the build should be there in success / failure notification. The job was already had time based triggered, hence could not use wrap([$class: 'BuildUser']) directly.
I used below step, which print username if the job is triggered manually or timer triggered. So, I used this:
pipeline {
agent any
stages {
stage('Test') {
steps {
script{
env.buildCauses = currentBuild.rawBuild.getCauses()
if (buildCauses.contains("hudson.triggers.TimerTrigger")){
env.builduser = "TimerTrigger"
} else {
wrap([$class: 'BuildUser']) {
env.builduser = "${BUILD_USER}"
}
}
}
echo "Initiated by: ${env.builduser}"
}
}
}
}

Custom changelog in Jenkins Pipelines

I was wondering if it is possible to have a custom changelog appear for Jenkins Pipelines. Ideally, I'd like to propagate the downstream changelogs, but failing that I've tried to create a custom changelog derived from the downstream builds. However, it doesn't appear to work (with no option for viewing the pipeline's workspace either).
I was wondering if this is something that I'm just getting wrong or whether it's actually supported or not.
This is the sample code I'm testing with
node('master')
{
stage('Source')
{
build 'SourceBuild'
def rootDir = currentBuild.rawBuild.getRootDir().toString()
echo rootDir
def changelog = new File(rootDir, "changelog.xml")
PrintWriter writer = new PrintWriter(new FileWriter(changelog));
writer.println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
writer.println("<changelog>");
writer.println(String.format("\t\t<user>%s</user>", 'User'));
writer.println(String.format("\t\t<comment>Change</comment>", 'Comment'));
writer.println("\t</changeset>");
writer.println("</changelog>");
writer.close();
}
}
Many thanks
In Jenkins pipeline, I noticed that there is a global variable named currentBuild. It has a readable property called changeSets. I would rather take this approach with pipeline instead of playing around with changelog.xml
stage('some name') {
def gitChangeSetList = currentBuild.changeSets
formatGitChangeLog(gitChangeSetList)
}
def formatGitChangeLog(GitChangeSetList changeSetList) {
def formatStr = ""
for (setItem in changeSetList) {
for (change in setItem.getLogs()) {
formatStr += "${change.getAuthor().getDisplayName()}: ${change.getMsg()}\n"
}
}
return formatStr
}
currentBuild.changeSets is of type GitChangeSetList. From the javadoc, we can derive various methods involved in GitChangeSet.

How to programmatically trigger a specific jenkins build managed via github-organization?

I am using the github-organization plugin to manage jenkins jobs from github but I discovered that Jenkins API does not report these builds.
In fact the API list the entire organization as a single job.
How can I build a specific repository and branch using the API?
To be clear, I am looking for some groovy code to add inside the Jenkinsfile
#!groovy
stage 'test-downstream'
node {
def job = build job: 'some-job'
}
Now, the problem is that Jenkins is seeing the entire organization as a single job!
If I use Jenkins API to retrieve the jobs, it will return only the organization, and not all the repositories and jobs inside it.
I suspect that's because the way this plugin was implemented and I suppose that I need to give some extra parameters in order to specify which repository and branch I want to build inside the organization.... building an organization does not make much sense.
The question is vague, but I am guessing “API” in this context means the REST API to trigger builds. You can use for example
curl -X POST -u user:apitoken http://jenkins/job/yourorg/job/yourrepo/job/master/build
The following code trigger job via System Groovy build step. Please note that system groovy always run on master so passing info from previous build steps might be tricky.
import jenkins.model.*
import hudson.model.*
import java.util.concurrent.*
def run_job(job_name) {
def currentBuild = Thread.currentThread().executable
def jenkins = jenkins.model.Jenkins.getInstance();
def job = jenkins.getItemByFullName(job_name);
if (job == null)
throw new hudson.AbortException("Cannot find job:" + job_name);
def params =[
new StringParameterValue('PARAMETER1', "invoke 1 param1"),
new StringParameterValue('PARAMETER2', ",invoke 1 param2")
]
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
def future_build = job.scheduleBuild2(0,causeAction,paramsAction);
def running_build = future_build.waitForStart()
return running_build
}
run_job("runner1")

Resources