How to limit Jenkins concurrent multibranch pipeline builds? - jenkins

I am looking at limiting the number of concurrent builds to a specific number in Jenkins, leveraging the multibranch pipeline workflow but haven't found any good way to do this in the docs or google.
Some docs say this can be accomplished using concurrency in the stage step of a Jenkinsfile but I've also read elsewhere that that is a deprecated way of doing it.
It looks like there was something released fairly recently for limiting concurrency via Job Properties but I couldn't find documentation for it and I'm having trouble following the code. The only thing I found a PR that shows the following:
properties([concurrentBuilds(false)])
But I am having trouble getting it working.
Does anybody know or have a good example of how to limit the number of concurrent builds for a given, multibranch project? Maybe a Jenkinsfile snippet that shows how to limit or cap the number of multibranch concurrent builds?

Found what I was looking for. You can limit the concurrent builds using the following block in your Jenkinsfile.
node {
// This limits build concurrency to 1 per branch
properties([disableConcurrentBuilds()])
//do stuff
...
}
The same can be achieved with a declarative syntax:
pipeline {
options {
disableConcurrentBuilds()
}
}

Limiting concurrent builds or stages are possible with the Lockable Resources Plugin (GitHub). I always use this mechanism to ensure that no publishing/release step is executed at the same time, while normal stages can be build concurrently.
echo 'Starting'
lock('my-resource-name') {
echo 'Do something here that requires unique access to the resource'
// any other build will wait until the one locking the resource leaves this block
}
echo 'Finish'

As #VadminKotov indicated it is possible to disable concurrentbuilds using jenkins declarative pipelines as well:
pipeline {
agent any
options { disableConcurrentBuilds() }
stages {
stage('Build') {
steps {
echo 'Hello Jenkins Declarative Pipeline'
}
}
}
}
disableConcurrentBuilds
Disallow concurrent executions of the Pipeline. Can be useful for
preventing simultaneous accesses to shared resources, etc. For
example: options { disableConcurrentBuilds() }

Thanks Jazzschmidt, I looking to lock all stages easily, this works for me (source)
pipeline {
agent any
options {
lock('shared_resource_lock')
}
...
...
}

I got the solution for multibranch locking too, with de lockable-resources plugin and the shared-libs here is :
Jenkinsfile :
#Library('my_pipeline_lib#master') _
myLockablePipeline()
myLockablePipeline.groovy :
call(Map config){
def jobIdentifier = env.JOB_NAME.tokenize('/') as String[];
def projectName = jobIdentifier[0];
def repoName = jobIdentifier[1];
def branchName = jobIdentifier[2];
//now you can use either part of the jobIdentifier to lock or
limit the concurrents build
//here I choose to lock any concurrent build for PR but you can choose all
if(branchName.startsWith("PR-")){
lock(projectName+"/"+repoName){
yourTruePipelineFromYourSharedLib(config);
}
}else{
// Others branches can freely concurrently build
yourTruePipelineFromYourSharedLib(config);
}
}
To lock for all branches just do in myLockablePipeline.groovy :
call(Map config){
def jobIdentifier = env.JOB_NAME.tokenize('/') as String[];
def projectName = jobIdentifier[0];
def repoName = jobIdentifier[1];
def branchName = jobIdentifier[2];
lock(projectName+"/"+repoName){
yourTruePipelineFromYourSharedLib(config);
}
}

Related

Executing multiple jenkins jobs

Can someone please let me know how i can trigger multiple Jenkins jobs? We have 5-6 jobs that we need to trigger manually post some technical upgrades. We are having to navigate to these jobs manually and click on 'Build'. Is there anyway i can create a new job or shell script that will help me in triggering all of these jobs with a single click/run.
Simplest approach will be creating a new pipeline which will trigger other pipelines.
You can also make stages parallel to faster execution if order is not issue.
You can find detailed answer and approach here
Yes, you can do it in both ways that you mentioned. Creating a new job that will run all these other jobs as downstream jobs is the fastest and easiest.
To do this, create a new job, and if you are using declarative Jenkins, then specify the jobs to be triggered:
stage ('trigger-multiple-jobs') {
parallel {
stage ('first job') {
steps {
build([
job : 'JobName',
wait : false,
parameters: [
string(name: 'PARAM_1', value: "${PARAM_1}")
]
])
}
}
stage ('second job') {
steps {
build([
job : 'JobName2',
wait : false
])
}
}
}
}
You can alternatively create a freestyle job, navigate down to the Trigger downstream jobs section, and set the jobs you want to be triggered from the drop down.
If you want to use a shell script, then you can trigger the jobs using API calls. This is described well in this answer.

Jenkins pipeline milestone not cancelling previous ongoing build

I am experimenting with Jenkins pipeline and milestones and cannot figure out why Jenkins is not cancelling the previous build when a new build crosses the milestone.
Example Jenkinsfile
pipeline {
agent any
parameters {
booleanParam(defaultValue: true, description: '', name: 'userFlag')
}
stages {
stage("foo") {
steps {
milestone(ordinal: 1, label: "BUILD_START_MILESTONE")
sh 'sleep 1000'
}
}
}
}
Triggering this pipeline twice does not cancel the 1st job
Try this:
/* This method should be added to your Jenkinsfile and called at the very beginning of the build*/
#NonCPS
def cancelPreviousBuilds() {
def jobName = env.JOB_NAME
def buildNumber = env.BUILD_NUMBER.toInteger()
/* Get job name */
def currentJob = Jenkins.instance.getItemByFullName(jobName)
/* Iterating over the builds for specific job */
for (def build : currentJob.builds) {
/* If there is a build that is currently running and it's not current build */
if (build.isBuilding() && build.number.toInteger() != buildNumber) {
/* Than stopping it */
build.doStop()
}
}
}
I don't think the behavior is "If I'm a newer build that crosses this milestone, then all older build that crossed this milestone will be cancelled"
The actual behavior of the milestone step is that when a more recent pipeline crosses it first, then it prevents older pipeline from crossing that milestone.
I have a simple work around with milestone plugin, according to the document:
Builds pass milestones in order (taking the build number as sorter field).
Older builds will not proceed (they are aborted) if a newer one already passed the milestone.
When a build passes a milestone, any older build that passed the previous milestone but not this one is aborted.
Once a build passes the milestone, it will never be aborted by a newer build that didn't pass the milestone yet.
you can try something like this:
pipeline {
agent any
stages {
stage('Stop Old Build') {
steps {
milestone label: '', ordinal: Integer.parseInt(env.BUILD_ID) - 1
milestone label: '', ordinal: Integer.parseInt(env.BUILD_ID)
}
}
}
}
you can put this at the start of any pipeline.
Assume you just start a new build, #5. The first milestone, will be used to passes #4's second milestone, and the second milestone(of #5) will be used to kill #4's process, if it's currently running.
The disableConcurrentBuilds property has been added to Pipeline. The Pipeline syntax snippet generator offers the following syntax hint:
properties([disableConcurrentBuilds(abortPrevious: true)])
That property is used on ci.jenkins.io to cancel older plugin build jobs when newer plugin build jobs start.
Declarative Pipeline also includes the disableConcurrentBuilds option that is documented in the Pipeline syntax page.
The declarative directive generator suggests the following:
options {
disableConcurrentBuilds abortPrevious: true
}
as per https://jenkins.io/blog/2016/10/16/stage-lock-milestone/, a pair of 'milestone()' works for me to kill the previous jobs while the pipeline kicked off for times,
stage('Build') {
// The first milestone step starts tracking concurrent build order
milestone()
node {
echo "Building"
}}
// The Deploy stage does not limit concurrency but requires manual input
// from a user. Several builds might reach this step waiting for input.
// When a user promotes a specific build all preceding builds are aborted,
// ensuring that the latest code is always deployed.
stage('Deploy') {
timeout(time: 60, unit: 'SECONDS') {input "Deploy?"}
milestone()
node {
echo "Deploying"
}
}
The last milestone helps kill previous builds if reached, say deploy button clicked for the above case. Or the locked resource released for the below case,
// This locked resource contains both Test stages as a single concurrency Unit.
// Only 1 concurrent build is allowed to utilize the test resources at a time.
// Newer builds are pulled off the queue first. When a build reaches the
// milestone at the end of the lock, all jobs started prior to the current
// build that are still waiting for the lock will be aborted
lock(resource: 'myResource', inversePrecedence: true){
node('test') {
stage('Unit Tests') {
echo "Unit Tests"
}
stage('System Tests') {
echo "System Tests"
}
}
milestone()
}
Building on #D.W.'s answer, i found a simple pattern that works. It seems to fit into D.W.'s bullet #3 (which is the official doc): When a build passes a milestone, Jenkins aborts older builds that passed the previous milestone but not this one.
Adding an earlier milestone that everything will pass, and then one after the thing that is going to wait, makes it all work like you think it should. In my case:
steps {
milestone 1
input 'ok'
milestone 2
}
Create two active builds with this, and only approve the second one. You'll see the first one get automatically canceled, because build 2 passed milestone 2 first.
Try taking out milestone 1, and you'll see that build 1 does not get canceled when build 2 passes milestone 2.
Adding the early milestone satisfies the requirement. It seems that a build has to pass any milestone before a future milestone passed by a newer build will cause it to cancel.

How to differentiate build triggers in Jenkins Pipeline

I'm hoping to add a conditional stage to my Jenkinsfile that runs depending on how the build was triggered. Currently we are set up such that builds are either triggered by:
changes to our git repo that are picked up on branch indexing
a user manually triggering the build using the 'build now' button in the UI.
Is there any way to run different pipeline steps depending on which of these actions triggered the build?
The following code should works to determine if a user has started the pipeline or a timer/other trigger:
def isStartedByUser = currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) != null
In Jenkins Pipeline without currentBuild.rawBuild access the build causes could be retrieved in the following way:
// started by commit
currentBuild.getBuildCauses('jenkins.branch.BranchEventCause')
// started by timer
currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')
// started by user
currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
You can get a boolean value with:
isTriggeredByTimer = !currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').isEmpty()
Or, as getBuildCauses() returns an array, the array's size will work correctly with Groovy truthy semantics:
if (currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')) {
The ability to get causes for a workflow run was released in version 2.22 (2018 Nov 02) to the Pipeline Supporting APIs Plugin. The feature was requested in JENKINS-41272.
A couple methods were added to the currentBuild global variable with that release:
getBuildCauses
Returns a JSON array of build causes for the current build
EXPERIMENTAL - MAY CHANGE getBuildCauses(String causeClass)
Takes a string representing the fully qualified Cause class and returns a JSON array of build causes filtered by that type for the current build, or an empty JSON array if no causes of the specified type apply to the current build
And an example from me submitting:
echo "${currentBuild.buildCauses}" // same as currentBuild.getBuildCauses()
echo "${currentBuild.getBuildCauses('hudson.model.Cause$UserCause')}"
echo "${currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')}"
And the output:
[Pipeline] echo
[[_class:hudson.model.Cause$UserIdCause, shortDescription:Started by user anonymous, userId:null, userName:anonymous], [_class:org.jenkinsci.plugins.workflow.cps.replay.ReplayCause, shortDescription:Replayed #12]]
[Pipeline] echo
[]
[Pipeline] echo
[]
[Pipeline] End of Pipeline
Finished: SUCCESS
NOTE
There appears to be an issue with the currentBuild.getBuildCauses(type) when the type is a type of Cause contributed by a plugin. For example, currentBuild.getBuildCauses('org.jenkinsci.plugins.workflow.cps.replay.ReplayCause') fails with a java.lang.ClassNotFoundException. This was reported in JENKINS-54673 for the 2.22 version of the Pipeline: Supporting APIs (workflow-support) plugin. It is reportedly fixed in the 2.24 version.
I might be missing something, but you can achieve what you want easily by making use of the when directive:
pipeline {
agent any
stages {
stage('Always') {
steps {
echo "I am always executed"
}
}
stage('ManualTimed') {
steps {
echo "I am only executed when triggered manually or timed"
}
when {
beforeAgent true
anyOf {
triggeredBy 'TimerTrigger'
triggeredBy cause: 'UserIdCause'
}
}
}
stage('GitLabWebHookCause') {
steps {
echo "I am only executed when triggered by SCM push"
}
when {
beforeAgent true
triggeredBy 'GitLabWebHookCause'
}
}
}
}
You will find many similar useful examples for various use cases in the documentation of the when directive.
Edit:
thanks to Jean-Francois Larvoire's answer, I was able to figure out 'my trigger' GitLabWebHookCause I required for my use case.
#vitalii-blagodir:
Your answer works for detecting builds triggered by users and timers, but not by commits.
Instead, I found this to work in my case:
def isTriggeredByIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
def isTriggeredByCommit = currentBuild.getBuildCauses('com.cloudbees.jenkins.GitHubPushCause').size()
def isTriggeredByUser = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').size()
def isTriggeredByTimer = currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').size()
The .size() suffix returns 0 if the object is missing, or 1 if it's present. This makes the result usable as a boolean.
For finding the object name to use, I found it convenient to display this in the log:
echo "# Build causes"
def buildCauses = currentBuild.buildCauses
def numCause = 0
for (cause in buildCauses) {
echo "${numCause++}: ${cause.shortDescription}" // Display a human-readable index and description
echo "${cause}" // Display the object class name. This allows knowing what names to use in getBuildCauses(name) calls below.
}
Finally, if the goal is to abort a pipeline build in specific cases, then the test must be done before the beginning of the pipeline.
For example, we had a problem with the branch indexing triggering extra useless builds. This was fixed by adding this before the pipeline:
// Avoid useless buils: The branch indexing should only trigger the initial build of a new branch.
def isTriggeredByBranchIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
if (isTriggeredByBranchIndexing && currentBuild.previousBuild) { // Then it's not the initial build.
echo "# Reindexing a branch already built. It is useless to rebuild it now. Aborting."
currentBuild.result = 'SUCCESS' // Make sure the build is not displayed in red in the Jenkins UI.
return // Abort before the pipeline even starts. (Inside the pipeline, this would only abort one stage.)
}
I think that the answers here are incomplete and do not provide an actual ready to use answer. Here's my code to get it working:
import com.cloudbees.groovy.cps.NonCPS
#NonCPS
def isStartedByTimer() {
def buildCauses = currentBuild.rawBuild.getCauses()
echo buildCauses
boolean isStartedByTimer = false
for (buildCause in buildCauses) {
if ("${buildCause}".contains("hudson.triggers.TimerTrigger\$TimerTriggerCause")) {
isStartedByTimer = true
}
}
echo isStartedByTimer
return isStartedByTimer
}
// [...]
// Other pipeline stuff
script {
isStartedByTimer()
}
When started by user:
00:00:01.353 [hudson.model.Cause$UserIdCause#fa5cb22a]
[Pipeline] echo
00:00:01.358 false
When started by timer:
00:00:01.585 [hudson.triggers.TimerTrigger$TimerTriggerCause#5]
[Pipeline] echo
00:00:01.590 true
Note: the NonCPS decorator is needed because otherwise the next non-script step will throw.
Assuming the two different build causes are "timer" and "push" (to a git repo), you can add the following stage to your Jenkinsfile (in a declarative Jenkins pipeline) to make use of getBuildCauses():
pipeline {
stages {
stage('preparation') {
steps {
script {
// get build cause (time triggered vs. SCM change)
def buildCause = currentBuild.getBuildCauses()[0].shortDescription
echo "Current build was caused by: ${buildCause}\n"
// e.g. "Current build was caused by: Started by GitHub push by mirekphd"
// vs. "Started by timer"
}
}
}
}
}
Then I can decide whether to perform certain stages conditionally (depending on the build cause). For example, pulling a docker base image and inspecting for changes in system libraries (likely security updates) should be done periodically, regardless of whether there was a source code change or not.
We can use "BUILD_CAUSE" variable for getting the information about who initiated the run
for [jenkins-pipeline] you may use
currentBuild.rawBuild.getCauses()
(see github.com/jenkinsci/pipeline-examples/blob/master/… for more details)
There was a similar requirement, where user detail who triggered the build should be there in success / failure notification. The job was already had time based triggered, hence could not use wrap([$class: 'BuildUser']) directly.
I used below step, which print username if the job is triggered manually or timer triggered. So, I used this:
pipeline {
agent any
stages {
stage('Test') {
steps {
script{
env.buildCauses = currentBuild.rawBuild.getCauses()
if (buildCauses.contains("hudson.triggers.TimerTrigger")){
env.builduser = "TimerTrigger"
} else {
wrap([$class: 'BuildUser']) {
env.builduser = "${BUILD_USER}"
}
}
}
echo "Initiated by: ${env.builduser}"
}
}
}
}

Set the pipeline name and description from Jenkinsfile

I am trying to do a poc of jenkins pipeline as code. I am using the Github organization folder plugin to scan Github orgs and create jobs per branch. Is there a way to explicitly define the names for the pipeline jobs that get from Jenkinsfile? I also want to add some descriptions for the jobs.
You need to use currentBuild like below. The node part is important
node {
currentBuild.displayName = "$yournamevariable-$another"
currentBuild.description = "$yourdescriptionvariable-$another"
}
Edit: Above one renames build where as Original question is about renaming jobs.
Following script in pipeline will do that(this requires appropriate permissions)
item = Jenkins.instance.getItemByFullName("originalJobName")
item.setDescription("This description was changed by script")
item.save()
item.renameTo("newJobName")
I'm late to the party on this one, but this question forced me in the #jenkins chat where I spent most of my day today. I would like to thank #tang^ from that chat for helping solve this in a graceful way for my situation.
To set the JOB description and JOB display name for a child in a multi-branch DECLARATIVE pipeline use the following steps block in a stage:
steps {
script {
if(currentBuild.rawBuild.project.displayName != 'jobName') {
currentBuild.rawBuild.project.description = 'NEW JOB DESCRIPTION'
currentBuild.rawBuild.project.setDisplayName('NEW JOB DISPLAY NAME')
}
else {
echo 'Name change not required'
}
}
}
This will require that you approve the individual script calls through the Jenkins sandbox approval method, but it was far simpler than anything else I'd found across the web about renaming the actual children of the parent pipeline. The last thing to note is that this should work in a Jenkinsfile where you can use the environment variables to manipulate the job items being set.
I tried to used code snippet from accepted answer to describe my Jenkins pipeline in Jenkinsfile. I had to wrap code snippet into function with #NonCPS annotation and use def for item variable. I have placed code snippet in root of Jenkinsfile, not in node section.
#NonCPS
def setDescription() {
def item = Jenkins.instance.getItemByFullName(env.JOB_NAME)
item.setDescription("Some description.")
item.save()
}
setDescription()

Run Parts of a Pipeline as Separate Job

We're considering using the Jenkins Pipeline plugin for a rather complex project consisting of several deliveries that need to be build using different tools (on different machines) before being merged. Still, it seems to be easy enough to do a complete build with a single Jenkinsfile, and I like the automatic discovery of git branches that comes with Pipeline.
However, at this point, we have jobs for each of the deliveries and use a build-flow based "meta" job to orchestrate the individual jobs. The nice thing about this is that it also allows starting just one individual job if only small changes were made, just to see whether this delivery still compiles.
To emulate this, some ideas came to mind:
Use different Jenkinsfiles for the deliveries and load them in the top-level Jenkinsfile; it seems that the Multibranch Pipeline job does not allow configuring the Jenkinsfile to use yet (https://issues.jenkins-ci.org/browse/JENKINS-35415), however, so creating the jobs for the individual deliveries is still open.
Provide a configuration option for the "top-level" job and have ifs for all deliveries in the Jenkinsfile to be able to select which should be build. This would mix different build types in one pipeline, though, and, at the very least, mess up the estimation of the build time.
Are those viable options, or is there a better one?
What you could do is to write a pipelining script that has has "if"-guards around the single stages, like this:
stage "s1"
if (theStage in ["s1","all"]) {
sleep 2
}
stage "s2"
if (theStage in ["s2", "all"]) {
sleep 2
}
stage "s3"
if (theStage in ["s3", "all"]) {
sleep 2
}
Then you can make a "main" job that uses this script and runs all stages at once by setting the parameter "theStage" to "all". This job will collect the statistics when all stages are run at once and give you useful estimation times.
Furthermore, you can make a "partial run" job that uses this script and that is parametrized with the stage that you want to run. The estimation will not be very useful, though.
Note that I put the stage itself to the main script and put only the execution code into the conditional, as suggested by Martin Ba. This makes sure that the visualization of the job is more reliable
As an expansion of the previous answer, I would propose something like that:
def stageIf(String name, Closure body) {
if (params.firstStage <= name && params.lastStage >= name) {
stage(name, body)
} else {
stage(name) {
echo "Stage skipped: $name"
}
}
}
node('linux') {
properties([
parameters([
choiceParam(
name: 'firstStage',
choices: '1.Build\n' +
'2.Docker\n' +
'3.Deploy',
description: 'First stage to start',
defaultValue: '1.Build',
),
choiceParam(
name: 'lastStage',
choices: '3.Deploy\n' +
'2.Docker\n' +
'1.Build',
description: 'Last stage to start',
defaultValue: '3.Deploy',
),
])
])
stageIf('1.Build') {
// ...
}
stageIf('3.Deploy') {
// ...
}
}
Not as perfect as I wish but at least its working.

Resources