I am trying to create Multibranch Pipeline Jobs using Job DSL, but I want to disable concurrent builds on each branch. I have tried the following code snippet but it didn't work, "Do not allow concurrent builds" is still unchecked on new branches.
multibranchPipelineJob("${FOLDER_NAME}/${JOB_NAME}") {
branchSources {
git {
remote("https://gitlab.com/${REPO_PATH}")
credentialsId('gitlab_credentials')
includes('*')
}
}
configure {
def factory = it / factory(class: 'com.cloudbees.workflow.multibranch.CustomBranchProjectFactory')
factory << disableConcurrentBuilds()
}
orphanedItemStrategy {
discardOldItems {
numToKeep(1)
}
}
}
I also tried this in configure closure:
factory << properties {
disableConcurrentBuilds()
}
But this one caused following exception to be thrown:
19:03:50 groovy.lang.GroovyRuntimeException: Ambiguous method overloading for method groovy.util.Node#leftShift.
19:03:50 Cannot resolve which method to invoke for [null] due to overlapping prototypes between:
19:03:50 [class groovy.util.Node]
19:03:50 [class java.lang.String]
I have this need as well. I notice that in my jenkins instance the jobDSL api docs indicate that disableConcurrentBuilds() property is NOT supported in multibranch pipeline jobs.
I just returned to a related discussion I was having with #tknerr in which he pointed out that there IS a rate limiting feature available to multibranch pipelines via jobDSL.
My team just ran into a problem with pollSCM triggering running amok due to this Jenkins bug, and so I'm implementing this in jobDSL to make our jobs more robust to this. Like you, I had wanted to just "disableConcurrentBuilds" like can be done in pipelines, but since rate limiting appears to be the only solution currently available to multibranch pipelines, I experimented with putting this in our jobDSLs:
strategy {
defaultBranchPropertyStrategy {
props {
rateLimitBranchProperty {
count(2)
durationName("hour")
}
}
}
}
This is of course a horrible workaround, since it places a nasty dependency in the jobDSL of needing to know how long builds take, but i'm willing to accept this alternative to having to push disableConcurrentBuilds option to the Jenkinsfile on hundreds of branches.
It is also barely even effective at achieving the goal, since we want to allow concurrent builds across branches, but want to prevent individual branch jobs from being built "too fast".
We should check if there is a feature request in Jenkins for this (your original request).
In my jenkins instance (v2.222.3, Pipeline:multibranch v2.22), the setting is described here for applying it to "all branches":
https://<my_jenkins_fqdn>/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource-strategy-allBranchesSame-props-rateLimit
and here for applying it to specific branches:
https://<my_jenkins_fqdn>/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource-strategy-namedBranchesDifferent-defaultProperties-rateLimit
EDIT: Also wanted to link to a related Jenkins issue here.
Related
I am developing software for an embedded device. The steps involved in building and verifying it all are complicated: creating the build environment (via containers), building the actual SD card image, running unit tests, automated tests on target hardware, license compliance checks and so on - details aren't important here.
Currently I have this in one long declarative Jenkinsfile as a multibranch-pipeline (for all intents and purpose here, we're doing gitflow). In doing this I've hit a limit on the size of a Jenkinsfile (https://issues.jenkins.io/browse/JENKINS-37984) and can't actually get all the stages in that I want to.
It's too big so i need to cut this massive pipeline up. I broke this all up in little pipeline jobs with parameters to pass data/context between each part of the pipeline and came up with something like this:
I've colour-coded the A and B artifacts as they're used a lot and the lines would make things messy. What this tries to show is an order of running things, where things in a column depend on artifacts created in column to the left.
I'm struggling to discover how to do the "waiting" for multiple upstream jobs (for instance in Job Foxtrot in the diagram) before starting another downstream job that depends on them.
I specifically do not want to turn each column in the diagram into a parallel group of things, because for instance Job Delta might take 2 minutes but Job Charlie take 20 minutes. The exact duration of each job is variable and unpredictable as for some parameter combinations will mean building from scratch and others will cause an existing artifact to be output.
I think I need something like the join plugin (https://plugins.jenkins.io/join/), but for pipeline jobs (join only works on freestyle jobs and is quite aged).
The one approach I've explored is to have a "controller" job (maybe job Alpha in the diagram?) that uses the build step (https://www.jenkins.io/doc/pipeline/steps/pipeline-build-step/_) with the wait parameter set to false to trigger the downstream jobs in correct order, with the correct parameters. It would involve searching Jenkins.instance.getItems() to locate the Runs for the downstream projects, which have an upstream cause that matches the currently executing "controller" job. This involves polling waiting for the job to appear and then polling for the job to complete. This feels like I'm "doing it wrong". Below is the source for this polling approach - be gentle, i'm new to groovy!
Is this polling approach a good way? What problems could I encounter with this approach? Should I be using the ItemListener Jenkins ExtensionPoint and writing a plugin to do this sort of thing in a generic way? Is there another way I've not found?
I feel like I'm not "holding it right" when it comes to the overall pipeline design/architecture here.
Finally after writing this I notice that Jobs India, Juliet and Kilo could be collapsed into a single Job, but I don't think that solve much.
#NonCPS
Integer getTriggeredBuildNumber(String project, String causeJobName, Integer causeBuildNumber) {
//find the job/project first
def job = Jenkins.instance.getAllItems(org.jenkinsci.plugins.workflow.job.WorkflowJob.class).find { job -> job.getFullName() == project }
//find a build for this job that was caused by the current build
def build = job.getBuilds().find { build ->
build.getCauses().findAll{ it.class == hudson.model.Cause.UpstreamCause.class }.find { cause ->
cause.getUpstreamProject() == causeJobName && cause.getUpstreamBuild() == causeBuildNumber
} != null
}
if(build != null) {
return build.getNumber()
} else {
return -1
}
}
#NonCPS
Boolean isBuildComplete(String jobName, Integer buildNumber) {
def job = Jenkins.instance.getAllItems(org.jenkinsci.plugins.workflow.job.WorkflowJob.class).find { job -> job.getFullName() == jobName }
if(job) {
def build = job.getBuildByNumber(buildNumber)
return build.isBuilding() == false && build.getResult() != null
} else {
println "WARNING: job '" + jobName + "' not found."
return false
}
}
We've hit the "Code too large" too many times, but the way to cope with it is to refactor your pipeline to remain deep under the limit. The following may be used:
You can run a combination of scripted and declarative pipeline. So some stages in the beginning and/or in the end may be refactored out.
You can build some of the parallel stages dynamically. This code would not be counted towards the limited code size.
Lastly, the issue mentions transformation variables, and that can help too.
We used the combination of the above and have expanded our pipeline well beyond what it was when we first encountered the issue you're facing.
You can use propagate on a build job as described here:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
So you can use something like this to prevent a failing step from failing the complete build:
build(job: 'example-job', propagate: false)
Is there a way to use this for a stage or a step? I know i can surround it with a try/catch and that does works almost as i want. It does ignore the failing the stage and resumes the rest of the build, but it does not display the stage as failed. For now i write all failing stages to a variable and output that on a later stage, but this is not ideal.
If i cant suppress propagation in a stage/step, is there maybe a way to use the build() call to do the same? Maybe if i move it to another pipeline and call that via build()?
Any help appreciated.
With catchError you can prevent a failing step from failing the complete build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this is a fairly new feature.
There are currently lots of suggestions for the scripted syntax, but for the declarative syntax work is in progress to support this.
Track the progress of https://issues.jenkins-ci.org/browse/JENKINS-26522 which groups all of the pieces together to achieve this. It has some interesting bits already marked as 'Resolved' (meaning a code change was made), such as https://issues.jenkins-ci.org/browse/JENKINS-49764 ( "Allow define a custom status for pipeline stage"). Unfortunately, I cannot find references to any of the tickets involved in the Jenkins changelog which would make sense since the parent ticket is not yet finished.
Of interest might also be the following : https://issues.jenkins-ci.org/browse/JENKINS-45579 which was reopened due to an issue. The environment for this is :
Admittedly, there are a confusing number of tickets tracking this work, but that is probably due to the fact that the functionality being implemented has a number of use-cases.
Another interesting ticket is "Individual Pipeline steps and stages/blocks should have Result statuses" , for which I was able to find a related PR: https://github.com/jenkinsci/workflow-api-plugin/pull/63
It is worth noting that the declarative pipeline was always designed as being opinionated and as such was not meant to support everything possible with the scripted syntax. For more complicated workflows and use-cases where it does not serve your needs, scripted syntax may be the only (and recommended?) option.
For needs such as the one you've stated, if enough noise is made, the declarative pipeline will probably be modified in due course to support it.
Trying to create multijobs in Jenkins with DSL scripting.
There are multiple jobs in a phase and I want to create a consolidated report for the multijob from downstream jobs.
I am using copy artifact to copy the results of downstream jobs to the multijob's target dir. Using selector - lastCompleted()
However I am getting this an error saying multiple extensions providing the method and tests are failing. lastCompleted() is apparently present in copyArtifact and multijob plugins where in this case I require both.
Here is my script:
multiJob('dailyMultiJob') {
concurrentBuild(true)
logRotator(-1, 10, -1, 10)
triggers {
cron('H H(0-4) * * 0-6')
}
steps {
phase('Smoke Tests'){
phaseJob('JobA')
phaseJob('JobB')
phaseJob('JobC')
}
copyArtifacts{
selector{
lastCompleted()
}
projectName('JobA')
filter('target/allure-results/*.*')
target('/path/to/this/multijob/workspace')
flatten(false)
}
copyArtifacts{
selector{
lastCompleted()
}
projectName('JobB')
filter('target/allure-results/*.*')
target('/path/to/this/multijob/workspace')
flatten(false)
}
copyArtifacts{
selector{
lastCompleted()
}
projectName('JobC')
filter('target/allure-results/*.*')
target('/path/to/this/multijob/workspace')
flatten(false)
}
}
publishers {
allure {
results {
resultsConfig {
path('target/allure-results')
}
}
}
archiveArtifacts {
pattern('target/reports/**/*.*')
pattern('target/allure-results/**/*.*')
allowEmpty(true)
}
}
}
Getting this below error after running gradle tests
Caused by: javaposse.jobdsl.dsl.DslException: Found multiple extensions which provide method lastCompleted with arguments []: [[hudson.plugins.copyartifact.LastCompletedBuildSelector, com.tikal.jenkins.plugins.multijob.MultiJobBuildSelector]]
I am not sure if there is a way to indicate use specific artifact's method.
Been stuck on this for quite some time. Any helps are highly appreciated. Thank you in advance!
I had come across the same issue few months back.
There are two possible solutions to this issue.
1 - Keep only one plugin that will avoid the conflict. (Not recommended as it might break other jobs)
2- Use configure block to modify the xml file which will avoid this conflict & you can keep multiple plugins that support the same extensions. (Recommended solution)
Thanks,
Late update:
What I had to do is to switch to scripted pipeline jobs instead.
Configure blocks are not really allowed on all the methods you want to use and they are limited by design. I believe some plugins also don't allow it for security reasons.
Better do use Pipelines.
Problem
Our source is a single large repository that contains multiple projects. We need to be able to avoid building all projects within the repository if a commit happens within specific areas. We are managing our build process using pipelines.
Research
The git plugin provides the ability to ignore commits from certain user, paths, and message content. However, as we are using the pipeline, we believe we are experiencing the issue described by JENKINS-36195. In one of the most recent comments, Jesse suggests examining the changeset and returning early if the changes look boring. He mentions that a return statement does not work inside a library, closure, etc), but he doesn't mention how a job could be aborted.
Potential Approaches
We have considered using the error step, but this would result in the job being marked as having a failure and would need to be investigated.
While a job result could be marked as NOT_BUILT, the job is not aborted but continues to process all stages.
Question
How would you abort a job during an early step without marking it as a failure and processing all stages of the pipeline (and potentially additional pipelines)?
Can you try using try catch finally block in the pipeline.
try{
}
catch {
}
finally{
}
I suppose you can also use post build action in the pipeline.
pipeline {
agent any
stages {
stage('Test') {
steps {
sh 'make check'
}
}
}
post {
always {
junit '**/target/*.xml'
}
failure {
mail to: team#example.com, subject: 'The Pipeline failed :('
}
}
}
The documentation is below https://jenkins.io/doc/book/pipeline/syntax/#post
you can also try using the below one outside of build step with your conditions as specified by Slawomir Demichowicz in the ticket.
if (stash.isJenkinsLastAuthor() && !params.SKIP_CHANGELOG_CHECK) {
echo "No more new changes, stopping pipeline"
currentBuild.result = "SUCCESS"
return
}
I am not sure this could help you.
The Jenkins Pipeline plugin (aka Workflow) can be extended with other Multibranch plugins to build branches and pull requests automatically.
What would be the preferred way to run multiple configurations? For example, building with Java 7 and Java 8. This is often called matrix configuration (because of the multiple combinations such as language version, framework version, ...) or build variants.
I tried:
executing them serially as separate stage steps. Good, but takes more time than necessary.
executing them inside a parallel step, with or without nodes allocated inside them. Works but I cannot use the stage step inside parallel for known limitations on how it would be visualized.
Is there a recommended way to do this?
TLDR: Jenkins.io wants you to use nodes for each build.
Jenkins.io: In pipeline coding contexts, a "node" is a step that does two things, typically by enlisting help from available executors on agents:
Schedules the steps contained within it to run by adding them to the Jenkins build queue (so that as soon as an executor slot is free on a node, the appropriate steps run)
It is a best practice to do all material work, such as building or running shell scripts, within nodes, because node blocks in a stage tell Jenkins that the steps within them are resource-intensive enough to be scheduled, request help from the agent pool, and lock a workspace only as long as they need it.
Vanilla Jenkins Node blocks within a stage would look like:
stage 'build' {
node('java7-build'){ ... }
node('java8-build'){ ... }
}
Further extending this notion Cloudbees writes about parallelism and distributed builds with Jenkins. Cloudbees workflow for you might look like:
stage 'build' {
parallel 'java7-build':{
node('mvn-java7'){ ... }
}, 'java8-build':{
node('mvn-java8'){ ... }
}
}
Your requirements of visualizing the different builds in the pipeline would could be satisfied with either workflow, but I trust the Jenkins documentation for best practice.
EDIT
To address the visualization #Stephen would like to see, He's right - it doesn't work! The issue has been raised with Jenkins and is documented here, the resolution of involving the use of 'labelled blocks' is still in progress :-(
Q: Is there documentation letting pipeline users not to put stages inside of parallel steps?
A: No, and this is considered to be an incorrect usage if it is done; stages are only valid as top-level constructs in the pipeline, which is why the notion of labelled blocks as a separate construct has come to be ... And by that, I mean remove stages from parallel steps within my pipeline.
If you try to use a stage in a parallel job, you're going to have a bad time.
ERROR: The ‘stage’ step must not be used inside a ‘parallel’ block.
I would suggest Declarative Matrix as a preferred way to run multiple configurations in Jenkins. It allows you to execute the defined stages for every configuration without code duplication.
Example:
pipeline {
agent none
stages {
stage('Test') {
matrix {
agent {
label "${NODENAME}"
}
axes {
axis {
name 'NODENAME'
values 'java7node', 'java8node'
}
}
stages {
stage('Test') {
steps {
echo "Do Test for ${NODENAME}"
}
}
}
}
}
}
}
Note that declarative Matrix is a native declarative Pipeline feature, so no additional Plugin installation needed.
Jenkins blog post about the matrix directive.
As noted by #StephenKing, Blue Ocean will show parallel branches better than the current stage view. A planned upcoming version of the stage view will be able to show all the branches, though it will not visually indicate any nesting structure (would look the same as if you ran the configurations serially).
In any event, the deeper issue is that you will essentially only get a pass/fail status for the build overall, pending a resolution to JENKINS-27395 and related requests.
In order to test each commit on several platforms, I've used this base Jenkinsfile skeleton:
def test_platform(label, with_stages = false)
{
node(label)
{
// Checkout
if (with_stages) stage label + ' Checkout'
...
// Build
if (with_stages) stage label + ' Build'
...
// Tests
if (with_stages) stage label + ' Tests'
...
}
}
/*
parallel ( failFast: false,
Windows: { test_platform("Windows") },
Linux: { test_platform("Linux") },
Mac: { test_platform("Mac") },
)
*/
test_platform("Windows", true)
test_platform("Mac", true)
test_platform("Linux", true)
With this it's relatively easy to switch from a sequential to a parallel execution, each of them having their pros and cons:
Parallel execution runs much faster, but it doesn't contain the stages labelling
Sequential execution is much slower, but you get a detailed report thanks to stages, labelled as "Windows Checkout", "Windows Build", "Windows Tests", "Mac Checkout", etc.)
I'm using the sequential execution for the time being, until I find a better solution.
It seems like there is relief coming at least with the BlueOcean UI. Here is what I got (the tk-* nodes are the parallel steps):