We currently generate a lot of Jenkins jobs on a per Git branch basis using Jenkins job DSL; the multi-branch pipeline plugin looks like an interesting way to potentially get first-class job generation support using Jenkinsfiles and reduce the amount of Job DSL we maintain.
For example we have libwidget-server and widget-server develop branch projects. When the libwidget-server build finishes then the widget-server job is triggered (for the develop branch). This applies to other branches too.
This makes use of the Build after other projects are built to trigger upon completion of an upstream build (e.g. libwidget-server causes widget-server to be built).
It seems that the multi-branch pipeline plugin lacks the Build after other projects are built setting - how would we accomplish the above in the multi-branch pipeline build?
You should add the branch name to your upstream job (assuming you are using a multi-branch pipeline for the upstream job too).
Suppose you have a folder with two jobs, both multi-branch pipeline jobs: jobA and jobB; jobB should trigger after jobA's master.
You can add this code snippet to jobB's Jenkinsfile:
properties([
pipelineTriggers([
upstream(
threshold: 'SUCCESS',
upstreamProjects: '../jobA/master'
)
])
])
(Mind that any branch of jobB here will trigger after jobA's master!)
I'm currently trying to get this to work for our deployment.
The closest I've got is adding the following to the downstream Jenkinsfile;
properties([
pipelineTriggers([
triggers: [
[
$class: 'jenkins.triggers.ReverseBuildTrigger',
upstreamProjects: "some_project", result: hudson.model.Result.SUCCESS
]
]
]),
])
That at least gets Jenkins to acknowledge that it should be triggering when
'some_project' get's built i.e it appears in the "View Configuration" page.
However so far builds of 'some_project' still don't trigger the downstream
project as expected.
That being said maybe you'll have more luck.
Let me know if it works for you.
(Someone else has asked a similar question here -> Jenkins: Trigger Multi-branch pipeline on upstream change )
Related
I have a Jenkinsfile setup for our CI/CD pipeline, and it runs through the pipeline on git actions like Pull Requests, Branch Creation, Tag Pushes, etc..
Prior to this setup, I was used to setting up Jenkins build jobs in the Jenkins UI. The advantage of this, was that I could setup dedicated build jobs that I could trigger remotely, and independently of git webhook actions. I could do a POST to the job endpoint with parameters to trigger various actions.
Documentation for this process would be referenced here - see "Trigger Builds Remotely"
I could also hit the big button that says "Build", or "Build with Parameters" in the UI, which was super nice.
How would one do this with a Jenkinsfile? Is this even possible to define build jobs in a pipeline definition within a Jenkinsfile? I.E. define functions / build jobs that have dedicated URLs that could be called on the Jenkins URL independent of webhook callbacks?
What's the best practice here?
Thanks for any tips, references, suggestions!
I would recommend starting with Multibranch pipelines. In general you get all the things you mentioned, but a little better. Because thhe paramteres can be defined within your Jenkinsfile. In short just do it like this:
Create a Jenkinsfile an check this into a Git Repository.
To create a Multibranch Pipeline: Click New Item on Jenkins home page.
Enter a name for your Pipeline, select Multibranch Pipeline and click OK.
Add a Branch Source (for example, Git) and enter the location of the repository.
Save the Multibranch Pipeline project.
A declarative Jenkinsfile can look like this:
pipeline {
agent any
parameters {
string(name: 'Greeting', defaultValue: 'Hello', description: 'How should I greet the world?')
}
stages {
stage('Example') {
steps {
echo "${params.Greeting} World!"
}
}
}
}
A good tutorial with screenshhots can be found here: https://www.jenkins.io/doc/book/pipeline/multibranch/
I have a multi-branch pipeline job which does a build/test on a specific branch.
I would like to create a new Jenkins job, which 'loads' a new branch into the multi-branch pipeline, runs the pipeline 3 times with 3 different options then cleans up and removes the branch.
Here is my use case example:
Set up - add branch release-19.0.4 branch to multi-branch pipeline
Run multi-branch pipeline on branch release-19.0.4 with parameter support-version=19.0.3
Run multi-branch pipeline on branch release-19.0.4 with parameter support-version=19.0.0
Run multi-branch pipeline on branch release-19.0.4 with parameter support-version=18.0.0
Write report: support-version=19.0.3 - ok, support-version=19.0.0 - ok, support-version=18.0.0 - failed
Tear down - remove release-19.0.4 branch from mule-branch pipeline job.
I can use build pipeline task to build another job with different options and read the success or failure of that job in order to create my report. However I'm stuck on how to do steps 1) and 6) from within a Jenkinsfile.
I've tried to read through the Jenkins pipeline API's but nothing seems to fit. Any ideas how to do this via automation?
In a Jenkinsfile, to start a parameterized pipeline job from another job, I have this code snippet:
build job: 'build-sharpen-branch', parameters: [
[$class: 'StringParameterValue', name: 'BRANCHNAME', value: mergeBranchname]
]
This already works as expected, and it will start a job at URL https://$JENKINS_URL/job/build-sharpen-branch/.
Now I want to start a job, that is one branch of a multibranch project inside a Bitbucket folder. The URL of the job is https://$JENKINS_URL/job/iText%207%20.NET/job/sharpen/job/feature%2FQA-10738/.
iText%207%20.NET is the name of the Bitbucket project.
sharpen is the name of the Multibranch job.
feature%2FQA-10738 is the name of the branch, urlencoded.
I read the following questions about starting a multibranch job NOT inside a folder:
Trigger Multibranch Job from another
Triggering a multibranch pipeline job from an other multibranch pipeline
How to trigger Multibranch Pipeline Jenkins Job within regular pipeline job?
From the answers there, I gather that the syntax is $JOB/$BRANCH (where $BRANCH is URL-encoded to rename branches like feature/foo to feature%2Ffoo).
From Jenkins pipeline with folder plugin. How to build a job located different folder I gather that the syntax for a job inside a folder is $FOLDER/$JOB.
Combining the two, I conclude that the syntax for folder+job+branch is most likely $FOLDER/$JOB/$BRANCH.
So I tried with this code:
build job: "iText%207%20.NET/sharpen/${java.net.URLEncoder.encode branchName, 'UTF-8'}"
with
folder = iText%207%20.NET
job = sharpen
branch = ${java.net.URLEncoder.encode branchName, 'UTF-8'} (URLEncoder to change / in the branch name to %2F)
To my surprise, when I ran this, I got an error:
ERROR: No item named iText%207%20.NET/sharpen/feature%2FQA-10738 found
As already stated above, a job exists on URL https://$JENKINS_URL/job/iText%207%20.NET/job/sharpen/job/feature%2FQA-10738/.
What is the correct syntax for a multibranch job inside a Bitbucket folder?
The problem was with the folder name iText%207%20.NET, which is an urlencoded version of iText 7 .NET. Apparently Jenkins can't handle urlencoded spaces in folder names.
I renamed the folder to itext_7_dotnet and then used
build job: "itext_7_dotnet/sharpen/${java.net.URLEncoder.encode branchName, 'UTF-8'}"
This works.
Moral of the story: never use spaces in your folder names!
For us, this works:
build job: "${folder}/${repo}/${branch.replace('/','%2F')}", wait: false, propagate: false
You can also trigger build by using curl, with crumbs and all that.
Consider the following setup using Jenkins 2.176.1:
A new pipeline project named Foobar
Poll SCM as (only) build trigger, with: H/5 * * * * ... under the assumption that this refers to the SCM configured in the next step
Pipeline script from SCM with SCM Git and a working Git repository URL
Uncheck Lightweight checkout because of JENKINS-42971 and JENKINS-48431 (I am using build variables in the real project and Jenkinsfile; also this may affect how pollSCM works, so I include this step here)
Said repository contains a simple Jenkinsfile
The Jenkinsfile looks approximately like this:
#!groovy
pipeline {
agent any
triggers { pollSCM 'H/5 * * * *' }
stages {
stage('Source checkout') {
steps {
checkout(
[
$class: 'GitSCM',
branches: [],
browser: [],
doGenerateSubmoduleConfigurations: false,
extensions: [],
submoduleCfg: [],
userRemoteConfigs: [
[
url: 'git://server/project.git'
]
]
]
)
stash 'source'
}
}
stage('OS-specific binaries') {
parallel {
stage('Linux') {
agent { label 'gcc && linux' }
steps {
unstash 'source'
echo 'Pretending to do a build here'
}
}
stage('Windows') {
agent { label 'windows' }
steps {
unstash 'source'
echo 'Pretending to do a build here'
}
}
}
}
}
}
My understanding so far was that:
a change to the Jenkinsfile (not the whole repo) triggers the pipeline on any registered agent (or as configured in the pipeline project).
said agent (which is random) uses the pollSCM trigger in the Jenkinsfile to trigger the pipeline stages.
But where does the pollSCM trigger poll (what SCM repo)? And if it's a random agent then how can it reasonably detect changes across poll runs?
then the stages are being executed on the agents as allocated ...
Now I am confused what refers to what. So here my questions (all interrelated which is why I keep it together in one question):
The pipeline project polls the SCM just for the Jenkinsfile or for any changes? The repository in my case is the same (for Jenkinsfile and source files to build binaries from).
If the (project-level) polling triggers at any change rather than changes to the Jenkinsfile
Does the pollSCM trigger in the Jenkinsfile somehow automagically refer to the checkout step?
Then ... what would happen, would I have multiple checkout steps with differing settings?
What determines what repository (and what contents inside of that) gets polled?
... or is this akin to the checkout scm shorthand and pollSCM actually refers to the SCM configured in the pipeline project and so I can shorten the checkout() to checkout scm in the steps?
Unfortunately the user handbook didn't answer any of those questions and pollSCM has a total of four occurrences on a single page within the entire handbook.
I'll take a crack at this one:
The pipeline project polls the SCM just for the Jenkinsfile or for any
changes? The repository in my case is the same (for Jenkinsfile and
source files to build binaries from).
The pipeline project will poll the repo for ANY file changes, not just the Jenkinsfile. A Jenkinsfile in the source repo is common practice.
If the (project-level) polling triggers at any change rather than
changes to the Jenkinsfile Does the pollSCM trigger in the Jenkinsfile
somehow automagically refer to the checkout step?
Your pipeline will be executed when a change to the repo is seen, and the steps are run in the order that they appear in your Jenkinsfile.
Then ... what would happen, would I have multiple checkout steps with
differing settings?
If you defined multiple repos with the checkout step (using multiple checkout SCM calls) then the main pipeline project repo would be polled for any changes and the repos you define in the pipeline would be checked out regardless of whether they changed or not.
What determines what repository (and what contents inside of that)
gets polled? ... or is this akin to the checkout scm shorthand and
pollSCM actually refers to the SCM configured in the pipeline project
and so I can shorten the checkout() to checkout scm in the steps?
pollSCM refers to the pipeline project's repo. The entire repo is cloned unless the project is otherwise configured (shallow clone, lightweight checkout, etc.).
The trigger defined as pollSCM polls the source-control-management (SCM), at the repository and branch in which this jenkinsfile itself (and other code) is located.
For Pipelines which are integrated with a source such as GitHub or BitBucket, triggers may not be necessary as webhooks-based integration will likely already be present. The triggers currently available are cron, pollSCM and upstream.
It works for a multibranch-pipeline as trigger to execute the pipeline.
When Jenkins polls the SCM, exactly this repository and branch, and detects a change (i.e. new commit), then this Pipeline (defined in jenkinsfile) is executed.
Usually then the following SCM Step checkout will be executed, so that the specified project(s) can be built, tested and deployed.
See also:
SCM Poll in jenkins multibranch pipeline
SehllHacks(2020): Jenkins: Scan Multibranch Pipeline Without Build
I have a scenario where but I have 2 projects (A and B), both are configured in Jenkins with multibranch pipeline jobs, problem is that Project B depends on Project A.
So I find that some times when I check in code in Project A, I also need to build ProjectB once A was built. Now before I started investigating pipeline builds, I'd have a job per branch and then trigger in Jenkins the appropriate job for Project B for the appropriate branch.
What I'd like to set up in a Jenkinsfile so that when ProjectA/develop executes it then triggers the multibranch pipeline job for ProjectB and the same branch.
I have:
stage ('Trigger Tenant Builds') {
build job: "ProjectB/${branch}", wait: false
}
But my ProjectA pipeline fails with:
ERROR: No parameterized job named ProjectB/develop found
Any ideas?
I've resolved this now. What I am doing is defining an upstream trigger in project B's Jenkinsfile:
pipelineTriggers([
upstream(
threshold: hudson.model.Result.SUCCESS,
upstreamProjects: "/ProjectA/" + env.BRANCH_NAME.replaceAll("/", "%2F")
)
])