I've set up a number of Multibranch Pipeline jobs in Jenkins (running 2.46.2 LTS, Branch API 2.0.8, GitHub Branch Source 2.0.5, and Pipeline Multibranch 2.14) and have just noted that branch indexing -- and thus any cleanup of old branches -- does not appear to be triggered by the webhook calls from GitHub. It only appears to be triggered if someone manually clicks the "Scan Repository Now" link, or if the job configuration in Jenkins is re-saved. I'm using the timestamp shown in the "Scan Repository Log" page as an indication of when the branch indexing occurs.
It seems that new branches or changes to existing ones are being detected correctly and built, so the webhooks from source control (GitHub) are working, but was surprised that this wasn't also triggering the branch indexing and thus the old branch cleanup. I just can't tell from the documentation whether this is correct and expected behavior or if something is incorrect in my setup.
I note that the help text for the "Periodically if not otherwise run" setting says:
Some kinds of folders are reindexed automatically and immediately upon receipt of an external event. For example, a multi-branch project will recheck its SCM repository for new or removed or modified branches when it receives an SCM change notification. (Push notification may be configured as per the SCM plugin used for each respective branch source.) Such notifications can occasionally be unreliable, however, or Jenkins might not even be running to receive them. In some cases no immediate notification is even possible, for example because Jenkins is behind a firewall and can only poll an external system.
This trigger allows for a periodic fallback, but when necessary. If no indexing has been performed in the specified interval, then an indexing will be scheduled. For example, in the case of a multi-branch project, if the source control system is not configured for push notification, set a short interval (most people will pick between 15 minutes and 1 hour). If the source control system is configured for push notification, set an interval that corresponds to the maximum acceptable delay in the event of a lost push notification as the last commit of the day. (Subsequent commits should trigger indexing anyway and result in the commit being picked up, so most people will pick between 4 hours and 1 day.)
This certainly implies that indexing of a Multibranch Pipeline job should be re-triggered by branch events (e.g., pushes from GitHub via webhook), but the timestamp on my indexing log seems to belie that.
So, is what I'm observing the intended behavior? If so, and I want a regular cleanup of old branches, do I need to select the "Periodically if not otherwise run" checkbox under "Scan repository triggers"? Or is there something wrong with my setup, which is preventing it from working as intended?
According to the official documentation:
By default, Jenkins will not automatically re-index the repository for branch additions or deletions (unless using an Organization Folder), so it is often useful to configure a Multibranch Pipeline to periodically re-index in the configuration.
I depend on "Periodically if not otherwise run" for 1) cleanup of branches and 2) creation of container jobs for brand new repos (i use "Bitbucket Team/Project", the bitbucket version of "Github Organization", which basically creates a multibranch pipeline for every repo in your organization). I have "Periodically if not otherwise run" set to run once a day for each project.
It does seem like these things could work via webhook, but they do not in my experience.
Related
I have a multibranch project in jenkins, and every time I press Scan Repository Now it queue a new build just because
‘Jenkinsfile’ found
Met criteria
What I'd like to do is whenever I scan the repository, it only add Pull Request to the project without triggering a build. And also, if I turn on scan repository trigger, periodically if not otherwise run, every time it branch indexing, it also build the pull request even after I turn on Skip initial build on first branch indexing.
What I'd like to do is whenever there's a comment 'build' in the pull request, then it builds the branch, so if the pr doesn't contain the comment, it should not build anything.
How can I achieve this?
This is my setup
I use Jenkins 2.180
According to the pipeline documentation there should be a option do dissable Index Triggers on Multibranch types, see https://jenkins.io/doc/book/pipeline/syntax/#options
But i didn't find the option itself either, so i deactivated it per Pipeline definition in the Jenkinsfile of every branch :/
overrideIndexTriggers
Allows overriding default treatment of branch indexing triggers. If branch indexing triggers are disabled at the multibranch or
organization label, options { overrideIndexTriggers(true) } will
enable them for this job only. Otherwise, options {
overrideIndexTriggers(false) } will disable branch indexing triggers
for this job only.
I also use SCM webhooks to autmatically trigger the branch indexing etc
The default for Build Strategies is to OR the list together. You'll need to remove the existing build strategies and add an "All Strategies Match" build strategy and add "Change Requests" and "Skip Initial Build on First Branch Indexing" to that.
Source: https://issues.jenkins-ci.org/browse/JENKINS-58442
I'm setting up a new Jenkins job using multibranch pipeline and I have noticed that when a branch is deleted, it only has a strikethrough and isn't actually removed on Jenkins. This is solved by re-running branch indexing. However, I cannot really use this as it will also cause every other branch to rebuild (a consequence of how the repository is updated). Is there some custom code or pipeline/script I can run to re-index without building?
I've already looked at various UI methods such as suppressing SCM triggers, but this also negates push events from Github which is something we want to use.
The deleted/merged branch build will disappear after a period of time (<24 hours). It is not removed immediately to show the recently deleted/merged branches and give a chance to review the prior build statuses. It is relatively harmless since the jobs for these branches are deactivated (read-only).
Note that the removal is based on the branch indexing job running at regular interval, so if you have this disabled, it probably won't do it (not sure the SCM webhook calls are enough).
I've been using Jenkins and I've seen a lot of Pipeline examples (declarative ones) and I've seen some using the pollSCM property in the Jenkinsfile to trigger a build, like this:
triggers {
pollSCM('H/5 * * * *')
}
However, I've seen this Scan Multibranch Pipeline Triggers option when configuring a Multibranch pipeline. I'm not sure what's the difference between those.
All this problem came to me because I'm facing some cases where two builds are being triggered for the same job, and I thought it was because I have both these options configured.
Can anyone please help me understand this difference?
Thank you!
Scan Multibranch Pipeline Trigger
The ‘Scan Multibranch Pipeline’ trigger will scan the repository for new branches and changes in existing branches. By default it will trigger a new build for all branches which have been updated. However in the multibranch job configuration you can disable this automatic trigger for specific - or all - branches. However, please note that you have to set up the web hook for your repository so that Jenkins will be notified of any changes. As the hook setup will depend both on the Jenkins plugin used to checkout the Jenkinsfile as well as on the Git server, you will have to look this up accordingly.
Poll SCM
The pollSCM trigger is branch-specific. Within a Jenkinsfile you may configure different options for different branches. This option will never be able to trigger the very first build for a branch as it would need at least one build so the properties step gets executed and the pollSCM option set. That is: Any change here will only get effective AFTER the next build.
You can use pollSCM trigger in two ways:
Real polling. That is: Tell Jenkins to check the repository for changes in certain time intervals.
Waiting for notification by some repository hook. For this you'd need to
Keep the string passed to pollSCM empty:
triggers {
pollSCM('')
}
Set up a hook on your git server to notify Jenkins (See How to configure Git post commit hook)
Recommendation
Therefore I’d recommend to stick to the trigger based on the Multibranch branch scan - if possible. However in some special cases (e.g. if the first build on a new branch should never be built automatically) it still might be useful to use the poll SCM feature. In that case you might want to disable the automatic trigger as required.
Last but not least the poll SCM feature may use a different plugin than the Scan Multibranch Pipeline, e.g. for Bitbucket.
AFAIK for Bitbucket the multibranch trigger is little bit more flexible, allows to trigger a build on more events compared the the plain Bitbucket trigger.
I think pollSCM must be the jenkins plugin
https://wiki.jenkins.io/display/JENKINS/PollSCM+Plugin
Multibranch pipeline : This is the type of pipeline, where jenkins scan and pull from all the branch within the repository , so the build will trigger automatically when some code is checked in within the branch (if you have configured it)
I have just setup a basic multibranch pipeline build job and have a feature branch with a jenkinsfile present which I am experimenting with.
I have the job configured to poll the scm every 5 mins and trigger a build if necessary.
I am finding however at times when I manually start a build for my branch after I have pushed up a tweak on my Jenkinsfile for example (as I don't want to wait for the next scm poll interval), the branch re-indexing activity can still trigger another build to be performed.
See the image below for what I mean, so here build 7 is one I kicked off manually on jenkins, so it picked up my commit, but then the branch indexing kicked off build 8 but there were no new changes for the branch.
Is there a way to prevent this from happening? Other than me being patient and waiting 5 mins of course!
Thanks
I experienced this also. I tried playing with the polling settings in the Multibranch pipeline to see if this could fix it but that only led to no builds being spawned at all in response to SCM changes.
So this behaviour to which you refer was the best of a bad lot.
At time of writing, I think this behaviour is a bug with no current resolution.
Can't remember if I raised a ticket or not as I am in Azure DevOps world now.
Using Jenkins, I am running 2 builds (Linux+Windows) and one Doxygen job
At the moment, I am using 3 separate SCM polling triggers pointing to the same source code
How can I use a single trigger for all three jobs provided that I still want to get separate statuses
For the record; the underlying SCM is Git
Off the top of my head, some solutions which might do what you are looking for:
Instead of setting an SCM trigger, use a post-receive hook in your repository, which can send a signal for Jenkins that there are new changes (see: https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository). This way Jenkins doesn't have to constantly poll the repository (multiple times for the different jobs), and the trigger would be faster, since there is no waiting for the next polling, but when there is a push, a build will be started.
Use an extra job, that would do nothing else, just have the SCM polling trigger, and start all three original jobs (without waiting for any of them to finish).
If the configuration is similar for all three jobs, you could consider creating a single project with a matrix configuration. Roughly what it does, is that you could have a variable for the build type, with values like linux, windows, doxygen. When the job is triggered, it would start multiple builds with all the possible values - of course you would have to set up the job in a way that the current parameter changes the build process according to what needs to be done. Actually I haven't had to use a matrix configuration yet, so my example may be not the best, but you can probably find lots of examples on the Jenkins wiki, if you think this is a good direction.