I have two Jenkins multibranch pipeline projects.
1st must be triggered by the Github and by the 2nd one success build.
2nd must be triggered by the Github only.
I added Properties to Jenkinsfiles (no upstream for 2nd of course)
properties([
// Builds rotation
buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10')),
disableConcurrentBuilds(),
// Git project
[$class: 'GithubProjectProperty', displayName: '', projectUrlStr: G_giturl],
// Trigger build from:
pipelineTriggers([upstream(G_artifactsource), githubPush()])
])
The new properties appeared at graphical interface and all worked fine, for awhile.
I do not know what i did, or maybe did nothing, but now all "Build Triggers" of all branches are empty. Github webhooks are still working, but not the upstream triggers. For testing purposes i made two new repos and did the same projects for them. New projects work good.
There is no "apply" buttons inside branches at the Multibranch projects, I can't add or delete build triggers by interface. Changing of Jenkinsfiles don't help too.
Is it bug or i missed something?
No "properties" for multibranch. Only "options" and "triggers"
Next code works fine.
pipeline {
agent none
triggers {
upstream G_artifactsource
}
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10')
disableConcurrentBuilds()
}
....
Related
I want to set up a Jenkins job with a declarative pipeline from a Jenkinsfile, using Subversion as the SCM, which should
do a scheduled SCM poll to detect changes in the trunk
be manually startable to build the trunk or any branch of choice
So I've set up the Jenkins job with the List Subversion tags (and more) parameter that collects existing SVN branches from an SVN url and lets the user select one. The selected value is stored in a variable, for instance $svnBranch, and I defined 'trunk' as its default value.
Then this variable is used to build the resulting SCM url, for example
svn+ssh://svn.mydomain.org/Reponame/projectname/$svnBranch/componentname
Now here's the problem:
This setup does work as long as the job ist started manually. But if it is started by the cron schedule, Jenkins keeps detecting SCM changes each and every time and always starts a new build. The SCN polling log shows
Workspace doesn't contain Reponame/projectname/$svnBranch/componentname. Need a new build.
So the problem is obviously caused by Jenkins not resolving the variable when polling the SCM for changes. To verify this assumption, I changed the job to use a fixed string variable, and the same happened again.
I was wondering if it possible to solve the problem by moving the polling and checkout logic to the Jenkinsfile. The idea would be to always poll the trunk, but checkout and build based on $svnBranch, but I'm unsure how to do this. Is it possible to define different SCM urls for polling and checkouts? According to my research, all checkout urls in a Jenkinsfile would automatically be used for polling, so how to accomplish that?
Any other working solution would be welcome, too.
Note that there's a similar question Jenkins Pipeline - SVN polling that stumbled upon the same issue, but no solution that would fit my scenario.
Also note that there is an issue reported at JENKINS-10628: SCM build trigger not working correctly with variables in SVN URL that describes my problem, but it's said to be resolved with a new version of the Subversion plugin since 2015. I've updated to the latest version 2.16.0, yet it did not resolve the problem.
This is the solution I've found (I'm open to better ones - for example, I'm not happy with configuring the same SCM url in two different places):
First, in the Jenkins job, under Pipeline from SCM, I configured the trunk url that contains no variable. This url will be used to poll for changes in trunk.
svn+ssh://svn.mydomain.org/Reponame/projectname/trunk/componentname
Second, I created thid function to replace the "trunk" part by the branch name:
def call(Map param = [ : ]) {
if ( param.branch == null ) {
return param.trunkUrl
}
url = param.trunkUrl.replaceAll('/trunk(?=/|$)', '/'+param.branch) // replaces /trunk if followed by / or if at end of url
return url
}
I have moved this function to a shared library, so I can use it from any pipeline.
This is then used to derive the checkout url with the svnBranch being selected in the user interface:
environment {
// Set actual checkout url, because SVN_URL_1 will always contain the fixed url of the trunk used for polling
checkoutUrl = composeSvnUrl(trunkUrl: env.SVN_URL_1, branch: env.svnBranch)
}
Finally, I added a checkout stage (as the first stage) to my Jenkinsfile:
stage('Checkout') {
steps {
/* Checkout for actual build (may be different if started manually) */
checkout(
poll: false, changelog: false, // = do not use this for polling
scm: [
$class: 'SubversionSCM',
quietOperation: false,
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: '',
filterChangelog: false,
ignoreDirPropChanges: false,
includedRegions: '',
locations: [[
credentialsId: 'id.from.jenkins.credentials',
depthOption: 'infinity',
ignoreExternalsOption: true,
local: '.',
remote: checkoutUrl,
workspaceUpdater: [$class: 'CheckoutUpdater']
]
)
}
}
The important parts are:
poll: false, changelog: false means that Jenkins should not use these checkout information for polling. As mentioned on Pipeline: SCM Step - checkout: Check out from version control at the very bottom of the page:
If 'Include in polling' is disabled and 'Include in changelog' is disabled, then when polling occurs, changes that are detected from this repository will be ignored.
workspaceUpdater: [$class: 'CheckoutUpdater'] is probably important, too, as this will wipe the workspace before checking out again.
I am using a multibranch pipeline with SVN. I currently don't use an explicit checkout scm command but I am using the declarative default checkout. My problem is, that this only seems to perform an svn update. I would however like to use the UpdateWithCleanUpdater update strategy.
I have seen here that there is a possibility to modify the scm object - but not without granting further permissions:
import hudson.scm.subversion.UpdateWithCleanUpdater
scm.setWorkspaceUpdater(new UpdateWithCleanUpdater())
Is there a simple way to only configure the updater while keeping all the other information from the multibranch configuration and having to set the branch, credentials, server, .... again manually?
I have tried checkout scm: [workspaceUpdater: [$class: 'UpdateWithCleanUpdater']] but this doesn't work
There is a built-in generator that comes with Jenkins at <jenkins-url>/pipeline-syntax e.g. http://localhost/pipeline-syntax
Under Sample Step, use the "checkout" step instead of "svn".
SCM: Subversion
Enter the URL and Credentials to use
Check-out Strategy: can be "Always check out fresh" or "svn revert before update". Up to you, the latter one is faster and has not yet failed in my experience. (see image below)
Click "Generate Pipeline Script", and copy the output into your pipeline.
The sample output will look like this:
checkout(
[$class: 'SubversionSCM',
additionalCredentials: [], excludedCommitMessages: '',
excludedRegions: '', excludedRevprop: '', excludedUsers: '',
filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '',
locations: [[cancelProcessOnExternalsFail: true,
credentialsId: '********-********-*******',
depthOption: 'infinity', ignoreExternalsOption: true,
local: '.', remote: 'http://*****/*****/trunk']],
quietOperation: true,
workspaceUpdater: [$class: 'CheckoutUpdater']]
)
You can have as many of these lines as you want on your pipeline script, with the remote URL changing depending on the branch. In my case, I have two repositories that need to sit next to each other when checked out, so I have two checkout commands in my script.
Note: "Always fresh" uses the CheckoutUpdater class, and "revert before update" uses UpdateWithRevertUpdater. The generated output is usable in both Declarative and Scripted pipeline syntax.
In a declarative pipeline stage, make sure that the options contain:
options {
skipDefaultCheckout()
}
and the steps section starts with a script block like:
script {
myScm = scm
myScm.setWorkspaceUpdater(new hudson.scm.subversion.UpdateWithRevertUpdater())
checkout myScm
}
More info on the different types of workspace updaters for Subversion is here: https://javadoc.jenkins.io/plugin/subversion/hudson/scm/subversion/WorkspaceUpdater.html
And yes, you may need to approve some scripts on the approval page:
http://<myjenkins>/scriptApproval/
My Jenkinsfile has two SCM checkouts, primary, and secondary. I only want to have the build triggered when commits are made in primary. I've set the poll argument in the obvious way, but it does not seem to be honored; the build gets triggered when commits are made to either repository.
node {
stage("checkout") {
checkout scm: [$class: "MercurialSCM", source: "/var/jenkins_home/hg/primary", subdir: "hg/primary", clean: true], poll: true
checkout scm: [$class: "MercurialSCM", source: "/var/jenkins_home/hg/secondary", subdir: "hg/secondary", clean: true], poll: false
}
stage("do something") {
echo 'Hello World'
sh 'sleep 30s'
echo 'Done'
}
}
I could not figure out how to do this from within Jenkinsfile only.
To solve this problem for myself, I ended up creating a separate Jenkins job (Free Style project, not pipeline) which was setup to Poll SCM on the primary repo. This job does nothing except in the Post-build Actions it triggers my actual Jenkins Pipeline job which loads the Jenkinsfile like you showed.
The trigger passes a Predefined parameter set to the change ID that the poller found. In my case it was Git so I set change=${GIT_COMMIT}.
In my Pipeline job, I created a String parameter called change.
In my Jenkinsfile, I used env.change in the checkout line to checkout the specific commit.
In a multibranch pipeline job, I have configured builds (basic linting) to scan across branches for a jenkins file. I still have to perform this build manually however. What is the property I can set to enable polling of GitHub or, even better, triggered on new commits.
In general, I'm trying to find a way to learn how all GUI fields map to keys I can use in the properties(); method. There is no way for me to translate between GUI form field and script key-value option.
node('master') {
properties([
[$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10']],
[$class: 'BuildTriggerProperty???', strategy: 'Build when a change is pushed to GitHub???']
]);
...
}
Jenkins version 2.7
I'm trying to find a way to learn how all GUI fields map to keys I can use in the properties(); method.
If I got you correctly, the answer is:
Go to to your Pipeline project page
Find Pipeline Syntax link in the left-side menu and follow it
Find Snippet Generator link in the left-side menu and follow it
Select properties: Set job properties from Sample Step dropdown
Choose whatever you want and click Generate Groovy
Profit =)
This does not work (anymore?) as the only options are:
Build Triggers:
Build periodically
Build when another project is promoted
Build whenever a SNAPSHOT dependency is built
Monitor Docker Hub/Registry for image changes
Periodically if not otherwise run
Stash Pull Requests Builder
On a simple "pipeline" build, you can specify:
Build when a change is pushed to BitBucket
But MultiBranch doesn't have this option.
We are using the pipeline plugin with multibranch configuration for our CD.
We have checked in the Jenkinsfile which works off git.
git url: "$url",credentialsId:'$credentials'
The job works fine, but does not auto trigger when a change is pushed to github.
I have set up the GIT web hooks correctly.
Interestingly, when I go into a branch of the multibranch job and I click "View Configuration", I see that the "Build when a change is pushed to Github" is unchecked. There is no way to check it since I can not modify the configuration of the job (since it takes from parent) and the same option is not there in parent.
Any ideas how to fix this?
For declarative pipelines try:
pipeline {
...
triggers {
githubPush()
}
...
}
For me this enables the checkbox "GitHub hook trigger for GITScm polling", but polling is not actually required. This requires the GitHub plugin.
I found a way to check the checkbox "Build when a change is pushed to Github".
This line did the trick:
properties([pipelineTriggers([[$class: 'GitHubPushTrigger'], pollSCM('H/15 * * * *')])])
I think the polling is needed to make it work. Would be nice if no polling is needed.
Here's a Jenkinsfile example with this implemented:
#!/usr/bin/env groovy
node ('master'){
stage('Build and Test') {
properties([pipelineTriggers([[$class: 'GitHubPushTrigger'], pollSCM('H/15 * * * *')])])
checkout scm
env.PATH = "${tool 'Maven 3'}/bin:${env.PATH}"
sh 'mvn clean package'
}
}
For declarative pipelines, try this:
pipeline {
agent any
triggers {
pollSCM('') //Empty quotes tells it to build on a push
}
}
If you use Stash for example you can register a Post-Receive WebHook where you have to insert your URL form Jenkins like : http://jenkinsHost:9090/git/notifyCommit?url=ssh://git#gitHost:1234/test.git
In your jenkins Job you have to set at least the Build trigger "Poll SCM".
And set a polling time of e.g 5 mins.
This enables also the automatic branch indexing for your multibranch project configuration.
Resorting to polling adds latency - time that it takes for a build to start and hence finish giving back a result.
It seemed to me that the basic plugins have a low level of abstraction, so I switched to the Github Organization Folder plugin, which depends on all of them and sets up an organization hook for triggering builds branches and/or pull requests.
Before I start, I would like to emphasize that I had no previous experience with Jenkins so far, so there might be a bunch of better solutions out there.
What I wanted to achieve in a nutshell:
After every push made to a Bitbucket repo(test2), on every branch,
pull and build another Bitbucket repo(test1), from an identical
branch name and right after that, build test2 using test1 as a
dependency.
How I managed to achieve that?
I started a new job with type 'Multibranch Pipeline'
I added the following Jenkinsfile to test2:
pipeline {
agent any
stages {
stage('build') {
steps {
dir('test1') {
git branch: BRANCH_NAME, url: 'git#bitbucket.org:user/test1.git', credentialsId: 'credentials_id'
}
sh('build_process')
}
}
}
}
I come across the issue that you can't set up a Bitbucket hook for pipelines
I added Bitbucket Branch Source Plugin to Jenkins
I selected Bitbucket at 'Branch Sources' when setting up the job
I added credentials and put a checkmark to Auto-register webhook
Under 'Scan Multibranch Pipeline Triggers' I put a checkmark to Periodically if not otherwise run, with an interval of 1 min
I added a webhook to my Bitbucket repo
I updated all my plugins, restarted Jenkins and it's ready to go
Other plugins I have installed: Bitbucket Plugin, Pipeline plugin. Hope this helps for somebody, I did manage to solve it this way after hours of struggling without the Bitbucket Branch Source Plugin.
node{
stage('Build and Test') {
properties([pipelineTriggers([[$class: 'GitHubPushTrigger'], pollSCM('* * * * *')])])
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'xxx-xxx-xxxx[your credentails Id]', url: 'https://github.com/git']]])
echo 'Build and Test has been done'
}
}