How to do a clean SVN checkout/update in multibranch pipeline - jenkins

I am using a multibranch pipeline with SVN. I currently don't use an explicit checkout scm command but I am using the declarative default checkout. My problem is, that this only seems to perform an svn update. I would however like to use the UpdateWithCleanUpdater update strategy.
I have seen here that there is a possibility to modify the scm object - but not without granting further permissions:
import hudson.scm.subversion.UpdateWithCleanUpdater
scm.setWorkspaceUpdater(new UpdateWithCleanUpdater())
Is there a simple way to only configure the updater while keeping all the other information from the multibranch configuration and having to set the branch, credentials, server, .... again manually?
I have tried checkout scm: [workspaceUpdater: [$class: 'UpdateWithCleanUpdater']] but this doesn't work

There is a built-in generator that comes with Jenkins at <jenkins-url>/pipeline-syntax e.g. http://localhost/pipeline-syntax
Under Sample Step, use the "checkout" step instead of "svn".
SCM: Subversion
Enter the URL and Credentials to use
Check-out Strategy: can be "Always check out fresh" or "svn revert before update". Up to you, the latter one is faster and has not yet failed in my experience. (see image below)
Click "Generate Pipeline Script", and copy the output into your pipeline.
The sample output will look like this:
checkout(
[$class: 'SubversionSCM',
additionalCredentials: [], excludedCommitMessages: '',
excludedRegions: '', excludedRevprop: '', excludedUsers: '',
filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '',
locations: [[cancelProcessOnExternalsFail: true,
credentialsId: '********-********-*******',
depthOption: 'infinity', ignoreExternalsOption: true,
local: '.', remote: 'http://*****/*****/trunk']],
quietOperation: true,
workspaceUpdater: [$class: 'CheckoutUpdater']]
)
You can have as many of these lines as you want on your pipeline script, with the remote URL changing depending on the branch. In my case, I have two repositories that need to sit next to each other when checked out, so I have two checkout commands in my script.
Note: "Always fresh" uses the CheckoutUpdater class, and "revert before update" uses UpdateWithRevertUpdater. The generated output is usable in both Declarative and Scripted pipeline syntax.

In a declarative pipeline stage, make sure that the options contain:
options {
skipDefaultCheckout()
}
and the steps section starts with a script block like:
script {
myScm = scm
myScm.setWorkspaceUpdater(new hudson.scm.subversion.UpdateWithRevertUpdater())
checkout myScm
}
More info on the different types of workspace updaters for Subversion is here: https://javadoc.jenkins.io/plugin/subversion/hudson/scm/subversion/WorkspaceUpdater.html
And yes, you may need to approve some scripts on the approval page:
http://<myjenkins>/scriptApproval/

Related

Make Jenkins with declarative pipeline work with SVN trunk as well as branches

I want to set up a Jenkins job with a declarative pipeline from a Jenkinsfile, using Subversion as the SCM, which should
do a scheduled SCM poll to detect changes in the trunk
be manually startable to build the trunk or any branch of choice
So I've set up the Jenkins job with the List Subversion tags (and more) parameter that collects existing SVN branches from an SVN url and lets the user select one. The selected value is stored in a variable, for instance $svnBranch, and I defined 'trunk' as its default value.
Then this variable is used to build the resulting SCM url, for example
svn+ssh://svn.mydomain.org/Reponame/projectname/$svnBranch/componentname
Now here's the problem:
This setup does work as long as the job ist started manually. But if it is started by the cron schedule, Jenkins keeps detecting SCM changes each and every time and always starts a new build. The SCN polling log shows
Workspace doesn't contain Reponame/projectname/$svnBranch/componentname. Need a new build.
So the problem is obviously caused by Jenkins not resolving the variable when polling the SCM for changes. To verify this assumption, I changed the job to use a fixed string variable, and the same happened again.
I was wondering if it possible to solve the problem by moving the polling and checkout logic to the Jenkinsfile. The idea would be to always poll the trunk, but checkout and build based on $svnBranch, but I'm unsure how to do this. Is it possible to define different SCM urls for polling and checkouts? According to my research, all checkout urls in a Jenkinsfile would automatically be used for polling, so how to accomplish that?
Any other working solution would be welcome, too.
Note that there's a similar question Jenkins Pipeline - SVN polling that stumbled upon the same issue, but no solution that would fit my scenario.
Also note that there is an issue reported at JENKINS-10628: SCM build trigger not working correctly with variables in SVN URL that describes my problem, but it's said to be resolved with a new version of the Subversion plugin since 2015. I've updated to the latest version 2.16.0, yet it did not resolve the problem.
This is the solution I've found (I'm open to better ones - for example, I'm not happy with configuring the same SCM url in two different places):
First, in the Jenkins job, under Pipeline from SCM, I configured the trunk url that contains no variable. This url will be used to poll for changes in trunk.
svn+ssh://svn.mydomain.org/Reponame/projectname/trunk/componentname
Second, I created thid function to replace the "trunk" part by the branch name:
def call(Map param = [ : ]) {
if ( param.branch == null ) {
return param.trunkUrl
}
url = param.trunkUrl.replaceAll('/trunk(?=/|$)', '/'+param.branch) // replaces /trunk if followed by / or if at end of url
return url
}
I have moved this function to a shared library, so I can use it from any pipeline.
This is then used to derive the checkout url with the svnBranch being selected in the user interface:
environment {
// Set actual checkout url, because SVN_URL_1 will always contain the fixed url of the trunk used for polling
checkoutUrl = composeSvnUrl(trunkUrl: env.SVN_URL_1, branch: env.svnBranch)
}
Finally, I added a checkout stage (as the first stage) to my Jenkinsfile:
stage('Checkout') {
steps {
/* Checkout for actual build (may be different if started manually) */
checkout(
poll: false, changelog: false, // = do not use this for polling
scm: [
$class: 'SubversionSCM',
quietOperation: false,
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: '',
filterChangelog: false,
ignoreDirPropChanges: false,
includedRegions: '',
locations: [[
credentialsId: 'id.from.jenkins.credentials',
depthOption: 'infinity',
ignoreExternalsOption: true,
local: '.',
remote: checkoutUrl,
workspaceUpdater: [$class: 'CheckoutUpdater']
]
)
}
}
The important parts are:
poll: false, changelog: false means that Jenkins should not use these checkout information for polling. As mentioned on Pipeline: SCM Step - checkout: Check out from version control at the very bottom of the page:
If 'Include in polling' is disabled and 'Include in changelog' is disabled, then when polling occurs, changes that are detected from this repository will be ignored.
workspaceUpdater: [$class: 'CheckoutUpdater'] is probably important, too, as this will wipe the workspace before checking out again.

What to enter for "scm" in BlueOcean editor

I got stuck at first step using the easy BlueOcean editor. I am trying to create a first step Build / Check out from version control.
With one required field to enter scm. I tried the URL to my repo but the editor shows a message
Expecting "class hudson.scm.SCM for parameter "scm" but got "whatever
I enter" of type class java.lang.String instead.
How can I enter a class and not a string in a string field?
What does BlueOcean expect from me at this point?
Sounds like you have selected the Git option in the pipeline creation step. It will accept ssh connection to the repository that you would like to create a Blue ocean Pipeline. Select Github/Bitbucket if you have your code on one of those repositories.
Also, the blue ocean pipeline adds a Jenkinsfile to the repository so it needs SSH access.
So follow below steps.
Create a new SSH key on Blue Ocean instead of specifying a https URL
Add that key to Git repo settings
You should be able to create a new pipeline and authentication will happen.
I outlined this here
Hope this helps. I dont have enough reputation to ask questions, hence answering straight away
the BlueOcean Editor expects something like this (Example for SVN):
[$class: 'SubversionSCM', additionalCredentials: [], excludedCommitMessages: '', excludedRegions: '', excludedRevprop: '', excludedUsers: '', filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '', locations: [[cancelProcessOnExternalsFail: true, credentialsId: 'Your Credential ID', depthOption: 'infinity', ignoreExternalsOption: true, local: '.', remote: 'SVN-REPO-Location']], quietOperation: true, workspaceUpdater: [$class: 'UpdateUpdater']
you can create this code with the code snippet editor as described here:(Checkout SVN with credentials in Jenkins pipeline?), just generate the snippet and cut out the class discription. Or paste the whole snippet into your pipeline as described here:(Jenkins Blue Ocean Provide Credentials for Subversion SCM Step)
Hope this helps!
Greetings,
bluescreenterror

Jenkins: Pipeline upstream trigger stop working at multibranch project

I have two Jenkins multibranch pipeline projects.
1st must be triggered by the Github and by the 2nd one success build.
2nd must be triggered by the Github only.
I added Properties to Jenkinsfiles (no upstream for 2nd of course)
properties([
// Builds rotation
buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10')),
disableConcurrentBuilds(),
// Git project
[$class: 'GithubProjectProperty', displayName: '', projectUrlStr: G_giturl],
// Trigger build from:
pipelineTriggers([upstream(G_artifactsource), githubPush()])
])
The new properties appeared at graphical interface and all worked fine, for awhile.
I do not know what i did, or maybe did nothing, but now all "Build Triggers" of all branches are empty. Github webhooks are still working, but not the upstream triggers. For testing purposes i made two new repos and did the same projects for them. New projects work good.
There is no "apply" buttons inside branches at the Multibranch projects, I can't add or delete build triggers by interface. Changing of Jenkinsfiles don't help too.
Is it bug or i missed something?
No "properties" for multibranch. Only "options" and "triggers"
Next code works fine.
pipeline {
agent none
triggers {
upstream G_artifactsource
}
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '10')
disableConcurrentBuilds()
}
....

I want to checkout the 2nd Repo in my Jenkins multibranch pipeline

In my Jenkinsfile, the "checkout scm" command will checkout whatever repo I have configured in the configuration panel.
But what if I add a 2nd repo to the Jenkins file - is there any way to check that out to a sepcific directory within the workspace? The catch is that I don't want to hard-code any URLs into my Jenkinsfile. Here's an illustration of what I'm trying to achive:
stage("Checkout") {
checkout scm // Works fine, checks out the 1st consifured repo to workspace.
dir("src") {
checkout scm // Checks out the exact same repo again, but how can I change this to colone the 2nd repo instead?
}
}
Basically - what could I put instead of the 2nd "checkout scm" that would make it pull the 2nd repo configured in the Multibranch pipeline web config?
And supposing this isn't actually possible - what's even the point of allowing users to provide more than one repo in the config-form if there's no way of checking it out in the script?
Use the url found at yourjenkinshostname.com/pipeline-syntax/ to generate a step for "checkout: General SCM". After that, fill out the info for the repo you want, and click "Additional Behaviors" and add one for "Checkout to Subdirectory".
Lastly click "Generate Pipeline Script". The output from that should be useable in your Jenkinsfile. Completed, the process looks like this:
Syntax Generator Example
Alternatively, if you're used to the checkout step, the "RelativeTargetDirectory" extension class can be used to do this. A checkout step with that included looks like this:
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'test-dir']], submoduleCfg: [], userRemoteConfigs: [[url: 'https://github.com/jenkinsci/puppet-jenkins.git']]])
The key part being...
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: 'test-dir']]
EDIT: According to issues.jenkins-ci.org/browse/JENKINS-32018, the multiple sources of a multibranch job is not for two different repositories, but rather for multiple sources of a single repository.
You'll need to hardcode in URLs, I'm afraid. On approach is to have two multibranch jobs. One has the SCM as repo A and hardcodes a checkout of repo B, the other has repo B as the SCM, and hardcodes a checkout of repo A.

Clean builds with Multibranch Workflow

Using Multibranch Workflow, the command to check out looks like
checkout scm
I can't find a way to tell Jenkins to perform a clean checkout. By "clean," I mean it should remove all files from the workspace that aren't under version control.
I'm not sure if this answers the original question or not (I couldn't tell if the intention was to leave some files in the workspace) but why not just remove the workspace first, this would allow a clean checkout:
stage ('Clean') {
deleteDir()
}
stage ('Checkout') {
checkout scm
}
I run into the same problem and here is my workaround.
I created a new scm object for the checkout and extended the extensions with the CleanBeforeCheckout. But i kept the other configurations like branches and userRemoteConfigs.
checkout([
$class: 'GitSCM',
branches: scm.branches,
extensions: scm.extensions + [[$class: 'CleanBeforeCheckout']],
userRemoteConfigs: scm.userRemoteConfigs
])
It's still not perfect because you have to create a new object :(
First, you can not assume that a workflow job has a workspace as it was for freestyle jobs. Actually, a workflow job can use more than one workspace (one for each node or ws block).
Said that, what I'm going to propose is a kind of hacky: modify the scm object before checkout to set up a CleanCheckout extension (you will have to approve some calls there).
import hudson.plugins.git.extensions.impl.CleanCheckout
scm.extensions.replace(new CleanCheckout())
checkout scm
But I'd prefer Christopher Orr's proposal, use a shell step after checkout (sh 'git clean -fdx').
Behaviors can be added when configuring the source. clean before checkout, clean after checkout and Wipe out repository and force clone. This removes the need to add logic to the declarative / scripted pipelines.
Adding Christopher-Orr's comment as an answer to just do:
stage('Checkout') {
checkout scm
sh 'git clean -fdx'
}
Jenkins currently contains a page to generate groovy pipeline syntax. Selecting the checkout step you should be able to add all the additional options that you're used to.
I generated the following which should do what you want:
checkout poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'CleanBeforeCheckout']], submoduleCfg: [], userRemoteConfigs: [[url: 'ssh://repo/location.git']]]

Resources