When scanning my multibranch pipeline, I am expecting old branches that are deleted from bitbucket to get deleted in Jenkins.
This is the output of scanning my pipeline:
Evaluating orphaned items in *****
Will remove ****
Will remove ****
Finished: NOT_BUILT
But it doesn't remove the branches.
I have Prune setup and orphaned setting are all set to empty
Edit:
Tried to add extensions: [[$class: 'PruneStaleBranch']] to my checkout step, it still doesn't remove old branches
Related
In a Jenkinsfile, to start a parameterized pipeline job from another job, I have this code snippet:
build job: 'build-sharpen-branch', parameters: [
[$class: 'StringParameterValue', name: 'BRANCHNAME', value: mergeBranchname]
]
This already works as expected, and it will start a job at URL https://$JENKINS_URL/job/build-sharpen-branch/.
Now I want to start a job, that is one branch of a multibranch project inside a Bitbucket folder. The URL of the job is https://$JENKINS_URL/job/iText%207%20.NET/job/sharpen/job/feature%2FQA-10738/.
iText%207%20.NET is the name of the Bitbucket project.
sharpen is the name of the Multibranch job.
feature%2FQA-10738 is the name of the branch, urlencoded.
I read the following questions about starting a multibranch job NOT inside a folder:
Trigger Multibranch Job from another
Triggering a multibranch pipeline job from an other multibranch pipeline
How to trigger Multibranch Pipeline Jenkins Job within regular pipeline job?
From the answers there, I gather that the syntax is $JOB/$BRANCH (where $BRANCH is URL-encoded to rename branches like feature/foo to feature%2Ffoo).
From Jenkins pipeline with folder plugin. How to build a job located different folder I gather that the syntax for a job inside a folder is $FOLDER/$JOB.
Combining the two, I conclude that the syntax for folder+job+branch is most likely $FOLDER/$JOB/$BRANCH.
So I tried with this code:
build job: "iText%207%20.NET/sharpen/${java.net.URLEncoder.encode branchName, 'UTF-8'}"
with
folder = iText%207%20.NET
job = sharpen
branch = ${java.net.URLEncoder.encode branchName, 'UTF-8'} (URLEncoder to change / in the branch name to %2F)
To my surprise, when I ran this, I got an error:
ERROR: No item named iText%207%20.NET/sharpen/feature%2FQA-10738 found
As already stated above, a job exists on URL https://$JENKINS_URL/job/iText%207%20.NET/job/sharpen/job/feature%2FQA-10738/.
What is the correct syntax for a multibranch job inside a Bitbucket folder?
The problem was with the folder name iText%207%20.NET, which is an urlencoded version of iText 7 .NET. Apparently Jenkins can't handle urlencoded spaces in folder names.
I renamed the folder to itext_7_dotnet and then used
build job: "itext_7_dotnet/sharpen/${java.net.URLEncoder.encode branchName, 'UTF-8'}"
This works.
Moral of the story: never use spaces in your folder names!
For us, this works:
build job: "${folder}/${repo}/${branch.replace('/','%2F')}", wait: false, propagate: false
You can also trigger build by using curl, with crumbs and all that.
Jenkins build job started to fail if the tag was pushed to the origin/master. We're using a private repository, credentials are Ok, validate works correctly. In case of commit to master build successfully build.
Steps to reproduce:
git push origin <tagname>
press build job -> will lead to error described below in log section
Pipeline Steps:
Start of Pipeline - (5.3 sec in block) __________Success
Read trusted file from SCM - (0.76 sec in self) _ Failed
If I remove the tag and commit some changes the new build will be Successfully completed
Here is the log of the job:
17:06:49 Connecting to https://api.github.com using ...build/****** (*** GitHub Build User)
ERROR: Could not do lightweight checkout, falling back to heavyweight
java.io.FileNotFoundException: https://raw.githubusercontent.com/***/myrepository/7ae6037fe8/Jenkinsfile?token=ALR5FLRJSV5UVVRE442KWCC6C42PA
at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.asStream(Requester.java:340)
Caused: org.kohsuke.github.GHFileNotFoundException: 404: Not Found
at org.kohsuke.github.Requester.handleApiError(Requester.java:699)
at org.kohsuke.github.Requester.asStream(Requester.java:342)
at org.kohsuke.github.GHContent.read(GHContent.java:117)
at org.jenkinsci.plugins.github_branch_source.GitHubSCMFile.content(GitHubSCMFile.java:183)
at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:335)
at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:299)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
Here is the git plugin configuration:
Previously tags were building fine, but can't what was changed.
What can be a reason for such error, is it no permissions to access the file?
Update1:
The problem is not with tag, but with Rebuild the same build. When Build Now button triggres again, this error will reproduce
Thanks in advance
I would clean the workspace at the very beginning of the stage.
On a traditional free style job I would try the Workspace Cleanup plugin
https://wiki.jenkins.io/display/JENKINS/Workspace+Cleanup+Plugin
on a Jenkins pipeline DSL I would use
checkout([$class: 'GitSCM',
branches: scm.branches,
extensions: scm.extensions + [[$class: 'WipeWorkspace']],
userRemoteConfigs: scm.userRemoteConfigs
])
I'm trying to setup a CI process using Jenkins with the source code being held in Mercurial, so I have the Jenkins Mercurial Plugin installed.
It's a simple pipeline script just now which polls my repository every 10 minutes. Using the checkout command below and hitting Build Now on my job it successfully clones my repository and continues with the pipeline as expected. It also successfully sets up polling and I can see from the polling log that it is checking the repository for changes every 10 minutes.
checkout changelog: true, poll: true, scm: [$class: 'MercurialSCM', credentialsId: 'xxx', installation: 'TortoiseHg', source: 'C:\\Path\\to\\repo']
However, when I push to my repository, Jenkins on next running the job spots these changes, pulls them down but then reports back 'No changes' and so the job stops. I expected that at this point the job would continue because there are changes. Of course if there were no changes I would expect it to stop the job at this point.
The Mercurial Polling Log shows the changes have been pulled but hg update has not been run. This is confirmed by looking at the Jenkins-created-repo in the Tortoise Workbench. However, even putting a hook in Mercurial configuration in Jenkins such that it does update after pulling does not fix the issue. Jenkins still reports 'No changes' after the checkout step.
So after the initial build, which works fine, Jenkins never runs the job all the way through again. It always bails out after the checkout step because despite pulling down any changes it reports there are none.
I have checked the permissions and everything seems fine, the fact that it completely runs on demand and can subsequently pull also suggests there is nothing wrong with the pipeline or elsewhere.
Any suggestions as to how to get the job to continue after the checkout stage would be much appreciated.
Below is an example pipeline script:
pipeline {
agent any
triggers {
pollSCM '*/10 * * * *'
}
stages {
stage('Checkout') {
steps {
checkout changelog: true, poll: true, scm: [$class: 'MercurialSCM', credentialsId: 'xxx', installation: 'TortoiseHg', source: 'C:\\Path\\to\\repo']
}
}
stage('Build') {
steps {
echo Continuing with build...
}
}
}
UPDATE I have noticed that if I manually build after the poll has pulled down but not updated the local repository the job then updates the repository and the build continues as normal.
It looks like you have omitted to specify a branch in your scm checkout. Are you using the default branch in HG?
This Jenlins issue and the source code show that if you are not using the deault branch, you'll likely not see your changes pulled down even if the references are there.
If you are trying to build a non default branch give this a try
checkout changelog: true, poll: true, scm: [$class: 'MercurialSCM', branches: [[name: '*/yourbranch']], credentialsId: 'xxx', installation: 'TortoiseHg', source: 'C:\\Path\\to\\repo']
I have a Jenkinsfile located in [my svn branch]\build folder, and it checks out code to the slave node and builds.
My multi branch project finds the branch correctly, but it checks out the entire svn branch on the master just to read the jenkinsfile instead of checking out just the jenkinsfile itself of just [my svn branch]\build folder.
This is a major problem because of storage and performance, are there any solutions for that?
In your multibranch pipeline config in 'include' field type: branches/*/build (i assume that you have all svn branches in folder 'branches', and url to your build folder is something like: svn_url/branches/my_new_branch/build)
Then it will scan only build folder in each branch.
Warning - after changing that config property your multibranch pipeline will only diacover 'build', if you want to index other build folders, you can list them in that property, i.e.:
Include: trunk/build, trunk/other_build, branches/*/build, branches/*/other_build
But more clean approach is to get only one build per multibranch pipeline
I have Jenkins LTS 2.60.2 on Windows Server 2016 and using these plugins:
Folders plugin (6.1.0)
Copy Artifact plugin (1.38.1)
Pipeline plugin (2.5) + all dependent pipeline sub-plugins
Various other dependent plugins...
See Pipeline to use artifacts from 2 projects associated by the same git branch name for more details about my setup, but to sum it up I have these items:
playground (a folder created with the Folders plugin to group all these following items)
frontend (multibranch pipeline)
backend (multibranch pipeline)
configure (pipeline with a parameter called BRANCH_NAME)
The frontend and backend git repos, both have a branch called master and one called release/2017.2.
The idea is to call the configure pipeline automatically after each successful build, passing the git branch name. Automatically triggering the configure pipeline works.
What doesn't work and I need your help to fix, is the step inside the configure pipeline to copy the artifacts from a multibranchPipeline/specificBranch.
If for the BRANCH_NAME parameter (or the upstream pipeline) is master it works. If BRANCH_NAME is: release/2017.2 I get this error:
ERROR: Unable to find project for artifact copy:
playground/frontend/release%2f2017.2 This may be due to incorrect project
name or permission settings; see help for project name in job
configuration. Finished: FAILURE
The configure pipeline looks like this:
node {
stage('Prepare') {
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2f")
step([$class: 'CopyArtifact', projectName: "${projectname}", selector: [$class: 'StatusBuildSelector', stable: false]])
}
stage('Archive') {
archiveArtifacts '**'
}
}
As you can see I already replace / with %2f (it's needed).
If I don't use the "playground" folder (all my pipelines as is, not inside a folder item), it works. If I use the folder and use the master branch, it works. It doesn't work if I use the folder and a branch name like 2017.2. What am I doing wrong? Can you help making it work? Of well if it's a bug (I searched in https://issues.jenkins-ci.org and found some bugs where a similar setup with folder doesn't work, but they have been fixed... so I really wonder...) in the copy artifact plugin, please file the bug and share the link here, so we can all monitor its progress...
Thank you.
I finally found the issue. The configure pipeline was failing to find a branch with a slash because the encoding was incorrect.
So, in my question, in the configure pipeline:
this (replace / with %2f) is wrong and generates the error:
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2f")
this is the proper way to encode the slash, and it works:
def projectname = "playground/frontend/" + URLEncoder.encode("${BRANCH_NAME}", "UTF-8").replace("+", "%20")
Credits to: http://www.pipegrep.se/copy-artifacts-from-jenkins-pipeline-jobs.html
UPDATE: actually, I investigated a bit further and added echo "${projectname}" just before step, with the previous and fixed projectname, and I noticed that the difference was %2f lowercase.
Uppercase, like this: %2F works:
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2F")
So, the fixed configure pipeline looks like this (I kept my replace function, which was enough for my case):
node {
stage('Prepare') {
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2F")
step([$class: 'CopyArtifact', projectName: "${projectname}", selector: [$class: 'StatusBuildSelector', stable: false]])
}
stage('Archive') {
archiveArtifacts '**'
}
}
I created a sample project to try and recreate what you were seeing, and I was able to do so, after a fashion, except that the build that I was having trouble on was master instead of release/2017.2. Eventually, I realized that I was doing the build job incorrectly from the frontend project, and it was giving me the same error as you because I hadn't ever completed a successful build of the frontend/master branch (I had completed a successful build of the release/2017.2 branch because I didn't have it triggering the configure build initially, so it didn't give me the same error once I did configure it to trigger the configure build).
What worked was changing the build job in the frontend Jenkinsfile to this:
build job: 'playground/configure', parameters: [[$class: 'StringParameterValue', name: 'BRANCH_NAME', value: env.BRANCH_NAME]], quietPeriod: 2, wait: false
Adding in the quietPeriod gives a couple seconds of quiet time between completing the previous job (I'm not certain that this is critical, but it seems like it might be a nice fail-safe, to try and make sure there's enough time for the build to complete), but the important part is the wait: false, which instructs Jenkins that this build shouldn't wait for the triggered build to complete. Once I changed that, the frontend/master branch completed successfully, and the configure build that it triggered also completed successfully.
Hopefully this helps. I was able to get both my master and release/2017.2 branches to build properly, so I don't believe there's any intrinsic problem with the / in the project name. You can see my simple Jenkinsfiles in the referenced repo, and I used the same pipeline script as you posted in your question.