I am trying to activate a pipeline on any merge request change. This works as long as my pipeline script is in Jenkins UI. Now I outsourced my script on GitLab, and the checkout should happen via the pipeline via scm option.
But all I get on build (yes, it triggers) is:
java.lang.IllegalArgumentException: Invalid refspec refs/heads/**
This happens if I leave the branch specifier empty, this is because I want to listen to any change. If I specify the branch, the build goes through.
My refspec:
+refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*
Most likely this is a Jenkins bug. https://issues.jenkins-ci.org/browse/JENKINS-46588
There seems to a solution anyway:
In your project configuration under Pipeline -> SCM -> Branches to build -> "Branch Specifier (blank for 'any'):
Do not use blank for any or * or .* or **. Use:
*/*
Another workaround would be to disable Lightweight Checkout.
PS:
Big thanks to ChrisAnnODell and Omurbek Kadyrbekov for linking the solutions in first place.
I'm still a bit puzzled that there is no fix for over 2 years now...
Related
In my Jenkins project, I've configured "Pipeline script from SCM" with Subversion as the SCM, a Repository URL corresponding to a repository on our test VisualSVN server (for sake of discussion, call it https://foo.bar.com/svn/Kofax/) and a Script Path of "Jenkins files/Jenkinsfile".
When I trigger a build, I get the following console output. The repository path appears to have been improperly concatenated (e.g. it has two copies of 'svn'), but I don't know where the extraneous characters are coming from:
Started by remote host
org.tmatesoft.svn.core.SVNException: svn: E160013: '/svn/Kofax/!svn/bc/10/Jenkins%20files/Jenkinsfile' path not found: 404 Not Found (https://foo.bar.com)
I've tried removing the repository name from the Repository URL, but then Jenkins says it can't connect to the repository (as you might expect). I've also tried using an underscore rather than a space in the Script Path.
Any suggestions would be appreciated.
Unchecking the Lightweight Checkout option allowed the build to continue, so I think this might be a bug in Jenkins.
Also it appears that the repository name has to be part of the Script Path, so in my case the correct Script Path was Kofax/Jenkins_files/JenkinsFile.
Hope this helps some other Jenkins newbie.
We have branches set up like this:
origin/develop
origin/release/1.0.0
origin/release/1.1.0
Bitbucket
In the BitBucket, we have added Hook called "Bitbucket Server Webhooks to Jenkins" & Automatic merging is enabled.
Under the Advanced Configuration:
Branch Options is set to All and refs/remotes/origin/develop refs/remotes/origin/release/* in the input box.
Jenkins
For Jenkins, under "Source Code Management" section and "Branches to Build" option we have added the following:
Branch Specific (blank for 'any'): ${Branch}
Branch Specific (blank for 'any'): **release/*
Problem
Jenkins picks up build when we push to develop. Nothing happens when we push to any of the release branches.
Also tried manually: http://server/job/Jenkins%20Bitbucket%20Test/buildWithParameters?Branch=release/1.0.0, but this triggers a build for release/1.1.0 and not release 1.0.0.
The syntax is of the form: REPOSITORYNAME/BRANCH. In
addition, BRANCH is recognized as a shorthand of */BRANCH, * is
recognized as a wildcard, and ** is recognized as wildcard that
includes the separator /. Therefore, origin/branches* would match
origin/branches-foo but not origin/branches/foo, while
origin/branches** would match both origin/branches-foo and
origin/branches/foo.
Therefore my **release/* needs to be *release/**
I seem unable to create a Jenkins Pipeline job that builds a specific branch, where that branch is a build parameter.
Here's some configuration screenshots:
(i've tried with a Git Parameter and a String Parameter, same outcome)
(I've tried $BRANCH_NAME_PARAM, ${BRANCH_NAME_PARAM} and ${env.BRANCH_NAME_PARAM}, same outcome for all variations)
And the build log:
hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/${BRANCH_NAME_PARAM}:refs/remotes/origin/${BRANCH_NAME_PARAM} --prune" returned status code 128:
stdout:
stderr: fatal: Couldn't find remote ref refs/heads/${BRANCH_NAME_PARAM}
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1970)
I'm obviously doing something wrong - any ideas on what?
https://issues.jenkins-ci.org/plugins/servlet/mobile#issue/JENKINS-28447
Appears that its something to do with a lightweight checkout. if i deselect this option in my config, my parameter variables are resolved
a bit more detailed with examples combined with VonC answer
1. Configure extended choice parameter named BRANCH:
specify delimiter
specify groovy script or path to groovy file:
def command = "git ls-remote -h $gitURL"
def proc = command.execute()
proc.waitFor()
if ( proc.exitValue() != 0 ) {
println "Error, ${proc.err.text}"
System.exit(-1)
}
def branches = proc.in.text.readLines().collect {
it.replaceAll(/[a-z0-9]*\trefs\/heads\//, '')
}
return branches.join(",")
2. Set Branches to build: $BRANCH
3. Disable "Lightweight checkout" checkbox in the "Pipeline" secton of Jenkins job configuration:
Otherwise job will fail with following message:"stderr: fatal: Couldn't find remote ref refs/heads/${BRANCH"}"
4. Build with parameter executes groovy script and you will then get a dropdown list of branches
I have tried the above solution but it didn't work for me. I have chosen a slightly different approach. I am posting because it will help someone in future.
Goto configures the pipeline job.
Check the option "This project is parameterized"
Add git paramter.
Note: If it doesn't show the option, please goto manage plugins and install git parameter plugin.
My pipeline Configure looks like
Uncheck lightweight checkout and update the "branch to build" in pipeline section.
Save the configuration.
Each time I had a job based on a branch, I had to put a groovy script with EnvInject plugin in order to remove the ref/heads part of the git branch selected in parameters.
If you keep refs/heads/xxx, Jenkins will look for the branch ref/heads/ref/heads/xxx.
I have Jenkins LTS 2.60.2 on Windows Server 2016 and using these plugins:
Folders plugin (6.1.0)
Copy Artifact plugin (1.38.1)
Pipeline plugin (2.5) + all dependent pipeline sub-plugins
Various other dependent plugins...
See Pipeline to use artifacts from 2 projects associated by the same git branch name for more details about my setup, but to sum it up I have these items:
playground (a folder created with the Folders plugin to group all these following items)
frontend (multibranch pipeline)
backend (multibranch pipeline)
configure (pipeline with a parameter called BRANCH_NAME)
The frontend and backend git repos, both have a branch called master and one called release/2017.2.
The idea is to call the configure pipeline automatically after each successful build, passing the git branch name. Automatically triggering the configure pipeline works.
What doesn't work and I need your help to fix, is the step inside the configure pipeline to copy the artifacts from a multibranchPipeline/specificBranch.
If for the BRANCH_NAME parameter (or the upstream pipeline) is master it works. If BRANCH_NAME is: release/2017.2 I get this error:
ERROR: Unable to find project for artifact copy:
playground/frontend/release%2f2017.2 This may be due to incorrect project
name or permission settings; see help for project name in job
configuration. Finished: FAILURE
The configure pipeline looks like this:
node {
stage('Prepare') {
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2f")
step([$class: 'CopyArtifact', projectName: "${projectname}", selector: [$class: 'StatusBuildSelector', stable: false]])
}
stage('Archive') {
archiveArtifacts '**'
}
}
As you can see I already replace / with %2f (it's needed).
If I don't use the "playground" folder (all my pipelines as is, not inside a folder item), it works. If I use the folder and use the master branch, it works. It doesn't work if I use the folder and a branch name like 2017.2. What am I doing wrong? Can you help making it work? Of well if it's a bug (I searched in https://issues.jenkins-ci.org and found some bugs where a similar setup with folder doesn't work, but they have been fixed... so I really wonder...) in the copy artifact plugin, please file the bug and share the link here, so we can all monitor its progress...
Thank you.
I finally found the issue. The configure pipeline was failing to find a branch with a slash because the encoding was incorrect.
So, in my question, in the configure pipeline:
this (replace / with %2f) is wrong and generates the error:
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2f")
this is the proper way to encode the slash, and it works:
def projectname = "playground/frontend/" + URLEncoder.encode("${BRANCH_NAME}", "UTF-8").replace("+", "%20")
Credits to: http://www.pipegrep.se/copy-artifacts-from-jenkins-pipeline-jobs.html
UPDATE: actually, I investigated a bit further and added echo "${projectname}" just before step, with the previous and fixed projectname, and I noticed that the difference was %2f lowercase.
Uppercase, like this: %2F works:
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2F")
So, the fixed configure pipeline looks like this (I kept my replace function, which was enough for my case):
node {
stage('Prepare') {
def projectname = "playground/frontend/" + "${BRANCH_NAME}".replace("/", "%2F")
step([$class: 'CopyArtifact', projectName: "${projectname}", selector: [$class: 'StatusBuildSelector', stable: false]])
}
stage('Archive') {
archiveArtifacts '**'
}
}
I created a sample project to try and recreate what you were seeing, and I was able to do so, after a fashion, except that the build that I was having trouble on was master instead of release/2017.2. Eventually, I realized that I was doing the build job incorrectly from the frontend project, and it was giving me the same error as you because I hadn't ever completed a successful build of the frontend/master branch (I had completed a successful build of the release/2017.2 branch because I didn't have it triggering the configure build initially, so it didn't give me the same error once I did configure it to trigger the configure build).
What worked was changing the build job in the frontend Jenkinsfile to this:
build job: 'playground/configure', parameters: [[$class: 'StringParameterValue', name: 'BRANCH_NAME', value: env.BRANCH_NAME]], quietPeriod: 2, wait: false
Adding in the quietPeriod gives a couple seconds of quiet time between completing the previous job (I'm not certain that this is critical, but it seems like it might be a nice fail-safe, to try and make sure there's enough time for the build to complete), but the important part is the wait: false, which instructs Jenkins that this build shouldn't wait for the triggered build to complete. Once I changed that, the frontend/master branch completed successfully, and the configure build that it triggered also completed successfully.
Hopefully this helps. I was able to get both my master and release/2017.2 branches to build properly, so I don't believe there's any intrinsic problem with the / in the project name. You can see my simple Jenkinsfiles in the referenced repo, and I used the same pipeline script as you posted in your question.
We want to use Jenkins to generate releases/deployments on specific project milestones. Is it possible to trigger a Jenkins Pipeline (defined in a Jenkinsfile or Groovy script) when a tag is pushed to a Git repository?
We host a private Gitlab server, so Github solutions are not applicable to our case.
This is currently something that is sorely lacking in the pipeline / multibranch workflow. See a ticket around this here: https://issues.jenkins-ci.org/browse/JENKINS-34395
If you're not opposed to using release branches instead of tags, you might find that to be easier. For example, if you decided that all branches that start with release- are to be treated as "release branches", you can go...
if( env.BRANCH_NAME.startsWith("release-") ) {
// groovy code on release goes here
}
And if you need to use the name that comes after release-, such as release-10.1 turning into 10.1, just create a variable like so...
if( env.BRANCH_NAME.startsWith("release-") ) {
def releaseName = env.BRANCH_NAME.drop(8)
}
Both of these will probably require some method whitelisting in order to be functional.
I had the same desire and rolled my own, maybe not pretty but it worked...
In your pipeline job, mark that "This project is parameterized" and add a parameter for your tag. Then in the pipeline script checkout the tag if it is present.
Create a freestyle job that runs a script to:
Checkout
Run git describe --tags --abbrev=0 to get the latest tag.
Check that tag against a running list of builds (like in a file).
If the build hasn't occurred, trigger the pipeline job via a url passing your tag as a parameter (in your pipeline job under "Build Triggers" set "Trigger builds remotely (e.g. from scripts) and it will show the correct url.
Add the tag to your running list of builds so it doesn't get triggered again.
Have this job run frequently.
if you use multibranch pipeline, there is a discover tag. Use that plus Spencer solution