When promoting a build, I want to expose some choices of how/what to promote. To do this properly, I need some info about the build which I am promoting, and I'd really like to use the Extended Choice Parameter to do this.
How can I access a build's parameters in an Extended Choice Parameter block in a build promotion step? For example, say the build I want to promote was built with a parameter myvar="1,2,5", and I want to let the build promoter select 1, 2, 5, or some combination. How can I access that particular build's myvar value, in an extended choice parameter, in a promotion process?
I can auto-populate these with available Git branches for a certain repo, with global environment variables, but I haven't seen the groovy magic I need to access info about the build I'm (about to) promote.
This was not obvious to me, but I found that environment variables such as PROMOTED_URL, PROMOTED_JOB_NAME, etc, were not available until the promotion steps were in progress. I need this information exposed when the promotion parameters are being populated via Extended Choice Parameter's Groovy Script fields. I accomplished it by peeking into the thread name:
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
def jenkins = Jenkins.instance
def thread = Thread.currentThread().toString()
def (jobName,buildNumber) = thread.replaceAll(/.*GET \/job\/(.*)\/promotion\/.*/, '$1').split('/')
def job = jenkins.getJob(jobName)
def build = job.getBuild(buildNumber)
return build.getBuildVariables()["SOME_BUILD_PARAM"]
What do I use this for? Say I can select certain targets to build but not others, when building. I want to be able to promote only some subset of those that were actually built, so the parameter can't just hold a default value. Pretty narrow edge-case, I'll grant, but sometimes it seems like my entire life, as far as Jenkins is concerned, is dealing with some edge-case or another. If there is a more elegant way, I am all ears!
building on the #courtlandj solution (with the latest hudson API):
import hudson.model.*
def paramToLookup = "SOME_BUILD_VARIABLE_NAME"
// you can of course parse the jobName from the currentThread
// but that is a case-by-case basis depending on how you name them
// build numbers are strictly numeric - and more predictable to parse with regex
def jobName = "some-jenkins-full-job-name"
int buildNumber = 0
// Similar approach to courtlandj to get THIS build number (of page being viewed):
def thisPromotionInfo = Thread.currentThread().toString()
// groovy regex that gets the promotion digits (\d)
def buildRegex = /.*\/(\d+)\/promotion\/.*/
def matcher = ( thisPromotionInfo =~ buildRegex )
if (matcher.matches()) {
buildNumber = matcher[0][1].toInteger()
} else {
return "unable to identify buildNumber in Extended Choice Parameter (see Jenkins Config)"
}
// borrowed slightly from here: https://wiki.jenkins-ci.org/display/JENKINS/Display+job+parameters
try {
def job = Hudson.instance.getItemByFullName(jobName)
def build = job.getBuildByNumber(buildNumber)
return build.getProperties()["buildVariableResolver"].resolve(paramToLookup).toString()
} catch(Exception ex) {
return ex.toString()
}
relevant links I borrowed from:
https://groups.google.com/forum/#!topic/jenkinsci-users/cHClVbjuSN4
http://groovy.jmiguel.eu/groovy.codehaus.org/Tutorial+5+-+Capturing+regex+groups.html
http://hudson-ci.org/javadoc/hudson/model/Hudson.html
Related
I'm implementing functionality to expose the triggering user for a Jenkins pipeline to our CD system, so I've grabbed the build user vars plugin: https://plugins.jenkins.io/build-user-vars-plugin/
The way the plugin seems to work is that you wrap the code you need the variables exposed to, like so:
wrap([$class: 'BuildUser']) {
userId = env.BUILD_USER_ID
}
I tried this on a general pipeline and just echoed it out, all was well.
I then tried implementing it in our shared library so that this would happen on all calls to CD, and I'm hitting an error.
wrap([$class: 'BuildUser']) {
jobBuildUrl ="${jobBuildUrl}&USER_ID=${env.BUILD_USER_ID}"
}
[2021-08-19T10:20:22.852Z] hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: com.company.jenkins.pipelines.BuildManager.wrap() is applicable for argument types: (java.util.LinkedHashMap, org.jenkinsci.plugins.workflow.cps.CpsClosure2) values: [[$class:BuildUser], org.jenkinsci.plugins.workflow.cps.CpsClosure2#1c9a210c]
Is there any way to use this plugin code within a shared library? If so how?
I'm lead to believe not, but I thought it was worth the asking. For reference, there is this outstanding issue: https://issues.jenkins.io/browse/JENKINS-44741
Side note, I'm trying to do this without touching everybody's pipelines themselves. If this is impossible with this plugin, I'll probably just implement my own version of it within the shared lib.
That plugin is hard to use and has many issues, one of them is with shared libraries.
Instead just implement it by yourself, it is relatively easy and allows you much more control over the logic of what is done, error handling and which parameters you return.
You can use something like the following:
/**
* Get the last upstream build that triggered the current build
* #return Build object (org.jenkinsci.plugins.workflow.job.WorkflowRun) representing the upstream build
*/
#NonCPS
def getUpstreamBuild() {
def build = currentBuild.rawBuild
def upstreamCause
while (upstreamCause = build.getCause(hudson.model.Cause$UpstreamCause)) {
build = upstreamCause.upstreamRun
}
return build
}
/**
* Get the properties of the build Cause (the event that triggered the build)
* #param upstream If true (Default) return the cause properties of the last upstream job (If the build was triggered by another job or by a chain of jobs)
* #return Map representing the properties of the user that triggered the current build.
* Contains keys: USER_NAME, USER_ID
*/
#NonCPS
def getUserCauseProperties(Boolean upstream = true) {
def build = upstream ? getUpstreamBuild() : currentBuild.rawBuild
def cause = build.getCause(hudson.model.Cause$UserIdCause)
if (cause) {
return ['USER_NAME': cause.userName, 'USER_ID': cause.userId]
}
println "Job was not started by a user, it was ${build.getCauses()[0].shortDescription}"
return [:]
}
I was also not able to use wrap([$class: 'BuildUser']) into groovy method. Therefore I think I found alternative way which seems to do the trick. You may get the username and userid (which might return email, depends on your setup) by just the following:
def build = currentBuild.rawBuild
def cause = build.getCause(hudson.model.Cause.UserIdCause.class)
def userName = cause.getUserName() // or use getUserId()
Original situation:
I have a job in Jenkins that is running an ant script. I easily managed to test this ant script on more then one software version using a "Multi-configuration project".
This type of project is really cool because it allows me to specify all the versions of the two software that I need (in my case Java and Matlab) an it will run my ant script with all the combinations of my parameters.
Those parameters are then used as string to be concatenated in the definition of the location of the executable to be used by my ant.
example: env.MATLAB_EXE=/usr/local/MATLAB/${MATLAB_VERSION}/bin/matlab
This is working perfectly but now I am migrating this scripts to a pipline version of it.
Pipeline migration:
I managed to implement the same script in a pipeline fashion using the Parametrized pipelines pluin. With this I achieve the point in which I can manually select which version of my software is going to be used if I trigger the build manually and I also found a way to execute this periodically selecting the parameter I want at each run.
This solution seems fairly working however is not really satisfying.
My multi-config project had some feature that this does not:
With more then one parameter I can set to interpolate them and execute each combination
The executions are clearly separated and in build history/build details is easy to recognize which settings hads been used
Just adding a new "possible" value to the parameter is going to spawn the desired executions
Request
So I wonder if there is a better solution to my problem that can satisfy also the point above.
Long story short: is there a way to implement a multi-configuration project in jenkins but using the pipeline technology?
I've seen this and similar questions asked a lot lately, so it seemed that it would be a fun exercise to work this out...
A matrix/multi-config job, visualized in code, would really just be a few nested for loops, one for each axis of parameters.
You could build something fairly simple with some hard coded for loops to loop over a few lists. Or you can get more complicated and do some recursive looping so you don't have to hard code the specific loops.
DISCLAIMER: I do ops much more than I write code. I am also very new to groovy, so this can probably be done more cleanly, and there are probably a lot of groovier things that could be done, but this gets the job done, anyway.
With a little work, this matrixBuilder could be wrapped up in a class so you could pass in a task closure and the axis list and get the task map back. Stick it in a shared library and use it anywhere. It should be pretty easy to add some of the other features from the multiconfiguration jobs, such as filters.
This attempt uses a recursive matrixBuilder function to work through any number of parameter axes and build all the combinations. Then it executes them in parallel (obviously depending on node availability).
/*
All the config axes are defined here
Add as many lists of axes in the axisList as you need.
All combinations will be built
*/
def axisList = [
["ubuntu","rhel","windows","osx"], //agents
["jdk6","jdk7","jdk8"], //tools
["banana","apple","orange","pineapple"] //fruit
]
def tasks = [:]
def comboBuilder
def comboEntry = []
def task = {
// builds and returns the task for each combination
/* Map the entries back to a more readable format
the index will correspond to the position of this axis in axisList[] */
def myAgent = it[0]
def myJdk = it[1]
def myFruit = it[2]
return {
// This is where the important work happens for each combination
node(myAgent) {
println "Executing combination ${it.join('-')}"
def javaHome = tool myJdk
println "Node=${env.NODE_NAME}"
println "Java=${javaHome}"
}
//We won't declare a specific agent this part
node {
println "fruit=${myFruit}"
}
}
}
/*
This is where the magic happens
recursively work through the axisList and build all combinations
*/
comboBuilder = { def axes, int level ->
for ( entry in axes[0] ) {
comboEntry[level] = entry
if (axes.size() > 1 ) {
comboBuilder(axes[1..-1], level + 1)
}
else {
tasks[comboEntry.join("-")] = task(comboEntry.collect())
}
}
}
stage ("Setup") {
node {
println "Initial Setup"
}
}
stage ("Setup Combinations") {
node {
comboBuilder(axisList, 0)
}
}
stage ("Multiconfiguration Parallel Tasks") {
//Run the tasks in parallel
parallel tasks
}
stage("The End") {
node {
echo "That's all folks"
}
}
You can see a more detailed flow of the job at http://localhost:8080/job/multi-configPipeline/[build]/flowGraphTable/ (available under the Pipeline Steps link on the build page.
EDIT:
You can move the stage down into the "task" creation and then see the details of each stage more clearly, but not in a neat matrix like the multi-config job.
...
return {
// This is where the important work happens for each combination
stage ("${it.join('-')}--build") {
node(myAgent) {
println "Executing combination ${it.join('-')}"
def javaHome = tool myJdk
println "Node=${env.NODE_NAME}"
println "Java=${javaHome}"
}
//Node irrelevant for this part
node {
println "fruit=${myFruit}"
}
}
}
...
Or you could wrap each node with their own stage for even more detail.
As I did this, I noticed a bug in my previous code (fixed above now). I was passing the comboEntry reference to the task. I should have sent a copy, because, while the names of the stages were correct, when it actually executed them, the values were, of course, all the last entry encountered. So I changed it to tasks[comboEntry.join("-")] = task(comboEntry.collect()).
I noticed that you can leave the original stage ("Multiconfiguration Parallel Tasks") {} around the execution of the parallel tasks. Technically now you have nested stages. I'm not sure how Jenkins is supposed to handle that, but it doesn't complain. However, the 'parent' stage timing is not inclusive of the parallel stages timing.
I also noticed is that when a new build starts to run, on the "Stage View" of the job, all the previous builds disappear, presumably because the stage names don't all match up. But after the build finishes running, they all match again and the old builds show up again.
And finally, Blue Ocean doesn't seem to vizualize this the same way. It doesn't recognize the "stages" in the parallel processes, only the enclosing stage (if it is present), or "Parallel" if it isn't. And then only shows the individual parallel processes, not the stages within.
Points 1 and 3 are not completely clear to me, but I suspect you just want to use “scripted” rather than “Declarative” Pipeline syntax, in which case you can make your job do whatever you like—anything permitted by matrix project axes and axis filters and much more, including parallel execution. Declarative syntax trades off syntactic simplicity (and friendliness to “round-trip” editing tools and “linters”) for flexibility.
Point 2 is about visualization of the result, rather than execution per se. While this is a complex topic, the usual concrete request which is not already supported by existing visualizations like Blue Ocean is to be able to see test results distinguished by axis combination. This is tracked by JENKINS-27395 and some related issues, with design in progress.
I am using the github-organization plugin to manage jenkins jobs from github but I discovered that Jenkins API does not report these builds.
In fact the API list the entire organization as a single job.
How can I build a specific repository and branch using the API?
To be clear, I am looking for some groovy code to add inside the Jenkinsfile
#!groovy
stage 'test-downstream'
node {
def job = build job: 'some-job'
}
Now, the problem is that Jenkins is seeing the entire organization as a single job!
If I use Jenkins API to retrieve the jobs, it will return only the organization, and not all the repositories and jobs inside it.
I suspect that's because the way this plugin was implemented and I suppose that I need to give some extra parameters in order to specify which repository and branch I want to build inside the organization.... building an organization does not make much sense.
The question is vague, but I am guessing “API” in this context means the REST API to trigger builds. You can use for example
curl -X POST -u user:apitoken http://jenkins/job/yourorg/job/yourrepo/job/master/build
The following code trigger job via System Groovy build step. Please note that system groovy always run on master so passing info from previous build steps might be tricky.
import jenkins.model.*
import hudson.model.*
import java.util.concurrent.*
def run_job(job_name) {
def currentBuild = Thread.currentThread().executable
def jenkins = jenkins.model.Jenkins.getInstance();
def job = jenkins.getItemByFullName(job_name);
if (job == null)
throw new hudson.AbortException("Cannot find job:" + job_name);
def params =[
new StringParameterValue('PARAMETER1', "invoke 1 param1"),
new StringParameterValue('PARAMETER2', ",invoke 1 param2")
]
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
def future_build = job.scheduleBuild2(0,causeAction,paramsAction);
def running_build = future_build.waitForStart()
return running_build
}
run_job("runner1")
I try to launch a job from a parametrized trigger and I would compute the name from a given variable.
Is it possible to set in field :
Build Triggers Projects to build
a value like this
${RELEASE}-MAIN-${PROJECT}-LOAD_START
?
Unfortunately, this isn't possible with the Build Triggers. I looked for a solution for this "higher order build job" that would allow you to create a dynamic build name with a one of the parameterized build plugins, but I couldn't find one.
However, using the Groovy Postbuild Plugin, you can do a lot of powerful things. Below is a script that can be modified to do what you want. In particular, notice that it gets environmental variables using build.buildVariables.get("MY_ENV_VAR"). The environmental variable TARGET_BUILD_JOB specifies the name of the build job to build. In your case, you would want to build TARGET_BUILD_JOB using these two environmental variables:
build.buildVariables.get("RELEASE")
build.buildVariables.get("PROJECT")
The script is commented so that if you're not familiar with Groovy, which is based off Java, it should hopefully make sense!
import hudson.model.*
import hudson.model.queue.*
import hudson.model.labels.*
import org.jvnet.jenkins.plugins.nodelabelparameter.*
def failBuild(msg)
{
throw new RuntimeException("[GROOVY] User message, exiting with error: " + msg)
}
// Get the current build job
def thr = Thread.currentThread()
def build = thr?.executable
// Get the parameters for the current build job
// For ?:, see "Elvis Operator" (http://groovy.codehaus.org/Operators#Operators-ElvisOperator)
def currentParameters = build.getAction(ParametersAction.class)?.getParameters() ?:
failBuild("There are no parameters to pass down.")
def nodeName = build.getBuiltOnStr()
def newParameters = new ArrayList(currentParameters); newParameters << new NodeParameterValue("param_NODE",
"Target node -- the node of the previous job", nodeName)
// Retrieve information about the target build job
def targetJobName = build.buildVariables.get("TARGET_BUILD_JOB")
def targetJobObject = Hudson.instance.getItem(targetJobName) ?:
failBuild("Could not find a build job with the name $targetJobName. (Are you sure the spelling is correct?)")
println("$targetJobObject, $targetJobName")
def buildNumber = targetJobObject.getNextBuildNumber()
// Add information about downstream job to log
def jobUrl = targetJobObject.getAbsoluteUrl()
println("Starting downstream job $targetJobName ($jobUrl)" + "\n")
println("======= DOWNSTREAM PARAMETERS =======")
println("$newParameters")
// Start the downstream build job if this build job was successful
boolean targetBuildQueued = targetJobObject.scheduleBuild(5,
new Cause.UpstreamCause(build),
new ParametersAction(newParameters)
);
if (targetBuildQueued)
{
println("Build started successfully")
println("Console (wait a few seconds before clicking): $jobUrl/$buildNumber/console")
}
else
failBuild("Could not start target build job")
I need to pass the username of the requester of a build down to the script that is actually doing the work. Looking at the console output for a particular build, the first line is always "Started by user foo," so Jenkins is clearly keeping track of who triggered the build. So it should be possible to pass that information down to the job. The question is, how?
user30997
Please check out Jenkins Build User Vars plugin, it does what you need:
It is used to set following user build variables:
BUILD_USER – full name of user started build,
BUILD_USER_FIRST_NAME – first name of user started build,
BUILD_USER_LAST_NAME – last name of user started build,
BUILD_USER_ID – id of user started build.
The username isn't put in an easy-to-fetch environment variable, but you can get it using the xml (or json or python) api - as soon as you start a build, http://[jenkins-server]/job/[job-name]/[build-number]/api/xml is populated with details:
<freeStyleBuild>
<action>
<cause>
<shortDescription>Started by user foobar</shortDescription>
<userName>foobar</userName>
</cause>
</action>
<building>true</building>
[...]
I tried to use Jenkins Build User Vars plugin and notify a HipChat room that a build was started by a certain user, but BUILD_USER variable was not available to HipChat plugin, possibly because HipChat action happened before Build User Vars plugin injects the variable.
So I installed pre-scm-buildstep plugin and added:
]
// Inject environment variables using Groovy
import hudson.model.*
def build = Thread.currentThread().executable
def userCause = build.getCause(hudson.model.Cause$UserIdCause)
def userName = userCause?.userId ?: 'Jenkins'
def envVars = ['BUILD_USER': userName]
for (item in envVars) {
build.addAction(new ParametersAction([
new StringParameterValue(item.key, item.value)
]))
}
In your Job add "Execute system Groovy script":
def yourUserName = build.causes[0].userId
I managed to get it (on Jenkins 2.58):
currentBuild.getRawBuild().getCauses()[0].getUserId()
Of course you need to set permissions in Jenkins to be able to call these methods.
It's not always the 0th Cause object you are looking for, e.g. it may be another one if you replay another user's build (did not test this).
import os
import jenkinsapi.build
import jenkinsapi.jenkins
#Required Environment variables example:
#JENKINS_URL=http://jenkinsserver/
#JOB_NAME=TEST_GT
#BUILD_NUMBER=8
jenkins_inst = None
def get_jenkins_inst():
if jenkins_inst == None:
jenkins_url = os.environ['JENKINS_URL']
print("Connect to jenkins " + jenkins_url)
jenkins_inst = jenkinsapi.jenkins.Jenkins(jenkins_url)
return jenkins_inst
def get_jenkins_job():
jenkins_inst = get_jenkins_inst()
jenkins_job_name = os.environ['JOB_NAME']
print("Get jenkins job " + jenkins_job_name)
return jenkins_inst.get_job(jenkins_job_name)
def get_jenkins_user():
jenkins_job = get_jenkins_job()
jenkins_buildno = int(os.environ['BUILD_NUMBER'])
print("Get jenkins job build " + str(jenkins_buildno))
cur_build = jenkins_job.get_build(jenkins_buildno)
return cur_build.get_actions()['causes'][0]['userId']