I'm trying to construct a convenient shared library for my scripted pipelines.
I want to be able to use an object from my Jenkinsfile that has several functional methods, but also stores some initial variables that are referenced by several methods.
So, I've defined a file in "vars" with a bunch of methods for functionality, but also a handful of get/set methods for the properties I want to store in the object for reference by several methods.
My initial testing is just to set some of the variables (not all of them) and call the "toString()" method which I've defined in the file (just for convention, obviously).
In my test Jenkinsfile, If I do set all of the variables and then call "toString()", it works fine and completes.
However, if I try commenting out one of the variable initializations, when it hits the line in the "toString" method that constructs the return value, Jenkins just hangs forever. I eventually just kill the job.
I've been able to avoid the hang by qualifying each reference with 'if binding.variables.containsKey("foo")) {', only referencing the variable if that is true. I made this a tiny bit cleaner by putting the binding check in each of the getter methods instead.
I don't really like this workaround. It just seems odd that I would have to do this.
I've tried several variations, but I don't set the variable and I try to reference it in a gstring, the job hangs forever, every time.
This is a brief excerpt from the "vars" file:
def setJobName(value) { jobName = value; }
//def getJobName() { return jobName }
def getJobName() { return (binding.variables.containsKey("jobName") ? jobName : "") }
def setMechIdCredentials(value) { mechIdCredentials = value; }
//def getMechIdCredentials() { return mechIdCredentials }
def getMechIdCredentials() { return (binding.variables.containsKey("mechIdCredentials") ? mechIdCredentials : "") }
... more get/set methods
String toString() {
echo "Inside uslutils.toString()."
This is an excerpt from the Jenkinsfile that uses this:
uslutils.currentBuild = currentBuild
uslutils.jobName = env.JOB_NAME
uslutils.buildURL = env.BUILD_URL
//uslutils.mechIdCredentials = "abc"
return "[currentBuild[${currentBuild}] mechIdCredentials[${mechIdCredentials}] " +
"baseStashURL[${baseStashURL}] jobName[${jobName}] codeBranch[${codeBranch}] " +
"codeURL[${codeURL}] buildURL[${buildURL}] pullRequestURL[${pullRequestURL}] QBotUserID[${QBotUserID}] " +
"QBotPassword[${QBotPassword}]]"
}
So, for instance, if I swapped the two variations of "getMechIdCredentials", leaving the "plain" one, this combination of samples will hang, until I click on the red X on the "Progress" indicator in Jenkins.
Update:
So, based on feedback, I defined variables in the file and changed my getters/setters to use the "this.#var" syntax. As a result, the build fails with "No such field: var for class: uslutils".
I imagine my syntax for defining the field isn't correct, but here's an excerpt of what I have:
def currentBuild = ""
String jobName = ""
String buildURL = ""
def mechIdCredentials = ""
def setCurrentBuild(value) { this.#currentBuild = value; }
def getCurrentBuild() { return this.#currentBuild }
def setJobName(value) { this.#jobName = value; }
def getJobName() { return this.#jobName }
def setBuildURL(value) { this.#buildURL = value; }
def getBuildURL() { return this.#buildURL }
def setMechIdCredentials(value) { this.#mechIdCredentials = value; }
def getMechIdCredentials() { return this.#mechIdCredentials }
Note that this is NOT within a "class" declaration, it's in a Groovy script file called "uslutils.groovy".
To be clear, here is an excerpt of the stack trace I get:
groovy.lang.MissingFieldException: No such field: mechIdCredentials for class: uslutils
at groovy.lang.MetaClassImpl.getAttribute(MetaClassImpl.java:2823)
at groovy.lang.MetaClassImpl.getAttribute(MetaClassImpl.java:3759)
at org.codehaus.groovy.runtime.InvokerHelper.getAttribute(InvokerHelper.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getField(ScriptBytecodeAdapter.java:306)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.getAttribute(DefaultInvoker.java:42)
at com.cloudbees.groovy.cps.impl.AttributeAccessBlock.rawGet(AttributeAccessBlock.java:20)
at uslutils.getMechIdCredentials(/opt/app/jenkins/sdt-usl/data/jobs/uslutils-tests/builds/33/libs/usl-pipeline-library/vars/uslutils.groovy:197)
at uslutils.toString(/opt/app/jenkins/sdt-usl/data/jobs/uslutils-tests/builds/33/libs/usl-pipeline-library/vars/uslutils.groovy:242)
at WorkflowScript.run(WorkflowScript:13)
Line 242 is shown here:
String toString() {
echo "Inside uslutils.toString()x."
return "[currentBuild[${currentBuild}] mechIdCredentials[${mechIdCredentials}] " + // line 242
"baseStashURL[${baseStashURL}] jobName[${jobName}] codeBranch[${codeBranch}] " +
"codeURL[${codeURL}] buildURL[${buildURL}] pullRequestURL[${pullRequestURL}] QBotUserID[${QBotUserID}] " +
"QBotPassword[${QBotPassword}]]"
}
Related
I have a jenkins pipeline job that has been working fine. I have a small handful of similar pipelines, and I've been duplicating a small set of reusable utility methods into each one. So, I've started to construct a shared library to reduce that duplication.
I'm using the following page for guidance: https://jenkins.io/doc/book/pipeline/shared-libraries/ .
For each method that I move into the shared library, I create a "vars/methodname.groovy" file in the shared library, and change the method name to "call".
I've been doing these one at a time and verifying the pipeline job still works, and this is all working fine.
The original set of methods would reference several "global" variables, like "env.JOB_NAME" and "params.". In order for the method to work in the shared library, I would add references to those env vars and params as parameters to the methods. This also works fine.
However, I don't like the fact that I have to pass these "global" variables, that are essentially static from the start of the job, sometimes through a couple of levels of these methods that I've put into the shared library.
So, I've now created something like the "vars/acme.groovy" example from that doc page. I'm going to define instance variables to store all of those "global" variables, and move each of the single methods defined in each of the "vars/methodname.groovy" files as instance variables in this new class.
I also defined a "with" method in the class for each of the instance variables (setter that returns "this" for chaining).
I initially would configure it inside my "node" block with something like the following (the file in the library is called "vars/uslutils.groovy"):
uslutils.withCurrentBuild(currentBuild).with...
And then when I need to call any of the reused methods, I would just do "uslutils.methodname(optionalparameters)".
I also added a "toString()" method to the class, just for debugging (since debugging Jenkinsfiles is so easy :) ).
What's odd is that I'm finding that if I call this toString() method from the pipeline script, the job hangs forever, and I have to manually kill it. I imagine I'm hitting some sort of non-obvious recursion in some Groovy AST, but I don't see what I'm doing wrong.
Here is my "vars/uslutils.groovy" file in the shared library:
import hudson.model.Cause
import hudson.triggers.TimerTrigger
import hudson.triggers.SCMTrigger
import hudson.plugins.git.GitStatus
class uslutils implements Serializable {
def currentBuild
String mechIdCredentials
String baseStashURL
String jobName
String codeBranch
String buildURL
String pullRequestURL
String qBotUserID
String qBotPassword
def getCurrentBuild() { return currentBuild }
String getMechIdCredentials() { return mechIdCredentials }
String getBaseStashURL() { return baseStashURL }
String getJobName() { return jobName }
String getCodeBranch() { return codeBranch }
String getBuildURL() { return buildURL }
String getPullRequestURL() { return pullRequestURL }
String getQBotUserID() { return qBotUserID }
String getQBotPassword() { return qBotPassword }
def withCurrentBuild(currentBuild) { this.currentBuild = currentBuild; return this }
def withMechIdCredentials(String mechIdCredentials) { this.mechIdCredentials = mechIdCredentials; return this }
def withBaseStashURL(String baseStashURL) { this.baseStashURL = baseStashURL; return this }
def withJobName(String jobName) { this.jobName = jobName; return this }
def withCodeBranch(String codeBranch) { this.codeBranch = codeBranch; return this }
def withBuildURL(String buildURL) { this.buildURL = buildURL; return this }
def withPullRequestURL(String pullRequestURL) { this.pullRequestURL = pullRequestURL; return this }
def withQBotUserID(String qBotUserID) { this.qBotUserID = qBotUserID; return this }
def withQBotPassword(String qBotPassword) { this.qBotPassword = qBotPassword; return this }
public String toString() {
// return "[currentBuild[${this.currentBuild}] mechIdCredentials[${this.mechIdCredentials}] " +
// "baseStashURL[${this.baseStashURL}] jobName[${this.jobName}] codeBranch[${this.codeBranch}] " +
// "buildURL[${this.buildURL}] pullRequestURL[${this.pullRequestURL}] qBotUserID[${this.qBotUserID}] " +
// "qBotPassword[${this.qBotPassword}]]"
return this.mechIdCredentials
}
Note that I've simplified the toString() method temporarily until I figure out what I'm doing wrong here.
This is what I added at the top of my "node" block:
uslutils.currentBuild = currentBuild
println "uslutils[${uslutils}]"
When I run the job, it prints information from lines that come before this, and then it just shows the rotating thing forever, until I kill the job. If I comment out the "println", it works fine.
So the problem is, I am not able to get the default value for controllerIP variable using the getControllerIP method without calling setControllerIP. I tried similar groovy code locally and it works, but not working on my jenkins server. Also tried lots of other combination in my groovy script but nothing worked.
Note that we are using Jenkins: pipeline shared groovy libraries plugin.
This is my pipeline job on Jenkins:
node{
def controllerParameters = new com.company.project.controller.DeploymentParameters() as Object
controllerParameters.setOSUsername('jenkins')
controllerParameters.setOSPassword('jenkins123')
controllerParameters.setBuildNumber(33)
//controllerParameters.setControllerIP('192.44.44.44')
//if I uncomment above line everything works fine but I need to get default value in a case
echo "I want the default value from other file"
controllerParameters.getControllerIP()
echo "my code hangs on above line"
}
This is my other file ../controller/DeploymentParameters.groovy
package com.company.project.controller
import groovy.transform.Field
def String osUsername
def String osPassword
#Field String controllerIP = "NotCreated" //tried few combinations
//Open Stack username
def String setOSUsername(String osUsername) {
this.osUsername = osUsername
}
def String getOSUsername() {
return this.osUsername
}
//Open Stack password
void setOSPassword(String osPassword) {
this.osPassword = osPassword
}
def String getOSPassword() {
return this.osPassword
}
//Open Stack floating ip of master vm
void setControllerIP(String controllerIP) {
this.controllerIP = controllerIP
}
def String getControllerIP() {
return this.controllerIP
}
When groovy executes lines like this.osUsername = osUsername or return this.osUsername it actually calls getters and setters instead of direct field access.
So this:
def String getOSPassword() {
return this.osPassword
}
behaves like this:
def String getOSPassword() {
return this.getOsPassword()
}
And you code enters infinite recursion (same for setter and assignment).
Within your setters and getters you need to use Groovy direct field access operator
def String getOSPassword() {
return this.#osPassword
}
Ran into this while doing a refactor. Calls to getProperties() were causing our CPU usage to spike. What we discovered is that if you have a getter without an associated attribute, when you make a call to getProperties() that getter is called over 1000 times. The fix/workaround is obvious and we know it has something to do with metaprogramming but why is this happening (what point in the groovy source)? See groovy script code below:
class tester {
int count = 0
public getVar() {
println count++ + " getVar() called!"
return var
}
}
def t = new tester()
t.getProperties()
println "done!"
You should see getVar() called over 1000 times. 1068 to be exact for us.
The question has probably already been answered in the comments but I dug a little deeper to also answer the "what point in the groovy source" part.
When you call getProperties() on the instance of tester Groovy will do its magic and finally call DefaultGroovyMethods#getProperties(Object) which (in Groovy 2.4.7) looks like this:
public static Map getProperties(Object self) {
List<PropertyValue> metaProps = getMetaPropertyValues(self); // 1
Map<String, Object> props = new LinkedHashMap<String, Object>(metaProps.size());
for (PropertyValue mp : metaProps) {
try {
props.put(mp.getName(), mp.getValue()); // 2
} catch (Exception e) {
LOG.throwing(self.getClass().getName(), "getProperty(" + mp.getName() + ")", e);
}
}
return props;
}
First, Groovy determines the meta properties of the given object (see 1). This will return three properties:
var: getter only (getVar()), no setter, no field
class: getter only (inherited from Object), no setter, no field
count: getter, setter (both generated by Groovy) and field
You can easily verify this by calling t.getMetaPropertyValues().
Next, Groovy tries to get the current value of each property and puts it in a map (see 2). When it reaches var, it remembers that var has a getter (namely getVar()) and calls it. getVar() however, returns var again. For Groovy, this is the exact same property as determined in the first step. Once again, it calls its getter getVar() and the endless loop begins.
At some point, depending on the JVM, this results in a StackOverflowError, which is exactly what this site is all about :-D
From the called build flow job, I've tried both:
build.environment['AOEU'] = 'aoeu' // callee would `println called.environment['AOEU']`
and:
upstream.environment['AOEU'] = 'aoeu' // callee would `println build.environment['AOEU']`
with no luck.
I fought a lot with that too and found no clean way to do it. I finally used EnvInjectPlugin in some ugly way to do this.
def buildEnv = build.getEnvVars();
varsToAdd = [:]
// add here your custom properties
buildEnv.putAll(varsToAdd)
import org.jenkinsci.plugins.envinject.EnvInjectPluginAction
def envInjectAction = build.getAction(EnvInjectPluginAction.class);
envInjectAction.overrideAll(buildEnv)
... the EnvInject plugin did the magic
I first tried to implement EnvironmentContributingAction
and add it as build.addAction(...) but did not work for me for unknown reason.
Be sure to set 'Flow run needs a workspace' in the called job.
#NoelYap: I can't comment on Hristo's answer, but it's the correct answer. The missing detail is that importing EnvInject only works if your job has the 'Flow run needs a workspace' option selected.
A simpler version based on Michael's solution
import hudson.model.*
def setEnvVariable(final String key, final String value) {
Thread.currentThread().getCurrentExecutable()
.addAction(new ParametersAction(new StringParameterValue(key, value))
);
}
setEnvVariable("THIS_ENV_VAR", "Hello World")
The variable is then available in the post-build actions.
An old question I know, but this is how i get around it. The beautiful part is this code works well in Jenkins both within the system evaluated groovy and within the Build FLow Job DSL.
Just drop the import statements when running it from the System Groovy job/console as they are already available.
As a bonus this class also helps get the environment variable regardless of the Groovy context. Still in the build flow you should favor build.environment.get it is more natural.
Note: 'Flow run needs a workspace' should be checked.
//Import the required bindings for a Build Flow DSL
import hudson.model.AbstractBuild
import hudson.model.EnvironmentContributingAction
import hudson.EnvVars
//Drop the above if running with a System Groovy Console/Job
class Job {
static def getVariable(String key) {
def config = new HashMap()
def thr = Thread.currentThread()
def build = thr?.getCurrentExecutable()
def envVarsMap = build.parent.builds[0].properties.get("envVars")
config.putAll(envVarsMap)
return config.get(key)
}
static def setVariable(String key, String value) {
def build = Thread.currentThread().getCurrentExecutable()
def action = new VariableInjectionAction(key, value)
build.addAction(action)
build.getEnvironment()
}
static def setVariable(String key, Integer value) {
setVariable(key, value.toString())
}
}
class VariableInjectionAction implements EnvironmentContributingAction {
private String key
private String value
public VariableInjectionAction(String key, String value) {
this.key = key
this.value = value
}
public void buildEnvVars(AbstractBuild build, EnvVars envVars) {
if (envVars != null && key != null && value != null) {
envVars.put(key, value);
}
}
public String getDisplayName() { return "VariableInjectionAction"; }
public String getIconFileName() { return null; }
public String getUrlName() { return null; }
}
Job.setVariable("THIS_ENV_VAR", "Hello World")
Will set the environment variable THIS_ENV_VAR to "Hello World"
I'm developing a Groovy library providing variable binding/syncronization with simple syntax and rich event handler.
Firstly I'm aiming to archive running the following:
def a = 1
def b = 1
def c = 1
a bind { b + c }
assert a==2 & b==1 & c==1
b = 3
assert a==4 & b==3 & c==1
c = -1
assert a==2 & b==3 & c==-1
I'd like avoid Java FX approach of providing additional classes but rather enhance objects with this functionality.
I'm stuck at finding some tracking of objects. Groovy doesn't feature overload of assignment expression.
Currently I see solution in heavy AST transformation (maybe dsld might help).
Maybe there is some object tracking, events or whatever, I've missed?
I'm presuming you are writing your code using scripts. Did what you wanted with the following ideas:
Instead of binding values to keys using the Script.binding, it is better if you create a Variable class which represents both the value and the variable name;
Every method called on the Variable object gets delegated to its value;
A Listener class which encapsulates a reference to the updating operation AND the Variable object which needs to be updated;
An updateListeners() method, to, well, update the listeners;
A bind(closure) method that just bypasses the closure to the variable object. It could pretty much be dropped and written as a { b + c };
class Listener { Closure operation; Variable variable }
class Variable implements GroovyInterceptable {
String name; def value
def invokeMethod(String method, args) {
def var = args[0]
var.class == Variable ?
value."$method"(var.value) : value."$method"(var)
}
}
Variable.metaClass.call = { Closure operation ->
binding['listeners'] <<
new Listener(operation: operation, variable: delegate)
updateListeners()
}
def bind(Closure operation) { operation }
def updateListeners() {
binding.listeners.each {
it.variable.value = it.operation()
}
}
void setProperty(String prop, value) {
if (!binding.hasVariable('listeners')) binding['listeners'] = []
binding[prop] = new Variable(name: prop, value: value)
updateListeners()
}
And your tests worked like a charm.