In Jenkins pipelines, if I assign any key-value to env, I can access it like normal variables in string interpolation and environment variables in shell script. How does this work?
I see the probable implementation here but can't figure out how it works.
[Update]
In below code snippet, I can access the env properties without accessor -->
node {
stage('Preparation') {
env.foo = "bar"
echo "foo is $foo"
}
}
I haven't delved into the Jenkins code, but you could implement something like this by implementing the propertyMissing() method in a class, which would write to the script binding. The propertyMissing method gets called when the code is trying to access a property that is not declared in the class.
class MyEnv {
groovy.lang.Script script
MyEnv( groovy.lang.Script script ) {
this.script = script
}
def propertyMissing( String name ) {
script.getProperty( name )
}
def propertyMissing( String name, value ) {
script.setProperty( name, value )
}
}
def env = new MyEnv( this ) // pass the script context to the class
env.foo = 42 // actually assigns to the script binding
println "env.foo: $env.foo" // prints 42
println "foo: $foo" // prints 42
// It also works the other way around...
foo = 21 // assign to the script binding (Note: "def foo" would not work!)
println "env.foo: $env.foo" // prints 21
Related
I would like to know if there is a way to access the Jenkins Workflow script object during its execution.
I have a shared library, and I can pass this object to any groovy class as an argument, either directly from the Jenkins file, using 'this' keyword, or from any DSL in the vars folder, also using the 'this' keyword.
But I would like to access it using a method, even if this imply using reflexivity.
Is that possible?
Here example with pipeline, where this is a script object. Some other examples here:
MyClass myClass = new MyClass()
pipeline {
agent any
environment {
VAR1 = "var1"
VAR2 = sh(returnStdout: true, script: "echo var2").trim()
VAR3 = "var3"
}
stages {
stage("Stage 1") {
steps {
script {
myClass.myPrint(this, "${VAR1}", "${VAR2}", "${VAR3}")
}
}
}
}
}
class MyClass implements Serializable {
void myPrint(def script, String var1, String var2, String... vars) {
script.echo "myPrint: ${var1}"
}
}
When I run the below Jenkins pipeline script:
def some_var = "some value"
def pr() {
def another_var = "another " + some_var
echo "${another_var}"
}
pipeline {
agent any
stages {
stage ("Run") {
steps {
pr()
}
}
}
}
I get this error:
groovy.lang.MissingPropertyException: No such property: some_var for class: groovy.lang.Binding
If the def is removed from some_var, it works fine. Could someone explain the scoping rules that cause this behavior?
TL;DR
variables defined with def in the main script body cannot be accessed from other methods.
variables defined without def can be accessed directly by any method even from different scripts. It's a bad practice.
variables defined with def and #Field annotation can be accessed directly from methods defined in the same script.
Explanation
When groovy compiles that script it actually moves everything to a class that roughly looks something like this
class Script1 {
def pr() {
def another_var = "another " + some_var
echo "${another_var}"
}
def run() {
def some_var = "some value"
pipeline {
agent any
stages {
stage ("Run") {
steps {
pr()
}
}
}
}
}
}
You can see that some_var is clearly out of scope for pr() becuse it's a local variable in a different method.
When you define a variable without def you actually put that variable into a Binding of the script (so-called binding variables). So when groovy executes pr() method firstly it tries to find a local variable with a name some_var and if it doesn't exist it then tries to find that variable in a Binding (which exists because you defined it without def).
Binding variables are considered bad practice because if you load multiple scripts (load step) binding variables will be accessible in all those scripts because Jenkins shares the same Binding for all scripts. A much better alternative is to use #Field annotation. This way you can make a variable accessible in all methods inside one script without exposing it to other scripts.
import groovy.transform.Field
#Field
def some_var = "some value"
def pr() {
def another_var = "another " + some_var
echo "${another_var}"
}
//your pipeline
When groovy compiles this script into a class it will look something like this
class Script1 {
def some_var = "some value"
def pr() {
def another_var = "another " + some_var
echo "${another_var}"
}
def run() {
//your pipeline
}
}
Great Answer from #Vitalii Vitrenko!
I tried program to verify that. Also added few more test cases.
import groovy.transform.Field
#Field
def CLASS_VAR = "CLASS"
def METHOD_VAR = "METHOD"
GLOBAL_VAR = "GLOBAL"
def testMethod() {
echo "testMethod starts:"
def testMethodLocalVar = "Test_Method_Local_Var"
testMethodGlobalVar = "Test_Metho_Global_var"
echo "${CLASS_VAR}"
// echo "${METHOD_VAR}" //can be accessed only within pipeline run method
echo "${GLOBAL_VAR}"
echo "${testMethodLocalVar}"
echo "${testMethodGlobalVar}"
echo "testMethod ends:"
}
pipeline {
agent any
stages {
stage('parallel stage') {
parallel {
stage('parallel one') {
agent any
steps {
echo "parallel one"
testMethod()
echo "${CLASS_VAR}"
echo "${METHOD_VAR}"
echo "${GLOBAL_VAR}"
echo "${testMethodGlobalVar}"
script {
pipelineMethodOneGlobalVar = "pipelineMethodOneGlobalVar"
sh_output = sh returnStdout: true, script: 'pwd' //Declared global to access outside the script
}
echo "sh_output ${sh_output}"
}
}
stage('parallel two') {
agent any
steps {
echo "parallel two"
// pipelineGlobalVar = "new" //cannot introduce new variables here
// def pipelineMethodVar = "new" //cannot introduce new variables here
script { //new variable and reassigning needs scripted-pipeline
def pipelineMethodLocalVar = "new";
pipelineMethodLocalVar = "pipelineMethodLocalVar reassigned";
pipelineMethodGlobalVar = "new" //no def keyword
pipelineMethodGlobalVar = "pipelineMethodGlobalVar reassigned"
CLASS_VAR = "CLASS TWO"
METHOD_VAR = "METHOD TWO"
GLOBAL_VAR = "GLOBAL TWO"
}
// echo "${pipelineMethodLocalVar}" only script level scope, cannot be accessed here
echo "${pipelineMethodGlobalVar}"
echo "${pipelineMethodOneGlobalVar}"
testMethod()
}
}
}
}
stage('sequential') {
steps {
script {
echo "sequential"
}
}
}
}
}
Observations:
Six cases of variables declarations
a. Three types (with def, without def, with def and with #field) before/above pipeline
b. within scripted-pipeline (with def, without def) within pipeline
c. Local to a method (with def) outside pipeline
new variable declaration and reassigning needs scripted-pipeline within pipeline.
All the variable declared outside pipeline can be accessed between the stages
Variable with def keyword generally specific to a method, if it is declared inside script then will not be available outside of it. So need to declare global variable (without def) within script to access outside of script.
I'm trying to construct a convenient shared library for my scripted pipelines.
I want to be able to use an object from my Jenkinsfile that has several functional methods, but also stores some initial variables that are referenced by several methods.
So, I've defined a file in "vars" with a bunch of methods for functionality, but also a handful of get/set methods for the properties I want to store in the object for reference by several methods.
My initial testing is just to set some of the variables (not all of them) and call the "toString()" method which I've defined in the file (just for convention, obviously).
In my test Jenkinsfile, If I do set all of the variables and then call "toString()", it works fine and completes.
However, if I try commenting out one of the variable initializations, when it hits the line in the "toString" method that constructs the return value, Jenkins just hangs forever. I eventually just kill the job.
I've been able to avoid the hang by qualifying each reference with 'if binding.variables.containsKey("foo")) {', only referencing the variable if that is true. I made this a tiny bit cleaner by putting the binding check in each of the getter methods instead.
I don't really like this workaround. It just seems odd that I would have to do this.
I've tried several variations, but I don't set the variable and I try to reference it in a gstring, the job hangs forever, every time.
This is a brief excerpt from the "vars" file:
def setJobName(value) { jobName = value; }
//def getJobName() { return jobName }
def getJobName() { return (binding.variables.containsKey("jobName") ? jobName : "") }
def setMechIdCredentials(value) { mechIdCredentials = value; }
//def getMechIdCredentials() { return mechIdCredentials }
def getMechIdCredentials() { return (binding.variables.containsKey("mechIdCredentials") ? mechIdCredentials : "") }
... more get/set methods
String toString() {
echo "Inside uslutils.toString()."
This is an excerpt from the Jenkinsfile that uses this:
uslutils.currentBuild = currentBuild
uslutils.jobName = env.JOB_NAME
uslutils.buildURL = env.BUILD_URL
//uslutils.mechIdCredentials = "abc"
return "[currentBuild[${currentBuild}] mechIdCredentials[${mechIdCredentials}] " +
"baseStashURL[${baseStashURL}] jobName[${jobName}] codeBranch[${codeBranch}] " +
"codeURL[${codeURL}] buildURL[${buildURL}] pullRequestURL[${pullRequestURL}] QBotUserID[${QBotUserID}] " +
"QBotPassword[${QBotPassword}]]"
}
So, for instance, if I swapped the two variations of "getMechIdCredentials", leaving the "plain" one, this combination of samples will hang, until I click on the red X on the "Progress" indicator in Jenkins.
Update:
So, based on feedback, I defined variables in the file and changed my getters/setters to use the "this.#var" syntax. As a result, the build fails with "No such field: var for class: uslutils".
I imagine my syntax for defining the field isn't correct, but here's an excerpt of what I have:
def currentBuild = ""
String jobName = ""
String buildURL = ""
def mechIdCredentials = ""
def setCurrentBuild(value) { this.#currentBuild = value; }
def getCurrentBuild() { return this.#currentBuild }
def setJobName(value) { this.#jobName = value; }
def getJobName() { return this.#jobName }
def setBuildURL(value) { this.#buildURL = value; }
def getBuildURL() { return this.#buildURL }
def setMechIdCredentials(value) { this.#mechIdCredentials = value; }
def getMechIdCredentials() { return this.#mechIdCredentials }
Note that this is NOT within a "class" declaration, it's in a Groovy script file called "uslutils.groovy".
To be clear, here is an excerpt of the stack trace I get:
groovy.lang.MissingFieldException: No such field: mechIdCredentials for class: uslutils
at groovy.lang.MetaClassImpl.getAttribute(MetaClassImpl.java:2823)
at groovy.lang.MetaClassImpl.getAttribute(MetaClassImpl.java:3759)
at org.codehaus.groovy.runtime.InvokerHelper.getAttribute(InvokerHelper.java:145)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getField(ScriptBytecodeAdapter.java:306)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.getAttribute(DefaultInvoker.java:42)
at com.cloudbees.groovy.cps.impl.AttributeAccessBlock.rawGet(AttributeAccessBlock.java:20)
at uslutils.getMechIdCredentials(/opt/app/jenkins/sdt-usl/data/jobs/uslutils-tests/builds/33/libs/usl-pipeline-library/vars/uslutils.groovy:197)
at uslutils.toString(/opt/app/jenkins/sdt-usl/data/jobs/uslutils-tests/builds/33/libs/usl-pipeline-library/vars/uslutils.groovy:242)
at WorkflowScript.run(WorkflowScript:13)
Line 242 is shown here:
String toString() {
echo "Inside uslutils.toString()x."
return "[currentBuild[${currentBuild}] mechIdCredentials[${mechIdCredentials}] " + // line 242
"baseStashURL[${baseStashURL}] jobName[${jobName}] codeBranch[${codeBranch}] " +
"codeURL[${codeURL}] buildURL[${buildURL}] pullRequestURL[${pullRequestURL}] QBotUserID[${QBotUserID}] " +
"QBotPassword[${QBotPassword}]]"
}
So the problem is, I am not able to get the default value for controllerIP variable using the getControllerIP method without calling setControllerIP. I tried similar groovy code locally and it works, but not working on my jenkins server. Also tried lots of other combination in my groovy script but nothing worked.
Note that we are using Jenkins: pipeline shared groovy libraries plugin.
This is my pipeline job on Jenkins:
node{
def controllerParameters = new com.company.project.controller.DeploymentParameters() as Object
controllerParameters.setOSUsername('jenkins')
controllerParameters.setOSPassword('jenkins123')
controllerParameters.setBuildNumber(33)
//controllerParameters.setControllerIP('192.44.44.44')
//if I uncomment above line everything works fine but I need to get default value in a case
echo "I want the default value from other file"
controllerParameters.getControllerIP()
echo "my code hangs on above line"
}
This is my other file ../controller/DeploymentParameters.groovy
package com.company.project.controller
import groovy.transform.Field
def String osUsername
def String osPassword
#Field String controllerIP = "NotCreated" //tried few combinations
//Open Stack username
def String setOSUsername(String osUsername) {
this.osUsername = osUsername
}
def String getOSUsername() {
return this.osUsername
}
//Open Stack password
void setOSPassword(String osPassword) {
this.osPassword = osPassword
}
def String getOSPassword() {
return this.osPassword
}
//Open Stack floating ip of master vm
void setControllerIP(String controllerIP) {
this.controllerIP = controllerIP
}
def String getControllerIP() {
return this.controllerIP
}
When groovy executes lines like this.osUsername = osUsername or return this.osUsername it actually calls getters and setters instead of direct field access.
So this:
def String getOSPassword() {
return this.osPassword
}
behaves like this:
def String getOSPassword() {
return this.getOsPassword()
}
And you code enters infinite recursion (same for setter and assignment).
Within your setters and getters you need to use Groovy direct field access operator
def String getOSPassword() {
return this.#osPassword
}
My goal is to execute groovy script with binding, where functions are predefined and interceptor log out execution time and result of closure evaluation. My sample code is:
binding.login = { ->
binding.event.appname=='login'
} def gse = new GroovyScriptEngine("src/main/resources/rules")
gse.run('DSL.groovy', binding)
Inside my script I am making a call to login method. Everything works except I can't fugure out how to intercept it using MetaClass. My attempts like
Binding.metaClass.invokeMethod = { String name, args ->
println ("Call to $name intercepted... ")
did not work. Later I figured out that closure is a property of the binding, not a method.
Is there any way to perform interception in this scenario and how to do it? What would be a correct object for metaclass? As of note, my closure executed inside another nested closures.
Thanks
I don't know if it's the better solution, but i managed to do what you wanted by decorating the closures in the binding
binding = new Binding([
login : { -> println "binding.login called" },
echo : { String text -> println "test for $text" },
foo : { a, b, c -> println "$a $b $c" }
])
binding.variables.each { key, value ->
if (value instanceof Closure)
{
binding.variables[key] = { Object[] args ->
long t0 = System.currentTimeMillis()
value( *args )
print "[$key method"
print " args: $args "
println " time: ${System.currentTimeMillis() - t0}]"
}
}
}
def gse = new GroovyScriptEngine(".")
gse.run('Bind.groovy', binding)
And this is my Bind.groovy:
println " ** executing bind"
login()
echo("echo")
foo("test", 4, "echo")
println " ** bind script done"
You could also try/catch a MissingMethodException if you didn't defined the method as a closure in the binding.
I also recommend you Laforge's slideshare in creating DSLs:
http://www.slideshare.net/glaforge/going-to-mars-with-groovy-domainspecific-languages
In this slideshare, Laforge shows binding using a class that extends script; i think that's a good approach. More OO.
Update
Take a look at mrhaki's suggestion to delegate method calls to a base Script class (also in Guillaume' slideshare):
http://mrhaki.blogspot.com.br/2011/11/groovy-goodness-create-our-own-script.html
I just used it to implement a DSL over JFugue and it worked right away.