Jenkins: get environment variables in the body of a global function - jenkins

I have a shared global function on PublishGitHub.groovy looks like this:
#!/usr/bin/env groovy
def call(body)
{
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
echo "\u001B[32mINFO: Publishing...\u001B[m"
body()
echo "\u001B[32mINFO: End Publish...\u001B[m"
}
And a code on my JenkinsFile:
environment {
VERSION = "v1.3.${env.BUILD_NUMBER}"
}
stages {
stage ('Publish WebAPI'){
steps{
echo "\u001B[32mINFO: Start Publish...\u001B[m"
PublishGitHub{
echo "This is a body with version: ${env.VERSION}"
}
}
}
}
And this is my output:
[Pipeline] echo
INFO: Start Publish...
[Pipeline] echo
INFO: Publishing...
[Pipeline] }
And follow next error:
java.lang.NullPointerException: Cannot get property 'VERSION' on null
object
Because inside the body I do not have access to the environment variables?

Your shared library code runs outside of the workflow CPS context, that is why closure you pass to the vars script does not recognize env property. You can fix this problem by passing a reference to the workflow script. If you call your function like this
PublishGitHub(this) {
echo "This is a body with version: ${env.VERSION}"
}
and you apply a small modification to vars/PublishGitHub.groovy script like:
#!/usr/bin/env groovy
def call(config, body) {
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
echo "\u001B[32mINFO: Publishing...\u001B[m"
body()
echo "\u001B[32mINFO: End Publish...\u001B[m"
}
then you will run your pipeline successfully:
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Publish WebAPI)
[Pipeline] echo
[32mINFO: Start Publish...[m
[Pipeline] echo
[32mINFO: Publishing...[m
[Pipeline] echo
This is a body with version: v1.3.537
[Pipeline] echo
[32mINFO: End Publish...[m
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
If you want to limit the scope for the shared library, you can always simply pass env instead of this and change vars/PublishGitHub.groovy to something like this:
#!/usr/bin/env groovy
def call(env, body) {
def config = [
env: env
]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
echo "\u001B[32mINFO: Publishing...\u001B[m"
body()
echo "\u001B[32mINFO: End Publish...\u001B[m"
}
In this scenario you give your shared library an access to environment variables only.

In order to make the environment variables that you have defined in your Jenkinsfile available in your shared library code you have to pass a this parameter on the call to your shared library method.
For example (below is a partial extract only of a full pipeline file):
// JENKINS-42730
#Library('pipeline-shared-library')_
import org.blah.MySharedLibraryClass
// END JENKINS_42730
pipeline {
agent { any }
environment {
FOO = (new MySharedLibraryClass(config, this)).myMethod("StringVar1", "StringVar2")
}
}
My Shared Library:
package org.blah
import groovy.text.SimpleTemplateEngine
public class MySharedLibraryClass implements Serializable {
def engine = new SimpleTemplateEngine()
def config
def steps
def ArtifactoryHelper(config, steps) {
this.config = config
this.steps = steps
}
def log(msg){
//Allows me to print to Jenkins console
steps.println(msg)
}
def myMethod(var1, var2) {
....
}
The this parameter I referred to above maps to steps in the shared library code. You should then be able to resolve "VERSION=${steps.env.VERSION}" in your shared library code.
Also see this post.
Notes:
pipeline-shared-library is the ID I gave the library in Manage Jenkins > Configure System

Szymon's answer works. I added "p.env = env"
def toParam(final Closure body) {
final def p = [:]
p.env = env
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = p
body()
return p
}

Related

Importing map variable to Jenkinsfile environment stage

My project has many common variables for many other projects, so I use Jenkins Shared Library and created a vars/my_vars.groovy file where I defined my variables and return Map of them:
class my_vars {
static Map varMap = [:]
static def loadVars (Map config) {
varMap.var1 = "val1"
varMap.var2 = "val2"
// Many more variables ...
return varMap
}
}
I load the Shared Library in my Jenkinsfile, and call the function in the environment bullet, as I want those variables to be as environment variables .
Jenkinsfile:
pipeline {
environment {
// initialize common vars
common_vars = my_vars.loadVars()
} // environment
stages {
stage('Some Stage') {
// ...
}
}
post {
always {
script {
// Print environment variables
sh "env"
} // script
} // always
} // post
} // pipeline
The thing is that the environment bullet gets KEY=VALUE pairs, thus my common_vars map is loaded like a String value (I can see that on sh "env").
...
vars=[var1:val1, var2:val2]
...
What is the correct way to declare those values as an environment variables?
My target to get this:
...
var1=val1
var2=val2
...
Pipeline's environment variables store only String values. That is why when you assign a map to env.common_vars variables it stores map.toString() equivalent.
If you want to rewrite key-values from a map to the environment variables, you can iterate the variables map and assign each k-v pair to something like env."$k" = v. You can do that by calling a class method inside the environment block - that way you can be sure that the environment variables are assigned no matter which stage your pipeline gets restarted from. Consider the following example:
class MyVars {
private Map config = [
var1: "val1",
var2: "val2"
]
String initializeEnvironmentVariables(final Script script) {
config.each { k,v ->
script.env."$k" = v
}
return "Initialization of env variables completed!"
}
}
pipeline {
agent any
environment {
INITIALIZE_ENV_VARIABLES_FROM_MAP = "${new MyVars().initializeEnvironmentVariables(this)}"
}
stages {
stage("Some stage") {
steps {
echo "env.var1 = ${env.var1}"
}
}
}
post {
always {
script {
sh 'printenv | grep "var[0-9]\\+"'
}
}
}
}
In this example, we use MyVars class to store some global config map (it can be a part of a shared library, here, for simplicity, it is a part of the Jenkinsfile). We use INITIALIZE_ENV_VARIABLES_FROM_MAP environment variable assignment to call MyVars.initializeEnvironmentVariables(this) method that can access env from the script parameter. Calling this method from inside environment block has one significant benefit - it guarantees that environment variables will be initialized even if you restart the pipeline from any stage.
And here is the output of this exemplary pipeline:
Running on Jenkins in /home/wololock/.jenkins/workspace/pipeline-env-map
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Some stage)
[Pipeline] echo
env.var1 = val1
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ grep 'var[0-9]\+'
+ printenv
var1=val1
var2=val2
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
As you can see we it sets env.var1 and env.var2 from the map encapsulated in MyVars class. Both variables can be accessed inside the pipeline step, script block or even inside the shell environment variables.
As far as I know there is no easy way to do this in declarative pipeline (e.g. in the environment directive. Instead, what you can do is to setup the environment outside of the declarative definition, like this:
my_vars.loadVars().each { key, value ->
env[key] = value
}
// Followed by your pipelines definition:
pipeline {
stages {
stage('Some Stage') {
// ...
}
}
// ...
} // pipeline
As an full example:
class my_vars {
static Map varMap = [:]
static def loadVars (Map config) {
varMap.var1 = "val1"
varMap.var2 = "val2"
// Many more variables ...
return varMap
}
}
my_vars.loadVars().each { key, value ->
env[key] = value
}
pipeline {
agent any
stages {
stage("Some stage") {
steps {
echo "env.var1 = ${env.var1}"
}
}
}
}
Which outputs the following when built:
Started by user xxx
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on yyy in /zzz
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Some stage)
[Pipeline] echo
env.var1 = val1
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Edit; If your class (my_vars) is located in a shared library (MySharedLibrary):
library 'MySharedLibrary' // Will load vars/my_vars.groovy
my_vars.loadVars().each { key, value ->
env[key] = value
}
pipeline {
agent any
stages {
stage("Some stage") {
steps {
echo "env.var1 = ${env.var1}"
}
}
}
}
You don't have to return a map of your environment variables from your shared library. You can simply set them in a shared library method, the method will run in the same container as your pipeline.
In you shared library vars/ directory:
def setVars() {
env.var1 = "var1"
env.var2 = "var2"
env.var3 = "var3"
}
In your pipeline:
pipeline {
agent any
stages {
stage("Setup") {
steps {
script {
imported_shared_lib.setVars()
}
}
}
}
}
Others mentioned the need to preserve the environment variables even if you restart the pipeline from a certain stage. In my experiments, the variables are preserved using this method, even if the setVars() method is not called in the environment{} block.

Expression depeding on Jenkins build boolean parameter doesn't work in pipeline

I've inherited some Jenkins pipeline and try to improve it. Jenkins and groovy is quite fresh topic for me, so most probably I'm doing something wrong.
I'm using Jenkins ver. 2.121.3
Main aim was to add build parameter to do some extra cleaning during build. So I've added parameter CLEAN_FIRST with Boolean type and default value false to a job configuration and did something like this in pipeline:
// CLEAN_FIRST = false
// def prefix = CLEAN_FIRST ? "" : "REM"
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('master') {
cleanWs()
try {
def prefix = CLEAN_FIRST ? "" : "REM"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
bat (label: 'build third party',
script: """
$prefix call cleanSomthing.bat
call doOtherStuff.bat
"""
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
Now this doesn't work as expected. "REM" prefix is not added.
Echo prints:
CLEAN_FIRST=false prefix=
And bat invokes cleanSomthing.bat which I wish to avoid (to save on build times).
I've tried to make prefix global, but with same result.
Most probably this is caused by some evaluation order or scoping issue, but I can't put finger on it.
Can someone give me a clue why it doesn't work? How to fix it?
Answered own question. Is this problem fixed on some version of Jenkins?
replace
def prefix = CLEAN_FIRST ? "" : "REM"
with
def prefix = params.CLEAN_FIRST ? "" : "REM"
Ok I've found source of problems. It is a bit funny.
When running this pipeline (tested on Mac machine since it had empty job queue):
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('Mac') {
cleanWs()
try {
def logic = true
def prefix = CLEAN_FIRST ? "Ole" : "REM"
def typeLogic = logic.getClass()
def typeParam = CLEAN_FIRST.getClass()
echo "typeLogic=$typeLogic typeParam=$typeParam"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
sh (script: """
echo prefix=$prefix
""")
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
I've got this outcome:
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] stage
[Pipeline] { (Some step)
[Pipeline] script
[Pipeline] {
[Pipeline] node
Running on master in /Users/builder/jenkins/workspace/EIbuild_MacOS
[Pipeline] {
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] echo
typeLogic=class java.lang.Boolean typeParam=class java.lang.String
[Pipeline] echo
CLEAN_FIRST=false prefix=Ole
[Pipeline] sh
[EIbuild_MacOS] Running shell script
+ echo prefix=Ole
prefix=Ole
[Pipeline] echo
some stuff
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
So now source the problem is obvious.
Jenkins in configuration promises variable of type Boolean, but in fact provides type String with values are "true" or "false" which are always evaluated as true when used as condition since both values are not empty strings :).

pipeline evaluate variable in yaml file

I have a yaml file that contains reference to some variable that define in pipeline.
I need some way to evaluate this $ to real value after I read the yaml.
yaml file
chart_folder: test_chart_${my_suffix}
lint:
enable: false
pipeline look like below
pipeline{
agent{
label "my_node"
}
stages{
stage("test"){
steps{
script {
def my_suffix = "test"
def my_yaml = readYaml file: "my_file.yaml"
echo my_yaml.chart_folder
}
}
}
}
}
The output of execution is
....
[Pipeline] readYaml
[Pipeline] echo
test_chart_${my_suffix}
[Pipeline] }
....
and I want to get the chart_folder as evaluated string
....
[Pipeline] readYaml
[Pipeline] echo
test_chart_test
[Pipeline] }
....
How can I do it?
You can't interpolate, but you can override.
def my_yaml = readYaml file: 'my_file.yaml', text: "chart_folder: 'test_chart_test'"

Can I define custom steps using a hash-table for params for a declarative pipeline?

I want to create a custom step as detailed here: https://jenkins.io/doc/book/pipeline/shared-libraries/#defining-custom-steps
The script looks like this:
// vars/buildPlugin.groovy
def call(body) {
// evaluate the body block, and collect configuration into the object
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()
...
And I can run it like this in a scripted pipeline:
buildPlugin {
name = 'git'
}
Which means in a declarative pipeline i gotta wrap it in a script block:
script {
buildPlugin {
name = 'git'
}
}
I have a lot of custom scripts and groovy classes and it clutters things up to have to wrap them in script blocks in my pipeline. Can I write groovy scripts in a way the declarative pipeline can use without script{}?
EDIT:
Calling a groovy script from the pipeline like this works:
myCustomStep('sldkfjlskdf')
But I want to use a hashtable like they have in the examples:
# In myCustomStep.grooy
def call(body) {
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()
In order to call it now I have to do:
myCustomStep{
param1 = 'sldkfjlskdf'
param2 = 'sdfsdfsdfdf'
}
Doing this I get Expected a step # line.... and have to wrap it in a step
Is there a way to get nice named params like with the hash-table approach but not have to wrap in a step? I also tried calling it like myCustomStep({param1 = 'sdfsdf'}) which did not work
you can use it also in declarative pipline without script wrapper
Here is example that works well:
script
//vars/shOut.groovy
def call(shellScript) {
return sh(returnStdout: true, script: shellScript).trim()
}
Jenkinsfile
#Library('modelsLib') _
pipeline {
agent { label 'master' }
stages {
stage('some stage') {
steps {
echo "hello!"
shOut 'touch xxyyzz'
sh 'ls -otr'
}
}
}
}
output
[Pipeline] {
[Pipeline] stage
[Pipeline] { (some stage)
[Pipeline] echo
hello!
[Pipeline] sh
[test-pipeline] Running shell script
+ touch xxyyzz
[Pipeline] sh
[test-pipeline] Running shell script
+ ls -otr
total 32
-rw-r--r-- 1 jenkins 0 Dec 20 17:59 xxyyzz
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Defining the closure outside of the pipeline works.
def buildOpts = {name = 'git'}
pipeline {
...
steps {
buildPlugin(buildOpts)
}
}
hopefully that works for your case

How to call a groovy function from a Jenkinsfile?

Despite following this answer and others, I am unable to successfully use a local groovy file in my Jenkinsfile (both are in the same repository).
def deployer = null
...
...
...
pipeline {
agent {
label 'cf_slave'
}
options {
skipDefaultCheckout()
disableConcurrentBuilds()
}
stages {
stage ("Checkout SCM") {
steps {
checkout scm
}
}
...
...
...
stage ("Publish CF app") {
steps {
script {
STAGE_NAME = "Publish CF app"
deployer = fileLoader.load ('deployer')
withCredentials(...) {
if (BRANCH_NAME == "develop") {
...
...
...
} else {
deployer.generateManifest()
}
}
}
}
}
...
...
}
deployer.groovy:
#!/usr/bin/env groovy
def generateManifest() {
sh "..."
echo "..."
}
In the console log (stack):
[Pipeline] stage
[Pipeline] { (Publish CF app)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
before loading groovy file
[Pipeline] echo
Loading from deployer.groovy
[Pipeline] load
[Pipeline] // load
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
Update:
It seems the problem was not with loading the file but rather with the contents of the file, where I execute the following which apparently does not play well:
sh "node $(pwd)/config/mustacher manifest.template.yml config/environments/common.json config/environments/someFile.json"
echo "..."
When only the echo is there, this is the stack.
So not the sh "node ..." nor the echo work. Even changing it just to sh "pwd" fails as well. What could it be? the syntax in the file? the way it is called in the pipeline?
If I will make the same node call in the pipeline (for example in the withCredentials if statement, it works.
Add a return this to the bottom of the deployer.groovy file, and then change you load step to use relative path and extension to groovy file like load('deployer.groovy').
The return this is documented on jenkins.io:
Takes a filename in the workspace and runs it as Groovy source text.
The loaded file can contain statements at top level or just load and run a closure. For example:
def pipeline
node('slave') {
pipeline = load 'pipeline.groovy'
pipeline.functionA()
}
pipeline.functionB()
pipeline.groovy
def pipelineMethod() {
...code
}
return this
Where pipeline.groovy defines functionA and functionB functions (among others) before ending with return this

Resources