How to use declared variables in when conditional step? - jenkins

I am declaring a variable outside the pipeline tags and initializing in one of the stages inside script using readFile. When I try to use this declared variable inside when condition, the when condition always skips.
When I initialize the variable with var1='myValue' and check for
when{ expression{var1=='myValue'} } works .
I tried making the varible into environment variable but not else works
def test1=''
pipeline{
agent { label 'local'}
stages{
stage('initialization'){
steps{
sh """echo "myValue" > user_file
pwd
"""
script{
test1=readFile file: 'user_file
//env.TEST=readFile file: 'user_file', encoding: 'UTF-8' doesn't work
//test1='myValue' only this works
}
}
}
stage('test'){
when {
expression { return test1 == 'myValue'}
//environment name: 'TEST', value: 'myValue'
}
steps{
echo " user :${env.TEST}"
//this step always gets skipped because of when condition above
}
}
}
}
I expect the steps under when to run, but it gets skips every time.

As Matt commented on the question, I had a new line character in the file. The new line character was introduced by echo by default. I had to suppress it using the -n option from echo.
echo -n "myValue" > user_file
The above command only puts the value without the newline in the file
Sorry that my shell-foo is bad and didn't realize that echo puts in a new line by default.

Related

How to update Env vars from jenkins Powershell script

Hello Trying to update jenkins Env variables from Powershell script written in stage
pipeline {
agent { label 'master' }
environment {
def var1 = "default_value"
def var2 = "default var 2"
}
stages {
stage ('step-1') {
steps {
script {
powershell (returnStdout: true, script: '''
${env:var1} = "changed from powershell"
${env:var2} = "var2 changed from powershell"
''').trim()
}
}
}
stage ("step-2"){
steps {
echo var1
echo var2
}
}
}
}
I can access values but cannot change the same, Does Scope is limited to Powershell only?
By design, a subshell cannot modify the environment of its parent shell. Every powershell, sh, or bat step called within a Jenkinsfile spawns a new subshell to run the child processes. Therefore, any environment variable declared inside these steps are lost as soon as the child processes end and are not available to the pipeline Groovy script.
One way to do this currently in a Jenkinsfile is to assign the stdout of your subshell to a variable in the Groovy script itself.
script {
env.var1 = powershell(returnStdout: true, script: '<single command>').trim()
}
Evidently, this approach has serious limitations as you can run only one command at a time and that too, only if it would return an appropriate output to be used as a variable.
Another way is to write the values to a file and then read these values using Groovy script to set the environment variables downstream in your pipeline.
Edit
Based on your comment, if your stdout returns two distinct values as a string, you can try parsing them into separate values. For example:
script {
def output = powershell(returnStdout: true, script: '''
<command1>
<command2>
''')
println(output)
/*Prints:
value1
value2
*/
env.var1 = output.tokenize('\n')[0].trim() \\value1
env.var2 = output.tokenize('\n')[1].trim() \\value2
}

Passing variables extracted from shell in Jenkinsfile

I am trying to pass variables extracted in a stage in Jenkinsfile between stages. For example:
stage('Dummy Stage') {
sh '''#!/bin/bash -l
export abc=`output of some command`
.....
.....
'''
Now, how can I pass the variable abc to a subsequent stage? I have tried setting the variable by adding a def section at the top of the file but looks like it doesnt work. In the absence of a neater way, I am having to retype the commands
Here is what I do to get the number of commits on master as a global environment variable:
pipeline {
agent any
environment {
COMMITS_ON_MASTER = sh(script: "git rev-list HEAD --count", returnStdout: true).trim()
}
stages {
stage("Print commits") {
steps {
echo "There are ${env.COMMITS_ON_MASTER} commits on master"
}
}
}
}
You can use the longer form of the sh step and return the output (see Pipeline document). Your variable should be defined outside the stages.
You can use an imperatively created environment variable inside a script block in you stage steps, for example:
stage("Stage 1") {
steps {
script {
env.RESULT_ON_STAGE_1 = sh (
script: 'echo "Output of some command"',
returnStdout: true
)
}
echo "In stage 1: ${env.RESULT_ON_STAGE_1}"
}
}
stage("Stage 2") {
steps {
echo "In stage 2: ${env.RESULT_ON_STAGE_1}"
}
}
This guide explains use of environment variables in pipelines with examples.
My issue concerned having two 'sh' commands where one uses single quotes (where I set a variable) and the other uses double quotes (where I access 'env' variables set in the jenkinsfile such as BUILD_ID).
Here's how I solved it.
script {
env.TEST = sh(
script:
'''
echo "TEST"
''',
returnStdout: true
)
sh """
echo ${env.BUILD_ID}
echo ${env.TEST}
"""
}

iterate over environment variables in Jenkins Pipeline Groovy [duplicate]

Given a jenkins build pipeline, jenkins injects a variable env into the node{}. Variable env holds environment variables and values.
I want to print all env properties within the jenkins pipeline. However, I do no not know all env properties ahead of time.
For example, environment variable BRANCH_NAME can be printed with code
node {
echo ${env.BRANCH_NAME}
...
But again, I don't know all variables ahead of time. I want code that handles that, something like
node {
for(e in env){
echo e + " is " + ${e}
}
...
which would echo something like
BRANCH_NAME is myBranch2
CHANGE_ID is 44
...
I used Jenkins 2.1 for this example.
According to Jenkins documentation for declarative pipeline:
sh 'printenv'
For Jenkins scripted pipeline:
echo sh(script: 'env|sort', returnStdout: true)
The above also sorts your env vars for convenience.
Another, more concise way:
node {
echo sh(returnStdout: true, script: 'env')
// ...
}
cf. https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#code-sh-code-shell-script
The following works:
#NonCPS
def printParams() {
env.getEnvironment().each { name, value -> println "Name: $name -> Value $value" }
}
printParams()
Note that it will most probably fail on first execution and require you approve various groovy methods to run in jenkins sandbox. This is done in "manage jenkins/in-process script approval"
The list I got included:
BUILD_DISPLAY_NAME
BUILD_ID
BUILD_NUMBER
BUILD_TAG
BUILD_URL
CLASSPATH
HUDSON_HOME
HUDSON_SERVER_COOKIE
HUDSON_URL
JENKINS_HOME
JENKINS_SERVER_COOKIE
JENKINS_URL
JOB_BASE_NAME
JOB_NAME
JOB_URL
You can accomplish the result using sh/bat step and readFile:
node {
sh 'env > env.txt'
readFile('env.txt').split("\r?\n").each {
println it
}
}
Unfortunately env.getEnvironment() returns very limited map of environment variables.
Why all this complicatedness?
sh 'env'
does what you need (under *nix)
Cross-platform way of listing all environment variables:
if (isUnix()) {
sh env
}
else {
bat set
}
Here's a quick script you can add as a pipeline job to list all environment variables:
node {
echo(env.getEnvironment().collect({environmentVariable -> "${environmentVariable.key} = ${environmentVariable.value}"}).join("\n"))
echo(System.getenv().collect({environmentVariable -> "${environmentVariable.key} = ${environmentVariable.value}"}).join("\n"))
}
This will list both system and Jenkins variables.
I use Blue Ocean plugin and did not like each environment entry getting its own block. I want one block with all the lines.
Prints poorly:
sh 'echo `env`'
Prints poorly:
sh 'env > env.txt'
for (String i : readFile('env.txt').split("\r?\n")) {
println i
}
Prints well:
sh 'env > env.txt'
sh 'cat env.txt'
Prints well: (as mentioned by #mjfroehlich)
echo sh(script: 'env', returnStdout: true)
The pure Groovy solutions that read the global env variable don't print all environment variables (e. g. they are missing variables from the environment block, from withEnv context and most of the machine-specific variables from the OS). Using shell steps it is possible to get a more complete set, but that requires a node context, which is not always wanted.
Here is a solution that uses the getContext step to retrieve and print the complete set of environment variables, including pipeline parameters, for the current context.
Caveat: Doesn't work in Groovy sandbox. You can use it from a trusted shared library though.
def envAll = getContext( hudson.EnvVars )
echo envAll.collect{ k, v -> "$k = $v" }.join('\n')
Show all variable in Windows system and Unix system is different, you can define a function to call it every time.
def showSystemVariables(){
if(isUnix()){
sh 'env'
} else {
bat 'set'
}
}
I will call this function first to show all variables in all pipline script
stage('1. Show all variables'){
steps {
script{
showSystemVariables()
}
}
}
The easiest and quickest way is to use following url to print all environment variables
http://localhost:8080/env-vars.html/
The answers above, are now antiquated due to new pipeline syntax. Below prints out the environment variables.
script {
sh 'env > env.txt'
String[] envs = readFile('env.txt').split("\r?\n")
for(String vars: envs){
println(vars)
}
}
Includes both system and build environment vars:
sh script: "printenv", label: 'print environment variables'
if you really want to loop over the env list just do:
def envs = sh(returnStdout: true, script: 'env').split('\n')
envs.each { name ->
println "Name: $name"
}
I found this is the most easiest way:
pipeline {
agent {
node {
label 'master'
}
}
stages {
stage('hello world') {
steps {
sh 'env'
}
}
}
}
You can get all variables from your jenkins instance. Just visit:
${jenkins_host}/env-vars.html
${jenkins_host}/pipeline-syntax/globals
ref: https://www.jenkins.io/doc/pipeline/tour/environment/
node {
sh 'printenv'
}
You can use sh 'printenv'
stage('1') {
sh "printenv"
}
another way to get exactly the output mentioned in the question:
envtext= "printenv".execute().text
envtext.split('\n').each
{ envvar=it.split("=")
println envvar[0]+" is "+envvar[1]
}
This can easily be extended to build a map with a subset of env vars matching a criteria:
envdict=[:]
envtext= "printenv".execute().text
envtext.split('\n').each
{ envvar=it.split("=")
if (envvar[0].startsWith("GERRIT_"))
envdict.put(envvar[0],envvar[1])
}
envdict.each{println it.key+" is "+it.value}
I suppose that you needed that in form of a script, but if someone else just want to have a look through the Jenkins GUI, that list can be found by selecting the "Environment Variables" section in contextual left menu of every build
Select project => Select build => Environment Variables

Load file with environment variables Jenkins Pipeline

I am doing a simple pipeline:
Build -> Staging -> Production
I need different environment variables for staging and production, so i am trying to source variables.
sh 'source $JENKINS_HOME/.envvars/stacktest-staging.sh'
But it returns Not found
[Stack Test] Running shell script
+ source /var/jenkins_home/.envvars/stacktest-staging.sh
/var/jenkins_home/workspace/Stack Test#tmp/durable-bcbe1515/script.sh: 2: /var/jenkins_home/workspace/Stack Test#tmp/durable-bcbe1515/script.sh: source: not found
The path is right, because i run the same command when i log via ssh, and it works fine.
Here is the pipeline idea:
node {
stage name: 'Build'
// git and gradle build OK
echo 'My build stage'
stage name: 'Staging'
sh 'source $JENKINS_HOME/.envvars/stacktest-staging.sh' // PROBLEM HERE
echo '$DB_URL' // Expects http://production_url/my_db
sh 'gradle flywayMigrate' // To staging
input message: "Does Staging server look good?"
stage name: 'Production'
sh 'source $JENKINS_HOME/.envvars/stacktest-production.sh'
echo '$DB_URL' // Expects http://production_url/my_db
sh 'gradle flywayMigrate' // To production
sh './deploy.sh'
}
What should i do?
I was thinking about not using pipeline (but i will not be able to use my Jenkinsfile).
Or make different jobs for staging and production, using EnvInject Plugin (But i lose my stage view)
Or make withEnv (but the code gets big, because today i am working with 12 env vars)
One way you could load environment variables from a file is to load a Groovy file.
For example:
Let's say you have a groovy file in '$JENKINS_HOME/.envvars' called 'stacktest-staging.groovy'.
Inside this file, you define 2 environment variables you want to load
env.DB_URL="hello"
env.DB_URL2="hello2"
You can then load this in using
load "$JENKINS_HOME/.envvars/stacktest-staging.groovy"
Then you can use them in subsequent echo/shell steps.
For example, here is a short pipeline script:
node {
load "$JENKINS_HOME/.envvars/stacktest-staging.groovy"
echo "${env.DB_URL}"
echo "${env.DB_URL2}"
}
From the comments to the accepted answer
Don't use global 'env' but use 'withEnv' construct, eg see:
issue #9: don't set env vars with global env in top 10 best practices jenkins pipeline plugin
In the following example: VAR1 is a plain java string (no groovy variable expansion), VAR2 is a groovy string (so variable 'someGroovyVar' is expanded).
The passed script is a plain java string, so $VAR1 and $VAR2 are passed literally to the shell, and the echo's are accessing environment variables VAR1 and VAR2.
stage('build') {
def someGroovyVar = 'Hello world'
withEnv(['VAR1=VALUE ONE',
"VAR2=${someGroovyVar}"
]) {
def result = sh(script: 'echo $VAR1; echo $VAR2', returnStdout: true)
echo result
}
}
For secrets / passwords you can use credentials binding plugin
Example:
NOTE: CREDENTIALS_ID1 is a registered username/password secret on the Jenkins settings.
stage('Push') {
withCredentials([usernamePassword(
credentialsId: 'CREDENTIALS_ID1',
passwordVariable: 'PASSWORD',
usernameVariable: 'USER')]) {
echo "User name: $USER"
echo "Password: $PASSWORD"
}
}
The jenkisn console log output hides the real values:
[Pipeline] echo
User name: ****
[Pipeline] echo
Password: ****
Jenkins and credentials is a big issue, probably see: credentials plugin
For completeness: Most of the time, we need the secrets in environment variables, as we use them from shell scripts, so we combine the withCredentials and withEnv like follows:
stage('Push') {
withCredentials([usernamePassword(
credentialsId: 'CREDENTIALS_ID1',
passwordVariable: 'PASSWORD',
usernameVariable: 'USER')]) {
withEnv(["ENV_USERNAME=${USER}",
"ENV_PASSWORD=${PASSWORD}"
]) {
def result = sh(script: 'echo $ENV_USERNAME', returnStdout: true)
echo result
}
}
}
Another way to resolve this install 'Pipeline Utility Steps' plugin that provides us readProperties method ( for reference please go to the link https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#pipeline-utility-steps)
Here in the example we can see that they are storing the keys into an array and using the keys to retrieve the value.
But in that case the in production the problem will be like if we add any variable later into property file that variable needs to be added into the array of Jenkins file as well.
To get rid of this tight coupling, we can write code in such a way so that the Jenkins build environment can get information automatically about all the existing keys which presents currently in the Property file. Here is an example for the reference
def loadEnvironmentVariables(path){
def props = readProperties file: path
keys= props.keySet()
for(key in keys) {
value = props["${key}"]
env."${key}" = "${value}"
}
}
And the client code looks like
path = '\\ABS_Output\\EnvVars\\pic_env_vars.properties'
loadEnvironmentVariables(path)
With declarative pipeline, you can do it in one line ( change path by your value):
script {
readProperties(file: path).each {key, value -> env[key] = value }
}
Using withEnv() to pass environment variables from file splitted by new line and casted to List:
writeFile file: 'version.txt', text: 'version=6.22.0'
withEnv(readFile('version.txt').split('\n') as List) {
sh "echo ${version}"
}
If you are using Jenkins 2.0 you can load the property file (which consists of all required Environment variables along with their corresponding values) and read all the environment variables listed there automatically and inject it into the Jenkins provided env entity.
Here is a method which performs the above stated action.
def loadProperties(path) {
properties = new Properties()
File propertiesFile = new File(path)
properties.load(propertiesFile.newDataInputStream())
Set<Object> keys = properties.keySet();
for(Object k:keys){
String key = (String)k;
String value =(String) properties.getProperty(key)
env."${key}" = "${value}"
}
}
To call this method we need to pass the path of property file as a string variable For example, in our Jenkins file using groovy script we can call like
path = "${workspace}/pic_env_vars.properties"
loadProperties(path)
Please ask me if you have any doubt
Here is a complete example of externalizing environment variables and loading them in Jenkins pipeline execution. The pipeline is written in a declarative style.
stage('Reading environment variable defined in groovy file') {
steps {
script {
load "./pipeline/basics/extenvvariable/env.groovy"
echo "${env.env_var1}"
echo "${env.env_var2}"
}
}
}
Complete code example:
https://github.com/dhruv-bansal/jenkins-pipeline-exploration/blob/master/pipeline/basics/extenvvariable/Jenkinsfile
Where variables are loaded from a groovy file placed with the pipeline code only.
https://github.com/dhruv-bansal/jenkins-pipeline-exploration/blob/master/pipeline/basics/extenvvariable/env.groovy
This pattern comes very handy when you are creating a generic pipeline that could be used across teams.
You can externalize the dependent variable in such groovy file and each team can define their values according to their ecosystem.
Another solution is to use a custom method without allowing extra permissions such as for new Properties() which leads to this error before allowing:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.util.Properties
or adding extra plugin methods such as readProperties.
here is a method which reads a simple file named env_vars in this format:
FOO=bar
FOO2=bar
pipeline {
<... skipped lines ...>
script {
loadEnvironmentVariablesFromFile("env_vars")
echo "show time! ${BAR} ${BAR2}"
}
<... skipped lines ...>
}
private void loadEnvironmentVariablesFromFile(String path) {
def file = readFile(path)
file.split('\n').each { envLine ->
def (key, value) = envLine.tokenize('=')
env."${key}" = "${value}"
}
}

How to access parameters in a Parameterized Build?

How do you access parameters set in the "This build is parameterized" section of a "Workflow" Jenkins job?
TEST CASE
Create a WORKFLOW job.
Enable "This build is parameterized".
Add a STRING PARAMETER foo with default value bar text.
Add the code below to Workflow Script:
node()
{
print "DEBUG: parameter foo = ${env.foo}"
}
Run job.
RESULT
DEBUG: parameter foo = null
I think the variable is available directly, rather than through env, when using Workflow plugin.
Try:
node()
{
print "DEBUG: parameter foo = ${foo}"
}
I tried a few of the solutions from this thread. It seemed to work, but my values were always true and I also encountered the following issue:
JENKINS-40235
I managed to use parameters in groovy jenkinsfile using the following syntax: params.myVariable
Here's a working example:
Solution
print 'DEBUG: parameter isFoo = ' + params.isFoo
print "DEBUG: parameter isFoo = ${params.isFoo}"
A more detailed (and working) example:
node() {
// adds job parameters within jenkinsfile
properties([
parameters([
booleanParam(
defaultValue: false,
description: 'isFoo should be false',
name: 'isFoo'
),
booleanParam(
defaultValue: true,
description: 'isBar should be true',
name: 'isBar'
),
])
])
// test the false value
print 'DEBUG: parameter isFoo = ' + params.isFoo
print "DEBUG: parameter isFoo = ${params.isFoo}"
sh "echo sh isFoo is ${params.isFoo}"
if (params.isFoo) { print "THIS SHOULD NOT DISPLAY" }
// test the true value
print 'DEBUG: parameter isBar = ' + params.isBar
print "DEBUG: parameter isBar = ${params.isBar}"
sh "echo sh isBar is ${params.isBar}"
if (params.isBar) { print "this should display" }
}
Output
[Pipeline] {
[Pipeline] properties
WARNING: The properties step will remove all JobPropertys currently configured in this job, either from the UI or from an earlier properties step.
This includes configuration for discarding old builds, parameters, concurrent builds and build triggers.
WARNING: Removing existing job property 'This project is parameterized'
WARNING: Removing existing job property 'Build triggers'
[Pipeline] echo
DEBUG: parameter isFoo = false
[Pipeline] echo
DEBUG: parameter isFoo = false
[Pipeline] sh
[wegotrade-test-job] Running shell script
+ echo sh isFoo is false
sh isFoo is false
[Pipeline] echo
DEBUG: parameter isBar = true
[Pipeline] echo
DEBUG: parameter isBar = true
[Pipeline] sh
[wegotrade-test-job] Running shell script
+ echo sh isBar is true
sh isBar is true
[Pipeline] echo
this should display
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
I sent a Pull Request to update the misleading pipeline tutorial#build-parameters quote that says "they are accessible as Groovy variables of the same name.". ;)
Edit: As Jesse Glick pointed out:
Release notes go into more details
You should also update the Pipeline Job Plugin to 2.7 or later, so that build parameters are defined as environment variables and thus accessible as if they were global Groovy variables.
When you add a build parameter, foo,
it gets converted to something which acts like a "bare variable",
so in your script you would do:
node {
echo foo
}
If you look at the implementation of the workflow script, you will see that when a script is executed, a class called WorkflowScript is
dynamically generated. All statements in the script are executed in the context of this class. All build parameters passed down to this script are converted to properties which are accessible from this class.
For example, you can do:
node {
getProperty("foo")
}
If you are curious, here is a workflow script I wrote which attempts to print out the build parameters, environment variables, and methods on the WorkflowScript class.
node {
echo "I am a "+getClass().getName()
echo "PARAMETERS"
echo "=========="
echo getBinding().getVariables().getClass().getName()
def myvariables = getBinding().getVariables()
for (v in myvariables) {
echo "${v} " + myvariables.get(v)
}
echo STRING_PARAM1.getClass().getName()
echo "METHODS"
echo "======="
def methods = getMetaClass().getMethods()
for (method in methods) {
echo method.getName()
}
echo "PROPERTIES"
echo "=========="
properties.each{ k, v ->
println "${k} ${v}"
}
echo properties
echo properties["class"].getName()
echo "ENVIRONMENT VARIABLES"
echo "======================"
echo "env is " + env.getClass().getName()
def envvars = env.getEnvironment()
envvars.each{ k, v ->
println "${k} ${v}"
}
}
Here is another code example I tried, where I wanted to test to see
if a build parameter was set or not.
node {
groovy.lang.Binding myBinding = getBinding()
boolean mybool = myBinding.hasVariable("STRING_PARAM1")
echo mybool.toString()
if (mybool) {
echo STRING_PARAM1
echo getProperty("STRING_PARAM1")
} else {
echo "STRING_PARAM1 is not defined"
}
mybool = myBinding.hasVariable("DID_NOT_DEFINE_THIS")
if (mybool) {
echo DID_NOT_DEFINE_THIS
echo getProperty("DID_NOT_DEFINE_THIS")
} else {
echo "DID_NOT_DEFINE_THIS is not defined"
}
}
Use double quotes instead of single quotes
e.g. echo "$foo" as opposed to echo '$foo'
If you configured your pipeline to accept parameters using the Build with Parameters option, those parameters are accessible as Groovy variables of the same name. See Here.
You can drop the semicolon (;), drop the parentheses (( and )), and use single quotes (') instead of double (") if you do not need to perform variable substitutions. See Here. This clued me into my problem, though I've found that only the double (") is required to make it work.
To parameter variable add prefix "params."
For example:
params.myParam
Don't forget: if you use some method of myParam, may be you should approve it in "Script approval".
You can also try using parameters directive for making your build parameterized and accessing parameters:
Doc:
Pipeline syntax: Parameters
Example:
pipeline{
agent { node { label 'test' } }
options { skipDefaultCheckout() }
parameters {
string(name: 'suiteFile', defaultValue: '', description: 'Suite File')
}
stages{
stage('Initialize'){
steps{
echo "${params.suiteFile}"
}
}
}
Hope the following piece of code works for you:
def item = hudson.model.Hudson.instance.getItem('MyJob')
def value = item.lastBuild.getEnvironment(null).get('foo')
The following snippet gives you access to all Job params
def myparams = currentBuild.rawBuild.getAction(ParametersAction)
for( p in myparams ) {
pMap[p.name.toString()] = p.value.toString()
}
Please note, the way that build parameters are accessed inside pipeline scripts (pipeline plugin) has changed. This approach:
getBinding().hasVariable("MY_PARAM")
Is not working anymore. Please try this instead:
def myBool = env.getEnvironment().containsKey("MY_BOOL") ? Boolean.parseBoolean("$env.MY_BOOL") : false
As per Pipeline plugin tutorial:
If you have configured your pipeline to accept parameters when it is built — Build with Parameters — they are accessible as Groovy variables of the same name.
So try to access the variable directly, e.g.:
node()
{
print "DEBUG: parameter foo = " + foo
print "DEBUG: parameter bar = ${bar}"
}

Resources