Passing variables extracted from shell in Jenkinsfile - jenkins

I am trying to pass variables extracted in a stage in Jenkinsfile between stages. For example:
stage('Dummy Stage') {
sh '''#!/bin/bash -l
export abc=`output of some command`
.....
.....
'''
Now, how can I pass the variable abc to a subsequent stage? I have tried setting the variable by adding a def section at the top of the file but looks like it doesnt work. In the absence of a neater way, I am having to retype the commands

Here is what I do to get the number of commits on master as a global environment variable:
pipeline {
agent any
environment {
COMMITS_ON_MASTER = sh(script: "git rev-list HEAD --count", returnStdout: true).trim()
}
stages {
stage("Print commits") {
steps {
echo "There are ${env.COMMITS_ON_MASTER} commits on master"
}
}
}
}

You can use the longer form of the sh step and return the output (see Pipeline document). Your variable should be defined outside the stages.

You can use an imperatively created environment variable inside a script block in you stage steps, for example:
stage("Stage 1") {
steps {
script {
env.RESULT_ON_STAGE_1 = sh (
script: 'echo "Output of some command"',
returnStdout: true
)
}
echo "In stage 1: ${env.RESULT_ON_STAGE_1}"
}
}
stage("Stage 2") {
steps {
echo "In stage 2: ${env.RESULT_ON_STAGE_1}"
}
}
This guide explains use of environment variables in pipelines with examples.

My issue concerned having two 'sh' commands where one uses single quotes (where I set a variable) and the other uses double quotes (where I access 'env' variables set in the jenkinsfile such as BUILD_ID).
Here's how I solved it.
script {
env.TEST = sh(
script:
'''
echo "TEST"
''',
returnStdout: true
)
sh """
echo ${env.BUILD_ID}
echo ${env.TEST}
"""
}

Related

How to pass variables set in sh script to subsequent Jenkins Pipeline Steps

I have a jenkins pipeline file where i need to call an sh file
node {
stage("Stage1") {
checkout scm
sh '''
echo "Invoking the sh script"
valueNeedstobepassed = "test"
'''
}
stage ('stage2') {
Need to refer the "valueNeedstobepassed" varaible in my
pipleline step
}
}
I am not able to refer the variable "valueNeedstobepassed" on stage 2
Any help please?

Jenkins pipeline "when" condition with sh defined variable

I'm trying to create a Jenkins pipeline where I in my first stage I define a variable in a sh shell script.
Then I want to run the next stages using a "when" condition depending on the previous defined variable.
pipeline {
agent { label 'php71' }
stages {
stage('Prepare CI...') {
steps{
sh '''
# Get the comment that was made on the PR
COMMENT=`echo $payload | jq .comment.body | tr -d '"'`
if [ "$COMMENT" = ":repeat: Jenkins" ]; then
BUILD="build"
fi
'''
}
}
stage('Build Pre Envrionment') {
agent { label 'php71' }
when {
expression { return $BUILD == "build" }
}
steps('Build') {
sh '''
echo $BUILD
echo $COMMENT
'''
}
}
}
}
This gives me an error:
groovy.lang.MissingPropertyException: No such property: $BUILD for class: groovy.lang.Binding
How can I do it? Is it possible?
Thank you!
Probably use Jenkins scripted pipeline which is more flexible than declarative.
Print the value in the sh script and use returnStdout to make it available to the pipeline script. See How to do I get the output of a shell command executed using into a variable from Jenkinsfile (groovy)? for more details.

Using env variables to set other variables in Jenkins pipeline as code

I cannot use environment variables set in previous blocks in access stage below.
pipeline{
agent any
stages{
stage("set env variable"){
steps{
script{
env.city = "Houston"
}
}
}
}
stage("access"){
steps{
sh """
set brf = ${env.city}
echo $brf
"""
}
}
}
}
ERROR: groovy.lang.MissingPropertyException: No such property: brf for class: groovy.lang.Binding
What is an easier way to use jenkins declarative pipeline env variables ?
I cannot use environment variables set in previous blocks in access stage below.
If you look closely at the error, you can see Jenkins is actually unable to access brf, not env.city.
The issue here is caused by the way Jenkins interprets $var inside sh block:
if you use "double quotes", $var in sh "... $var ..." will be interpreted as Jenkins variable;
if you use 'single quotes', $var in sh '... $var ...' will be interpreted as shell variable.
Since the sh code in your script is wrapped in "double quotes", $brf is considered to be a Jenkins variable, while there is no such variable defined, therefore the error occurs.
To use shell variable inside double-quoted block add \ before $:
sh "echo \$var"
works the same way as
sh 'echo $var'
This should fix your pipeline script:
pipeline{
agent any
stages{
stage("set env variable"){
steps{
script{
env.city = "Houston"
}
}
}
stage("access"){
steps{
sh """
brf=${env.city}
echo \$brf
"""
}
}
}
}
Output from the pipeline:
[test] Running shell script
+ brf=Houston
+ echo Houston
Houston
You should not have any problem to get the variables with this code:
stage("access"){
steps{
sh "set brf = ${env.city}"
echo '$brf'
//or
sh "set brf = ${env.city} && echo $brf"
}
}
I think this is what you had asked but let me know if you have another doubt.

Pass variables between Jenkins stages

I want to pass a variable which I read in stage A towards stage B somehow. I see in some examples that people write it to a file, but I guess that is not really a nice solution. I tried writing it to an environment variable, but I'm not really successful on that. How can I set it up properly?
To get it working I tried a lot of things and read that I should use the """ instead of ''' to start a shell and escape those variables to \${foo} for example.
Below is what I have as a pipeline:
#!/usr/bin/env groovy
pipeline {
agent { node { label 'php71' } }
environment {
packageName='my-package'
packageVersion=''
groupId='vznl'
nexus_endpoint='http://nexus.devtools.io'
nexus_username='jenkins'
nexus_password='J3nkins'
}
stages{
// Package dependencies
stage('Install dependencies') {
steps {
sh '''
echo Skip composer installation
#composer install --prefer-dist --optimize-autoloader --no-interaction
'''
}
}
// Unit tests
stage('Unit Tests') {
steps {
sh '''
echo Running PHP code coverage tests...
#composer test
'''
}
}
// Create artifact
stage('Package') {
steps {
echo 'Create package refs'
sh """
mkdir -p ./build/zpk
VERSIONTAG=\$(grep 'version' composer.json)
REGEX='"version": "([0-9]+.[0-9]+.[0-9]+)"'
if [[ \${VERSIONTAG} =~ \${REGEX} ]]
then
env.packageVersion=\${BASH_REMATCH[1]}
/usr/bin/zs-client packZpk --folder=. --destination=./build/zpk --name=${env.packageName}-${env.packageVersion}.zpk --version=${env.packageVersion}
else
echo "No version found!"
exit 1
fi
"""
}
}
// Publish ZPK package to Nexus
stage('Publish packages') {
steps {
echo "Publish ZPK Package"
sh "curl -u ${env.nexus_username}:${env.nexus_password} --upload-file ./build/zpk/${env.packageName}-${env.packageVersion}.zpk ${env.nexus_endpoint}/repository/zpk-packages/${groupId}/${env.packageName}-${env.packageVersion}.zpk"
archive includes: './build/**/*.{zpk,rpm,deb}'
}
}
}
}
As you can see the packageVersion which I read from stage Package needs to be used in stage Publish as well.
Overall tips against the pipeline are of course always welcome as well.
A problem in your code is that you are assigning version of environment variable within the sh step. This step will execute in its own isolated process, inheriting parent process environment variables.
However, the only way of passing data back to the parent is through STDOUT/STDERR or exit code. As you want a string value, it is best to echo version from the sh step and assign it to a variable within the script context.
If you reuse the node, the script context will persist, and variables will be available in the subsequent stage. A working example is below. Note that any try to put this within a parallel block can be of failure, as the version information variable can be written to by multiple processes.
#!/usr/bin/env groovy
pipeline {
environment {
AGENT_INFO = ''
}
agent {
docker {
image 'alpine'
reuseNode true
}
}
stages {
stage('Collect agent info'){
steps {
echo "Current agent info: ${env.AGENT_INFO}"
script {
def agentInfo = sh script:'uname -a', returnStdout: true
println "Agent info within script: ${agentInfo}"
AGENT_INFO = agentInfo.replace("/n", "")
env.AGENT_INFO = AGENT_INFO
}
}
}
stage("Print agent info"){
steps {
script {
echo "Collected agent info: ${AGENT_INFO}"
echo "Environment agent info: ${env.AGENT_INFO}"
}
}
}
}
}
Another option which doesn't involve using script, but is just declarative, is to stash things in a little temporary environment file.
You can then use this stash (like a temporary cache that only lives for the run) if the workload is sprayed out across parallel or distributed nodes as needed.
Something like:
pipeline {
agent any
stages {
stage('first stage') {
steps {
// Write out any environment variables you like to a temporary file
sh 'echo export FOO=baz > myenv'
// Stash away for later use
stash 'myenv'
}
}
stage ("later stage") {
steps {
// Unstash the temporary file and apply it
unstash 'myenv'
// use the unstashed vars
sh 'source myenv && echo $FOO'
}
}
}
}

iterate over environment variables in Jenkins Pipeline Groovy [duplicate]

Given a jenkins build pipeline, jenkins injects a variable env into the node{}. Variable env holds environment variables and values.
I want to print all env properties within the jenkins pipeline. However, I do no not know all env properties ahead of time.
For example, environment variable BRANCH_NAME can be printed with code
node {
echo ${env.BRANCH_NAME}
...
But again, I don't know all variables ahead of time. I want code that handles that, something like
node {
for(e in env){
echo e + " is " + ${e}
}
...
which would echo something like
BRANCH_NAME is myBranch2
CHANGE_ID is 44
...
I used Jenkins 2.1 for this example.
According to Jenkins documentation for declarative pipeline:
sh 'printenv'
For Jenkins scripted pipeline:
echo sh(script: 'env|sort', returnStdout: true)
The above also sorts your env vars for convenience.
Another, more concise way:
node {
echo sh(returnStdout: true, script: 'env')
// ...
}
cf. https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#code-sh-code-shell-script
The following works:
#NonCPS
def printParams() {
env.getEnvironment().each { name, value -> println "Name: $name -> Value $value" }
}
printParams()
Note that it will most probably fail on first execution and require you approve various groovy methods to run in jenkins sandbox. This is done in "manage jenkins/in-process script approval"
The list I got included:
BUILD_DISPLAY_NAME
BUILD_ID
BUILD_NUMBER
BUILD_TAG
BUILD_URL
CLASSPATH
HUDSON_HOME
HUDSON_SERVER_COOKIE
HUDSON_URL
JENKINS_HOME
JENKINS_SERVER_COOKIE
JENKINS_URL
JOB_BASE_NAME
JOB_NAME
JOB_URL
You can accomplish the result using sh/bat step and readFile:
node {
sh 'env > env.txt'
readFile('env.txt').split("\r?\n").each {
println it
}
}
Unfortunately env.getEnvironment() returns very limited map of environment variables.
Why all this complicatedness?
sh 'env'
does what you need (under *nix)
Cross-platform way of listing all environment variables:
if (isUnix()) {
sh env
}
else {
bat set
}
Here's a quick script you can add as a pipeline job to list all environment variables:
node {
echo(env.getEnvironment().collect({environmentVariable -> "${environmentVariable.key} = ${environmentVariable.value}"}).join("\n"))
echo(System.getenv().collect({environmentVariable -> "${environmentVariable.key} = ${environmentVariable.value}"}).join("\n"))
}
This will list both system and Jenkins variables.
I use Blue Ocean plugin and did not like each environment entry getting its own block. I want one block with all the lines.
Prints poorly:
sh 'echo `env`'
Prints poorly:
sh 'env > env.txt'
for (String i : readFile('env.txt').split("\r?\n")) {
println i
}
Prints well:
sh 'env > env.txt'
sh 'cat env.txt'
Prints well: (as mentioned by #mjfroehlich)
echo sh(script: 'env', returnStdout: true)
The pure Groovy solutions that read the global env variable don't print all environment variables (e. g. they are missing variables from the environment block, from withEnv context and most of the machine-specific variables from the OS). Using shell steps it is possible to get a more complete set, but that requires a node context, which is not always wanted.
Here is a solution that uses the getContext step to retrieve and print the complete set of environment variables, including pipeline parameters, for the current context.
Caveat: Doesn't work in Groovy sandbox. You can use it from a trusted shared library though.
def envAll = getContext( hudson.EnvVars )
echo envAll.collect{ k, v -> "$k = $v" }.join('\n')
Show all variable in Windows system and Unix system is different, you can define a function to call it every time.
def showSystemVariables(){
if(isUnix()){
sh 'env'
} else {
bat 'set'
}
}
I will call this function first to show all variables in all pipline script
stage('1. Show all variables'){
steps {
script{
showSystemVariables()
}
}
}
The easiest and quickest way is to use following url to print all environment variables
http://localhost:8080/env-vars.html/
The answers above, are now antiquated due to new pipeline syntax. Below prints out the environment variables.
script {
sh 'env > env.txt'
String[] envs = readFile('env.txt').split("\r?\n")
for(String vars: envs){
println(vars)
}
}
Includes both system and build environment vars:
sh script: "printenv", label: 'print environment variables'
if you really want to loop over the env list just do:
def envs = sh(returnStdout: true, script: 'env').split('\n')
envs.each { name ->
println "Name: $name"
}
I found this is the most easiest way:
pipeline {
agent {
node {
label 'master'
}
}
stages {
stage('hello world') {
steps {
sh 'env'
}
}
}
}
You can get all variables from your jenkins instance. Just visit:
${jenkins_host}/env-vars.html
${jenkins_host}/pipeline-syntax/globals
ref: https://www.jenkins.io/doc/pipeline/tour/environment/
node {
sh 'printenv'
}
You can use sh 'printenv'
stage('1') {
sh "printenv"
}
another way to get exactly the output mentioned in the question:
envtext= "printenv".execute().text
envtext.split('\n').each
{ envvar=it.split("=")
println envvar[0]+" is "+envvar[1]
}
This can easily be extended to build a map with a subset of env vars matching a criteria:
envdict=[:]
envtext= "printenv".execute().text
envtext.split('\n').each
{ envvar=it.split("=")
if (envvar[0].startsWith("GERRIT_"))
envdict.put(envvar[0],envvar[1])
}
envdict.each{println it.key+" is "+it.value}
I suppose that you needed that in form of a script, but if someone else just want to have a look through the Jenkins GUI, that list can be found by selecting the "Environment Variables" section in contextual left menu of every build
Select project => Select build => Environment Variables

Resources