Jenkins - matrix jobs - variables on different slaves overwrite each other? - jenkins

I think i dont get how matrix builds work. When i set some variable in some stage depending on which node i run, then on rest of the stage sometimes this variable is set as it should and sometimes it gets values from other nodes (axes). In example below its like job which runs on ub18-1 sometimes has VARIABLE1='Linux node' and sometimes is VARIABLE1='Windows node'. Or gitmethod sometimes it is created from LinuxGitInfo and sometimes WindowsGitInfo.
Source i based on
https://jenkins.io/doc/book/pipeline/syntax/#declarative-matrix
Script almost exactly the same as real one
#Library('firstlibrary') _
import mylib.shared.*
pipeline {
parameters {
booleanParam name: 'AUTO', defaultValue: true, description: 'Auto mode sets some parameters for every slave separately'
choice(name: 'SLAVE_NAME', choices:['all', 'ub18-1','win10'],description:'Run on specific platform')
string(name: 'BRANCH',defaultValue: 'master', description: 'Preferably common label for entire group')
booleanParam name: 'SONAR', defaultValue: false, description: 'Scan and gateway'
booleanParam name: 'DEPLOY', defaultValue: false, description: 'Deploy to Artifactory'
}
agent none
stages{
stage('BuildAndTest'){
matrix{
agent {
label "$NODE"
}
when{ anyOf{
expression { params.SLAVE_NAME == 'all'}
expression { params.SLAVE_NAME == env.NODE}
}}
axes{
axis{
name 'NODE'
values 'ub18-1', 'win10'
}
}
stages{
stage('auto mode'){
when{
expression { return params.AUTO }
}
steps{
echo "Setting parameters for each slave"
script{
nodeLabelsList = env.NODE_LABELS.split()
if (nodeLabelsList.contains('ub18-1')){
println("Setting params for ub18-1");
VARIABLE1 = 'Linux node'
}
if (nodeLabelsList.contains('win10')){
println("Setting params for Win10");
VARIABLE1 = 'Windows node'
}
if (isUnix()){
gitmethod = new LinuxGitInfo(this,env)
} else {
gitmethod = new WindowsGitInfo(this, env)
}
}
}
}
stage('GIT') {
steps {
checkout scm
}
}
stage('Info'){
steps{
script{
sh 'printenv'
echo "branch: " + env.BRANCH_NAME
echo "SLAVE_NAME: " + env.NODE_NAME
echo VARIABLE1
gitinfo = new GitInfo(gitmethod)
gitinfo.init()
echo gitinfo.author
echo gitinfo.id
echo gitinfo.msg
echo gitinfo.buildinfo
}
}
}
stage('install'){
steps{
sh 'make install'
}
}
stage('test'){
steps{
sh 'make test'
}
}
}
}
}
}
}

Ok i solved the problem by defining variables maps with node/slave names as keys. Some friend even suggested to define variables in yml/json file in repository and parse them. Maybe i will, but so far this works well
example:
before the pipelines
def DEPLOYmap = [
'ub18-1': false,
'win10': true
]
in stages
when {
equals expected: true, actual: DEPLOYmap[NODE]
}

Related

Access a Groovy variable within shell step in Jenkins pipeline

SSHUER in stage one prints as required. But in stage two it doesn't print and gives me a null value. All three attempt statements in stage two are not working. I can't use script block as I have some special characters in the actual code. Any suggestions as to how to fetch the value of SSHUSER in stage two.
def SSHUSER
environment {
if(params.CLOUD == 'google') {
SSHUSER = "test1"
} else if (params.CLOUD == 'azure') {
SSHUSER = "test2"
}
}
pipeline {
stage('stage one') {
steps {
echo "first attempt '${SSHUSER}'"
}
}
stage('stage two') {
steps {
sh '''#!/bin/bash
echo "first attempt '${SSHUSER}'"
echo "second attempt \${SSHUSER}"
echo "thrid attempt \$SSHUSER"
if ssh \${SSHUSER}#$i 'test -e /tmp/test'";
then
echo "$i file exists "
fi
'''
}
}
}
your usage of the environment block is a bit weird, it should reside inside the pipeline block and variables should be initialized inside the environment block.
If it is defined outside the variables will not be passed to the shell script as environment variables.
If you want to use the environment block inside the `pipeline' block just define the parameters (as strings) you want and they will be available for all stages, and they will be passed as environment variables for the shell.
For example:
pipeline {
agent any
parameters{
string(name: 'CLOUD', defaultValue: 'google', description: '')
}
environment {
SSHUSER = "${params.CLOUD == 'google' ? 'test1' : 'test2'}"
}
stages {
stage('stage one') {
steps {
echo "first attempt '${SSHUSER}'"
}
}
stage('stage two') {
steps {
sh '''#!/bin/bash
echo "first attempt '${SSHUSER}'"
echo "second attempt \${SSHUSER}"
echo "thrid attempt \$SSHUSER"
if ssh \${SSHUSER}#$i 'test -e /tmp/test'";
then
echo "$i file exists "
fi
'''
}
}
}
}
If you don't want to use the environment block or mange the parameters by yourself you can do it outside the pipeline block using Global variables but then they will not be passed environment variables and you will need to use string interpolation (""" instead of ''') to calculate the values when passing the command to the shell:
SSHUSER = ''
if(params.CLOUD == 'google') {
SSHUSER = "test1"
} else if (params.CLOUD == 'azure') {
SSHUSER = "test2"
}
pipeline {
agent any
parameters{
string(name: 'CLOUD', defaultValue: 'google', description: 'Bitbucket Payload', trim: true)
}
stages {
stage('stage one') {
steps {
echo "first attempt '${SSHUSER}'"
}
}
stage('stage two') {
steps {
sh """#!/bin/bash
echo "first attempt '${SSHUSER}'"
echo "second attempt ${SSHUSER}"
echo "thrid attempt $SSHUSER"
if ssh ${SSHUSER}#\$i 'test -e /tmp/test'";
then
echo "\$i file exists "
fi
"""
}
}
}
}

Jenkins parameterized input in stage

I know how to request for the user input for the whole pipeline using parameters directive. Now i want to be able to request for the user input inside a specific stage and be able to access that value inside the stage. Here's my pipeline:
pipeline {
agent none
stages {
stage('Stage 1') {
when{
beforeAgent true
expression {
timeout (time: 30, unit: "SECONDS"){
input message: 'Should we continue?', ok: 'Yes',
parameters:[[
$class: 'ChoiceParameterDefinition',
choices: ['Stable release', 'SNAPSHOT'],
description: 'Which Version?',
name: 'version'
]]
}
return true
}
}
agent any
steps {
echo 'Checking dependencies ...'
echo "${params}"
}
}
}
}
In this pipeline I'm able to prompt the user to choose between Stable release and SNAPSHOT inside Stage 1 stage. However I'm not able to access this variable using ${params.version}. Any ideas how to solve this?
I managed to work around the problem and read the input chosen by user as in the following pipeline:
def version //define a global variable for the whole pipeline.
pipeline {
agent none
stages {
stage('Stage 1') {
when{
beforeAgent true
expression {
timeout (time: 30, unit: "SECONDS"){
//Assign the variable here.
version = input message: 'Should we continue?', ok: 'Yes',
parameters:[[
$class: 'ChoiceParameterDefinition',
choices: ['Stable release', 'SNAPSHOT'],
description: 'Which Version?',
name: 'v'
]]
}
return true
}
}
agent any
steps {
// And finally access it.
echo "${version}"
}
}
}
}

Jenkins: Trigger another job with branch name

I am using Jenkins Pipeline via declarative and I would like to trigger another job with branch name.
For instance, I have two different pipeline(PipelineA -PipelineB) with stages JobA and JobB.
One of the stage for JobA should trigger the JobB via paramater using env.GIT_BRANCH. What I mean, if we trigger the JobA via origin/develop, then it should trigger the 'JobB' and run the stages where it has origin/develop condition.
Meanwhile, we also making some separate changes on JobB and it also has its own GIT_BRANCH expression.Thus I could not able to find a way to manage this separately without affecting JobA. To be clarify, when JobA trigger JobB with origin/stage parameter, due to latest changes on JobB is origin/development whereas GIT_BRANCH is origin/development, I can not able to run the stages which has stage condition.
Here is my script.
stage ('Job A') {
steps {
script {
echo "Triggering job for branch ${env.GIT_BRANCH}"
ret = build(job: "selenium_tests",
parameters: [
string(name: "projectName", value: "Project1"),
string(name: "branchName", value: "env.GIT_BRANCH")
],
propagate: true,
wait: true)
echo ret.result
currentBuild.result = ret.result
}
}
}
parameters {
string(defaultValue: "project1", description: 'Which project do you want to test?', name: 'projectName')
string(defaultValue: "origin/development", description: 'Environment for selenium tests', name:'branchName')
}
stage ('Job B') {
when {
beforeAgent true
expression { params.projectName == 'Project1' }
expression { params.branchName == "origin/stage"}
expression{ return env.GIT_BRANCH == "origin/stage"}
}
steps {
script {
//Do something
}
}
}
Pass down one more param for branch when trigger Job B
stage('Trigger Job A') {}
stage('Trigger Job B') {
when {
allOf {
beforeAgent true
expression { params.projectName == 'Project1' }
expression{ return env.GIT_BRANCH == "origin/stage"}
}
}
steps {
build(job: "selenium_tests/Job B",
parameters: [
string(name: "projectName", value: "Project1")
strint(name: "branchName", value: "${env.GIT_BRANCH}")
],
propagate: true,
wait: true)
}
}
In Job B' Jenkinsfile add one stage as the first stage to switch to desired branch
pipeline {
parameters {
string(name: 'branchName', defaultValue: 'develop')
}
stages {
stage('Switch branch') {
steps {
sh "git checkout ${params.branchName}"
}
}
// other stages
}
}

Store current workspace path in variable for a later stage

I am using the declarative syntax for my pipeline, and would like to store the path to the workspace being used on one of my stages, so that same path can be used in a later stage.
I have seen I can call pwd() to get the current directory, but how do I assign to a variable to be used between stages?
EDIT
I have tried to do this by defining by own custom variable and using like so with the ws directive:
pipeline {
agent { label 'master' }
stages {
stage('Build') {
steps {
script {
def workspace = pwd()
}
sh '''
npm install
bower install
gulp set-staging-node-env
gulp prepare-staging-files
gulp webpack
'''
stash includes: 'dist/**/*', name: 'builtSources'
stash includes: 'config/**/*', name: 'appConfig'
node('Protractor') {
dir('/opt/foo/deploy/') {
unstash 'builtSources'
unstash 'appConfig'
}
}
}
}
stage('Unit Tests') {
steps {
parallel (
"Jasmine": {
node('master') {
ws("${workspace}"){
sh 'gulp karma-tests-ci'
}
}
},
"Mocha": {
node('master') {
ws("${workspace}"){
sh 'gulp mocha-tests'
}
}
}
)
}
post {
success {
sh 'gulp combine-coverage-reports'
sh 'gulp clean-lcov'
publishHTML(target: [
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: false,
reportDir: 'test/coverage',
reportFiles: 'index.html',
reportName: 'Test Coverage Report'
])
}
}
}
}
}
In the Jenkins build console, I see this happens:
[Jasmine] Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ#2
[Pipeline] [Jasmine] {
[Pipeline] [Jasmine] ws
[Jasmine] Running in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ#2#2
The original workspace allocated from the first stage is actually _Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ
So it doesnt look like it working, what am I doing wrong here?
Thanks
pipeline {
agent none
stages {
stage('Stage-One') {
steps {
echo 'StageOne.....'
script{ name = 'StackOverFlow'}
}
}
stage('Stage-Two'){
steps{
echo 'StageTwo.....'
echo "${name}"
}
}
}
}
Above prints StackOverFlow in StageTwo for echo "${name}"
You can also use sh "echo ${env.WORKSPACE}" to get The absolute path of the directory assigned to the build as a workspace.
You could put the value into an environment variable like described in this answer
CURRENT_PATH= sh (
script: 'pwd',
returnStdout: true
).trim()
Which version are you running? Maybe you can just assign the WORKSPACE variable to an environment var?
Or did i totally misunderstand and this is what you are looking for?

How do I pass variables between stages in a declarative Jenkins pipeline?

How do I pass variables between stages in a declarative pipeline?
In a scripted pipeline, I gather the procedure is to write to a temporary file, then read the file into a variable.
How do I do this in a declarative pipeline?
E.g. I want to trigger a build of a different job, based on a variable created by a shell action.
stage("stage 1") {
steps {
sh "do_something > var.txt"
// I want to get var.txt into VAR
}
}
stage("stage 2") {
steps {
build job: "job2", parameters[string(name: "var", value: "${VAR})]
}
}
If you want to use a file (since a script is the thing generating the value you need), you could use readFile as seen below. If not, use sh with the script option as seen below:
// Define a groovy local variable, myVar.
// A global variable without the def, like myVar = 'initial_value',
// was required for me in older versions of jenkins. Your mileage
// may vary. Defining the variable here maybe adds a bit of clarity,
// showing that it is intended to be used across multiple stages.
def myVar = 'initial_value'
pipeline {
agent { label 'docker' }
stages {
stage('one') {
steps {
echo "1.1. ${myVar}" // prints '1.1. initial_value'
sh 'echo hotness > myfile.txt'
script {
// OPTION 1: set variable by reading from file.
// FYI, trim removes leading and trailing whitespace from the string
myVar = readFile('myfile.txt').trim()
}
echo "1.2. ${myVar}" // prints '1.2. hotness'
}
}
stage('two') {
steps {
echo "2.1 ${myVar}" // prints '2.1. hotness'
sh "echo 2.2. sh ${myVar}, Sergio" // prints '2.2. sh hotness, Sergio'
}
}
// this stage is skipped due to the when expression, so nothing is printed
stage('three') {
when {
expression { myVar != 'hotness' }
}
steps {
echo "three: ${myVar}"
}
}
}
}
Simply:
pipeline {
parameters {
string(name: 'custom_var', defaultValue: '')
}
stage("make param global") {
steps {
tmp_param = sh (script: 'most amazing shell command', returnStdout: true).trim()
env.custom_var = tmp_param
}
}
stage("test if param was saved") {
steps {
echo "${env.custom_var}"
}
}
}
I had a similar problem as I wanted one specific pipeline to provide variables and many other ones using it to get this variables.
I created a my-set-env-variables pipeline
script
{
env.my_dev_version = "0.0.4-SNAPSHOT"
env.my_qa_version = "0.0.4-SNAPSHOT"
env.my_pp_version = "0.0.2"
env.my_prd_version = "0.0.2"
echo " My versions [DEV:${env.my_dev_version}] [QA:${env.my_qa_version}] [PP:${env.my_pp_version}] [PRD:${env.my_prd_version}]"
}
I can reuse these variables in a another pipeline my-set-env-variables-test
script
{
env.dev_version = "NOT DEFINED DEV"
env.qa_version = "NOT DEFINED QA"
env.pp_version = "NOT DEFINED PP"
env.prd_version = "NOT DEFINED PRD"
}
stage('inject variables') {
echo "PRE DEV version = ${env.dev_version}"
script
{
def variables = build job: 'my-set-env-variables'
def vars = variables.getBuildVariables()
//println "found variables" + vars
env.dev_version = vars.my_dev_version
env.qa_version = vars.my_qa_version
env.pp_version = vars.my_pp_version
env.prd_version = vars.my_prd_version
}
}
stage('next job') {
echo "NEXT JOB DEV version = ${env.dev_version}"
echo "NEXT JOB QA version = ${env.qa_version}"
echo "NEXT JOB PP version = ${env.pp_version}"
echo "NEXT JOB PRD version = ${env.prd_version}"
}
there is no need for (hidden plugin) parameter definitions or temp-file access. Sharing varibles across stages can be acomplished by using global Groovy variables in a Jenkinsfile like so:
#!/usr/bin/env groovy
def MYVAR
def outputOf(cmd) { return sh(returnStdout:true,script:cmd).trim(); }
pipeline {
agent any
stage("stage 1") {
steps {
MYVAR = outputOf('echo do_something')
sh "echo MYVAR has been set to: '${MYVAR}'"
}
}
stage("stage 2") {
steps {
sh '''echo "...in multiline quotes: "''' + MYVAR + '''" ... '''
build job: "job2", parameters[string(name: "var", value: MYVAR)]
}
}
}
I have enhanced the existing solution by correcting syntax .Also used hidden parameter plugin so that it does not show up as an extra parameter in Jenkins UI. Works well :)
properties([parameters([[$class: 'WHideParameterDefinition', defaultValue: 'yoyo', name: 'hidden_var']])])
pipeline {
agent any
stages{
stage("make param global") {
steps {
script{
env.hidden_var = "Hello"
}
}
}
stage("test if param was saved") {
steps {
echo"About to check result"
echo "${env.hidden_var}"
}
}
}
}

Resources