I am using a declarative pipeline in a Jenkinsfile but I would like to derive some variables from a parameter.
For example given:
parameters {
choice(name: 'Platform',choices: ['Debian9', 'CentOS7'], description: 'Target OS platform', )
}
I would like to add a block like:
script {
switch(param.Platform) {
case "Centos7":
def DockerFile = 'src/main/docker/Jenkins-Centos.Dockerfile'
def PackageType = 'RPM'
def PackageSuffix = '.rpm'
break
case "Debian9":
default:
def DockerFile = 'src/main/docker/Jenkins-Debian.Dockerfile'
def PackageType = 'DEB'
def PackageSuffix = '.deb'
break
}
}
Such that I can use variables elsewhere in the pipeline. For example:
agent {
dockerfile {
filename "$DockerFile"
}
}
etc..
but script is illegal in the parameter, environment & agent sections.
It can only be used in steps.
I need to use the parameter in the agent block and I want to avoid repeating myself where the variables are used in different steps.
Is there a sane way to achieve this? My preferences in order are:
a declarative pipeline
a scripted pipeline (less good)
via a plugin to the Jenkins UI (least good)
A shared library might be appropriate here regardless of whether it is actually shared.
The intention is to support a multi-configuration project by creating a parameterised build and invoking it for different parameter sets with a red/blue status light for each configuration.
It could be that I have assumed an 'old fashioned' design. In which case an acceptable answer would explain the modern best practice for creating a multi-configuration multi-branch pipeline. Something like: https://support.cloudbees.com/hc/en-us/articles/115000088431-Create-a-Matrix-like-flow-with-Pipeline or Jenkins Pipeline Multiconfiguration Project
See also Multiconfiguration / matrix build pipeline in Jenkins for less specific discussion of best practices.
Never really used the Jenkins declarative pipeline before but I think the way you refer to params is incorrect?
I think it might be: ${params.Platform} or params.Platform instead of param.
So something like the below maybe?
pipeline {
agent any
stages {
stage('example') {
steps {
script {
switch(${params.Platform}) {
...
}
}
}
}
}
}
As I said, never really used it before so not 100%. I was just looking at the syntax used for parameters on the docs: https://jenkins.io/doc/book/pipeline/syntax/#parameters
I think that the key for solving your issue is the declaration of your variables. Do not use def if you want your variable to be accessible from other stages.
Here is an example of a solution for your issue :
pipeline{
agent none
parameters {
choice(name: 'Platform',choices: ['Debian9', 'CentOS7'], description: 'Target OS platform', )
}
stages{
stage('Setting stage'){
agent any
steps {
script {
switch(params.Platform){
case 'CentOS7' :
DockerFile = 'src/main/docker/Jenkins-Centos.Dockerfile'
PackageType = 'RPM'
PackageSuffix = '.rpm'
break
case 'Debian9' :
DockerFile = 'src/main/docker/Jenkins-Debian.Dockerfile'
PackageType = 'DEB'
PackageSuffix = '.deb'
break
}
}
}
}
stage('Echo stage'){
agent {
dockerfile {
filename "$DockerFile"
}
}
steps{
echo PackageType
echo PackageSuffix
}
}
}
}
What is definitely possible on windows:
stage('var from groovy') {
steps {
script {
anvar = "foo"
}
bat "${anyvar}"
}
}
This is an example that I have in production
def dest_app_instance = "${params.Destination}"
switch(dest_app_instance) {
case "CASE1":
dest_server = "server1"
break
case "CASE2":
dest_server = "server2"
break
}
Related
From my experience with Jenkins declarative-syntax pipelines, I'm aware that you can conditionally skip a stage with a when clause. E.g.:
run_one = true
run_two = false
run_three = true
pipeline {
agent any
stages {
stage('one') {
when {
expression { run_one }
}
steps {
echo 'one'
}
}
stage('two') {
when {
expression { run_two }
}
steps {
echo 'two'
}
}
stage('three') {
when {
expression { run_three }
}
steps {
echo 'three'
}
}
}
}
...in the above code block, there are three stages, one, two, and three, each of whose execution is conditional on a boolean variable.
I.e. the paradigm is that there is a fixed superset of known stages, of which individual stages may be conditionally skipped.
Does Jenkins pipeline script support a model where there is no fixed superset of known stages, and stages can be "looked up" for conditional execution?
To phrase it as pseudocode, is something along the lines of the following possible:
my_list = list populated _somehow_, maybe reading a file, maybe Jenkins build params, etc.
pipeline {
agent any
stages {
if (stage(my_list[0]) exists) {
run(stage(my_list[0]))
}
if (stage(my_list[1]) exists) {
run(stage(my_list[1]))
}
if (stage(my_list[2]) exists) {
run(stage(my_list[2]))
}
}
}
?
I think another way to think about what I'm asking is: is there a way to dynamically build a pipeline from some dynamic assembly of stages?
For dynamic stages you could write either a fully scripted pipeline or use a declarative pipeline with a scripted section (e. g. by using the script {…} step or calling your own function). For an overview see Declarative versus Scripted Pipeline syntax and Pipeline syntax overview.
Declarative pipeline is better supported by Blue Ocean so I personally would use that as a starting point. Disadvantage might be that you need to have a fixed root stage, but I usually name that "start" or "init" so it doesn't look too awkward.
In scripted sections you can call stage as a function, so it can be used completely dynamic.
pipeline {
agent any
stages {
stage('start') {
steps {
createDynamicStages()
}
}
}
}
void createDynamicStages() {
// Stage list could be read from a file or whatever
def stageList = ['foo', 'bar']
for( stageName in stageList ) {
stage( stageName ) {
echo "Hello from stage $stageName"
}
}
}
This shows in Blue Ocean like this:
Reading the properties file for the node label and triggerConfigURL, node label works, but I couldn't read and set triggerConfigURL from environment.
def propFile = "hello/world.txt" //This is present in workspace, and it works.
pipeline {
environment {
nodeProp = readProperties file: "${propFile}"
nodeLabel = "$nodeProp.NODE_LABEL"
dtcPath = "$nodeProp.DTC"
}
agent { label env.nodeLabel } // this works!! sets NODE_LABEL value from the properties file.
triggers {
gerrit dynamicTriggerConfiguration: 'true',
triggerConfigURL: env.dtcPath, // THIS DON'T WORK, tried "${env.dtcPath}" and few other notations too.
serverName: 'my-gerrit-server',
triggerOnEvents: [commentAddedContains('^fooBar$')]
}
stages {
stage('Print Env') {
steps {
script {
sh 'env' // This prints "dtcPath=https://path/of/the/dtc/file", so the dtcPath env is set.
}
}
}
After running the job, the configuration is as below:
Of the env and triggers clauses Jenkins runs one before the other, and it looks like you have experimentally proven that triggers run first and env second. It also looks like agent runs after env as well.
While I don't know why the programmers have made this specific decision, I think you are in a kind of a chicken-and-egg problem, where you want to define the pipeline using a file but can only read the file once the pipeline is defined and running.
Having said that, the following might work:
def propFile = "hello/world.txt"
def nodeProp = null
node {
nodeProp = readProperties file: propFile
}
pipeline {
environment {
nodeLabel = nodeProp.NODE_LABEL
dtcPath = nodeProp.DTC
}
agent { label env.nodeLabel }
triggers {
gerrit dynamicTriggerConfiguration: 'true',
triggerConfigURL: nodeProp.DTC,
//etc.
My Jenkins pipeline is specified in a declarative way. However, in the place I'd like to introduce a parallel step in, I'm forced down into plain Groovy methods:
def call(Map parameters) {
runFirstStep(parameters)
// add second step that runs in parallel to runFirstStep
}
Unexperienced as I am with Groovy and Jenkins, I hoped to make something simple like this work:
def call(Map parameters) {
parallel(
'First step': {
runFirstStep(parameters)
},
'Second step': {
runSecondStep(parameters)
}
)
}
However, this seems to mix up declarative and imperative code in an unsuited way and leads to cryptic NullPointerExceptions.
Is there another, feasible solution for this? Or is overriding the declarative part that surrounds this the only option to squeeze in another parallel step?
def call(Map parameters) {
def steps = [:]
steps['First step'] = {
node('master') {
runFirstStep(parameters)
}
}
steps['Second step'] = {
node('master') {
runSecondStep(parameters)
}
}
parallel steps.plus([failFast: true])
}
I'm trying to get a declarative pipeline that looks like this:
pipeline {
environment {
ENV1 = 'default'
ENV2 = 'default also'
}
}
The catch is, I'd like to be able to override the values of ENV1 or ENV2 based on an arbitrary condition. My current need is just to base it off the branch but I could imagine more complicated conditions.
Is there any sane way to implement this? I've seen some examples online that do something like:
stages {
stage('Set environment') {
steps {
script {
ENV1 = 'new1'
}
}
}
}
But I believe this isn't setting the actually environment variable, so much as it is setting a local variable which is overriding later calls to ENV1. The problem is, I need these environment variables read by a nodejs script, and those need to be real machine environment variables.
Is there any way to set environment variables to be dynamic in a jenkinsfile?
Maybe you can try Groovy's ternary-operator:
pipeline {
agent any
environment {
ENV_NAME = "${env.BRANCH_NAME == "develop" ? "staging" : "production"}"
}
}
or extract the conditional to a function:
pipeline {
agent any
environment {
ENV_NAME = getEnvName(env.BRANCH_NAME)
}
}
// ...
def getEnvName(branchName) {
if("int".equals(branchName)) {
return "int";
} else if ("production".equals(branchName)) {
return "prod";
} else {
return "dev";
}
}
But, actually, you can do whatever you want using the Groovy syntax (features that are supported by Jenkins at least)
So the most flexible option would be to play with regex and branch names...So you can fully support Git Flow if that's the way you do it at VCS level.
use withEnv to set environment variables dynamically for use in a certain part of your pipeline (when running your node script, for example). like this (replace the contents of an sh step with your node script):
pipeline {
agent { label 'docker' }
environment {
ENV1 = 'default'
}
stages {
stage('Set environment') {
steps {
sh "echo $ENV1" // prints default
// override with hardcoded value
withEnv(['ENV1=newvalue']) {
sh "echo $ENV1" // prints newvalue
}
// override with variable
script {
def newEnv1 = 'new1'
withEnv(['ENV1=' + newEnv1]) {
sh "echo $ENV1" // prints new1
}
}
}
}
}
}
Here is the correct syntax to conditionally set a variable in the environment section.
environment {
MASTER_DEPLOY_ENV = "TEST" // Likely set as a pipeline parameter
RELEASE_DEPLOY_ENV = "PROD" // Likely set as a pipeline parameter
DEPLOY_ENV = "${env.BRANCH_NAME == 'master' ? env.MASTER_DEPLOY_ENV : env.RELEASE_DEPLOY_ENV}"
CONFIG_ENV = "${env.BRANCH_NAME == 'master' ? 'MASTER' : 'RELEASE'}"
}
I managed to get this working by explicitly calling shell in the environment section, like so:
UPDATE_SITE_REMOTE_SUFFIX = sh(returnStdout: true, script: "if [ \"$GIT_BRANCH\" == \"develop\" ]; then echo \"\"; else echo \"-$GIT_BRANCH\"; fi").trim()
however I know that my Jenkins is on nix, so it's probably not that portable
Here is a way to set the environment variables with high flexibility, using maps:
stage("Environment_0") {
steps {
script {
def MY_MAP = [ME: "ASSAFP", YOU: "YOUR_NAME", HE: "HIS_NAME"]
env.var3 = "HE"
env.my_env1 = env.null_var ? "not taken" : MY_MAP."${env.var3}"
echo("env.my_env1: ${env.my_env1}")
}
}
}
This way gives a wide variety of options, and if it is not enough, map-of-maps can be used to enlarge the span even more.
Of course, the switching can be done by using input parameters, so the environment variables will be set according to the input parameters value.
pipeline {
agent none
environment {
ENV1 = 'default'
ENV2 = 'default'
}
stages {
stage('Preparation') {
steps {
script {
ENV1 = 'foo' // or variable
ENV2 = 'bar' // or variable
}
echo ENV1
echo ENV2
}
}
stage('Build') {
steps {
sh "echo ${ENV1} and ${ENV2}"
}
}
// more stages...
}
}
This method is more simple and looks better. Overridden environment variables will be applied to all other stages also.
I tried to do it in a different way, but unfortunately it does not entirely work:
pipeline {
agent any
environment {
TARGET = "${changeRequest() ? CHANGE_TARGET:BRANCH_NAME}"
}
stages {
stage('setup') {
steps {
echo "target=${TARGET}"
echo "${BRANCH_NAME}"
}
}
}
}
Strangely enough this works for my pull request builds (changeRequest() returning true and TARGET becoming my target branch name) but it does not work for my CI builds (in which case the branch name is e.g. release/201808 but the resulting TARGET evaluating to null)
I'd like to use a function to build some of the stages of my Jenkinsfile. This is going to be a build with a number of repetitive stages/steps - I'd not like to generate everything manually.
I was wondering if it's possible to do something like this:
_make_stage() {
stage("xx") {
step("A") {
echo "A"
}
step("B") {
echo "B"
}
}
}
_make_stages() {
stages {
_make_stage()
}
}
// pipeline starts here!
pipeline {
agent any
_make_stages()
}
Unfortunately Jenkins doesn't like this - when I run I get the error:
WorkflowScript: 24: Undefined section "_make_stages" # line 24, column 5.
_make_stages()
^
WorkflowScript: 22: Missing required section "stages" # line 22, column 1.
pipeline {
^
So what's going wrong here? The function _make_stages() really looks like it returns whatever the stages object returns. Why does it matter whether I put that in a function call or just inline it into the pipeline definition?
As explained here, Pipeline "scripts" are not simple Groovy scripts, they are heavily transformed before running, some parts on master, some parts on slaves, with their state (variable values) serialized and passed to the next step. As such, every Groovy feature is not supported, and what you see as simple functions really is not.
It does not mean what you want to achieve is impossible. You can create stages programmatically, but apparently not with the declarative syntax. See also this question for good suggestions.
You can define a declarative pipeline in a shared library, for example:
// vars/evenOrOdd.groovy
def call(int buildNumber) {
if (buildNumber % 2 == 0) {
pipeline {
agent any
stages {
stage('Even Stage') {
steps {
echo "The build number is even"
}
}
}
}
} else {
pipeline {
agent any
stages {
stage('Odd Stage') {
steps {
echo "The build number is odd"
}
}
}
}
}
}
// Jenkinsfile
#Library('my-shared-library') _
evenOrOdd(currentBuild.getNumber())
See Defining Declarative Pipelines