Reading the properties file for the node label and triggerConfigURL, node label works, but I couldn't read and set triggerConfigURL from environment.
def propFile = "hello/world.txt" //This is present in workspace, and it works.
pipeline {
environment {
nodeProp = readProperties file: "${propFile}"
nodeLabel = "$nodeProp.NODE_LABEL"
dtcPath = "$nodeProp.DTC"
}
agent { label env.nodeLabel } // this works!! sets NODE_LABEL value from the properties file.
triggers {
gerrit dynamicTriggerConfiguration: 'true',
triggerConfigURL: env.dtcPath, // THIS DON'T WORK, tried "${env.dtcPath}" and few other notations too.
serverName: 'my-gerrit-server',
triggerOnEvents: [commentAddedContains('^fooBar$')]
}
stages {
stage('Print Env') {
steps {
script {
sh 'env' // This prints "dtcPath=https://path/of/the/dtc/file", so the dtcPath env is set.
}
}
}
After running the job, the configuration is as below:
Of the env and triggers clauses Jenkins runs one before the other, and it looks like you have experimentally proven that triggers run first and env second. It also looks like agent runs after env as well.
While I don't know why the programmers have made this specific decision, I think you are in a kind of a chicken-and-egg problem, where you want to define the pipeline using a file but can only read the file once the pipeline is defined and running.
Having said that, the following might work:
def propFile = "hello/world.txt"
def nodeProp = null
node {
nodeProp = readProperties file: propFile
}
pipeline {
environment {
nodeLabel = nodeProp.NODE_LABEL
dtcPath = nodeProp.DTC
}
agent { label env.nodeLabel }
triggers {
gerrit dynamicTriggerConfiguration: 'true',
triggerConfigURL: nodeProp.DTC,
//etc.
Related
In this Jenkinsfile I am trying to execute the same stage for different branches in Multibranch Pipeline. I need to configure the environment variable of each branch-name every time. Is there any better way to do this?
stage('Create New AMI for master branch') {
when { branch 'master' }
environment {
BRANCH_NAME = "${env.BRANCH_NAME}"
ENV_NAME = "prod"
}
steps {
sh "packer build jenkins/${PROJECT_NAME}/${PROJECT_NAME}-ami.json"
}
}
stage('Create New AMI for development branch') {
when { branch 'development' }
environment {
BRANCH_NAME = "${env.BRANCH_NAME}"
ENV_NAME = "dev"
}
steps {
sh "packer build jenkins/${PROJECT_NAME}/${PROJECT_NAME}-ami.json"
}
}
stage('Create New AMI for staging branch') {
when { branch 'staging' }
environment {
BRANCH_NAME = "${env.BRANCH_NAME}"
ENV_NAME = "staging"
}
steps {
sh "packer build jenkins/${PROJECT_NAME}/${PROJECT_NAME}-ami.json"
}
}
Please use shared library in this case which will contain all your stage implementations.Reference Example implemenation as shown below.
Create a file named : sharedlibraries (groovy file)
#!groovy
// Write or add Functions(definations of stages) which will be called from your jenkins file
def Create_AMI(PROJECT_NAME, ENV_NAME)
{
echo ENV_NAME
sh "packer build jenkins/${PROJECT_NAME}/${PROJECT_NAME}-ami.json"
// You can also set the environment below example:
env.ENV_NAME ="dev"
}
return this
In your Jenkinsfile write below code:
// Global variable is used to get data from groovy file(shared library)
def stagelibrary
stage('Create New AMI') {
steps {
script {
// Load Shared library Groovy file stagelibraries.Give your path of stagelibraries file whic is created
stagelibrary = load 'C:\\Jenkins\\stagelibraries'
// Execute Create_AMI function. You can add if else conditions over here, based on your branches. But am not sure of your conditions.
// You can also pass your environment variable
// in the Crete_AMI function using env.<YOURENVVARIABLE>
stagelibrary.Create_AMI(PROJECT_NAME,env.ENV_NAME)
}
}
}
The above example was shown to provide overview of shared library and you dont need to write same functions / or redundant stages.
I'm trying to dynamically set environment variables in the jenkins pipeline script.
I'm using a combination of .groovy and .jenkinsfile scripts to generate the stage{} definitions for a pipeline as DRY as possible.
I have a method below:
def generateStage(nameOfTestSet, pathToTestSet, machineLabel, envVarName, envVarValue)
{
echo "Generating stage for ${nameOfTestSet} on ${machineLabel}"
return node("${machineLabel}") {
stage(nameOfTestSet)
{
/////// Area of interest ////////////
environment {
"${envVarName} = ${envVarValue}"
}
/////////////////////////////////////
try {
echo "Would run: "+pathToTestSet
} finally {
echo "Archive results here"
}
}
}
}
There's some wrapper code running this, but abstracting away we'd have the caller essentially use:
generateStage("SimpleTestSuite", "path.to.test", "MachineA", "SOME_ENV_VAR", "ENV_VALUE")
Where the last two parameters are the environment name (SOME_ENV_VAR) and the value (ENV_VALUE)
The equivalent declarative code would be:
stage("SimpleTestSuite")
{
agent {
label "MachineA"
}
environment = {
SOME_ENV_VAR = ENV_VALUE
}
steps {
echo "Would run" + "path.to.test"
}
post {
always {
echo "Archive results"
}
}
}
However, when running this script, the environment syntax in first code block doesn't seem to affect the actual execution at all. If I echo the ${SOME_ENV_VAR} (or even echo ${envVarName} in case it took this variable name as the actual environment value) they both return null.
I'm wondering what's the best way to make this environment{} section as DRY / dynamic as possible?
I would prefer it if there's an extendable solution that can take in a list of environmentName=Value pairs, as this would be more general case.
Note: I have tried the withEnv[] solution for scripted pipelines, however this seems to have the same issue.
I figured out the solution to this.
It is to use the withEnv([]) step.
def generateStage(nameOfTestSet, pathToTestSet, machineLabel, listOfEnvVarDeclarations=[])
{
echo "Generating stage for ${nameOfTestSet} on ${machineLabel}"
return node("${machineLabel}") {
stage(nameOfTestSet)
{
withEnv(listOfEnvVarDeclarations) {
try {
echo "Would run: "+pathToTestSet
} finally {
echo "Archive results here"
}
}
}
}
}
And the caller method would be:
generateStage("SimpleTestSuite", "path.to.test", "MachineA", ["SOME_ENV_VAR=\"ENV_VALUE\""])
Since the withEnv([]) step can take in multiple environment variables, we can also do:
generateStage("SimpleTestSuite", "path.to.test", "MachineA", ["SOME_ENV_VAR=\"ENV_VALUE\"", "SECOND_VAR=\"SECOND_VAL\""])
And this would be valid and should work.
I'm trying to use an env var in Jenkins' pipeline options {} block, but it seems they aren't extrapolated at that stage. Am I missing something or is it intentional and there's no way to make it work?
Example:
pipeline {
agent {
docker {
image '...'
label 'docker'
}
}
environment {
MAGIC_APP_NAME = "xxx"
MAGIG_APP_ID = "yyy"
}
options {
connection: gitLabConnection("GitLab-${env.MAGIC_APP_ID}")
}
}
I am using a declarative pipeline in a Jenkinsfile but I would like to derive some variables from a parameter.
For example given:
parameters {
choice(name: 'Platform',choices: ['Debian9', 'CentOS7'], description: 'Target OS platform', )
}
I would like to add a block like:
script {
switch(param.Platform) {
case "Centos7":
def DockerFile = 'src/main/docker/Jenkins-Centos.Dockerfile'
def PackageType = 'RPM'
def PackageSuffix = '.rpm'
break
case "Debian9":
default:
def DockerFile = 'src/main/docker/Jenkins-Debian.Dockerfile'
def PackageType = 'DEB'
def PackageSuffix = '.deb'
break
}
}
Such that I can use variables elsewhere in the pipeline. For example:
agent {
dockerfile {
filename "$DockerFile"
}
}
etc..
but script is illegal in the parameter, environment & agent sections.
It can only be used in steps.
I need to use the parameter in the agent block and I want to avoid repeating myself where the variables are used in different steps.
Is there a sane way to achieve this? My preferences in order are:
a declarative pipeline
a scripted pipeline (less good)
via a plugin to the Jenkins UI (least good)
A shared library might be appropriate here regardless of whether it is actually shared.
The intention is to support a multi-configuration project by creating a parameterised build and invoking it for different parameter sets with a red/blue status light for each configuration.
It could be that I have assumed an 'old fashioned' design. In which case an acceptable answer would explain the modern best practice for creating a multi-configuration multi-branch pipeline. Something like: https://support.cloudbees.com/hc/en-us/articles/115000088431-Create-a-Matrix-like-flow-with-Pipeline or Jenkins Pipeline Multiconfiguration Project
See also Multiconfiguration / matrix build pipeline in Jenkins for less specific discussion of best practices.
Never really used the Jenkins declarative pipeline before but I think the way you refer to params is incorrect?
I think it might be: ${params.Platform} or params.Platform instead of param.
So something like the below maybe?
pipeline {
agent any
stages {
stage('example') {
steps {
script {
switch(${params.Platform}) {
...
}
}
}
}
}
}
As I said, never really used it before so not 100%. I was just looking at the syntax used for parameters on the docs: https://jenkins.io/doc/book/pipeline/syntax/#parameters
I think that the key for solving your issue is the declaration of your variables. Do not use def if you want your variable to be accessible from other stages.
Here is an example of a solution for your issue :
pipeline{
agent none
parameters {
choice(name: 'Platform',choices: ['Debian9', 'CentOS7'], description: 'Target OS platform', )
}
stages{
stage('Setting stage'){
agent any
steps {
script {
switch(params.Platform){
case 'CentOS7' :
DockerFile = 'src/main/docker/Jenkins-Centos.Dockerfile'
PackageType = 'RPM'
PackageSuffix = '.rpm'
break
case 'Debian9' :
DockerFile = 'src/main/docker/Jenkins-Debian.Dockerfile'
PackageType = 'DEB'
PackageSuffix = '.deb'
break
}
}
}
}
stage('Echo stage'){
agent {
dockerfile {
filename "$DockerFile"
}
}
steps{
echo PackageType
echo PackageSuffix
}
}
}
}
What is definitely possible on windows:
stage('var from groovy') {
steps {
script {
anvar = "foo"
}
bat "${anyvar}"
}
}
This is an example that I have in production
def dest_app_instance = "${params.Destination}"
switch(dest_app_instance) {
case "CASE1":
dest_server = "server1"
break
case "CASE2":
dest_server = "server2"
break
}
I'm trying to get a declarative pipeline that looks like this:
pipeline {
environment {
ENV1 = 'default'
ENV2 = 'default also'
}
}
The catch is, I'd like to be able to override the values of ENV1 or ENV2 based on an arbitrary condition. My current need is just to base it off the branch but I could imagine more complicated conditions.
Is there any sane way to implement this? I've seen some examples online that do something like:
stages {
stage('Set environment') {
steps {
script {
ENV1 = 'new1'
}
}
}
}
But I believe this isn't setting the actually environment variable, so much as it is setting a local variable which is overriding later calls to ENV1. The problem is, I need these environment variables read by a nodejs script, and those need to be real machine environment variables.
Is there any way to set environment variables to be dynamic in a jenkinsfile?
Maybe you can try Groovy's ternary-operator:
pipeline {
agent any
environment {
ENV_NAME = "${env.BRANCH_NAME == "develop" ? "staging" : "production"}"
}
}
or extract the conditional to a function:
pipeline {
agent any
environment {
ENV_NAME = getEnvName(env.BRANCH_NAME)
}
}
// ...
def getEnvName(branchName) {
if("int".equals(branchName)) {
return "int";
} else if ("production".equals(branchName)) {
return "prod";
} else {
return "dev";
}
}
But, actually, you can do whatever you want using the Groovy syntax (features that are supported by Jenkins at least)
So the most flexible option would be to play with regex and branch names...So you can fully support Git Flow if that's the way you do it at VCS level.
use withEnv to set environment variables dynamically for use in a certain part of your pipeline (when running your node script, for example). like this (replace the contents of an sh step with your node script):
pipeline {
agent { label 'docker' }
environment {
ENV1 = 'default'
}
stages {
stage('Set environment') {
steps {
sh "echo $ENV1" // prints default
// override with hardcoded value
withEnv(['ENV1=newvalue']) {
sh "echo $ENV1" // prints newvalue
}
// override with variable
script {
def newEnv1 = 'new1'
withEnv(['ENV1=' + newEnv1]) {
sh "echo $ENV1" // prints new1
}
}
}
}
}
}
Here is the correct syntax to conditionally set a variable in the environment section.
environment {
MASTER_DEPLOY_ENV = "TEST" // Likely set as a pipeline parameter
RELEASE_DEPLOY_ENV = "PROD" // Likely set as a pipeline parameter
DEPLOY_ENV = "${env.BRANCH_NAME == 'master' ? env.MASTER_DEPLOY_ENV : env.RELEASE_DEPLOY_ENV}"
CONFIG_ENV = "${env.BRANCH_NAME == 'master' ? 'MASTER' : 'RELEASE'}"
}
I managed to get this working by explicitly calling shell in the environment section, like so:
UPDATE_SITE_REMOTE_SUFFIX = sh(returnStdout: true, script: "if [ \"$GIT_BRANCH\" == \"develop\" ]; then echo \"\"; else echo \"-$GIT_BRANCH\"; fi").trim()
however I know that my Jenkins is on nix, so it's probably not that portable
Here is a way to set the environment variables with high flexibility, using maps:
stage("Environment_0") {
steps {
script {
def MY_MAP = [ME: "ASSAFP", YOU: "YOUR_NAME", HE: "HIS_NAME"]
env.var3 = "HE"
env.my_env1 = env.null_var ? "not taken" : MY_MAP."${env.var3}"
echo("env.my_env1: ${env.my_env1}")
}
}
}
This way gives a wide variety of options, and if it is not enough, map-of-maps can be used to enlarge the span even more.
Of course, the switching can be done by using input parameters, so the environment variables will be set according to the input parameters value.
pipeline {
agent none
environment {
ENV1 = 'default'
ENV2 = 'default'
}
stages {
stage('Preparation') {
steps {
script {
ENV1 = 'foo' // or variable
ENV2 = 'bar' // or variable
}
echo ENV1
echo ENV2
}
}
stage('Build') {
steps {
sh "echo ${ENV1} and ${ENV2}"
}
}
// more stages...
}
}
This method is more simple and looks better. Overridden environment variables will be applied to all other stages also.
I tried to do it in a different way, but unfortunately it does not entirely work:
pipeline {
agent any
environment {
TARGET = "${changeRequest() ? CHANGE_TARGET:BRANCH_NAME}"
}
stages {
stage('setup') {
steps {
echo "target=${TARGET}"
echo "${BRANCH_NAME}"
}
}
}
}
Strangely enough this works for my pull request builds (changeRequest() returning true and TARGET becoming my target branch name) but it does not work for my CI builds (in which case the branch name is e.g. release/201808 but the resulting TARGET evaluating to null)