I would like to set the variable that will be available to all stages. Variable that depends on chosen parameter, something like this:
parameters {
choice(name: 'Environment', choices: ['Dev', 'Stage'], description: 'Deploy to chosen environment')
}
environment {
//set the config file which depends on params.Environment e.g.
//case params.Environment of
// Dev -> CONFIG_FILE="deploy/file_1.conf"
// Stage -> CONFIG_FILE="deploy/other_file.conf"
}
stages {
stage('check-params') {
steps {
sh "echo \"config file: ${CONFIG_FILE}\""
}
}
stage('build-frontend') {
steps {
sh "build-fronted.sh ${CONFIG_FILE}"
}
}
stage('deploy-backend') {
steps {
sh "deploy-backend.sh ${CONFIG_FILE}"
}
}
but according to the Pipeline Syntax it is not allowed (I get ERROR: Expected name=value pairs).
Does anyone know how can I achieve this without using scripts { ... } in every stage->step as described in this post?
You can have a init stage which will set correct variable depending on parameter, something like this:
parameters {
choice(name: 'Environment', choices: ['Dev', 'Stage'], description: 'Deploy to chosen environment')
}
environment {
//set the config file which depends on params.Environment e.g.
//case params.Environment of
// Dev -> CONFIG_FILE="deploy/file_1.conf"
// Stage -> CONFIG_FILE="deploy/other_file.conf"
}
stages {
// ----------------------------
stage('init-env-variables') {
steps {
script {
switch(params.Environment) {
case "Dev":
env.setProperty('CONFIG_FILE', 'deploy/file_1.conf')
break;
case "Stage":
env.setProperty('CONFIG_FILE', 'deploy/other_file.conf')
break;
}
}
}
}
// -----------------------
stage('check-params') {
steps {
sh "echo \"config file: ${CONFIG_FILE}\""
}
}
stage('build-frontend') {
steps {
sh "build-fronted.sh ${CONFIG_FILE}"
}
}
stage('deploy-backend') {
steps {
sh "deploy-backend.sh ${CONFIG_FILE}"
}
}
Related
I wanna build one Jenkins pipeline that builds and runs tests on multiple versions of a program (e.g. Different databases)
But when any step fails, I want to skip the following steps only for that "branch" so to speak..
This is my example Code where Stage 1 is run first, with possible parallel steps (1.a, 1.b). The code does not work and is only some sort of example of how I would like it to work:
pipeline {
agent any
environment {
stageOneFailed = "false"
stageTwoFailed = "false"
}
stages {
stage ("Stage 1") {
parallel {
stage("Stage 1.a") {
// Something like this maybe?
steps {
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
// Do stuff here..
}
}
post {
unsuccessful {
// When stage did not succeed..
// Set stageOneFailed = "true"
}
}
}
stage("Stage 1.b") {
// Do Stuff..
// If Stage 1.b fails, set stageTwoFailed="true"
}
}
}
stage("Stage 2") {
parallel {
// Only run stages if earlier steps didn't fail
stage("Stage 2.a") {
when {
environment(name: "stageOneFailed", value: "false")
}
steps {
// Do stuff..
// If Stage 2.a fails, set stageOneFailed="true"
}
}
stage("Stage 2.b") {
when {
environment(name: "stageTwoFailed", value: "false")
}
steps {
// Do stuff..
// If Stage 2.b fails, set stageTwoFailed="true"
}
}
}
}
// stage()
}
}
Can anyone give any advice on how to do this the proper way?
Thanks in advance
EDIT: Changed code example. The example runs now!
pipeline {
agent any
environment {
stageOneFailed = "false"
stageTwoFailed = "false"
}
stages {
stage ("Stage 1") {
parallel {
stage("Stage 1.a") {
steps {
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
bat "ech Stage 1.a" // Should fail because ech is no valid command
}
}
post {
failure {
script {
env.stageOneFailed = "true"
}
}
}
}
stage("Stage 1.b") {
steps {
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
bat "echo Stage 1.b" // Should not fail
}
}
post {
failure {
script {
env.stageTwoFailed = "true"
}
}
}
}
}
}
stage("Stage 2") {
parallel {
// Only run stages if earlier steps didn't fail
stage("Stage 2.a") {
when {
environment(name: "stageOneFailed", value: "false")
}
steps {
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
bat "echo Stage 2.a"
}
}
post {
failure {
script {
env.stageOneFailed = "true"
}
}
}
}
stage("Stage 2.b") {
when {
environment(name: "stageTwoFailed", value: "false")
}
steps {
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
bat "echo Stage 2.b"
}
}
post {
failure {
script {
env.stageTwoFailed = "true"
}
}
}
}
}
}
}
}
But when running the example, the Stage 1.a fails but Stage 2.a is still run, maybe anyone could help out here..
EDIT: I added output to see, what value stageNFailed is set to. Even after calling env.stageOneFailed, when going into next stage, it takes the old value false..
My assumption is, that when calling script env.stageNFailed = "true", the value is only set temporarily for that stage..
The example you have used is a perfectly acceptable way to do it. You have introduced 2 env variables that get used to determine if the previous step failed. You have used catchError to ensure that the pipeline doesn't fail when the stage fails. You have to use catchError in every stage to prevent the pipeline from failing (but I guess you already know that). In the post part of the stage, you have set the appropriate env variable to true, which is also correct.
post {
failure {
script {
env.stageOneFailed = true
}
}
}
Then when the next relevant stage starts, you have used the when condition to check if the stage should be run (you could also do something like this):
when {
expression { stageOneFailed == false }
}
So basically you have done everything right.
Is it possible to specify agent 'any' only when a condition is met?
Context: I have a parallel pipeline with some common steps. The pipeline runs tests on Linux and Windows on two different agents. There are parameter defining if both, or only Linux, or only Windows should run. I want to use as little agents as possible.
With the following pipeline, the Linux tests are executed on the main agent. The Windows tests on a different agent. So if only Linux needs to run, I only use 1 agent, which is good. But if only Windows needs to run, the main agent will be blocked until the Windows agent finishes.
pipeline {
agent any // main agent
parameters {
booleanParam( name: 'RUN_WINDOWS' )
booleanParam( name: 'RUN_LINUX' )
}
stages {
stage('Common step') {
...
}
stage("Test") {
parallel {
stage('Windows') {
agent any
when {
expression { return params.RUN_WINDOWS }
}
stages {
stage('Test Windows') {
...
}
}
}
stage('Linux') { // no agent specified, so will use the main agent
when {
expression { return params.RUN_LINUX }
}
stages {
stage('Test Linux') {
...
}
}
}
}
post {
...
}
}
}
}
I'd need something like this for the Windows stage, where it only specifies an agent based on a condition:
if (params.RUN_WINDOWS && params.LINUX) { agent any }
EDIT: I'm looking for a solution without using labels.
As a workaround, instead of setting any you can set agent labels conditionally. Something like below.
def labels = "Windows"
if (params.RUN_WINDOWS && params.LINUX) {
labels = "windows || linux"
}
pipeline {
agent any // main agent
parameters {
booleanParam( name: 'RUN_WINDOWS' )
booleanParam( name: 'RUN_LINUX' )
}
stages {
stage('Common step') {
...
}
stage("Test") {
parallel {
stage('Windows') {
agent {
label "${labelSelected}"
}
when {
expression { return params.RUN_WINDOWS }
}
stages {
stage('Test Windows') {
...
}
}
}
stage('Linux') { // no agent specified, so will use the main agent
when {
expression { return params.RUN_LINUX }
}
stages {
stage('Test Linux') {
...
}
}
}
}
post {
...
}
}
}
}
I want to execute all the stages in parallel with the loop based on user input.
This gives error because script is not allowed under stages.
How should I achieve the same?
pipeline {
agent {
node {
label 'ec2'
}
}
stages{
script{
int[] array = params.elements;
for(int i in array) {
parallel{
stage('Preparation') {
echo 'Preparation'
println(i);
}
stage('Build') {
echo 'Build'
println(i);
}
}
}
}
}
}
If you are using declarative pipelines you have two options, first is to use static parallel stages which is an integral part of the declarative syntax but does not allow dynamic or runtime modifications.
The second option (which is probably what you attempted) is to use the scripted parallel function:
parallel firstBranch: {
// do something
}, secondBranch: {
// do something else
},
failFast: true|false```
When using it inside a declarative pipeline it should be used inside a script block like you did but the declarative basic directive must still be kept: pipeline -> stages -> stage -> steps -> script. In addition the scripted parallel function receives a specifically formatted map alike the example above.
In your case it can look somethong like:
pipeline {
agent {
node {
label 'ec2'
}
}
stages {
stage('Parallel Execution') {
steps {
script {
parallel params.elements.collectEntries {
// the key of each entry is the parallel execution branch name
// and the value of each entry is the code to execute
["Iteration for ${it}" : {
stage('Preparation') {
echo 'Preparation'
println(it);
}
stage('Build') {
echo 'Build'
println(it);
}
}]
}
}
}
}
}
}
Or if you want to use the for loop:
pipeline {
agent {
node {
label 'ec2'
}
}
stages {
stage('Parallel Execution') {
steps {
script {
map executions = [:]
for(int i in params.elements) {
executions["Iteration for ${it}" ] = {
stage('Preparation') {
echo 'Preparation'
println(i);
}
stage('Build') {
echo 'Build'
println(i);
}
}]
}
parallel executions
}
}
}
}
}
Other useful examples for the parallel function can be found here
I'm using Jenkins declarative pipeline and I want to make a conditional step depending on an environment variable, which is set according the existence of a file.
So I just want to make something like that : if Dockerfile exist, perform next stage, else don't.
To perform this I tried :
pipeline {
// ...
stage {
stage('Docker') {
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
when {
environment name: 'IS_DOCKERFILE', value: true
}
stage('Build') {
// ...
}
}
}
}
Or :
pipeline {
// ...
stage {
stage('Docker') {
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
when {
expression {
env.IS_DOCKERFILE == true
}
}
stage('Build') {
// ...
}
}
}
}
In both cases, the Dockerfile exist and it is in the workspace. I also tried with strings ("true") but everytime, the pipeline continue without executing the stage 'Build'.
Any suggestions ?
This is because the exprsssion:
IS_DOCKERFILE = fileExists 'Dockerfile'
Creates the environment variable with boolean value as string:
$ set
IS_DOCKERFILE='false'
So the solution would be to use .toBoolean() like this:
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
stages {
stage("build docker image") {
when {
expression {
env.IS_DOCKERFILE.toBoolean()
}
}
steps {
echo 'fileExists'
}
}
stage("build libraries") {
when {
expression {
!env.IS_DOCKERFILE.toBoolean()
}
}
steps {
echo 'fileNotExists'
}
}
}
As #Sergey already posted, the problem is that you're comparing a string to a boolean. See fileExists: Verify if file exists in workspace.
Besides his answer, you can compare directly to a string:
environment {
IS_DOCKERFILE = fileExists 'Dockerfile'
}
stages {
stage("build docker image") {
when {
expression {IS_DOCKERFILE == 'true'}
}
steps {
echo 'fileExists'
}
}
stage("build libraries") {
when {
expression {IS_DOCKERFILE == 'false'}
}
steps {
echo 'fileNotExists'
}
}
}
I would like to configure an environment variable for my Jenkins pipeline, but dynamically based on an input parameter to the build. I'm trying to configure my pipeline to set the KUBECONFIG environment variable for kubectl commands.
My pipeline is as follows (slightly changed):
pipeline {
parameters {
choice(name: 'CLUSTER_NAME', choices: 'cluster1/cluster2')
}
stages {
// Parallel stages since only one environment variable should be set, based on input
stage ('Set environment variable') {
parallel {
stage ('Set cluster1') {
when {
expression {
params.CLUSTER_NAME == "cluster1"
}
}
environment {
KUBECONFIG = "~/kubeconf/cluster1.conf"
}
steps {
echo "Using KUBECONFIG: ${env.KUBECONFIG}"
}
}
stage ('Set cluster2') {
when {
expression {
params.CLUSTER_NAME == "cluster2"
}
}
environment {
KUBECONFIG = "~/kubeconf/cluster2.conf"
}
steps {
echo "Using KUBECONFIG: ${env.KUBECONFIG}"
}
}
}
}
stage ('Test env') {
steps {
sh "cat ${env.KUBECONFIG}"
}
}
}
}
However, while the stage where I set the environment variable can print it, once I move to another stage I only get null.
Is there some way of sharing env variables between stages? Since I'd like to use the default KUBECONFIG command (and not specify a file/context in my kubectl commands), it would be much easier to find a way to dynamically set the env variable.
I've seen the EnvInject plugin mentioned, but was unable to get it working for a pipeline, and was struggling with the documentation.
I guess that with the environment{} you are setting the environment variable only for the stage where it runs - it is not affecting the context of environment of the pipeline itself. Set environment variables like below to affect the main context. Works for me.
pipeline {
agent any
parameters {
choice(name: 'CLUSTER_NAME', choices: 'cluster1\ncluster2')
}
stages {
// Parallel stages since only one environment variable should be set, based on input
stage ('Set environment variable') {
parallel {
stage ('Set cluster1') {
when {
expression {
params.CLUSTER_NAME == "cluster1"
}
}
steps {
script{
env.KUBECONFIG = "~/kubeconf/cluster1.conf"
echo "Using KUBECONFIG: ${env.KUBECONFIG}"
}
}
}
stage ('Set cluster2') {
when {
expression {
params.CLUSTER_NAME == "cluster2"
}
}
steps {
script{
env.KUBECONFIG = "~/kubeconf/cluster2.conf"
echo "Using KUBECONFIG: ${env.KUBECONFIG}"
}
}
}
}
}
stage ('Test env') {
steps {
sh "cat ${env.KUBECONFIG}"
}
}
}
}