I need to be able to queue more than 2 jobs in a Jenkins pipeline.
In https://stackoverflow.com/a/24918670/8369030 it is suggested to use the Random String Parameter Plugin, how ever I can not find any documentation how to use it.
Alternatively I tried to do it with a random value like showed in https://stackoverflow.com/a/67110959/8369030, how ever this seems only to work in a Stage but not in a Parameter. Specifically, I always get null as default value when doing this:
pipeline {
environment {
max = 50
random_num = "${Math.abs(new Random().nextInt(max+1))}"
}
parameters {
string(name: 'JOB_ID', defaultValue: "${env.random_num}",
description: "Enter a random value to allow more than 2 jobs in the queue")
}
But it does not solve the problem with queues, because the parameter is overridden after execution, not before.
max = 50
random_num = "${Math.abs(new Random().nextInt(max+1))}"
println(random_num)
pipeline {
agent any
parameters {
string(name: 'rn', defaultValue: random_num,
description: "Enter a random value to allow more than 2 jobs in the queue")
}
stages {
stage('Randon number') {
steps {
println(random_num)
println(rn)
}
}
}
}
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] echo
31
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/test2
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Randon number)
[Pipeline] echo
31
[Pipeline] echo
25
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Solution:
enter image description here
Related
Currently Jenkins pipeline input step time is shown as paused for X duration in stage view.
I want the inputStep time to be included, currently I am using an ugly way to run parallel branches something like below to simulate the time but the downside is the build log still get populated with ugly blocks like below
def gotInput = false
def branches = [:]
branches['durationRep'] = {
if (stageName != 'none') {
stage("${stageName}") {
this.logger.info("Waiting for response")
waitUntil(initialRecurrencePeriod: 10000, quiet: true) {
return gotInput
}
}
}
}
branches['input'] = {
inputParams = input(
id: "fooBar",
message: "Waiting for input",
)
gotInput = true
}
timeout(time: 6, unit: 'HOURS') {
parallel branches
}
Current console log example
18:04:50 Waiting for input
18:04:50 [Pipeline] waitUntil
18:04:50 [Pipeline] {
18:04:50 [Pipeline] }
18:05:00 [Pipeline] {
18:05:00 [Pipeline] }
18:05:12 [Pipeline] {
18:05:12 [Pipeline] }
18:05:27 [Pipeline] {
18:05:27 [Pipeline] }
18:05:42 [Pipeline] {
18:05:42 [Pipeline] }
18:05:57 [Pipeline] {
18:05:57 [Pipeline] }
Does anyone know if I can manually edit the duration of a stage using flowNodes somehow or if its possible to include pauseDuration caused by input step in Jenkins pipelines?
I am New to Jenkins. Trying to create one basic Pipeline which is using choicebased parameters. Following is my script.
Code ----
pipeline{
agent {
label 'agent'
}
parameters {
choice choices: ['John', 'Stacy'], description: 'Choose one', name: 'Person'
}
stages {
stage('Print') {
steps {
echo "Hello ${params.Person}"
sh """if (${params.Person} = "John")
then
echo "Person is male."
else
echo "Person is female."
fi"""
}
}
}
}
Now my build complete successfully regardless of whatever option I choose. It always display result "Person is female.
Following is result of one of my build.
Started by user ****
[Pipeline] Start of Pipeline
[Pipeline] node
Running on agent in
/home/temp/jenkins_agent/workspace/ChoiceBased PL
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Print)
[Pipeline] echo
Hello John
[Pipeline] sh
+ John = John
/home/temp/jenkins_agent/workspace/ChoiceBased PL#tmp/durable-
b7e98c46/script.sh: 1: John: not found
+ echo Person is female.
Person is female.
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: Success
Please suggest what I am missing ?
I would change this just to be in Groovy rather than doing the compare in sh
stage('Print') {
steps {
echo "Hello ${params.Person}"
script {
if (params.Person == "John") {
echo "Person is male."
} else {
echo "Person is female."
}
}
}
}
Then when you choose Stacey you will get
[Pipeline] echo
Hello Stacy
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Person is female.
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
may I ask if it is do-able if I can get the user input and run a script based on what was chosen by the user? the logic whould be
IF User choose Proceed THEN run a script (in my case promote or Blue/Green deployment)
IF User choose Abort THEN don't terminate the jenkins job, run a script (in my case rollback)
This is the script I'm using however I have no idea how to apply after validate stage
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sh """#!/bin/bash +x
echo "performing sts assume role"
//SCRIPT-HERE//
echo "performing ansible deploy"
//SCRIPT-HERE//
"""
}
}
stage('validate') {
steps {
timeout(30) {
script {
input(message: 'Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided', ok: 'Proceed')
}
}
}
}
}
}
One more issue I'm seeing is, although this pipeline script is running, The Proceed / Abort is not clickable on the jenkins job Console output, is it a bug? I'm referring to the image shown below
I was able to add nandilov's suggestion however it seems that the logic is still not applying, please advise on what is missed here
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sh """#!/bin/bash +x
echo "performing sts assume role"
echo "performing ansible deploy"
"""
}
}
stage('validate') {
steps {
script {
env.flagError = "false"
try {
input(message: 'Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided', ok: 'Proceed')
}catch(e){
println "input aborted or timeout expired, will try to rollback."
env.flagError = "true"
}
}
}
}
stage("If user selects Proceed"){
when{
expression { env.inputValue == "value1" }
}
steps{
sh """#!/bin/bash +x
echo "User selected proceed"
"""
}
}
stage("rollback if flag error true"){
when{
expression { env.inputValue == "value2" }
}
steps{
sh """#!/bin/bash +x
echo "User selected Abort"
"""
}
}
}
}
from the pipeline view it never triggered either one of the last two stages when choosing "Abort" or "Proceed"
These are the logs when choosing Abort or Proceed
ABORT
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/test-job-lagot
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] sh
performing sts assume role
performing ansible deploy
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (validate)
[Pipeline] script
[Pipeline] {
[Pipeline] input
Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided
Proceed or Abort
[Pipeline] echo
input aborted or timeout expired, will try to rollback.
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (If user selects Proceed)
Stage "If user selects Proceed" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (rollback if flag error true)
Stage "rollback if flag error true" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
PROCEED
Started by user lagot
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/test-job-lagot
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Deploy)
[Pipeline] sh
performing sts assume role
performing ansible deploy
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (validate)
[Pipeline] script
[Pipeline] {
[Pipeline] input
Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided
Proceed or Abort
Approved by lagot
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (If user selects Proceed)
Stage "If user selects Proceed" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (rollback if flag error true)
Stage "rollback if flag error true" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
If you want to have the job aborted and do something, you can try/catch it:
try{
input(message: 'Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided', ok: 'Proceed')
}catch(e){
println "input aborted or timeout expired, will try to rollback."
// execute rollback
}
You also can do it in another stage:
pipeline {
agent any
stages {
stage('Deploy') {
steps {
sh """#!/bin/bash +x
echo "performing sts assume role"
echo "performing ansible deploy"
"""
}
}
stage('validate') {
steps {
script {
env.flagError = "false"
try {
input(message: 'Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided', ok: 'Proceed')
}catch(e){
println "input aborted or timeout expired, will try to rollback."
env.flagError = "true"
}
}
}
}
stage("If user selects Proceed"){
when{
expression { env.flagError == "false" }
}
steps{
sh """#!/bin/bash +x
echo "User selected proceed"
"""
}
}
stage("rollback if flag error true"){
when{
expression { env.flagError == "true" }
}
steps{
sh """#!/bin/bash +x
echo "User selected Abort"
"""
}
}
}
Try with this, you will need to click on Proceed and then to choose between "deploy" and "rollback". The result we'll be store in an environment variable and you can use it as a condition on the next stages
stage('validate') {
steps {
timeout(30) {
script {
CHOICES = ["deploy", "rollback"];
env.yourChoice = input message: 'Please validate, this job will automatically ABORTED after 30 minutes even if no user input provided', ok : 'Proceed',id :'choice_id',
parameters: [choice(choices: CHOICES, description: 'Do you want to deploy or to rollback?', name: 'CHOICE'),
string(defaultValue: 'rollback', description: '', name: 'rollback value')]
}
}
}
}
}
stage('Deploy') {
when {
expression { env.yourChoice == 'deploy' }
}
steps {
...
}
}
stage('Rollback') {
when {
expression { env.yourChoice == 'rollback' }
}
steps {
...
}
}
I've inherited some Jenkins pipeline and try to improve it. Jenkins and groovy is quite fresh topic for me, so most probably I'm doing something wrong.
I'm using Jenkins ver. 2.121.3
Main aim was to add build parameter to do some extra cleaning during build. So I've added parameter CLEAN_FIRST with Boolean type and default value false to a job configuration and did something like this in pipeline:
// CLEAN_FIRST = false
// def prefix = CLEAN_FIRST ? "" : "REM"
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('master') {
cleanWs()
try {
def prefix = CLEAN_FIRST ? "" : "REM"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
bat (label: 'build third party',
script: """
$prefix call cleanSomthing.bat
call doOtherStuff.bat
"""
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
Now this doesn't work as expected. "REM" prefix is not added.
Echo prints:
CLEAN_FIRST=false prefix=
And bat invokes cleanSomthing.bat which I wish to avoid (to save on build times).
I've tried to make prefix global, but with same result.
Most probably this is caused by some evaluation order or scoping issue, but I can't put finger on it.
Can someone give me a clue why it doesn't work? How to fix it?
Answered own question. Is this problem fixed on some version of Jenkins?
replace
def prefix = CLEAN_FIRST ? "" : "REM"
with
def prefix = params.CLEAN_FIRST ? "" : "REM"
Ok I've found source of problems. It is a bit funny.
When running this pipeline (tested on Mac machine since it had empty job queue):
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('Mac') {
cleanWs()
try {
def logic = true
def prefix = CLEAN_FIRST ? "Ole" : "REM"
def typeLogic = logic.getClass()
def typeParam = CLEAN_FIRST.getClass()
echo "typeLogic=$typeLogic typeParam=$typeParam"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
sh (script: """
echo prefix=$prefix
""")
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
I've got this outcome:
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] stage
[Pipeline] { (Some step)
[Pipeline] script
[Pipeline] {
[Pipeline] node
Running on master in /Users/builder/jenkins/workspace/EIbuild_MacOS
[Pipeline] {
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] echo
typeLogic=class java.lang.Boolean typeParam=class java.lang.String
[Pipeline] echo
CLEAN_FIRST=false prefix=Ole
[Pipeline] sh
[EIbuild_MacOS] Running shell script
+ echo prefix=Ole
prefix=Ole
[Pipeline] echo
some stuff
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
So now source the problem is obvious.
Jenkins in configuration promises variable of type Boolean, but in fact provides type String with values are "true" or "false" which are always evaluated as true when used as condition since both values are not empty strings :).
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
echo "whoami".execute().text
script {
File f = new File('/home/jenkins/test2.txt');
f.createNewFile();
}
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
Jenkins console log: (got exception: Started by user Edgar Yu Running
in Durability level: MAX_SURVIVABILITY [Pipeline] node Running on
Jenkins in /var/jenkins_home/workspace/test2 [Pipeline] { [Pipeline]
stage [Pipeline] { (Build) [Pipeline] echo Building.. [Pipeline] echo
jenkins
[Pipeline] script [Pipeline] { [Pipeline] } [Pipeline] // script
[Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Test)
Stage 'Test' skipped due to earlier failure(s) [Pipeline] } [Pipeline]
// stage [Pipeline] stage [Pipeline] { (Deploy) Stage 'Deploy' skipped
due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline]
} [Pipeline] // node [Pipeline] End of Pipeline
java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at
java.io.File.createNewFile(File.java:1012)
This is due to Jenkins not implementing Groovy itself but an interpreter (CPS) - https://github.com/cloudbees/groovy-cps
To help deal with the complexities introduced, there are some common Steps implemented to take the trouble out of tasks such as creating a file.
To use Jenkins pipeline steps out of the box, use writeFile:
https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#code-writefile-code-write-file-to-workspace
writeFile([file: 'file.txt', text: filetxt])
If your deadset on writing your own, I suggest splitting it out into a Shared library, note this will probably cause ScriptSecurity alerts that will require approval:
final class PipelineUtils implements Serializable {
private script=null
private static final PipelineUtils instance = new PipelineUtils()
#NonCPS
String saveFile(String filename, String text) {
String PWD = script.pwd()
String filePath = "${PWD}/${filename}"
File file = new File(filePath)
file.text = text
}
}
See https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md for information regarding #NonCPS and nonserializable objects.