In Jenkins, when someone selects "Build with Parameters" options, how can I:
call an API that returns an Array
use that Array values list dynamically as checkboxes
get selected values in the job execution
You can do something like this and This is just generalised overview of you problem statement
arrayName = functionWhichReturnsArray()
pipeline {
agent any
parameters {
booleanParam(name: 'NAME_OF_CHECKBOX_YOU_WANT', defaultValue: arrayName, description: 'Tick the box to exceute the checkbox')
}
stages {
stage('Test') {
}
}
}
def functionWhichReturnsArray() {
add your api logic and return what you want to return
}
I'm struggling to run the following Jenkins pipeline.
I added a UI parameter (TARGET_FILTER).
The idea is:
if I select tst, it excludes qua, prd; it executes only 1 value (tst1)
if I select qua, it excludes tst, prd; it executes 2 values (qua1, qua2)
if I select prd, it excludes tst, qua; it executes 2 values (prd1, prd2)
I can't make it work, the entire matrix is executed and my excludes are ignored.
pipeline {
parameters {
choice(name: 'TARGET_FILTER', choices: ['tst', 'qua', 'prd'], description: 'Run on specific target')
}
agent any
stages {
stage('go') {
matrix {
when { anyOf {
expression { params.TARGET_FILTER == 'tst' }
} }
axes {
axis {
name 'TARGET'
values 'tst', 'qua', 'prd'
}
axis {
name 'REMOTE'
values 'tst1', 'qua1', 'qua2', 'prd1', 'prd2'
}
}
excludes {
exclude {
axis {
name 'TARGET'
values 'tst'
}
axis {
name 'REMOTE'
values 'qua1', 'qua2', 'prd1', 'prd2'
}
}
exclude {
axis {
name 'TARGET'
values 'qua'
}
axis {
name 'REMOTE'
values 'tst1', 'prd1', 'prd2'
}
}
exclude {
axis {
name 'TARGET'
values 'prd'
}
axis {
name 'REMOTE'
values 'tst1', 'qua1', 'qua2'
}
}
}
stages {
stage('Build') {
steps {
echo "${params.TARGET}"
echo "Do Build for ${TARGET} - ${REMOTE}"
}
}
}
}
}
}
}
I'm converting our enterprise build so that it uses the 'matrix' directive (https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/), but am having trouble optimizing it. In the pre-matrix world, we have a build step (produces jars), followed by a parallel 'package' step (produces linux and windows distros), essentially followed by a 'system test' phase (execute the windows and linux distros on various JVMs), and we do some 'code quality' inspection in parallel to the package-and-system-test phases.
Much of this seems to lend itself to using 'matrix'. So, the 'packaging' is clearly a matrix to get windows and linux platforms crafted. And the 'system test' is another dual-axis matrix of platform and jvm. I can easily make one matrix follow the other. So far, so good.
However, I'm stuck with the 'code quality' stuff as an outlier. Is there a way to run those stages parallel to the matrix runs. They are independent once the build is created (they do not need to wait for packaging). They are also time consuming, so running them serially with the two matrix phases makes the build longer. Jenkins complains if you put a matrix inside a parallel stage.
Any ideas on how to run non-matrix stages in parallel to a matrix?
I came up with a somewhat IMHO "better" solution:
stage {
matrix {
axes {
axis {
name 'ORIGAXIS'
values 'ALPHA','BETA','BAR','BAZ'
}
}
stages {
stage ("alpha") {
when { expression { env.ORIGAXIS == "ALPHA" } }
steps {
alpha()
}
stage ("beta") {
when { expression { env.ORIGAXIS == "BETA" } }
steps {
beta()
}
}
stage ("Tests") {
when { allOf
expression { env.ORIGAXIS != "ALPHA" }
expression { env.ORIGAXIS != "BETA" }
}
stages {
stage("First") {
originalFirst( ...)
}
stage("Second") {
originalSecond(...)
}
}
}
}
}
Of course, the final layout is not perfect. But, it works, is not that much cumbersome and still easily maintainable.
I have a work-around to my own question.
According to the documentation (https://www.jenkins.io/doc/book/pipeline/syntax/#declarative-matrix) what I am looking for is not possible. "Each cell in a matrix can include one or more stages to be run sequentially using the configuration for that cell. Note that a stage must have one and only one of steps, stages, parallel, or matrix. It is not possible to nest a parallel or matrix block within a stage directive if that stage directive is nested within a parallel or matrix block itself."
But.... you can cheat. I was able to turn the matrix into a generic dispatch queue. i.e. each combination invokes N stages - for me, 2 and I call them "prepare" and "execute". I pass an additional "matrixtype" argument in. The matrixtype gets 1 value for the matrix itself, as well as additional values for each line of the non-matrix. I then use the matrix 'excludes' to insure that the non-matrix lines execute only once. Effectively, this folds the non matrix into the matrix differentiated by matrix type.
Original (parallel-stages in series with a subsequent matrix)
stage {
parallel {
stage("alpha") {
alpha(..)
}
stage("beta") {
beta(..)
}
// etc
}
}
stage {
matrix {
axes {
axis {
name 'ORIGAXIS'
values 'FOO','BAR','BAZ'
}
}
stages {
stage("First") {
originalFirst( ...)
}
stage("Second") {
originalSecond(...)
}
}
}
}
Replacement (parallel folded into matrix)
stage {
matrix {
axes {
axis {
name 'MATRIXTYPE
values 'ORIGINAL', 'ALPHA', 'BETA'
axis {
name 'ORIGAXIS'
values 'FOO','BAR','BAZ'
}
excludes {
// Execute Alpha and Beta only once (during 'FOO')
exclude {
axis {
name 'MATRIXTYPE'
values 'ALPHA', 'BETA'
}
axis {
name 'ORIGAXIS'
values 'BAR','BAZ'
}
}
}
}
stages {
stage("First") {
dispatchFirst( "${MATRIXTYPE}", ...)
}
stage("Second") {
dispatchSecond( "${MATRIXTYPE}", ...)
}
}
}
}
The dispatchFirst(..) and dispatchSecond(..) are then simple dispatch methods in the shared lib that examine matrixtype, and invoke originalFirst(..), originalSecond(..), alpha(..), beta(..) or a no-op as appropriate. It's slightly clumsy, and amounts to shoehorning the parallel stages into the matrix, but it works. And, you get the benefit of parallelization (build speed-optimization)
Hopefully in the future, there will be something more elegant.
Updated (2022-09-22): What the 'replacement' does is brute-force the heck out of it. It creates a matrix with 9 permutations:
ORIGINAL-FOO, ORIGINAL-BAR, ORIGINAL-BAZ, ALPHA-FOO, ALPHA-BAR, ALPHA-BAZ, BETA-FOO, BETA-BAR, BETA-BAZ. The exclude then gets rid of ALPHA-BAR, ALPHA-BAZ, BETA-BAR, BETA-BAZ, leaving 5 executable elements (emphazied above) in the matrix. For each of the 5 elements, the matrix will call the "dispatchFirst" then "dispatchSecond" method, passing the matrix variables over to it.
You then implement dispatchFirst such that:
if matrixType == ALPHA, invoke alpha() [only gets called once]
if matrixType == BETA, invoke beta() [only gets called once]
otherwise, invoke originalFirst(..) using FOO/BAR/BAZ as passed in
And dispatchSecond only operates if matrixType=ORIGINAL, and it calls originalSecond(..) using FOO/BAR/BAZ as passed in.
Based on the parameter selected (Eg. choice - in this case), need to scale up/down the pipeline stages.
if(choice.equals("four")){
pipeline{
<4 stages>
}
}else{
pipeline{
<3 stages>
}
}
Is it possible to implement something like this?
You can define a super-set of all the stages and then run only the stages you need, like this:
pipeline {
agent any
stages {
stage('Always') {
steps {
...
}
}
stage('Only when Four') {
when {
environment name: 'CHOICE', value: 'four'
beforeAgent true
}
steps {
...
}
}
}
}
I have a job with multiple parameters, but one is a choice parameter and it contains 10 choices, i need to build this job with all these choices one by one.
is that possible?
You can achieve this by using Jenkins Declarative Pipelines.
Here is an example pipeline which iterates through selected multi-choice parameter:
pipeline {
agent any
parameters {
choice(name: 'CHOICE', choices: ['One', 'Two', 'Three'], description: 'Please select one/multiple options.')
}
stages {
stage('Build') {
steps {
script {
for (String selectedChoice : params.CHOICE) {
// do something
}
}
}
}
}
}