For example these:
when { changeRequest() }
when { buildingTag() }
when { changeset "**/*.js" }
I don't want this information just in when conditions I want to access it in code in script{} directives and even before the pipeline directive potentially.
For example I want to do things like this:
IS_A_PR = changeRequest()
IS_A_TAG = buildingTag()
pipeline {
stages {
stage('Checkout') {
script{
sh "someCommand ${IS_A_TAG}"
I don't want to only capture this in when conditions that turn stages on or off.
Related
From my experience with Jenkins declarative-syntax pipelines, I'm aware that you can conditionally skip a stage with a when clause. E.g.:
run_one = true
run_two = false
run_three = true
pipeline {
agent any
stages {
stage('one') {
when {
expression { run_one }
}
steps {
echo 'one'
}
}
stage('two') {
when {
expression { run_two }
}
steps {
echo 'two'
}
}
stage('three') {
when {
expression { run_three }
}
steps {
echo 'three'
}
}
}
}
...in the above code block, there are three stages, one, two, and three, each of whose execution is conditional on a boolean variable.
I.e. the paradigm is that there is a fixed superset of known stages, of which individual stages may be conditionally skipped.
Does Jenkins pipeline script support a model where there is no fixed superset of known stages, and stages can be "looked up" for conditional execution?
To phrase it as pseudocode, is something along the lines of the following possible:
my_list = list populated _somehow_, maybe reading a file, maybe Jenkins build params, etc.
pipeline {
agent any
stages {
if (stage(my_list[0]) exists) {
run(stage(my_list[0]))
}
if (stage(my_list[1]) exists) {
run(stage(my_list[1]))
}
if (stage(my_list[2]) exists) {
run(stage(my_list[2]))
}
}
}
?
I think another way to think about what I'm asking is: is there a way to dynamically build a pipeline from some dynamic assembly of stages?
For dynamic stages you could write either a fully scripted pipeline or use a declarative pipeline with a scripted section (e. g. by using the script {…} step or calling your own function). For an overview see Declarative versus Scripted Pipeline syntax and Pipeline syntax overview.
Declarative pipeline is better supported by Blue Ocean so I personally would use that as a starting point. Disadvantage might be that you need to have a fixed root stage, but I usually name that "start" or "init" so it doesn't look too awkward.
In scripted sections you can call stage as a function, so it can be used completely dynamic.
pipeline {
agent any
stages {
stage('start') {
steps {
createDynamicStages()
}
}
}
}
void createDynamicStages() {
// Stage list could be read from a file or whatever
def stageList = ['foo', 'bar']
for( stageName in stageList ) {
stage( stageName ) {
echo "Hello from stage $stageName"
}
}
}
This shows in Blue Ocean like this:
Is there a way to execute step outside of the Jenkins agent?
Suppose that I have following structure of Jenkinsfile:
pipeline {
agent none
stages {
stage('Example Stage') {
agent { someAgent }
steps {
run something ...
input ...
}
}
}
}
I would like to execute input outside of an agent to not block it for hours (timeout is not the answer ;))
One of the possible solutions is to execute the logic in separate stages but i'm trying to avoid creating additional ones.
You could use node instead of agent:
pipeline {
agent none
stages {
stage('Example Stage') {
steps {
node( someAgent ) {
run something ...
}
// outside of any agent
input ...
}
}
}
}
I'm using Declarative Pipelines 1.3.2 plugin and I want to use the same agent (as in only specifying the agent directive once) in multiple parallel stages:
stage('Parallel Deployment')
{
agent { dockerfile { label 'docker'; filename 'Dockerfile'; } }
parallel
{
stage('A') { steps { ... } }
stage('B') { steps { ... } }
}
}
However, Jenkins complains:
"agent" is not allowed in stage "Parallel Deployment" as it contains parallel stages
A solution is to duplicate the agent directive for each parallel stage, but this is tedious and leads to lot of duplicated code with many parallel stages:
stage('Parallel Deployment')
{
parallel
{
stage('A') {
agent { dockerfile { label 'docker'; filename 'Dockerfile'; } }
steps { ... }
}
stage('B') {
agent { dockerfile { label 'docker'; filename 'Dockerfile'; } }
steps { ... }
}
}
}
Is there a more idiomatic solution, or is duplicating agent directive necessary for each of the parallel stages?
Specifying the agent at pipeline level can be a solution, but has the potential downside that the agent is up & running for the whole duration of the build.
Also note that this means each stage (that doesn't define its own agent) is run on the same agent instance, not agent type. If the parallel processes are CPU / resource intensive, this might not be what you want.
Still, if you want to run parallel stages on one instance, and can't or want not to define the agent at pipeline level, here's a workaround for the declarative syntax:
stage('Parallel Deployment') {
agent { dockerfile { label 'docker'; filename 'Dockerfile'; } }
stages {
stage('A & B') {
parallel {
stage('A') { steps { ... } }
stage('B') { steps { ... } }
}
}
}
}
Or you go for a scripted pipeline, which doesn't have this limitation.
Declare the agent at Pipeline level so all stages run on the same agent.
I have a list of long running Gradle tasks on different sub projects in my project. I would like to run these in parallel using Jenkins declarative pipeline.
I was hoping something like this might work:
projects = [":a", ":b", ":c"]
pipeline {
stage("Deploy"){
parallel {
for(project in projects){
stage(project ) {
when {
expression {
someConditionalFunction(project)
}
}
steps {
sh "./gradlew ${project}:someLongrunningGradleTask"
}
}
}
}
}
}
Needless to say that gives a compile error since it was expecting stage instead of for. Any ideas on how to overcome this? Thanks
I was trying to reduce duplicated code in my existing Jenkinsfile using declarative pipeline syntax. Finally I was able to wrap my head around the difference between scripted and declarative syntax.
It is possible to use scripted pipeline syntax in a declarative pipeline by wrapping it with a script {} block.
Check out my example below: you will see that all three parallel stages finish at the same time after waking up from the sleep command.
def jobs = ["JobA", "JobB", "JobC"]
def parallelStagesMap = jobs.collectEntries {
["${it}" : generateStage(it)]
}
def generateStage(job) {
return {
stage("stage: ${job}") {
echo "This is ${job}."
sh script: "sleep 15"
}
}
}
pipeline {
agent any
stages {
stage('non-parallel stage') {
steps {
echo 'This stage will be executed first.'
}
}
stage('parallel stage') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
Parallel wants a map structure. You are doing this a little inside-out. Build your map and then just pass it to parallel, rather than trying to iterate inside parallel.
Option 2 on this page shows you a way to do something similar to what you are trying.
At this link you can find a complex way I did this similar to a matrix/multi-config job:
I'd like to use a function to build some of the stages of my Jenkinsfile. This is going to be a build with a number of repetitive stages/steps - I'd not like to generate everything manually.
I was wondering if it's possible to do something like this:
_make_stage() {
stage("xx") {
step("A") {
echo "A"
}
step("B") {
echo "B"
}
}
}
_make_stages() {
stages {
_make_stage()
}
}
// pipeline starts here!
pipeline {
agent any
_make_stages()
}
Unfortunately Jenkins doesn't like this - when I run I get the error:
WorkflowScript: 24: Undefined section "_make_stages" # line 24, column 5.
_make_stages()
^
WorkflowScript: 22: Missing required section "stages" # line 22, column 1.
pipeline {
^
So what's going wrong here? The function _make_stages() really looks like it returns whatever the stages object returns. Why does it matter whether I put that in a function call or just inline it into the pipeline definition?
As explained here, Pipeline "scripts" are not simple Groovy scripts, they are heavily transformed before running, some parts on master, some parts on slaves, with their state (variable values) serialized and passed to the next step. As such, every Groovy feature is not supported, and what you see as simple functions really is not.
It does not mean what you want to achieve is impossible. You can create stages programmatically, but apparently not with the declarative syntax. See also this question for good suggestions.
You can define a declarative pipeline in a shared library, for example:
// vars/evenOrOdd.groovy
def call(int buildNumber) {
if (buildNumber % 2 == 0) {
pipeline {
agent any
stages {
stage('Even Stage') {
steps {
echo "The build number is even"
}
}
}
}
} else {
pipeline {
agent any
stages {
stage('Odd Stage') {
steps {
echo "The build number is odd"
}
}
}
}
}
}
// Jenkinsfile
#Library('my-shared-library') _
evenOrOdd(currentBuild.getNumber())
See Defining Declarative Pipelines