In Jenkins about to define an agent is possible use either:
#1
agent {
label 'something'
}
#2
agent {
node {
label 'something'
}
}
Both work, but when is mandatory use an approach over the other?
Related
I am currently facing a problem, I have about 90 jenkinsfile, we recently updated one of the Jenkins agent and it has a new label now, which means that we have to go and update every jenkinsfile with the new label of that agent, you agree that this is a bit of a pain, especially since we will have to do it every time we update the agent.I was thinking if we can define all of the agents is a single file (variable=value) than we reference the variable in our jenkinsfile, so next time we upgrade the agent we do the changes in that particular file instead of 90 jenkinsfile
Yes, you can do this. I'm assuming you have the agent details in the same SCM repo you have the Pipelines. In this case, you can do something like the below.
pipeline {
agent {label getAgentFromFile()}
stages {
stage('Hello6') {
steps {
script {
echo "Hello Something"
}
}
}
}
}
def getAgentFromFile(){
def agent = "default"
node {
agent = new File( pwd() + '/agent.txt').text.trim()
println agent
}
return agent
}
I have this example pipeline(part of it):
pipeline {
parameters {
choice(name: 'myParam', choices: ['1','2'], description: 'param desc')
}
options { timestamps () }
agent {
node {
label 'myLabel'
}
}
my question is this:
I would like to change myLabel for my agent label based on users choice of myParam.
in this case i want that when myParam is 1 then myLabel would be equal to "linux_first" and when myParam is 2 then myLabel would be "linux_second".
Does anyone know a way to do it?
Thanks in advance,
Alon
You can parameterize the agent this way in the declarative pipeline
properties([
string(description: 'build Agent', name: 'build_agent')
])
pipeline {
agent {
label params['build_agent']
}
stages {
// steps
}
}
Approach 1-
You can try "nodeLabelParameter" plugin. It might help you based on your requirement.
https://plugins.jenkins.io/nodelabelparameter/
This plugin adds two new parameter types to job configuration - node and label. The new parameters allow dynamic selection of the node or label where a job should be executed.
Approach 2-
Also Basic syntax would be to define parameter for pipeline and consume it.
agent {
label "${build_agent}"
}
In my pipeline, I have a stage that checks to see if a specific computer (node) is offline. If it is, I want to skip the next stage. However, the next stage is set to use the offline agent, so it doesn't seem to be able to check the When clause.
Here's a simplified version of my pipeline:
pipeline {
agent none
environment {
CONTINUERUN = true
}
stages {
stage('Check Should Run') {
agent any
steps {
script {
CONTINUERUN = false
}
}
}
stage('Skip this stage') {
agent {
label 'offlineAgent'
}
when {
expression {
CONTINUERUN
}
}
steps {
//Do stuff here
}
}
}
}
When I run this, the build just hangs at the 'Skip this stage' stage. I'm assuming, because the agent is offline. How can I skip this stage, when the agent is known to be offline?
In order to evaluate expression before allocating agent, you need to add beforeAgent directive to the when block.
Relevant part of documentation:
Evaluating when before entering agent in a stage
By default, the when condition for a stage will be evaluated after entering the agent for that stage, if one is defined. However, this can be changed by specifying the beforeAgent option within the when block. If beforeAgent is set to true, the when condition will be evaluated first, and the agent will only be entered if the when condition evaluates to true.
What is modern best practice for multi-configuration builds (with Jenkins)?
I want to support multiple branches and multiple configurations.
For example for each version V1, V2 of the software I want builds targeting
platforms P1 and P2.
We have managed to set up multi-branch declarative pipelines. Each build has its own docker so its easy to support multiple platforms.
pipeline {
agent none
stages {
stage('Build, test and deploy for P1) {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P1.Dockerfile'
}
}
steps {
sh buildit...
}
}
stage('Build, test and deploy for P2) {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P2.Dockerfile'
}
}
steps {
sh buildit...
}
}
}
}
This gives one job covering multiple platforms but there is no separate red/blue status for each platform.
There is good argument that this does not matter as you should not release unless the build works for all platforms.
However, I would like a separate status indicator for each configuration. This suggests I should use a multi-configuration build which triggers a parameterised build for each configuration as below (and the linked question):
pipeline {
parameters {
choice(name: 'Platform',choices: ['P1', 'P2'], description: 'Target OS platform', )
}
agent {
filename someMagicToGetDockerfilePathFromPlatform()
}
stages {
stage('Build, test and deploy for P1) {
steps {
sh buildit...
}
}
}
}
There are several problems with this:
A declarative pipeline has more constraints over how it is scripted
Multi-configuration builds cannot trigger declarative pipelines (even with the parameterized triggers plugin I get "project is not buildable").
This also begs the question what use are parameters in declarative pipelines?
Is there a strategy that gives the best of both worlds i.e:
pipeline as code
separate status indicators
limited repetition?
This is a partial answer. I think others with better experience will be able to improve on it.
This is currently untested. I may be barking up the wrong tree.
Please comment or add a better answer.
Do not use pipeline parameters except where you need user input
Use a hybrid of a scripted and declarative pipeline
(see also https://stackoverflow.com/a/46675227/1569204)
Have a function which declares a pipeline based on parameters:
(see also https://jenkins.io/doc/book/pipeline/shared-libraries/)
Use nodes to create visible indicators in the pipeline (at least in blue ocean)
So something like the following:
def build(string platform) {
switch(platform) {
case P1:
dockerFile = 'foo'
indicator = 'build for foo'
break
case P2:
dockerFile = 'bar'
indicator = 'build for bar'
break
}
pipeline {
agent {
dockerfile {
filename "$dockerFile"
}
node {
label "$indicator"
}
}
stages {
steps {
echo "build it"
}
}
}
}
The relevant code could be moved to a shared library (even if you don't actually need to share it).
I think the cleanest approach is to have this all in a pipeline similar to the first one you presented, the only modification I would see here is making those parallel, so you would actually try and build/test for both platforms.
To reuse the previous stage's workspace you could do: reuseNode true
Something similar to this flow, that would have parallel build for platforms
pipeline {
agent 'docker'
stages {
stage('Common pre') { ... }
stage('Build all platforms') {
parallel {
stage('Build, test and deploy for P1') {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P1.Dockerfile'
reuseNode true
}
}
steps {
sh buildit...
}
}
stage('Build, test and deploy for P2') {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P2.Dockerfile'
reuseNode true
}
}
steps {
sh buildit...
}
}
}
}
stage('Common post parallel') { ... }
}
}
I'm using declarative Jenkins pipelines to run some of my build pipelines and was wondering if it is possible to define multiple agent labels.
I have a number of build agents hooked up to my Jenkins and would like for this specific pipeline to be able to be built by various agents that have different labels (but not by ALL agents).
To be more concrete, let's say I have 2 agents with a label 'small', 4 with label 'medium' and 6 with label 'large'. Now I have a pipeline that is very resource-low and I want it to be executed on only a 'small'- or 'medium'-sized agent, but not on a large one as it may cause larger builds to wait in the queue for an unnecessarily long time.
All the examples I've seen so far only use one single label.
I tried something like this:
agent { label 'small, medium' }
But it failed.
I'm using version 2.5 of the Jenkins Pipeline Plugin.
You can see the 'Pipeline-syntax' help within your Jenkins installation and see the sample step "node" reference.
You can use exprA||exprB:
node('small||medium') {
// some block
}
EDIT: I misunderstood the question. This answer is only if you know
which specific agent you want to run for each stage.
If you need multiple agents you can declare agent none and then declare the agent at each stage.
https://jenkins.io/doc/book/pipeline/jenkinsfile/#using-multiple-agents
From the docs:
pipeline {
agent none
stages {
stage('Build') {
agent any
steps {
checkout scm
sh 'make'
stash includes: '**/target/*.jar', name: 'app'
}
}
stage('Test on Linux') {
agent {
label 'linux'
}
steps {
unstash 'app'
sh 'make check'
}
post {
always {
junit '**/target/*.xml'
}
}
}
stage('Test on Windows') {
agent {
label 'windows'
}
steps {
unstash 'app'
bat 'make check'
}
post {
always {
junit '**/target/*.xml'
}
}
}
}
}
This syntax appears to work for me:
agent { label 'linux && java' }
As described in Jenkins pipeline documentation and by Vadim Kotov one can use operators in label definition.
So in your case if you want to run your jobs on nodes with specific labels, the declarative way goes like this:
agent { label('small || medium') }
And here are some examples from Jenkins page using different operators
// with AND operator
agent { label('windows && jdk9 )') }
// a more complex one
agent { label('postgres && !vm && (linux || freebsd)') }
Notes
When constructing those definitions one just needs to consider following rules/restrictions:
All operators are left-associative
Labels or agent names can be surrounded with quotation marks if they contain characters that would conflict with the operator syntax
Expressions can be written without whitespace
Jenkins will ignore whitespace when evaluating expressions
Matching labels or agent names with wildcards or regular expressions is not supported
An empty expression will always evaluate to true, matching all agents
Create a another label call 'small-or-medium' that has 6 all agents. Then in Jenkinsfile:
agent { label 'small-or-medium' }