I want to be able select whether a pipeline stage is going to be executed with the dockerfile agent depending on the presence of a Dockerfile in the repository. If there's no Dockerfile, the stage should be run locally.
I tried something like
pipeline {
stage('AwesomeStage') {
when {
beforeAgent true
expression { return fileExists("Dockerfile") }
}
agent { dockerfile }
steps {
// long list of awesome steps that should be run either on Docker either locally, depending on the presence of a Dockerfile
}
}
}
But the result is that the whole stage is skipped when there's no Dockerfile.
Is it possible to do something like the following block?
//...
if (fileExists("Dockerfile")) {
agent {dockerfile}
}
else {
agent none
}
//...
I came up with this solution that relies on defining a function to avoid repetion and defines two different stages according to type of agent.
If anyone has a more elegant solution, please let me know.
def awesomeScript() {
// long list of awesome steps that should be run either on Docker either locally, depending on the presence of a Dockerfile
}
pipeline {
stage('AwesomeStageDockerfile') {
when {
beforeAgent true
expression { return fileExists("Dockerfile") }
}
agent { dockerfile }
steps {
awesomeScript()
}
}
stage('AwesomeStageLocal') {
when {
beforeAgent true
expression { return !fileExists("Dockerfile") }
}
agent none
steps {
awesomeScript()
}
}
}
Related
I have multiple jenkinsifles, doing basically the same thing:
pipeline{
parameters { ... }
environment { ... }
stages {
stage ('setup') { ... }
stage ('run') {
agent { AGENT }
steps { STEPS }
}
}
The STEPS & AGENT parameters are values I get in the setup stage. Is it possible to define somewhere a function that returns a stage?
e.g.
def stage_factory(name, agent, steps, post ...){
return
stage (name) {
agent { agnet }
steps { steps }
post { post }
}
}
}
which later will be called inside the pipeline, right after the setup stage
?
The following works in scripted pipeline, you need to try the declarative syntax yourself. Note the use of surrounding {}
def stage_factory(name, agent, steps, post ...){
return {
node(agent){
stage (name) {
steps()
}
}
}
}
With this approach you need to put the post action in try-catch blocks, but this is the gist of it.
If you change it like so, you can even pass the steps to it as you would expect from a Jenkins stage.
def stage_factory(name, agent){
return { steps ->
node(agent){
stage (name) {
steps()
}
}
}
}
usage:
def myDtage = stage_factory("foo", "bar")
myStage{
//...
}
Say I have a simple Jenkins pipeline file as below:
pipeline {
agent any
stages {
stage('Test') {
steps {
sh ...
}
}
stage('Build') {
steps {
sh ...
}
}
stage('Publish') {
when {
buildingTag()
}
steps {
sh ...
send_slack_message("Built tag")
}
}
}
post {
failure {
send_slack_message("Error building tag")
}
}
}
Since there's a lot non-tag builds everyday, I don't want to send any slack message about non-tag builds. But for the tag builds, I want to send either a success message or a failure message, despite of which stage it failed.
So for the above example, I want:
When it's a tag build, and stage 'Test' failed, I shall see a "Error building tag" message. (This is a yes in the example)
When it's a tag build, and all stages succeeded, I shall see a "Built tag" message. (This is also a yes in the example)
When it's not a tag build, no slack message will ever been sent. (This is not the case in the example, for example, when the 'Test' stage fails, there's will be a "Error building tag" message)
As far as I know, there's no such thing as "conditional post section" in Jenkins pipeline syntax, which could really help me out here. So my question is, is there any other way I can do this?
post {
failure {
script {
if (isTagBuild) {
send_slack_message("Error building tag")
}
}
}
}
where isTagBuild is whatever way you have to differentiate between a tag or no tag build.
You could also apply the same logic, and move send_slack_message("Built tag") down to a success post stage.
In the postbuild step you can also use script step inside and use if. And inside this if step you can add emailext plugin.
Well, for those who just want some copy-pastable code, here's what I ended-up with based on #eez0's answer.
pipeline {
agent any
environment {
BUILDING_TAG = 'no'
}
stages {
stage('Setup') {
when {
buildingTag()
}
steps {
script {
BUILDING_TAG = 'yes'
}
}
}
stage('Test') {
steps {
sh ...
}
}
stage('Build') {
steps {
sh ...
}
}
stage('Publish') {
when {
buildingTag()
}
steps {
sh ...
}
}
}
post {
failure {
script {
if (BUILDING_TAG == 'yes') {
slackSend(color: '#dc3545', message: "Error publishing")
}
}
}
success {
script {
if (BUILDING_TAG == 'yes') {
slackSend(color: '#28a745', message: "Published")
}
}
}
}
}
As you can see, I'm really relying on Jenkins built-in buidingTag() function to help me sort things out, by using an env-var as a "bridge". I'm really not good at Jenkins pipeline, so please leave comments if you have any suggestions.
I'm trying to set up a jenkins multibranch pipeline to run all my code validation steps in a docker container, then build the docker image and push it outside of said docker container.
Currently, my Jenkinsfile looks sort of like this (slimmed down for readability):
pipeline {
agent {
label 'AWS'
}
stages {
stage('stuff in docker') {
agent {
dockerfile {
filename 'Dockerfile.jenkins'
reuseNode true
}
}
steps {
stuff
}
}
stage('more stuff in docker') {
agent {
dockerfile {
filename 'Dockerfile.jenkins'
reuseNode true
}
}
steps {
stuff
}
}
stage('stuff not in docker') {
steps {
stuff
}
}
stage('more stuff not in docker') {
steps {
stuff
}
}
}
post {
always {
cleanWs()
}
}
}
The problem here is that every stage that I use a dockerfile agent, jenkins will attempt to rebuild the docker image. The stages are all cached, but sending build context and actually processing everything still takes more time than I'd like. If I use the dockerfile as the root agent, I get to run everything inside the same docker container, but I lose the ability to do git stuff and build the docker image (at least without connecting to the external docker process sock, which seems like more hassle than should be necessary).
I'd like to know if there's some way I can use the same docker image for multiple steps, but then pull out of that docker image for some other steps.
Figured it out!
pipeline {
agent {
label 'AWS'
}
stages {
stage('do everything in docker') {
agent {
dockerfile {
filename 'Dockerfile.jenkins'
reuseNode true
}
}
stages {
stage('stuff in docker') {
steps {
stuff
}
stage('more stuff in docker') {
steps {
stuff
}
}
}
}
stage('stuff not in docker') {
steps {
stuff
}
}
stage('more stuff not in docker') {
steps {
stuff
}
}
}
post {
always {
cleanWs()
}
}
}
tl;dr you can nest stages
Using dockerfile as build agent will always result in a container rebuild. If you just want to run some steps inside a (pre-build) container use docker as agent. Of course, this can also be a container build locally inside the same pipeline. You can also utilize bind mounts to share data between containers.
If unsure, check out the documentation 1 and 2. Maybe it's also advisable to set the global agent to none in your case.
I try to configure different pipelines in jenkins 2. My Problem ist that all my pipelines need the same workspace path (configugerd with customWorkspace in my configuration script).
Now I have to prevent that more than one pipeline is running.
My search always leads me back to the same pages, which unfortunately do not help me :-(
Has anyone already solved the same problem and can give me a hint?
Thank you very much
def locked = false;
pipeline {
agent any
stages {
stage('check workspace lock status') {
steps {
script {
locked = fileExists file: '.lock'
if(locked == false) {
touch file: '.lock'
}
}
}
}
stage('build') {
when {
beforeAgent true
expression { locked == false }
}
steps {
// do something you want
}
}
}
post {
always {
sh 'rm -f .lock'
}
}
}
I'm trying to run a jenkins file with multiple agents in it, but I'm running into errors. Here's a snippet of my jenkins file:
pipeline {
agent {
docker {
label 'agentAAA'
...
}
node {
label 'agentBBB'
...
}
}
...
stages {
stage('to run on AAA') {
agent {
label 'agentAAA'
}
...
}
stage('to run on BBB') {
agent {
label 'agentBBB'
}
...
}
stage('to run on BBB') {
agent {
label 'agentBBB'
}
...
}
I'm getting these errors:
Only one agent type is allowed per agent section
No agent type specified. Must be one of [any, docker, dockerfile, label, none]
I can't find any examples in the documentation of how to refer to a previously declared agent. I see how to declare the agent in each individual stage, but I'd end up with many repeated declarations in my file.
You need to specify agent as none for overall pipeline, then you can specify agent for each stage explicitly as shown in below example. Populate the details as and what required.
pipeline {
agent none
stages {
stage ('Stage-1') {
agent { label 'agent-1' }
steps {
script {
}
}
}
stage ('Stage-2') {
agent { label 'agent-2' }
steps {
script {
}
}
}
}
}
Refer link for further details - https://jenkins.io/doc/book/pipeline/jenkinsfile/#using-multiple-agents