I running our maven project on a jenkins server with multiple stages inside the pipeline.
Every time I decide that the branch test does not need to continue and click on abort in the jenkins ui, I need to repeat this many times until the jenkins pipeline really stops.
I guess that our jenkinsfile does not really pick up that the job was aborted and I need to abort every stage to come to the end.
Is there a way to help jenkins to get out of the pipeline?
For example a variable I can check?
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
if (!currentBuild.isAborted) {
stage('Unit Tests') {
steps {
echo 'Unit Testing'
}
}
}
if (!currentBuild.isAborted) {
stage('Deploy') {
steps {
echo 'Deploying'
}
}
}
if (!currentBuild.isAborted) {
stage('Backend Integration Tests') {
steps {
echo 'Backend Int Tests'
}
}
}
if (!currentBuild.isAborted) {
stage('Frontend Integration Tests') {
steps {
echo 'Deploying....'
}
}
}
// done
}
}
Related
I have some parallel stages in my Jenkins pipeline. They are all identical, except that they run on different agents:
stage {
parallel {
stage {
agent {
label 'agent-1'
}
steps {
sh 'do task number 468'
}
}
stage {
agent {
label 'agent-2'
}
steps {
sh 'do task number 468'
}
}
stage {
agent {
label 'agent-3'
}
steps {
sh 'do task number 468'
}
}
}
}
I want to add more parallel stages on more nodes, but the script is long and repetetive. What's the best way to rewrite this to tell jenkins to parallelize the same steps across agents 1, 2, 3, 4...etc?
Please see below code which will create and run the stage on multiple agents:
// Define your agents
def agents = ['agent-1','agent-2','agent-3']
def createStage(label) {
return {
stage("Runs on ${label}") {
node(label) {
// build steps that should happen on all nodes go here
echo "Running on ${label}"
sh 'do task number 468'
}
}
}
}
def parallelStagesMap = agents.collectEntries {
["${it}" : createStage(it)]
}
pipeline {
agent none
stages {
stage('parallel stage') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
More information is available at : Jenkins examples
We have a pipeline like this:
pipeline {
agent none
stages {
stage('Build') {
// ...
}
stage('Test') {
parallel {
stage('Test on Debian') {
agent {
label 'debian'
}
steps {
unstash 'compile-artifacts'
unstash 'dot-gradle'
sh './gradlew check --stacktrace'
}
post {
always {
junit '*/build/test-results/**/*.xml'
}
}
}
stage('Test on CentOS') {
agent {
label 'centos'
}
steps {
unstash 'compile-artifacts'
unstash 'dot-gradle'
sh './gradlew check --stacktrace'
}
post {
always {
junit '*/build/test-results/**/*.xml'
}
}
}
stage('Test on Windows') {
agent {
label 'windows'
}
steps {
unstash 'compile-artifacts'
unstash 'dot-gradle'
bat "gradlew.bat check --stacktrace"
}
post {
always {
junit '*/build/test-results/**/*.xml'
}
}
}
stage('Test on macOS') {
agent {
label 'macos'
}
steps {
unstash 'compile-artifacts'
unstash 'dot-gradle'
sh './gradlew check --stacktrace'
}
post {
always {
junit '*/build/test-results/**/*.xml'
}
}
}
}
}
}
}
Every stage is essentially identical, save for one line in the Windows block which I already know how to deal with, so is there a way to template out the common parts of these stages to remove the duplication?
I already tried putting a loop inline, but it's not something that declarative pipelines let you do. :(
You can refactor your step{}-blocks with groovy-methods:
def stageX(boolean linux) {
unstash 'compile-artifacts'
unstash 'dot-gradle'
if (linux) {
sh './gradlew check --stacktrace' }
else {
bat "gradlew.bat check --stacktrace" }
}
which you have to call like the following in your step{}:
steps {
script { stageX( true) } // or with false for your windows agent
}
Of course you can do the same for your junit-plugin-call:
def junitCall() {
junit '*/build/test-results/**/*.xml'
}
and call it like:
post {
always {
script { junitCall()
}
}
}
You won't win a lot of lines but it will improve the handling of the code a lot. If you want to cleanup your Jenkinsfile even more you could put the methods into a shared-library which you import so they aren't even declared in your Jenkinsfile.
Essentially what you want to do is currently not possible. As https://jenkins.io/doc/book/pipeline/shared-libraries/#defining-declarative-pipelines states:
Only entire pipelines can be defined in shared libraries as of this
time. This can only be done in vars/*.groovy, and only in a call
method. Only one Declarative Pipeline can be executed in a single
build, and if you attempt to execute a second one, your build will
fail as a result.
So you can define methods to bundle several steps or you can bundle a whole pipeline in a shared library but nothing in between. Which is a shame, really.
I'm using Jenkins Pipeline with the declarative syntax, currently with the following stages:
Prepare
Build (two parallel sets of steps)
Test (also two parallel sets of steps)
Ask if/where to deploy
Deploy
For steps 1, 2, 3, and 5 I need and agent (an executor) because they do actual work on the workspace. For step 4, I don't need one, and I would like to not block my available executors while waiting for user input. This seem to be referred to as either a "flyweight" or "lightweight" executor for the classic, scripted syntax, but I cannot find any information on how to achieve this with the declarative syntax.
So far I've tried:
Setting an agent directly in the pipeline options, and then setting agent none on the stage. This has no effect, and the pipeline runs as normalt, blocking the executor while waiting for input. It is also mentioned in the documentation that it will have no effect, but I thought I'd give it a shot anyway.
Setting agent none in the pipeline options, and then setting an agent for each stage except #4. Unfortunately, but expectedly, this allocates a new workspace for every stage, which in turn requires me to stash and unstash. This is both messy and gives me further problems in the parallel stages (2 and 3) because I cannot have code outside the parallel construct. I assume the parallel steps run in the same workspace, so stashing/unstashing in both would have unfortunate results.
Here is an outline of my Jenkinsfile:
pipeline {
agent {
label 'build-slave'
}
stages {
stage("Prepare build") {
steps {
// ...
}
}
stage("Build") {
steps {
parallel(
frontend: {
// ...
},
backend: {
// ...
}
)
}
}
stage("Test") {
steps {
parallel(
jslint: {
// ...
},
phpcs: {
// ...
},
)
}
post {
// ...
}
}
stage("Select deploy target") {
steps {
script {
// ... code that determines choiceParameterDefinition based on branch name ...
try {
timeout(time: 5, unit: 'MINUTES') {
deployEnvironment = input message: 'Deploy target', parameters: [choiceParameterDefinition]
}
} catch(ex) {
deployEnvironment = null
}
}
}
}
stage("Deploy") {
when {
expression {
return binding.variables.get("deployEnvironment")
}
}
steps {
// ...
}
}
}
post {
// ...
}
}
Am I missing something here, or is it just not possible in the current version?
Setting agent none at the top level, then agent { label 'foo' } on every stage, with agent none again on the input stage seems to work as expected for me.
i.e. Every stage that does some work runs on the same agent, while the input stage does not consume an executor on any agent.
pipeline {
agent none
stages {
stage("Prepare build") {
agent { label 'some-agent' }
steps {
echo "prepare: ${pwd()}"
}
}
stage("Build") {
agent { label 'some-agent' }
steps {
parallel(
frontend: {
echo "frontend: ${pwd()}"
},
backend: {
echo "backend: ${pwd()}"
}
)
}
}
stage("Test") {
agent { label 'some-agent' }
steps {
parallel(
jslint: {
echo "jslint: ${pwd()}"
},
phpcs: {
echo "phpcs: ${pwd()}"
},
)
}
}
stage("Select deploy target") {
agent none
steps {
input message: 'Deploy?'
}
}
stage("Deploy") {
agent { label 'some-agent' }
steps {
echo "deploy: ${pwd()}"
}
}
}
}
However, there are no guarantee that using the same agent label within a Pipeline will always end up using the same workspace, e.g. as another build of the same job while the first build is waiting on the input.
You would have to use stash after the build steps. As you note, this cannot be done normally with parallel at the moment, so you'd have to additionally use a script block, in order to write a snippet of Scripted Pipeline for the stashing/unstashing after/before the parallel steps.
There is a workaround to use the same build slave in the other stages.
You can set a variable with the node name and use it in the others.
ie:
pipeline {
agent none
stages {
stage('First Stage Gets Agent Dynamically') {
agent {
node {
label "some-agent"
}
}
steps {
echo "first stage running on ${NODE_NAME}"
script {
BUILD_AGENT = NODE_NAME
}
}
}
stage('Second Stage Setting Node by Name') {
agent {
node {
label "${BUILD_AGENT}"
}
}
steps {
echo "Second stage using ${NODE_NAME}"
}
}
}
}
As of today (2021), you can use nested stages (https://www.jenkins.io/doc/book/pipeline/syntax/#sequential-stages) to group all the stages that must run in the same workspace before the input step, and all the stages that must be run in the same workspace after the input step. Of course, you need to stash or to store artifacts in some external repository before the input step, because the second workspace may not be the same than the first one:
pipeline {
agent none
stages {
stage('Deployment to Preproduction') {
agent any
stages {
stage('Stage PRE.1') {
steps {
echo "StagePRE.1"
sleep(10)
}
}
stage('Stage PRE.2') {
steps {
echo "Stage PRE.2"
sleep(10)
}
}
}
}
stage('Stage Ask Deploy') {
steps {
input message: 'Deploy to production?'
}
}
stage('Deployment to Production') {
agent any
stages {
stage('Stage PRO.1') {
steps {
echo "Stage PRO.1"
sleep(10)
}
}
stage('Stage PRO.2') {
steps {
echo "Stage PRO.2"
sleep(10)
}
}
}
}
}
}
How can the Jenkins Continuous Integration Game plugin (ci-game) be used in a Jenkins pipeline as code (Jenkinsfile) job?
Unfortunately, the ci-game plugin does not (yet) support pipelines.
The plugin does not appear in the Plugin Compatibility with Pipeline list.
There's already an open ticket on this issue (JENKINS-42683).
It seems that the latest update 1.26 includes the DSL for ci-game (see https://github.com/jenkinsci/ci-game-plugin/pull/19/commits/89e6c3e6ff11294418c2e741ebade5cfaa53ba1d
)
I tested it out and it seems to work when you put ciGame() :
post {
always {
ciGame()
}
}
However, this writer complained that it doesn't work:
https://github.com/jenkinsci/ci-game-plugin/commit/89e6c3e6ff11294418c2e741ebade5cfaa53ba1d
Simple Jenkins declarative pipeline with single Stage
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Hello world!'
}
}
}
}
Simple Jenkins declarative pipeline with multiple Stage
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Inside Stage 1'
}
}
stage('Stage 2') {
steps {
echo 'Inside Stage 2'
}
}
}
}
Simple Jenkins declarative pipeline with Post Actions
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo 'Inside Stage 1'
}
post {
failure {
script { echo 'failure Inside Stage 1' }
}
success {
script { echo 'failure Inside Stage 1' }
}
}
}
stage('Stage 2') {
steps {
echo 'Inside Stage 2'
}
post {
failure {
script { echo 'failure Inside Stage 2' }
}
success {
script { echo 'failure Inside Stage 1' }
}
}
}
}
https://devopsdiagnosis.wixsite.com/tech/forum/jenkins/jenkins-pipeline
I am looking for a Jenkinsfile example of having a step that is always executed, even if a previous step failed.
I want to assure that I archive some builds results in case of failure and I need to be able to have an always-running step at the end.
How can I achieve this?
We switched to using Jenkinsfile Declarative Pipelines, which lets us do things like this:
pipeline {
agent any
stages {
stage('Test') {
steps {
sh './gradlew check'
}
}
}
post {
always {
junit 'build/reports/**/*.xml'
}
}
}
References:
Tests and Artifacts
Jenkins Pipeline Syntax
try {
sh "false"
} finally {
stage 'finalize'
echo "I will always run!"
}
Another possibility is to use a parallel section in combination with a lock. For example:
pipeline {
stages {
parallel {
stage('Stage 1') {
steps {
lock('MY_LOCK') {
echo 'do stuff 1'
}
}
}
stage('Stage 2') {
steps {
lock('MY_LOCK') {
echo 'do stuff 2'
}
}
}
}
}
}
Parallel stages in a parallel section only abort other stages in the same parallel section if the fail fast option for the parallel section is set. See the docs.