Nested docker agents in a Jenkinspipeline - jenkins

I have the following setup:
Jenkinsmaster, no docker installed
Jenkinsslave, docker is installed, label dockerslave
When I run the following pipeline:
pipeline {
agent { node { label 'dockerslave' } }
stages {
stage('Example Build') {
agent { docker { image 'maven:3-alpine' } }
steps {
echo 'Hello, Maven'
sh 'mvn --version'
}
}
stage('Example Test') {
agent { docker { image 'openjdk:8-jre' } }
steps {
echo 'Hello, JDK'
sh 'java -version'
}
}
}
}
I get the following logoutput:
[Pipeline] node
Running on dockerslave in /home/jenkins/workspace/docker-
declarative
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Example Build)
[Pipeline] node
Still waiting to schedule task
There are no nodes with the label ?latest?
The job doesn't proceed and hangs.
What is the problem here?

The problem was the missing:
reuseNode true
The fixed example:
pipeline {
agent {
node { label 'dockerslave' } }
stages {
stage('Example Build') {
agent {
docker {
reuseNode true
image 'maven:3-alpine'
}
}
steps {
echo 'Hello, Maven'
sh 'mvn --version'
}
}
stage('Example Test') {
agent {
docker {
reuseNode true
image 'openjdk:8-jre'
}
}
steps {
echo 'Hello, JDK'
sh 'java -version'
}
}
}
}

Related

How to achive nested parallel in Jenkins declarative pipeline?

I have declarative pipeline as below and it runs 2* and 3* stages in parallel, pasted the blue ocean diagram below.
pipeline {
agent { label 'my_node' }
options {
timestamps()
parallelsAlwaysFailFast()
}
stages {
stage('1') {
steps {
script {
step([$class: 'WsCleanup'])
}
}
}
stage('2') {
parallel {
stage("2.1") {
steps {
script {
sh 'echo hi 2.1'
}
}
}
stage("2p") {
steps {
script {
sh 'echo hi 2p'
}
}
}
}
}
stage('3') {
parallel {
stage('3.1') {
steps {
script {
sh """
echo hi 3.1
"""
}
}
}
stage('3.2') {
steps {
script {
sh """
echo "hi 3.2"
"""
}
}
}
}
}
stage('4') {
steps {
script {
sh "echo end"
}
}
}
}
}
But I am looking to run 2p in parallel to 2* and 3*, like shown below, is there a way?
I tried to use paralle under parallel, to start 2p in parallel to 2 and 3, and nested parallel to run 3.1. and 3.2 underneath, but declarative pipeline is not allowing nested parallel.
You can't do this only with Declarative syntax. But you can achieve this with a combination of Scripted and Declarative syntax. One thing to note is, AFAIK there is no visualization support for nested parallel stages as of now. There is a feature request for this here.
Following is a sample pipeline you can use as a reference for your use case.
pipeline {
agent any
options {
timestamps()
parallelsAlwaysFailFast()
}
stages {
stage('1') {
steps {
script {
step([$class: 'WsCleanup'])
}
}
}
stage('2 AND 3') {
steps {
script {
parallel getWrappedStages()
}
}
}
stage('4') {
steps {
script {
sh "echo end"
}
}
}
}
}
def getWrappedStages() {
stages = [:]
stages["Step2.1"] = { stage('2.1') {
sh """
echo hi 2.1
"""
}
parallel parallel3xstages()
}
stages["Step2.p"] = { stage('2.p') {
sh """
echo hi 2.p
"""
}
}
return stages
}
def parallel3xstages() {
stages = [:]
stages["Step3.1"] = { stage('3.1') {
sh """
echo hi 3.1
"""
}
}
stages["Step3.2"] = { stage('3.2') {
sh """
echo hi 3.2
"""
}
}
return stages
}

How to configure SCM (gitHub) checkout re-try in multibranch declarative Jenkins pipeline?

My multibranch declarative Jenkins pipeline is failing very frequently during SCM checkout with timeout error and it works after one or two retries. is there anyway to automate the retry the SCM checkout?
Jenkinsfile
agent {
label "agent1"
}
stages {
stage('Test') {
parallel {
stage('Test1') {
steps { sh 'echo Test 1 passed' }
}
stage('Test2') {
steps {
sh 'echo Test2 is passed'
}
} stage('Test3') {
steps {
sh 'echo Test 3 passed'
}
}
}
}
}
You can add wrapper steps such as "retry" to retry the SCM checkout step.
agent {
label "agent1"
}
stages {
stage('Test') {
parallel {
stage('Test1') {
steps {
retry(3) {
echo "Test 1 passed"
}
}
}
stage('Test2') {
steps {
retry(3) {
echo "Test 1 passed"
}
}
}
}
}
}

How to move steps inside stage to a function in Jenkins pipeline

I have a Jenkinsfile like this
pipeline {
agent { label 'master' }
stages {
stage('1') {
steps {
script {
sh '''#!/bin/bash
source $EXPORT_PATH_SCRIPT
cd $SCRIPT_PATH
python -m scripts.test.test
'''
}
}
}
stage('2') {
steps {
script {
sh '''#!/bin/bash
source $EXPORT_PATH_SCRIPT
cd $SCRIPT_PATH
python -m scripts.test.test
'''
}
}
}
}
}
As you can see, In both stages I am using the same script and calling the same file.
Can I move this step to a function in Jenkinsfile and call that function in script? like this
def execute script() {
return {
sh '''#!/bin/bash
source $EXPORT_PATH_SCRIPT
cd $SCRIPT_PATH
python -m scripts.test.test
'''
}
}
Yes, It's possible like below example:
Jenkinsfile
def doIt(name) {
return "The name is : ${name}"
}
def executeScript() {
sh "echo HelloWorld"
}
pipeline {
agent any;
stages {
stage('01') {
steps {
println doIt("stage 01")
executeScript()
}
}
stage('02') {
steps {
println doIt("stage 02")
executeScript()
}
}
}
}

Jenkinsfile nested stage(s) throwing error

I have the following Jenkinsfile which I believe is setup correctly. I used https://jenkins.io/doc/book/pipeline/syntax/#sequential-stages as an example but for some reason when I run this in jenkins I am receiving,
WorkflowScript: 11: Unknown stage section "stages". Starting with
version 0.5, steps in a stage must be in a steps block
Can someone tell me what I am missing or doing wrong?
pipeline {
agent {label 'windows'}
stages {
stage('Quick Build') {
steps {
echo 'Building'
}
}
stage('Deploy to Dev') {
// when {
// branch 'develop'
// }
stages {
stage('Building Distributable Package') {
steps {
echo 'Building'
}
}
stage('Archiving Package') {
steps {
echo 'Archiving Aritfacts'
archiveArtifacts artifacts: '/*.zip', fingerprint: true
}
}
stage('Deploying Dev') {
steps {
echo 'Deploying'
timeout(time:3, unit:'DAYS') {
input message: "Approve build?"
}
}
}
}
}
stage('Deploy to Test') {
when {
branch 'develop'
}
steps {
echo 'deploying..'
timeout(time:3, unit:'DAYS') {
input message: "Approve build?"
}
}
}
stage('Deploy to Prod') {
when {
branch 'release'
}
steps {
timeout(time:3, unit:'DAYS') {
input message: "Deploy to Prod?"
}
echo 'Deploying....'
}
}
}
}
Thanks in advance!
This ended up being a problem in version 2.107.3. Once upgraded to 2.121.2 this functionality started working.

How to declare many agents in a Jenkinsfile with Declarative syntax?

In the Pipeline scripted syntax we use this syntax to declare specific steps to specific nodes:
steps{
node('node1'){
// sh
}
node('node2'){
// sh
}
}
But I want to use the Pipeline Declarative Syntax, can I put many agents?
Sure you can. Just take a look at example (from docs):
pipeline {
agent none
stages {
stage('Build') {
agent any
steps {
checkout scm
sh 'make'
stash includes: '**/target/*.jar', name: 'app'
}
}
stage('Test on Linux') {
agent {
label 'linux'
}
steps {
unstash 'app'
sh 'make check'
}
post {
always {
junit '**/target/*.xml'
}
}
}
stage('Test on Windows') {
agent {
label 'windows'
}
steps {
unstash 'app'
bat 'make check'
}
post {
always {
junit '**/target/*.xml'
}
}
}
}
}

Resources