I have below code which is executing fine on 1 node - Server1:
pipeline {
agent {
node {
label 'Server1'
}
}
stages {
stage('Stage1') {
steps {
callStage1()
}
}
stage('Stage2') {
steps {
callStage2()
}
}
}
}
def callStage1()
{
sh ''' #shell script
'''
}
def callStage2()
{
sh ''' #shell script
'''
}
I want to execute Stage 1 on Server1 only and Stage2 on 3 nodes serially - Server1,Server2,Server3.
How can this be achieved?
Related
I want to execute multiple jobs from a single pipeline using declarative syntax in parallel. Can this be possible!! I know we can make a declarative parallel pipeline using "parallel" parameter.
pipeline {
agent any
parallel{
stages {
stage('Test1') {
steps {
sh 'pip install -r requirements.txt'
}
}
stage('Test2') {
steps {
echo 'Stage 2'
sh 'behave -f allure_behave.formatter:AllureFormatter -o allure-results features/scenarios/**/*.feature'
}
}
stage('Test3') {
steps {
script {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'allure-results']]
])
}
}
}
}
}
}
Below image will show you the proper flow that I want. Any approach how to do it?
// Pipeline project: SO-69680107-1-parallel-downstream-jobs-matrix
pipeline {
agent any
stages {
stage('Clean Workspace') {
steps {
cleanWs()
}
}
stage('Job matrix') {
matrix {
axes {
axis {
name 'job'
values 'SO-69680107-2', 'SO-69680107-3', 'SO-69680107-k' // , ...
}
}
stages {
stage('Run job') {
steps {
build "$job"
copyFiles( "$WORKSPACE\\..\\$job", "$WORKSPACE")
}
} // stage 'Run job'
}
} // matrix
} // stage 'Job matrix'
stage('List upstream workspace') {
steps {
bat "#dir /b \"$WORKSPACE\""
}
}
} // stages
}
def copyFiles( downstreamWorkspace, upstreamWorkspace ) {
dir("$downstreamWorkspace") {
bat """
#set prompt=\$g\$s
#echo Begin: %time%
dir /b
xcopy /f *.* \"$upstreamWorkspace\\\"
#echo End: %time%
"""
}
}
Template for downstream projects SO-69680107-2, SO-69680107-3, SO-69680107-k:
// Pipeline project: SO-69680107-X
pipeline {
agent any
stages {
stage('Stage X') {
steps {
sh 'set +x; echo "Step X" | tee SO-69680107-X.log; date; sleep 3; date'
}
}
}
}
I have some parallel stages in my Jenkins pipeline. They are all identical, except that they run on different agents:
stage {
parallel {
stage {
agent {
label 'agent-1'
}
steps {
sh 'do task number 468'
}
}
stage {
agent {
label 'agent-2'
}
steps {
sh 'do task number 468'
}
}
stage {
agent {
label 'agent-3'
}
steps {
sh 'do task number 468'
}
}
}
}
I want to add more parallel stages on more nodes, but the script is long and repetetive. What's the best way to rewrite this to tell jenkins to parallelize the same steps across agents 1, 2, 3, 4...etc?
Please see below code which will create and run the stage on multiple agents:
// Define your agents
def agents = ['agent-1','agent-2','agent-3']
def createStage(label) {
return {
stage("Runs on ${label}") {
node(label) {
// build steps that should happen on all nodes go here
echo "Running on ${label}"
sh 'do task number 468'
}
}
}
}
def parallelStagesMap = agents.collectEntries {
["${it}" : createStage(it)]
}
pipeline {
agent none
stages {
stage('parallel stage') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
More information is available at : Jenkins examples
I have list of items, each of them need to go through two stages, when for each of the items the stages must to be one after the other,
but the items can run parallelly.
So, I tried to create a nested parallel stages for every item in the list, so all the items will run parallelly, and for each of them the stages will run one after the other
when I tried to do it without a loop (only for 2 items from my list) it worked this way:
stage('nested parallel stage') {
parallel {
stage("stages for item1") {
steps {
script {
stage("stage1 for item1 ") {
sh "echo stage 1 for item1"
} //stage 1
stage("stage 2 for item1") {
sh "echo stage 2 to item1"
} // stage 2
} //script
} // steps
} // stage
stage("stages for item2") {
steps {
script {
stage("staqge1 for item 2") {
sh "echo stage 1 for item2"
} //stage 1
stage("stage 2 for item2") {
sh "echo stage 2 to item2"
} // stage2
} //script
} // steps
} // stage
} //parallel
} // nested parallel stage
It worked without any problem.
But, when I tried to do it in a loop, like this:
def itemsMap = getitemsMap ()
def parallelItemsStages = itemsMap.collectEntries{
[ "${it.value.item}": {
stage("stages for ${it.value.item}") {
steps{
script {
stage("stage 1 for ${it.value.item}") {
sh "echo stage 1 for ${it.value.item}"
} //stage 1
stage("stage 2 for ${it.value.item}") {
sh "echo stage 2 for ${it.value.item}"
} // stage 2
} // script
} // steps
} // stage
} // nested stages
}] // collectEntries
} // collectEntries
parallel parallelItemsStages
I got this error:
ava.lang.NoSuchMethodError: No such DSL method 'steps' found among steps
Trying to understand getitemsMap (); May I ask what is the role of def itemsMap = getitemsMap ()
try this:
def itemsMap = getitemsMap ()
def parallelItemsStages = itemsMap.collectEntries{
[ "${it.value.item}": {
script {
stage("stage 1 for ${it.value.item}") {
sh "echo stage 1 for ${it.value.item}"
} //stage 1
stage("stage 2 for ${it.value.item}") {
sh "echo stage 2 for ${it.value.item}"
} // stage 2
} // script
} // nested stages
}] // collectEntries
} // collectEntries
parallel parallelItemsStages
that worked for me just now :)
I have a jenkinsfile that calls a function from groovy:
jenkinsfile:
pipeline {
agent none
environment {
HOME = '.'
}
stages {
stage("initiating"){
agent {
docker {
image 'docker-image'
}
}
stages {
stage('scanning') {
steps {
script {
workloadPipeline = load("Pipeline.groovy")
workloadPipeline.loopImages1(Images)
}
}
}
}
}
}
}
groovy functions:
def loopImages1(Images){
Images.each { entry ->
parallel {
stage('test-' + entry.key) {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
script {
sh """
docker run -d $entry.value
"""
}
}
}
}
}
}
Images returns a map, something like this:
image-1 : 123.dkr.ecr.eu-west-1.amazonaws.com....
image-2 : 123.dkr.ecr.eu-west-1.amazonaws.com....
image-3 : 123.dkr.ecr.eu-west-1.amazonaws.com....
And I was trying to run it with parallel, which in this case should run 3 jobs in parallel, but it gives me the following error message:
java.lang.IllegalArgumentException: Expected named arguments but got
org.jenkinsci.plugins.workflow.cps.CpsClosure2#19027e83
What do I need to change in order to get this to work? From what I read it needs a map as input, which I'm already giving.
In case anyone has a similar question, here is the answer that solved my problem:
groovy function:
def loopImages1(Images){
**def parallelStage = [:]**
Images.each { entry ->
**parallelStage[entry] = {**
stage('test-' + entry.key) {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
script {
sh """
docker run -d $entry.value
"""
}
}
}
}
}
**parallel parallelStage**
}
Ｉhave noticed that Jenkins pipeline file -- Jenkinsfile which have two syntax
Declarative
Scripted
I have made Declarative Script work to specify node to run my task. However I don't know how to modify my script to Scripted syntax.
My Declarative Script
pipeline {
agent none
stages {
stage('Build') {
agent { label 'my-label' }
steps {
echo 'Building..'
sh '''
'''
}
}
stage('Test') {
agent { label 'my-label' }
steps {
echo 'Testing..'
sh '''
'''
}
}
stage('Deploy') {
agent { label 'my-label' }
steps {
echo 'Deploying....'
sh '''
'''
}
}
}
}
I have tried to use in this way:
node('my-label') {
stage 'SCM'
git xxxx
stage 'Build'
sh ''' '''
}
But it seems Jenkins cannot find my node to run.
How about this simple example?
stage("one") {
node("linux") {
echo "One"
}
}
stage("two") {
node("linux") {
echo "two"
}
}
stage("three") {
node("linux") {
echo "three"
}
}
Or the below answer, this way you are guaranteed to have the stages run on the same node if there are multiple nodes with the same label and run interrupted by another job.
The above example will release the node after every stage, the below example will hold the node for all three stages.
node("linux") {
stage("one") {
echo "One"
}
stage("two") {
echo "two"
}
stage("three") {
echo "three"
}
}