I have a jenkins workflow configuration that is running several test jobs in parallel. What I would like to do, is aggregate all test results and display them on the 'workflow' job page.
Here is my current configuration:
node('git && linux') {
// dome some stuff here
}
stage "Running unit tests for $REVISION"
build job: 'unit-tests', parameters: [[$class: 'StringParameterValue', name: 'REVISION', value: REVISION], [$class: 'StringParameterValue', name: 'REFSPEC', value: REFSPEC]]
stage "Running integration tests $REVISION"
def jobs = [:]
jobs['integration-tests-1']={build job: 'integration-tests-job-1', parameters: [[$class: 'StringParameterValue', name: 'REVISION', value: REVISION], [$class: 'StringParameterValue', name: 'REFSPEC', value: REFSPEC]]}
jobs['integration-tests-2']={build job: 'integration-tests-job-2', parameters: [[$class: 'StringParameterValue', name: 'REVISION', value: REVISION], [$class: 'StringParameterValue', name: 'REFSPEC', value: REFSPEC]]}
jobs['integration-tests-3']={build job: 'integration-tests-job-3', parameters: [[$class: 'StringParameterValue', name: 'REVISION', value: REVISION], [$class: 'StringParameterValue', name: 'REFSPEC', value: REFSPEC]]}
parallel jobs
// Here I would like to add something like
// aggregateDownstream('unit-tests', 'integration-tests-1', 'integration-tests-2', 'integration-tests-3'
Do you know if it would be possible to achieve this?
Related
I am running a Jenkins job where I call another Jenkins job to build azure environments.
I create a 2d array [:] and store 3 jobs inside.
When I call the keyword 'parallel' on the array, the 3 jobs should run in parallel. This has worked for all my past Jenkins files, but when I run it here, it only runs one or two of the three jobs.
node(label: 'master')
{
def branches = [:]
stage ('Parallel Builds')
{
for (int i = 0; i < 3; i++)
{
branches["branch${i}"] = prepare(i)
}
echo "branches: ${branches}"
parallel branches
}
}
def prepare(def num)
{
return {
build job: 'Azure/Environment-General/Environment - Create', parameters: [
[$class: 'StringParameterValue', name: 'BOHSnapshotName', value: 'snap-win10-19.6.9-boh-cfc-qs'],
[$class: 'StringParameterValue', name:'Terminal1SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'Terminal2SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'EnvironmentPrefix', value: 'jl250638-'+num]
]
}
}
Jenkins console - skipping job when running in parallel
I am expecting all parallel jobs to run together but it keeps skipping one or two.
EDIT: I have also implemented a retry(3) for the failed branches but the jobs that fail just hang infinitely... below is the retry code as well as a picture to the jenkins console..
Jenkins console - jobs that fail hang
node(label: 'master')
{
def branches = [:]
branches.failFast = false
stage ('Parallel Builds')
{
for (int i = 0; i < 3; i++)
{
branches["branch${i}"] = prepare(i)
}
echo "branches: ${branches}"
}
try
{
parallel branches
}
catch(Exception e)
{
e.getCauses().each
{
echo "${it.getShortDescription()}"
}
}
}
def prepare(def num)
{
return {
try
{
build job: 'Azure/Environment-General/Environment - Create', parameters: [
[$class: 'StringParameterValue', name: 'BOHSnapshotName', value: 'snapPOSQSWithEDC1.2'],
[$class: 'StringParameterValue', name:'Terminal1SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'Terminal2SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'EnvironmentPrefix', value: 'jl250638-0-PARALLEL-'+num],
[$class: 'StringParameterValue', name:'ResourceGroupName', value: 'rg-aloha-pos-automation']
]
}
catch(error)
{
echo "First build failed, let's retry if accepted"
retry(3)
{
input "***Retry the job***"
build job: 'Azure/Environment-General/Environment - Create', parameters: [
[$class: 'StringParameterValue', name: 'BOHSnapshotName', value: 'snapPOSQSWithEDC1.2'],
[$class: 'StringParameterValue', name:'Terminal1SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'Terminal2SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'EnvironmentPrefix', value: 'jl250638-PARALLEL-'+num],
[$class: 'StringParameterValue', name:'ResourceGroupName', value: 'rg-aloha-pos-automation']
]
}
}
}
}
Try adding propagate: false to the build command.
build job: 'Azure/Environment-General/Environment - Create', propagate: false, parameters: [
[$class: 'StringParameterValue', name: 'BOHSnapshotName', value: 'snap-win10-19.6.9-boh-cfc-qs'],
[$class: 'StringParameterValue', name:'Terminal1SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'Terminal2SnapshotName', value: 'none'],
[$class: 'StringParameterValue', name:'EnvironmentPrefix', value: 'jl250638-'+num]
Found a very stupid solution..... The job I was calling had the parameter checked off 'Do not allow concurrent builds'..... I unchecked and parallel works fine.
Will keep this up just in case anyone makes this mistake as well Lol
I've Pipeline Generic Webhook from Bitbucket, this is a job to trigger another job.
currentBuild.displayName = "Generic-Job-#" + env.BUILD_NUMBER
pipeline {
agent any
triggers {
GenericTrigger(
genericVariables: [
[key: 'actorName', value: '$.actor.display_name'],
[key: 'TAG', value: '$.push.changes[0].new.name'],
[key: 'REPONAME', value: '$.repository.name'],
[key: 'GIT_URL', value: '$.repository.links.html.href'],
],
token: '11296ae8d97b2134550f',
causeString: ' Triggered on $actorName version $TAG',
printContributedVariables: true,
printPostContent: true
)
}
stages {
stage('Build Job DEVELOPMENT') {
when {
expression { return params.TARGET_ENV == 'DEVELOPMENT' }
}
steps {
build job: 'DEVELOPMENT',
parameters: [
[$class: 'StringParameterValue', name: 'FROM_BUILD', value: "${BUILD_NUMBER}"],
[$class: 'StringParameterValue', name: 'TAG', value: "${TAG}"],
[$class: 'StringParameterValue', name: 'GITURL', value: "${GIT_URL}"],
[$class: 'StringParameterValue', name: 'REPONAME', value: "${REPONAME}"],
[$class: 'StringParameterValue', name: 'REGISTRY_URL', value: "${REGISTRY_URL}"],
]
}
}
}
}
Another Pipeline
pipeline {
agent any
stages {
stage('Cleaning') {
steps {
cleanWs()
}
}
def jenkinsFile
stage('Loading Jenkins file') {
jenkinsFile = fileLoader.fromGit('Jenkinsfile', "${GIT_URL}", "${TAG}", null, '')
}
jenkinsFile.start()
}
}
can i run Jenkinsfile in Pipeline ? Because every project I make has a different Jenkinsfile, it can't be the same, but when I run this it doesn't execute the Jenkinsfile
it works for me :D
Sample Pipeline
stage 'Load a file from GitHub'
def jenkinsFile = fileLoader.fromGit('<path-jenkinsfile>', "<path-git>", "<branch>", '<credentials>', '')
stage 'Run method from the loaded file'
jenkinsFile
pipeline {
agent any
stages {
stage('Print Hello World Ke #1') {
steps {
echo "Hello Juan"
}
}
}
}
before run the pipeline, you must install plugin "Pipeline Remote Loader Plugin Version"
Below is the pipeline script for my job:
pipeline {
agent any
stages {
stage('Runbook database deploy job') {
steps {
withCredentials([
usernamePassword(credentialsId: 'RUNBOOK_RDS_DBADMIN', passwordVariable: 'DB_APPTEAM_PASS', usernameVariable: 'DB_APPTEAM_USER'),
usernamePassword(credentialsId: 'nexus_download_user', passwordVariable: 'NEXUS_APPTEAM_PASS', usernameVariable: 'NEXUS_APPTEAM_USER')
]) {
build job: 'DBA-TSS/db_aurora-mysql_DEPLOY/', parameters: [
string(name: 'INSTALL_TYPE', value: "${INSTALL_TYPE}"),
string(name: 'SDLC', value: "${SDLC}"),
string(name: 'DB_ARTIFACT', value: "${DB_ARTIFACT}"),
string(name: 'DEV_NEXUS_ID', value: NEXUS_APPTEAM_USER),
[$class: 'hudson.model.PasswordParameterValue', name: 'DEV_NEXUS_PASSWORD', value:hudson.util.Secret.fromString(NEXUS_APPTEAM_PASS)],
string(name: 'DEV_NEXUS_REPO', value: "${DEV_NEXUS_REPO}"),
string(name: 'DB_USERNAME', value: DB_APPTEAM_USER),
[$class: 'hudson.model.PasswordParameterValue', name: 'DB_PASSWORD', value:hudson.util.Secret.fromString(DB_APPTEAM_PASS)],
string(name: 'REQUESTER_EMAIL', value: "${REQUESTER_EMAIL}")
]}
}
}
}
}
And now as hudson is not supported , we want to replace it with Credentials id or password parameter to pass it to downstream job for password variable. can you please help us on this.
I have this jenkins pipe line which has multiple stages. Inside these stages, there are multiple jobs being executed.
When I build the job I'd like to have a set of check boxes and the pipe line should build only what I've checked inside the pipeline stages. Is there any plugins or methods I can use to achieve this?
Sample pipeline code.
As per below example, there are jobs called job_A1, job_B1, job_C1, job_D1, job_A2, job_B2, job_C2 and job_D2. If I click Build with parameters, it should prompt me check boxes and I should be able to check any job I want so that the pipe line will build only the ones I checked.
Thanks in Advance.
pipeline {
agent {label 'server01'}
stages {
stage('Build 01') {
steps {
parallel (
"BUILD A1" : {
build job: 'job_A1',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD B1" : {
build job: 'job_B1',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD C1" : {
build job: 'job_C1',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD D1" : {
build job: 'job_D1',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
)
}
}
stage('Build 02') {
steps {
parallel (
"BUILD A2" : {
build job: 'job_A2',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD B2" : {
build job: 'job_B2',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD C2" : {
build job: 'job_C2',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
"BUILD D2" : {
build job: 'job_D2',
parameters:[
string(name: 'PARAM01', value: "$PARAM01"),
string(name: 'PARAM02', value: "$PARAM02")
]
},
)
}
}
}
}
Thanks #mbn217 for your answer, but ExtendedChoice parameter didn't help much in my scenario.
Anyway, I could do it using boolean parameters and calling it inside the pipeline using the script tag.
Example pipeline script
stage ('BUILD A') {
steps {
script {
if (params.get('boolA',true)) {
build job: '_build_A', parameters: [string(name: 'param1', value: "$param1"),string(name: 'param2', value: "$param2")]
} else {
echo "A is not selected to build"
}
}
}
}
stage ('BUILD B') {
steps {
script {
if (params.get('boolB',true)) {
build job: '_build_B', parameters: [string(name: 'param1', value: "$param1"),string(name: 'param2', value: "$param2")]
} else {
echo "B is not selected to build"
}
}
}
}
You can use ExtendedChoiceParameter to accomplish what you want. Basically you will nee to parametrize job Names too using this jenkins plugin.
You can use a list of checkboxes as shown in the screen shot
We have a jenkins jobs that run autotests with parameters:
HOST;
EXPEIMENT;
TAKE_NEW_SCREENSHOT;
XML_NAME.
All of this parameters have default values,
see screenshot before running parametrizing job:
I need to run several jobs simultaneously with only 2 parameters: HOST and EXPERIMENT.
I created next pipeline-script:
def tasks = [:]
parameters {
string(name: 'HOST', defaultValue: 'www', description: 'host: www, dev3, etc',)
string(name: 'EXPERIMENT', defaultValue: 'withoutExperiment',)
}
tasks['Actions MyBox'] = {
build job: 'MyDocs_Actions_And_Manage_Buttons_MyBox_Tests', parameters: [
string(name: 'HOST', value: 'www'),
string(name: 'EXPERIMENT', value: 'withoutExperiment'),
booleanParam(name: 'TAKE_NEW_SCREENSHOT', value: false),
string(name: 'XML_NAME', value: 'my_docs_actions_buttons_mybox_tests')
]
}
tasks['DashBoard General'] = {
build job: 'DashBoard_General_Tests', parameters: [
string(name: 'HOST', value: 'www'),
string(name: 'EXPERIMENT', value: 'withoutExperiment'),
booleanParam(name: 'TAKE_NEW_SCREENSHOT', value: false),
string(name: 'XML_NAME', value: 'my_docs_dash_board_general_tests')
]
}
tasks['Actions InBox'] = {
build job: 'MyDocs_Actions_Buttons_InBox_Tests', parameters: [
string(name: 'HOST', value: 'www'),
string(name: 'EXPERIMENT', value: 'withoutExperiment'),
booleanParam(name: 'TAKE_NEW_SCREENSHOT', value: false),
string(name: 'XML_NAME', value: 'my_docs_actions_buttons_inbox_tests')
]
}
parallel tasks
and specified parameters in "General" pipeline configuration:
But when I run this pipeline item with parameter value != default value, for example specify HOST = dev12,
anyway all jobs running simultaneously with default parameter values and build shows null specified parameter,
Help me please define a problem.
You're passing hardcoded values to your tasks. For example, you defined
tasks['Actions MyBox'] = {
build job: 'MyDocs_Actions_And_Manage_Buttons_MyBox_Tests', parameters: [
string(name: 'HOST', value: 'www'),
string(name: 'EXPERIMENT', value: 'withoutExperiment'),
booleanParam(name: 'TAKE_NEW_SCREENSHOT', value: false),
string(name: 'XML_NAME', value: 'my_docs_actions_buttons_mybox_tests')
]
}
In this case all parameters are hardcoded and each time when pipeline is executed the value of HOST will be www. And that's why you have null in the HOST parameter description in build execution info (because you're not specifying it in build job command).
So, you need to use something like string(name:'HOST', value: "${params.HOST}")