I have two Jenkins pipelines, let's say pipeline-A and pipeline-B. I want to invoke pipeline-A in pipeline-B. How can I do this?
(pipeline-A is a subset of pipeline-B. Pipeline-A is responsible for doing some routine stuff which can be reused in pipeline-B)
I have installed Jenkins 2.41 on my machine.
Following solution works for me:
pipeline {
agent
{
node {
label 'master'
customWorkspace "${env.JobPath}"
}
}
stages
{
stage('Start') {
steps {
sh 'ls'
}
}
stage ('Invoke_pipeline') {
steps {
build job: 'pipeline1', parameters: [
string(name: 'param1', value: "value1")
]
}
}
stage('End') {
steps {
sh 'ls'
}
}
}
}
Adding link of the official documentation of "Pipeline: Build Step" here:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
A little unclear if you want to invoke another pipeline script or job, so I answer both:
Pipeline script
The "load" step will execute the other pipeline script. If you have both scripts in the same directory, you can load it like this:
def pipelineA = load "pipeline_A.groovy"
pipelineA.someMethod()
Other script (pipeline_a.groovy):
def someMethod() {
//do something
}
return this
Pipeline job
If you are talking about executing another pipeline job,
the "build job" step can accomplish this:
build job: '<Project name>', propagate: true, wait: true
propagate: Propagate errors
wait: Wait for completion
If you have paramters on the job, you can add them like this:
build job: '<Project name>', parameters: [[$class: 'StringParameterValue', name: 'param1', value: 'test_param']]
As mentioned by #Matias Snellingen and #Céline Aussourd, in the case of launching a multibranch job you have to specify the branch to build like this :
stage ('Invoke_pipeline') {
steps {
build job: 'pipeline1/master', parameters: [
string(name: 'param1', value: "value1")
]
}
}
In my case it solved the problem.
I am going to post my solution, which is similar to #Michael COLL, #Matias Snellingen, and #Céline Aussourd.
For the multibranch pipeline I am using the following code in Jenkinsfile to trigger my multibranch B with multibranch A (in the example there are two cases for pipeline and multibranch pipeline):
post {
always {
echo 'We are in post part and Jenkins build with QA tests is going to be triggered.'
// For triggering Pipeline
//build job: 'WGF-QA WITH ALLURE', parameters: [string(name: 'QA-Automation', value: 'value from Build pipeline')]
// For triggering Multibranch Pipeline
build job: 'Testing QA/QA Selenium Tests/feature%2FGET-585', parameters: [string(name: 'QA-Automation', value: 'value from Build pipeline')]
}
}
Just be sure to define the whole path to the branch as is defined in the case and instead of / in branch name use %2F (feature/GET-585 -> feature%2FGET-585).
To add to what #matias-snellingen said. If you have multiple functions, the return this should be under the function that will be called in the main pipeline script. For example in :
def someMethod() {
helperMethod1()
helperMethod2()
}
return this
def helperMethod1(){
//do stuff
}
def helperMethod2(){
//do stuff
}
The someMethod() is the one that will be called in the main pipeline script
Another option is to create a package, load it and execute it from the package.
package name.of.package
import groovy.json.*
def myFunc(var1) {
return result
}
Than consume it
#Library('name_of_repo')
import name.of.package.*
utils = new name_of_pipeline()
// here you can invoke
utils.myFunc(var)
hope it helps
Related
As the title states, I want to capture the logs for all the stages in my build, which looks like this:
pipeline {
agent any
stages {
stage('Build First Repo') {
steps {
build job: 'jobOne', parameters: [string(name: 'branch', value: "${params.branch}")], quietPeriod: 1
}
}
stage ('Build Second Repo') {
steps {
build job: 'jobTwo', parameters: [string(name: 'branch', value: "${params.someOtherBranch}")], quietPeriod: 1
}
}
stage ('Deploy') {
steps {
build job: 'jobThree', parameters: [string(name: 'buildEnvironment', value: "${params.environment}")], quietPeriod: 1
}
}
stage ('Remote Build') {
steps {
build job: 'jobFour', parameters: [string(name: 'Hosts', value: "${params.hosts}")], quietPeriod: 1
}
}
}
post {
always {
mail to:me#mydomain.com, subject: "${currentBuild.currentResult} - ${currentBuild.fullDisplayName}", body:"...${currentBuild.rawBuild.getLog(100)}"
}
}
}
Currently, I can only get the pipeline build log (which I am e-mailing in the post/always section), which is helpful but not sufficient; I'd like to get the logs from each of the stages. I thought of maybe capturing them per stage and creating an environment variable or something but I'm not sure how to even access the logs for the build of those jobs. Can someone point me in the right direction on how to capture the logs for those jobs?
You can add a post section after an any stage.
And you can setup sendind emails there.
To send single email from the build you can use stash/unstash command. You can stash log at each section and finally unstash them and send an email.
Im trying to convert my freestyle job to a declarative pipeline job since the pipeline provides more flexibility. I cannot figure out how to use the NodeLabel parameter plugin (https://wiki.jenkins.io/display/JENKINS/NodeLabel+Parameter+Plugin) in a pipeline however.
pipeline {
agent any
parameters {
// Would like something like LabelParameter here
}
stages {
stage('Dummy1') {
steps {
cleanWs()
sh('ls')
sh('pwd')
sh('hostname')
}
}
stage('Dummy2') {
steps {
node("comms-test02") {
sh('ls')
sh('pwd')
sh('hostname')
}
}
}
}
I basically just need a way to start the job using a parameter that specifies where to build the job (using slave label).
Jenkins requires an agent field to be present which i set to 'any'. But it doesnt seem like there is a labelparameter available ?
As an alternative I tried using the 'node' command (https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#-node- allocate node). But that leaves me with two running jobs which, while working, doesnt look that pretty.
Does anyone if the NodeLabel parameter plugin can be used ? or maybe someone has a cleaner approach ?
Edit: Maybe I wasn't clear. I need to be able to run jobs on different nodes. The node to run on should be decided when triggering the job through a parameter. The node label plugin does this perfectly. However, I have not been able to reproduce this behavior in pipeline.
Here's a full example:
pipeline {
parameters {
choice(name: 'node', choices: [nodesByLabel('label')], description: 'The node to run on') //example 1: just listing all the nodes with label
choice(name: 'node2', choices: ['label'] + nodesByLabel('label'), description: 'The node to run on') //example 2: add the label itself as the first choice to make "Any of the nodes" the default choice
}
agent none
stages {
stage('Test') {
agent { label params.node}
stages {
stage('Print environment settings') {
steps {
echo "running on ${env.NODE_NAME}"
sh 'printenv | sort'
}
}
}
}
}
}
Let's say you added the parameter(say named slaveName) using the NodeLabel plugin on your pipeline. You now need to extract the value of slaveName and feed it into the agent->node->label field.
You can specify the node using the node property inside the agent.
Like this -
agent
{
node
{
label "${slaveName}"
}
}
The following script worked for me to run the multiple jobs parallelly on different Node.
I have taken the reference from the build step plugin documentation.
https://www.jenkins.io/doc/pipeline/steps/pipeline-build-step/
def build_one()
{
parallel one: {
stage('XYZ') {
catchError(buildResult: 'SUCCESS', stageResult:'FAILURE') {
build job: 'yourDownStreamJob', parameters: [[$class: 'NodeParameterValue', name: 'NodeToRun',labels: ['nodeName'], nodeEligibility: [$class: 'AllNodeEligibility']], string(name: 'ParentBuildName', value: "XX"), string(name: 'Browser', value: 'chrome'), string(name: 'Environment', value: 'envName')]
}
}
},
two : {
stage('SecondArea') {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
build job: 'yourDownStreamJob', parameters: [[$class: 'NodeParameterValue', name: 'NodeToRun',labels: ['Your'], nodeEligibility: [$class: 'AllNodeEligibility']], string(name: 'ParentBuildName', value: "XYX"), string(name: 'Browser', value: 'firefox'), string(name: 'Environment', value: 'envName')]
}
}
}
}
build_one()
Below is my pipeline script
node(Slave01) {
currentBuild.displayName = "${URL_Name}"
}
stage 'Pt2ctf process'
node(Slave01) {
build job: 'Pt2ctf_16_7', parameters: [string(name: 'URL_Name', value: "${URL_name}"), string(name: 'Display_Name', value: "${Display_Name}")]
}
stage 'add_fields'
node(Slave01) {
build job: 'add_fields_16_7', parameters: [string(name: 'URL_Name', value: "${URL_Name}")]
}
The above groovy script would trigger multiple builds in sequence. I have another build to be run once the sequence is completed. I don't see any post build option in the pipeline job configuration.
Is it possible that we can add few more lines like below:
post
node(Slave01){
build job: 'testing_build'
}
Or do we have any other option? please suggest
You can simply add post action to your pipeline script, in case of using declarative pipeline.
It is explained in Pipeline syntax reference.
You can add a stage for post build to add post build action in pipeline:
stage 'post-build'
node(Slave01){
build job: 'testing_build'
}
You can use this stage as:
try {
//Stages to be included in build
...
} catch {
...
} finally {
stage 'post-build'
...
}
I have three pipeline projects, project-a, project-b and project-c. project-c takes a parameter. On successful completion of either project-a or project-b I want to trigger a build of project-c with a parameter.
I can do this in project-a and project-b with this code in the pipeline:
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
But this requires two executors. I want project-a or project-b to completely finish before running project-c with parameters.
Your pipeline most likely looks like this:
node {
stage('build') {
// sh "make"
}
// ...
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
}
By wrapping everything inside the node closure, the downstream job project-c is triggered inline, without the upstream job being paused / releasing an executor.
Therefore, things that do essentially nothing for a long time should not be wrapped within a node step, in order to not block an executor. A very similar case is when using the input step to wait for user feedback.
Instead, your pipeline should look e.g. as follows, which is - so to say - the best practice (as you don't block your executor):
stage('build') {
node {
// sh "make"
}
}
// or
node {
stage('build') {
// sh "make"
}
stage('unit') {
// sh "make"
}
} // node
// note: the following code is _not_ wrapped inside a `node` step
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
There is no need to wrap the build step within a node, i.e., block an executor for it. For other steps (like sh), pipeline execution would trigger an error and remind you that it cannot be run outside of a node allocation.
Add the parameter wait: false to the build step to continue pipeline execution without waiting for the downstream job.
EDIT: this would help, if you don't care for the success of the other downstream job for this pipeline. If you need to wait until it is finished and then continue the own (upstream job's) pipeline, then see my other answer.
How can I trigger build of another job from inside the Jenkinsfile?
I assume that this job is another repository under the same github organization, one that already has its own Jenkins file.
I also want to do this only if the branch name is master, as it doesn't make sense to trigger downstream builds of any local branches.
Update:
stage 'test-downstream'
node {
def job = build job: 'some-downtream-job-name'
}
Still, when executed I get an error
No parameterized job named some-downtream-job-name found
I am sure that this job exists in jenkins and is under the same organization folder as the current one. It is another job that has its own Jenkinsfile.
Please note that this question is specific to the GitHub Organization Plugin which auto-creates and maintains jobs for each repository and branch from your GitHub Organization.
In addition to the above mentioned answers: I wanted to start a job with a simple parameter passed to a second pipeline and found the answer on http://web.archive.org/web/20160209062101/https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow
So i used:
stage ('Starting ART job') {
build job: 'RunArtInTest', parameters: [[$class: 'StringParameterValue', name: 'systemname', value: systemname]]
}
First of all, it is a waste of an executor slot to wrap the build step in node. Your upstream executor will just be sitting idle for no reason.
Second, from a multibranch project, you can use the environment variable BRANCH_NAME to make logic conditional on the current branch.
Third, the job parameter takes an absolute or relative job name. If you give a name without any path qualification, that would refer to another job in the same folder, which in the case of a multibranch project would mean another branch of the same repository.
Thus what you meant to write is probably
if (env.BRANCH_NAME == 'master') {
build '../other-repo/master'
}
You can use the build job step from Jenkins Pipeline (Minimum Jenkins requirement: 2.130).
Here's the full API for the build step: https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
How to use build:
job: Name of a downstream job to build. May be another Pipeline job, but more commonly a freestyle or other project.
Use a simple name if the job is in the same folder as this upstream Pipeline job;
You can instead use relative paths like ../sister-folder/downstream
Or you can use absolute paths like /top-level-folder/nested-folder/downstream
Trigger another job using a branch as a param
At my company many of our branches include "/". You must replace any instances of "/" with "%2F" (as it appears in the URL of the job).
In this example we're using relative paths
stage('Trigger Branch Build') {
steps {
script {
echo "Triggering job for branch ${env.BRANCH_NAME}"
BRANCH_TO_TAG=env.BRANCH_NAME.replace("/","%2F")
build job: "../my-relative-job/${BRANCH_TO_TAG}", wait: false
}
}
}
Trigger another job using build number as a param
build job: 'your-job-name',
parameters: [
string(name: 'passed_build_number_param', value: String.valueOf(BUILD_NUMBER)),
string(name: 'complex_param', value: 'prefix-' + String.valueOf(BUILD_NUMBER))
]
Trigger many jobs in parallel
Source: https://jenkins.io/blog/2017/01/19/converting-conditional-to-pipeline/
More info on Parallel here: https://jenkins.io/doc/book/pipeline/syntax/#parallel
stage ('Trigger Builds In Parallel') {
steps {
// Freestyle build trigger calls a list of jobs
// Pipeline build() step only calls one job
// To run all three jobs in parallel, we use "parallel" step
// https://jenkins.io/doc/pipeline/examples/#jobs-in-parallel
parallel (
linux: {
build job: 'full-build-linux', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
mac: {
build job: 'full-build-mac', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
windows: {
build job: 'full-build-windows', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
failFast: false)
}
}
Or alternatively:
stage('Build A and B') {
failFast true
parallel {
stage('Build A') {
steps {
build job: "/project/A/${env.BRANCH}", wait: true
}
}
stage('Build B') {
steps {
build job: "/project/B/${env.BRANCH}", wait: true
}
}
}
}
The command build in pipeline is there to trigger other jobs in jenkins.
Example on github
The job must exist in Jenkins and can be parametrized.
As for the branch, I guess you can read it from git
Use build job plugin for that task in order to trigger other jobs from jenkins file.
You can add variety of logic to your execution such as parallel ,node and agents options and steps for triggering external jobs. I gave some easy-to-read cookbook example for that.
1.example for triggering external job from jenkins file with conditional example:
if (env.BRANCH_NAME == 'master') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam1',value: 'valueOfParam1')
booleanParam(name: 'keyNameOfParam2',value:'valueOfParam2')
]
}
2.example triggering multiple jobs from jenkins file with conditionals example:
def jobs =[
'job1Title'{
if (env.BRANCH_NAME == 'master') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam1',value: 'valueNameOfParam1')
booleanParam(name: 'keyNameOfParam2',value:'valueNameOfParam2')
]
}
},
'job2Title'{
if (env.GIT_COMMIT == 'someCommitHashToPerformAdditionalTest') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam3',value: 'valueOfParam3')
booleanParam(name: 'keyNameOfParam4',value:'valueNameOfParam4')
booleanParam(name: 'keyNameOfParam5',value:'valueNameOfParam5')
]
}
}