I am trying to write a jenkinsfile which will take the data from shared libraries in jenkins for multibranch pipeline, something like below:-
#Library('Template')_
if (env.BRANCH_NAME == 'master') {
jenkins1(PROJECTNAME: 'test', GITURL: 'http://test/test.git')
} else {
jenkins2(PROJECTNAME: 'test1', GITURL: 'http:////test/test.git')
}
so the pipeline take the shared library depending upon the if condition, if the branch is master if statement data should work or else should be build.
Yes that’s possible. Actually we’re using a multibranch project to test our changes to our shared library that way.
You have to use the library step to load the library instead of the #Library annotation, like:
if (condition) {
library(‘someLib#${env.BRANCH_NAME}’)
} else {
library(‘someOtherLib’)
}
See https://jenkins.io/doc/pipeline/steps/workflow-cps-global-lib/#library-load-a-shared-library-on-the-fly for all details.
By the way: In case you’re planning to do Pull Requests the following Post might be useful to you as well: https://stackoverflow.com/a/51915362/4279361
tell me please, can I integrate one jenkinsfile with declarative pipeline into another?
The idea is to run in parallel several processes that are divided into different jenkinsfiles.
stage('run-parallel-branches') {
steps {
parallel(
a: {
echo "call Jenkinsfile 1"
},
b: {
echo "call Jenkinsfile 2"
}
)
}
}
Thanks for the help.
You have two solutions:
If your goal is to avoid repeating code, you can use Shared Libraries and extract the work done in Jenkinsfile1 and Jenkinsfile2 into a library, and then call this library instead.
Your two Jenkinsfile can have their dedicated jobs, and you would call them in the parallel stages (it will wait and propagate errors): build 'myJob1' & build 'myJob2'
The problem with this approach, however, is that the invoked job will not necessarily have the same SCM commit as the parent one (if commits entered meanwhile).
I have a jenkins file as below
pipelineJob('My pipeline job'){
displayName('display name')
logRotator {
numToKeep(10)
daysToKeep(30)
artifactDaysToKeep(7)
artifactNumToKeep(1)
}
definition{
cps {
script(readFileFromWorkspace('./cicd/pipelines/clone_git_code.groovy'))
script(readFileFromWorkspace('./cicd/pipelines/install_dependencies_run_quality_checks.groovy'))
}
}
}
with above jenkinsfile the last script file is replacing other scripts.
Basically I have split tasks into multiple groovy files so that i wont repeat the same code in all jenkinsfile and reuse the same for other jobs as well, like I can now use the clone_git_code.groovy script in dev build as well as QA builds.
You have to use shared libraries (https://jenkins.io/doc/book/pipeline/shared-libraries/). You can define multiple groovy files with classes to return a processed object or simply creating calls with method where you define a step and the execution will be sequential.
I had this same issue when trying to include multiple scripts into a Jenkins job. After doing some research, I found the below solution to be the simplest:
definition {
cps {
script (
ScriptsLibrary.pipelineTest('did it work?') +
ScriptsLibrary.scmConf('repoURL_input', 'accessCredentials', 'activeBranch')
)
}
}
Add the "+" to concatenate the Strings. Got the job done for me :)
Problem
Our source is a single large repository that contains multiple projects. We need to be able to avoid building all projects within the repository if a commit happens within specific areas. We are managing our build process using pipelines.
Research
The git plugin provides the ability to ignore commits from certain user, paths, and message content. However, as we are using the pipeline, we believe we are experiencing the issue described by JENKINS-36195. In one of the most recent comments, Jesse suggests examining the changeset and returning early if the changes look boring. He mentions that a return statement does not work inside a library, closure, etc), but he doesn't mention how a job could be aborted.
Potential Approaches
We have considered using the error step, but this would result in the job being marked as having a failure and would need to be investigated.
While a job result could be marked as NOT_BUILT, the job is not aborted but continues to process all stages.
Question
How would you abort a job during an early step without marking it as a failure and processing all stages of the pipeline (and potentially additional pipelines)?
Can you try using try catch finally block in the pipeline.
try{
}
catch {
}
finally{
}
I suppose you can also use post build action in the pipeline.
pipeline {
agent any
stages {
stage('Test') {
steps {
sh 'make check'
}
}
}
post {
always {
junit '**/target/*.xml'
}
failure {
mail to: team#example.com, subject: 'The Pipeline failed :('
}
}
}
The documentation is below https://jenkins.io/doc/book/pipeline/syntax/#post
you can also try using the below one outside of build step with your conditions as specified by Slawomir Demichowicz in the ticket.
if (stash.isJenkinsLastAuthor() && !params.SKIP_CHANGELOG_CHECK) {
echo "No more new changes, stopping pipeline"
currentBuild.result = "SUCCESS"
return
}
I am not sure this could help you.
The Jenkins Pipeline plugin (aka Workflow) can be extended with other Multibranch plugins to build branches and pull requests automatically.
What would be the preferred way to run multiple configurations? For example, building with Java 7 and Java 8. This is often called matrix configuration (because of the multiple combinations such as language version, framework version, ...) or build variants.
I tried:
executing them serially as separate stage steps. Good, but takes more time than necessary.
executing them inside a parallel step, with or without nodes allocated inside them. Works but I cannot use the stage step inside parallel for known limitations on how it would be visualized.
Is there a recommended way to do this?
TLDR: Jenkins.io wants you to use nodes for each build.
Jenkins.io: In pipeline coding contexts, a "node" is a step that does two things, typically by enlisting help from available executors on agents:
Schedules the steps contained within it to run by adding them to the Jenkins build queue (so that as soon as an executor slot is free on a node, the appropriate steps run)
It is a best practice to do all material work, such as building or running shell scripts, within nodes, because node blocks in a stage tell Jenkins that the steps within them are resource-intensive enough to be scheduled, request help from the agent pool, and lock a workspace only as long as they need it.
Vanilla Jenkins Node blocks within a stage would look like:
stage 'build' {
node('java7-build'){ ... }
node('java8-build'){ ... }
}
Further extending this notion Cloudbees writes about parallelism and distributed builds with Jenkins. Cloudbees workflow for you might look like:
stage 'build' {
parallel 'java7-build':{
node('mvn-java7'){ ... }
}, 'java8-build':{
node('mvn-java8'){ ... }
}
}
Your requirements of visualizing the different builds in the pipeline would could be satisfied with either workflow, but I trust the Jenkins documentation for best practice.
EDIT
To address the visualization #Stephen would like to see, He's right - it doesn't work! The issue has been raised with Jenkins and is documented here, the resolution of involving the use of 'labelled blocks' is still in progress :-(
Q: Is there documentation letting pipeline users not to put stages inside of parallel steps?
A: No, and this is considered to be an incorrect usage if it is done; stages are only valid as top-level constructs in the pipeline, which is why the notion of labelled blocks as a separate construct has come to be ... And by that, I mean remove stages from parallel steps within my pipeline.
If you try to use a stage in a parallel job, you're going to have a bad time.
ERROR: The ‘stage’ step must not be used inside a ‘parallel’ block.
I would suggest Declarative Matrix as a preferred way to run multiple configurations in Jenkins. It allows you to execute the defined stages for every configuration without code duplication.
Example:
pipeline {
agent none
stages {
stage('Test') {
matrix {
agent {
label "${NODENAME}"
}
axes {
axis {
name 'NODENAME'
values 'java7node', 'java8node'
}
}
stages {
stage('Test') {
steps {
echo "Do Test for ${NODENAME}"
}
}
}
}
}
}
}
Note that declarative Matrix is a native declarative Pipeline feature, so no additional Plugin installation needed.
Jenkins blog post about the matrix directive.
As noted by #StephenKing, Blue Ocean will show parallel branches better than the current stage view. A planned upcoming version of the stage view will be able to show all the branches, though it will not visually indicate any nesting structure (would look the same as if you ran the configurations serially).
In any event, the deeper issue is that you will essentially only get a pass/fail status for the build overall, pending a resolution to JENKINS-27395 and related requests.
In order to test each commit on several platforms, I've used this base Jenkinsfile skeleton:
def test_platform(label, with_stages = false)
{
node(label)
{
// Checkout
if (with_stages) stage label + ' Checkout'
...
// Build
if (with_stages) stage label + ' Build'
...
// Tests
if (with_stages) stage label + ' Tests'
...
}
}
/*
parallel ( failFast: false,
Windows: { test_platform("Windows") },
Linux: { test_platform("Linux") },
Mac: { test_platform("Mac") },
)
*/
test_platform("Windows", true)
test_platform("Mac", true)
test_platform("Linux", true)
With this it's relatively easy to switch from a sequential to a parallel execution, each of them having their pros and cons:
Parallel execution runs much faster, but it doesn't contain the stages labelling
Sequential execution is much slower, but you get a detailed report thanks to stages, labelled as "Windows Checkout", "Windows Build", "Windows Tests", "Mac Checkout", etc.)
I'm using the sequential execution for the time being, until I find a better solution.
It seems like there is relief coming at least with the BlueOcean UI. Here is what I got (the tk-* nodes are the parallel steps):