I have the following Jenkinsfile:
pipeline {
agent any
environment { }
stages {
stage('stageA') {
steps {
... Do something with arg1, arg2 or arg3
}
}
stage('stageB') {
steps {
... Do something with arg1, arg2 or arg3
}
}
...
}
}
Is there anywhere I can specify a universal "pre-stage" or "post-stage" set of actions to perform? A use-case would be sending logging information at the end of a stage to a log manager, but it would be preferable to not copy and paste those invocations at the end of each and every stage.
As far as I know there is no generic post- or pre-stage hook in Jenkins pipelines. You can define post steps in a post section but you need one per stage.
However, if you don't want to repeat yourself, you have some options.
Use a shared lib
The place to put repeating code to it a shared library. That way allows you to declare your own steps using Groovy.
You need another repository to define a shared lib, but apart from that it is a pretty strait forward way and you can reuse the code in all of your Jenkins' pipelines.
Use a function
If you declare a function outside of the pipeline, you can call it from any stage. This is not really documented and might be prevented in the future. As far as I understand it messes with the coordination between master and agents. However, it works:
pipeline {
agent any
stages {
stage ("First") {
steps {
writeFile file: "resultFirst.txt", text: "all good"
}
post {
always {
cleanup "first"
}
}
}
stage ("Second") {
steps {
writeFile file: "resultSecond.txt", text: "all good as well"
}
post {
always {
cleanup "second"
}
}
}
post {
always {
cleanup "global" // this is only triggered after all stages, not after every
}
}
}
}
void cleanup(String stage) {
echo "cleanup ${stage}"
archiveArtifacts artifacts: "result*"
}
Related
I have 2 questions regarding jenkins
1. What is the difference between the 2 methods?
Method 1
pipeline {
agent any
stages {
stage('Example') {
steps {
echo 'Hello World'
}
}
}
Method 2
node ("jenkins-nodes") {
stage("git clone"){
echo 'Hello World' }
}
As I understand in the first method I can add Post section that will be running regardless of the result of the job. I wish to add the same post section for the second method, but it is not working. Any ideas?
What is the difference between the 2 methods?
As Noam Helmer wrote, pipeline{} is declarative syntax and and node{} is scripted syntax.
In general I recommend to always use pipeline{} as it makes common tasks easier to write and visualization using Blue Ocean plugin works best with declarative pipeline.
When declarative pipeline becomes too unflexible, you can insert scripted blocks using the script{} step in a declarative pipeline:
pipeline {
agent any
stages {
stage('Example') {
steps {
script {
echo 'Hello World'
}
}
}
}
}
A cleaner approach is to define a function, which is scripted by definition and use that as a custom step in declarative pipeline. Note this works even without script{}!
pipeline {
agent any
stages {
stage('Example') {
steps {
myStep 42
}
}
}
}
void myStep( def x ) {
echo "Hello World $x" // prints "Hello World 42"
}
In complex pipelines that use lots of custom code, I usually have one function per stage. This keeps the pipeline{} clean and makes it easy to see the overall structure of the pipeline, without script{} clutter all over the place.
As I understand in the first method I can add Post section that will
be running regardless of the result of the job. I wish to add the same
post section for the second method, but it is not working. Any ideas?
post{} is only available in declarative pipeline, but not within a scripted pipeline or a scripted section of a declarative pipeline. You can use try{} catch{} finally{} instead. catch{} runs only when an error occurs, finally{} always runs. You can use both or either of catch{} and finally{}.
node ("jenkins-nodes") {
stage("git clone"){
echo 'Hello World'
try {
// some steps that may fail
}
catch( Exception e ) {
echo "An error happened: $e"
// Rethrow exception, to let the build fail.
// If you remove "throw", the error would be ignored by Jenkins.
throw
}
finally {
echo "Cleaning up some stuff"
}
}
}
What is modern best practice for multi-configuration builds (with Jenkins)?
I want to support multiple branches and multiple configurations.
For example for each version V1, V2 of the software I want builds targeting
platforms P1 and P2.
We have managed to set up multi-branch declarative pipelines. Each build has its own docker so its easy to support multiple platforms.
pipeline {
agent none
stages {
stage('Build, test and deploy for P1) {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P1.Dockerfile'
}
}
steps {
sh buildit...
}
}
stage('Build, test and deploy for P2) {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P2.Dockerfile'
}
}
steps {
sh buildit...
}
}
}
}
This gives one job covering multiple platforms but there is no separate red/blue status for each platform.
There is good argument that this does not matter as you should not release unless the build works for all platforms.
However, I would like a separate status indicator for each configuration. This suggests I should use a multi-configuration build which triggers a parameterised build for each configuration as below (and the linked question):
pipeline {
parameters {
choice(name: 'Platform',choices: ['P1', 'P2'], description: 'Target OS platform', )
}
agent {
filename someMagicToGetDockerfilePathFromPlatform()
}
stages {
stage('Build, test and deploy for P1) {
steps {
sh buildit...
}
}
}
}
There are several problems with this:
A declarative pipeline has more constraints over how it is scripted
Multi-configuration builds cannot trigger declarative pipelines (even with the parameterized triggers plugin I get "project is not buildable").
This also begs the question what use are parameters in declarative pipelines?
Is there a strategy that gives the best of both worlds i.e:
pipeline as code
separate status indicators
limited repetition?
This is a partial answer. I think others with better experience will be able to improve on it.
This is currently untested. I may be barking up the wrong tree.
Please comment or add a better answer.
Do not use pipeline parameters except where you need user input
Use a hybrid of a scripted and declarative pipeline
(see also https://stackoverflow.com/a/46675227/1569204)
Have a function which declares a pipeline based on parameters:
(see also https://jenkins.io/doc/book/pipeline/shared-libraries/)
Use nodes to create visible indicators in the pipeline (at least in blue ocean)
So something like the following:
def build(string platform) {
switch(platform) {
case P1:
dockerFile = 'foo'
indicator = 'build for foo'
break
case P2:
dockerFile = 'bar'
indicator = 'build for bar'
break
}
pipeline {
agent {
dockerfile {
filename "$dockerFile"
}
node {
label "$indicator"
}
}
stages {
steps {
echo "build it"
}
}
}
}
The relevant code could be moved to a shared library (even if you don't actually need to share it).
I think the cleanest approach is to have this all in a pipeline similar to the first one you presented, the only modification I would see here is making those parallel, so you would actually try and build/test for both platforms.
To reuse the previous stage's workspace you could do: reuseNode true
Something similar to this flow, that would have parallel build for platforms
pipeline {
agent 'docker'
stages {
stage('Common pre') { ... }
stage('Build all platforms') {
parallel {
stage('Build, test and deploy for P1') {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P1.Dockerfile'
reuseNode true
}
}
steps {
sh buildit...
}
}
stage('Build, test and deploy for P2') {
agent {
dockerfile {
filename 'src/main/docker/Jenkins-P2.Dockerfile'
reuseNode true
}
}
steps {
sh buildit...
}
}
}
}
stage('Common post parallel') { ... }
}
}
So I have a use case with Jenkinsfile that I know is not common, and I haven't found a solution for it yet.
Background
We currently have a multi-branch pipeline job configured to build multiple branches. This is uses to run system-testing of the products across multiple release. The Jenkins job
Clone all required repositories
Deploy the environment
Execute the automated test cases
Undeploy the environment
In order to avoid having to define the same Jenkinsfile on each branches, we created a shared library. The shared library defines the Declarative pipeline stages for the Jenkins file. The shared library has the following:
/* File name var/myStep.groovy */
def call(Map pipelineParams) {
callASharedLibraryFunction()
properties([
parameters(sharedLibraryGetParameters(pipelineParams))
])
pipeline {
// snip
stages {
stage("clone repos") { }
stage("Deploy environment") { }
stage("Executed Tests") { }
stage("Undeploy environment") { }
}
// post directives
}
}
And the Jenkins file simply defines a map, and then call myStep call.
e.g.:
/* Sample Jenkinsfile */
pipelineParams = [
FOO = "foo"
]
myStep pipelineParams
The problem
We now have a need for another Jenkins job, where some of the stages will be the same. For example, the new jobs will need to
Clone all required repositories
Deploy the environment
Do something else
And changing the behaviour of a common stage (e.g.: Clone the repo), should take effect across all the jobs that define this stage. I know we can use the when directive in the stage, however from a usability perspective, I want the jobs to be different as they are exercising different things. And the users of one job don't care about the additional stages the other job runs.
I want to avoid code duplication, and better yet, I don't want to duplicate the stage code. (including steps, when, post, etc..).
Is there a way a shared library can define the stage "implementation" with all the directives (steps, when, post, etc) once, but have it get called multiple times?
e.g.:
/* File: vars/cloneReposStageFunction.groovy */
def call() {
stage("Clone Repos") { }
}
/* File: vars/myStep.groovy */
def call(Map pipelineParams) {
pipeline {
// snip
stages {
cloneReposStageFunction()
stage("Deploy environment") { }
stage("Executed Tests") { }
stage("Undeploy environment") { }
}
// post directives
}
}
/* File: vars/myNewStep.groovy */
def call(Map pipelineParams) {
pipeline {
// snip
stages {
cloneReposStageFunction()
stage("Deploy environment") { }
stage("Do something else") { }
}
// post directives
}
}
It's an open Jenkins' Feature Request.
I've seen different ways to template a pipeline, but it's far from what you'd like to achieve.
I have a Jenkinsfile for a multibranch pipeline like this:
pipeline {
agent any
stages {
// ...
}
post {
failure {
mail to: 'team#example.com',
subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
body: "Something is wrong with ${env.BUILD_URL}"
}
}
}
I want to only send email for failures on the master branch. Is there a way to make the mail step conditional? Based on the documentation a when directive may only be used inside a stage.
https://jenkins.io/doc/book/pipeline/syntax/#when
https://jenkins.io/doc/pipeline/tour/post/
like you've noted when only works inside a stage. And only valid steps can be used inside the post conditions.
You can still use scripted syntax inside of a script block, and script blocks are a valid step. So you should be able to use if inside a script block to get the desired behavior.
...
post {
failure {
script {
if (env.BRANCH_NAME == 'master') {
... # your code here
}
}
}
}
}
see JENKINS-52689
I'm using a declarative pipeline. This part works:
steps {
script {
if ('xxx' == 'cloud') {
sh 'xxx'
} else {
sh 'xxx'
}
}
}
But I want to follow the more declarative pipeline syntax using when.
I tried something like this:
stage ('Newman run') {
when {
expression { "xxx" == "cloud" }
}
steps {
echo 'Use proxy for cloud'
sh 'xx'
}
when {
expression { "cloud" == "cloud" }
}
steps {
echo 'Do not use proxy (non-cloud)'
sh 'xxx'
}
But it didn't work. How I can specify a sort of 'else' statement after a when in a declarative pipeline. Or if this not works, the recommended way to do this?
Possible duplicate of How to do simple if-statements inside a declarative pipeline in Jenkins
The first snippet you posted is a good way to go. You could use the 'not' directive in a second stage like this:
stage('then') {
when {
<condition>
}
steps {
...
}
}
stage('else') {
when {
not {
<condition>
}
}
steps {
...
}
}
But that doesn't look better than wrapping it in a 'script' statement, because you would even maintain the condition twice. And it's two separate stages that appear in your pipeline.
So stay with the if-then-else inside a script block.
Rationale: The idea of the declarative pipeline is to keep it simple without complex algorithms in it. When you are tempted to use if statements, loops, etc. in your Jenkinsfile, ask yourself if you are doing the right thing. Using these things if not wrong in general, but it could be that there is a simpler solution for your problem.
Only one when clause is allowed per stage. You need two stages excluding each other by when clauses to achieve the if-else behavior you want.