I want to update my Jenkins pipeline in way that certain stages are only build when some specific files are changed (git is already integrated in the pipeline). I found a promising solution on this site, which would go like this for my use case (this code run successful):
stage("TEST STAGE 1") {
when {
anyOf { changeset "*dir1/*"; changeset "*dir2/*"; changeset "*somefile" }
}
steps {
// Do stuff
}
}
But I have more stages (TEST STAGE 2 and TEST STAGE 3) which should also be triggered only when these files are changed. To avoid writing the same code over and over (which would be bad practice), I implemented a function (I got the code from here):
def runStage(changeset) {
return {
changeset ==~ ("*dir1/*"||"*dir2/*"||"*somefile")
}
}
I call this function in the TEST stages (also TEST 2 and 3):
stage("TEST STAGE 1") {
when {
expression{ runStage(changeset) }
}
steps {
// Do stuff
}
}
But now my pipeline fails when entering the first TEST stage. I get this error:
hudson.remoting.ProxyException: groovy.lang.MissingPropertyException: No such property: changeset for class: WorkflowScript
Do you have an idea what I am doing wrong?
I found a solution.
This is my function:
def runStage() {
CHANGE_SET = sh (
script: 'git log -2 --name-only --oneline --pretty="format:"',
returnStdout: true
).trim()
echo "Current changeset: ${CHANGE_SET}"
return (CHANGE_SET ==~ "(.*)dir1(.*)|(.*)dir2(.*)|(.*)somefile")
}
I call it in my pipeline stage like this:
stage("TEST STAGE 1") {
when {
expression { runStage() }
}
steps {
//stuff
}
}
I would have prefered using changeset in the when block instead of git log, but it looks like it can't be done for my case.
Related
This looks like very basic question about Jenkins usage.
I have Jenkinsfile located in root folder of my Subversion repository tree. There are many branches (versions/tags) of the product - everywhere is the same Jenkinsfile. So far very basic setup, I suppose.
I need to provide some steps with current Subversion repository branch/url.
There are some similar questions like this or this, but none is working solution for Subversion.
pipeline {
agent { label 'master' }
stages {
stage("test") {
steps {
echo "Start pipeline "
// commented-out = not working
//echo scm.getUserRemoteConfigs()
//echo scm
script {
println "Current svn url/branch: "//??? + scm.getUserRemoteConfigs()[0].getUrl()
}
}
}
}
}
It will be like this
pipeline {
agent any;
stages {
stage('test'){
steps {
script {
def s = checkout scm;
if (s.GIT_URL != null) {
print s.GIT_URL
print s.GIT_BRANCH
print s.GIT_COMMIT
}
else if (s.SVN_URL != null) {
print s.SVN_REVISION
print s.SVN_REVISION_1
print s.SVN_URL
print s.SVN_URL_1
}
}
}
}
}
}
Note- This works fine with GIT and SVN, but a bit differently.
In a pipeline script of mine, I have the following:
pipeline {
agent any
options {
skipDefaultCheckout(true)
}
stages {
stage('Checkout') {
//Checkout steps
}
stage('Clean') {
when {
expression {
def result = input(message: 'Clean?', ok: 'Go for it!')
return true
}
beforeAgent true
}
parallel {
stage("Clean Project 1") {
// Clean steps
}
stage("Clean Project 2") {
// Clean steps
}
}
}
}
}
I'm expecting there to be a waiting step before the parallel stage "Clean" begins to run however the pipeline ends up looking like this:
I have even tried to add another step between checkout and clean with just an input as a step but that hasn't worked either. I could put the input st the end of the checkout steps but that is not desired.
As far as declarative pipelines go in Jenkins, I'm having trouble with the when keyword.
I keep getting the error No such DSL method 'when' found among steps. I'm sort of new to Jenkins 2 declarative pipelines and don't think I am mixing up scripted pipelines with declarative ones.
The goal of this pipeline is to run mvn deploy after a successful Sonar run and send out mail notifications of a failure or success. I only want the artifacts to be deployed when on master or a release branch.
The part I'm having difficulties with is in the post section. The Notifications stage is working great. Note that I got this to work without the when clause, but really need it or an equivalent.
pipeline {
agent any
tools {
maven 'M3'
jdk 'JDK8'
}
stages {
stage('Notifications') {
steps {
sh 'mkdir tmpPom'
sh 'mv pom.xml tmpPom/pom.xml'
checkout([$class: 'GitSCM', branches: [[name: 'origin/master']], doGenerateSubmoduleConfigurations: false, submoduleCfg: [], userRemoteConfigs: [[url: 'https://repository.git']]])
sh 'mvn clean test'
sh 'rm pom.xml'
sh 'mv tmpPom/pom.xml ../pom.xml'
}
}
}
post {
success {
script {
currentBuild.result = 'SUCCESS'
}
when {
branch 'master|release/*'
}
steps {
sh 'mvn deploy'
}
sendNotification(recipients,
null,
'https://link.to.sonar',
currentBuild.result,
)
}
failure {
script {
currentBuild.result = 'FAILURE'
}
sendNotification(recipients,
null,
'https://link.to.sonar',
currentBuild.result
)
}
}
}
In the documentation of declarative pipelines, it's mentioned that you can't use when in the post block. when is allowed only inside a stage directive.
So what you can do is test the conditions using an if in a script:
post {
success {
script {
if (env.BRANCH_NAME == 'master')
currentBuild.result = 'SUCCESS'
}
}
// failure block
}
Using a GitHub Repository and the Pipeline plugin I have something along these lines:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
make
'''
}
}
}
post {
always {
sh '''
make clean
'''
}
success {
script {
if (env.BRANCH_NAME == 'master') {
emailext (
to: 'engineers#green-planet.com',
subject: "${env.JOB_NAME} #${env.BUILD_NUMBER} master is fine",
body: "The master build is happy.\n\nConsole: ${env.BUILD_URL}.\n\n",
attachLog: true,
)
} else if (env.BRANCH_NAME.startsWith('PR')) {
// also send email to tell people their PR status
} else {
// this is some other branch
}
}
}
}
}
And that way, notifications can be sent based on the type of branch being built. See the pipeline model definition and also the global variable reference available on your server at http://your-jenkins-ip:8080/pipeline-syntax/globals#env for details.
Ran into the same issue with post. Worked around it by annotating the variable with #groovy.transform.Field. This was based on info I found in the Jenkins docs for defining global variables.
e.g.
#!groovy
pipeline {
agent none
stages {
stage("Validate") {
parallel {
stage("Ubuntu") {
agent {
label "TEST_MACHINE"
}
steps {{
sh "run tests command"
recordFailures('Ubuntu', 'test-results.xml')
junit 'test-results.xml'
}
}
}
}
}
post {
unsuccessful {
notify()
}
}
}
// Make testFailures global so it can be accessed from a 'post' step
#groovy.transform.Field
def testFailures = [:]
def recordFailures(key, resultsFile) {
def failures = ... parse test-results.xml script for failures ...
if (failures) {
testFailures[key] = failures
}
}
def notify() {
if (testFailures) {
... do something here ...
}
}
I am creating a sample jenkins pipeline, here is the code.
pipeline {
agent any
stages {
stage('test') {
steps {
sh 'echo hello'
}
}
stage('test1') {
steps {
sh 'echo $TEST'
}
}
stage('test3') {
if (env.BRANCH_NAME == 'master') {
echo 'I only execute on the master branch'
} else {
echo 'I execute elsewhere'
}
}
}
}
this pipeline fails with following error logs
Started by user admin
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 15: Not a valid stage section definition: "if (env.BRANCH_NAME == 'master') {
echo 'I only execute on the master branch'
} else {
echo 'I execute elsewhere'
}". Some extra configuration is required. # line 15, column 9.
stage('test3') {
^
WorkflowScript: 15: Nothing to execute within stage "test3" # line 15, column 9.
stage('test3') {
^
But when i execute the following example from this url, it executes successfully and print the else part.
node {
stage('Example') {
if (env.BRANCH_NAME == 'master') {
echo 'I only execute on the master branch'
} else {
echo 'I execute elsewhere'
}
}
}
The only difference i can see is that in the working example there is no stages but in my case it has.
What is wrong here, can anyone please suggest?
your first try is using declarative pipelines, and the second working one is using scripted pipelines. you need to enclose steps in a steps declaration, and you can't use if as a top-level step in declarative, so you need to wrap it in a script step. here's a working declarative version:
pipeline {
agent any
stages {
stage('test') {
steps {
sh 'echo hello'
}
}
stage('test1') {
steps {
sh 'echo $TEST'
}
}
stage('test3') {
steps {
script {
if (env.BRANCH_NAME == 'master') {
echo 'I only execute on the master branch'
} else {
echo 'I execute elsewhere'
}
}
}
}
}
}
you can simplify this and potentially avoid the if statement (as long as you don't need the else) by using "when". See "when directive" at https://jenkins.io/doc/book/pipeline/syntax/. you can also validate jenkinsfiles using the jenkins rest api. it's super sweet. have fun with declarative pipelines in jenkins!
It requires a bit of rearranging, but when does a good job to replace conditionals above. Here's the example from above written using the declarative syntax. Note that test3 stage is now two different stages. One that runs on the master branch and one that runs on anything else.
stage ('Test 3: Master') {
when { branch 'master' }
steps {
echo 'I only execute on the master branch.'
}
}
stage ('Test 3: Dev') {
when { not { branch 'master' } }
steps {
echo 'I execute on non-master branches.'
}
}
If you wanted to create a condition to execute only a stage based on expression you can use keyword when
stage ('test3'){
when { expression { return env.BRANCH_NAME == 'master'} }
steps {
echo 'I only execute on the master branch.'
}
}
}
With the expression key word you can add any condition.
e.g. if stage is dependent on generated file in workspace.
stage ('File Dependent stage'){
when { expression { return fileExists ('myfile') } }
steps {
echo "file exists"
}
}
}
if ( params.build_deploy == '1' ) {
println "build_deploy 是 ${params.build_deploy}"
jobB = build job: 'k8s-core-user_deploy', propagate: false, wait: true, parameters: [
string(name:'environment', value: "${params.environment}"),
string(name:'branch_name', value: "${params.branch_name}"),
string(name:'service_name', value: "${params.service_name}"),
]
println jobB.getResult()
}
I am looking for a Jenkinsfile example of having a step that is always executed, even if a previous step failed.
I want to assure that I archive some builds results in case of failure and I need to be able to have an always-running step at the end.
How can I achieve this?
We switched to using Jenkinsfile Declarative Pipelines, which lets us do things like this:
pipeline {
agent any
stages {
stage('Test') {
steps {
sh './gradlew check'
}
}
}
post {
always {
junit 'build/reports/**/*.xml'
}
}
}
References:
Tests and Artifacts
Jenkins Pipeline Syntax
try {
sh "false"
} finally {
stage 'finalize'
echo "I will always run!"
}
Another possibility is to use a parallel section in combination with a lock. For example:
pipeline {
stages {
parallel {
stage('Stage 1') {
steps {
lock('MY_LOCK') {
echo 'do stuff 1'
}
}
}
stage('Stage 2') {
steps {
lock('MY_LOCK') {
echo 'do stuff 2'
}
}
}
}
}
}
Parallel stages in a parallel section only abort other stages in the same parallel section if the fail fast option for the parallel section is set. See the docs.