Jenkinsfile and different strategies for branches - jenkins

I'm trying to use Jenkins file for all our builds in Jenkins, and I have following problem.
We basically have 3 kind of builds:
pull-request build - it will be merged to master after code review, and if build works
manual pull-request build - a build that does the same as above, but can be triggered manually by the user (e.g. in case we have some unstable test)
an initial continuous deliver pipeline - this will build the code, deploy to repository, install artifacts from repository on the target server and start the application there
How should I contain all of the above builds into a single Jenkinsfile.
Right now the only idea I have is to make a giant if that will check which branch it is and will do the steps.
So I have two questions:
1. Is that appropriate way to do it in Jenkinsfile?
How to get the name of currently executing branch in multi-branch job type?
For reference, here's my current Jenkinsfile:
def servers = ['server1', 'server2']
def version = "1.0.0-${env.BUILD_ID}"
stage 'Build, UT, IT'
node {
checkout scm
env.PATH = "${tool 'Maven'}/bin:${env.PATH}"
withEnv(["PATH+MAVEN=${tool 'Maven'}/bin"]) {
sh "mvn -e org.codehaus.mojo:versions-maven-plugin:2.1:set -DnewVersion=$version -DgenerateBackupPoms=false"
sh 'mvn -e clean deploy'
sh 'mvn -e scm:tag'
}
}
def nodes = [:]
for (int i = 0; i < servers.size(); i++) {
def server = servers.get(i)
nodes["$server"] = {
stage "Deploy to INT ($server)"
node {
sshagent(['SOME-ID']) {
sh """
ssh ${server}.example.com <<END
hostname
/apps/stop.sh
yum -y update-to my-app.noarch
/apps/start.sh
END""".stripIndent()
}
}
}
}
parallel nodes
EDIT: removed opinion based question

You can add If statement for multiple stages if you want to skip multiple stages according to the branch as in:
if(env.BRANCH_NAME == 'master'){
stage("Upload"){
// Artifact repository upload steps here
}
stage("Deploy"){
// Deploy steps here
}
}
or, you can add it to individual stage as in:
stage("Deploy"){
if(env.BRANCH_NAME == 'master'){
// Deploy steps here
}
}

Using this post, this worked for me:
stage('...') {
when {
expression { env.BRANCH_NAME == 'master' }
}
steps {
...
}
}

1) I don't know if it is appropriate, but if it resolves your problem, I think is appropriate enough.
2) In order to know the name of the branch you can use BRANCH_NAME variable, its name is taken from the branch name.
${env.BRANCH_NAME}
Here is the answer:
Jenkins Multibranch pipeline: What is the branch name variable?

We followed the model used by fabric8 for builds, tweaking it as we needed, where the Jenkinsfile is used to define the branch and deployment handling logic, and a release.groovy file for build logic.
Here's what our Jenkinsfile looks like for a pipeline that continuously deploys into DEV from master branch:
#!groovy
import com.terradatum.jenkins.workflow.*
node {
wrap([$class: 'TimestamperBuildWrapper']) {
checkout scm
echo "branch: ${env.BRANCH_NAME}"
def pipeline = load "${pwd()}/release.groovy"
if (env.DEPLOY_ENV != null) {
if (env.DEPLOY_ENV.trim() == 'STAGE') {
setDisplayName(pipeline.staging() as Version)
} else if (env.DEPLOY_ENV.trim() == 'PROD') {
setDisplayName(pipeline.production() as Version)
}
} else if (env.BRANCH_NAME == 'master') {
try {
setDisplayName(pipeline.development() as Version)
} catch (Exception e) {
hipchatSend color: 'RED', failOnError: true, message: "<p>BUILD FAILED: </p><p>Check console output at <a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a></p><p><pre>${e.message}</pre></p>", notify: true, room: 'Aergo', v2enabled: false
throw e; // rethrow so the build is considered failed
}
} else {
setDisplayName(pipeline.other() as Version)
}
}
}
def setDisplayName(Version version) {
if (version) {
currentBuild.displayName = version.toString()
}
}
Note: you can find the code for our global pipeline library here.

Don't know if this what you want..
I prefer because it's look more structured.
Jenkinsfile
node {
def rootDir = pwd()
def branchName = ${env.BRANCH_NAME}
// Workaround for pipeline (not multibranches pipeline)
def branchName = getCurrentBranch()
echo 'BRANCH.. ' + branchName
load "${rootDir}#script/Jenkinsfile.${branchName}.Groovy"
}
def getCurrentBranch () {
return sh (
script: 'git rev-parse --abbrev-ref HEAD',
returnStdout: true
).trim()
}
Jenkinsfile.mybranch.Groovy
echo 'mybranch'
// Pipeline code here

for questions 2 you may be able to do
sh 'git branch > GIT_BRANCH'
def gitBranch = readFile 'GIT_BRANCH'
since you're checking out from git

In my scenarium, I have needed run a stage Deploy Artifactory only if the branch was master(webhook Gitlab), otherwise I couldn't perform the deploy.
Below the code of my jenkinsfile:
stages {
stage('Download'){
when{
environment name: 'gitlabSourceBranch', value: 'master'
}
steps{
echo "### Deploy Artifactory ###"
}
}
}

Related

Build stages in Jenkins only when specific files are changed but use a function

I want to update my Jenkins pipeline in way that certain stages are only build when some specific files are changed (git is already integrated in the pipeline). I found a promising solution on this site, which would go like this for my use case (this code run successful):
stage("TEST STAGE 1") {
when {
anyOf { changeset "*dir1/*"; changeset "*dir2/*"; changeset "*somefile" }
}
steps {
// Do stuff
}
}
But I have more stages (TEST STAGE 2 and TEST STAGE 3) which should also be triggered only when these files are changed. To avoid writing the same code over and over (which would be bad practice), I implemented a function (I got the code from here):
def runStage(changeset) {
return {
changeset ==~ ("*dir1/*"||"*dir2/*"||"*somefile")
}
}
I call this function in the TEST stages (also TEST 2 and 3):
stage("TEST STAGE 1") {
when {
expression{ runStage(changeset) }
}
steps {
// Do stuff
}
}
But now my pipeline fails when entering the first TEST stage. I get this error:
hudson.remoting.ProxyException: groovy.lang.MissingPropertyException: No such property: changeset for class: WorkflowScript
Do you have an idea what I am doing wrong?
I found a solution.
This is my function:
def runStage() {
CHANGE_SET = sh (
script: 'git log -2 --name-only --oneline --pretty="format:"',
returnStdout: true
).trim()
echo "Current changeset: ${CHANGE_SET}"
return (CHANGE_SET ==~ "(.*)dir1(.*)|(.*)dir2(.*)|(.*)somefile")
}
I call it in my pipeline stage like this:
stage("TEST STAGE 1") {
when {
expression { runStage() }
}
steps {
//stuff
}
}
I would have prefered using changeset in the when block instead of git log, but it looks like it can't be done for my case.

JenkinsFile, aproaching to enviroment dependent

I'm new on Jenkins and I don't know how to approach to a multi-environment jenkinsfile.
Note: I'm using multibranch pipeline
Firstly I thought about writing 3 different files of Jenkins, depending of the branch. But, I think there must be another way to do it with only 1 Jenkinsfile which could execute different process depending of the branch you are working on.
Something like:
if branch == 'master' then
procces to do
else if branch == 'test' then
other proccess
else
process for developer branch
edit:
I found that there's the possibility of using env variables. I tried:
echo "$BRANCH_NAME"
echo "$env.BRANCH_NAME"
echo 'BRANCH NAME: ' + env.BRANCH_NAME
echo ${env.BRANCH_NAME}
echo "${env.BRANCH_NAME}"
...
but nothing works.
In Jenkins you use when clauses to define when stages should be run
pipeline {
agent any
stages {
stage("Release if Required") {
when {
anyOf {
branch 'master'
branch 'main'
}
}
steps {
script {
publishRelease()
}
}
}
}
}
If you need to do the check while in the steps you can use the environment vars
script {
if(env.BRANCH_NAME =~ 'master' || env.BRANCH_NAME =~ 'main') {
echo "On the main branch"
}
}
Problem was not using it on script { section.
stage('Checkout code') {
script {
echo "${env.BRANCH_NAME}"
if ("master" == env.BRANCH_NAME) {
do something
...
} else if ("Test" == env.BRANCH_NAME) {
do something
...
} else {
do something
...
}
}

Is it possible to check if checked out from a specific repository in a Jenkins declarative pipeline?

I would like to have a release stage in my Jenkinsfile that only runs when it's checked out from the original repository. This is to avoid error messages on cloned repositories, because of missing keys etc. there.
stage('Release')
{
when
{
allOf
{
// TODO Check for repository url https://github.com/PowerStat/TemplateEngine.git
branch 'master'
}
}
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode release:clean'
sh 'mvn --batch-mode release:prepare'
sh 'mvn --batch-mode release:perform'
}
else
{
bat 'mvn --batch-mode release:clean'
bat 'mvn --batch-mode release:prepare'
bat 'mvn --batch-mode release:perform'
}
}
}
}
I have studied Pipeline Syntax: when but have no idea how to do the test I would like to have.
Also I thought about using an environment variable Global Variable Reference, but found non with the repository URL in it.
So my question is: how to implement this check in a decalarative pipeline?
You can get remote repository URL from git config remote.origin.url command. You can execute this command using expression directive inside the when block - it defines a closure that returns a boolean value.
Consider the following example:
def expectedRemoteUrl = "https://github.com/PowerStat/TemplateEngine.git"
pipeline {
agent any
stages {
stage("Release") {
when {
allOf {
branch 'tmp'
expression {
def remoteUrl = isUnix() ?
sh(script: "git config remote.origin.url", returnStdout: true)?.trim() :
bat(script: "git config remote.origin.url", returnStdout: true)?.trim()
return expectedRemoteUrl == remoteUrl
}
}
}
steps {
echo "Do your release steps here..."
}
}
}
}
Alternatively, if git command is not available in the node that runs the pipeline, you can get the remote repository URL with scm.userRemoteConfigs?.first()?.url. Consider the following example:
def expectedRemoteUrl = "https://github.com/PowerStat/TemplateEngine.git"
pipeline {
agent any
stages {
stage("Release") {
when {
allOf {
branch 'tmp'
expression {
def remoteUrl = scm.userRemoteConfigs?.first()?.url
return expectedRemoteUrl == remoteUrl
}
}
}
steps {
echo "Do your release steps here..."
}
}
}
}

Declarative Jenkins Pipeline; How to declare a variable and use it in script or mail notification?

(update below)
I have a declarative pipeline job which can take an argument VERSION.
pipeline {
parameters {
string(name: VERSION, defaultValue: '')
}
// ...
}
If no VERSION is given, like when Gitlab send a hook to this job, I want to compute it from git, so I do something like this
stages {
stage('Prepare') {
steps {
// ...
if (! env.VERSION) {
VERSION = sh(script: "git describe", returnStdout: true).trim()
}
}
}
}
Now I want to "inject" this variable to
my build script. It needs to find "VERSION" in the environment variables
to the jenkins mail notificator. And get it to retreive ${VERSION} in subject or body text
I tried changing above code with
stages {
stage('Prepare') {
steps {
// ...
if (! env.VERSION) {
env.VERSION = sh(script: "git describe", returnStdout: true).trim()
}
}
}
}
Got this error groovy.lang.MissingPropertyException: No such property: VERSION for class: groovy.lang.Binding
I then tried to add a "environment" step below
environment {
VERSION = ${VERSION}
}
but it didn't solve my problem.
I'm looking for any help to solve it.
UPDATE
I now have a working pipeline which looks like
pipeline {
agent any
parameters {
string(name: 'VERSION', defaultValue: '')
}
environment {
def VERSION = "${params.VERSION}"
}
stages {
stage('Prepare & Checkout') {
steps {
script {
if (! env.VERSION) {
VERSION = sh(script: "date", returnStdout: true).trim()
}
echo "** version: ${VERSION} **"
}
}
}
stage('Build') {
steps {
// sh "./build.sh"
echo "** version2: ${VERSION} **"
}
}
} // stages
post {
always {
mail to: 'foo#example.com',
subject: "SUCCESS: ${VERSION}",
body: """<html><body><p>SUCCESS</p></body></html>""",
mimeType: 'text/html',
charset: 'UTF-8'
deleteDir()
}
}
} // pipeline
I needed to add the "environment" step to be able to get $VERSION in all Stages (not only in the one it is manipulated).
I still need to find a way to inject this $VERSION variable in the environment variables, so that my build script can find it.
If you want to inject the variable in the environment so that you can use it later, you could define another variable that is equal to env.VERSION or the output of the shell scrip. Then use that variable in your pipeline eg:
pipeline {
parameters {
string(name: VERSION, defaultValue: '')
}
def version = env.VERSION
stages {
stage('Prepare') {
steps {
// ...
if (!version) {
version = sh(script: "git describe", returnStdout: true).trim()
}
}
}
mail subject: "$version build succeeded", ...
}
If you want other jobs to be able to access the value of VERSION after the build is run, you can write it in a file and archive it.
Edit:
In order for your script to be able to use the version variable, you can either make your script take version as a parameter or you can use the withEnv step.
Assuming you are using Parametrized pipelines, you should call variable as ${params.parameterName}
Although parameters are available in env they currently are created before the first time the pipeline is run, therefore you should access them via params:
In your case:
${params.VERSION}

Declarative pipeline when condition in post

As far as declarative pipelines go in Jenkins, I'm having trouble with the when keyword.
I keep getting the error No such DSL method 'when' found among steps. I'm sort of new to Jenkins 2 declarative pipelines and don't think I am mixing up scripted pipelines with declarative ones.
The goal of this pipeline is to run mvn deploy after a successful Sonar run and send out mail notifications of a failure or success. I only want the artifacts to be deployed when on master or a release branch.
The part I'm having difficulties with is in the post section. The Notifications stage is working great. Note that I got this to work without the when clause, but really need it or an equivalent.
pipeline {
agent any
tools {
maven 'M3'
jdk 'JDK8'
}
stages {
stage('Notifications') {
steps {
sh 'mkdir tmpPom'
sh 'mv pom.xml tmpPom/pom.xml'
checkout([$class: 'GitSCM', branches: [[name: 'origin/master']], doGenerateSubmoduleConfigurations: false, submoduleCfg: [], userRemoteConfigs: [[url: 'https://repository.git']]])
sh 'mvn clean test'
sh 'rm pom.xml'
sh 'mv tmpPom/pom.xml ../pom.xml'
}
}
}
post {
success {
script {
currentBuild.result = 'SUCCESS'
}
when {
branch 'master|release/*'
}
steps {
sh 'mvn deploy'
}
sendNotification(recipients,
null,
'https://link.to.sonar',
currentBuild.result,
)
}
failure {
script {
currentBuild.result = 'FAILURE'
}
sendNotification(recipients,
null,
'https://link.to.sonar',
currentBuild.result
)
}
}
}
In the documentation of declarative pipelines, it's mentioned that you can't use when in the post block. when is allowed only inside a stage directive.
So what you can do is test the conditions using an if in a script:
post {
success {
script {
if (env.BRANCH_NAME == 'master')
currentBuild.result = 'SUCCESS'
}
}
// failure block
}
Using a GitHub Repository and the Pipeline plugin I have something along these lines:
pipeline {
agent any
stages {
stage('Build') {
steps {
sh '''
make
'''
}
}
}
post {
always {
sh '''
make clean
'''
}
success {
script {
if (env.BRANCH_NAME == 'master') {
emailext (
to: 'engineers#green-planet.com',
subject: "${env.JOB_NAME} #${env.BUILD_NUMBER} master is fine",
body: "The master build is happy.\n\nConsole: ${env.BUILD_URL}.\n\n",
attachLog: true,
)
} else if (env.BRANCH_NAME.startsWith('PR')) {
// also send email to tell people their PR status
} else {
// this is some other branch
}
}
}
}
}
And that way, notifications can be sent based on the type of branch being built. See the pipeline model definition and also the global variable reference available on your server at http://your-jenkins-ip:8080/pipeline-syntax/globals#env for details.
Ran into the same issue with post. Worked around it by annotating the variable with #groovy.transform.Field. This was based on info I found in the Jenkins docs for defining global variables.
e.g.
#!groovy
pipeline {
agent none
stages {
stage("Validate") {
parallel {
stage("Ubuntu") {
agent {
label "TEST_MACHINE"
}
steps {{
sh "run tests command"
recordFailures('Ubuntu', 'test-results.xml')
junit 'test-results.xml'
}
}
}
}
}
post {
unsuccessful {
notify()
}
}
}
// Make testFailures global so it can be accessed from a 'post' step
#groovy.transform.Field
def testFailures = [:]
def recordFailures(key, resultsFile) {
def failures = ... parse test-results.xml script for failures ...
if (failures) {
testFailures[key] = failures
}
}
def notify() {
if (testFailures) {
... do something here ...
}
}

Resources