We are using a Jenkins multibranch pipeline with BitBucket to build pull request branches as part of our code review process.
We wanted to abort any queued or in-progress builds so that we only run and keep the latest build - I created a function for this:
def call(){
def jobName = env.JOB_NAME
def buildNumber = env.BUILD_NUMBER.toInteger()
def currentJob = Jenkins.instance.getItemByFullName(jobName)
for (def build : currentJob.builds){
def exec = build.getExecutor()
if(build.isBuilding() && build.number.toInteger() != buildNumber && exec != null){
exec.interrupt(
Result.ABORTED,
new CauseOfInterruption.UserInterruption("Job aborted by #${currentBuild.number}")
)
println("Job aborted previously running build #${build.number}")
}
}
}
Now in my pipeline, I want to run this function when the build is triggered by the creation or push to a PR branch.
It seems the only way I can do this is to set the agent to none and then set it to the correct node for each of the subsequent stages. This results in missing environment variables etc. since the 'environment' section runs on the master.
If I use agent { label 'mybuildnode' } then it won't run the stage until the agent is free from the currently in-progress/running build.
Is there a way I can get the 'cancelpreviousbuilds()' function to run before the agent is allocated as part of my pipeline?
This is roughly what I have currently but doesn't work because of environment variable issues - I had to do the skipDefaultCheckout so I could do it manually as part of my build:
pipeline {
agent none
environment {
VER = '1.2'
FULLVER = "${VER}.${BUILD_NUMBER}.0"
PROJECTPATH = "<project path>"
TOOLVER = "2017"
}
options {
skipDefaultCheckout true
}
stages {
stage('Check Builds') {
when {
branch 'PR-*'
}
steps {
// abort any queued/in-progress builds for PR branches
cancelpreviousbuilds()
}
}
stage('Checkout') {
agent {
label 'buildnode'
}
steps {
checkout scm
}
}
}
}
It works and aborts the build successfully but I get errors related to missing environment variables because I had to start the pipeline with agent none and then set each stage to agent label 'buildnode' to.
I would prefer that my entire pipeline ran on the correct agent so that workspaces / environment variables were set correctly but I need a way to trigger the cancelpreviousbuilds() function without requiring the buildnode agent to be allocated first.
You can try combining the declarative pipeline and the scripted pipeline, which is possible.
Example (note I haven't tested it):
// this is scripted pipeline
node('master') { // use whatever node name or label you want
stage('Cancel older builds') {
cancel_old_builds()
}
}
// this is declarative pipeline
pipeline {
agent { label 'buildnode' }
...
}
As a small side comment, you seem to use: build.number.toInteger() != buildNumber which would abort not only older builds but also newer ones. In our CI setup, we've decided that it's best to abort the current build if a newer build has already been scheduled.
I have jenkins pipeline and also using a shared library for jenkins.
In my multibranch pipeline three to four repo clone while executing build using bitbucket plugin.
my question is how to get the last build revision from the previous build.
I have tried currentBuild.changeSets approach but for multiple repositories clone, it fails.
I had to get SCM revisions from the previous builds too. I didn't find any API to get it nicely, so I implemented a workaround. It is not great, but at least it works ;-)
When you save an environment variable by using env.setProperty(name, value) it is saved in the build metadata as a build variable. You can read it at any moment.
pipeline {
agent any
stages {
stage('Test') {
script {
env.setProperty('MY_ENV', env.BUILD_NUMBER)
def previousBuild = currentBuild.previousBuild
if (previousBuild != null) {
echo previousBuild.buildVariables['MY_ENV'] // prints env.BUILD_NUMBER - 1
}
}
}
}
}
In your case you have 4 checkouts. I don't know how you close sources, so let's imagine that you have a cloneRepo method and it sets the GIT_COMMIT environment variable. They you may use:
def previousBuild = currentBuild.previousBuild
if (previousBuild != null) {
echo previousBuild.buildVariables['GIT_COMMIT_REPO_1']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_2']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_3']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_4']
}
cloneRepo(repo1)
env.setProperty('GIT_COMMIT_REPO_1', env.GIT_COMMIT)
cloneRepo(repo2)
env.setProperty('GIT_COMMIT_REPO_2', env.GIT_COMMIT)
cloneRepo(repo3)
env.setProperty('GIT_COMMIT_REPO_3', env.GIT_COMMIT)
cloneRepo(repo4)
env.setProperty('GIT_COMMIT_REPO_4', env.GIT_COMMIT)
If you use the checkout step, then you may do:
def commitId = checkout(scm).find { it.key == 'GIT_COMMIT' }
env.setProperty('GIT_COMMIT_REPO_1', commitId)
I am trying to print/list all our jenkins (Freestyle and Pipeline) Jobs separately along with SCM Details such as (Git URL & Branch details) using below groovy. I am able to list our freestyle & scripted pipeline jobs names separately.
import jenkins.model.*
import hudson.model.*
import hudson.triggers.*
import org.jenkinsci.plugins.workflow.job.*
println("--- Jenkins Pipeline jobs List ---")
Jenkins.getInstance().getAllItems(WorkflowJob.class).each() { println(it.fullName) };
println("\n--- Jenkins FreeStyle jobs List ---")
Jenkins.getInstance().getAllItems(FreeStyleProject.class).each() { println(it.fullName) };
println '\nDone.'
With the below groovy code i can able to print the both freestyle & pipeline Git URLs, But it is printing separately.
Jenkins.instance.getAllItems(hudson.model.AbstractProject.class).each {it ->
scm = it.getScm()
if(scm instanceof hudson.plugins.git.GitSCM)
{
println scm.getUserRemoteConfigs()[0].getUrl()
}
}
println "Done"
Need help in listing/printing both Job name & Git URL along with respective Job.
If I run this code in Jenkins Script console,i see in output DSL scripted pipeline (Jenkinsfile) jobs as well
import org.jenkinsci.plugins.workflow.job.WorkflowJob;
def printScm(project, scm){
if (scm instanceof hudson.plugins.git.GitSCM) {
scm.getRepositories().each {
it.getURIs().each {
println(project + "\t"+ it.toString());
}
}
}
}
Jenkins.instance.getAllItems(Job.class).each {
project = it.getFullName()
if (it instanceof AbstractProject){
printScm(project, it.getScm())
} else if (it instanceof WorkflowJob) {
it.getSCMs().each {
printScm(project, it)
}
} else {
println("project type unknown: " + it)
}
}
Using a single declarative pipeline (not multibranch pipeline)
Is there a way I can trigger a certain stage only if its the Master Branch ?
I've been unsuccessful with the following:
Stage('Deploy') {
steps {
script {
if (env.BRANCH_ENV == 'master') {
sh "mvn deploy"
} else {
echo 'Ignoring'
}
}
}
}
No matter what branch i'm deploying, everything gets ignored
any help or advice would be great
I had the same issue before and figured that env.BRANCH_ENV does not return what I expected. You can echo env.BRANCH_ENV in your pipeline to confirm.
My solution to this was to get the git branch manually:
scmVars = checkout scm
gitBranch = sh(
script: "echo ${scmVars.GIT_BRANCH} | cut -d '/' -f2",
returnStdout: true
).trim()
Here some approaches:
use return command to finalize prematurely the stage
https://stackoverflow.com/a/51406870/3957754
use when directive
when directive allows the Pipeline to determine whether the stage should be executed depending on the given condition
built-in conditions: branch, expression, allOf, anyOf, not etc.
when {
// Execute the stage when the specified Groovy expression evaluates to true
expression {
return params.ENVIRONMENT ==~ /(?i)(STG|PRD)/
}
}
Complete sample :
https://gist.github.com/HarshadRanganathan/97feed7f91b7ae542c994393447f3db4
I delete old jenkins builds with rm where job is hosted:
my_job/builds/$ rm -rf [1-9]*
These old builds are still visible in job page.
How to remove them with command line?
(without the delete button in each build user interface)
Here is another option: delete the builds remotely with cURL. (Replace the beginning of the URLs with whatever you use to access Jenkins with your browser.)
$ curl -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The above deletes build #1 to #56 for job myJob.
If authentication is enabled on the Jenkins instance, a user name and API token must be provided like this:
$ curl -u userName:apiToken -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The API token must be fetched from the /me/configure page in Jenkins. Just click on the "Show API Token..." button to display both the user name and the API token.
Edit: As pointed out by yegeniy in a comment below, one might have to replace doDeleteAll by doDelete in the URLs above to make this work, depending on the configuration.
It looks like this has been added to the CLI, or is at least being worked on: http://jenkins.361315.n4.nabble.com/How-to-purge-old-builds-td385290.html
Syntax would be something like this: java -jar jenkins-cli.jar -s http://my.jenkins.host delete-builds myproject '1-7499' --username $user --password $password
Check your home jenkins directory:
"Manage Jenkins" ==> "Configure System"
Check field "Home directory" (usually it is /var/lib/jenkins)
Command for delete all jenkins job builds
/jenkins_home/jobs> rm -rf */builds/*
After delete should reload config:
"Manage Jenkins" ==> "Reload Configuration from Disk"
You can do it by Groovy Scripts using Hudson API.. Access your jenkins instalation
http://localhost:38080/script.
For Example, for deleting all old builds of all projects using the follow script:
Note: Take care if you use Finger Prints , you will lose all history.
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
System.out.println("Deleting all builds of job "+item.name)
for(build in item.getBuilds()){
build.delete()
}
}
else {
System.out.println("Skipping job "+item.name+", currently building")
}
}
Or for cleaning all workspaces :
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
println("Wiping out workspace of job "+item.name)
item.doDoWipeOutWorkspace()
}
else {
println("Skipping job "+item.name+", currently building")
}
}
There are a lot of examples on the Jenkins wiki
Is there a reason you need to do this manually instead of letting Jenkins delete old builds for you?
You can change your job configuration to automatically delete old builds, based either on number of days or number of builds. No more worrying about it or having to keep track, Jenkins just does it for you.
The following script cleans old builds of jobs. You should reload config from disk if you delete build manually:
import hudson.model.*
for(item in Hudson.instance.items) {
if (!item.isBuilding()) {
println("Deleting old builds of job " + item.name)
for (build in item.getBuilds()) {
//delete all except the last
if (build.getNumber() < item.getLastBuild().getNumber()) {
println "delete " + build
try {
build.delete()
} catch (Exception e) {
println e
}
}
}
} else {
println("Skipping job " + item.name + ", currently building")
}
}
From Script Console Run this, but you need to change the job name:
def jobName = "name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
From Jenkins Scriptler console run the following Groovy script to delete all the builds of jobs listed under a view:
import jenkins.model.Jenkins
hudson.model.Hudson.instance.getView('<ViewName>').items.each() {
println it.fullDisplayName
def jobname = it.fullDisplayName
def item = hudson.model.Hudson.instance.getItem(jobname)
def build = item.getLastBuild()
if (item.getLastBuild() != null) {
Jenkins.instance.getItemByFullName(jobname).builds.findAll {
it.number <= build.getNumber()
}.each {
it.delete()
}
}
}
def jobName = "MY_JOB_NAME"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().findAll { it.number < 10 }.each { it.delete() }
if you had 12 builds this would clear out builds 0-9 and you'd have 12,11,10 remaining. Just drop in the script console
This script will configure the build retention settings of all of the Jenkins jobs.
Change the values from 30 and 200 to suite you needs, run the script, then restart the Jenkins service.
#!/bin/bash
cd $HOME
for xml in $(find jobs -name config.xml)
do
sed -i 's#<daysToKeep>.*#<daysToKeep>30</daysToKeep>#' $xml
sed -i 's#<numToKeep>.*#<numToKeep>200</numToKeep>#' $xml
done
The script below works well with Folders and Multibranch Pipelines. It preserves only 10 last builds for each job. That could be adjusted or removed (proper if) if needed. Run that from web script console (example URL: https://jenkins.company.com/script)
def jobs = Hudson.instance.getAllItems(hudson.model.Job.class)
for (job in jobs){
println(job)
def recent = job.builds.limit(10)
for(build in job.builds){
if(!recent.contains(build)){
println("\t Deleting build: " + build)
build.delete()
}
}
}
From my opinion all those answers are not sufficient, you have to do:
echo "Cleaning:"
echo "${params.PL_JOB_NAME}"
echo "${params.PL_BUILD_NUMBER}"
build_number = params.PL_BUILD_NUMBER as Integer
sleep time: 5, unit: 'SECONDS'
wfjob = Jenkins.instance.getItemByFullName(params.PL_JOB_NAME)
wfjob.getBuilds().findAll { it.number >= build_number }.each { it.delete() }
wfjob.save()
wfjob.nextBuildNumber = build_number
wfjob.save()
wfjob.updateNextBuildNumber(build_number)
wfjob.save()
wfjob.doReload()
Or the job will not be correctly reset and you have to hit build until you reach next free number in the meanwhile the jenkins log will show:
java.lang.IllegalStateException: JENKINS-23152: ****/<BUILD_NUMBER> already existed;