So, I start my pipeline script with this piece of code:
node {
if (env.JOB_NAME != 'Company/project/develop' && env.JOB_NAME != 'Company/project/master'){
def jobName = env.JOB_NAME
def job = Jenkins.instance.getItemByFullName(jobName)
for (build in job.builds) {
if (build.isBuilding() && build != job.builds.first()) {
println '*' * 30
println 'All previous builds for this job have been aborted!'
println '*' * 30
build.doStop()
}}
}
}
And it works perfectly. It stops all previous builds for the same job name. The point of this is to optimize Jenkins when developers are pushing too much code, but latest push is only important one. Master and develop branches are excluded from this rule.
I would like to expand this a little bit, by adding interruption cause. I would like to print into stopped build for example: "This job has been stop because is obsolete..."
I have tried a lot of things posted on stackoverflow, but I didn't succeed.
Thanks
Here is my implementation of similar function. It will abort previous running build, and show cause "Aborted by newer build #123"
import hudson.model.Result
import hudson.model.Run
import jenkins.model.CauseOfInterruption.UserInterruption
Run previousBuild = currentBuild.rawBuild.getPreviousBuildInProgress()
while (previousBuild != null) {
if (previousBuild.isInProgress()) {
def executor = previousBuild.getExecutor()
if (executor != null) {
echo ">> Aborting older build #${previousBuild.number}"
executor.interrupt(Result.ABORTED, new UserInterruption("Aborted by newer build #${currentBuild.number}"))
}
}
previousBuild = previousBuild.getPreviousBuildInProgress()
}
Related
We are using a Jenkins multibranch pipeline with BitBucket to build pull request branches as part of our code review process.
We wanted to abort any queued or in-progress builds so that we only run and keep the latest build - I created a function for this:
def call(){
def jobName = env.JOB_NAME
def buildNumber = env.BUILD_NUMBER.toInteger()
def currentJob = Jenkins.instance.getItemByFullName(jobName)
for (def build : currentJob.builds){
def exec = build.getExecutor()
if(build.isBuilding() && build.number.toInteger() != buildNumber && exec != null){
exec.interrupt(
Result.ABORTED,
new CauseOfInterruption.UserInterruption("Job aborted by #${currentBuild.number}")
)
println("Job aborted previously running build #${build.number}")
}
}
}
Now in my pipeline, I want to run this function when the build is triggered by the creation or push to a PR branch.
It seems the only way I can do this is to set the agent to none and then set it to the correct node for each of the subsequent stages. This results in missing environment variables etc. since the 'environment' section runs on the master.
If I use agent { label 'mybuildnode' } then it won't run the stage until the agent is free from the currently in-progress/running build.
Is there a way I can get the 'cancelpreviousbuilds()' function to run before the agent is allocated as part of my pipeline?
This is roughly what I have currently but doesn't work because of environment variable issues - I had to do the skipDefaultCheckout so I could do it manually as part of my build:
pipeline {
agent none
environment {
VER = '1.2'
FULLVER = "${VER}.${BUILD_NUMBER}.0"
PROJECTPATH = "<project path>"
TOOLVER = "2017"
}
options {
skipDefaultCheckout true
}
stages {
stage('Check Builds') {
when {
branch 'PR-*'
}
steps {
// abort any queued/in-progress builds for PR branches
cancelpreviousbuilds()
}
}
stage('Checkout') {
agent {
label 'buildnode'
}
steps {
checkout scm
}
}
}
}
It works and aborts the build successfully but I get errors related to missing environment variables because I had to start the pipeline with agent none and then set each stage to agent label 'buildnode' to.
I would prefer that my entire pipeline ran on the correct agent so that workspaces / environment variables were set correctly but I need a way to trigger the cancelpreviousbuilds() function without requiring the buildnode agent to be allocated first.
You can try combining the declarative pipeline and the scripted pipeline, which is possible.
Example (note I haven't tested it):
// this is scripted pipeline
node('master') { // use whatever node name or label you want
stage('Cancel older builds') {
cancel_old_builds()
}
}
// this is declarative pipeline
pipeline {
agent { label 'buildnode' }
...
}
As a small side comment, you seem to use: build.number.toInteger() != buildNumber which would abort not only older builds but also newer ones. In our CI setup, we've decided that it's best to abort the current build if a newer build has already been scheduled.
I have jenkins pipeline and also using a shared library for jenkins.
In my multibranch pipeline three to four repo clone while executing build using bitbucket plugin.
my question is how to get the last build revision from the previous build.
I have tried currentBuild.changeSets approach but for multiple repositories clone, it fails.
I had to get SCM revisions from the previous builds too. I didn't find any API to get it nicely, so I implemented a workaround. It is not great, but at least it works ;-)
When you save an environment variable by using env.setProperty(name, value) it is saved in the build metadata as a build variable. You can read it at any moment.
pipeline {
agent any
stages {
stage('Test') {
script {
env.setProperty('MY_ENV', env.BUILD_NUMBER)
def previousBuild = currentBuild.previousBuild
if (previousBuild != null) {
echo previousBuild.buildVariables['MY_ENV'] // prints env.BUILD_NUMBER - 1
}
}
}
}
}
In your case you have 4 checkouts. I don't know how you close sources, so let's imagine that you have a cloneRepo method and it sets the GIT_COMMIT environment variable. They you may use:
def previousBuild = currentBuild.previousBuild
if (previousBuild != null) {
echo previousBuild.buildVariables['GIT_COMMIT_REPO_1']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_2']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_3']
echo previousBuild.buildVariables['GIT_COMMIT_REPO_4']
}
cloneRepo(repo1)
env.setProperty('GIT_COMMIT_REPO_1', env.GIT_COMMIT)
cloneRepo(repo2)
env.setProperty('GIT_COMMIT_REPO_2', env.GIT_COMMIT)
cloneRepo(repo3)
env.setProperty('GIT_COMMIT_REPO_3', env.GIT_COMMIT)
cloneRepo(repo4)
env.setProperty('GIT_COMMIT_REPO_4', env.GIT_COMMIT)
If you use the checkout step, then you may do:
def commitId = checkout(scm).find { it.key == 'GIT_COMMIT' }
env.setProperty('GIT_COMMIT_REPO_1', commitId)
I need to check if specific job is running from another job. I dont want use API or plugins. Just bare hudson model.
I already tried some properties of Jenkins.instance.getItemByFullName('folder/job').lastBuild but they either returning whole job status (stable/unstable) or there is no RUNNING results. I even seen that there is a script which finds current running executors. So i dont believe that there is no way of checking running jobs.
https://wiki.jenkins.io/display/JENKINS/Find+builds+currently+running+that+has+been+executing+for+more+than+N+seconds
import com.cloudbees.groovy.cps.NonCPS
import jenkins.model.*
import hudson.model.Result
#NonCPS
def getProject(projectName) {
def project = jenkins.model.Jenkins.instance.getItemByFullName(projectName)
if (!project) {error("Project not found: $projectName")}
return project
}
project = getProject('folder/job')
build = project.lastBuild
def checkStatus2(){
//return Jenkins.instance.getItemByFullName('folder/job').lastBuild.getResult()
return Jenkins.instance.getItemByFullName('folder/job').lastBuild.buildStatusSummary.message
}
stage('check Job A status'){
//def status = checkStatus2()
//echo status
//if(hudson.model.Result.RUNNING.equals(status)){
if(build.#result == hudson.model.Result.NOT_BUILT){
echo "running"
}else{
echo "not running"
}
}
I found a thread which gave me a hint
Jenkins workflow check if job is running or schedulled
So i tried ...
if(Jenkins.instance.getItemByFullName('folder/job').isBuilding())
... and it works.
I have a Jenkins declarative pipeline I have been running on the Jenkins master and it works fine. However, now that I have moved to trying to execute this on a slave node, the groovy scripts which are called in the pipeline can not access the files in the workspace.
My jenkinsfile looks like this...
pipeline {
agent {
label {
label "windows"
customWorkspace "WS-${env.BRANCH_NAME}"
}
}
stages {
stage('InitialSetup') {
steps {
"${env.WORKSPACE}/JenkinsScripts/myScript.groovy"
}
}
}
I can see on the slave that it is creating the workspace, doing the checkout from git, and executing the script correctly. However, if something in the script try's to interact with the files in the workspace it fails.
If I have something simple like this...
def updateFile(String filename) {
echo env.NODE_NAME
filename = "${env.WORKSPACE}/path/to/file"
def myFile = new File(filename)
<do other things with the file>
}
...it says it can not find the file specified. It gives me the path it is looking for and I can confirm the file exists, and that the code runs when just building on the master.
Why can the script not find the files this way when in can just running on the master node? I added the "echo env.NODE_NAME" command into my groovy file and it says the script is executing on the correct node.
Thanks.
Turns out Groovy File commands are considered insecure, and although they will run on the master, they will not run on the slave. If you call them from a script that has the agent set to another node, it will still execute the command just fine, just on the master node, not the agent. Here's an excerpt of an article post https://support.cloudbees.com/hc/en-us/articles/230922508-Pipeline-Files-manipulation
The operation with File class are run on master, so only works if build is run on master, in this example I create a file and check if I can access it on a node with method exists, it does not exist because the new File(file) is executed on master, to check this I search for folder Users that exist on my master but not in the node.
stage 'file move wrong way'
//it only works on master
node('slave') {
def ws = pwd()
def context = ws + "/testArtifact"
def file = ws + '/file'
sh 'touch ' + file
sh 'ls ' + ws
echo 'File on node : ' + new File(file).exists()
echo 'Users : ' + new File('/Users').exists()
sh 'mv ' + file + ' ' + context
sh 'ls ' + ws
}
To execute file manipulation command we recommend to use native commands.
This is a simple example of operations in shell
stage 'Create file'
sh 'touch test.txt'
stage 'download file'
def out='$(pwd)/download/maven.tgz'
sh 'mkdir -p ./download'
sh 'curl -L http://ftp.cixug.es/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -o ' + out
stage 'move/rename'
def newName = 'mvn.tgz'
sh 'mkdir -p $(pwd)/other'
sh 'mv ' + out + ' ' + newName
sh 'cp ' + newName + ' ' + out
}
I run into this same issue recently. I had a python file that runs and writes the results to a JSON file. I was trying to access the JSON file to retrieve the data from there. Here is the code I was using inside a stage block of a declarative pipeline:
script {
def jsonSlurper = new JsonSlurper()
def fileParsed = new File("parameters.json")
def dataJSON = jsonSlurper.parse(fileParsed)
}
As everyone stated already, the above was failing with FileNotFoundException because anything inside script{} will only run on master and not the agent.
To work around the issue, I have used the Pipeline Utility Steps plugin (reference: https://plugins.jenkins.io/pipeline-utility-steps/ -- How to use: https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#writejson-write-json-to-a-file-in-the-workspace)
The plugin will allow you to do any read/write operation on multiple file formats.
Here is an example of the code I used after installing the plugin:
script {
def props = readJSON file: 'parameters.json'
println("just read it..")
println(props)
}
Note: I was using jenkins 2.249.1
I have implemented the code which automatically installs Groovy on slave (for scripted pipeline). Perhaps this solution is a little bit cumbersome, but pipelines don't offer any other way to achieve the same functionality as "Execute Groovy Script" stuff from the old Jenkins, because the plugin https://wiki.jenkins.io/display/JENKINS/Groovy+plugin is not supported yet for pipeline.
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.tools.ToolDescriptor;
import hudson.tools.ToolInstallation;
import hudson.tools.ToolInstaller;
import hudson.util.DescribableList;
import hudson.plugins.groovy.GroovyInstaller;
import hudson.plugins.groovy.GroovyInstallation;
/*
Installs Groovy on the node.
The idea was taken from: https://devops.lv/2016/12/05/jenkins-groovy-auto-installer/
and https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/configMavenAutoInstaller.groovy
COMMENT 1: If we use this code directly (not as a separate method) then we get
java.io.NotSerializableException: hudson.plugins.groovy.GroovyInstaller
COMMENT 2: For some reason inst.getExecutable(channel) returns null. I use inst.forNode(node, null).getExecutable(channel) instead.
TODO: Check if https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.step.MultiJobStepContext.groovyCommand
works better.
*/
#NonCPS
def installGroovyOnSlave(String version) {
if ((version == null) || (version == "")) {
version = "2.4.7" // some default should be
}
/* Set up properties for our new Groovy installation */
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def proplist = new DescribableList<ToolProperty<?>, ToolPropertyDescriptor>()
def installers = new ArrayList<GroovyInstaller>()
def autoInstaller = new GroovyInstaller(version)
installers.add(autoInstaller)
def InstallSourceProperty isp = new InstallSourceProperty(installers)
proplist.add(isp)
def inst = new GroovyInstallation("Groovy", "", proplist)
/* Download and install */
autoInstaller.performInstallation(inst, node, null)
/* Define and add our Groovy installation to Jenkins */
def descriptor = Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy")
descriptor.setInstallations(inst)
descriptor.save()
/* Output the current Groovy installation's path, to verify that it is ready for use */
def groovyInstPath = getGroovyExecutable(version)
println("Groovy " + version + " is installed in the node " + node.getDisplayName())
}
/* Returns the groovy executable path on the current node
If version is specified tries to find the specified version of groovy,
otherwise returns the first groovy installation that was found.
*/
#NonCPS
def getGroovyExecutable(String version=null) {
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def channel = node.getComputer().getChannel()
for (ToolInstallation tInstallation : Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy").getInstallations()) {
if (tInstallation instanceof GroovyInstallation) {
if ((version == null) || (version == "")) {
// any version is appropriate for us
return tInstallation.forNode(node, null).getExecutable(channel)
}
// otherwise check for version
for (ToolProperty prop in tInstallation.getProperties()) {
if (prop instanceof InstallSourceProperty) {
for (ToolInstaller tInstaller: prop.installers) {
if (
(tInstaller instanceof GroovyInstaller) &&
(tInstaller.id.equals(version))
)
return tInstallation.forNode(node, null).getExecutable(channel)
}
}
}
}
}
return null
}
/* Wrapper function. Returns the groovy executable path as getGroovyExecutable()
but additionally tries to install if the groovy installation was not found.
*/
def getGroovy(String version=null) {
def installedGroovy = getGroovyExecutable(version)
if (installedGroovy != null) {
return installedGroovy
} else {
installGroovyOnSlave(version)
}
return getGroovyExecutable(version)
}
Just put these 3 methods to your pipeline script and you will be able to get the Groovy executable path with the help of the method getGroovy(). If it is not installed yet then the installation will be done automatically. You can test this code with the simple pipeline, like this:
// Main
parallel(
'Unix' : {
node ('build-unix') {
sh(getGroovy() + ' --version')
}
},
'Windows' : {
node ('build-win') {
bat(getGroovy() + ' --version')
}
}
)
For me the output was:
[build-unix] Groovy Version: 2.4.7 JVM: 1.8.0_222 Vendor: Private Build OS: Linux
[build-win] Groovy Version: 2.4.7 JVM: 11.0.1 Vendor: Oracle Corporation OS: Windows 10
To work with files on the slave workspace use the readFile, writeFile, findFiles etc steps.
Or if they are large as FloatingCoder said use native tooling; which may be running a groovy script.
A workaround could be load the library via sh command in Jenkinsfile.
So, if you use in Jenkinsfile:
sh 'groovy libraryName.groovy'
You can load the lib locally and in this way you can store File on the slave node.
Even without pipelines, there is no option to restrict a job based on slave agent label. So, I think, pipelines are only for master node execution.
Starting from release 2.4 of the Groovy plugin there is withGroovy step available which sets up the environment on the agent so that you can do sh 'groovy yourscript.groovy' with expected environments. It also enables limited interaction between Pipeline and groovy script.
See https://www.jenkins.io/doc/pipeline/steps/groovy/ for some details about the step.
I delete old jenkins builds with rm where job is hosted:
my_job/builds/$ rm -rf [1-9]*
These old builds are still visible in job page.
How to remove them with command line?
(without the delete button in each build user interface)
Here is another option: delete the builds remotely with cURL. (Replace the beginning of the URLs with whatever you use to access Jenkins with your browser.)
$ curl -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The above deletes build #1 to #56 for job myJob.
If authentication is enabled on the Jenkins instance, a user name and API token must be provided like this:
$ curl -u userName:apiToken -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The API token must be fetched from the /me/configure page in Jenkins. Just click on the "Show API Token..." button to display both the user name and the API token.
Edit: As pointed out by yegeniy in a comment below, one might have to replace doDeleteAll by doDelete in the URLs above to make this work, depending on the configuration.
It looks like this has been added to the CLI, or is at least being worked on: http://jenkins.361315.n4.nabble.com/How-to-purge-old-builds-td385290.html
Syntax would be something like this: java -jar jenkins-cli.jar -s http://my.jenkins.host delete-builds myproject '1-7499' --username $user --password $password
Check your home jenkins directory:
"Manage Jenkins" ==> "Configure System"
Check field "Home directory" (usually it is /var/lib/jenkins)
Command for delete all jenkins job builds
/jenkins_home/jobs> rm -rf */builds/*
After delete should reload config:
"Manage Jenkins" ==> "Reload Configuration from Disk"
You can do it by Groovy Scripts using Hudson API.. Access your jenkins instalation
http://localhost:38080/script.
For Example, for deleting all old builds of all projects using the follow script:
Note: Take care if you use Finger Prints , you will lose all history.
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
System.out.println("Deleting all builds of job "+item.name)
for(build in item.getBuilds()){
build.delete()
}
}
else {
System.out.println("Skipping job "+item.name+", currently building")
}
}
Or for cleaning all workspaces :
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
println("Wiping out workspace of job "+item.name)
item.doDoWipeOutWorkspace()
}
else {
println("Skipping job "+item.name+", currently building")
}
}
There are a lot of examples on the Jenkins wiki
Is there a reason you need to do this manually instead of letting Jenkins delete old builds for you?
You can change your job configuration to automatically delete old builds, based either on number of days or number of builds. No more worrying about it or having to keep track, Jenkins just does it for you.
The following script cleans old builds of jobs. You should reload config from disk if you delete build manually:
import hudson.model.*
for(item in Hudson.instance.items) {
if (!item.isBuilding()) {
println("Deleting old builds of job " + item.name)
for (build in item.getBuilds()) {
//delete all except the last
if (build.getNumber() < item.getLastBuild().getNumber()) {
println "delete " + build
try {
build.delete()
} catch (Exception e) {
println e
}
}
}
} else {
println("Skipping job " + item.name + ", currently building")
}
}
From Script Console Run this, but you need to change the job name:
def jobName = "name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
From Jenkins Scriptler console run the following Groovy script to delete all the builds of jobs listed under a view:
import jenkins.model.Jenkins
hudson.model.Hudson.instance.getView('<ViewName>').items.each() {
println it.fullDisplayName
def jobname = it.fullDisplayName
def item = hudson.model.Hudson.instance.getItem(jobname)
def build = item.getLastBuild()
if (item.getLastBuild() != null) {
Jenkins.instance.getItemByFullName(jobname).builds.findAll {
it.number <= build.getNumber()
}.each {
it.delete()
}
}
}
def jobName = "MY_JOB_NAME"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().findAll { it.number < 10 }.each { it.delete() }
if you had 12 builds this would clear out builds 0-9 and you'd have 12,11,10 remaining. Just drop in the script console
This script will configure the build retention settings of all of the Jenkins jobs.
Change the values from 30 and 200 to suite you needs, run the script, then restart the Jenkins service.
#!/bin/bash
cd $HOME
for xml in $(find jobs -name config.xml)
do
sed -i 's#<daysToKeep>.*#<daysToKeep>30</daysToKeep>#' $xml
sed -i 's#<numToKeep>.*#<numToKeep>200</numToKeep>#' $xml
done
The script below works well with Folders and Multibranch Pipelines. It preserves only 10 last builds for each job. That could be adjusted or removed (proper if) if needed. Run that from web script console (example URL: https://jenkins.company.com/script)
def jobs = Hudson.instance.getAllItems(hudson.model.Job.class)
for (job in jobs){
println(job)
def recent = job.builds.limit(10)
for(build in job.builds){
if(!recent.contains(build)){
println("\t Deleting build: " + build)
build.delete()
}
}
}
From my opinion all those answers are not sufficient, you have to do:
echo "Cleaning:"
echo "${params.PL_JOB_NAME}"
echo "${params.PL_BUILD_NUMBER}"
build_number = params.PL_BUILD_NUMBER as Integer
sleep time: 5, unit: 'SECONDS'
wfjob = Jenkins.instance.getItemByFullName(params.PL_JOB_NAME)
wfjob.getBuilds().findAll { it.number >= build_number }.each { it.delete() }
wfjob.save()
wfjob.nextBuildNumber = build_number
wfjob.save()
wfjob.updateNextBuildNumber(build_number)
wfjob.save()
wfjob.doReload()
Or the job will not be correctly reset and you have to hit build until you reach next free number in the meanwhile the jenkins log will show:
java.lang.IllegalStateException: JENKINS-23152: ****/<BUILD_NUMBER> already existed;