Related
I have a jenkinsfile dropped into the root of my project and would like to pull in a groovy file for my pipeline and execute it. The only way that I've been able to get this to work is to create a separate project and use the fileLoader.fromGit command. I would like to do
def pipeline = load 'groovy-file-name.groovy'
pipeline.pipeline()
If your Jenkinsfile and groovy file in one repository and Jenkinsfile is loaded from SCM you have to do:
Example.Groovy
def exampleMethod() {
//do something
}
def otherExampleMethod() {
//do something else
}
return this
JenkinsFile
node {
def rootDir = pwd()
def exampleModule = load "${rootDir}#script/Example.Groovy "
exampleModule.exampleMethod()
exampleModule.otherExampleMethod()
}
If you have pipeline which loads more than one groovy file and those groovy files also share things among themselves:
JenkinsFile.groovy
def modules = [:]
pipeline {
agent any
stages {
stage('test') {
steps {
script{
modules.first = load "first.groovy"
modules.second = load "second.groovy"
modules.second.init(modules.first)
modules.first.test1()
modules.second.test2()
}
}
}
}
}
first.groovy
def test1(){
//add code for this method
}
def test2(){
//add code for this method
}
return this
second.groovy
import groovy.transform.Field
#Field private First = null
def init(first) {
First = first
}
def test1(){
//add code for this method
}
def test2(){
First.test2()
}
return this
You have to do checkout scm (or some other way of checkouting code from SCM) before doing load.
Thanks #anton and #Krzysztof Krasori, It worked fine if I combined checkout scm and exact source file
Example.Groovy
def exampleMethod() {
println("exampleMethod")
}
def otherExampleMethod() {
println("otherExampleMethod")
}
return this
JenkinsFile
node {
// Git checkout before load source the file
checkout scm
// To know files are checked out or not
sh '''
ls -lhrt
'''
def rootDir = pwd()
println("Current Directory: " + rootDir)
// point to exact source file
def example = load "${rootDir}/Example.Groovy"
example.exampleMethod()
example.otherExampleMethod()
}
Very useful thread, had the same problem, solved following you.
My problem was: Jenkinsfile -> call a first.groovy -> call second.groovy
Here my solution:
Jenkinsfile
node {
checkout scm
//other commands if you have
def runner = load pwd() + '/first.groovy'
runner.whateverMethod(arg1,arg2)
}
first.groovy
def first.groovy(arg1,arg2){
//whatever others commands
def caller = load pwd() + '/second.groovy'
caller.otherMethod(arg1,arg2)
}
NB: args are optional, add them if you have or leave blank.
Hope this could helps further.
In case the methods called on your loaded groovy script come with their own node blocks, you should not call those methods from within the node block loading the script. Otherwise you'd be blocking the outer node for no reason.
So, building on #Shishkin's answer, that could look like
Example.Groovy
def exampleMethod() {
node {
//do something
}
}
def otherExampleMethod() {
node {
//do something else
}
}
return this
Jenkinsfile
def exampleModule
node {
checkout scm // could not get it running w/o checkout scm
exampleModule = load "script/Example.Groovy"
}
exampleModule.exampleMethod()
exampleModule.otherExampleMethod()
Jenkinsfile using readTrusted
When running a recent Jenkins, you will be able to use readTrusted to read a file from the scm containing the Jenkinsfile without running a checkout - or a node block:
def exampleModule = evaluate readTrusted("script/Example.Groovy")
exampleModule.exampleMethod()
exampleModule.otherExampleMethod()
I have set up some folders (Using Cloudbees Folder Plugin).
It sounds like the simplest possible command to be able to tell Jenkins: Build every job in Folder X.
I do not want to have to manually create a comma-separated list of every job in the folder. I do not want to add to this list whenever I want to add a job to this folder. I simply want it to find all the jobs in the folder at run time, and try to build them.
I'm not finding a plugin that lets me do that.
I've tried using the Build Pipeline Plugin, the Bulk Builder Plugin, the MultiJob plugin, and a few others. None seem to support the use case I'm after. I simply want any Job in the folder to be built. In other words, adding a job to this build is as simple as creating a job in this folder.
How can I achieve this?
I've been using Jenkins for some years and I've not found a way of doing what you're after.
The best I've managed is:
I have a "run every job" job (which contains a comma-separated list of all the jobs you want).
Then I have a separate job that runs periodically and updates the "run every job" job as new projects come and go.
One way to do this is to create a Pipeline job that runs Groovy script to enumerate all jobs in the current folder and then launch them.
The version below requires the sandbox to be disabled (so it can access Jenkins.instance).
def names = jobNames()
for (i = 0; i < names.size(); i++) {
build job: names[i], wait: false
}
#NonCPS
def jobNames() {
def project = Jenkins.instance.getItemByFullName(currentBuild.fullProjectName)
def childItems = project.parent.items
def targets = []
for (i = 0; i < childItems.size(); i++) {
def childItem = childItems[i]
if (!childItem instanceof AbstractProject) continue;
if (childItem.fullName == project.fullName) continue;
targets.add(childItem.fullName)
}
return targets
}
If you use Pipeline libraries, then the following is much nicer (and does not require you to allow a Groovy sandbox escape:
Add the following to your library:
package myorg;
public String runAllSiblings(jobName) {
def names = siblingProjects(jobName)
for (def i = 0; i < names.size(); i++) {
build job: names[i], wait: false
}
}
#NonCPS
private List siblingProjects(jobName) {
def project = Jenkins.instance.getItemByFullName(jobName)
def childItems = project.parent.items
def targets = []
for (def i = 0; i < childItems.size(); i++) {
def childItem = childItems[i]
if (!childItem instanceof AbstractProject) continue;
if (childItem.fullName == jobName) continue;
targets.add(childItem.fullName)
}
return targets
}
And then create a pipeline with the following code:
(new myorg.JobUtil()).runAllSiblings(currentBuild.fullProjectName)
Yes, there are ways to simplify this further, but it should give you some ideas.
I developed a Groovy script that does this. It works very nicely. There are two Jobs, initBuildAll, which runs the groovy script and then launches the 'buildAllJobs' jobs. In my setup, I launch the InitBuildAll script daily. You could trigger it another way that works for you. We aren't full up CI, so daily is good enough for us.
One caveat: these jobs are all independent of one another. If that's not your situation, this may need some tweaking.
These jobs are in a separate Folder called MultiBuild. The jobs to be built are in a folder called Projects.
import com.cloudbees.hudson.plugins.folder.Folder
import javax.xml.transform.stream.StreamSource
import hudson.model.AbstractItem
import hudson.XmlFile
import jenkins.model.Jenkins
Folder findFolder(String folderName) {
for (folder in Jenkins.instance.items) {
if (folder.name == folderName) {
return folder
}
}
return null
}
AbstractItem findItem(Folder folder, String itemName) {
for (item in folder.items) {
if (item.name == itemName) {
return item
}
}
null
}
AbstractItem findItem(String folderName, String itemName) {
Folder folder = findFolder(folderName)
folder ? findItem(folder, itemName) : null
}
String listProjectItems() {
Folder projectFolder = findFolder('Projects')
StringBuilder b = new StringBuilder()
if (projectFolder) {
for (job in projectFolder.items.sort{it.name.toUpperCase()}) {
b.append(',').append(job.fullName)
}
return b.substring(1) // dump the initial comma
}
return b.toString()
}
File backupConfig(XmlFile config) {
File backup = new File("${config.file.absolutePath}.bak")
FileWriter fw = new FileWriter(backup)
config.writeRawTo(fw)
fw.close()
backup
}
boolean updateMultiBuildXmlConfigFile() {
AbstractItem buildItemsJob = findItem('MultiBuild', 'buildAllProjects')
XmlFile oldConfig = buildItemsJob.getConfigFile()
String latestProjectItems = listProjectItems()
String oldXml = oldConfig.asString()
String newXml = oldXml;
println latestProjectItems
println oldXml
def mat = newXml =~ '\\<projects\\>(.*)\\<\\/projects\\>'
if (mat){
println mat.group(1)
if (mat.group(1) == latestProjectItems) {
println 'no Change'
return false;
} else {
// there's a change
File backup = backupConfig(oldConfig)
def newProjects = "<projects>${latestProjectItems}</projects>"
newXml = mat.replaceFirst(newProjects)
XmlFile newConfig = new XmlFile(oldConfig.file)
FileWriter nw = new FileWriter(newConfig.file)
nw.write(newXml)
nw.close()
println newXml
println 'file updated'
return true
}
}
false
}
void reloadMultiBuildConfig() {
AbstractItem job = findItem('MultiBuild', 'buildAllProjects')
def configXMLFile = job.getConfigFile();
def file = configXMLFile.getFile();
InputStream is = new FileInputStream(file);
job.updateByXml(new StreamSource(is));
job.save();
println "MultiBuild Job updated"
}
if (updateMultiBuildXmlConfigFile()) {
reloadMultiBuildConfig()
}
A slight variant on Wayne Booth's "run every job" approach. After a little head scratching I was able to define a "run every job" in Job DSL format.
The advantage being I can maintain my job configuration in version control. e.g.
job('myfolder/build-all'){
publishers {
downstream('myfolder/job1')
downstream('myfolder/job2')
downstream('myfolder/job2')
}
}
Pipeline Job
When running as a Pipeline job you may use something like:
echo jobNames.join('\n')
jobNames.each {
build job: it, wait: false
}
#NonCPS
def getJobNames() {
def project = Jenkins.instance.getItemByFullName(currentBuild.fullProjectName)
project.parent.items.findAll {
it.fullName != project.fullName && it instanceof hudson.model.Job
}.collect { it.fullName }
}
Script Console
Following code snippet can be used from the script console to schedule all jobs in some folder:
import hudson.model.AbstractProject
Jenkins.instance.getAllItems(AbstractProject.class).each {
if(it.fullName =~ 'path/to/folder') {
(it as AbstractProject).scheduleBuild2(0)
}
}
With some modification you'd be able to create a jenkins shared library method (requires to run outside the sandbox and needs #NonCPS), like:
import hudson.model.AbstractProject
#NonCPS
def triggerItemsInFolder(String folderPath) {
Jenkins.instance.getAllItems(AbstractProject.class).each {
if(it.fullName =~ folderPath) {
(it as AbstractProject).scheduleBuild2(0)
}
}
}
Reference pipeline script to run a parent job that would trigger other jobs as suggested by #WayneBooth
pipeline {
agent any
stages {
stage('Parallel Stage') {
parallel {
stage('Parallel 1') {
steps {
build(job: "jenkins_job_1")
}
}
stage('Parallel 2') {
steps {
build(job: "jenkins_job_2")
}
}
}
}
}
The best way to run an ad-hoc command like that would be using the Script Console (can be found under Manage Jenkins).
The console allows running Groovy Script - the script controls Jenkins functionality. The documentation can be found under Jenkins JavaDoc.
A simple script triggering immediately all Multi-Branch Pipeline projects under the given folder structure (in this example folder/subfolder/projectName):
import org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject
import hudson.model.Cause.UserIdCause
Jenkins.instance.getAllItems(WorkflowMultiBranchProject.class).findAll {
return it.fullName =~ '^folder/subfolder/'
}.each {
it.scheduleBuild(0, new UserIdCause())
}
The script was tested against Jenkins 2.324.
I am using the Pipeline plugin in Jenkins by Clouldbees (the name was Workflow plugin before), I am trying to get the user name in the Groovy script but I am not able to achieve it.
stage 'checkout svn'
node('master') {
// Get the user name logged in Jenkins
}
Did you try installing the Build User Vars plugin? If so, you should be able to run
node {
wrap([$class: 'BuildUser']) {
def user = env.BUILD_USER_ID
}
}
or similar.
To make it work with Jenkins Pipeline:
Install user build vars plugin
Then run the following:
pipeline {
agent any
stages {
stage('build user') {
steps {
wrap([$class: 'BuildUser']) {
sh 'echo "${BUILD_USER}"'
}
}
}
}
}
Here's a slightly shorter version that doesn't require the use of environment variables:
#NonCPS
def getBuildUser() {
return currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
}
The use of rawBuild requires that it be in a #NonCPS block.
It is possible to do this without a plugin (assuming JOB_BASE_NAME and BUILD_ID are in the environment):
def job = Jenkins.getInstance().getItemByFullName(env.JOB_BASE_NAME, Job.class)
def build = job.getBuildByNumber(env.BUILD_ID as int)
def userId = build.getCause(Cause.UserIdCause).getUserId()
There is also a getUserName, which returns the full name of the user.
This works for me without the Build User plugin:
// get first entry of JSONArray
def buildCause = currentBuild.getBuildCauses()[0]
def buildPrincipal = [type:"unknown", name:""]
if (buildCause._class ==~ /.+BranchEventCause/) {
def branchCause = currentBuild.getRawBuild().getCause(jenkins.branch.BranchEventCause)
buildPrincipal = [type:"branch",name:buildCause.shortDescription]
} else
if (buildCause._class ==~ /.+TimerTriggerCause/) {
def timerCause = currentBuild.getRawBuild().getCause(hudson.triggers.TimerTrigger.TimerTriggerCause)
buildPrincipal = [type:"timer", name:"Timer event"]
} else
if (buildCause._class ==~ /.+UserIdCause/) {
def buildUserCause = currentBuild.getRawBuild().getCause(hudson.model.Cause.UserIdCause)
buildPrincipal = [type:"user", name:buildCause.userId]
} else
// ... other causes
def jobUserId, jobUserName
//then somewhere
wrap([$class: 'BuildUser']) {
jobUserId = "${BUILD_USER_ID}"
jobUserName = "${BUILD_USER}"
}
//then
println("Started By: ${jobUserName}")
We were using this plugin : Build User Vars Plugin. More variables are available.
//Below is a generic groovy function to get the XML metadata for a Jenkins build.
//curl the env.BUILD_URL/api/xml parse it with grep and return the string
//I did an or true on curl, but possibly there is a better way
//echo -e "some_string \c" will always return some_string without \n char
//use the readFile() and return the string
def GetUserId(){
sh """
/usr/bin/curl -k -s -u \
\$USERNAME:\$PASSWORD -o \
/tmp/api.xml \
\$BUILD_URL/api/xml || true
THE_USERID=`cat /tmp/api.xml | grep -oP '(?<=<userId>).*?(?=</userId>)'`
echo -e "\$THE_USERID \\c" > /tmp/user_id.txt
"""
def some_userid = readFile("/tmp/user_id.txt")
some_userid
}
I modified #shawn derik response to get it to work in my pipeline:
stage("preserve build user") {
wrap([$class: 'BuildUser']) {
GET_BUILD_USER = sh ( script: 'echo "${BUILD_USER}"', returnStdout: true).trim()
}
}
Then I can reference that variable later on by passing it or in the same scope as ${GET_BUILD_USER} . I installed the same plugin referenced.
Edit: I re-read the question - the below only gets you the user running the build (which technically is often more interesting), not the one triggering the build in the frontend (be it REST-API or WebUI).
If you have Jenkins impersonation enabled, then I believe the result should be equivalent, otherwise this will only get you the user who owns the jenkins agent on the build machine.
Original answer:
Another way would be to
sh 'export jenkins_user=$(whoami)'
Downside: Linux-dependent, difficult to port across multiple agents in a single build (but then, the auth context may be different on each slave)
Upside: No need to install plugins (which on shared/large Jenkins instances can be tricky)
The Build User Vars Plugin is useful when you are executing the stage on an agent.
The alternative is to use the current build clause (see https://code-maven.com/jenkins-get-current-user), which also works when your stage is set with agent none.
The following code is inspired by Juergen's solution but I added more possible trigger reason and display them in a formatted manner:
String getTriggerReason() {
def buildCause = currentBuild.getBuildCauses()[0]
if (buildCause._class ==~ /.+(BranchEventCause|BranchIndexingCause)/) {
if (env.JOB_BASE_NAME == 'master') {
return 'Triggered by master commit'
} else {
return "Triggered by ${buildCause.shortDescription}"
}
}
if (buildCause._class ==~ /.+TimerTriggerCause/) {
return 'Triggered by timer'
}
if (buildCause._class ==~ /.+BuildUpstreamCause/) {
return "Triggered by build #${buildCause.upstreamBuild}"
}
if (buildCause._class ==~ /.+UserIdCause/) {
def userName = buildCause.userName.replaceFirst(/\s?\(.*/, '')
return "Triggered by user ${userName}"
}
return 'Unknown trigger'
}
I need to update 'execute shell' command of Build section in Series of Jenkins Jobs. And i am using groovy for it.
Here is starting script. Although it does not seems to update.
import hudson.model.*
for(item in Hudson.instance.items) {
if (item.name == 'TEMP-RELEASE-UPDATE') {
println("--- Parameters for :" + item.name)
def branches = item.scm.getBranches()
for (builder in item.buildersList) {
new_command = builder.command.replaceAll('PATTERN_1','PATTERN_2')
builder.command = new_command
builder.save()
}
}
}
It normally breaks at 'builder.command = new_command'.
Can someone help to modify this script and save resultant to 'execute shell' block successfully?
Thanks
builder.command = new_command breaks because builder.command is read-only.
What you need to do is to instantiate new builder instance:
new hudson.tasks.Shell(new_command)
add that to the list and remove the old one.
My full script:
jobsChanged = new java.util.ArrayList()
jobsNotChanged = new java.util.ArrayList()
for(project in Hudson.instance.items) {
new_builder = null
old_builder = null
for (builder in project.buildersList) {
if (!(builder instanceof hudson.tasks.Shell)) {
jobsNotChanged.add "$project.name"
continue;
}
new_command = builder.command.replace(SOMETHING, SOMETHINGELSE)
new_builder = new hudson.tasks.Shell(new_command)
old_builder = builder;
jobsChanged.add "$project.name"
}
if (new_builder != null) {
// comment out below for test run
project.buildersList.add(new_builder)
project.buildersList.remove(old_builder)
}
}
println ""
println "Jobs changed"
println ""
jobsChanged.each { line -> println line }
println ""
println "Jobs not changed"
println ""
jobsNotChanged.each { line -> println line }
println ""
""
You can make use of Jenkins Configuration Slicer to edit the Shell command of jobs in bulk.
You need to save the job for it to actually be saved.
in your case it would be:
item.save()
For Jenkins using a Groovy System Script, is there a way to easily search the build queue and list of executing builds for some criteria (specifically a parameter that matches some condition) and then kill/cancel them?
I cannot seem to find any way to do this, but it seems like it should be possible.
I haven't tested it myself, but looking at the API it should be possible in the following way:
import hudson.model.*
import jenkins.model.Jenkins
def q = Jenkins.instance.queue
q.items.findAll { it.task.name.startsWith('my') }.each { q.cancel(it.task) }
Relevant API links:
http://javadoc.jenkins-ci.org/jenkins/model/Jenkins.html
http://javadoc.jenkins-ci.org/hudson/model/Queue.html
I know it's kind of an old question, but Google points me to this one. The scripts shown here only remove the jobs from the queue, and don't stop running builds. The following script, just removes everything from the queue and kills all running builds:
import java.util.ArrayList
import hudson.model.*;
import jenkins.model.Jenkins
// Remove everything which is currently queued
def q = Jenkins.instance.queue
for (queued in Jenkins.instance.queue.items) {
q.cancel(queued.task)
}
// stop all the currently running jobs
for (job in Jenkins.instance.items) {
stopJobs(job)
}
def stopJobs(job) {
if (job in com.cloudbees.hudson.plugins.folder.Folder) {
for (child in job.items) {
stopJobs(child)
}
} else if (job in org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject) {
for (child in job.items) {
stopJobs(child)
}
} else if (job in org.jenkinsci.plugins.workflow.job.WorkflowJob) {
if (job.isBuilding()) {
for (build in job.builds) {
build.doKill()
}
}
}
}
Referencie: https://xanderx.com/post/cancel-all-queued-jenkins-jobs/
Run this in Manage Jenkins > Script Console:
Jenkins.instance.queue.clear()
After some investigation, I came up with this code which works absolutely fine for me. It clears the queue and also aborts all the jobs currently getting executed.
Prerequisites:
'All' view contains all the jobs
Use System groovy
import jenkins.model.Jenkins
import hudson.*
import hudson.model.*
import jenkins.*
// Remove everything which is currently queued
Jenkins.instance.queue.clear()
def buildingJobs = Jenkins.instance.getAllItems(Job.class).findAll {
it.isBuilding()
}
buildingJobs.each {
def jobName = it.toString()
def val = jobName.split("\\[|\\]")
// 'Abort jobs' is the name of the job I have created, and I do not want it to abort itself.
if((val[1].trim())!='Abort jobs') {
def job = Jenkins.instance.getItemByFullName(val[1].trim())
for (build in job.builds) {
if (build.isBuilding()) {
println(build)
build.doStop();
}
}
}
}
couldn't add as a comment, but as of today with latest jenkins, Andrey's script (nice) requires another import to work. Executing as system Groovy script.
Jenkins errors and mentions the missing class. I included the url that mentioned the issue:
//import hudson.model.*
// per http://stackoverflow.com/questions/17429050/running-groovy-command-from-jenkins-using-groovy-script-plugin
// requires this now
import jenkins.model.Jenkins
def q = Jenkins.instance.queue
q.items.findAll { it.task.name.startsWith('my') }.each { q.cancel(it.task) }
Here is my solution, if you want to run only the newest job of same project from the build queue and cancel other:
def q = Jenkins.instance.queue
//Find items in queue that match <project name>
def queue = q.items.findAll { it.task.name.startsWith('sample_project') }
//get all jobs id to list
def queue_list = []
queue.each { queue_list.add(it.getId()) }
//sort id's, remove last one - in order to keep the newest job, cancel the rest
queue_list.sort().take(queue_list.size() - 1).each { q.doCancelItem(it) }
import hudson.model.*
def queue = Hudson.instance.queue
println "Queue contains ${queue.items.length} items"
queue.clear()
println "Queue cleared"
paste this in "Script Console" - jenkins > Manage Jenkins > Script Console and hit run.
This helped in clearing the queue and also displays the number that is cleared.
Use the jenkins groovy postbuild plugin:
I think this would be the groovy script:
import hudson.model.*
def q = jenkins.model.Jenkins.getInstance().getQueue()
def items = q.getItems()
for (i=0;i<items.length;i++){
if(items[i].task.getName() == "job_name"){
items[i].doCancelQueue()
}
}
I've expanded upon the snippet by Igor Zilberman so that it also aborts running jobs when there is a job in the queue with the same cause (what you see when you hover over the job in the build queue, only looking at the first line). I’m running this as a job with build step “Execute System Groovy Script”.
import hudson.model.Result
import jenkins.model.CauseOfInterruption
import jenkins.model.*;
[ // setup job names here
'my-jobname-here'
].each {jobName ->
def queue = Jenkins.instance.queue
def q = queue.items.findAll { it.task.name.equals(jobName) }
def r = [:]
def projs = jenkins.model.Jenkins.instance.items.findAll { it.name.equals(jobName) }
projs.each{p ->
x = p._getRuns()
x.each{id, y ->
r.put(id, y)
}
}
TreeMap queuedMap = [:]
TreeMap executingMap = [:]
q.each{i->
queuedMap.put(i.getId(), i.getCauses()[0].getShortDescription()) //first line
}
r.each{id, run->
def exec = run.getExecutor()
if(exec != null){
executingMap.put(id, run.getCauses()[0].getShortDescription()) //first line
}
}
println("Queued:")
queuedMap.each{ k, v -> println "${k}:${v}" }
println("Executing:")
executingMap.each{ k, v -> println "${k}:${v}" }
// First, if there is more than one queued entry, cancel all but the highest one.
// Afterwards, if there is a queued entry, cancel the running ones
def queuedNames = queuedMap.values();
queuedNames.each{n ->
def idsForName = []
queuedMap.each{ id, name ->
if(name.equals(n)){
idsForName.add(id)
}
}
if (idsForName.size() > 1){
println("Cancelling queued job: "+n)
}
// remove all but the latest from queue
idsForName.sort().take(idsForName.size() - 1).each { queue.doCancelItem(it) }
}
executingMap.each{ id, name ->
if(queuedMap.values().contains(name)){
r.each{rid, run->
if (id == rid){
def exec = run.getExecutor()
if(exec != null){
println("Aborting running job: "+id+": "+name)
exec.interrupt(Result.ABORTED)
}
}
}
}
}
}
return "Done"
To control Job build queue, you can use this Plugin also:
https://wiki.jenkins-ci.org/display/JENKINS/Block+queued+job+plugin
To block job when last build of defined target project is in building
status
To block job when last build of defined target project has result