In my scripted pipeline, I want to get changes since last successful build and based on files which have changed I want to enable or disable some parts of the pipeline. I am using Global Shared Library which contains definitions of some additional steps and the whole pipeline. To print changes since last successful build I am using the following code:
def showChanges(def build) {
if ((build != null) && (build.result != 'SUCCESS')) {
def changeLogSets = build.rawBuild.changeSets
for (int i = 0; i < changeLogSets.size(); i++) {
def entries = changeLogSets[i].items
for (int j = 0; j < entries.length; j++) {
def entry = entries[j]
echo "${entry.commitId} by ${entry.author} on ${new Date(entry.timestamp)}: ${entry.msg}"
def files = new ArrayList(entry.affectedFiles)
for (int k = 0; k < files.size(); k++) {
def file = files[k]
echo " ${file.editType.name} ${file.path}"
}
}
}
showChanges(build.getPreviousBuild())
}
}
However, when I do some change in global library then it prints just this change and not the change which happened on the main repository. The changeSet contains no info regarding files which have changed in the main cloned repository.
This is because Jenkins loads all changes from all repositories and shared libraries referenced in your Pipeline into rawBuild.changeSets. There's nothing you can really do about this except manually filter out repositories. For instance, if you only want changes that come from the my_awesome_repo repository:
changeSets = rawBuild.changeSets.findAll { changeSet->
try {
changeSet.getBrowser().getRepoUrl() =~ /my_awesome_repo/
} catch(groovy.lang.MissingMethodException e) {
false // repository has no `browser` property
}
}
Related
I want to be able to execute specific components that I organized in Files in the repository with only one main jenkinsfile.
For example I have this repo structure:
And I have three different components: Topic_A, Topic_B, Topic_C (same type of components but will be created for different teams).
I want to be able to modify only Topic_A and C and after I push the branch I want my jenkinsfile to able to execute just those changes instead also redeploying Topic_B which it was not modified.
My question is if this possible? Could it be done with a jenkins pipeline? or any other component? (script)
Thank you.
There is a changeset directive that allows you to check whether a file has changed in the Git repository. But it doesn't support checking directories, hence you can do something like the below.
pipeline {
agent any
stages {
stage('Cloning') {
steps {
// Get some code from a GitHub repository
git branch: 'main', url: 'https://github.com/xxxx/sample.git'
}
}
stage('TOPIC_A') {
when { expression { isChanged("Topic_A") } }
steps {
echo 'Doing something for Topic A'
}
}
stage('TOPIC_B') {
when { expression { isChanged("Topic_B") } }
steps {
echo 'Doing something for Topic B'
}
}
}
}
def isChanged(dirName) {
def changeLogSets = currentBuild.changeSets
def folderName = dirName
for (int i = 0; i < changeLogSets.size(); i++) {
def entries = changeLogSets[i].items
for (int j = 0; j < entries.length; j++) {
def entry = entries[j]
def files = new ArrayList(entry.affectedFiles)
for (int k = 0; k < files.size(); k++) {
def file = files[k]
if(file.path.contains(folderName)){
return true
}
}
}
}
return false
}
Anyone have a Jenkins Pipeline script that can stuff all the changes since the previous successful build in a variable? I'm using git and a multibranch pipeline job.
Well I managed to cobble something together. I'm pretty sure you can iterate the arrays better but here's what I've got for now:
node('Android') {
passedBuilds = []
lastSuccessfulBuild(passedBuilds, currentBuild);
def changeLog = getChangeLog(passedBuilds)
echo "changeLog ${changeLog}"
}
def lastSuccessfulBuild(passedBuilds, build) {
if ((build != null) && (build.result != 'SUCCESS')) {
passedBuilds.add(build)
lastSuccessfulBuild(passedBuilds, build.getPreviousBuild())
}
}
#NonCPS
def getChangeLog(passedBuilds) {
def log = ""
for (int x = 0; x < passedBuilds.size(); x++) {
def currentBuild = passedBuilds[x];
def changeLogSets = currentBuild.rawBuild.changeSets
for (int i = 0; i < changeLogSets.size(); i++) {
def entries = changeLogSets[i].items
for (int j = 0; j < entries.length; j++) {
def entry = entries[j]
log += "* ${entry.msg} by ${entry.author} \n"
}
}
}
return log;
}
Based on the answer from CaptRespect I came up with the following script for use in the declarative pipeline:
def changes = "Changes:\n"
build = currentBuild
while(build != null && build.result != 'SUCCESS') {
changes += "In ${build.id}:\n"
for (changeLog in build.changeSets) {
for(entry in changeLog.items) {
for(file in entry.affectedFiles) {
changes += "* ${file.path}\n"
}
}
}
build = build.previousBuild
}
echo changes
This is quite useful in stage->when->expression parts to run a stage only when certain files were changed. I haven't gotten to that part yet though, I'd love to create a shared library from this and make it possible to pass it some globbing patterns to check for.
EDIT: Check the docs btw, in case you want to delve a little deeper. You should be able to convert all the object.getSomeProperty() calls into just entry.someProperty.
This is what I've used:
def listFilesForBuild(build) {
def files = []
currentBuild.changeSets.each {
it.items.each {
it.affectedFiles.each {
files << it.path
}
}
}
files
}
def filesSinceLastPass() {
def files = []
def build = currentBuild
while(build.result != 'SUCCESS') {
files += listFilesForBuild(build)
build = build.getPreviousBuild()
}
return files.unique()
}
def files = filesSinceLastPass()
There's the Changes Since Last Success Plugin that could help you with that.
For anyone using Accurev here is an adaptation of andsens answer. andsens answer can't be used because the Accurev plugin doesn't implement getAffectedFiles. Documentation for the AccurevTransaction that extends the ChangeLogSet.Entry class can be found at here.
import hudson.plugins.accurev.*
def changes = "Changes: \n"
build = currentBuild
// Go through the previous builds and get changes until the
// last successful build is found.
while (build != null && build.result != 'SUCCESS') {
changes += "Build ${build.id}:\n"
for (changeLog in build.changeSets) {
for (AccurevTransaction entry in changeLog.items) {
changes += "\n Issue: " + entry.getIssueNum()
changes += "\n Change Type: " + entry.getAction()
changes += "\n Change Message: " + entry.getMsg()
changes += "\n Author: " + entry.getAuthor()
changes += "\n Date: " + entry.getDate()
changes += "\n Files: "
for (path in entry.getAffectedPaths()) {
changes += "\n " + path;
}
changes += "\n"
}
}
build = build.previousBuild
}
echo changes
writeFile file: "changeLog.txt", text: changes
In order to return the changes as a list of strings, instead of just printing them, you may use this function (based on #andsens answer):
def getChangesSinceLastSuccessfulBuild() {
def changes = []
def build = currentBuild
while (build != null && build.result != 'SUCCESS') {
changes += (build.changeSets.collect { changeSet ->
(changeSet.items.collect { item ->
(item.affectedFiles.collect { affectedFile ->
affectedFile.path
}).flatten()
}).flatten()
}).flatten()
build = build.previousBuild
}
return changes.unique()
}
I'm new to groovy and the workflow plugin, so perhaps this is something obvious. Q1: I'm try to run jobs read under a view in parallel. I do like this:
jenkins = Hudson.instance
parallel getBranches()
#NonCPS def getBranches() {
def jobBranches = [:]
for (int i = 0; i < getJobs().size(); i++) {
jobBranches["branch_${i}"] = {
build job : getJobs()[i]
}
}
return jobBranches
}
#NonCPS def getJobs() {
def jobArray = []
jenkins.instance.getView("view_A").items.each{jobArray.add(it.displayName)}
return jobArray
}
I got:
But if I wrote it like this:
jenkins = Hudson.instance
def jobBranches = [:]
for (int i = 0; i < getJobs().size(); i++) {
jobBranches["branch_${i}"] = {
build job : getJobs()[i]
}
}
parallel jobBranches
#NonCPS def getJobs() {
def jobArray = []
jenkins.instance.getView("view_A").items.each{jobArray.add(it.displayName)}
return jobArray
}
Then I got something like this:
What am I doing wrong? Or Is there another way to accomplish the same thing.
Q2: BTW, If there are three jobs, like j1, j2, j3. j1 and j2 are executed first and in parallel, when one of them are finished, j3 will be executed. so how to do this?
I figured out why.
for (int i = 0; i < getJobs().size(); i++) {
def j=i
jobBranches["branch_${i}"] = {
build job : getJobs()[j]
}
Then it will work!
I am trying to create an instance of a domain class inside a grails 2.3.6 script:
def player = new Player(name:"Bob")
player.save()
But I keep getting an exception
java.lang.NoClassDefFoundError: gaming/Player
I've tried all the different bootstrapping tricks I've managed to find on the internet but they don't really change the result:
I've tried importing:
import gaming.Player
I've tried loading the bootstrap script:
includeTargets << grailsScript("_GrailsBootstrap")
I've tried depending on every task I managed to find:
depends(configureProxy, packageApp, classpath, loadApp, configureApp, compile, bootstrap)
I've even tried loading the class at runtime:
ApplicationHolder.application.getClassForName("gaming.Player")
Interestingly enough, this last line doesn't barf which suggests that grails can find my class, but chooses to ignore the find when I actually go to use it.
Edit. As requested, here is the current version of the script
import gaming.Player
import org.codehaus.groovy.grails.commons.ApplicationHolder
includeTargets << grailsScript("_GrailsInit")
includeTargets << grailsScript("_GrailsBootstrap")
includeTargets << grailsScript("_GrailsClasspath")
def handleHeaderLine(line) {
def retval = []
line.each {
if(!it.equals("Game Name") && !it.equals("Total # of Copies")) {
println("Creating Player: " + it)
def player = new Player(name:it)
player.save
retval << it
} else {
retval << null
}
}
return retval;
}
def handleGameLine(header, line) {
println("Creating Game: " + line[0])
for(int i = 1; i < line.length - 1; i++) {
if(!header[i].equals("Total # of Copies")) {
def count = line[i] == "" ? 0 : Integer.parseInt(line[i]);
for(int j = 0; j < count; j++) {
println "Creating copy of " + line[0] + " owned by " + header[i]
}
}
}
}
target(loadAssets: "The description of the script goes here!") {
depends(configureProxy, packageApp, classpath, loadApp, configureApp, compile, bootstrap)
ApplicationHolder.application.getClassForName("gaming.Player")
def tsv = new File("...")
def header = null;
tsv.eachLine {
def line = it.split("\t")
if(header == null) {
header = handleHeaderLine(line)
println header
} else {
handleGameLine(header, line)
}
}
}
setDefaultTarget(loadAssets)
You do not have to do all the boiler plate effort to bring up the environment while running your script. run-script does that for you. When grails run-script is used following targets are run by default: checkVersion, configureProxy, bootstrap. And finally the script run-script is run.
run-script runs your custom script in GroovyShell by providing ApplicationContext and grailsApplication as bindings to shell. So what you would end up with your script is shown below as if it is written in Groovy console/shell:
//scripts/player/PlayerScript.groovy
def handleHeaderLine(line) {
def retval = []
line.each {
if(!it.equals("Game Name") && !it.equals("Total # of Copies")) {
println("Creating Player: " + it)
def player = new Player(name: it)
player.save(flush: true)
retval << it
} else {
retval << null
}
}
return retval
}
def handleGameLine(header, line) {
println("Creating Game: " + line[0])
for(int i = 1; i < line.length - 1; i++) {
if(!header[i].equals("Total # of Copies")) {
def count = line[i] == "" ? 0 : Integer.parseInt(line[i]);
for(int j = 0; j < count; j++) {
println "Creating copy of " + line[0] + " owned by " + header[i]
}
}
}
}
def tsv = new File("...")
def header = null
tsv.eachLine {
def line = it.split("\t")
if(header == null) {
header = handleHeaderLine(line)
println header
} else {
handleGameLine(header, line)
}
}
And then use run-script as below:
grails run-script scripts/player/PlayerScript.groovy
which would by default run the script in dev environment. If you want for other envirnments then use as
grails test run-script scripts/player/PlayerScript.groovy
BUT
Due to a major bug in latest version of grails, you won't be able to run script the above mentioned way because run-script always depends on bootstrap target and would always try to bring tomcat up while running script as the plugin scope in build which would result in Error loading plugin manager: TomcatGrailsPlugin. The workaround is also mentioned in the defect but here is a groovier implementation. Change in BuildConfig.groovy as:
plugins {
if ( !System.getProperty("noTomcat") ) {
build ":tomcat:7.0.52.1"
}
....
}
and then issue run-script command as:
grails -DnoTomcat=true run-script scripts/player/PlayerScript.groovy
On a side note, the reason your script was not running is that the class Player will not be loaded at this time while running script, for use. It has to be loaded manually using classLoader and then create an instance off of it. Something like:
includeTargets << grailsScript("_GrailsInit")
includeTargets << grailsScript("_GrailsBootstrap")
target(playerScript: "The description of the script goes here!") {
depends configureProxy, packageApp, classpath, loadApp, configureApp
def playerClass = classLoader.loadClass("gaming.Player")
//Skeptical about how a domain class would behave
//But a normal POGO should be good being used this way
def player = playerClass.newInstance([[name: "Bob"]] as Object[])
player.save(flush: true)
println player
}
setDefaultTarget(playerScript)
How I can get all the active jobs scheduled in the Quartz.NET scheduler? I tried the GetCurrentlyExecutingJobs() but it is returning always 0.
That method doesn't seem to work.
The only solution I had found was to loop through all the jobs:
var groups = sched.JobGroupNames;
for (int i = 0; i < groups.Length; i++)
{
string[] names = sched.GetJobNames(groups[i]);
for (int j = 0; j < names.Length; j++)
{
var currentJob = sched.GetJobDetail(names[j], groups[i]);
}
}
When a job is found it means that it is still active.
If you set your job as durable, though, it will never be deleted if there are no associated trigger.
In that situation this code works better:
var groups = sched.JobGroupNames;
for (int i = 0; i < groups.Length; i++)
{
string[] names = sched.GetJobNames(groups[i]);
for (int j = 0; j < names.Length; j++)
{
var currentJob = sched.GetJobDetail(names[j], groups[i]);
if (sched.GetTriggersOfJob(names[j], groups[i]).Count() > 0)
{
// still scheduled.
}
}
}
UPDATE:
I did some debugging to see what happens with GetCurrentlyExecutingJobs().
As a matter of fact it returns the job being executed but the elements are remove from the collection as soon as the job is executed.
You can check the 2 functions JobToBeExecuted and JobWasExecuted in the QuartzScheduler class.
A simpler loop option would be to get all of the job keys and iterate over them. This implementation is for a minimal API example. It gets all JobKeys from the scheduler and then iterates over each on to get the details and execution schedule. More details available in this sample repo: QuartzScheduler. If a job doesn't have a schedule, or it's scheduled execution has completed and there are no future executions planned then the job will not be included in the list of returned jobs.
app.MapGet("/schedules", async (ISchedulerFactory sf) =>
{
var scheduler = await sf.GetScheduler();
var definedJobDetails = new List<JobDetailsDto>();
var jobKeys = await scheduler.GetJobKeys(GroupMatcher<JobKey>.AnyGroup());
foreach (var jobKey in jobKeys)
{
var jobDetail = await scheduler.GetJobDetail(jobKey);
var jobSchedule = await scheduler.GetTriggersOfJob(jobKey);
if (jobDetail != null && jobSchedule != null)
{
definedJobDetails.Add(new JobDetailsDto(
jobDetail.Key.Name,
jobDetail.Key.Group,
jobDetail.Description,
jobSchedule.First().GetPreviousFireTimeUtc(),
jobSchedule.First().GetNextFireTimeUtc())
);
}
}
return definedJobDetails;
})