Jenkins pipeline script - wait for running build - jenkins

Question

I have jenkins groovy pipeline which triggers other builds. It is done in following script:
for (int i = 0; i < projectsPath.size(); i++) {
stepsForParallel[jenkinsPath] = {
stage("build-${jenkinsPath}") {
def absoluteJenkinsPath = "/${jenkinsPath}/BUILD"
build job: absoluteJenkinsPath, parameters: [[$class: 'StringParameterValue', name: 'GIT_BRANCH', value: branch],
[$class: 'StringParameterValue', name: 'ROOT_EXECUTOR', value: rootExecutor]]
}
}
}
parallel stepsForParallel
The problem is that my jobs depend on other common job, i.e. job X triggers job Y and job Z triggers job Y. What I'd like to achieve is that the job X triggers job Y and job Z waits for result of Y triggered by X.
I suppose I need to iterate over all running builds and check if any build of the same type is running. If yes then wait for it. Following code could wait for build to be done:
def busyExecutors = Jenkins.instance.computers
.collect {
c -> c.executors.findAll { it.isBusy() }
}
.flatten()
busyExecutors.each { e ->
e.getCurrentWorkUnit().context.future.get()
}
My problem is that I need to tell which running job I need to wait. To do so I need to check:
build parameters
build environments variables
job name
How can i retreive this kind of data?
I know that jenkins have silent period feature but after period expires new job will be triggered.
EDIT1
Just to clarify why I need this function. I have jobs which builds applications and libs. Applications depend on libs and libs depend on other libs. When build is triggered then it triggers downstream jobs (libs on which it depends).
Sample dependency tree:
A -> B,C,D,E
B -> F
C -> F
D -> F
E -> F
So when I trigger A then B,C,D,E are triggered and F is also triggered (4 times). I'd like to trigger F only once.
I have beta/PoC solution (below) which almost work. Right now I have following problems with this code:
echo with text "found already running job" is not flushed to the screen until job.future.get() ends
I have this ugly "wait" (for(i = 0; i < 1000; ++i){}). It is because result field isn't set when get method returns
import hudson.model.*
def getMatchingJob(projectName, branchName, rootExecutor){
result = null
def busyExecutors = []
for(i = 0; i < Jenkins.instance.computers.size(); ++i){
def computer = Jenkins.instance.computers[i]
for(j = 0; j < computer.getExecutors().size(); ++j){
def executor = computer.executors[j]
if(executor.isBusy()){
busyExecutors.add(executor)
}
}
}
for(i = 0; i < busyExecutors.size(); ++i){
def workUnit = busyExecutors[i].getCurrentWorkUnit()
if(!projectName.equals(workUnit.work.context.executionRef.job)){
continue
}
def context = workUnit.context
context.future.waitForStart()
def parameters
def env
for(action in context.task.context.executionRef.run.getAllActions()){
if(action instanceof hudson.model.ParametersAction){
parameters = action
} else if(action instanceof org.jenkinsci.plugins.workflow.cps.EnvActionImpl){
env = action
}
}
def gitBranchParam = parameters.getParameter("GIT_BRANCH")
def rootExecutorParam = parameters.getParameter("ROOT_EXECUTOR")
gitBranchParam = gitBranchParam ? gitBranchParam.getValue() : null
rootExecutorParam = rootExecutorParam ? rootExecutorParam.getValue() : null
println rootExecutorParam
println gitBranchParam
if(
branchName.equals(gitBranchParam)
&& (rootExecutor == null || rootExecutor.equals(rootExecutorParam))
){
result = [
"future" : context.future,
"run" : context.task.context.executionRef.run,
"url" : busyExecutors[i].getCurrentExecutable().getUrl()
]
}
}
result
}
job = getMatchingJob('project/module/BUILD', 'branch', null)
if(job != null){
echo "found already running job"
println job
def done = job.future.get()
for(i = 0; i < 1000; ++i){}
result = done.getParent().context.executionRef.run.result
println done.toString()
if(!"SUCCESS".equals(result)){
error 'project/module/BUILD: ' + result
}
println job.run.result
}

Answer 1

I have a similar problem to solve. What I am doing, though, is iterating over the jobs (since an active job might not be executed on an executor yet).
The triggering works like this in my solution:
if a job has been triggered manually or by VCS, it triggers all its (recursive) downstream jobs
if a job has been triggered by another upstream job, it does not trigger anything
This way, the jobs are grouped by their trigger cause, which can be retrieved with
#NonCPS
def getTriggerBuild(currentBuild)
{
def triggerBuild = currentBuild.rawBuild.getCause(hudson.model.Cause$UpstreamCause)
if (triggerBuild) {
return [triggerBuild.getUpstreamProject(), triggerBuild.getUpstreamBuild()]
}
return null
}
I give each job the list of direct upstream jobs it has. The job can then check whether the upstream jobs have finished the build in the same group with
#NonCPS
def findBuildTriggeredBy(job, triggerJob, triggerBuild)
{
def jobBuilds = job.getBuilds()
for (buildIndex = 0; buildIndex < jobBuilds.size(); ++buildIndex)
{
def build = jobBuilds[buildIndex]
def buildCause = build.getCause(hudson.model.Cause$UpstreamCause)
if (buildCause)
{
def causeJob = buildCause.getUpstreamProject()
def causeBuild = buildCause.getUpstreamBuild()
if (causeJob == triggerJob && causeBuild == triggerBuild)
{
return build.getNumber()
}
}
}
return null
}
From there, once the list of upstream builds have been made, I wait on them.
def waitForUpstreamBuilds(upstreamBuilds)
{
// Iterate list -- NOTE: we cannot use groovy style or even modern java style iteration
for (upstreamBuildIndex = 0; upstreamBuildIndex < upstreamBuilds.size(); ++upstreamBuildIndex)
{
def entry = upstreamBuilds[upstreamBuildIndex]
def upstreamJobName = entry[0]
def upstreamBuildId = entry[1]
while (true)
{
def status = isUpstreamOK(upstreamJobName, upstreamBuildId)
if (status == 'OK')
{
break
}
else if (status == 'IN_PROGRESS')
{
echo "waiting for job ${upstreamJobName}#${upstreamBuildId} to finish"
sleep 10
}
else if (status == 'FAILED')
{
echo "${upstreamJobName}#${upstreamBuildId} did not finish successfully, aborting this build"
return false
}
}
}
return true
}
And abort the current build if one of the upstream builds failed (which nicely translates as a "aborted build" instead of a "failed build").
The full code is there: https://github.com/doudou/autoproj-jenkins/blob/use_autoproj_to_bootstrap_in_packages/lib/autoproj/jenkins/templates/library.pipeline.erb
The major downside of my solution is that the wait is expensive CPU-wise when there are a lot of builds waiting. There's the built-in waitUntil, but it led to deadlocks (I haven't tried on the last version of the pipeline plugins, might have been solved). I'm looking for ways to fix that right now - that's how I found your question.