jenkins fails on building a downstream job - jenkins

I'm trying to trigger a downstream job from my current job like so
pipeline {
stages {
stage('foo') {
steps{
build job: 'my-job', propagate: true, wait: true
}
}
}
}
The purpose is to wait on the job result and fail or succeed according to that result. Jenkins is always failing with the message Waiting for non-job items is not supported . The job mentioned above does not have any parameters and is defined like the rest of my jobs, using multibranch pipeline plugin.
All i can think of is that this type of jenkins item is not supported as a build step input, but that seems counterintuitive and would prove to be a blocker to me. Can anyone confirm if this is indeed the case?
If so, can anyone suggest any workarounds?
Thank you

I actually managed to fix this by paying more attention to the definition of the build step. Since all my downstream jobs are defined as multibranch pipeline jobs, their structure is folder-like, with each item in the folder representing a separate job. Thus the correct way to call the downstream jobs was not build job: 'my-job', propagate: true, wait: true, but rather build job: "my-job/my-branch-name", propagate: true, wait: true.
Also, unrelated to the question but related to the issue at hand, make sure you always have at least one more executor free on the jenkins machine, since the wait on syntax will consume one thread for the waiting job and one for the job being waited on, and you can easily find yourself in a resource-starvation type situation.
Hope this helps

This looks like JENKINS-45443 which includes the comment
Pipeline has no support for the upstream/downstream job system, in part due to technical limitations, in part due to the fact that there is no static job configuration that would make this possible except by inspecting recent build metadata.
But it also offer the workaround:
as long as the solution is still ongoing, I include here our workaround. It is based in the rtp (Rich Text Publisher) plugin, that you should have installed to make it work:
At the end of our Jenkinsfile and after triggering the job, we wait it to finish. In that case, build() returns the object used to run the downstream job. We get the info from it.
Warning: getAbsoluteUrl() function is a critical one. Use it at your own risk!
def startedBld = build(
job: YOUR_DOWNSTREAM_JOB,
wait: true, // VERY IMPORTANT, otherwise build () does not return expected object
propagate: true
)
// Publish the started build information in the Build result
def text = '<h2>Downstream jobs</h2>Started job ' + startedBld.rawBuild.toString () + ''
rtp (nullAction: '1',parserName: 'HTML', stableText: text)
This issue is part of JENKINS-29913, opened for the past two years:
Currently DependencyGraph is limited to AbstractProject, making it impossible for Workflows to participate in upstream/downstream relationships (in cases where job chaining is required, for example due to security constraints).
It refers the RFE (Request for Enhancement) JENKINS-37718, based on another (unanswered) Stack Overflow question.

Related

Result of each jenkins job and send by email

In my company I have a pipeline that runs several jobs. I wanted to get the result of each job and write each of these results in a file or variable, later email it to me. Is there such a possibility? Remembering that: I don't want the result of the pipeline, but the result of each of the jobs that are inside it.
I even tried to make requests via api, but for each pipeline it would have to have a code and that is not feasible at all, the maintenance issue.
When you trigger a job inside a pipeline, you use the build job step.
This step has a property called propagate that:
If enabled (default state), then the result of this step is that of the downstream build (e.g., success, unstable, failure, not built, or aborted). If disabled, then this step succeeds even if the downstream build is unstable, failed, etc.; use the result property of the return value as needed.
You can write a wrapper for calling jobs, that stores the result of each job (and maybe other data useful for debugging, like build url), so you can use it later to construct the contents of an email.
E.g.
def jobResults = [:]
def buildJobAndStoreResult(jobName, jobParams) {
def run = build job: jobName, parameters: jobParams, propagate: false
jobResults[jobName] = [
result: run.result
]
}
Then you can constuct the body of an email by iterating through the map e.g.
emailBody = "SUMMARY\n\n"
jobResults.each() { it ->
​ str += "${it.key}: ${it.value.result}\n"
}
And use the mail step to send out a report.
It's worth thinking if you want your pipeline to fail after sending the email if any of the called jobs failed, and adding links from your email report to the failed jobs and caller pipeline.

How to discard old builds in Jenkins, based on their completion status?

I have a Jenkins job set up to poll for a change in a git repository every 10 minutes. If it doesn't find one (99/100 times, this is what happens) it aborts the build early and marks it as UNSTABLE . If it finds a change, it goes through with it and marks it as SUCCESS, storing its artifacts. I know I can use a plugin to discard old builds, but it only allows for these options:
As you can see, there is no option to filter by completion status.
Ideally, I want to discard all but the latest UNSTABLE build and keep all SUCCESS or FAILED builds and their artifacts. If this is not possible, simply discarding all UNSTABLE builds would also work.
Note: I am using a declarative Pipeline
One possibility would be to discard builds programmatically. Get your job object with def job = Jenkins.instance.getItem("JobName") Since you are using declarative pipeline, job is of type WorkflowJob [1] and you can get all its builds with
job.getBuilds(). Now you can check the result of each build (WorkflowRun objects [2]) and decide if you want to delete it or not. Something like follows should work
def job = Jenkins.instance.getItem("JobName")
job.getBuilds().each {
if(it.result.toString() == "UNSTABLE") {
it.delete()
job.save()
}
}
You could create a new job that executes the code above and is triggered after YourJob has been built.
[1] https://javadoc.jenkins.io/plugin/workflow-job/org/jenkinsci/plugins/workflow/job/WorkflowJob.html
[2] https://javadoc.jenkins.io/plugin/workflow-job/org/jenkinsci/plugins/workflow/job/WorkflowRun.html

Is there a way to use "propagate=false" in a Jenkinsfile with declarative syntax directly for stage/step?

You can use propagate on a build job as described here:
https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
So you can use something like this to prevent a failing step from failing the complete build:
build(job: 'example-job', propagate: false)
Is there a way to use this for a stage or a step? I know i can surround it with a try/catch and that does works almost as i want. It does ignore the failing the stage and resumes the rest of the build, but it does not display the stage as failed. For now i write all failing stages to a variable and output that on a later stage, but this is not ideal.
If i cant suppress propagation in a stage/step, is there maybe a way to use the build() call to do the same? Maybe if i move it to another pipeline and call that via build()?
Any help appreciated.
With catchError you can prevent a failing step from failing the complete build:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As you might have guessed, you can freely choose the buildResult and stageResult, in case you want it to be unstable or anything else. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this is a fairly new feature.
There are currently lots of suggestions for the scripted syntax, but for the declarative syntax work is in progress to support this.
Track the progress of https://issues.jenkins-ci.org/browse/JENKINS-26522 which groups all of the pieces together to achieve this. It has some interesting bits already marked as 'Resolved' (meaning a code change was made), such as https://issues.jenkins-ci.org/browse/JENKINS-49764 ( "Allow define a custom status for pipeline stage"). Unfortunately, I cannot find references to any of the tickets involved in the Jenkins changelog which would make sense since the parent ticket is not yet finished.
Of interest might also be the following : https://issues.jenkins-ci.org/browse/JENKINS-45579 which was reopened due to an issue. The environment for this is :
Admittedly, there are a confusing number of tickets tracking this work, but that is probably due to the fact that the functionality being implemented has a number of use-cases.
Another interesting ticket is "Individual Pipeline steps and stages/blocks should have Result statuses" , for which I was able to find a related PR: https://github.com/jenkinsci/workflow-api-plugin/pull/63
It is worth noting that the declarative pipeline was always designed as being opinionated and as such was not meant to support everything possible with the scripted syntax. For more complicated workflows and use-cases where it does not serve your needs, scripted syntax may be the only (and recommended?) option.
For needs such as the one you've stated, if enough noise is made, the declarative pipeline will probably be modified in due course to support it.

How to use upstream triggers in declarative Jenkisfile

What is the correct usage of the upstream trigger in a declarative Jenkinsfile?
I'm trying to add dependency triggers, so that the pipeline is triggered after another project has built successfully.
The jenkisci doku on github is listing upstream events as possible pipeline triggers here.
My Jenkisfile is currently looking like this:
pipeline {
agent {
docker {
...
}
}
options {
timeout(time: 1, unit: 'HOURS')
buildDiscarder(logRotator(numToKeepStr:'10'))
}
triggers {
upstream 'project-name,other-project-name', hudson.model.Result.SUCCESS
}
which leads to the following error:
WorkflowScript: 16: Arguments to "upstream" must be explicitly named. # line 16, column 9.
upstream 'project-name,other-project-name', hudson.model.Result.SUCCESS
^
Update
I changed the syntax for the upstream trigger according to the code snippet here. So, now there is at least no syntax error anymore. But the trigger is still not working as intended.
triggers {
upstream(
upstreamProjects: 'project-name,other-project-name',
threshold: hudson.model.Result.SUCCESS)
}
If I understand the documentation correctly this pipeline should be triggered if one of the two declared jobs has completed successfully right?
If both projects are in same folder and jenkins is aware of jenkinsfile (with below mentioned code) of downstream project(i.e downstream project has ran atleast once), this should work for you :
triggers {
upstream(upstreamProjects: "../projectA/master", threshold: hudson.model.Result.SUCCESS)
}
I'm still relatively new to Jenkins and I have been struggling to get this to work, too. Specifically, I thought that just saving the pipeline code with the triggers directive in the web UI editor would connect the two jobs. It doesn't.
However, if one then manually runs the downstream job, the upstream... code in its pipeline appears to modify the job definition and configure the triggers, resulting in the same situation one would have if they had just set things up in the "build triggers" section of the job config form. In other words, the directive appears to be just a way to tell Jenkins to do the web UI config work for you, when/if the job gets run for some reason.
I spent hours on this and it could have been documented better. Also, the "declarative directive generator" in Jenkins for triggers gave me something like upstream threshold: 'FAILURE' which resulted in something like:
WorkflowScript: 5: Expecting "class hudson.model.Result" for parameter "threshold" but got "FAILURE" of type class java.lang.String instead # line 5, column 23.
To fix this I had to change the parameter to read upstream threshold: hudson.model.Result.FAILURE. Fair enough, but then the generator is broken. Not impressed.

Multiple concurrent builds of the same project in Jenkins

On my team, we have a project that we want to do continuous-integration-style testing on. Our build takes around 2 hours and is triggered by the "Poll SCM" trigger (using Perforce as the server), and we have two build nodes.
Currently, if someone checks in a change, one build node will start up pretty much right away, but if another change gets checked in, the other node will not kick in, as it's waiting for the previous job to finish. However, I could like the other build node to start a build with the newer checkin as soon as possible, so that we can maximize the amount of continuous testing that's occurring (so that if e.g. one build fails we know sooner rather than later).
Is there any simple way to configure a Jenkins job (using Poll SCM against a Perforce server) to not block while another instance of the job is already running?
Unfortunately, due to the nature of the project it's not possible to simply break the project up into multiple build jobs that get pipelined across multiple slaves (as much as I'd like to change it to work in this way).
Use the "Execute concurrent builds if necessary" option in Jenkins configuration.
Just to register here in case someone needs it, in the version I'm using (Jenkins 2.249.3) I had to uncheck the option Do not allow concurrent builds in the child job that is called multiple times from the parent job.
The code is more or less like that:
stage('STAGE IN THE PARENT JOB') {
def subParallelJobs = [:]
LIST_OF_PARAMETERS = LIST_OF_PARAMETERS.split(",")
for (int i = 0; i < LIST_OF_PARAMETERS.size(); i++) {
MY_PARAMETER_VALUE = LIST_OF_PARAMETERS[i].trim()
MY_KEY_USING_THE_PARAMETER_TO_MAKE_IT_UNIQUE = "JOB_KEY_${MY_PARAMETER_VALUE}"
def jobParams = [ string(name: 'MY_JOB_PARAMETER', value: MY_PARAMETER_VALUE) ]
subParallelJobs.put("MY_KEY_USING_THE_PARAMETER_TO_MAKE_IT_UNIQUE", {build (job: "MY_CHILD_JOB", parameters: jobParams)})
}
parallel(subParallelJobs)
}
}

Resources