I have a jenkins job that I'm using to aggregate the execution of multiple other jobs that only perform testing. Because they are testing, I want all the jobs to run regardless of any failures. I do want to keep track of wether or not there has been a failure so that I can set the end result to FAILURE rather than SUCCESS if need be.
At the moment I am calling 1 remote job via bash script and jenkins-cli. I have a 2nd child job that is local, so I'm using "trigger/call builds on other jobs" build step to run that one.
Any ideas on how to accomplish this?
If you can use build_flow-plugin it is easy, if you use pipeline it is possible too but can't give you example. Have to look it up if that is the case.
https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin:
def result = SUCCESS
ignore(FAILURE){
def job1 = build('job1')
result = job1.result.combine(result)
}
ignore(FAILURE){
def job2 = build('job2')
result = job1.result.combine(result)
}
build.result = result.combine(build.result)
http://javadoc.jenkins.io/hudson/model/Result.html
Related
In my company I have a pipeline that runs several jobs. I wanted to get the result of each job and write each of these results in a file or variable, later email it to me. Is there such a possibility? Remembering that: I don't want the result of the pipeline, but the result of each of the jobs that are inside it.
I even tried to make requests via api, but for each pipeline it would have to have a code and that is not feasible at all, the maintenance issue.
When you trigger a job inside a pipeline, you use the build job step.
This step has a property called propagate that:
If enabled (default state), then the result of this step is that of the downstream build (e.g., success, unstable, failure, not built, or aborted). If disabled, then this step succeeds even if the downstream build is unstable, failed, etc.; use the result property of the return value as needed.
You can write a wrapper for calling jobs, that stores the result of each job (and maybe other data useful for debugging, like build url), so you can use it later to construct the contents of an email.
E.g.
def jobResults = [:]
def buildJobAndStoreResult(jobName, jobParams) {
def run = build job: jobName, parameters: jobParams, propagate: false
jobResults[jobName] = [
result: run.result
]
}
Then you can constuct the body of an email by iterating through the map e.g.
emailBody = "SUMMARY\n\n"
jobResults.each() { it ->
str += "${it.key}: ${it.value.result}\n"
}
And use the mail step to send out a report.
It's worth thinking if you want your pipeline to fail after sending the email if any of the called jobs failed, and adding links from your email report to the failed jobs and caller pipeline.
I have a Jenkins job set up to poll for a change in a git repository every 10 minutes. If it doesn't find one (99/100 times, this is what happens) it aborts the build early and marks it as UNSTABLE . If it finds a change, it goes through with it and marks it as SUCCESS, storing its artifacts. I know I can use a plugin to discard old builds, but it only allows for these options:
As you can see, there is no option to filter by completion status.
Ideally, I want to discard all but the latest UNSTABLE build and keep all SUCCESS or FAILED builds and their artifacts. If this is not possible, simply discarding all UNSTABLE builds would also work.
Note: I am using a declarative Pipeline
One possibility would be to discard builds programmatically. Get your job object with def job = Jenkins.instance.getItem("JobName") Since you are using declarative pipeline, job is of type WorkflowJob [1] and you can get all its builds with
job.getBuilds(). Now you can check the result of each build (WorkflowRun objects [2]) and decide if you want to delete it or not. Something like follows should work
def job = Jenkins.instance.getItem("JobName")
job.getBuilds().each {
if(it.result.toString() == "UNSTABLE") {
it.delete()
job.save()
}
}
You could create a new job that executes the code above and is triggered after YourJob has been built.
[1] https://javadoc.jenkins.io/plugin/workflow-job/org/jenkinsci/plugins/workflow/job/WorkflowJob.html
[2] https://javadoc.jenkins.io/plugin/workflow-job/org/jenkinsci/plugins/workflow/job/WorkflowRun.html
have a pipeline script that executes child jobs in parallel.
Say I have 5 data (a,b,c,d,e) that has to be executed on 3 jobs (J1, J2, J3)
My pipeline script is in the below format
for (int i = 0; i < size; i++) { def index = i branches["branch${i}"] = { build job: 'SampleJob', parameters: [ string(name: 'param1', value:'${data}'), string(name:'dummy', value: "${index}")] } } parallel branches
My problem is, say the execution is happening on Job 1 with the data 1,2,3,4,5 and if the data 3 execution is failed on Job 1 then the data 3 execution should be stopped there itself and should not happen on the subsequent parallel execution on Jobs 2 and 3.
Is there any way that I can read the execution status of parallelly execution job status on the Pipeline script so that I can restrict data 3 execution to block in Jobs 2 and 3.
I am quite blocked here for a long time. Hoping for a solution from my community. Thanks a lot in advance.
In summary, it sounds like you want to
run multiple jobs in parallel against different pieces of data. I will call the set of related jobs the "batch".
avoid starting a queued job if any of the jobs in the batch have failed
automatically abort a running job if any of the jobs in the batch have failed
The jobs need some way to communicate their failure to the others. Use a shared storage location to store the "failure flag". If the file exists, then one or more of the jobs have failed.
For example, a shared NFS path: /shared/jenkins/jobstate/<BATCH_ID>/failed
At the start of the job, check for the existence of this path. Exit if it does. The file doesn't necessarily need to contain any data - its presence is enough.
Since you need running jobs to abort early if the failure flag exists, you will need to poll that location periodically. For example, after each unit of work. Again, if the file exists then exit early.
If you don't use NFS, that's ok. You could also use an object storage bucket. The important thing is that the state is accessible to all the relevant build jobs.
Right now, I have two sets of benchmarks, a short one and a long one. The short one runs on checkin for every branch. Which set to run is a parameter - SHORT or LONG. The long one always runs nightly on the dev branch. How can I trigger other branches to build and run the long benchmark if the branch was built successfully today?
If you want to run those long tests only over the night - I find it easiest to just duplicate the job and modify it so its triggered in the night and has additional checks added after the normal job, I.e. your post-commit jobs just do the short test, the nightly triggered do the short first and then (if no errors) the long one.
I find that much easier to handle then the added complexity of chaining jobs on some condition, like evaluating time of day to skip some tests.
Example 1st job that runs after every commit
node() {
stage('Build') {
// Build
}
stage('Short Test') {
// Short Test
}
}
2nd job that triggers nightly
node() {
stage('Build') {
// Build
}
stage('Short Test') {
// Short Test, fail the build here when not successful
}
stage('Long Tests')
// Long Test, runs only when short test successful
}
}
Edit
A solution that got it all in a single job, however it adds alot of complexity and makes some followup use cases harder to integrate, i.e. different notification for the integration test branch, tracking of build durations etc. I still find it more manageable to have it split in 2 jobs.
Following job must be configured to be triggered by post commit hook and a nightly timer. It runs the long test when
the last build is younger then set (you dont want it to trigger from the last nightly),
last run was successful (dont want to run long test for a broken build), and
was triggered by said timer (dont want to trigger on a check in).
def runLongTestMaxDiffMillis = 20000
def lastRunDiff = (currentBuild.getStartTimeInMillis().toInteger() - currentBuild.getPreviousBuild().getStartTimeInMillis().toInteger())
def lastBuildTooOld = (lastRunDiff > runLongTestMaxDiffMillis)
def isTriggeredByTimer = currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')
def lastBuildSuccessful = (currentBuild.getPreviousBuild().getResult() == 'SUCCESS')
def runLongTest = (!lastBuildTooOld && isTriggeredByTimer && lastBuildSuccessful)
node() {
if (runLongTest) {
println 'Running long test'
} else {
println 'Skipping long test'
}
}
You can create another pipeline that calls the parameterized pipeline with the LONG parameter, for example:
stage('long benchmark') {
build job: 'your-benchmark-pipeline', parameters: [string(name: 'type', value: 'LONG')]
}
When you configure this new pipeline you can tick the Build after other projects are built checkbox in the Build Triggers section and choose which short benchmarks should trigger it once they complete successfully (the default behavior).
You can use the Schedule Build Plugin to schedule a build of the long job when the short job succeed.
The short job runs on every branch, when a build succeed for a certain branch, it schedules a build of the long job (in the night) with the branch in parameter, so the long job will run on this particular branch.
On my team, we have a project that we want to do continuous-integration-style testing on. Our build takes around 2 hours and is triggered by the "Poll SCM" trigger (using Perforce as the server), and we have two build nodes.
Currently, if someone checks in a change, one build node will start up pretty much right away, but if another change gets checked in, the other node will not kick in, as it's waiting for the previous job to finish. However, I could like the other build node to start a build with the newer checkin as soon as possible, so that we can maximize the amount of continuous testing that's occurring (so that if e.g. one build fails we know sooner rather than later).
Is there any simple way to configure a Jenkins job (using Poll SCM against a Perforce server) to not block while another instance of the job is already running?
Unfortunately, due to the nature of the project it's not possible to simply break the project up into multiple build jobs that get pipelined across multiple slaves (as much as I'd like to change it to work in this way).
Use the "Execute concurrent builds if necessary" option in Jenkins configuration.
Just to register here in case someone needs it, in the version I'm using (Jenkins 2.249.3) I had to uncheck the option Do not allow concurrent builds in the child job that is called multiple times from the parent job.
The code is more or less like that:
stage('STAGE IN THE PARENT JOB') {
def subParallelJobs = [:]
LIST_OF_PARAMETERS = LIST_OF_PARAMETERS.split(",")
for (int i = 0; i < LIST_OF_PARAMETERS.size(); i++) {
MY_PARAMETER_VALUE = LIST_OF_PARAMETERS[i].trim()
MY_KEY_USING_THE_PARAMETER_TO_MAKE_IT_UNIQUE = "JOB_KEY_${MY_PARAMETER_VALUE}"
def jobParams = [ string(name: 'MY_JOB_PARAMETER', value: MY_PARAMETER_VALUE) ]
subParallelJobs.put("MY_KEY_USING_THE_PARAMETER_TO_MAKE_IT_UNIQUE", {build (job: "MY_CHILD_JOB", parameters: jobParams)})
}
parallel(subParallelJobs)
}
}