Is there a way to mark a test run failed if the number of tests decreases in jenkins. We have a test report in JUnit.
For some reasons, sometimes the number of tests decreases. Often that are critical errors. Is there a way to say jenkins that in such case the test status should be red? (Maybe some plugin)
Thanks a lot for any hint!
There is a way.
Install the Groovy Postbuild Plugin and add a Post Build Step
Post-Build-Actions -> Add Step -> Groovy Postbuild
import jenkins.model.*
def currentBuild = manager.build
def totalCurrent
def totalPrevious
// evaluate test count of current Build
def result = currentBuild .getAction(hudson.tasks.junit.TestResultAction.class).result
if (result != null) {
totalCurrent = result.hasProperty( 'totalCount' ) ? result.totalCount : null
}
// evaluate test count of previous Build
result = currentBuild .previousBuild.getAction(hudson.tasks.junit.TestResultAction.class).result
if (result != null) {
totalPrevious = result.hasProperty( 'totalCount' ) ? result.totalCount : null
}
// fail the build if test count reduced
if(totalCurrent < totalPrevious) {
manager.buildFailure()
}
You'll may need to add some more nullsafe checks.
Even if there is a way to do this, it sounds like a bad idea considering the fact that your number of tests executed could actually decrease from build to build for legitimate reasons.
First I would try to find out why this is happening unexpectedly.
Related
In Jenkins' Groovy, how can I detect that some other build is waiting for user input, as triggered by statements like input message: 'Retry?' ok: 'Restart'?
I checked the Executor and Build API docus, but couldn't identify something that matches. executor.isParking() sounded promising, but returns false.
The problem with Florian's answer is that if the input is waiting for a while, possibly across restarts of Jenkins, it might not be the last line in the log.
This is better:
import org.jenkinsci.plugins.workflow.job.WorkflowJob
import org.jenkinsci.plugins.workflow.support.steps.input.InputAction
def jobs = Jenkins.instance.getAllItems(WorkflowJob.class)
jobs.each { job ->
job.builds.each { build ->
if (build.isBuilding() && build.actions.last() instanceof InputAction) {
println "$job.fullName #$build.number"
}
}
}
This one is looking specifically for WorkflowJob jobs but you could change the class to be FreeStyleProject or whatever other kind of thing you want to look for. But checking that the last action is input seems like a good way to know if it's waiting for input.
Stupid as it may be, this is the only hack I found so far:
def isWaitingForInput(WorkflowRun otherBuild) {
def otherBuildsLog = otherBuild.getLog()
return otherBuildsLog.lastIndexOf("0mAbort") >= otherBuildsLog .length() - 8
}
0mAbort is the piece of code that defines the Abort link that the user can click to abort the build while it is waiting for input.
In jenkins pipelines you can define post steps to run under certain conditions. I am sending notifications of the build status to a mattermost channel and i cant figure out how I can report when a build returns to stable after a failure. The 'changed' option also runs when a build starts failing, but I want to use the 'failure' section for that to differentiate the type of message sent to mattermost.
Anyone know how to make this work?
So, ofcourse right after I post I found a working solution, but im still interested in other options
changed {
script {
if (env.CHANGE_ID == null && currentBuild.result == null) {
mattermostSend....
}
}
}
You can do this by combining https://support.cloudbees.com/hc/en-us/articles/226419147-How-can-I-check-previous-build-status-in-a-Pipeline-Script- and a comparison with the current build status.
if(currentBuild.rawBuild.getPreviousBuild()?.getResult().toString() != "SUCCESS" && ( currentBuild.result == "SUCCESS" || currentBuild.result == null ) ) {
echo "last build failed"
}
A few things to note: currentBuild.result defaults to null (which counts as a success), so if it's null or "SUCCESS", then you're good. Also, you'll need to whitelist some of these objects in your groovy script approval page found at jenkinsurl.com/scriptApproval/ (they will appear there after you try to run the script)
We are building a large number of variants of our code in every nightly build and naturally there will often be intermittent errors, even if the chance of a single error is only a fraction of a percent. Some of the most common ones are slaves that disconnect during a build and servers that don't respond.
The build failure analyzer plugin can categorize different failure causes but what we need is a plugin that can act on those problems and retrigger the build if there is an intermittent error. Preferebly the solution should fit into our build flow so that the results propagate to the job that creates the build report.
Is there such a plugin or other tool for doing this?
Here's Naginator Plugin!
you can :
Rerun build for unstable builds as well as failures
Only rebuild the job if the build's log output contains a given regular expression
Rerun build only for the failed parts of a matrix job
If you are using a Pipeline, you can define a generic retry method that takes an invocation and a retriable strategy as input.
What it does is to tee the execution in a separate file (using the tee step of Pipeline Utility Steps Plugin), then read the file and check if the retriable strategy applies.
Something like:
def retry(execution, isRetriable) {
def retryCount = 0
while (true) {
def file = "execution-${System.currentTimeMillis()}.log"
try {
tee(file) {
execution()
}
break
} catch (Exception e) {
def output = readFile(file: file)
if (isRetriable(output)) {
retryCount++
if (retryCount == 5) {
throw e
}
} else {
throw e
}
}
}
}
And then wrap your invocation with retry:
retry(
{ stepThatOccasionallyFails() },
{ output -> output.contains('a random error!') }
)
When promoting a build, I want to expose some choices of how/what to promote. To do this properly, I need some info about the build which I am promoting, and I'd really like to use the Extended Choice Parameter to do this.
How can I access a build's parameters in an Extended Choice Parameter block in a build promotion step? For example, say the build I want to promote was built with a parameter myvar="1,2,5", and I want to let the build promoter select 1, 2, 5, or some combination. How can I access that particular build's myvar value, in an extended choice parameter, in a promotion process?
I can auto-populate these with available Git branches for a certain repo, with global environment variables, but I haven't seen the groovy magic I need to access info about the build I'm (about to) promote.
This was not obvious to me, but I found that environment variables such as PROMOTED_URL, PROMOTED_JOB_NAME, etc, were not available until the promotion steps were in progress. I need this information exposed when the promotion parameters are being populated via Extended Choice Parameter's Groovy Script fields. I accomplished it by peeking into the thread name:
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
def jenkins = Jenkins.instance
def thread = Thread.currentThread().toString()
def (jobName,buildNumber) = thread.replaceAll(/.*GET \/job\/(.*)\/promotion\/.*/, '$1').split('/')
def job = jenkins.getJob(jobName)
def build = job.getBuild(buildNumber)
return build.getBuildVariables()["SOME_BUILD_PARAM"]
What do I use this for? Say I can select certain targets to build but not others, when building. I want to be able to promote only some subset of those that were actually built, so the parameter can't just hold a default value. Pretty narrow edge-case, I'll grant, but sometimes it seems like my entire life, as far as Jenkins is concerned, is dealing with some edge-case or another. If there is a more elegant way, I am all ears!
building on the #courtlandj solution (with the latest hudson API):
import hudson.model.*
def paramToLookup = "SOME_BUILD_VARIABLE_NAME"
// you can of course parse the jobName from the currentThread
// but that is a case-by-case basis depending on how you name them
// build numbers are strictly numeric - and more predictable to parse with regex
def jobName = "some-jenkins-full-job-name"
int buildNumber = 0
// Similar approach to courtlandj to get THIS build number (of page being viewed):
def thisPromotionInfo = Thread.currentThread().toString()
// groovy regex that gets the promotion digits (\d)
def buildRegex = /.*\/(\d+)\/promotion\/.*/
def matcher = ( thisPromotionInfo =~ buildRegex )
if (matcher.matches()) {
buildNumber = matcher[0][1].toInteger()
} else {
return "unable to identify buildNumber in Extended Choice Parameter (see Jenkins Config)"
}
// borrowed slightly from here: https://wiki.jenkins-ci.org/display/JENKINS/Display+job+parameters
try {
def job = Hudson.instance.getItemByFullName(jobName)
def build = job.getBuildByNumber(buildNumber)
return build.getProperties()["buildVariableResolver"].resolve(paramToLookup).toString()
} catch(Exception ex) {
return ex.toString()
}
relevant links I borrowed from:
https://groups.google.com/forum/#!topic/jenkinsci-users/cHClVbjuSN4
http://groovy.jmiguel.eu/groovy.codehaus.org/Tutorial+5+-+Capturing+regex+groups.html
http://hudson-ci.org/javadoc/hudson/model/Hudson.html
I'm trying to fail a Build Flow Plugin job if a parameter isn't set. When I try:
final parameter = params['PARAMETER']
if (parameter.isEmpty()) {
out.println('Error: PARAMETER must be defined')
System.exit(1)
}
the entire Jenkins master dies (double plus ungood).
How do I get the job to fail without killing Jenkins?
Set the Result on build.state then return so as not to execute anything else:
import hudson.model.Result
final parameter = params['PARAMETER']
if (parameter.isEmpty()) {
out.println('Error: PARAMETER must be defined')
build.state.setResult(Result.FAILURE)
return
}