How to fail a jenkins job based on a condition - jenkins

Question

I use Jmeter to validate functional test for our API's. I have created an HTML file which contains the total number of tests executed and the passing/failure ones. This is a customized report.
Can I set a condition such that the build fails if there is a single failure in the report?
Please not that I have already tried performance plugin. But my assertion here is to see if the test case fails or passes and not the reponse/throughtput time.
Not sure if there is a plugin which will enable me to read the HTML and then fail the report if it sees a failing test case.

Answer 1

The easiest option would be running your JMeter test using Taurus tool as a wrapper. Taurus has powerful and flexible Pass/Fail Criteria subsystem where you can configure any combination of test failure thresholds and if any failure occurs - Taurus will exit with non-zero exit status. Jenkins is smart enough to catch this non-zero exit code and fail the build.
JMeter-only solution will be adding a JSR223 Listener with the script like:
if (!prev.isSuccessful()) {
System.exit(1)
}
Which is basically doing the same - in case of any failure the test will immediately exit with status of 1 triggering the build failure.

Answer 2

You could just use a shell job, grep for the failures and exit if found, something like this :
#!/bin/sh
if grep 'failure' file.html; then
exit 1
fi

Answer 3

I don't know what is the structure of your HTML file, but since it's a test pass / fail report then I suppose it's pretty simple.
You may try adding "Execute Python script" (needs "Python plugin") to the job. With that you can easily parse HTML doc and check the results using a simple script.

Answer 4

I was having a similar issue with Jenkins and JMeter. Here's what I did:
Install this plugin: https://wiki.jenkins.io/display/JENKINS/Log+Parser+Plugin
for your logParserfile, add this:
error /FAILED/
Create a BeanShell Assertion for each sampler you are testing that contains something like this:
String testscenario = "TEST SCENARIO: ";
String requestName = "getRequest";
String ver = "${My_ver}";
String respData = new String(ResponseData);
if (respData.contains("OK"))
{
log.info(testscenario+"Passed. "+requestName+" version ="+ver);
print(testscenario+"Passed. "+requestName+" version ="+ver);
} else
{
log.error(testscenario+"*FAILED. "+requestName);
print(testscenario+"*FAILED. "+requestName);
}
When your Jenkins job runs and anything fails, the FAILED log entry will show