How to create pass threshold for TestCafe Tests on Jenkins - jenkins

Question

We've TestCafe.js UI tests that runs regression suite on Jenkins environment.
We're exploring a way to create a mechanism, wherein we can potentially set certain pass threshold for the test suite to make the Jenkins job status as Pass / Fail.
i.e. if 98% + tests pass then mark the test job as pass.
Under XUnit projects same could be achieved using XUnit test Plugin etc.
Example reference: How can I have Jenkins fail a build only when the number of test failures changes?
How to fail a Jenkins job based on pass rate threshold of testng tests
How to not mark Jenkins job as FAILURE when pytest tests fail
Is similar possible for TestCafe based tests either through TestCafe customization / through some Jenkins plugin?
Our Jenkins file:
#!groovy
pipeline {
environment {
CI = 'true'
}
options {
buildDiscarder(logRotator(numToKeepStr: '50'))
disableResume()
ansiColor('xterm')
}
agent none
// Define the stages of the pipeline:
stages {
stage('setup') {
steps {
script {
cicd.setupBuild()
}
}
}
// Use the make target to run tests:
stage('Tests') {
agent any
steps {
script {
cicd.withSecret(<keys>) {
cicd.runMake("test")
}
}
}
post {
cleanup {
archiveArtifacts artifacts: "screenshots/**", allowEmptyArchive: true
}
}
}
}
post {
success {
script { cicd.buildSuccess() }
}
failure {
script {
slackSend channel: "#<test-notifications-channel>", color: 'bad', message: "Regression tests failed or unstable <${env.RUN_DISPLAY_URL}|${env.JOB_NAME}>"
cicd.buildFailure()
}
}
}
}
enter code here

Answer 1

TestCafe provides a bunch of specified reporters, which generate a report in the special format. Once produced, CI system (or a plugin therein) can parse a report and perform threshold checks based on the number of failed/passed tests. TestCafe documentation includes an example with Jenkins integration. The Jenkins JUnit Plugin used in the example doesn't support set threshold yet: issue. But you can try to follow the steps in the guide in a similar way, except using Jenkins xUnit Plugin.