Jacoco 0% Code Coverage - jenkins

My Code Coverage in Sonar is showing 0% which isn't true as I do have Unit Tests.
Gradle
sonarqube {
properties {
property "sonar.binaries", "build/intermediates/classes/release"
property "sonar.java.binaries", "build/intermediates/classes/release"
property "sonar.java.test.binaries", "build/intermediates/classes/test/release"
property "sonar.sources", "src"
property "sonar.junit.reportsPath", "build/reports/tests/release"
property "sonar.java.junit.reportsPath", "build/reports/tests/release"
property "sonar.android.lint.report", "build/outputs/lint-results.xml"
property "sonar.jacoco.reportPath", "${project.buildDir}/jacoco/testReleaseUnitTest.exec"
}
}
When I open up the index.html inside build/reports/tests/release then I can see successful unit tests.
I run sonarqube as a gradle task within my Jenkins environment. My SonarQube instance shows Code Smells and everything but for code coverage, it shows 0%.
Update
I do get an index.html created for the Code Coverage but that's all showing 0% too:
app/build/reports/jacoco/jacocoTestDebugUnitTestReport/html/index.html
Update
Still getting 0% but this is what I have so far:
android {
...
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
testCoverageEnabled true
}
debug {
testCoverageEnabled true
}
}
jacoco {
version "0.7.8.201607051106"
}
}

Excerpt from SonarQube Documentation:
The Java Plugin reuses reports; it does not generate them. So before trying to configure your analysis to import these reports, make sure they are correctly generated and non-empty.
Since you don't seem to be using the Gradle Jacoco plugin, SonarQube is probably reporting that 0% because you have not generated a report. You will need to add Jacoco to your build and ensure that you have fed SonarQube the path of the generated report (sonar.jacoco.reportPath) so it can read it.
To add Jacoco to your project, you will need to add the following to build.gradle:
//...
apply plugin: "jacoco"
//...
jacoco {
toolVersion = "0.7.6.201602180812"
//Note: unless "reportsDir" is set here, default is “$buildDir/reports/jacoco”
}
You will also need to ensure the following: First of all, you need to ensure the jacocoTestReport task is being executed (either by adding it to tasks yourself; or by adding the task to your gradle invocation). Secondly, you need to make sure that SonarQube is looking in the correct location for the test report by setting sonar.jacoco.reportPath to point to your /reports/jacoco directory (it's defaulting to target/jacoco.exec, so it won't find a report on default settings).

I fixed the issue by using this plugin. The problem as I see it was that Jacoco was trying to look for Instrumentation Tests androidTests and not Unit Tests tests. The plugin which I used made sure that it ran the tests before hand, created a report based on the tests AND make Jacoco point to those tests.

Related

Turn off archiving of Allure report

Allure commandline 2.6.0;
Jenkins ver. 2.89.3;
I am using the following script (no any other post-build settings in job settings):
stage('Generate reports') {
allure([includeProperties: false,
reportBuildPolicy: 'ALWAYS',
results : [[path: allureResultsPath]]])
archive 'catalina.log'
}
This gives me the report, but also the following archive, attached to each run:
Is it required for Trend, history or something?
I'd like to turn it off as it is not used by me and only spends the disk's space.
Is it possible to turn it off using the pipeline script?
Okay, I looked at the plugin's code and as far as I can see, there is no way to turn off archiving the report. Because it is called right after the report is generated, without any conditions (see saveAllureArtifact at 306 and it's call at 299):
https://github.com/jenkinsci/allure-plugin/blob/master/src/main/java/ru/yandex/qatools/allure/jenkins/AllureReportPublisher.java#L299
In allure plugin you can add attribute disabled
disabled (optional)
Type: boolean
So add disabled: true to your configuration:
allure(disabled: true, ...

Multiple paths in sonar.jacoco.reportpath

I have a maven Jenkins job (Jenkins version 2.105) with Jacoco and Sonar (Version 6.0) configured. The project has multiple jacoco.exec created and I need to put the path for the same under sonar.jacoco.reportpath. The code coverage comes up in sonar if I add for only one exec. While adding the others are comma separted values, code coverage in not displayed in Sonar.
As the version of SonarQube is prior to 6.2 I understand we are required to use sonar.jacoco.reportPath property and not sonar.jacoco.reportPaths. How do we configure multiple path here?
You need to merge your JaCoCo .exec files into a single binary file.
To achieve this use JaCoCo's merge mojo.
Cristian (from cristian.io) has an excellent walkthrough of how to achieve this here. The following is a slightly modified version of the code from that blog post.
def allTestCoverageFile = "$buildDir/jacoco/allTestCoverage.exec"
task jacocoMergeTest(type: JacocoMerge) {
destinationFile = file(allTestCoverageFile)
executionData = project.fileTree(dir: '.', include:'**/build/jacoco/test.exec')
}
sonarqube {
properties {
property "sonar.projectKey", "your.org:YourProject"
property "sonar.projectName", "YourProject"
property "sonar.jacoco.reportPath", allTestCoverageFile
}
}

Combine Multiple json results in one, updated Cucumber-JVM Report

I have two runners in my automation project as follows:
Main runner - Executes all the #ui-test tagged test cases and if a scenario is failed target/rerun.txt will be populated with the scenario location (e.g. features/Dummy.feature:22):
#RunWith(Cucumber.class)
#CucumberOptions(
features = "classpath:features",
plugin = {"pretty", "html:target/cucumber-html-report", "json:target/cucumber.json", "rerun:target/rerun.txt"},
tags = {"#ui-test", "~#ignore"}
)
public class RunCukesTest {
}
Secondary runner - Re-executes the scenarios from target/rerun.txt:
#RunWith(Cucumber.class)
#CucumberOptions(
features = "#target/rerun.txt",
plugin = {"pretty", "html:target/cucumber-html-report-rerun", "json:target/cucumber_rerun.json"}
)
public class ReRunFailedCukesTest {
}
When the execution is performed two result json files are created:
cucumber.json
cucumber_rerun.json
Jenkins will collect the results via Cucumber-JVM Reports plugin and will create a combined report.
The problem is, even if all the target/rerun.txt tests are passed in the second run, the report status will remain failed because of the cucumber.json.
Is there a way (to set up Cucumber-JVM Reports plugin or modify the upper presented runners) to overwrite cucumber.json with the results from cucumber_rerun.json and to publish only the modified cucumber.json?
Another sub-keywords: maven, java, cucumber-java8, cucumber-junit, junit
I had problem similar to yours, though, I've used single runner, handled re-runs from testNG(re-runs was one of the reasons I've switched from JUnit to TestNG) directly and as a results I had increased amount of tests in my json report.
My solution was to clean json files afterwards, despite the fact that Jenkins knows about failed tests it won't mark build as failed or as unstable.
In your particular case you may try to somehow match tests from rerun.json and exclude them from regular json report.
For parsing jsons I may recommend using Jackson FasterXML
I use Jenkins cucumber reporting latest release with below config in Jenkins.
Image Of Config In Jenkins
1st Runner
#RunWith(Cucumber.class)
#CucumberOptions(
features="FolderFeature",
glue={"Gluefolder"},
plugin={"html:target/cucumberpf-html-report",
"json:target/cucumberpf.json"}
)
public class RunPF {
}
2nd Runner
#RunWith(Cucumber.class)
#CucumberOptions(
features="Blah/Test.feature",
glue={"mygluefolder"},
plugin={"html:target/cucumber-html-report",
"json:target/cucumber.json"}
)
public class RunRA {
}
I had failed in both .json files and when it passed both were merged and updated correctly in one cucumber report.
Here is the error:
[CucumberReport] Preparing Cucumber Reports
[CucumberReport] JSON report directory is "C:\Users\ajacobs\workspace\com.mytest.framework\target\"
[CucumberReport] Copied 2 json files from workspace "C:\Users\admin\workspace\yourtest\target" to
reports directory "C:\Users\admin\.jenkins\jobs\Regression\builds\21\cucumber-html-reports\.cache"
[CucumberReport] Processing 2 json files:
[CucumberReport] C:\Users\admin\yourtest\builds\21\cucumber-html-reports\.cache\cucumber.json
[CucumberReport] C:\Users\admin\yourtest\builds\21\cucumber-html-reports\.cache\cucumberpf.json
Finished: SUCCESS

gradle - make a step 100% optional

We use clover for code coverage testing but it interferes with stack traces and error information. I want to be able to use cloverGenerateReport when doing automated builds via jenkins but to skip this step entirely when doing local builds.
I've tried the various suggestions from searches for 'gradle optional dependencies' but I can't seem to get clover completely out of the way.
Suggestions?
You can use the method onlyIf.
cloverGenerateReport.onlyIf {
project.hasProperty('enableClover') ? Boolean.valueOf(project.getProperty('enableClover')) : false
}
On the command line you can enable it by providing the project property:
gradle cloverGenerateReport -PenableClover=true
One solution would be to check if the environment variable "JENKINS_HOME" exists. If it does, then set cloverGenerateReport as a dependency to another task.
In your build.gradle:
def env = System.getenv()
if(env.containsKey('JENKINS_HOME')){
reportTask.dependsOn cloverGenerateReport
}

Building grails project successfully only if coverage check passes

I am working on a grails application. We use cobertura to generate the code coverage reports. Now I want to modify the grails project such that the build should fail if the code coverage is less than say, 90%.
How can I achieve this in grails?
I don't think the code-coverage plugin supports this directly, but it's easy enough to do by hooking into the powerful grails build events infrastructure. By placing this in your scripts/_Events.groovy, the build will fail if coverage is below a certain threshold:
eventStatusFinal = { message ->
if (message ==~ /.*Cobertura Code Coverage Complete.*/) {
def report = new XmlSlurper().parse(new File("target/test-reports/cobertura/coverage.xml"))
if (Float.parseFloat(report.'#line-rate'.text()) < 0.90) {
throw new RuntimeException("coverage too low!")
}
}
}
This requires you to turn on the XML report generation with this in grails-app/conf/BuildConfig.groovy:
coverage {
xml = true
}
Adjust the attribute (line-rate, branch-rate) and value as needed.

Resources