How to configure sonar.coverage.jacoco.xmlReportPaths for JaCoCo/SonarQube? - jenkins

SonarQube 7.7 shows the following warning for a Java project analysis:
Property 'sonar.jacoco.reportPath' is deprecated (JaCoCo binary format). 'sonar.coverage.jacoco.xmlReportPaths' should be used instead (JaCoCo XML format).
The Gradle based project is configured via sonar-project.properties as follows:
sonar.projectKey=MyExampleLib
sonar.projectName=MyExample Library
sonar.sources=src/main/java
sonar.jacoco.reportPath=build/jacoco/test.exec
sonar.junit.reportsPath=build/test-results/test
sonar.java.test.binaries=build/classes/test
sonar.java.binaries=build/classes/java/main
sonar.binaries=build/classes
sonar.projectVersion=$libVersion
The SonarQube server URL is injected via (otherwise you end up with a "localhost:9000" error):
The SonarQube analysis is triggered via Jenkins and the JaCoCo plugin v.3.0.4 with the following Job configuration:
I read that a report.xml is picked up by xmlReportPaths. How can I generate it?
Related
https://github.com/jacoco/jacoco/issues/919
https://github.com/jacoco/jacoco/commit/6babdb5233217b0812a85f6b1673aabe7f0fd47e

We can generate Jacoco reports in XML format by specifying xml.enabled value to true and providing destination path in the reports section.
plugins {
id "org.sonarqube" version "2.8"
}
jacocoTestReport {
group = "Reporting"
reports {
xml.enabled true
csv.enabled false
//to create coverage report in html
html.destination file("${buildDir}/reports/coverage")
//for XML
xml.destination file("${buildDir}/reports/jacoco.xml")
}
}
The SonarQube properties can be also configured through the build.gradle file. As sonar.jacoco.reportPath and sonar.jacoco.reportPaths are deprecated properties from the sonar version of 7.7 which should be replaced with sonar.coverage.jacoco.xmlReportPaths.
Configuring the Sonarqube properties through the build.gradle
sonarqube {
properties {
property 'sonar.projectName', 'MyExample Library'
property 'sonar.projectKey', 'MyExampleLib'
property 'sonar.core.codeCoveragePlugin', 'jacoco'
property 'sonar.coverage.jacoco.xmlReportPaths', "${project.buildDir}/reports/jacoco.xml"
}
}
If you wish to do that through sonar-project.properties then update the deprecated properties mentioned below to the suggested one.
sonar.jacoco.reportPath=build/reports/jacoco.xml
Finally, by executing gradle jacocoTestReport sonarqube command, the jacoco test report files such as ${project.buildDir}/reports/jacoco.xml and ${project.buildDir}/jacoco/test.exec will be generated for SonarQube.

It seems that your build is based on Gradle. It would be easier to use jacoco and sonarqube plugins in the gradle build
plugins {
id "jacoco"
id "org.sonarqube" version "2.8"
}
you don't need sonar-project.properties, analysis is configured from your build. You can customize default values in sonarqube configuration
// in build.gradle
sonarqube {
properties {
property "sonar.exclusions", "**/*Generated.java"
}
}
To enable coverage you need to configure gradle to generate XML report
jacocoTestReport {
reports {
xml.enabled true
}
}
And then run with gradle build jacocoTestReport sonarqube. More details can be found here and in SonarScanner for Gradle doc

In Maven (pom.xml), simple add (under properties):
<properties>
<sonar.coverage.jacoco.xmlReportPaths>
../app-project-name/target/jacoco-report/jacoco.xml
</sonar.coverage.jacoco.xmlReportPaths>
</properties>

Related

Jenkins, Gradle : How to publish Dependency report to Sonar Dashboard

Currently we're using Jenkins free style job for Gradle project and using following commands to run Sonar and Dependencycheck
./gradlew clean build sonarqube dependencyCheckAnalyze \
and I'm getting following message
Analyzing /opt/jenkins_slave_home/workspace/AA/package-lock.json - however, the node_modules directory does not exist. Please run npm install prior to running dependency-check
Generating report for project AA_ArbitraryBuild
Found 0 vulnerabilities in project AA
and we can able to see a file inside "ws/build/reports/" but it dint scanned anything.
Following are the "build.gardle" file
buildscript {
repositories {
maven { url artifactoryRepoUrl }
mavenCentral()
}
dependencies {
classpath 'org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.7'
classpath 'org.owasp:dependency-check-gradle:6.0.3'
}
}
apply plugin: 'org.sonarqube'
apply plugin: 'org.owasp.dependencycheck'
sonarqube {
properties {
property 'sonar.projectName', sonarProjectName
property 'sonar.projectKey', sonarProjectKey
property 'sonar.host.url', sonarHostUrl
property 'sonar.login', sonarAuthToken
property 'sonar.dependencyCheck.reportPath', sonarDependencyCheckReport
property 'sonar.dependencyCheck.htmlReportPath', sonarDependencyCheckHTMLReport
}
}
Can you plz help on what are the additional steps that I need to add.
You've got all you need to push result to sonar. Make sure that you provide right path for your owasp vulnerabilities report for sonar plugin. It's sonar.dependencyCheck.reportPath and should point to build/reports directroy, and if you produce html report file you can point it with sonar.dependencyCheck.htmlReportPath.

SonarQube not showing test Jacoco coverage for JUnit tests in a Gradle multi-project

I would like to show the test coverage of a multiple project Spring boot application build with Gradle 6.0. We currently use JUnit5.
The test coverage shows 0% in SonarQube even though a few first tests exists.
The build.gradle files in the top level project (https://github.com/OpenReqEU/eclipse-plugin-vogella/blob/master/server/build.gradle) has the following input:
plugins {
id "org.sonarqube" version "2.7"
id 'jacoco'
}
repositories {
jcenter()
}
subprojects {
apply plugin: 'jacoco'
apply plugin: 'java'
apply plugin: 'eclipse'
repositories {
jcenter()
maven { url 'https://repo.spring.io/snapshot' }
maven { url 'https://repo.spring.io/milestone' }
maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
maven { url 'http://oss.jfrog.org/artifactory/oss-snapshot-local/' }
}
jacocoTestReport {
reports {
xml.enabled true
xml.destination file("${buildDir}/coverage-reports/coverage.xml")
//html.destination file("${buildDir}/coverage-reports")
}
}
ext {
springBootVersion = '2.1.1.RELEASE'
}
sourceCompatibility = 1.8
}
wrapper {
gradleVersion = '6.0'
}
In the Jenkins build we set the following parameters:
sonar.projectKey=eclipse-plugin-vogella
sonar.sources=server/com.vogella.prioritizer.server/src/main,server/com.vogella.prioritizer.server.bugzilla/src/main,server/com.vogella.prioritizer.server.issue.api/src/main
sonar.java.binaries=com.vogella.prioritizer.server/build/classes/java/main,com.vogella.prioritizer.server.bugzilla/build/classes/java/main,com.vogella.prioritizer.server.issue.api/build/classes/java/main
sonar.tests=server/com.vogella.prioritizer.server/src/test,server/com.vogella.prioritizer.server.bugzilla/src/test
sonar.coverage.jacoco.xmlReportsPath=server/com.vogella.prioritizer.server.bugzilla/build/jacoco/test.exec,server/com.vogella.prioritizer.server/build/jacoco/test.exec,server/com.vogella.prioritizer.server.issue.api/build/jacoco/test.exec
The result of the build shows an error:
INFO: parsing [/home/jenkins/workspace/issue-prioritizer/coverage-reports/coverage.xml]
ERROR: Reports path not found or is not a directory: /home/jenkins/workspace/issue-prioritizer/coverage-reports/coverage.xml
I see that each project has a generated ${buildDir}/coverage-reports/coverage.xml file but the root file is empty, which is expected as I did not configure anything related to this.
At some point I added a copy task which copied one of the generated xml files from one project into the root folder but the build job complained that the classes were not matching.
Does anybody know how this issue can be solved? I assume I must add a configuration to add a root coverage.xml file which is the aggregate of the individual ones but I have not found a solution for that.
I also tried to apply the jacoco to the root project but that also failed as the root project is not a Java project.
Need to generate report in xml format. Add the sonar property to the xml path as below.
jacocoTestReport {
reports {
xml.enabled true
}
}
sonarqube {
properties {
property "sonar.java.source", "1.8"
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.jacoco.reportPaths", "build/reports/jacoco/test/jacocoTestReport.xml"
}
}
Run the gradle command with the jacocoTestReport
gradlew sonarqube jacocoTestReport
I managed to create the aggregated coverage.xml file by changing the top level build.gradle to:
plugins {
// id "org.sonarqube" version "2.7"
id 'jacoco'
}
repositories {
jcenter()
}
subprojects {
println name;
apply plugin: 'jacoco'
apply plugin: 'java'
apply plugin: 'eclipse'
repositories {
jcenter()
maven { url 'https://repo.spring.io/snapshot' }
maven { url 'https://repo.spring.io/milestone' }
maven { url 'https://oss.sonatype.org/content/repositories/snapshots/' }
maven { url 'http://oss.jfrog.org/artifactory/oss-snapshot-local/' }
}
jacocoTestReport {
reports {
xml.enabled true
}
}
ext {
springBootVersion = '2.1.1.RELEASE'
}
sourceCompatibility = 1.8
}
// run the build with ./gradlew clean build generateMergedReport
task generateMergedReport(type: JacocoReport) {
dependsOn = subprojects.test
additionalSourceDirs.setFrom files(subprojects.sourceSets.main.allSource.srcDirs)
sourceDirectories.setFrom files(subprojects.sourceSets.main.allSource.srcDirs)
classDirectories.setFrom files(subprojects.sourceSets.main.output)
executionData.setFrom project.fileTree(dir: '.', include: '**/build/jacoco/test.exec')
println executionData;
reports {
xml.enabled true
xml.destination file("../coverage-reports/coverage.xml")
}
}
wrapper {
gradleVersion = '6.0'
}
And changing the Jenkins to build generateMergedReport:
cd server && ./gradlew build generateMergedReport
The SonarQube properties where changed to:
sonar.projectKey=eclipse-plugin-vogella
sonar.sources=server/com.vogella.prioritizer.server/src/main,server/com.vogella.prioritizer.server.bugzilla/src/main,server/com.vogella.prioritizer.server.issue.api/src/main
sonar.java.binaries=com.vogella.prioritizer.server/build/classes/java/main,com.vogella.prioritizer.server.bugzilla/build/classes/java/main,com.vogella.prioritizer.server.issue.api/build/classes/java/main
Unfortunately SonarQube still doesnt find the coverage.xml file.
INFO: parsing [/home/jenkins/workspace/issue-prioritizer/coverage-reports/coverage.xml]
ERROR: Reports path not found or is not a directory: /home/jenkins/workspace/issue-prioritizer/coverage-reports/coverage.xml
Does someone have an idea of what is missing?

How to publish a WAR file to maven (Nexus) repository with Jenkins via Gradle task

I'm struggling with deploying the war file to Nexus repository using Jenkinsfile via Gradle task.
The war is being created successfully. I have also no problem with deploying JARs (since there are examples everywhere how to do it).
So I have this publishing section in my build.grade:
publishing {
repositories {
maven {
URI releasesUrl = new URI("${UploadURL}/repository/releases")
URI snapshotsUrl = new URI("${UploadURL}/repository/snapshots")
afterEvaluate {
url version.endsWith("SNAPSHOT") ? snapshotsUrl : releasesUrl
}
credentials {
username "${user}"
password "${password}"
}
}
}
publications {
mavenWeb(MavenPublication) {
from components.web
artifact war.archivePath
}
}
}
With pluggins:
apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'maven-publish'
apply plugin: 'org.springframework.boot'
apply plugin: 'war'
The URL for repositories is also specified in the build script correctly (test publish with the jar works just fine)
And the Jenkinsfile:
stage ('Publish war') {
steps {
sh "sh gradlew publish"
}
}
Currently I'm getting this error from jenkins build:
Task :publishMavenWebPublicationToMavenRepository FAILED
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':publishMavenWebPublicationToMavenRepository'.
Failed to publish publication 'mavenWeb' to repository 'maven'
Invalid publication 'mavenWeb': multiple artifacts with the identical extension and classifier ('war', 'null').
I'm quite sure that the problem is within "publications" part of Gradle task.
For publishing the Jars I have been using it like this:
[...]
publications {
mavenJava(MavenPublication) {
from components.java
artifact sourceJar
}
}
[...]
task sourceJar(type: Jar) {
classifier 'sources'
from sourceSets.main.java
}
I do not know how to configure from, artifact and classifier for this task. I do not even know if all of these parameters should be configured... Could anyone help me with that?
It turned out, that the origin of the problem was this section:
afterEvaluate {
url version.endsWith("SNAPSHOT") ? snapshotsUrl : releasesUrl
}
This feature works with Gradle 5.X version however, I was using Gradle 4.8. That lead to null instead of propper url value...
Unfortunately, it took a while since the exception message does not suggest where the problem was.

Compile groovy project and run JUnit tests via Jenkins

I googled for ages now and I give up, the buzz word Groovy + Jenkins is bringing up so many false flags...
I have a Groovy project I developed in IntelliJ, it contains also a JUnit.groovy with unit tests. Now this is a script for SoapUI, no need for Maven, Ant nor Grails, but I would like to be able to compile those files on Jenkins and run the unit tests after. Is it possible to build and test those files on Jenkins? So far all solutions seem to be me manually running groovyc (commited with my repository) and then running JUnit on the JUnit.class.
So before I start to dig deeper and write a Maven, Grails or Ant file, is there another way that does not involve me pushing the GroovySDK on my git? Or is there may be a simple build script, not involving 20 libraries and steps that would build the groovy sources and run the JUnit tests :) ?
I'm new to Jenkins obviously ;), thanks for your input.
Update:
So for all as newbie as me, what was required? First I changed my local source code to a gradle project (remember to activate AutoImport in IntelliJ) and also activate the creation of the JUnit xml and since I do not use Maven and the system is "offline" we have the libs in git anyway so my build.gradle is:
version '2.5-SNAPSHOT'
apply plugin: 'groovy'
dependencies {
compile fileTree(dir: '../Library', include: ['*.jar'])
}
test {
reports {
junitXml.enabled = true
html.enabled = true
}
}
sourceCompatibility = 1.8
set up gradle wrapper for the project via gradle wrapper for the gradlew.bat
then I added a post-commit in my git-/.hooks/ so my Jenkins is triggered upon commit via curl http://jenkins:8080/git/notifyCommit?url=https://git.git&branches=dev
finally set up a pipeline on jenkins:
#!groovy
node {
stage('Checkout') {
git branch: 'dev', credentialsId: 'youwish', url: 'https://git.git'
}
stage('Build') {
dir('./Modules') {
gradle('clean')
gradle('compileTestGroovy')
}
}
stage('UnitTest') {
dir('./Modules') {
gradle('test')
junit '/build/test-results/**/TEST-*.xml'
}
}
stage('IntegrationTest') {
stage('CodeTableDownload') {
dir('./SoapUi') {
bat 'AutoRun.bat'
junit '/results/**/*-JUNIT.xml'
}
}
}
}
def gradle(command) {
bat "./gradlew.bat $command"
}
There's a Groovy plugin for Jenkins that will let you execute Groovy scripts on Jenkins.
But, why not let something like Gradle do the build and run the test for you? A minimal Gradle build file for Groovy that will do both is:
apply plugin: 'groovy'
repositories {
jcenter()
}
dependencies {
compile 'org.codehaus.groovy:groovy-all:2.4.12'
testCompile 'junit:junit:4.12'
}
You don't have to commit the GDK, just declare a dependency.

How perform OpenJPA Enhancement when using Gradle?

I've tried this gradle plugin https://github.com/schmutterer/gradle-openjpa but it complains that it cannot find certain libraries and doesn't support providedCompile which makes this unusable for me anyway.
I've also tried calling ANT tasks, my latest attempt below is throwing:
Caused by: C:\Work_Java\workspace\PaxHoldRelease\jpa_enhance.xml:5: taskdef class org.apache.openjpa.ant.PCEnhancerTask cannot be found
build.gralde
apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'ear'
// Java compilier compliance level
sourceCompatibility = 1.7
targetCompatibility = 1.7
repositories {
mavenLocal()
mavenCentral()
}
ant.importBuild 'jpa_enhance.xml'
war.dependsOn enhance
dependencies {
// Ensure ear plugin gets war file
deploy files(war)
providedCompile 'javax.servlet:javax.servlet-api:3.0.1'
compile 'javax.websocket:javax.websocket-api:1.1'
compile 'org.glassfish.jersey.containers:jersey-container-servlet:2.16'
compile 'com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.5.1'
compile 'org.glassfish:javax.json:1.0.4'
providedCompile 'org.apache.openjpa:openjpa:2.2.2'
providedCompile 'com.sybase:jconn3:6.05'
providedCompile files('libs/sqljdbc4-3.0.jar')
}
jpa_enhance.xml
This is the latest version in a long list of attempts and probably complete rubbish as I just ripped everything out in a fit of desperation :-(
<project>
<target name="enhance">
<taskdef name="openjpac" classname="org.apache.openjpa.ant.PCEnhancerTask"/>
<!-- invoke enhancer on all .java files below the model directory -->
<openjpac>
</openjpac>
<echo message="Enhancing complete!"/>
</target>
</project>
Try this Andrew - I loosely based this gradle on the nice Enhancer script provided on S.O. by another member (for the DataNucleus enhancer).
Note that you will need to modify the entity-files (include/exclude) to point to your specific 'to be/to not be' enhanced Java source files. Further, this approach assumes that classpath derives from your parent build.gradle.
task openJPAEnhance {
description "Enhance JPA model classes using OpenJPA Enhancer"
dependsOn compileJava
doLast {
// define the entity classes
def entityFiles = fileTree(sourceSets.main.output.classesDir).matching {
include 'org/foo/mypkg/entity/*.class'
exclude 'org/foo/mypkg/entity/DoNotEnhance.class'
}
println "Enhancing with OpenJPA, the following files..."
entityFiles.getFiles().each {
println it
}
// define Ant task for Enhancer
ant.taskdef(
name : 'openjpac',
classpath : sourceSets.main.runtimeClasspath.asPath,
classname : 'org.apache.openjpa.ant.PCEnhancerTask'
)
// Run the OpenJPA Enhancer as an Ant task
// - see OpenJPA 'PCEnhancerTask' for supported arguments
// - this invocation of the enhancer adds support for a default-ctor
// - as well as ensuring JPA property use is valid.
ant.openjpac(
classpath: sourceSets.main.runtimeClasspath.asPath,
addDefaultConstructor: true,
enforcePropertyRestrictions: true) {
entityFiles.addToAntBuilder(ant, 'fileset', FileCollection.AntType.FileSet)
}
}
}
I hope this helps, and the individual who wrote that first gradle script did not mind that we re-purposed it (from DataNucleus) to OpenJPA.

Resources