I'm using Cobertura 1.9.4 with NetBeans 7.2.1, Ant 1.8.3 and JDK 1.7.0_05 with JUnit 4.10.
My HTML coverage reports are being generated, but they all list 0% unit test coverage. This shouldn't be the case because I have unit test that cover my entire util class.
What's interesting is all my unit test start to fail if I include the instrumented classes in my test path by changing
run.test.classpath=\
${javac.test.classpath}:\
${build.test.classes.dir}
to
run.test.classpath=\
${build.instrumented.dir}:\
${javac.test.classpath}:\
${build.test.classes.dir}
Also, if I turn on the emma code coverage tool all my test cases will break as well.
NOTE: There is a nearly identical post Cobertura showing 0% coverage but since it is unanswered, slightly dated and my configuration differs I thought I'd capture my setup here. Here are two references that are similar to my config: 1)cobertura pdf 2)NetBeans CoberaturaAnt Wiki
Snipped from my build.xml
<!-- this strangely doesn't work so i'm temporarily using hardcoded path below
<taskdef classpath="cobertura.jar" resource="tasks.properties" />
-->
<path id="cobertura.class.path">
<pathelement location="libs/cobertura.jar" />
<pathelement location="libs/asm-3.0.jar" />
<pathelement location="libs/asm-tree-3.0.jar" />
<pathelement location="libs/log4j-1.2.9.jar" />
<pathelement location="libs/jakarta-oro-2.0.8.jar" />
</path>
<taskdef resource="tasks.properties" classpathref="cobertura.class.path" />
<target depends="-init-check,init" name="-init-check-cobertura">
<fail unless="build.classes.dir">Must set build.classes.dir</fail>
</target>
<target name="-post-compile" depends="-init-check-cobertura,init">
<delete dir="${build.instrumented.dir}" />
<delete dir="${build.test.coverage.dir}" />
<mkdir dir="${build.instrumented.dir}" />
<mkdir dir="${build.test.coverage.dir}" />
<cobertura-instrument todir="${build.instrumented.dir}"
datafile="${build.dir}/cobertura.ser">
<fileset dir="${build.classes.dir}">
<include name="**/*.class"/>
</fileset>
</cobertura-instrument>
</target>
<target name="coverage-report" depends="test">
<cobertura-report format="html" destdir="${build.test.coverage.dir}"
datafile="${build.dir}/cobertura.ser"/>
<delete file="${build.dir}/cobertura.ser" />
<delete dir="${build.instrumented.dir}" />
</target>
I've also defined these in my project.properties file
build.instrumented.dir=${build.dir}/instrumented
build.test.coverage.dir=${build.dir}/test/coverage
FYI I already tried adding these to my tag in build-impl.xml. still zero coverage.
<jvmarg value="-Dcobertura.use.java.nio=false"/>
<jvmarg value="-Dnet.sourceforge.cobertura.datafile=${basedir}/cobertura.ser"/>
The ant test target is from the netbeans generated build-impl.xml file
<target depends="init,compile-test,-pre-test-run,-do-test-run,test-report,-post-test-run,-test-browse"
description="Run unit tests." name="test"/>
Related
I am new to Ant and getting java.lang.ClassNotFoundException. when running my Junit from eclipse / Ant build.xml, However, when running my unit test from eclipse by itself my test passes with no issue. There should be something wrong with my classpath which i cannot figure out.
My envrmnts are:
Java_home: C:/Program Files/Java/jdk1.7.0_25
Ant_home: C:/Users/Armen/javaFolder/apache-ant-1.9.2/bin
JUnit_home: C:/Users/Armen/javaFolder/apache-ant-1.9.2/bin/junit-4.10.jar
My Build.xml
<property name="junitLocation">C:/Users/Armen/javaFolder/apache-ant-1.9.2/bin/junit-4.10.jar</property>
<property name="antLocation2">C:/Users/Armen/JavaFolder/apache-ant-1.9.2.jar</property>
<property name="testCode">C:/Users/Armen/workspace/MockingObjects/test/demo</property>
<property name="srcCode">C:/Users/Armen/workspace/MockingObjects/src/demo</property>
<target name="compile" depends="clean">
<javac includeantruntime="false" srcdir="./src" destdir="./staging" ></javac>
</target>
<target name="run" depends="compile, unitTest">
<java classname="demo.AccountService"><classpath path="./staging"></classpath></java>
</target>
<target name="unitTest" depends="compile">
<junit printsummary="true" showoutput="true" haltonfailure="false" fork="yes">
<formatter type="plain" usefile="true"/>
<test name="demo.TestAccountService" outfile="./result" ></test>
<classpath>
<pathelement location="${junitLocation}"/>
<pathelement location="${antLocation}"/>
<pathelement location="${testCode}" />
<pathelement location="${srcCode}"/>
</classpath>
</junit>
</target>
<target name="clean">
<delete dir="staging"></delete>
<mkdir dir="./staging"/>
</target>
enter image description here
You don't have any step in your build script to compile the unit tests, and when you tell JUnit to run, you also don't include the compiled classes in the classpath - only the source files.
There are therefore 2 things you need to do:
Add an additional build step similar to the following
<target name="test-compile" depends="compile">
<javac includeantruntime="false" srcdir="./test" destdir="./test-classes" />
</target>
then change your current unitTest target to depend on test-compile rather than compile.
Tell JUnit where your classes are, not your source code. Change your testCode and srcCode properties to
<property name="testCode">C:/Users/Armen/workspace/MockingObjects/test-classes</property>
<property name="srcCode">C:/Users/Armen/workspace/MockingObjects/staging</property>
Note From your compile and run steps, it isn't clear if your code is properly structured in Java Packages or, if it is in packages, that you understand how these work. I've made the assumption that your code is in a package of demo and therefore stripped that part of the path out of your compiled class locations.
The exact message received from jenkins is:
No test report files were found. Configuration error?
Build step 'Publish JUnit test result report' changed build result to FAILURE
When configuring the JUnit Test Result Report plugin, on entering the 'Test Report XMLs' path as '/reports/TEST-*.xml', the following error is displayed beneath the path:
'/reports/TEST-*.xml' doesn't match anything: '' exists but not '/reports/TEST-*.xml'
I have tried using the full path as well but that produces the same result. In both cases the paths should have picked up the 'TESTS-TestSuites.xml' file that was present in the /reports directory.
I'm not sure whether this is a problem with the plugin or the XML file being generated. I'm also aware that it could be an issue with the ant build script that I have written to run the JUnit tests and produce the XML result file therefore I have included the contents of this below in case something needs to be changed:
<?xml version="1.0" encoding="utf-8"?>
<project name="jenkins-tests" basedir="." default="linux">
<property name="junit.output.dir" value="output"/>
<property name="src.dir" value="src"/>
<property name="lib.dir" value="libs" />
<property name="bin.dir" value="bin" />
<property name="full-compile" value="true" />
<path id="classpath.base"/>
<path id="classpath.test">
<pathelement location="${bin.dir}" />
<pathelement location="${src.dir}" />
<pathelement location="${lib.dir}" />
<pathelement location="${lib.dir}/junit.jar" />
<path refid="classpath.base" />
</path>
<target name="clean" description="Clean up build artefacts">
<delete dir="${basedir}/${junit.output.dir}" />
</target>
<target name="prepare" depends="clean" description="Prepare for build">
<mkdir dir="${basedir}/${junit.output.dir}" />
<mkdir dir="${junit.output.dir}/reports"/>
</target>
<target name="compile" depends="prepare">
<javac srcdir="${src.dir}" destdir="${bin.dir}" verbose="${full-compile}" includeAntRuntime="false" >
<classpath refid="classpath.test"/>
</javac>
</target>
<target name="test" depends="compile">
<junit printsummary="true" haltonfailure="false">
<formatter type="xml" usefile="true"/>
<classpath refid="classpath.test" />
<batchtest fork="yes" todir="${junit.output.dir}">
<fileset dir="${src.dir}">
<include name="*.java"/>
</fileset>
</batchtest>
</junit>
</target>
<target name="test-reports" depends="test">
<junitreport tofile="TESTS-TestSuites.xml" todir="${junit.output.dir}/reports">
<fileset dir="${junit.output.dir}">
<include name="TEST-*.xml" />
</fileset>
<report format="frames" todir="${junit.output.dir}/reports" />
</junitreport>
</target>
</project>
I've been researching into this problem for a while now and haven't found any solution so I would appreciate any help. Thanks.
Jenkins looks for the path from the workspace root. Ensure that the given path is correct or use wildcards to look in multiple locations. Try using **/reports/TEST-*.xml
Are you sure the reports folder is right under the workspace? Verify manually if the test result files are indeed present in the location given in the path.
For my Android project which has multiple Gradle product flavors I used the following path for Test report XMLs:
**/build/test-results/**/TEST-*.xml
I have problem with ant sonar task. This task end with success but don't run unit tests and don't show code coverage.
Test task
<target name="test" depends=".....">
<path id="classpath">
<fileset dir="test/lib" includes="**/*.jar"/>
<fileset dir="lib" includes="**/*.jar"/>
<pathelement location="......."/>
</path>
<mkdir dir="build/tests"/>
<javac srcdir="test/src" destdir="build/tests" includes="**/*.java" debug="${debug}" deprecation="${deprecation}" optimize="${optimize}" nowarn="${nowarn}" fork="true">
<classpath refid="classpath"/>
</javac>
<copy todir="build/tests">
<fileset dir="test/src" excludes="**/*.java"/>
</copy>
<jacoco:coverage destfile="target/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
<junit printsummary="yes" fork="true" haltonfailure="false" showoutput="true" failureproperty="test.failed">
<formatter type="xml"/>
<classpath refid="classpath"/>
<classpath>
<pathelement location="build/tests"/>
</classpath>
<test name="com........MainTestSuite" todir="build"/>
</junit>
</jacoco:coverage>
<fail message="Test failure detected, check test results." if="test.failed"/>
</target>
and sonar task:
<target name="sonar" depends="build">
<property name="sonar.tests" value="test" />
<property name="sonar.libraries" value="" />
<property name="sonar.surefire.reportsPath" value="sonarWorkDir" />
<!-- The following properties are required to use JaCoCo: -->
<!-- 1. Tells Sonar to run the unit tests -->
<property name="sonar.dynamicAnalysis" value="true" />
<!-- 2. Tells Sonar which "tests" targets to run -->
<property name="sonar.jacoco.antTargets" value="test" />
<!-- 3. Tells Sonar to use JaCoCo as the code coverage engine -->
<property name="sonar.core.codeCoveragePlugin" value="jacoco" />
<!-- Execute Sonar -->
<sonar:sonar key="${JOB_NAME}" version="${VERSION}" xmlns:sonar="antlib:org.sonar.ant">
<sources>
<path location="...../src" />
</sources>
<binaries>
<path location="build/....." />
</binaries>
</sonar:sonar>
</target>
until sonar task runs i got a warning 10:00:20.290 WARN o.s.p.j.JaCoCoPlugin - Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
I would advise you to:
use the latest version of the Sonar Ant Task (from what I can see in your script, you're using the old syntax)
take a look at our sample Ant projects to find out where your issue is. Most probably, the following example is what you're looking for: https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests
Kind of a late response, but I just recently ran into this issue and couldn't find a simple solution on the internet. Instead, to solve it I simply took the advice of the stack trace. Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
And then went back to my ant file and made sure the source files (not test) were being compiled in debug mode. Apparently the jacoco plugin needs that extra info like line numbers in order to calculate the code coverage. Once you change your ant file you should finally see a code coverage percentage in sonar.
<javac srcdir="${source.dir}" target="1.6" source="1.6" destdir="${build.classes.dir}" debug="on">
...
</javac>
Also, make sure you include the following sonar properties :
sonar.binaries=build/classes
sonar.tests=junit/tests
sonar.dynamicAnalysis=reuseReports
sonar.junit.reportsPath=build/test-reports
sonar.java.coveragePlugin=jacoco
sonar.jacoco.reportPath=build/test-reports/jacoco.exec
So you probably want to remove sonar.antTargets and change sonar.dynamicAnalysis's value to reuseReports
I am newbie to use Emma. I am trying to add emma ant task for JUnit test case for modules in EAR project. I have few question here.
Should I use instrumented class for packaging my EAR projet?
What is good way to add emma ant task for junit? Should I use emmarun:on-th-fly mode or offline mode? For JUnit should I use fork or no fork?
I am using Emma Offline mode and Junit with fork. Here is my build.xml
<!--Target and task for EMMA -->
<taskdef resource="emma_ant.properties" classpathref="Emma.libraryclasspath" />
<target name="emma" description="turns on EMMA's instrumentation/reporting" >
<property name="emma.enabled" value="true" />
<mkdir dir="${out.instr.dir}" />
<property name="emma.filter" value="" />
</target>
<target name="test" depends="init, compile" description="Run JUnit Test cases under emma environment">
<!-- Emma instrumentation -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${class.dir}"
destdir="${out.instr.dir}"
metadatafile="${coverage.dir}/metadata.em"
merge="true"
mode="copy">
<filter value="${emma.filter}" />
</instr>
</emma>
<!-- JUnit Start -->
<junit printsummary="yes" fork="yes">
<test name="com.hf.platform.authorizer.WebTxnAuthorizerTest" todir="${test.report.dir}">
<formatter type="xml"/>
</test>
<classpath>
<path refid="HFPlatformWeb.classpath"/>
<path refid="Emma.libraryclasspath"/>
</classpath>
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.ec" />
<jvmarg value="-Demma.coverage.out.merge=false" />
</junit>
<!-- Junit End -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<report>
<sourcepath>
<dirset dir="${basedir}">
<include name="src"/>
<include name="test-src"/>
</dirset>
</sourcepath>
<fileset dir="${coverage.dir}">
<include name="*.em"/>
<include name="*.ec"/>
</fileset>
<xml outfile="${coverage.report.dir}/report.xml" />
<txt outfile="${coverage.report.dir}/report.txt" />
<html outfile="${coverage.report.dir}/report.html" />
</report>
</emma>
</target>
When I ran it for one test, it is not generating any report. But when i ran same unit test with EclEmma it gives correct output.
In above example we need to make sure following two things
The file path for metadatafile and coverage report file that is .ec, .em or .emma file should be absolute or relative to project.
e.g.
For running java/junit task sandwiched between the instrumentation and report task, it must use instrumented class file path.
e.g.
<classpath> <pathelement location="${out.instr.dir}" />
<path refid="Emma.libraryclasspath"/>
<path refid="HFPlatformEJB.classpath"/>
</classpath>
I tried to get Cobertura running inside my ant script. All is successfull (source code building, junit tests, cobertura reports (xml / html); but in html reports, the code coverage is always at 0% ...
Ant Script : make-instrument
<!-- Make instrument for Cobertura engine -->
<target name="make-instrument">
<!-- Remove the coverage data file and any old instrumentation. -->
<delete file="${cobertura.ser}" />
<!-- Instrument the application classes, writing the instrumented classes into ${build.instrumented.dir}. -->
<cobertura-instrument todir="${report.cobertura.dir}">
<!-- The following line causes instrument to ignore any source line containing a reference to log4j,
for the purposes of coverage reporting. -->
<ignore regex="org.apache.log4j.*" />
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<!-- Instrument all the application classes, but don't instrument the test classes. -->
<include name="**/*.class" />
<exclude name="**/*Test.class" />
</fileset>
</cobertura-instrument>
</target>
Ant Script : make-instrument
<target name="install-cobertura" if="is-hudson-env">
<path id="cobertura.classpath">
<fileset dir="${user.home.sharehunter.dir}/cobertura-${cobertura.rev}">
<include name="**/cobertura.jar" />
<include name="**/*.jar" />
</fileset>
</path>
<taskdef resource="tasks.properties" classpathref="cobertura.classpath" />
</target>
Ant Script : junit
<target name="run-tests" depends="make-instrument">
<path id="classpath.test">
<path path="${webcontent.dir}/WEB-INF/classes" />
<pathelement location="${webcontent.dir}/WEB-INF/classes" />
<fileset dir="${lib.dir}" includes="**/*.jar" />
<path location="${webcontent.dir}/WEB-INF/classes" />
<path location="${webcontent.dir}/WEB-INF" />
<path location="${webcontent.dir}" />
</path>
<junit fork="yes" failureProperty="test.failed">
<classpath refid="classpath.test" />
<classpath location="${user.home.dir}/junit-${junit.rev}.jar" />
<!-- Specify the name of the coverage data file to use.
The value specified below is the default. -->
<sysproperty key="net.sourceforge.cobertura.datafile" file="${cobertura.ser}" />
<!-- Note the classpath order: instrumented classes are before the original (uninstrumented) classes. -->
<classpath location="${report.cobertura.dir}" />
<!--
The instrumented classes reference classes used by the
Cobertura runtime, so Cobertura and its dependencies
must be on your classpath.
-->
<classpath refid="cobertura.classpath" />
<!-- Generate xml files for each junit tests runs -->
<formatter type="xml" />
<batchtest todir="${report.junit.dir}">
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<include name="**/*Test.class" />
</fileset>
</batchtest>
</junit>
<!-- Generate Cobertura xml file containing the coverage data -->
<cobertura-report format="xml" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
<!-- Generate Cobertura html file report containing the coverage data -->
<cobertura-report format="html" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
</target>
This is what Cobertura FAQ Says
When I generate coverage reports, why do they always show 0% coverage everywhere?
Cobertura is probably using the wrong .ser file when generating the reports. When you instrument your classes, Cobertura generates a .ser file containing basic information about each class. As your tests run, Cobertura adds additional information to this same data file. If the instrumented classes can not find the data file when running then they will create a new one. It is important that you use the same cobertura.ser file when instrumenting, running, and generating reports.
The best way to do this is to specify the location of the data file when running your tests. You should pass the -Dnet.sourceforge.cobertura.datafile=${basedir}/cobertura.ser sysproperty to the JUnit task.
Another common problem is that the cobertura.ser file is deleted, but the previously instrumented classes are not also deleted. Any time you delete your coverage data file you should also deleted all instrumented classes.
Ok I found the problem. To be sure have this :
<!--
Note the classpath order: instrumented classes are before the
original (uninstrumented) classes. This is important.
-->
<classpath location="${instrumented.dir}" />
<classpath location="${classes.dir}" />
Instrumented classes must are before the original (uninstrumented) classes.
I tried similar way. I also used instrumented code before actual source code, but I am getting 0 % in the report file.
<macrodef name="coberturaTestMacro">
<attribute name="moduleName" />
<attribute name="classpath.module" />
<attribute name="classpath.junit" />
<attribute name="failOnCoverageFall" />
<attribute name="fileCoberturaData"/>
<sequential>
<path id="classpathCobertura">
<fileset dir="${homeCobertura}">
<include name="cobertura.jar" />
<include name="lib/**/*.jar" />
</fileset>
</path>
<taskdef classpathref="classpathCobertura" resource="tasks.properties" />
<property name="cob.instrumented.dir" value="target/cobertura/instrumented" />
<delete dir="target/cobertura" />
<cobertura-instrument todir="${cob.instrumented.dir}" datafile="#{fileCoberturaData}" >
<fileset dir="target/classes">
<include name="**/*.class" />
</fileset>
</cobertura-instrument>
<delete dir="target/reports/test" />
<mkdir dir="target/cobertura/reports" />
<junit printsummary="false" failureproperty="junit.failure"
maxmemory="512m" fork="true" forkmode="perTest">
<jvmarg value="-Djava.awt.headless=true" />
<classpath location="${homeCobertura}/cobertura.jar" />
<classpath location="${cob.instrumented.dir}" />
<classpath>
<path refid="#{classpath.module}" />
<path refid="#{classpath.junit}" />
</classpath>
<classpath path="target/test-classes" />
<batchtest todir="target/cobertura/reports/">
<fileset dir="src/test/java">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<cobertura-report srcdir="src/main/java" destdir="target/cobertura/reports/" />
<echo message="${line.separator}" />
<echo message="COVERAGE: #{moduleName} module..." />
<echo message="${line.separator}" />
<if>
<available file="target/cobertura/#{moduleName}-cobertura.properties" />
<then>
<var name="total.line-rate" file="target/cobertura/#{moduleName}-cobertura.properties" />
<cobertura-check haltonfailure="#{failOnCoverageFall}"
datafile="#{fileCoberturaData}" totallinerate="${total.line-rate}" />
</then>
</if>
<delete file="${dirBuild}/coverage-summary.properties" />
<cobertura-report datafile="#{fileCoberturaData}" destdir="target/cobertura/" format="summaryXml" />
<var name="total.line-rate" file="target/cobertura/coverage-summary.properties" />
<echo message="Total line coverage: ${total.line-rate}%" />
<propertyfile file="target/cobertura//#{moduleName}-cobertura.properties">
<entry key="total.line-rate" value="${total.line-rate}" type="int" />
</propertyfile>
</sequential>
</macrodef>
The surprising thing is that the generated report says total 2% coverage, but summary file says 0% coverage.
Where old cobertura task shows 8% coverage. I am totally confused :(
Probably it is not applicable to everybody but I had the similar issue where coverage was 0 for all classes.
There were 2 issues in my case
1) it was reading wrong jdk version 1.8 off of PATH. I updated PATH to read 1.6 jdk.
2) it was initially using version 1.8 of cobertura. I ran the build and it would generate coverage report but all the classes were 0% always. I updated the javac target to include
debug="true" debuglevel="vars,lines,source" reference: cobertura 0 coverage
Then ran the build again and saw that the there was an error when running tests and traced that back to an issue with version 1.8 of cobertura.
So, I upgraded
Cobertura 1.9.4
asm 3.1 from 2.2.1
asm-tree 3.1
other dependencies
1. jakarta-oro 2.0.8
2. log4j-1.2.9
After that ran the task again and the reports were alright.