I have run into this specific problem while executing the junit with jacoco:coverage for code coverage. Tried a few things but no luck yet.
I have this junit build script.
<target name="executeJunitMain" depends="createJunitLibs" description="Executes All Junit and prepare report.">
<junit fork="yes" haltonfailure="off" failureProperty="junit.failure" includeantruntime="true" maxmemory="256m">
<classpath refid="compile.class.path" />
<formatter type="xml" />
<jvmarg value="-javaagent:${external.junit.lib.dir}/jmockit.jar"/>
<sysproperty key="jacoco-agent.destfile" file="${coverage.dir}/jacoco.exec"/>
<batchtest fork="yes" todir="${report.dir}" >
<fileset dir="${dest.dir}">
<include name="**/Test*.class" />
<exclude name="**/AllTests.class" />
<exclude name="**/*$*.class"/>
</fileset>
</batchtest>
</junit>
<antcall target="report"/>
</target>
<!-- Execute the coverage report.-->
<target name="report" description="Collect Coverage Report">
<!-- Creates the Junit Report -->
<junitreport todir="${report.dir}">
<fileset dir="${report.dir}">
<include name="TEST-*.xml"/>
</fileset>
<report format="frames" todir="${report.dir}/html"/>
</junitreport>
<!-- Creates the coverage Report.-->
<mkdir dir="${report.dir}/coverage"/>
<jacoco:report>
<executiondata>
<file file="${coverage.dir}/jacoco.exec" />
</executiondata>
<structure name="Code Coverage">
<classfiles>
<zipfileset src="${sources.lib.dir}/${test.jar.name}.jar"/>
</classfiles>
</structure>
<html destdir="${report.dir}/coverage" />
</jacoco:report>
</target>
using this ant script sometimes the junit build is failing. Error as below
BUILD FAILED
C:\Users\user\Codebases\junit.xml:111: The following error occurred while executing this line:
C:\Users\user\Codebases\junit.xml:147: The following error occurred while executing this line:
C:\Users\user\Codebases\junit.xml:164: Unable to read execution data file C:\Users\user\CodeBases\ju
nit\coverage\jacoco.exec
But sometimes it works perfectly. I am not sure when it works and when it does not. Any help will be appreciated.
Thanks
I have found the issue that was causing the jacoco to fail. I had a few instrumented jars imported in the classpath which were not required for the junit execution or coverage. These instrumented jars were causing jacoco session to be incorrect as we also had jmockit integration as Mocking Framework. The below link was very helpful for finding out the issue
eclemma.org/jacoco/trunk/doc/classids.html
Related
I was trying to integrate code coverage on my project by using jacoco, ant and teamcity. However, I realized that when jacoco task is around the junit task, teamcity does not catch the failing tests and everything is a success even with test failed.
Here are my 2 test tasks to test with and without jacoco and see teamcity bahaviours.
1- with jacoco activated
<target name="-test">
<echo message="JaCoCo activated"/>
<!-- Import the JaCoCo Ant Task -->
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml"/>
<!-- Run your unit tests, adding the JaCoCo agent -->
<jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
<junit fork="yes" printsummary="yes" haltonfailure="no" showoutput="false" failureProperty="test.failed" errorProperty="test.failed">
<classpath>
<path location="${lib}/${projectName}.jar"/>
<path refid="project.classpath"/>
</classpath>
<formatter type="xml"/>
<batchtest todir="${reportingHome}">
<fileset dir="${test}">
<include name="**/*Test.java"/>
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
<copy todir="${completeReportDir}" overwrite="true">
<fileset dir="${reportingHome}">
<include name="*.xml"/>
</fileset>
</copy>
</target>
2- without jacoco
<target name="-test">
<echo message="JaCoCo activated"/>
<!-- Import the JaCoCo Ant Task -->
<taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml"/>
<!-- Run your unit tests, adding the JaCoCo agent -->
<!--<jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">-->
<junit fork="yes" printsummary="yes" haltonfailure="no" showoutput="false" failureProperty="test.failed" errorProperty="test.failed">
<classpath>
<path location="${lib}/${projectName}.jar"/>
<path refid="project.classpath"/>
</classpath>
<formatter type="xml"/>
<batchtest todir="${reportingHome}">
<fileset dir="${test}">
<include name="**/*Test.java"/>
</fileset>
</batchtest>
</junit>
<!--</jacoco:coverage>-->
<copy todir="${completeReportDir}" overwrite="true">
<fileset dir="${reportingHome}">
<include name="*.xml"/>
</fileset>
</copy>
</target>
Only jacoco task has been commented between the 2 releases of test.
Teamcity output
[CommonBuildTasks.-test] echo
[08:26:21]: [echo] JaCoCo activated
[08:26:21]: [CommonBuildTasks.-test] jacoco:coverage (4s)
[08:26:21]: [jacoco:coverage] Enhancing junit with coverage.
[08:26:22]: [jacoco:coverage] Running ca.thalesgroup.socialnetworkanalysisorchestrator.impl.client.SocialNetworkAnalysisOrchestratorServiceProviderTest
[08:26:25]: [jacoco:coverage] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 3.511 sec
[08:26:26]: [jacoco:coverage] Test ca.thalesgroup.socialnetworkanalysisorchestrator.impl.client.SocialNetworkAnalysisOrchestratorServiceProviderTest FAILED
[08:26:26]: [CommonBuildTasks.-test] copy
[08:26:26]: [copy] Copying 1 file to C:\TeamCity\buildAgent\work\cc10e09e43249f57\reports
As you can see, a test failed but teamcity has reported a successfull build.
Any idea why I got this behaviour?
Thanks
The answer is hidden in your call to the JUnit-Task:
<junit haltonfailure="no">...</junit>
With this configuration, the JUnit task does not fail the build on failing tests. This should lead to the desired behaviour:
<junit haltonfailure="yes">...</junit>
See the Ant documentation for the configuration of the JUnit task.
I solved this issue by using agent task instead of the coverage task. So, instead of
<jacoco:coverage destfile="${bin}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
Use:
<jacoco:agent property="agentvmparam" destfile="${bin}/jacoco.exec"/>
<junit fork="yes"...
<jvmarg value="${agentvmparam}"/>
</junit>
Agent task uses the same properties as the coverage task. Then you can start your junit task without wrapping it in coverage task. That way teamcity is able to intercept junit task output.
I have problem with ant sonar task. This task end with success but don't run unit tests and don't show code coverage.
Test task
<target name="test" depends=".....">
<path id="classpath">
<fileset dir="test/lib" includes="**/*.jar"/>
<fileset dir="lib" includes="**/*.jar"/>
<pathelement location="......."/>
</path>
<mkdir dir="build/tests"/>
<javac srcdir="test/src" destdir="build/tests" includes="**/*.java" debug="${debug}" deprecation="${deprecation}" optimize="${optimize}" nowarn="${nowarn}" fork="true">
<classpath refid="classpath"/>
</javac>
<copy todir="build/tests">
<fileset dir="test/src" excludes="**/*.java"/>
</copy>
<jacoco:coverage destfile="target/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">
<junit printsummary="yes" fork="true" haltonfailure="false" showoutput="true" failureproperty="test.failed">
<formatter type="xml"/>
<classpath refid="classpath"/>
<classpath>
<pathelement location="build/tests"/>
</classpath>
<test name="com........MainTestSuite" todir="build"/>
</junit>
</jacoco:coverage>
<fail message="Test failure detected, check test results." if="test.failed"/>
</target>
and sonar task:
<target name="sonar" depends="build">
<property name="sonar.tests" value="test" />
<property name="sonar.libraries" value="" />
<property name="sonar.surefire.reportsPath" value="sonarWorkDir" />
<!-- The following properties are required to use JaCoCo: -->
<!-- 1. Tells Sonar to run the unit tests -->
<property name="sonar.dynamicAnalysis" value="true" />
<!-- 2. Tells Sonar which "tests" targets to run -->
<property name="sonar.jacoco.antTargets" value="test" />
<!-- 3. Tells Sonar to use JaCoCo as the code coverage engine -->
<property name="sonar.core.codeCoveragePlugin" value="jacoco" />
<!-- Execute Sonar -->
<sonar:sonar key="${JOB_NAME}" version="${VERSION}" xmlns:sonar="antlib:org.sonar.ant">
<sources>
<path location="...../src" />
</sources>
<binaries>
<path location="build/....." />
</binaries>
</sonar:sonar>
</target>
until sonar task runs i got a warning 10:00:20.290 WARN o.s.p.j.JaCoCoPlugin - Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
I would advise you to:
use the latest version of the Sonar Ant Task (from what I can see in your script, you're using the old syntax)
take a look at our sample Ant projects to find out where your issue is. Most probably, the following example is what you're looking for: https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/ut/ant/ut-ant-jacoco-runTests
Kind of a late response, but I just recently ran into this issue and couldn't find a simple solution on the internet. Instead, to solve it I simply took the advice of the stack trace. Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
And then went back to my ant file and made sure the source files (not test) were being compiled in debug mode. Apparently the jacoco plugin needs that extra info like line numbers in order to calculate the code coverage. Once you change your ant file you should finally see a code coverage percentage in sonar.
<javac srcdir="${source.dir}" target="1.6" source="1.6" destdir="${build.classes.dir}" debug="on">
...
</javac>
Also, make sure you include the following sonar properties :
sonar.binaries=build/classes
sonar.tests=junit/tests
sonar.dynamicAnalysis=reuseReports
sonar.junit.reportsPath=build/test-reports
sonar.java.coveragePlugin=jacoco
sonar.jacoco.reportPath=build/test-reports/jacoco.exec
So you probably want to remove sonar.antTargets and change sonar.dynamicAnalysis's value to reuseReports
I am newbie to use Emma. I am trying to add emma ant task for JUnit test case for modules in EAR project. I have few question here.
Should I use instrumented class for packaging my EAR projet?
What is good way to add emma ant task for junit? Should I use emmarun:on-th-fly mode or offline mode? For JUnit should I use fork or no fork?
I am using Emma Offline mode and Junit with fork. Here is my build.xml
<!--Target and task for EMMA -->
<taskdef resource="emma_ant.properties" classpathref="Emma.libraryclasspath" />
<target name="emma" description="turns on EMMA's instrumentation/reporting" >
<property name="emma.enabled" value="true" />
<mkdir dir="${out.instr.dir}" />
<property name="emma.filter" value="" />
</target>
<target name="test" depends="init, compile" description="Run JUnit Test cases under emma environment">
<!-- Emma instrumentation -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${class.dir}"
destdir="${out.instr.dir}"
metadatafile="${coverage.dir}/metadata.em"
merge="true"
mode="copy">
<filter value="${emma.filter}" />
</instr>
</emma>
<!-- JUnit Start -->
<junit printsummary="yes" fork="yes">
<test name="com.hf.platform.authorizer.WebTxnAuthorizerTest" todir="${test.report.dir}">
<formatter type="xml"/>
</test>
<classpath>
<path refid="HFPlatformWeb.classpath"/>
<path refid="Emma.libraryclasspath"/>
</classpath>
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.ec" />
<jvmarg value="-Demma.coverage.out.merge=false" />
</junit>
<!-- Junit End -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<report>
<sourcepath>
<dirset dir="${basedir}">
<include name="src"/>
<include name="test-src"/>
</dirset>
</sourcepath>
<fileset dir="${coverage.dir}">
<include name="*.em"/>
<include name="*.ec"/>
</fileset>
<xml outfile="${coverage.report.dir}/report.xml" />
<txt outfile="${coverage.report.dir}/report.txt" />
<html outfile="${coverage.report.dir}/report.html" />
</report>
</emma>
</target>
When I ran it for one test, it is not generating any report. But when i ran same unit test with EclEmma it gives correct output.
In above example we need to make sure following two things
The file path for metadatafile and coverage report file that is .ec, .em or .emma file should be absolute or relative to project.
e.g.
For running java/junit task sandwiched between the instrumentation and report task, it must use instrumented class file path.
e.g.
<classpath> <pathelement location="${out.instr.dir}" />
<path refid="Emma.libraryclasspath"/>
<path refid="HFPlatformEJB.classpath"/>
</classpath>
I am trying some code-coverage analysis for first time and I was working on getting cobertura using ANT. My questions might be silly, but thought of asking here. I have the following in my ANT scripts. While reading over through cobertura the next step was instrumentation. What is code coverage instrumentation?
<target name="cobertura" depends="checkstyle">
<property name="cobertura.dir" location="C:\\Softwares- packages\\Corbetura\\cobertura-1.9.4.1" />
<path id ="cobertura.classpath">
<fileset dir="${cobertura.dir}">
<include name="cobertura.jar"/>
<include name="lib/**/*.jar"/>
</fileset>
</path>
<taskdef resource="tasks.properties" classpathref="cobertura.classpath"/>
</target>
cobertura modifies your class files so that it can compute the coverage. I typically 'instrument' a copy of the jar files that I use for executing tests and use a copy that hasn't been instrument as my build artifact.
Here is the build file I used when I first set up cobertura via ant:
The cobertura-instrument target instruments my code and writes the instrumented classes to a separate directory like you said.
The junit target compiles the test, then instruments the tests, then runs the tests, then produces the report. These steps are all done by declaring dependent targets to the junit one.
<path id="cobertura.classpath">
<fileset dir="${cobertura.dir}">
<include name="cobertura.jar" />
<include name="lib/**/*.jar" />
</fileset>
</path>
<taskdef classpathref="cobertura.classpath" resource="tasks.properties" />
<!-- Delete an existing coburtura datafile -->
<delete file="${cobertura.datafile}"/>
<antcall target="cobertura.clean"/>
<!-- Instrument the code with cobertura to test for coverage -->
<cobertura-instrument todir="${cobertura.instrumented.classes}" datafile="${cobertura.datafile}">
<fileset dir="${build.dir}/classes/">
<include name="**/*.class"/>
</fileset>
</cobertura-instrument>
<fileset dir="${src.dir}">
<include name="**/*.java" />
</fileset>
<fileset dir="${tests.src.dir}">
<include name="**/*.java" />
</fileset>
I believe you're looking for the "cobertura-instrument" task. See here
I tried to get Cobertura running inside my ant script. All is successfull (source code building, junit tests, cobertura reports (xml / html); but in html reports, the code coverage is always at 0% ...
Ant Script : make-instrument
<!-- Make instrument for Cobertura engine -->
<target name="make-instrument">
<!-- Remove the coverage data file and any old instrumentation. -->
<delete file="${cobertura.ser}" />
<!-- Instrument the application classes, writing the instrumented classes into ${build.instrumented.dir}. -->
<cobertura-instrument todir="${report.cobertura.dir}">
<!-- The following line causes instrument to ignore any source line containing a reference to log4j,
for the purposes of coverage reporting. -->
<ignore regex="org.apache.log4j.*" />
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<!-- Instrument all the application classes, but don't instrument the test classes. -->
<include name="**/*.class" />
<exclude name="**/*Test.class" />
</fileset>
</cobertura-instrument>
</target>
Ant Script : make-instrument
<target name="install-cobertura" if="is-hudson-env">
<path id="cobertura.classpath">
<fileset dir="${user.home.sharehunter.dir}/cobertura-${cobertura.rev}">
<include name="**/cobertura.jar" />
<include name="**/*.jar" />
</fileset>
</path>
<taskdef resource="tasks.properties" classpathref="cobertura.classpath" />
</target>
Ant Script : junit
<target name="run-tests" depends="make-instrument">
<path id="classpath.test">
<path path="${webcontent.dir}/WEB-INF/classes" />
<pathelement location="${webcontent.dir}/WEB-INF/classes" />
<fileset dir="${lib.dir}" includes="**/*.jar" />
<path location="${webcontent.dir}/WEB-INF/classes" />
<path location="${webcontent.dir}/WEB-INF" />
<path location="${webcontent.dir}" />
</path>
<junit fork="yes" failureProperty="test.failed">
<classpath refid="classpath.test" />
<classpath location="${user.home.dir}/junit-${junit.rev}.jar" />
<!-- Specify the name of the coverage data file to use.
The value specified below is the default. -->
<sysproperty key="net.sourceforge.cobertura.datafile" file="${cobertura.ser}" />
<!-- Note the classpath order: instrumented classes are before the original (uninstrumented) classes. -->
<classpath location="${report.cobertura.dir}" />
<!--
The instrumented classes reference classes used by the
Cobertura runtime, so Cobertura and its dependencies
must be on your classpath.
-->
<classpath refid="cobertura.classpath" />
<!-- Generate xml files for each junit tests runs -->
<formatter type="xml" />
<batchtest todir="${report.junit.dir}">
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<include name="**/*Test.class" />
</fileset>
</batchtest>
</junit>
<!-- Generate Cobertura xml file containing the coverage data -->
<cobertura-report format="xml" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
<!-- Generate Cobertura html file report containing the coverage data -->
<cobertura-report format="html" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
</target>
This is what Cobertura FAQ Says
When I generate coverage reports, why do they always show 0% coverage everywhere?
Cobertura is probably using the wrong .ser file when generating the reports. When you instrument your classes, Cobertura generates a .ser file containing basic information about each class. As your tests run, Cobertura adds additional information to this same data file. If the instrumented classes can not find the data file when running then they will create a new one. It is important that you use the same cobertura.ser file when instrumenting, running, and generating reports.
The best way to do this is to specify the location of the data file when running your tests. You should pass the -Dnet.sourceforge.cobertura.datafile=${basedir}/cobertura.ser sysproperty to the JUnit task.
Another common problem is that the cobertura.ser file is deleted, but the previously instrumented classes are not also deleted. Any time you delete your coverage data file you should also deleted all instrumented classes.
Ok I found the problem. To be sure have this :
<!--
Note the classpath order: instrumented classes are before the
original (uninstrumented) classes. This is important.
-->
<classpath location="${instrumented.dir}" />
<classpath location="${classes.dir}" />
Instrumented classes must are before the original (uninstrumented) classes.
I tried similar way. I also used instrumented code before actual source code, but I am getting 0 % in the report file.
<macrodef name="coberturaTestMacro">
<attribute name="moduleName" />
<attribute name="classpath.module" />
<attribute name="classpath.junit" />
<attribute name="failOnCoverageFall" />
<attribute name="fileCoberturaData"/>
<sequential>
<path id="classpathCobertura">
<fileset dir="${homeCobertura}">
<include name="cobertura.jar" />
<include name="lib/**/*.jar" />
</fileset>
</path>
<taskdef classpathref="classpathCobertura" resource="tasks.properties" />
<property name="cob.instrumented.dir" value="target/cobertura/instrumented" />
<delete dir="target/cobertura" />
<cobertura-instrument todir="${cob.instrumented.dir}" datafile="#{fileCoberturaData}" >
<fileset dir="target/classes">
<include name="**/*.class" />
</fileset>
</cobertura-instrument>
<delete dir="target/reports/test" />
<mkdir dir="target/cobertura/reports" />
<junit printsummary="false" failureproperty="junit.failure"
maxmemory="512m" fork="true" forkmode="perTest">
<jvmarg value="-Djava.awt.headless=true" />
<classpath location="${homeCobertura}/cobertura.jar" />
<classpath location="${cob.instrumented.dir}" />
<classpath>
<path refid="#{classpath.module}" />
<path refid="#{classpath.junit}" />
</classpath>
<classpath path="target/test-classes" />
<batchtest todir="target/cobertura/reports/">
<fileset dir="src/test/java">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<cobertura-report srcdir="src/main/java" destdir="target/cobertura/reports/" />
<echo message="${line.separator}" />
<echo message="COVERAGE: #{moduleName} module..." />
<echo message="${line.separator}" />
<if>
<available file="target/cobertura/#{moduleName}-cobertura.properties" />
<then>
<var name="total.line-rate" file="target/cobertura/#{moduleName}-cobertura.properties" />
<cobertura-check haltonfailure="#{failOnCoverageFall}"
datafile="#{fileCoberturaData}" totallinerate="${total.line-rate}" />
</then>
</if>
<delete file="${dirBuild}/coverage-summary.properties" />
<cobertura-report datafile="#{fileCoberturaData}" destdir="target/cobertura/" format="summaryXml" />
<var name="total.line-rate" file="target/cobertura/coverage-summary.properties" />
<echo message="Total line coverage: ${total.line-rate}%" />
<propertyfile file="target/cobertura//#{moduleName}-cobertura.properties">
<entry key="total.line-rate" value="${total.line-rate}" type="int" />
</propertyfile>
</sequential>
</macrodef>
The surprising thing is that the generated report says total 2% coverage, but summary file says 0% coverage.
Where old cobertura task shows 8% coverage. I am totally confused :(
Probably it is not applicable to everybody but I had the similar issue where coverage was 0 for all classes.
There were 2 issues in my case
1) it was reading wrong jdk version 1.8 off of PATH. I updated PATH to read 1.6 jdk.
2) it was initially using version 1.8 of cobertura. I ran the build and it would generate coverage report but all the classes were 0% always. I updated the javac target to include
debug="true" debuglevel="vars,lines,source" reference: cobertura 0 coverage
Then ran the build again and saw that the there was an error when running tests and traced that back to an issue with version 1.8 of cobertura.
So, I upgraded
Cobertura 1.9.4
asm 3.1 from 2.2.1
asm-tree 3.1
other dependencies
1. jakarta-oro 2.0.8
2. log4j-1.2.9
After that ran the task again and the reports were alright.