Empty Junit reports from ant - ant

I am trying to use ant to run junit tests and generate reports.
I am able to successfully run the tests but the report files are empty.
What am I doing wrong ?
This is my build.xml :
<project name="JunitTest" default="test" basedir=".">
<property name="testdir" location="." />
<property name="srcdir" location="." />
<property name="full-compile" value="true" />
<property name="test.reports" value="./reports" />
<path id="classpath.base"/>
<path id="classpath.test">
<pathelement location="${testdir}" />
<pathelement location="${srcdir}" />
<path refid="classpath.base" />
</path>
<target name="clean" >
<delete verbose="${full-compile}">
<fileset dir="${testdir}" includes="**/*.class" />
</delete> `
</target>
<target name="compile" depends="clean">
<javac srcdir="${srcdir}" destdir="${testdir}" verbose="${full-compile}" >
<classpath refid="classpath.test"/>
</javac>
</target>
<target name="test" depends="compile">
<junit>
<classpath refid="classpath.test" />
<formatter type="brief" usefile="false" />
<test name="com.tests.nav1" />
</junit>
<junitreport todir="${test.reports}">
<fileset dir="${test.reports}">
<include name="TEST-*.xml" />
</fileset>
<report todir="${test.reports}" />
</junitreport>
</target>
</project>
and this is the output on the console :
[junit] Using CLASSPATH C:\eclipse\eclipse-java-helios-SR1-win32\eclipse\JunitWS\SeleniumTraining\src;C:\jars\junit.jar;C:\ant\lib\ant-launcher.jar;C:\ant\lib\ant.jar;C:\ant\lib\ant-junit.jar;C:\ant\lib\ant-junit4.jar
[junit] Testsuite: com.tests.nav1
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 48.187 sec
[junit] ------------- Standard Output ---------------
[junit] testnav2
[junit] ------------- ---------------- ---------------
[junitreport] Using class org.apache.tools.ant.taskdefs.optional.TraXLiaison
[junitreport] Processing C:\eclipse\eclipse-java-helios-SR1-win32\eclipse\JunitWS\SeleniumTraining\src\reports\TESTS-TestSuites.xml to C:\Users\pmahajan\AppData\Local\Temp\null236099757
[junitreport] Loading stylesheet jar:file:/C:/ant/lib/ant-junit.jar!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl
[junitreport] Transform time: 330ms
[junitreport] Deleting: C:\Users\pmahajan\AppData\Local\Temp\null236099757
BUILD SUCCESSFUL
Total time: 49 seconds

If you look at the ant snippet, there are a few issues:
You have set usefile=false, which means no output file is created
You have set formatter type=brief, which will print detailed information only for failed tests
You need to also specify the todir - the folder where the report has to go in the <test> tag - the default is current folder. This should match the folder you are using in <junitreport> task.
You can try with the following updated <junit> section...
<junit>
<classpath refid="classpath.test" />
<formatter type="xml"/>
<test name="com.tests.nav1" todir="${test.reports}"/>
</junit>

1 <target name="test" depends="compile">
2 <junit>
3 <classpath refid="classpath.test" />
4 <formatter type="brief" usefile="false" />
5 <test name="com.tests.nav1" />
6 </junit>
7 <junitreport todir="${test.reports}">
8 <fileset dir="${test.reports}">
9 <include name="TEST-*.xml" />
10 </fileset>
11 <report todir="${test.reports}" />
12 </junitreport>
13 </target>
The above segment of your coded needs following changes.
You have to specify the to directory option in the 5th line( for example todir = ${data.reports} )
In the 8th line the directory specified must me data.reports.
The 11th line must contain the option format with the value frames (format="frames").

The ant JUnit Task doc gives this example that might help you (as it apparently does exactly what you're trying to achieve):
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<pathelement location="${build.tests}"/>
<pathelement path="${java.class.path}"/>
</classpath>
<formatter type="plain"/>
<test name="my.test.TestCase" haltonfailure="no" outfile="result">
<formatter type="xml"/>
</test>
<batchtest fork="yes" todir="${reports.tests}">
<fileset dir="${src.tests}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
</junit>
Runs my.test.TestCase in the same VM, ignoring the given CLASSPATH; only a warning is printed if this test fails. In addition to the plain text test results, for this test a XML result will be output to result.xml. Then, for each matching file in the directory defined for ${src.tests} a test is run in a separate VM. If a test fails, the build process is aborted. Results are collected in files named TEST-name.txt and written to ${reports.tests}.
It is specified in the doc that printsummary can take values on and off, but they're using yes in the example which is on the same page, so I guess it's accepted too.

Please try formatter with "xml"
<formatter type="${junit.format}"/>
where junit.format is property with appropriate value.

Related

Jacoco coverage with Ant throws java.lang.instrument.IllegalClassFormatException: Error while instrumenting class

I integrated Jacoco with my Ant build. When I run the build, the test case is executed successfully followed by the below exception in my TEST-com.worker.ManagerTest.xml.
When I add excludes="*" the error is not thrown. But the jacoco.exec is generated with 1kb size and when I run the report nothing is generated. Can someone let me know what am I missing?
Exception:
<![CDATA[java.lang.instrument.IllegalClassFormatException: Error while instrumenting class com/dataaccess/GenericDao.
at org.jacoco.agent.rt_6qyg3i.CoverageTransformer.transform(CoverageTransformer.java:69)
Below is the jacoco build script.
<target name="test" depends="test-compile">
<mkdir dir="${report.dir}" />
<jacoco:coverage destfile="${report.dir}/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant" exclclassloader="sun.reflect.DelegatingClassLoader:javassist.Loader">
<junit fork="true" forkmode="once" printsummary="on">
<classpath>
<!--<pathelement location="${basedir}/../../../build/lib/aspectjtools.jar"/>
<pathelement location="${basedir}/../../../build/lib/aspectjrt.jar"/>-->
<pathelement path="${test.path}" />
<pathelement path="${dist.dir}/test/unittest-manager.jar" />
</classpath>
<formatter type="xml" />
<batchtest todir="${report.dir}" fork="yes">
<fileset dir="test">
<include name="**/*Test*" />
</fileset>
</batchtest>
</junit>
</jacoco:coverage>
</target>
<target name="report" depends="test">
<echo message="Generating Jacoco reports..." />
<property name="report.dir.file" value="${report.dir}/jacoco.exec"/>
<jacoco:report>
<executiondata>
<file file="${report.dir.file}"/>
</executiondata>
<structure name="JaCoCo Reports">
<classfiles>
<fileset dir="${dist.dir}/applications/lib/manager.jar">
<include name="**/*.class"/>
</fileset>
</classfiles>
<sourcefiles encoding="UTF-8">
<fileset dir="${src.dir}">
<include name="**/*.java"/>
</fileset>
</sourcefiles>
</structure>
<html destdir="${report.dir}/coverage"/>
</jacoco:report>
</target>
This is the empty report I get.
Regards,
Sat
I used the latest version 0.8.7 and it worked.

Jenkins JUnit Test Result Report plugin states that the JUnit xml file is not found?

The exact message received from jenkins is:
No test report files were found. Configuration error?
Build step 'Publish JUnit test result report' changed build result to FAILURE
When configuring the JUnit Test Result Report plugin, on entering the 'Test Report XMLs' path as '/reports/TEST-*.xml', the following error is displayed beneath the path:
'/reports/TEST-*.xml' doesn't match anything: '' exists but not '/reports/TEST-*.xml'
I have tried using the full path as well but that produces the same result. In both cases the paths should have picked up the 'TESTS-TestSuites.xml' file that was present in the /reports directory.
I'm not sure whether this is a problem with the plugin or the XML file being generated. I'm also aware that it could be an issue with the ant build script that I have written to run the JUnit tests and produce the XML result file therefore I have included the contents of this below in case something needs to be changed:
<?xml version="1.0" encoding="utf-8"?>
<project name="jenkins-tests" basedir="." default="linux">
<property name="junit.output.dir" value="output"/>
<property name="src.dir" value="src"/>
<property name="lib.dir" value="libs" />
<property name="bin.dir" value="bin" />
<property name="full-compile" value="true" />
<path id="classpath.base"/>
<path id="classpath.test">
<pathelement location="${bin.dir}" />
<pathelement location="${src.dir}" />
<pathelement location="${lib.dir}" />
<pathelement location="${lib.dir}/junit.jar" />
<path refid="classpath.base" />
</path>
<target name="clean" description="Clean up build artefacts">
<delete dir="${basedir}/${junit.output.dir}" />
</target>
<target name="prepare" depends="clean" description="Prepare for build">
<mkdir dir="${basedir}/${junit.output.dir}" />
<mkdir dir="${junit.output.dir}/reports"/>
</target>
<target name="compile" depends="prepare">
<javac srcdir="${src.dir}" destdir="${bin.dir}" verbose="${full-compile}" includeAntRuntime="false" >
<classpath refid="classpath.test"/>
</javac>
</target>
<target name="test" depends="compile">
<junit printsummary="true" haltonfailure="false">
<formatter type="xml" usefile="true"/>
<classpath refid="classpath.test" />
<batchtest fork="yes" todir="${junit.output.dir}">
<fileset dir="${src.dir}">
<include name="*.java"/>
</fileset>
</batchtest>
</junit>
</target>
<target name="test-reports" depends="test">
<junitreport tofile="TESTS-TestSuites.xml" todir="${junit.output.dir}/reports">
<fileset dir="${junit.output.dir}">
<include name="TEST-*.xml" />
</fileset>
<report format="frames" todir="${junit.output.dir}/reports" />
</junitreport>
</target>
</project>
I've been researching into this problem for a while now and haven't found any solution so I would appreciate any help. Thanks.
Jenkins looks for the path from the workspace root. Ensure that the given path is correct or use wildcards to look in multiple locations. Try using **/reports/TEST-*.xml
Are you sure the reports folder is right under the workspace? Verify manually if the test result files are indeed present in the location given in the path.
For my Android project which has multiple Gradle product flavors I used the following path for Test report XMLs:
**/build/test-results/**/TEST-*.xml

Ant JUnit task doesn't work in TeamCity

There is part of ant script with junit task:
...
<target name="test">
<mkdir dir="path_to_report_dir">
<junit fork="true" printsummary="true" showoutput="true" maxmemory="1024M">
<classpath ... />
<batchtest todir="path_to_report_dir">
<formatter type="xml" />
<fileset ... />
</batchtest>
</junit>
</target>
...
This script works from Eclipse and from command line. But it doesn't work in TeamCity. The last informative message in TeamCity is:
[mkdir] Created dir: path_to_report_dir
Process exit code: 0
It looks like junit task doesn't work and also it stops performting aff all script. Where is trouble in?
The cause was in <fileset> file list. The TeamCity version of Ant doesn't work with strings like "/test/" (this mean select all files recursively); it only works with strings like "**/test/*.class". The local version of Ant supports both variants.
Thanks.
Don't know if this helps.... but here's my standard test target:
<target name="test" depends="compile-tests">
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${classes.dir}"/>
<pathelement path="${test.classes.dir}"/>
</classpath>
<formatter type="xml"/>
<batchtest fork="yes" todir="${test.reports.dir}">
<fileset dir="${test.src.dir}">
<include name="**/*Test*.java"/>
<exclude name="**/AllTests.java"/>
</fileset>
</batchtest>
</junit>
</target>
Build output:
test:
[junit] Running org.demo.AppTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.056 sec
Notes
Using Junit 4.10.

Use of Emma for JUnit in ant build.xml

I am newbie to use Emma. I am trying to add emma ant task for JUnit test case for modules in EAR project. I have few question here.
Should I use instrumented class for packaging my EAR projet?
What is good way to add emma ant task for junit? Should I use emmarun:on-th-fly mode or offline mode? For JUnit should I use fork or no fork?
I am using Emma Offline mode and Junit with fork. Here is my build.xml
<!--Target and task for EMMA -->
<taskdef resource="emma_ant.properties" classpathref="Emma.libraryclasspath" />
<target name="emma" description="turns on EMMA's instrumentation/reporting" >
<property name="emma.enabled" value="true" />
<mkdir dir="${out.instr.dir}" />
<property name="emma.filter" value="" />
</target>
<target name="test" depends="init, compile" description="Run JUnit Test cases under emma environment">
<!-- Emma instrumentation -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${class.dir}"
destdir="${out.instr.dir}"
metadatafile="${coverage.dir}/metadata.em"
merge="true"
mode="copy">
<filter value="${emma.filter}" />
</instr>
</emma>
<!-- JUnit Start -->
<junit printsummary="yes" fork="yes">
<test name="com.hf.platform.authorizer.WebTxnAuthorizerTest" todir="${test.report.dir}">
<formatter type="xml"/>
</test>
<classpath>
<path refid="HFPlatformWeb.classpath"/>
<path refid="Emma.libraryclasspath"/>
</classpath>
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.ec" />
<jvmarg value="-Demma.coverage.out.merge=false" />
</junit>
<!-- Junit End -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<report>
<sourcepath>
<dirset dir="${basedir}">
<include name="src"/>
<include name="test-src"/>
</dirset>
</sourcepath>
<fileset dir="${coverage.dir}">
<include name="*.em"/>
<include name="*.ec"/>
</fileset>
<xml outfile="${coverage.report.dir}/report.xml" />
<txt outfile="${coverage.report.dir}/report.txt" />
<html outfile="${coverage.report.dir}/report.html" />
</report>
</emma>
</target>
When I ran it for one test, it is not generating any report. But when i ran same unit test with EclEmma it gives correct output.
In above example we need to make sure following two things
The file path for metadatafile and coverage report file that is .ec, .em or .emma file should be absolute or relative to project.
e.g.
For running java/junit task sandwiched between the instrumentation and report task, it must use instrumented class file path.
e.g.
<classpath> <pathelement location="${out.instr.dir}" />
<path refid="Emma.libraryclasspath"/>
<path refid="HFPlatformEJB.classpath"/>
</classpath>

Running jUnit with ANT - How to execute all tests without using #Suite?

I'd like to execute all tests in a /test directory without using annotations such as
#Suite.SuiteClasses( ....)
In the past i had a single class, which was calling many other classes to test them all. This approach is no longer acceptable.
I have a /test directory, underneath which i have a number of packages, each containing several tests.
In my current ANT script, i have:
<target name="compileTest" depends="compile" description="compile jUnit">
<javac srcdir="${test}" destdir="${bin}" includeantruntime="true" />
</target>
followed by
<target name="test" depends="compileTest">
<junit printsummary="yes" fork="no" haltonfailure="no">
<classpath location="${bin}" />
<formatter type="plain" />
</junit>
</target>
In the past, i had
<test name="MyCollectionOfTests" />
I'd rather not do this anymore.
What am i missing? Please advise.
You can use a nested batchtest. For instance:
<junit printsummary="on"
fork="on"
dir="${test.build}"
haltonfailure="false"
failureproperty="tests.failed"
showoutput="true">
<classpath>
<path refid="tests.classpath"/>
</classpath>
<batchtest todir="${test.report}">
<fileset dir="${test.gen}">
<include name="**/Test*.java"/>
</fileset>
<fileset dir="${test.src}">
<include name="**/Test*.java"/>
<exclude name="gen/**/*"/>
</fileset>
</batchtest>
</junit>
In its simplest form, you can simply add a nested:
<batchtest todir="report">
<fileset dir="test"/>
</batchtest>
to your junit call.

Resources