I want to execute test script on multiple devices(Android). when i run my java class with JUnit i am able to execute in only one device. How to execute in multiple devices at a time.
Any suggestion would be appreciated.
TestNG.xml file
<suite name="Default suite" thread-count="2" parallel="tests">
<test name="Nexus">
<Parameters>
<parameter name="platform" value="Nexus"/>
<parameter name="browsername" value="Android"/>
<parameter name="udid" value="xyz" />
<parameter name="remoteurl" value="http://0.0.0.0:4723/wd/hub"/>
</Parameters>
<classes>
<class name="AppiumTest">
<methods>
<include name="Test1"/>
<include name="Test2"/>
<include name="Test3"/>
</methods>
</class>
</classes>
</test>
<test name="Moto E">
<Parameters>
<parameter name="platform" value="Moto E"/>
<parameter name="browsername" value="Android"/>
<parameter name="udid" value="abc" />
<parameter name="remoteurl" value="http://0.0.0.0:4726/wd/hub"/>
</Parameters>
<classes>
<class name="AppiumTest">
<methods>
<include name="Test1"/>
<include name="Test2"/>
<include name="Test3"/>
</methods>
</class>
</classes></suite>
If you use testNG instead of JUnit you can create a test suite with a testng.xml file that should look like this:
<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite" parallel="tests" thread-count="2">
<test name="Nexus 7">
<parameter name="udid" value="XXXX" />
<classes>
<class name="testNG.TestOne"/>
</classes>
</test> <!-- Test -->
<test name="HTC desrire">
<parameter name="udid" value="XXXX" />
<classes>
<class name="testNG.TestOne"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->
Stating parallel tests and a thread-count of 2 allows two tests to be completed on seperate devices in parallel.
All you need to do from here is configure Selenium Grid nodes with capabilities of each device and in your tests script use the udid parameter passed in through the testng.xml.
Hope this helps.
Just launch another test after the first is launched. Of course each test must be pointed to different device.
Try to apply the concept shown in the below thread.It is using grid concept in selenium to start two appium sessions in parallel.By this we can run our scripts parallely on two android devices.
https://discuss.appium.io/t/connecting-appium-server-to-selenium-grid-for-android/804/10
Related
I have been using multiple testng files in my build.xml like in the example shown below:
<target name="run1" depends="compile">
<!-- config testng -->
<xmlfileset dir="." includes="test1.xml" />
<sysproperty key="testurl" value="${url}" />
<sysproperty Key="dryruntest" value="${name}" />
<!--<sysproperty key="logLevel" value="${testLogLevel}"/> -->
</testng>
<fail if="test.failure" message="Not all tests passed!" />
</target>
<target name="run2" depends="compile">
<xmlfileset dir="." includes="test2.xml" />
<sysproperty key="testurl" value="${url}" />
<sysproperty Key="dryruntest" value="${name}" />
<!--<sysproperty key="logLevel" value="${testLogLevel}"/> -->
</testng>
<fail if="test.failure" message="Not all tests passed!" />
</target>
But now I want to use just one testng.xml in both the target tabs in build.xml, for which I need the commontestng.xml to look like this:
When I kick off the job from Jenkin (which currently takes in one custom argument - which is url) and target name for build.xml, it works fine with different testng for respective target names in build.xml.
However, when I try to use a single testng file in build.xml for different target names it always runs the test1, and not test2.
My goal is to run TestBox scripts on Jenkins. But using the Ant script from
https://testbox.ortusbooks.com/content/running_tests/ant_runner.html
as a template, I get this error
BUILD FAILED
C:\public\data\trunk\AutomatedTesting\Box_Unit_Tests\build.xml:38: The reference to entity "bundles" must end with the ';' delimiter.
with this script:
<?xml version="1.0" encoding="UTF-8"?>
<project name="testbox-ant-runner" default="init" basedir=".">
<!-- THE URL TO THE RUNNER, PLEASE CHANGE ACCORDINGLY-->
<property name="basedir" value="C:\public\data\trunk\AutomatedTesting\Box_Unit_Tests" />
<property name="url.runner" value="C:\public\data\ColdBox\testbox\test-harness\runner.cfm?"/>
<!-- FILL OUT THE BUNDLES TO TEST, CAN BE A LIST OF CFC PATHS -->
<property name="test.bundles" value="http://localhost/application/testing/TestBox/Hello.cfc?method=runRemote" />
<!-- FILL OUT THE DIRECTORY MAPPING TO TEST -->
<property name="test.directory" value="test.specs" />
<!-- FILL OUT IF YOU WANT THE DIRECTORY RUNNER TO RECURSE OR NOT -->
<property name="test.recurse" value="true" />
<!-- FILL OUT THE LABELS YOU WANT TO APPLY TO THE TESTS -->
<property name="test.labels" value="" />
<!-- FILL OUT THE TEST REPORTER YOU WANT, AVAILABLE REPORTERS ARE: ANTJunit, Codexwiki, console, dot, doc, json, junit, min, raw, simple, tap, text, xml -->
<property name="test.reporter" value="simple" />
<!-- FILL OUT WHERE REPORTING RESULTS ARE STORED -->
<property name="report.dir" value="${basedir}\results" />
<property name="junitreport.dir" value="${report.dir}\junitreport" />
<target name="init" description="Init the tests">
<mkdir dir="${junitreport.dir}" />
<tstamp prefix="start">
<format property="TODAY" pattern="MM-dd-YYYY hh:mm:ss aa"/>
</tstamp>
<concat destfile="${report.dir}\Latestrun.log">Tests ran at ${start.TODAY}</concat>
</target>
<target name="run">
<get dest="${report.dir}/results.html"
src="${url.runner}&bundles=${test.bundles}&reporter=${test.reporter}"
verbose="true"/>
<-- Create fancy junit reports -->
<junitreport todir="${junitreport.dir}">
<fileset dir="${report.dir}">
<include name="TEST-*.xml"/>
</fileset>
<report format="frames" todir="${junitreport.dir}">
<param name="TITLE" expression="My Awesome TestBox Results"/>
</report>
</junitreport>
</target>
</project>
Any thoughts?
There are many testng.xml files which I run via same build.xml using command line option.
I want build.xml to parse the test name from each testng.xml that is passed via command line option
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="tests" verbose="1" threads="10" preserve-order="true">
<test name="first" preserve-order="true">
<classes>
<class name="com.etc.first">
<methods>
<include name="First" />
</methods>
</class>
</classes>
</test>
<test name="second" preserve-order="true">
<classes>
<class name="com.etc.Second.java">
<methods>
<include name="Req" />
</methods>
</class>
</classes>
</test>
</suite>
I am newbie to use Emma. I am trying to add emma ant task for JUnit test case for modules in EAR project. I have few question here.
Should I use instrumented class for packaging my EAR projet?
What is good way to add emma ant task for junit? Should I use emmarun:on-th-fly mode or offline mode? For JUnit should I use fork or no fork?
I am using Emma Offline mode and Junit with fork. Here is my build.xml
<!--Target and task for EMMA -->
<taskdef resource="emma_ant.properties" classpathref="Emma.libraryclasspath" />
<target name="emma" description="turns on EMMA's instrumentation/reporting" >
<property name="emma.enabled" value="true" />
<mkdir dir="${out.instr.dir}" />
<property name="emma.filter" value="" />
</target>
<target name="test" depends="init, compile" description="Run JUnit Test cases under emma environment">
<!-- Emma instrumentation -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${class.dir}"
destdir="${out.instr.dir}"
metadatafile="${coverage.dir}/metadata.em"
merge="true"
mode="copy">
<filter value="${emma.filter}" />
</instr>
</emma>
<!-- JUnit Start -->
<junit printsummary="yes" fork="yes">
<test name="com.hf.platform.authorizer.WebTxnAuthorizerTest" todir="${test.report.dir}">
<formatter type="xml"/>
</test>
<classpath>
<path refid="HFPlatformWeb.classpath"/>
<path refid="Emma.libraryclasspath"/>
</classpath>
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.ec" />
<jvmarg value="-Demma.coverage.out.merge=false" />
</junit>
<!-- Junit End -->
<emma enabled="${emma.enabled}" verbosity="verbose">
<report>
<sourcepath>
<dirset dir="${basedir}">
<include name="src"/>
<include name="test-src"/>
</dirset>
</sourcepath>
<fileset dir="${coverage.dir}">
<include name="*.em"/>
<include name="*.ec"/>
</fileset>
<xml outfile="${coverage.report.dir}/report.xml" />
<txt outfile="${coverage.report.dir}/report.txt" />
<html outfile="${coverage.report.dir}/report.html" />
</report>
</emma>
</target>
When I ran it for one test, it is not generating any report. But when i ran same unit test with EclEmma it gives correct output.
In above example we need to make sure following two things
The file path for metadatafile and coverage report file that is .ec, .em or .emma file should be absolute or relative to project.
e.g.
For running java/junit task sandwiched between the instrumentation and report task, it must use instrumented class file path.
e.g.
<classpath> <pathelement location="${out.instr.dir}" />
<path refid="Emma.libraryclasspath"/>
<path refid="HFPlatformEJB.classpath"/>
</classpath>
I tried to get Cobertura running inside my ant script. All is successfull (source code building, junit tests, cobertura reports (xml / html); but in html reports, the code coverage is always at 0% ...
Ant Script : make-instrument
<!-- Make instrument for Cobertura engine -->
<target name="make-instrument">
<!-- Remove the coverage data file and any old instrumentation. -->
<delete file="${cobertura.ser}" />
<!-- Instrument the application classes, writing the instrumented classes into ${build.instrumented.dir}. -->
<cobertura-instrument todir="${report.cobertura.dir}">
<!-- The following line causes instrument to ignore any source line containing a reference to log4j,
for the purposes of coverage reporting. -->
<ignore regex="org.apache.log4j.*" />
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<!-- Instrument all the application classes, but don't instrument the test classes. -->
<include name="**/*.class" />
<exclude name="**/*Test.class" />
</fileset>
</cobertura-instrument>
</target>
Ant Script : make-instrument
<target name="install-cobertura" if="is-hudson-env">
<path id="cobertura.classpath">
<fileset dir="${user.home.sharehunter.dir}/cobertura-${cobertura.rev}">
<include name="**/cobertura.jar" />
<include name="**/*.jar" />
</fileset>
</path>
<taskdef resource="tasks.properties" classpathref="cobertura.classpath" />
</target>
Ant Script : junit
<target name="run-tests" depends="make-instrument">
<path id="classpath.test">
<path path="${webcontent.dir}/WEB-INF/classes" />
<pathelement location="${webcontent.dir}/WEB-INF/classes" />
<fileset dir="${lib.dir}" includes="**/*.jar" />
<path location="${webcontent.dir}/WEB-INF/classes" />
<path location="${webcontent.dir}/WEB-INF" />
<path location="${webcontent.dir}" />
</path>
<junit fork="yes" failureProperty="test.failed">
<classpath refid="classpath.test" />
<classpath location="${user.home.dir}/junit-${junit.rev}.jar" />
<!-- Specify the name of the coverage data file to use.
The value specified below is the default. -->
<sysproperty key="net.sourceforge.cobertura.datafile" file="${cobertura.ser}" />
<!-- Note the classpath order: instrumented classes are before the original (uninstrumented) classes. -->
<classpath location="${report.cobertura.dir}" />
<!--
The instrumented classes reference classes used by the
Cobertura runtime, so Cobertura and its dependencies
must be on your classpath.
-->
<classpath refid="cobertura.classpath" />
<!-- Generate xml files for each junit tests runs -->
<formatter type="xml" />
<batchtest todir="${report.junit.dir}">
<fileset dir="${webcontent.dir}/WEB-INF/classes">
<include name="**/*Test.class" />
</fileset>
</batchtest>
</junit>
<!-- Generate Cobertura xml file containing the coverage data -->
<cobertura-report format="xml" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
<!-- Generate Cobertura html file report containing the coverage data -->
<cobertura-report format="html" srcdir="${src.main.java.dir}" destdir="${report.cobertura.dir}" datafile="${cobertura.ser}" />
</target>
This is what Cobertura FAQ Says
When I generate coverage reports, why do they always show 0% coverage everywhere?
Cobertura is probably using the wrong .ser file when generating the reports. When you instrument your classes, Cobertura generates a .ser file containing basic information about each class. As your tests run, Cobertura adds additional information to this same data file. If the instrumented classes can not find the data file when running then they will create a new one. It is important that you use the same cobertura.ser file when instrumenting, running, and generating reports.
The best way to do this is to specify the location of the data file when running your tests. You should pass the -Dnet.sourceforge.cobertura.datafile=${basedir}/cobertura.ser sysproperty to the JUnit task.
Another common problem is that the cobertura.ser file is deleted, but the previously instrumented classes are not also deleted. Any time you delete your coverage data file you should also deleted all instrumented classes.
Ok I found the problem. To be sure have this :
<!--
Note the classpath order: instrumented classes are before the
original (uninstrumented) classes. This is important.
-->
<classpath location="${instrumented.dir}" />
<classpath location="${classes.dir}" />
Instrumented classes must are before the original (uninstrumented) classes.
I tried similar way. I also used instrumented code before actual source code, but I am getting 0 % in the report file.
<macrodef name="coberturaTestMacro">
<attribute name="moduleName" />
<attribute name="classpath.module" />
<attribute name="classpath.junit" />
<attribute name="failOnCoverageFall" />
<attribute name="fileCoberturaData"/>
<sequential>
<path id="classpathCobertura">
<fileset dir="${homeCobertura}">
<include name="cobertura.jar" />
<include name="lib/**/*.jar" />
</fileset>
</path>
<taskdef classpathref="classpathCobertura" resource="tasks.properties" />
<property name="cob.instrumented.dir" value="target/cobertura/instrumented" />
<delete dir="target/cobertura" />
<cobertura-instrument todir="${cob.instrumented.dir}" datafile="#{fileCoberturaData}" >
<fileset dir="target/classes">
<include name="**/*.class" />
</fileset>
</cobertura-instrument>
<delete dir="target/reports/test" />
<mkdir dir="target/cobertura/reports" />
<junit printsummary="false" failureproperty="junit.failure"
maxmemory="512m" fork="true" forkmode="perTest">
<jvmarg value="-Djava.awt.headless=true" />
<classpath location="${homeCobertura}/cobertura.jar" />
<classpath location="${cob.instrumented.dir}" />
<classpath>
<path refid="#{classpath.module}" />
<path refid="#{classpath.junit}" />
</classpath>
<classpath path="target/test-classes" />
<batchtest todir="target/cobertura/reports/">
<fileset dir="src/test/java">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
<cobertura-report srcdir="src/main/java" destdir="target/cobertura/reports/" />
<echo message="${line.separator}" />
<echo message="COVERAGE: #{moduleName} module..." />
<echo message="${line.separator}" />
<if>
<available file="target/cobertura/#{moduleName}-cobertura.properties" />
<then>
<var name="total.line-rate" file="target/cobertura/#{moduleName}-cobertura.properties" />
<cobertura-check haltonfailure="#{failOnCoverageFall}"
datafile="#{fileCoberturaData}" totallinerate="${total.line-rate}" />
</then>
</if>
<delete file="${dirBuild}/coverage-summary.properties" />
<cobertura-report datafile="#{fileCoberturaData}" destdir="target/cobertura/" format="summaryXml" />
<var name="total.line-rate" file="target/cobertura/coverage-summary.properties" />
<echo message="Total line coverage: ${total.line-rate}%" />
<propertyfile file="target/cobertura//#{moduleName}-cobertura.properties">
<entry key="total.line-rate" value="${total.line-rate}" type="int" />
</propertyfile>
</sequential>
</macrodef>
The surprising thing is that the generated report says total 2% coverage, but summary file says 0% coverage.
Where old cobertura task shows 8% coverage. I am totally confused :(
Probably it is not applicable to everybody but I had the similar issue where coverage was 0 for all classes.
There were 2 issues in my case
1) it was reading wrong jdk version 1.8 off of PATH. I updated PATH to read 1.6 jdk.
2) it was initially using version 1.8 of cobertura. I ran the build and it would generate coverage report but all the classes were 0% always. I updated the javac target to include
debug="true" debuglevel="vars,lines,source" reference: cobertura 0 coverage
Then ran the build again and saw that the there was an error when running tests and traced that back to an issue with version 1.8 of cobertura.
So, I upgraded
Cobertura 1.9.4
asm 3.1 from 2.2.1
asm-tree 3.1
other dependencies
1. jakarta-oro 2.0.8
2. log4j-1.2.9
After that ran the task again and the reports were alright.