i have 2 test reports. One for backend i.e. junit and one for front end i.e. karma.
i have to publish test result to jenkins. For junit there is plugin. but for karma there is not.
What can be used.
As of now i tried using junit and xunit to publish the report. But what jenkins is doing here is -- "It is merging both the result reports into a single trend graph."
As noted, you can get Karma to publish in JUnit format.
Alternatively, if it can output HTML, you can publish those: https://jenkins.io/doc/pipeline/steps/htmlpublisher/
Karma is a test runner for JavaScript. (for angular that runs jasmine test cases)
For Karma, you can use "karma-junit-reporter" which will generate an XML file of test cases and then use publish JUnit reporter from post-build action and give path of that XML file
Related
Currently it consumes so much time to rerun the filed test cases from Jenkins in My LOCAL MACHINE.And most of the test would pass locally .
Is there anything I can do to cut short the time ?
TestNG generates all of the artifacts viz.,
Html reports
Emailable report
JUnit styled xml reports
TestNG XML reports
testng-failed.xml
by default in the output directory of your project. If its not defined, TestNG defaults to test-output folder.
When tests are run via build tools such as maven wherein you leverage the surefire plugin, then this output directory gets configured to point to target/surefire-reports directory.
So your testng-failed.xml if it gets generated, will be available in your $WORKSPACE\target\surefire-reports directory (Here $WORKSPACE represents your Jenkins job's workspace)
In our continuous integration process we are using Jenkins, NUnit and OpenCover.
Every Jenkins job runs NUnit and OpenCover, but OpenCover calls the NUnit batch file in order to determine code coverage; therefore NUnit is executed twice.
For example we have a first (simplified) batch:
nunit-console-x86 [PathToTestAssemblies] /xml=NunitResult.xml /noshadow /nodots /process=Separate
And we have a second batch for OpenCover:
OpenCover.Console.exe -target:"NUnit.bat" -output:"./OpenCoverResults.xml" -register -targetdir:".\bin" -coverbytest:*.dll
The problem is that OpenCover does not provide NUnit result (The NunitResult.xml file in my previous command). So in order to have less test time for every Jenkins job we want to get back the NUnit result or find a way to have the following features with OpenCover in the Jenkins job web page:
Latest Tests result from every Jenkins job, so it is easy for a developer to see the latest result.
Test result trend
Is there a way to have both NunitResult and OpenCover results from an single run of NUnit?
I finally found were was the NUnitResult.xml file. In fact it was in the folder were I put all test assemblies (ex: bin) while the Nunit batch put it at the root location.
I think this is because I use the -targetdir args with "bin" to indicate OpenCover were are my assemblies.
I have a Jenkins build which build all my java/angularJS project. It launch testNG tests for the java part and karma tests for the javascript part. So I can generate one testNG report (for java) and one junit report (for karma test) in my Jenkins build. This is working very well.
Until now, I used cobertura to report the coverage of my java tests. But now I would like to add also a coverage report for my karma tests (generated by Istanbul with cobertura type). The problem is that, in Jenkins, I'm allowed to generate only one coverage report in a build (I can't add more that one 'publish cobertura coverage report' post build action). So how can I have these two coverage reports in a single Jenkins build?
There's a nice plugin called HTML Publisher Plugin. You can generate HTML coverage reports and publish as much reports as you want under different titles in one Jenkins project.
For example I generate html reports using karma+istanbul and then publish them to Jenkins.
On JUnit xml report files. You should import JUnit once enumerating all files probably from different directories but you can differentiate them with proper package names inside files.
If I'm right, you can't use, as a post build action, the same plug-in twice( note that I'm not really sure). I faced this problem when I worked as Jenkins plug-in developer for a company and the solution was to use a plug-in that make the same thing.
For example: for JUnit reports there is an official JUnit plugin and also XUnit. For my problem it was simple.
So, maybe you can find a plug-in that do the same thing as Cobertura or you can change the output format of the java coverage or karma coverage. For example, for java you can use EclEmma or Jacoco...
I am running Sonar task through Ant, triggered by Jenkins in RHEL environment. I am successfully using Cobertura for Junit code coverage and Surefire for reporting. Sonar imports the Surefire reports fine.
However, now I am running Selenium tests using Ant in Jenkins. I would like to report code coverage and test results to Sonar. Apparently I need the JaCoCo plugin which analyses code coverage and reports tests. I presume like for unit tests, Jenkins does the job and Sonar only imports the reports into its own repository.
I am puzzled on how to actually do this. The web page http://www.sonarsource.org/measure-coverage-by-integration-tests-with-sonar-updated/ references to the JaCoCo page http://www.eclemma.org/jacoco/trunk/doc/ant.html. I am not sure what the steps are to be done. Do I need the coverage target? Should I only start the agent? Where in Ant do I start the agent? Do I need to dump stuff?
I really appreciate all the help I can get, thanks :)
Sonar 3.3 has a new feature for combine code coverage metrics generated by both unit tests and integration tests. This is done by using two properties to detail the two different report files generated by the jacoco too:
#Tells Sonar where the unit tests code coverage report is
sonar.jacoco.reportPath=reports/jacoco/jacoco-ut.exec
#Tells Sonar where the integration tests code coverage report is
sonar.jacoco.itReportPath=reports/jacoco/jacoco-it.exec
The Sonar examples project has an integration test example for ANT:
https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/it/ant/it-jacoco-ant
Unfortunately it doesn't give an example of running the actual tests, instead it just shows how to configure an ANT build to load data.
Finally the Sonar documentation has more details with links to the example projects.
I have started using Qunit to test my JS code. I am looking into JSCoverage to generate the coverage reports later. We have a CI server (Jenkins) which already do a few things with our PHP code and I was wondering if anyone can comment on how I can integrate the report from my Qunit and JSCoverage into Jenkins
Thanks
Sparsh
QUnit: use QUnit API to generate junit XML files. Here's a sample.
In Post-build Actions for your job you then check Publish JUnit test result report and specify your junit XML files (or their file pattern). Jenkins will then mark builds that have failed tests as unstable and produce a nice trend graph of successful/failing tests.
A few more details, for those actually attempting this:
Putting together QUnit and Jenkins
If you want to run QUnit and publish the results in Jenkins, you'll need to do the following:
Step 1: Getting QUnit to generate an XML file compatible with JUnit.
If you're using Apache Ant, this question explains how to get
QUnit to generate XML.
If not, you can use Grunt and
grunt-qunit-junit, together with grunt-contrib-qunit, to
run your .html tests.
And if you're not into either Ant or Grunt, here is
a script for PhantomJS to run your tests directly and produce
JUnit-style XML.
Step 2: Processing that XML file
This is the easy step - look in "Post-build Actions" for your job in Jenkins, and add the path to the XML file.