I am aware of using karma and phantomjs to run opa5 headless. I want to know if there any other options? Currently I am using grunt to run qunit and Nightwatch for unit and end-to-end tests. I added OPA5 tests and looking for ways to run it headlessly with coverage report which I can publish on TFS(Team Foundation Server)
Please guide.
You also can consider Selenium, check the different of them: Casperjs/PhantomJs vs Selenium.
Regarding test result, you can use trxReporter, related article: CI/CD Pipeline for Angular2 with VSTS.
Regarding Code Coverage, the TFS Code Coverage supports JaCoco and Cobertura (Publish code coverage task) and karma-coverage supports Cobertura format, so you can use cobertura code coverage report.
Related article: Include code coverage report in VSTS, Is Test Adapter mandatory for VSTS? (Apply to TFS)
Related
I am using .Net MVC with Web API project in Visual Studio 2015.
The project is configured with the SonarQube Version 8.7.
Currently the Code coverage percent is 0%.
Recently I have added a unit test project in my solution and added around 25+ test methods for my Web API Controllers.
All the test methods are passed. I am using MS Test for Unit testing.
On executing the SonarQube script I am not able to see any improvement in my Code coverage inspite of adding unit test cases.
Please guide me for what have been gone wrong resulting in 0 percent of Code coverage in SonarQube
Thank you in advance.
EDIT: After OP add a comment for more clarity, the following block is necessary for understanding issue:
From documentation that is linked below.
We support:
Import of coverage reports from VSTest, dotCover, OpenCover, Coverlet and NCover 3.
Import of unit test results from VSTest, NUnit and xUnit.
To include tests results and test coverage you need to create an XML report that will be read by SonarQube. Official documentation can be found here.
Since MS forum says that:
Visual Studio includes the VSTest and MSTest command-line tools for testing purposes.
We can use both VSTEST and MSTEST to run automated unit and coded UI tests from a command line.
My advice is to use VStest since MStest is not supported. Also, there are some cases that should be looked at, so please check the documentaion.
Run Unit Tests To Collect Code Coverage
"%VSINSTALLDIR%\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" /EnableCodeCoverage "UnitTestProject1\bin\Debug\UnitTestProject1.dll"
Convert the Code Coverage Report from Binary into XML
"%VSINSTALLDIR%\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output: "%CD%\VisualStudio.coveragexml" "%CD%\VisualStudio.coverage"
If anyone uses NUnit, here are steps to include it:
Run Unit Tests and Save Results in file "NUnitResults.xml"
packages\NUnit.ConsoleRunner.3.7.0\tools\nunit3-console.exe --result=NUnitResults.xml "NUnitTestProject1\bin\Debug\NUnitTestProject1.dll"
Import unit test results
To import a test execution report, during the Begin step you need to pass a parameter that points to the file that will be generated:
sonar.cs.nunit.reportsPaths for NUnit
Full command will be something like:
dotnet sonarscanner begin /k:PROJECT_NAME /d:sonar.login=LOGIN_TOKEN /d:sonar.host.url=URL /d:sonar.cs.nunit.reportsPath=PATH_TO_NUnitResults.xml
My first guess is that you're running VS community edition or pro. To have Visual Studio generate a coverage file, you need Enterprise, or a 3d party coverage plugin.
Requirements
The code coverage feature is available only in Visual Studio Enterprise edition.
In which case you'll need to configure dotCover or any of the other mentioned coverage tools or upgrade to VS Enterprise.
Is it possible to create a code coverage Cobertura report and get it published on Sonar Dashboard for Websphere Commmerce Server (WCS) projects ? I understand that the pre-requisite is that the team should have written the Junit test cases but still wondering whether code coverage report is possible to generate ?
Under the assumptions that
a Websphere Commerce Server project is written in Java
the project contains unit tests that are executed during the build
a code coverage report is generated during the run of unit tests and pointed to during the analysis
Then the answer is yes.
I have a Jenkins build which build all my java/angularJS project. It launch testNG tests for the java part and karma tests for the javascript part. So I can generate one testNG report (for java) and one junit report (for karma test) in my Jenkins build. This is working very well.
Until now, I used cobertura to report the coverage of my java tests. But now I would like to add also a coverage report for my karma tests (generated by Istanbul with cobertura type). The problem is that, in Jenkins, I'm allowed to generate only one coverage report in a build (I can't add more that one 'publish cobertura coverage report' post build action). So how can I have these two coverage reports in a single Jenkins build?
There's a nice plugin called HTML Publisher Plugin. You can generate HTML coverage reports and publish as much reports as you want under different titles in one Jenkins project.
For example I generate html reports using karma+istanbul and then publish them to Jenkins.
On JUnit xml report files. You should import JUnit once enumerating all files probably from different directories but you can differentiate them with proper package names inside files.
If I'm right, you can't use, as a post build action, the same plug-in twice( note that I'm not really sure). I faced this problem when I worked as Jenkins plug-in developer for a company and the solution was to use a plug-in that make the same thing.
For example: for JUnit reports there is an official JUnit plugin and also XUnit. For my problem it was simple.
So, maybe you can find a plug-in that do the same thing as Cobertura or you can change the output format of the java coverage or karma coverage. For example, for java you can use EclEmma or Jacoco...
I'm trying to use JaCoCo agent for collecting FT code coverage of a web-site by adding "-javaagent..." to Jetty and running external Selenium tests (or, for example, manual testing).
At the end of testing I have results only for web-module, that was actually run in Jetty, but not for "core" modules (in the same project) being used in web-module. So, is there any ability to collect calls for all modules of the project?
Indeed, I now tried to start all project webservices (it has 5) with the same JaCoCo report destination (i.e. /tmp/jacoco-res.exec), append=true and ran:
mvn sonar:sonar -Dsonar.jacoco.itReportPath=/tmp/jacoco-res.exec
So, sonar now used the same JaCoCo report file for every module in project and sonar shows tests coverage both for jetty-run and core modules.
Is this approach applicable and coverage results correct?
I had a simular question that I posed on the Sonar User forum a couple of days ago.
In short, your approach is correct and was also proposed by a Sonar consultant (as can be seen on the post).
Perform a Sonar analyze on each Maven project.
For each analysis, reuse the report and pass your JaCoCo report (using -Dsonar.jacoco.itReportPath)
Sonar will retrieve from this JaCoCo report
the coverage data related to the Maven project being analyzed. (one of your core projects)
Thus, for each Maven project, you'll get the coverage by your integration tests.
Finally, to get the coverage of your webservice by your integration
tests, you will have to use the Views plugin:
http://www.sonarsource.com/products/plugins/governance/portfolio-management/
Create a view that is an aggregation of all the Maven
projects composing the webservice.
Run a Sonar analysis of one of its
Maven projects in order for Sonar to compute the view.
On the view dashboard, you'll be able to get the code coverage
of the webservice by your integration tests.
I am running Sonar task through Ant, triggered by Jenkins in RHEL environment. I am successfully using Cobertura for Junit code coverage and Surefire for reporting. Sonar imports the Surefire reports fine.
However, now I am running Selenium tests using Ant in Jenkins. I would like to report code coverage and test results to Sonar. Apparently I need the JaCoCo plugin which analyses code coverage and reports tests. I presume like for unit tests, Jenkins does the job and Sonar only imports the reports into its own repository.
I am puzzled on how to actually do this. The web page http://www.sonarsource.org/measure-coverage-by-integration-tests-with-sonar-updated/ references to the JaCoCo page http://www.eclemma.org/jacoco/trunk/doc/ant.html. I am not sure what the steps are to be done. Do I need the coverage target? Should I only start the agent? Where in Ant do I start the agent? Do I need to dump stuff?
I really appreciate all the help I can get, thanks :)
Sonar 3.3 has a new feature for combine code coverage metrics generated by both unit tests and integration tests. This is done by using two properties to detail the two different report files generated by the jacoco too:
#Tells Sonar where the unit tests code coverage report is
sonar.jacoco.reportPath=reports/jacoco/jacoco-ut.exec
#Tells Sonar where the integration tests code coverage report is
sonar.jacoco.itReportPath=reports/jacoco/jacoco-it.exec
The Sonar examples project has an integration test example for ANT:
https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/it/ant/it-jacoco-ant
Unfortunately it doesn't give an example of running the actual tests, instead it just shows how to configure an ANT build to load data.
Finally the Sonar documentation has more details with links to the example projects.