I am using .Net MVC with Web API project in Visual Studio 2015.
The project is configured with the SonarQube Version 8.7.
Currently the Code coverage percent is 0%.
Recently I have added a unit test project in my solution and added around 25+ test methods for my Web API Controllers.
All the test methods are passed. I am using MS Test for Unit testing.
On executing the SonarQube script I am not able to see any improvement in my Code coverage inspite of adding unit test cases.
Please guide me for what have been gone wrong resulting in 0 percent of Code coverage in SonarQube
Thank you in advance.
EDIT: After OP add a comment for more clarity, the following block is necessary for understanding issue:
From documentation that is linked below.
We support:
Import of coverage reports from VSTest, dotCover, OpenCover, Coverlet and NCover 3.
Import of unit test results from VSTest, NUnit and xUnit.
To include tests results and test coverage you need to create an XML report that will be read by SonarQube. Official documentation can be found here.
Since MS forum says that:
Visual Studio includes the VSTest and MSTest command-line tools for testing purposes.
We can use both VSTEST and MSTEST to run automated unit and coded UI tests from a command line.
My advice is to use VStest since MStest is not supported. Also, there are some cases that should be looked at, so please check the documentaion.
Run Unit Tests To Collect Code Coverage
"%VSINSTALLDIR%\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" /EnableCodeCoverage "UnitTestProject1\bin\Debug\UnitTestProject1.dll"
Convert the Code Coverage Report from Binary into XML
"%VSINSTALLDIR%\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output: "%CD%\VisualStudio.coveragexml" "%CD%\VisualStudio.coverage"
If anyone uses NUnit, here are steps to include it:
Run Unit Tests and Save Results in file "NUnitResults.xml"
packages\NUnit.ConsoleRunner.3.7.0\tools\nunit3-console.exe --result=NUnitResults.xml "NUnitTestProject1\bin\Debug\NUnitTestProject1.dll"
Import unit test results
To import a test execution report, during the Begin step you need to pass a parameter that points to the file that will be generated:
sonar.cs.nunit.reportsPaths for NUnit
Full command will be something like:
dotnet sonarscanner begin /k:PROJECT_NAME /d:sonar.login=LOGIN_TOKEN /d:sonar.host.url=URL /d:sonar.cs.nunit.reportsPath=PATH_TO_NUnitResults.xml
My first guess is that you're running VS community edition or pro. To have Visual Studio generate a coverage file, you need Enterprise, or a 3d party coverage plugin.
Requirements
The code coverage feature is available only in Visual Studio Enterprise edition.
In which case you'll need to configure dotCover or any of the other mentioned coverage tools or upgrade to VS Enterprise.
Related
I am aware of using karma and phantomjs to run opa5 headless. I want to know if there any other options? Currently I am using grunt to run qunit and Nightwatch for unit and end-to-end tests. I added OPA5 tests and looking for ways to run it headlessly with coverage report which I can publish on TFS(Team Foundation Server)
Please guide.
You also can consider Selenium, check the different of them: Casperjs/PhantomJs vs Selenium.
Regarding test result, you can use trxReporter, related article: CI/CD Pipeline for Angular2 with VSTS.
Regarding Code Coverage, the TFS Code Coverage supports JaCoco and Cobertura (Publish code coverage task) and karma-coverage supports Cobertura format, so you can use cobertura code coverage report.
Related article: Include code coverage report in VSTS, Is Test Adapter mandatory for VSTS? (Apply to TFS)
Using Jenkins I am trying to pass the result of the Nunit task and the OpenCover task to SonarQube.
SonarQube displays properly the test coverage but there is no trace of unit test results.
Here is what I do :
Build Visual Studio project with MSBuild
Batch windows for opencover via nunit
Launch SonarQube scanner
Batch windows for deployement
Here is my actual properties for the Sonar Scanner task:
sonar.projectKey=FC
sonar.projectName=FC
sonar.projectVersion=%SVN_REVISION%
sonar.sources=src
sonar.binaries=build
sonar.exclusions=src/FC.Test/**/*.*
sonar.tests=src/FC.Test
sonar.cs.nunit.reportsPaths=TestResult.xml
sonar.cs.opencover.reportsPaths=reports/opencovertests.xml
I'm using SonarQube 5.6, Jenkins 2.10, SonarQube plugin 2.4.3, OpenCover 4.5 and Nunit 2.6.4
Is there any known problems (compatibility issues...) ?
Note that I tryed to wrap my process with the Prepare and Terminate Sonar analysis tasks but it systematicaly deletes the coverage panel. So i removed it.
EDIT 2017
I stopped doing this way. I understood that the good way to do this was using the SonarQube Scanner for MSBuild which is integrated at the Jenkins's plugin.
The coverage panel was not shown because i didn't configure my quality profiles properly.
But...
I still don't have my tests' result in Sonar.
I checked the absolute path of my report and its parsing by sonarQube : "INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=25ms". I also tryed with a .nunit file. I saw that apply a filter to opencover could help, so I did.
Do you have a clue?
Scanning C# projects requires to use the Sonar Scanner for MSBuild and follow a specific 3 steps process which I guess you don't follow for now. See doc about the 3 steps process at: http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+MSBuild and http://docs.sonarqube.org/display/SCAN/From+Jenkins
Your properties look correct. Yet they shall not be provided in a sonar properties file but instead in the command line arguments of the MSBuild.SonarQube.Runner.exe begin step, for example:
MSBuild.SonarQube.Runner.exe begin <allMandatoryParameters> /d:sonar.cs.nunit.reportsPaths=TestResult.xml /d:sonar.cs.opencover.reportsPaths=reports\opencovertests.xml
Olivier
Is it possible to create a code coverage Cobertura report and get it published on Sonar Dashboard for Websphere Commmerce Server (WCS) projects ? I understand that the pre-requisite is that the team should have written the Junit test cases but still wondering whether code coverage report is possible to generate ?
Under the assumptions that
a Websphere Commerce Server project is written in Java
the project contains unit tests that are executed during the build
a code coverage report is generated during the run of unit tests and pointed to during the analysis
Then the answer is yes.
Is it possible to use NCover plug-in in Jenkins to get code coverage on c# code with VSTest? I found help on how to user NCover with MSTests but not VSTests. Is there any other Jenkins tool to get code coverage using VSTests?
I take it by your reference to the NCover plug-in that you're using NCover 3?
In any case, NCover 5 can cover any .NET test runner from a Jenkins build script.
In the case of VSTest, it's usually a matter of profiling vstest.executionengine.
I had this working on a previous project and now on a new project I've setup SpecFlow, got it generating tests from my feature file but I can't run the tests from the feature file and instead have to go to the code behind to run the tests. I've also installed the VS extension "Spec Flow for Visual Studio". What can I try?
as Greg suggested the first thing to check is that your config is set up correctly for ms test. you basically need this:
<specflow>
<unitTestProvider name="MSTest"/>
</specflow>
Also worth checking your generated feature.cs tests to see what unit test language they are in