We can get unit test code coverage with bazel coverage, but I have to get interface test code coverage.
The interface test is that I upload my package to a remote docker container,and run it,Then I post some requests to it. I want to get these interface test coverage in bazel output coverage.dat files.
I use this command to build a golang package:
bazel build ... --compilation_mode=dbg --collect_code_coverage=true
it can run successfully, but it doesn't generate code coverage files.
Should I use some other build options to build my package?
If you know how to resolve it, please help me. Thank you very much.
Related
How do I include non-test targets in bazel coverage? Currently I use the following bazel command to get code coverage:
bazel coverage \
--instrument_test_targets \
--experimental_cc_coverage \
--combined_report=lcov \
//... --test_arg=--logtostderr
The project is written in C++. The command works fine. However, the output lcov trace file only includes the files that have coverage. If a C file does not have a test, it is not in the lcov trace file.
Does bazel coverage only executes the test targets? Is there a way to include all targets (the non-test targets)? So that even if a file has no test, I can still see it in the report (the report will show zero coverage). The intention for this is that if someone adds new files and doesn't write unit test, the file can be shown in coverage report.
Can you reproduce using --incompatible_cc_coverage?
I am using .Net MVC with Web API project in Visual Studio 2015.
The project is configured with the SonarQube Version 8.7.
Currently the Code coverage percent is 0%.
Recently I have added a unit test project in my solution and added around 25+ test methods for my Web API Controllers.
All the test methods are passed. I am using MS Test for Unit testing.
On executing the SonarQube script I am not able to see any improvement in my Code coverage inspite of adding unit test cases.
Please guide me for what have been gone wrong resulting in 0 percent of Code coverage in SonarQube
Thank you in advance.
EDIT: After OP add a comment for more clarity, the following block is necessary for understanding issue:
From documentation that is linked below.
We support:
Import of coverage reports from VSTest, dotCover, OpenCover, Coverlet and NCover 3.
Import of unit test results from VSTest, NUnit and xUnit.
To include tests results and test coverage you need to create an XML report that will be read by SonarQube. Official documentation can be found here.
Since MS forum says that:
Visual Studio includes the VSTest and MSTest command-line tools for testing purposes.
We can use both VSTEST and MSTEST to run automated unit and coded UI tests from a command line.
My advice is to use VStest since MStest is not supported. Also, there are some cases that should be looked at, so please check the documentaion.
Run Unit Tests To Collect Code Coverage
"%VSINSTALLDIR%\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" /EnableCodeCoverage "UnitTestProject1\bin\Debug\UnitTestProject1.dll"
Convert the Code Coverage Report from Binary into XML
"%VSINSTALLDIR%\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output: "%CD%\VisualStudio.coveragexml" "%CD%\VisualStudio.coverage"
If anyone uses NUnit, here are steps to include it:
Run Unit Tests and Save Results in file "NUnitResults.xml"
packages\NUnit.ConsoleRunner.3.7.0\tools\nunit3-console.exe --result=NUnitResults.xml "NUnitTestProject1\bin\Debug\NUnitTestProject1.dll"
Import unit test results
To import a test execution report, during the Begin step you need to pass a parameter that points to the file that will be generated:
sonar.cs.nunit.reportsPaths for NUnit
Full command will be something like:
dotnet sonarscanner begin /k:PROJECT_NAME /d:sonar.login=LOGIN_TOKEN /d:sonar.host.url=URL /d:sonar.cs.nunit.reportsPath=PATH_TO_NUnitResults.xml
My first guess is that you're running VS community edition or pro. To have Visual Studio generate a coverage file, you need Enterprise, or a 3d party coverage plugin.
Requirements
The code coverage feature is available only in Visual Studio Enterprise edition.
In which case you'll need to configure dotCover or any of the other mentioned coverage tools or upgrade to VS Enterprise.
running bazel build //... \
--aspects some-aspect.bzl%some_aspect \
--output_groups=some_new_output,default
does not create test jars outputs.
on the other hand running bazel test does create the test jar outputs:
bazel test //... \
--aspects some-aspect.bzl%some_aspect \
--output_groups=some_new_output,default
How come?
This question was updated to reflect use of aspects:
The original question:
running bazel build //... does not add test code to output jar.
on the other hand bazel test //... builds the test code but also
runs it.
Is there a way in bazel to build the test code without running the
tests?
I had a mistake in the values I gave the --output_groups flag.
It should have been --output_groups=+some_new_output,+default
The default can even be omitted:
--output_groups=+some_new_output
This flag is not documented at all. There is an open issue on this in bazel github repo.
You may be looking for --build_tests_only.
I'm possibly in over my head here, but I've been asked to set up a scheduled Team Foundation Build for our team's branch and then after the build completes for our automated tests to be executed using NUnit.
I've had a look at a few online tutorials on setting up the build definition in TFS, but I can't seem to figure out how to call NUnit after the build is successful. I was expecting to see or find some kind of "run this command line on success" option somewhere; the best I could find is "Pre/Post-test script path", but that's related to tests like **\*test*.dll;**\*test*.appx and I'm not sure what that is.
Just knowing what to Google for would be a help, as I am at a loss now.
If you use XAML build:
You can either install the NUnit Test Adapter NuGet package in the unit test project. Or you can check the assemblies into the Build Controller's Custom Assemblies Path.
Useful article:
https://blogs.msdn.microsoft.com/visualstudioalm/2013/06/11/part-3-unit-testing-with-traits-and-code-coverage-in-visual-studio-2012-using-the-tfs-build-and-the-new-nuget-adapter-approach/
https://www.codit.eu/blog/2015/03/18/continuous-integration-with-javascript-nunit-on-tfsbuild-part-3-of-3-/
If you use new task based build:
You can add the NUnit Test Adapter NuGet package to your solution, and specify the path of NUnit Test Adapter NuGet package in the Path to Custom Test Adapters field in VSTest task. Check the screenshot below:
Useful article:
http://bartwullems.blogspot.sg/2015/10/team-foundation-server-2015enable-nunit.html
I managed to install Google Test on Jenkins.
I use cmake to build the test executable and everything works fine.
The stupid question I have now is:
How do I automatically let Jenkins run google test?
Do I have to write a shell script for this or is there a better way?
I know that one could run it in ant but since I use cmake I doubt that this is the right way to go.
There's no builtin or pluggable "Googletest automation" for Jenkins.
You've built the test executable with CMake in a build step.
Execute the test executable in a subsequent build step
with xml test reporting enabled
and configure the build to archive or otherwise publish the test reports.