I am running a merge using ncover.reporting on a slew of assemblies (250+) and am trying to apply coverage filters to only include assemblies that start with my teams namespace prefix (Infrastructure*) and exclude coverage on getters and setters. I wrote a basic batch file that looks something like this:
NCover.Reporting
C:\hudson\jobs\Infrastructure.Members.Api\workspace\Infrastructure.Members.Api.Test-dll-coverage-results.xml
C:\hudson\jobs\Infrasctucture.REST.Proxy\workspace\Infrasctucture.REST.Proxy.Test-dll-coverage-results.xml
...
//s MergeCoverage.xml
//cf "Infrastructure*":Assembly:False:True
//cf "*get_*":Method
//cf "*set_*":Method
//or FullCoverageReport:Html
//op "C:\NCoverProjects\Reports\FullCoverageReport"
//p "Infrastructure"
However, when I view FullCoverageReport.html I still see the dependent assemblies and getters and setters being included. What am I doing wrong?
Thanks in advance!
Resolved on NCover Forum, re-posting answer here:
I tried these filters on one of my test projects: //cf "BusinessObjects*":Assembly:False:True
//cf "get":Method //cf "set":Method
Only BusinessObjects was included, and all my get and set methods were excluded. Hopefully, that modified wildcard syntax will work as you expect. I was also successful with regex pattern 'get.*'
Related
I am working on creating a Jenkins pipeline for unit-testing maybe with GTest.
My plan is to use the following tools:
GTest for Unit-Testing, gcov for generating gcda and gcno Files and gcovr for xml or Html outputs of the unit-testing results.
It's working well till now with the help from the internet and particularly stack overflow.
But I am struggling with 3 issues.
gcov is creating gcda and gcno files for gtest sources and my unit-tests. Because gcovr is invoking them and I see them in the HTML files. how can I avoid this? I only want my production code in the HTML files.
I can only see code coverage for template classes if gcov is generating gcda and gcno files for my unit-tests. So I need a simple idea for 1) Maybe can I use an exclude flag in gcovr?
Unused functions in template classes (inline functions) are not covered. Code coverage is always 100% but I tried with different flags, and nothing helped.
-fprofile-abs-path --coverage -fno-inline -fno-inline-small-functions -fno-default-inline -fkeep-inline-functions
I added a picture to show, what I am talking about. UnitTests and GTests covering results should not appear in gcovr HTML...
You can filter out unwanted coverage data, but you can't create data that doesn't exist.
1. gcov is creating gcda and gcno files for gtest sources and my unit-tests. Because gcovr is invoking them and I see them in the HTML files. how can I avoid this? I only want my production code in the HTML files.
Use gcovr --exclude GoogleTest/ --exclude UnitTests/
Gcovr has a per-file filtering system that allows you to specify which source code files to include/exclude. For a file to be included in the coverage report,
any --filter pattern must match, and
no --exclude pattern must match.
Or phrased in reverse: a file is excluded if it doesn't match any --filter or if it matches any --exclude pattern.
If you don't provide an explicit --filter, then the default filter is the --root directory, which in turn defaults to the current working directory.
These patterns are regexes. Usually, these are used to match paths relative to the current working directory. For example, you can limit the reports to a src/ directory with gcovr --filter src/. Or you can exclude the GoogleTest/ directory with gcovr --exclude GoogleTest/.
Gcovr also has a way to filter gcda/gcno files (search_paths and --gcov-filter), but that is mostly useful as a performance optimization.
2. I can only see code coverage for template classes if gcov is generating gcda and gcno files for my unit-tests. So I need a simple idea for 1) Maybe can I use an exclude flag in gcovr?
This is by design. As explained above, you can solve this via gcovr's exclude flag.
You get a gcda/gcno file per compilation unit. Header files are included into multiple compilation units, so their coverage information is essentially split across all compilation units that include it.
So, if you want coverage for code in header files, and you include these headers into your unit tests, then gcovr must also process the gcda/gcno files from those unit tests.
3. Unused functions in template classes (inline functions) are not covered. Code coverage is always 100% but I tried with different flags, and nothing helped. -fprofile-abs-path --coverage -fno-inline -fno-inline-small-functions -fno-default-inline -fkeep-inline-functions
The gcov coverage data model works on an assembly-code level. Counters are inserted by the compiler itself, but only for functions for which the compiler actually generates machine code. Thus, as far as gcov is concerned, inline functions, optimized-out code, and non-instantiated templates simply do not exist.
This is quite annoying, but it's potentially difficult to work around.
This can most clearly be avoided by making sure that all functions for which you want coverage data are referenced via your unit tests. It is not necessary to actually invoke that function, merely referencing it should be sufficient. For example, I'd write a function to ignore() arbitrary values despite optimizations, then:
ignore(&some_inline_function);
Possible implementation: template<class T> void ignore(T const& t) { volatile T sinkhole = t; }
Your suggested options like -fno-inline do not work because the code for these functions isn't generated in the first place.
With GCC and when using C++ (but not C), the -fkeep-inline-functions should work, but only for non-templated inline functions.
If a non-templated inline function is only used within one file and isn't provided in a header to multiple files, then it should instead be declared static (in C) or in an anonymous namespace (in C++11 or later), so that -Wunused-function or -Wall notify you if it isn't referenced.
Templates are more tricky in general. Each distinct instantiation of a template results in separate functions. Gcovr does aggregate coverage data across them, but in order for the template to appear in the coverage data it must be instantiated at least once. You will have to do this manually.
I want to query all gtest cases by bazel,
parameter "--gtest_filter" only can be used with "bazel test " cmd
and I am try to use "bazel query bazel query //xxx:all", but it will show the test list which defined in BUILD file , I want to get the cases list from xxx.cc files.
This is not a job that bazel query can do. Query operates on the graph structure of targets. A fundamental design decision of Bazel is that this graph can be computed by looking only at BUILD files and the .bzl files they depend on. In particular, parsing source files is not allowed.
(The argument to --test_filter is simply passed through the test runner; Bazel does not know what it represents.)
If you use CLion with the Bazel plugin you get the following view for googletest tests:
This works even with Catch2 (but for Catch2 the view is not so nice). I guess that's some IDE magic here - nevertheless, it gives you what you want. I assume you can also come up with some type of Bazel Aspect that produces this information for you.
I tested this also with Lavender (with minor modifications) and Visual Studio which gives me in the test overview also a list of all test:
I'm writing descriptors for somebody else's pipeline steps jenkins plugin. Most steps are straight forward, e.g.
mySimpleStep(param1: value1, param2: value2)
However one of the steps requires a parameter, which is a map of two other values, so the actual call syntax is the following:
myOtherStep(param1: value1, param2: [sub1: value2, sub2: value3])
I can't fathom how to specify the parameters in the config.jelly file for the step and/or update the actual Step class so that the call syntax is created correctly. How can I do that?
(param2 class does have its own #DataBoundConstructor if it matter)
Do note that this is somebody else's plugin, I am in no position to change the actual plugin.
After almost giving up, I stumbled upon the answer while looking at the source code of Microsoft Azure Storage plugin. Here are the steps I needed to do.
Ensure that the class of param2 implements Step and add Description inner class to it. It also needs to have a #DataBoundConstructor
Create a separate descriptor directory for the class in resources with its own config.jelly and help-*.html files
Change the config.jelly of myOtherStep to have something like this:
<f:section title="General">
<f:entry field="value1" title="First param" description="Simple parameter">
<f:textbox/>
</f:entry>
<f:property field="value2">
<st:include page="config.jelly"/>
</f:property>
</f:section>
Now the config.jelly class for the complex parameter will be included - and everything works as expected.
I am using //cf flag to exclude properties from the ncover coverage report like
//cf "*get_*":Method but i am getting error 2000 " top level exception" Unrecognised comandline option cf.
Please suggest what to do.
Thanks
priya
I used these filters on one of my test projects: //cf "BusinessObjects*":Assembly:False:True //cf "get":Method //cf "set":Method
Only BusinessObjects was included, and all my get and set methods were excluded. Hopefully, that modified wildcard syntax will work as you expect. I was also successful with regex pattern 'get.*'
How can I add filters to skip some of the classes in a namespace/assembly. For example: SYM.UI is the base assembly and i want to skip SYM.UI.ViewModels. Writing the below filter but it is including all of them and not fulfilling my request:
+[SYM.UI*]* -[SYM.UI.ViewModels*]*
Kindly help me correcting this?
The opencover wiki is a good place to start.
The usage is described as +/-[modulefilter]typefilter (this is based on how you would see the types in IL; where the type filter also includes the namespace and module filter usually is the name of the assembly (without the file extension).
Thus to exclude your types you could use
+[SYM.UI]* -[SYM.UI]SYM.UI.ViewModels.*
NOTE: Exclusion filters take preference over inclusion filters.
You can use following:
"-filter:+[*]* -[SYM.UI]SYM.UI.ViewModels.*"
Note that the quotes must be around the -filter: part, too