I am using //cf flag to exclude properties from the ncover coverage report like
//cf "*get_*":Method but i am getting error 2000 " top level exception" Unrecognised comandline option cf.
Please suggest what to do.
Thanks
priya
I used these filters on one of my test projects: //cf "BusinessObjects*":Assembly:False:True //cf "get":Method //cf "set":Method
Only BusinessObjects was included, and all my get and set methods were excluded. Hopefully, that modified wildcard syntax will work as you expect. I was also successful with regex pattern 'get.*'
Related
I want to query all gtest cases by bazel,
parameter "--gtest_filter" only can be used with "bazel test " cmd
and I am try to use "bazel query bazel query //xxx:all", but it will show the test list which defined in BUILD file , I want to get the cases list from xxx.cc files.
This is not a job that bazel query can do. Query operates on the graph structure of targets. A fundamental design decision of Bazel is that this graph can be computed by looking only at BUILD files and the .bzl files they depend on. In particular, parsing source files is not allowed.
(The argument to --test_filter is simply passed through the test runner; Bazel does not know what it represents.)
If you use CLion with the Bazel plugin you get the following view for googletest tests:
This works even with Catch2 (but for Catch2 the view is not so nice). I guess that's some IDE magic here - nevertheless, it gives you what you want. I assume you can also come up with some type of Bazel Aspect that produces this information for you.
I tested this also with Lavender (with minor modifications) and Visual Studio which gives me in the test overview also a list of all test:
I am running a merge using ncover.reporting on a slew of assemblies (250+) and am trying to apply coverage filters to only include assemblies that start with my teams namespace prefix (Infrastructure*) and exclude coverage on getters and setters. I wrote a basic batch file that looks something like this:
NCover.Reporting
C:\hudson\jobs\Infrastructure.Members.Api\workspace\Infrastructure.Members.Api.Test-dll-coverage-results.xml
C:\hudson\jobs\Infrasctucture.REST.Proxy\workspace\Infrasctucture.REST.Proxy.Test-dll-coverage-results.xml
...
//s MergeCoverage.xml
//cf "Infrastructure*":Assembly:False:True
//cf "*get_*":Method
//cf "*set_*":Method
//or FullCoverageReport:Html
//op "C:\NCoverProjects\Reports\FullCoverageReport"
//p "Infrastructure"
However, when I view FullCoverageReport.html I still see the dependent assemblies and getters and setters being included. What am I doing wrong?
Thanks in advance!
Resolved on NCover Forum, re-posting answer here:
I tried these filters on one of my test projects: //cf "BusinessObjects*":Assembly:False:True
//cf "get":Method //cf "set":Method
Only BusinessObjects was included, and all my get and set methods were excluded. Hopefully, that modified wildcard syntax will work as you expect. I was also successful with regex pattern 'get.*'
I was wondering if any of you guys had any experience generating code coverage reports in TFS Build Server 2010 while running NUnit tests.
I know it can be easily done with the packaged alternative (MSTest + enabling coverage on the testrunconfig file), but things are a little more involved when using NUnit. I've found some info here and there pointing to NCover, but it seems outdated. I wonder if there are other alternatives and whether someone has actually implemented this or not.
Here's more info about our environment/needs:
- TFS Build Server 2010
- Tests are in plain class libraries (not Test libraries - i.e., no testrunconfig files associated), and are implemented in NUnit. We have no MSTests.
- We are interested in running coverage reports as part of each build and if possible setting coverage threshold requirements for pass/fail criteria.
We 've done it with NUnit-NCover and are pretty happy with our results. NUnit execution is followed by NUnitTfs execution in order to get our testing results published in the Build Log. Then NCover kicks in, generating our code coverage results.
One major thing that poses as a disadvantage is fact that setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it.
Two things could pose as disadvantages:
NUnitTfs doesn't work well with NCover (at least I couldn't find a way to execute both in the same step, so (since NCover invokes NUnit) I have to run Unit tests twice: (1) to get the test results and (2) to get coverage results over NCover. Naturally, that makes my builds last longer.
Setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it .
In any case, the resulting reporting (especially the Trend aspect) is very useful in monitoring how our code evolves within time. Especially if you 're working on a Platform (as opposed to short-timed Projects), Trend reports are of great value.
EDIT
I 'll try to present in a quick & dirty manner how I 've implemented this, I hope it can be useful. We currently have NCover 3.4.12 on our build server.
Our simple naming convention regarding our NUnit assemblies is that if we have a production assembly "123.dll", then another assembly named "123_nunit.dll" exists that implements its tests. So, each build has several *_nunit.dll assemblies that are of interest.
The part in the build process template under "If not disable tests" is the one that has been reworked in order to achieve our goals, in particular the section that was named "Run MSTest for Test Assemblies". The whole implementation is here, after some cleanups to make the flow easier to be understood (pic was too large to be directly inserted here).
At first, some additional Arguments are implemented in the Build Process Template & are then available to be set in each build definition:
We then form the NUnit args in "Formulate nunitCommandLine":
String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL))
This is then used in the "Invoke NUnit"
In case this succeeds & we have set coverage for this build we move to "Generate NCover NCCOV" (the coverage file for this particular assembly). For this we invoke NCover.Console.exe with the following as Args:
String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""",
NUnitPath,
Path.GetFileName(nunitDLL),
Path.GetDirectoryName(nunitDLL),
Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)),
Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"),
Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""),
BuildDetail.BuildNumber)
All these run in the foreach loop "For all nunit dlls". When we exit the loop, we enter "Final NCover Activities" & at first the part "Merge NCCovs", where NCover.Console.exe is executed again - this time with different args:
String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject
)
When this has run, we have reached the point where all NCCOV files of this build are merged into one NCCOV-file named after the build + the Trend file (that monitors the build throughout its life) has been updated with the elements of this current build.
We now have to only generate the final HTML report, this is done in "Generate final NCover rep" where we invoke NCover.reporting with the following args:
String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
PathForNCoverResults,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject,
BuildType
)
Is there a way to customise the failure report from junit such that I can remove the time value from the test results? First prize - a way to do this without having to write code; second prize - knowing that there is no other option.
At first I thought this was an Ant thing, but looking at the
org.apache.tools.ant.taskdefs.optional.junit.JUnitResultFormatter
interface, it seems like the only thing I can control here is where the output goes, not what the actual output is. Ok - I could edit the output before allowing it out, but I could also just write a script to do that.
Sorry to disappoint, but you have to write code. Either XSLT to create a different report or post processing HTML code.
I'm using Emma in my ant build to perform coverage reporting. For those that have used Emma, is there a way to get the build to fail if the line coverage (or any type of coverage stat) does not meet a particular threshold? e.g. if the line coverage is not 100%
Not out of the box.
However, the report.metrics property or attribute of <report></report> can be set for name, class, method, block, and line. See Coverage Metrics in the Emma reference.
Use a plain-text report then a regexp filter to set up a fail condition.
I wrote an ant task to do this.
You should be able to find all the information you need on my EmmaCheck site.