I am using the coveralls.io service to display line coverage for my C++ project. I also want to track branch coverage, but cannot get it to work.
On Travis CI, I use this call to generate the coverage report:
coveralls -r <my_project_root> -b <my_build_dir> --verbose --gcov=gcov --gcov-options '\-lpbc';
The coveralls script is previously installed with pip
pip install cpp-coveralls urllib3[secure]
I get the line coverage shown correctly on coveralls.io, but not the branch coverage. I don't know what of the following things I am doing wrong.
Do I have to activate it on coveralls.io explicitly?
Is there something wrong with the coveralls command?
Can coveralls.io even show branch coverage?
Pretty late to the party, but to answer your question(s):
Yes, you will want to enable the Coveralls setting for BRANCH COVERAGE: INCLUDE IN AGGREGATE %:
Of course, this will only work if branch coverage is included in your original coverage report.
That happens in a prior step, when you compile the original project into an "instrumented" version of the source code and generate the GCOV coverage report, before you use the coveralls command to POST the coverage report to Coveralls.
Something like:
gcc -Wall -ftest-coverage -fprofile-arcs cov.c
gcov --branch-probabilities cov.c
Source: gcov Wiki - Example
Related
I am collecting code coverage for a Java-based repository using the following command:
bazel coverage --jobs=$PARALLEL_JOBS --keep_going --flaky_test_attempts=3 --cache_test_results=no --combined_report=lcov --coverage_report_generator="#bazel_tools//tools/test/CoverageOutputGenerator/java/com/google/devtools/coverageoutputgenerator:Main" -- //$TARGET/...
This creates a coverage report in this directory:
$(bazel info output_path)/_coverage/_coverage_report.dat
Problem: All of the files reported in this coverage report correspond to Java packages instead of actual files (e.g. com/my_package/path/my_class.java)
Question: How do I configure bazel coverage to report coverage data by original source files instead of Java packages?
I would like to post process these coverage files, but I don't have a clear way to map all of these Java packages into original source files.
Thank you!
How do I include non-test targets in bazel coverage? Currently I use the following bazel command to get code coverage:
bazel coverage \
--instrument_test_targets \
--experimental_cc_coverage \
--combined_report=lcov \
//... --test_arg=--logtostderr
The project is written in C++. The command works fine. However, the output lcov trace file only includes the files that have coverage. If a C file does not have a test, it is not in the lcov trace file.
Does bazel coverage only executes the test targets? Is there a way to include all targets (the non-test targets)? So that even if a file has no test, I can still see it in the report (the report will show zero coverage). The intention for this is that if someone adds new files and doesn't write unit test, the file can be shown in coverage report.
Can you reproduce using --incompatible_cc_coverage?
I am using Bazel with rules_scala. My problem now is how to exclude files from code coverage. So far this is how I am running coverage:
rm -rf coverage
bazel coverage --combined_report=lcov --coverage_report_generator="#bazel_tools//tools/test/CoverageOutputGenerator/java/com/google/devtools/coverageoutputgenerator:Main" ...
genhtml -o coverage --ignore-errors source bazel-out/_coverage/_coverage_report.dat
But there are some folders I would like to exclude from code coverage. I tried using the --instrumentation_filter flag, but no matter what I tried putting there Bazel still collect coverage for this folder.
Are there any examples how I should use this flag?
Thanks!
Use
bazel coverage --combined_report=lcov --coverage_report_generator="#bazel_tools//tools/test/CoverageOutputGenerator/java/com/google/devtools/coverageoutputgenerator:Main" -- //... -//package/to/ignore/...
This appears to be a bug with rules_scala. See this issue for more details.
I have written a sample program using C++. I have written corresponding unit tests using GUNIT framework. I was successfully able to generate.gcda and .gcno files for every source file. I used (gcov -b -l -p -c *.gcno) command in the folder where the .gcno files were generated. I am using gcov 7.5.0 . When i ran this command i saw that it gave me a)Lines covergae in percentage b)Branches covered in percentage c)Taken atleast once . Next i ran (gcovr --html -o Filename.html -r /path_to_C_sourceFiles/ .) command to generate the html output for this data. In the html file i see that the branch covergae data is extracted from the taken atleast once data which was generated by the gcov. Why is the html not taking the branches covered percentage from the gcov data and displaying it as branch covergae. Taken atleast data given by the gcov tool is a reduced number when compared to branches percentage. What is this taken atleast once?
A branch is covered if it was taken at least once. If a branch is executed multiple times, it is not more covered. So gcovr primarily considers covered/uncovered status for lines and branches, whereas GCC's gcov tool shows execution counts and branch probabilities.
Having access to the branch probabilities can be useful. Not in the context of testing, but perhaps for low-level code optimization. If you need that data, you will likely want to look at the gcov files yourself.
However, the next version of gcovr (expected to be gcovr 4.3) will show branch counts (not percentages) in the HTML report:
For each line with branch coverage data, there will be a popup that shows full branch counts.
You can use this functionality right now if you install gcovr's development version:
pip install git+https://github.com/gcovr/gcovr.git
I have a some 'custom code' that runs during the maven project build. I would like to use cobertura's line coverage feature to generate report of the executed lines of my 'custom code' for a build.
What I have achieved so far:
I am using the maven cobertura plugin to generate the report.
The execution of 'custom code' is bound to the test phase of maven build.
The maven goal cobertura:cobertura runs cobertura:instrumentation first and then in the test phase my 'custom code' is executed.
The unexpected behavior:
Although, I expected the cobertura reports to show line coverage, the line coverage reports shows just 0.
Further investigating the issue I found that the cobertura:instrumentation takes place in a separate vm. Following a snapshot of the processes that are listed by 'ps' command.
6647 ? S 0:00 /bin/sh -c /home/user/softwares/JDK/jdk1.7.0_79/jre/bin/java -Dlog4j.configuration=file:/tmp/log4j628942363594701372config.properties -Xmx64m net.sourceforge.cobertura.instrument.InstrumentMain --commandsfile /tmp/cobertura.5724213382380101768.cmdline
6649 ? D 0:00 /bin/sh -c /home/user/softwares/JDK/jdk1.7.0_79/jre/bin/java -Dlog4j.configuration=file:/tmp/log4j628942363594701372config.properties -Xmx64m net.sourceforge.cobertura.instrument.InstrumentMain --commandsfile /tmp/cobertura.5724213382380101768.cmdline
Attempts at solving this:
In order to resolve this, I've tried to find how jmockit achieves the same. Haven't really been able to figure that out yet.
I have also tried attaching the execution code to the already running VM.
com.sun.tools.attach.VirtualMachine.attach(arg0);
Can somebody please help me out here.
Update - 23th October 2015
As per an excerpt from Cobertura Github Wiki,
java -cp C:\cobertura\lib\cobertura.jar;C:\MyProject\build\instrumented;C:\MyProject\build\classes;C:\MyProject\build\test-classes -Dnet.sourceforge.cobertura.datafile=C:\MyProject\build\cobertura.ser ASimpleTestCase
This should run the tests with the cobertura's instrumented code. I tried this with an extra -javaagent:agent.jar. This doesn't seem to work either.