I am trying to get bazel coverage //my:test to output coverage data files, building with custom C rules and using a custom clang toolchain.
For Bazel's native C rules this is a solved problem. I can build coverage output using the cc_library and cc_test native rules by running the following command with env set:
export BAZEL_USE_LLVM_NATIVE_COVERAGE=1
export GCOV=/path/to/llvm-profdata
export BAZEL_LLVM_COV=/path/to/llvm-cov
export CC=/path/to/clang
bazel coverage //my:test --experimental_generate_llvm_lcov --combined_report=lcov
The test target has a coverage.dat file output and there is a combined dat report file too. I have noticed that the cc_library target returns an InstrumentedFilesInfo provider that has the "metadata files" attribute populated with the .gcno files output during compilation.
I am using the cc_common Starlark library to build custom C rules and my compile action is setup via cc_common.compile(). While *.gcno files are outputs that Bazel expects from this action [0], the compile function does not return any *.gcno File objects in the compilation context or compilation output so using them as the inputs to another action/returning in a provider/adding to the target's runfiles is not possible.
I understand that the .dat files are produced using the *.gcno compile output and the *.gcda sandboxed test execution output and combined in the collect_cc_coverage.sh script. Something in the plumbing of my rule implementation is missing that is not being fixed by returning a provider constructed by coverage_common.instrumented_files_info() and declaring extra outputs of cc_common.compile() is currently not possible.
[0]: Running under coverage rather than test has the toolchain feature where --compile is added the .gcno files are output and they appear in bazel-out.
My questions:
Has anyone had any experience implementing code coverage for custom C rules?
How do I get my test executable to take in .gcno files, generate the .gcda files and combine the two using my toolchain to produce the .dat files that are expected with the native C rules? (This question does not require .gcno - solutions involving profraw/profdata are equally valid.)
Related
I have a Bazel project (the new tcmalloc) I'm trying to integrate into a typical GNU Make project that uses it's own build of compiler/libc++. The goal is to not fork the upstream project.
If I pass all the C++ options correctly to bazel (one set of which is -nostdinc++ -I<path to libc++>), Bazel is uhappy The include path '/home/vlovich/myproject/deps/toolchain/libc++/trunk/include' references a path outside of the execution root. (tcmalloc is a git submodule sibling # deps/tcmalloc). It's possible to get this "working" by giving Bazel a custom script to invoke as the compiler that injects those flags so that Bazel never sees them. However, I'd like to just define a toolchain to work properly.
I've read all the documentation I could find on this topic but it's not clear to me how to glue all these docs together.
Specifically not really clear where I should place the toolchain definition files or how to tell Bazel to find those definitions. Is there a way to give bazel a directory that it uses to find toolchain definitions? Am I expected to create a top-level WORKSPACE # /home/vlovich/myproject & register tcmalloc and my toolchain there, & then invoke bazel from /home/vlovich/myproject instead of /home/vlovich/myproject/deps/tcmalloc?
Toolchain support is rather complicated, and it is hard to understand, if you are not a bazel maintainer.
You can use CC and CXX environment variables to set a different compiler like: CC=your_c_compiler CXX=your_c++_compiler bazel build .... You can write your own custom script wrapper which will act as a normal C++ compiler
That -I<path to libc++> does not work, because all normal include paths have to be defined in srcs attribute or via dependencies indicated by the deps attribute. For system-wide dependencies use -isystem Read more about it https://stackoverflow.com/a/44061589/4638604
I am using gcov to find code coverage of my application while running functional test cases.
Issue: .gcda files are not getting created for few classes though those classes are being executed
Steps:
Compile Application code with gcov and verify .gcno files are generated for all classes
Create binary image of compiled code and deploy server using that binary
Use cross profiling while deploying server. Home directory structure for my source code was "/proj/QQ/scm/tools/jenkins/db_ws/FunctionTestCoverage/ccode/" and I used below mentioned gcov parameters for cross profiling
GCOV_PREFIX=/automation/testCoverage
GCOV_PREFIX_STRIP=7
Run Functional Test Cases to hit application code
.gcda files gets created for few classes only. It is not getting created for all classes which are being executed.
.gcda files are generating under directory structure "/automation/testCoverage/ccode"
I believe this should not be issue with cross profiling. suppose I have 5 directories in parallel then .gcda files are generating for 2 directories only.
What can be root cause for my issue where .gcda are generating for only few files and how to resolve this?
We have a bunch of targets being built by Bazel in our build environment today, targets being cc_binary/cc_test/cc_library etc. Is there some Bazel option that dumps build time for each of these into a file so we can analyze it after the build is done. While building Bazel dumps some timing information that disappears fast but I am looking for something (either text/XML or any other readable format) that can be used in post-build analysis.
To some extent this is possible. Look at the documentation: Performance profiling
There is the build flag --experimental_generate_json_trace_profile that generates a file named command.profile.gz in our outputBase dir (details about the outputBase dir can be fond here).
Example:
bazel build --experimental_generate_json_trace_profile --experimental_profile_cpu_usage //:HelloWorld
command.profile.gz contains a file named command.profile. This can be view in Chrome (chrome://tracing/).
(Tested with Bazel 2.0.0)
I am migrating a project from cmake to bazel. I have a folder contains some python code and some genrules.
I had a test script run all the python tests in this folder recursively.
So basically I need all py files under this folder as data for the test script. But given there are some genrule I need to run, there are some BUILD files, so that glob(["**/*.py"]) can't get through.
For example, we have a folder python contains following files.
python/BUILD
python/test_a.py
python/folder_a/BUILD this one has genrule in it.
python/folder_a/folder_b/BUILD this one has genrule as well.
python/folder_a/folder_b/folder_c/test_b.py
I want to run the test script under python/, it will run all the test_*.py recursively. Now we want to wrap it as a sh_test in bazel. So we need to specify all the test_*.py in the data field. But there is no easy way to do that since glob() can't get through python/folder_a/BUILD and python/folder_a/folder_b/BUILD.
It will be quite convenience if I can run this script in the source tree. But it seems that bazel didn't provide this. Adding local = 1 in sh_test only make the runfiles tree writable.
I know it is not a good way to use bazel to test, but sometimes it is too much work for migrating everything at the same time.
I can't think of an easy way to obtain all of the target names in a BUILD file, but there is a Bazel query you can run to get the target names, which you can then collect in a filegroup target to reference in the sh_test.data attribute.
bazel query 'filter(".*:test_.*\.py", kind("source file", //python/...:*) + kind("generated file", //python/...:*))'
Breaking this down:
kind("source file", //python/...:*) queries all source file
targets in the //python package recursively. This collects the
normal source files.
kind("generated file", //python/...:*) queries all generated
file targets in the //python package recursively. This collects
the genrule'd files.
filter(".*:test_.*\.py", ...) filters the results that contain
only targets in the form of //any/package:test_name.py
For example, running
bazel query 'filter(".*:test_.*\.py", kind("source file", //src/...:* + //tools/...:*) + kind("generated file", //src/...:* + //tools/...:*))'
on Bazel's own source tree finds one target: //src/test/py/bazel:test_base.py
When i build my project for coverage testing with "--coverage -fprofile-arcs -ftest-coverage", and then move the build and source to the other user directory to execute testing. I will get so many problems such as "xxx/cc/cc/getopt_log.c:cannot open source file"
the details as the below:
Processing cs/CMakeFiles/cfa/__/src/base/fault_injection.c.gcda
/home/cov/build/xfcq/src/base/fault_injection.c:cannot open source file
the path of "/home/cov/build/xfcq/src/base/fault_injection.c" is the path of build environment, how to change it as the relative path or the path I specified.
I tried to use GCOV_PREFIX and GCOV_PREFIX_STRIP, these can't work well for me.
I also tried to add -b option for lcov, it does not also work well for me.
e.g., lcov --gcov-tool=/bin/gcov -d . -b xx/src -t "xfcq" -o test_cov.info
do you have idea to resolve it?
Well for using gcov coverage process you should never move the files after building your project, instead you should modify your automated build scripts to build everything to the desired location.
When you compile your project with the specified options it generates *.gcno files for each source file which are necessarily the flow chart like details of the relevant source file.
So, the object files are instrumented in such a way that they should trigger function(added by compiler to generate coverage info ) whenever any line of statement is executed to generate *.gcda files with all the execution information.
Note: I can see that you have specified three options in question (--coverage -fprofile-arcs -ftest-coverage) which is again wrong, as --coverage works as a replacement to the other two.
If you specify only --coverage then it will do for the compilation and the linking too.(remember to use it at both the places positively though)