I'm in the process of (finally!) setting up code coverage monitoring for my brand new C++ project. Due to the fact that I need some advanced C++20 features (read, coroutines), I am using clang 6 as compiler.
Now, I followed this guide on how to do basic code coverage for your project, and everything worked like magic. If I do:
clang++ -fprofile-instr-generate -fcoverage-mapping test.cpp -o test.out
LLVM_PROFILE_FILE="coverage/test.profraw" ./test.out
llvm-profdata merge -sparse coverage/test.profraw -o coverage/test.profdata
llvm-cov show ./test.out -instr-profile=coverage/test.profdata
I get a nice, colored report on my terminal that tells me what is covered and what is not.
So far so good! I thought I was close to what I wanted, but then the pain started when I tried to get the report uploaded to codecov.io.
I have tried a few things, including:
Running their https://codecov.io/bash script on my coverage folder in the hope that maybe it would catch on my test.profdata. No dice, and it makes sense, since even llvm-cov needs the path to the executable file to run.
Using the export functionality: when running llvm-cov export --instr-profile=coverage/test.profdata ./test.out I get a good-looking JSON file (via terminal). I tried throwing the output in a coverage.json file, which actually got uploaded, but then codecov just says that there was an error parsing it, with no further information.
I'm feeling completely lost. Everything seems so black-box-ish on their website that I just don't understand how to get anything done that doesn't by chance perfectly fit the cases that they can manage.
How can I get this working with codecov? If codecov can't handle my reports, is there any other equivalent online code coverage that I can use to get this to work?
It looks like the bash script codecov uses to upload coverage data to their site looks for files matching a wide range of patterns associated with formats that it understands. These are poorly documented, but you can at least see which patterns are viable by looking at the script on Github. Of course, this doesn't tell you what expectations codecov has about the format of files matching a given pattern, as you discovered when your coverage.json file was rejected.
Through trial and error I have found that the following produces a file that codecov will interpret correctly when you run the bash script:
llvm-cov show ./test.out -instr-profile=default.profdata > coverage.txt
I haven't extensively tested what file names are allowed, but it seems that you can put whatever additional characters you want between coverage and .txt in the name of the file that you're piping the coverage data to (e.g. you could call it coverage_my_file_name.txt).
EDIT: Just in case this is helpful to anyone, it turns out that an important corollary to the above is that it's critical that you avoid naming anything that isn't a coverage report something that matches this pattern. I just dealt with a scenario where I had a bunch of executables named coverage_[more_text_here].out that were getting uploaded with the reports. It turns out that attempting to parse assembly code as a coverage report can cause codecov to mysteriously fail without any useful errors.
Another option is to use GCOV profiling, which is a little less precise than source-based, but it is supported by codecov.io. You need the "--coverage" compiler flag to enable it.
You can use grcov (which you can also download from https://github.com/mozilla/grcov/releases) to parse the gcno/gcda files and upload them via the codecov.io bash uploader:
grcov OBJ_DIR -s SRC_DIR -t lcov --branch > lcov.info
bash codecov.sh -f "lcov.info"
I'm planning to add support for source-based reports to grcov, which will make it easier to support the format on codecov too.
Related
I followed The Clang manual and used __llvm_profile_write_buffer to collect coverage cprofile data inside the instrumented program.
This works well, but to actually generate a coverage report the recommended way is to use the llvm-cov tool like this:
llvm-cov show ./foo -instr-profile=foo.profdata
This tool needs access to the binary which does not play well with __llvm_profile_write_buffer .
Is there a way to generate a coverage report similar to what llvm-cov does, but inside the process, from the buffer updated by __llvm_profile_write_buffer ?
I guess this would involve accessing the symbol table from within the process, which I think is doable?
Use case : I would like to upload the coverage report from within the process to a remote server without having to execute an external tool.
Thanks for your help,
Antoine
It seems an obvious improvement, in Kate, to jump directly to the error location given in a gcc error message. It would great if it can be made to work directly in Kate's terminal, but I would settle for a tool that can yank text off the x clipboard. Failing that, is there a way to write an add-on for Kate to do this?
Thanks.
Mike.
from the terminal we can run something like this:
$ kate source.cpp --line=45
you could write a script to parse gcc output and re-open kate to that line in the source. If kate already has the source code open, it will simply emulate a jump to the line.
There is "Build" plugin for Kate, which does also stderr parsing, and understand GCC error syntax.
So for example you can easily configure in Kate to build your project with make (and KDevelop is sharing this part of Kate source and plugin, although KDevelop has extra plugins for project management), and you will see in the Build plugin output parsed errors, clicking on them will direct you to the particular source file and line.
(but I was just looking around if it can even mark the error lines in the source visually, and actually there's nothing like it? From my limited understanding of the Kate plugin API I would be slightly afraid this is not trivial to add to current Build plugin, but I didn't dig into the Kate sources enough to fully understand its architecture and what are the true limits for plugin code).
I am trying to automate a repetitive manual process for which I use WireShark:
1) Load a given pcap file
2) Apply a simple filter for a given protocol
3) Use the export dialog box to export the displayed packets to CSV file
4) Use the export dialog box to export the displayed packets in XML PDML form.
This is tedious, and requires human involvement in the middle of a process that is mostly automated (including the analysis of the files to produce reports).
Is there some way to either automate Wireshark, or do somehow access the underlying libraries used for export?
UPDATE: As several people here indicated, TShark turns out to be the way to go.
The exact command line I ended up using is:
tshark -r MyDataFile.pcap -T pdml -R MyProtocol > MyOutputFile.xml\
I then use an event based XML parser (Python's expat) to parse the generated 2GB file
I watched at the dependency list of wireshark on my debian system, and I found Tshark: it's the command line version of wireshark.
It seems interesting, but I didn't read the manual yet, however it's for sure more script friendly.
Also I'll stay tuned on this thread and post more info when I'll start using it.
I think what you should do is look into tshark. That is the linux command line version, which will allow exactly what you ask for (assuming you have access to it). And of course, this assumes that it is acceptable to run tshark and then review the outputs manually.
I haven't ever tried to automate Wireshark before, though I have had to do something similar to what you describe. I ended up reducing the handful of human (and thus error-prone) steps to one step that was automated.
Autohotkey is my solution for lots of repetitive GUI-based tasks. You can very easily write a keystroke playback script that will do all of the above steps. You'll probably have to have it increment the filename for you automatically. You could also have your other automated tool kick off the Autohotkey script with a keystroke.
My problem is to create an ant target for automating our installer running in console mode.
The installer is created using InstallAnywhere 2008, which UniversalExtractor recognizes as a 7-zip archive. Once I have the archive unpacked, it appears that the task can use an input file to drive the console (at the very least, it appears that emitting a quit shuts everything down correctly, and output is captured).
So it looks to me as though I have all of the pieces I need for proving out this idea except a clean way to perform-self-extraction-then-stop. Searching for a command-line argument to stop the auto execution has not produced a likely candidate, and the only suitable ant task I've found ( http://www.pharmasoft.be/7z/ ) isn't so clearly documented that I have a lot of confidence in it.
The completed completed is expected to work in Windows, Linux, and a small handful of other Unix environments.
What's the best practice to use here?
Since you control the installer creation, can you run the self-extraction step on your machine, package the results before the installer is launched in a ZIP file, etc. and use that instead of the single file executable? Not very elegant but it may work.
Also, I am a bit hesitant to blatantly promote my project :) but since it has been a while since you asked the question and nobody has answered, have you considered an alternative? Our project InstallBuilder allows you to install in unattended mode directly, without having to autoextract the contents. Just invoke the executable with --mode unattended, pass any additional options you may need from the command line or an external file and you are good to go. We have a lot of ex-InstallAnywhere customers :)
I'm using P4COM to communicate with our perforce server. I have written an little utility to simplify our QA of what files have changed from one release to another. I have been using the P4COM interface from Delphi. So far so good.
I though it might be nice to allow users to view the diff between the two versions of the file from within my little utility rather than going back to p4v. So I print (get) the files at each revision using p4COM and the following command
print -o "E:\Development\TempProjects\p4Changes\temp\File_dispatch.pas#25" "//depot/mydepotpath/File_dispatch.pas"#25
and
print -o "E:\Development\TempProjects\p4Changes\temp\File_dispatch.pas#26" "//depot/mydepotpath/File_dispatch.pas"#26
However when I do this from my app using P4COM I seem to get random files (and they appear to be deleted ones). If I run the exact same command from the command line I get perfect results. Running both of these does return a file and correctly dumps it to disk where I want it, its just not the file I've asked for.
Any ideas?
Could it be a backslash issue in the command string? This would work fine at the command line, but a single backslash may be being interpreted as an escape code by whatever language compiler you are using (if C or C++, then this will definitely cause a problem, and that maybe happening under the hood on the P4COM side).
Try using double backslashes and see if that fixes it.
You're probably better of asking this to Perforce support itself, as this sounds like a bug in their software.
As a sidenote : Why do you use p4v? (I hugely prefer p4win myself)