I use JaCoCo for IT coverage in Sonar in Java language. Some IT code coverage is reported, and the reported results appear to be sound. However, I noticed that not all source code was included in the IT coverage analysis. Looking at the "Components" view, many Java packages show rules %, cobertura unit test coverage %, but nothing for IT coverage. I have not included or excluded any files, I would therefore expect IT coverage to show 0% if no code from this package was covered? I know there should be some % on a set of files, but Sonar does not display anything for it (again, rules % and cobertura % are shows).
I will try to explicitly include some class files, but I am still puzzled regarding why all sources are not included for JaCoCo analysis. The files are imported to Sonar because rules violations work fine on them.
I guess that if you do not see any results on some specific files/packages, this probably means that no IT covers those parts of your source code.
Related
Reference:
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/codecoverage-for-pullrequests?view=azure-devops
https://github.com/MicrosoftDocs/codecoverage-yaml-samples
The links above, only talk about diff coverage as part of Pull request. My questions are as follows:
1. How can we configure YAML to do full coverage instead of differencial coverage and produce the output in PR
2. When diff coverage is run, it is even considering files where [ExcludeFromCodeCoverage] is mentioned as attributes. Do we have any settings for this to be excluded from target percentage?
does anybody know if Cobertura provides possibility to automatically compare code coverage reports of two builds? If needed, I am always doing it manually in separate tabs in browser, which takes some time for larger modules
I'm trying to get Sonar IT Code coverage to work for me when my unit tests are "covered" via Emma.
My situation :
(1) I have unit tests that make extensive use of JMockit. Removing JMockit is not an option. I want to get unit test coverage reports for these tests.
(2) I have integration tests that simply run the core server side portion of the app (its a Spring web app) with different input scenarios. I want IT code coverage reports for this.
For part (1) I have chosen to use Emma for my unit test coverage mainly because I was having issues with Jacoco and JMockit working together due to both tangling each other up while instrumenting the java classes on the fly. There are some issues with these tools working together - with no solution that I saw as being reliable and non-invasive to the unit tests themselves. So in the end, I chose to use Emma as its does "offline instrumentation" to perform the code coverage. Jacoco does offline as well, but I could not get Sonar to pick up its results in this mode (Maybe I forgot a semi-colon :-) ).
For part (2) [ the IT code coverage part ] I have only seen the ability to specify the following property
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
I can only assume, we need to use Jacoco to instrument the IT tests? Correct me if I'm wrong. So technically I can use Jacoco for this part of the tests because my integration tests do not use JMockit.
This is what I did. I have a separate ant task that recompiles the code base (stripping out the Emma instrumentation) and then repackages the application and runs the jacoco:coverage tool against my integration tests.
Problem is.. my unit tests are Emma based.. so how do you tell Sonar to use the Emma plugin for the Emma unit tests and the Jacoco plugin for the IT tests? Can this even be done?
Here is how I set up my Sonar properties:
sonar.test=test
sonar.sources=src
sonar.binaries=build/compile
# The value of the property must be the key of the language.
sonar.language=java
# Encoding of the source code
sonar.sourceEncoding=UTF-8
sonar.host.url=http://localhost:9000
sonar.dynamicAnalysis=reuseReports
sonar.java.coveragePlugin=emma
sonar.core.codeCoveragePlugin=emma
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
sonar.emma.reportPath=build/coverage
sonar.junit.reportsPath=build/test/report
sonar.surefire.reportsPath=build/test/report
My ant build runs the Emma unit tests (output in Emma form) and then the IT tests are run with the output to the jacoco.exec file. Then I upload to Sonar.
I get the unit test coverage stats - not the IT stats. I'm not really surprised with the results.
But does anyone have an idea where I made a wrong turn?
You already made some efforts using jacoco for your unit-test-coverage. I would continue trying to make that solution work and maybe post your difficulties with that here.
I recommend to try using jacoco for both coverage reports, because Integration-Coverage can only be analysed with jacoco, as you can read here:
http://www.sonarqube.org/measure-coverage-by-integration-tests-with-sonar-updated/
I have consulted the documentation of sonarqube and configuration-websites and did not find a possibility to use different coverage plugins.
You can find the documentation for
Unit-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Unit+Tests+for+Java+Project
and for Int-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Integration+Tests+for+Java+Project
we try to use gcovr to generate coverage report for our c++ project in Jenkins.
I was able to get it worked, but I'm stuck with one problem. gcovr doesn't show any statistics for files with zero coverage - they have only .gcno files, no .gcda files are produced and gcovr don't show it in results.
So I have 80% coverage for the whole project, but only 2 tests were written and it's actually 80% coverage only for source files involved in tests.
For large project it makes of course no sense to use such statistic.
I have found https://software.sandia.gov/trac/fast/changeset/2766 this changeset as solution for this ticket https://software.sandia.gov/trac/fast/ticket/3887, but it seems not to be working.
Did I miss something?
p.s. I use gcovr 3.1-prerelease
I am using the Ncover 2.1.2.3625. I want branch coverage and sequence coverage. How to get this?
The .xml generated is given to the NCoverExplorer to generate the full coverage report.
But the full coverage report is not generated. What is the reason for this?
Branch coverage should be displayed for any successful coverage run, but to get symbol/sequence coverage, the PDB files for the source code must be placed in the directory with the tested assemblies.
A valid coverage file should produce a full coverage report (unless the coverage file is empty). If you can post any error messages, that would help track down the problem.