Reasking my older question:
Java test coverage: who covers what?
Background: I look at sonar's coverage report for a class and want to know, which test contributes to the coverage of a specific line / branch, so that it easy to got to that test and add the test for the newly introduced if-branch.
Are there other (preferably free) alternatives to clover in the IDE? Perhaps even such that they can be included into sonar ?
Or maybe tricks to enhance, accumulate information with some scripting in emma-reports ?
Or even further, patch emma or cobertura to log the required info (instead of logging a "1" for counting, one could well log the names of class under test and the test, I assume)
Thanks!
You should definitely give a try to JaCoCo. Its integration with Sonar allows to benefit from new features, for example :
merge coverage by unit and integration tests. See http://www.sonarsource.org/sonar-3-3-in-screenshots/
track the relations between tests and tested code (since sonar 3.5). You can find a screenshot on the documentation page: http://docs.codehaus.org/display/SONAR/Resource+Viewer#ResourceViewer-CoverageTab
Related
I have an full fledged android application and its source code but there is no JUnit or any other automated test cases are available for the app. How can I do code coverage test for it?
Is Jacoco any plugin like Gradle can be helpful in this scenario?
Thanks in adavance.
Our Java Test Coverage Tool can collect code coverage statistics regardless of how the application is exercised. It is not dependent on any specific unit test framework (although it can work any, including none).
If you run it with tests, you get "code coverage from tests". If you simply exercise the application manually, you get "what code does this manual operation exercise?" If you do a thorough manual exercising of the code, you get the equivalent of "what code does all of my manual testing" exercise.
I'm trying to get Sonar IT Code coverage to work for me when my unit tests are "covered" via Emma.
My situation :
(1) I have unit tests that make extensive use of JMockit. Removing JMockit is not an option. I want to get unit test coverage reports for these tests.
(2) I have integration tests that simply run the core server side portion of the app (its a Spring web app) with different input scenarios. I want IT code coverage reports for this.
For part (1) I have chosen to use Emma for my unit test coverage mainly because I was having issues with Jacoco and JMockit working together due to both tangling each other up while instrumenting the java classes on the fly. There are some issues with these tools working together - with no solution that I saw as being reliable and non-invasive to the unit tests themselves. So in the end, I chose to use Emma as its does "offline instrumentation" to perform the code coverage. Jacoco does offline as well, but I could not get Sonar to pick up its results in this mode (Maybe I forgot a semi-colon :-) ).
For part (2) [ the IT code coverage part ] I have only seen the ability to specify the following property
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
I can only assume, we need to use Jacoco to instrument the IT tests? Correct me if I'm wrong. So technically I can use Jacoco for this part of the tests because my integration tests do not use JMockit.
This is what I did. I have a separate ant task that recompiles the code base (stripping out the Emma instrumentation) and then repackages the application and runs the jacoco:coverage tool against my integration tests.
Problem is.. my unit tests are Emma based.. so how do you tell Sonar to use the Emma plugin for the Emma unit tests and the Jacoco plugin for the IT tests? Can this even be done?
Here is how I set up my Sonar properties:
sonar.test=test
sonar.sources=src
sonar.binaries=build/compile
# The value of the property must be the key of the language.
sonar.language=java
# Encoding of the source code
sonar.sourceEncoding=UTF-8
sonar.host.url=http://localhost:9000
sonar.dynamicAnalysis=reuseReports
sonar.java.coveragePlugin=emma
sonar.core.codeCoveragePlugin=emma
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
sonar.emma.reportPath=build/coverage
sonar.junit.reportsPath=build/test/report
sonar.surefire.reportsPath=build/test/report
My ant build runs the Emma unit tests (output in Emma form) and then the IT tests are run with the output to the jacoco.exec file. Then I upload to Sonar.
I get the unit test coverage stats - not the IT stats. I'm not really surprised with the results.
But does anyone have an idea where I made a wrong turn?
You already made some efforts using jacoco for your unit-test-coverage. I would continue trying to make that solution work and maybe post your difficulties with that here.
I recommend to try using jacoco for both coverage reports, because Integration-Coverage can only be analysed with jacoco, as you can read here:
http://www.sonarqube.org/measure-coverage-by-integration-tests-with-sonar-updated/
I have consulted the documentation of sonarqube and configuration-websites and did not find a possibility to use different coverage plugins.
You can find the documentation for
Unit-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Unit+Tests+for+Java+Project
and for Int-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Integration+Tests+for+Java+Project
We have our software product which is written in perl. We have setup a Jenkins build for that which runs the automated tests, to get the coverage values we run the unit tests under Devel::Cover, and get the coverage values for each module. But I find it very difficult to go and check the coverge values for each module individually, also it doesn't tell us where are we heading as far as coverage is concerned (increasing/decreasing trend). So what I'd like is to have a history or graph of those values, so that I can have a better picture of where our coverage values are and how do they compare to, say a month ago.
Is there a plug-in available that would do this thing?
I searched the Internet and found some plugins like cobertura and emma that I hope would achieve that purpose, but they are for Java. Is there a good alternative for Perl?
Also, would it be worth it to develop our own tool to display such plots? We already have the coverage data in Jenkins, and all we need is just to get that data and plot it.
You can with luck use Devel::Cover's HTML output formatting, put by default in the cover_db directory by:
$ cover -test
The install and configure the HTML Publisher Plugin.
Configure it as so:
HTML directory: cover_db
index page: coverage.html
Report title: Coverage Report
I written up my notes for setting this up, and they are available at:
https://logiclab.jira.com/wiki/display/OPEN/Continuous+Integration#ContinuousIntegration-CoverageTests
I am currently looking into the Jenkins Plugin Clover, since Devel::Cover can also output in Clover format. I will update my notes soon.
You can with luck use my CPAN distribution Task::Jenkins, installing all of the CPAN dependencies needed for the setup on the referenced wiki page.
`No valid coverage data available
Tracking and trending code coverage works best when like is compared with like. In this regard it is best to only track builds when all unit tests are passing.
This plugin will not report code coverage until there is at least one stable build.`
what should i do to have a rails stats report
#krs - From what you've said in your comments, you are correct.
rails stats only provides code coverage metrics for tests at that point in time. rails stats does not intend to keep data between builds and report test results vs. code converage.
You can connect your CI to an external service to see test coverage vs. test results. (EG, https://coveralls.io).
Please add coverage.xml file outside the project.It works fine for me.
I am using sonar for the last few months and want to know that do sonar works in an incremental way or not i.e if i do soanr analysis for the first time on my project code it will definitely analyze all the code but if i do some enhancement on my core source code and update some files then after updation do sonar analysis again on the same code then will sonar analyze all the files OR only analyze files which i have updated?
I am using "Sonar way with Findbugs" as my default quality profile.
Is there any way to use sonar in an incremental way, to analyze only updated files?
Is this possible in sonar or not?
Kindly revert your help will be appreciated..
Thanks in advance..
Sonar does not currently handle incremental analyses.
If you want, you can watch and vote for the following ticket: http://jira.codehaus.org/browse/SONAR-2815
If your task if code quality metrics in a general way, then you can use these tools directly on Jenkins. See FindBugs Plugin , PMD plugin or Checkstyle Plugin. They independently give what you need in the form of "incremental code" reports. But if you have to use SONAR only, then Fabrice's answer is the correct one.
Here the post where the ticket that #fabrice-sonar-team comments was born:
does Sonar support Incremental code quality analysis
You can read Freddy Mallet's explanation about why this functionality is not trivial to be implemented. It requires lot of effort, so as Fabrice said, vote for the ticket! :)
Just adding on since this is the first google result for 'Sonar incremental preview analysis' in stack, and the answers are way too old.
Sonar has plugins available for IDEs such as Eclipse and NetBeans that can run incremental analysis on the changed files alone. This still needs a connection the SonarQube server though.
You may also run the analysis right from Jenkins by passing an additional sonar analysis property - sonar.analysis.mode=incrementalin your Sonar scan build step. This will report the code quality in a full report - will all code issues as well as a light report containing only new issues (since last full scan as recorded in server).
To take one step further use sonar.issuesReport.html.enable=true and publish the generated html reports to your Jenkins build page - Neat and Trim
Sonar documentation here