I faced problem during PIT Mutation Testing. I am able to build maven and able to run Android project as PIT Mutation test. After that, I am getting one index.html, with no coverage file. Can anyone suggest me how to test our Class through PIT Mutation
After set up robolectric,right click on test project and choose, run as Pit mutation test from Eclipse, and the below Report
Related
I'm trying to filter out some unit test marked with the xunit trait category. On my build server I don't want unit test to run with category integration
[Trait("Category", "Integration")]
When I don't use the 'Test Filter criteria' in TFS VS test step, the unit test all get executed. But when I apply the desired filter
Category!=Integration
It doesn't run any tests at all. I've tried all variations but no success. I also tried using the 2.* version of the test task.
Instead of using [Trait("Category", "Integration")], use [Trait("TestCategory", "Integration")]. XUnit doesn't care what you put in for these key-value pairs, but the TFS Test runner task DOES. The Test Filter Criteria box only works with a prescribed set of attributes. I have this working for our builds. Ran into the same problem you are describing. After analyzing the build output, (and getting ALL test projects switched to XUnit - that is also a key... don't have a mix of MSTest and XUnit).
Now (Februaray 2022), this is not problem anymore. However, I have had some issues getting it working and I have ended up here anyway, so just couple of points:
Must be in such configuration built where the test project is really built (e.g. in Release the test project is often not built)
Multiple detected test dlls seem to cause trouble (better to filter just really those you want)
The name of the trait (and the value ) can be arbitrary (16.8 and 17.0.0 at least), i.e. the text in Test Filter Criteria does NOT need anymore to be some of the vstest standard names, like TestCategory
I am working on a project in Xcode for iOS app and I enabled "Gather coverage data" in the "Edit scheme -> Test".
I also created a test target with an empty unit test, the only one in the whole project so far and now when I run the test I can see that around 12% of code has been covered (The project has around 500 files in the project and there are many classes in the coverage report that are displayed as covered).
How is this possible? Did I miss something?
In wiki there is "In computer science, code coverage is a measure used to describe the degree to which the source code of a program is tested by a particular test suite." We have only one test suite and it is empty.
As of Xcode 8, running the tests actually instantiates the app. That means that all the initialization code is executed. This will show as covered code in the test coverage report.
I have an full fledged android application and its source code but there is no JUnit or any other automated test cases are available for the app. How can I do code coverage test for it?
Is Jacoco any plugin like Gradle can be helpful in this scenario?
Thanks in adavance.
Our Java Test Coverage Tool can collect code coverage statistics regardless of how the application is exercised. It is not dependent on any specific unit test framework (although it can work any, including none).
If you run it with tests, you get "code coverage from tests". If you simply exercise the application manually, you get "what code does this manual operation exercise?" If you do a thorough manual exercising of the code, you get the equivalent of "what code does all of my manual testing" exercise.
I'm trying to get Sonar IT Code coverage to work for me when my unit tests are "covered" via Emma.
My situation :
(1) I have unit tests that make extensive use of JMockit. Removing JMockit is not an option. I want to get unit test coverage reports for these tests.
(2) I have integration tests that simply run the core server side portion of the app (its a Spring web app) with different input scenarios. I want IT code coverage reports for this.
For part (1) I have chosen to use Emma for my unit test coverage mainly because I was having issues with Jacoco and JMockit working together due to both tangling each other up while instrumenting the java classes on the fly. There are some issues with these tools working together - with no solution that I saw as being reliable and non-invasive to the unit tests themselves. So in the end, I chose to use Emma as its does "offline instrumentation" to perform the code coverage. Jacoco does offline as well, but I could not get Sonar to pick up its results in this mode (Maybe I forgot a semi-colon :-) ).
For part (2) [ the IT code coverage part ] I have only seen the ability to specify the following property
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
I can only assume, we need to use Jacoco to instrument the IT tests? Correct me if I'm wrong. So technically I can use Jacoco for this part of the tests because my integration tests do not use JMockit.
This is what I did. I have a separate ant task that recompiles the code base (stripping out the Emma instrumentation) and then repackages the application and runs the jacoco:coverage tool against my integration tests.
Problem is.. my unit tests are Emma based.. so how do you tell Sonar to use the Emma plugin for the Emma unit tests and the Jacoco plugin for the IT tests? Can this even be done?
Here is how I set up my Sonar properties:
sonar.test=test
sonar.sources=src
sonar.binaries=build/compile
# The value of the property must be the key of the language.
sonar.language=java
# Encoding of the source code
sonar.sourceEncoding=UTF-8
sonar.host.url=http://localhost:9000
sonar.dynamicAnalysis=reuseReports
sonar.java.coveragePlugin=emma
sonar.core.codeCoveragePlugin=emma
sonar.jacoco.itReportPath=build/coverage/jacoco.exec
sonar.emma.reportPath=build/coverage
sonar.junit.reportsPath=build/test/report
sonar.surefire.reportsPath=build/test/report
My ant build runs the Emma unit tests (output in Emma form) and then the IT tests are run with the output to the jacoco.exec file. Then I upload to Sonar.
I get the unit test coverage stats - not the IT stats. I'm not really surprised with the results.
But does anyone have an idea where I made a wrong turn?
You already made some efforts using jacoco for your unit-test-coverage. I would continue trying to make that solution work and maybe post your difficulties with that here.
I recommend to try using jacoco for both coverage reports, because Integration-Coverage can only be analysed with jacoco, as you can read here:
http://www.sonarqube.org/measure-coverage-by-integration-tests-with-sonar-updated/
I have consulted the documentation of sonarqube and configuration-websites and did not find a possibility to use different coverage plugins.
You can find the documentation for
Unit-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Unit+Tests+for+Java+Project
and for Int-Test-Coverage here: http://docs.codehaus.org/display/SONAR/Code+Coverage+by+Integration+Tests+for+Java+Project
Reasking my older question:
Java test coverage: who covers what?
Background: I look at sonar's coverage report for a class and want to know, which test contributes to the coverage of a specific line / branch, so that it easy to got to that test and add the test for the newly introduced if-branch.
Are there other (preferably free) alternatives to clover in the IDE? Perhaps even such that they can be included into sonar ?
Or maybe tricks to enhance, accumulate information with some scripting in emma-reports ?
Or even further, patch emma or cobertura to log the required info (instead of logging a "1" for counting, one could well log the names of class under test and the test, I assume)
Thanks!
You should definitely give a try to JaCoCo. Its integration with Sonar allows to benefit from new features, for example :
merge coverage by unit and integration tests. See http://www.sonarsource.org/sonar-3-3-in-screenshots/
track the relations between tests and tested code (since sonar 3.5). You can find a screenshot on the documentation page: http://docs.codehaus.org/display/SONAR/Resource+Viewer#ResourceViewer-CoverageTab