I am currently using Karma's coverage within a project, I would like to enforce a threshold for the coverage to be set at and therefore make my builds on Circle CI to fail and go red due to it being lower than a set percentage.
You can install karma-threshold-reporter and configure it as shown in the README.
Related
With codecov, you can set the target coverage for your project. Assuming a project has 90% coverage right now,
coverage:
status:
project:
default:
target: auto
threshold: 1%
this will allow the next pr to reduce the coverage only to a min. of 89%. But with subsequent PRs this could keep dropping. Whereas,
coverage:
status:
project:
default:
target: 80%
will only allow the PRs which maintain the project coverage above 80%. But a single PR might drop the coverage from 90% to 80%.
Is it possible to have configurations which is like a combination of these two, where a new pr can reduce the project coverage by a max of 1% but the coverage should not go below 80%.
You should be able to set both the target value and the threshold value.
You can also leverage flags to track coverage more granularly.
You can also set "patch" coverage targets, which might be more important since that will only "consider" the newly added or modified production code. Thus you can start with a low project coverage, but have stricter standards for your patch/diff coverage.
How to make jenkins build pass if cucumber JVM scenarios pass percentage is 80%
The cucumber reports plugin can be used for this.
The plugin has a lot of options where the build is marked as failure or unstable.
For example:
failedScenariosPercentage (optional)
Maximum percentage of failed scenarios above which build result is
changed.
My managers want we to determine which tests might have to be run, based on coding changes that were made to the application we are testing.
But, it is hard to know which tests are actually needed to be re-verified as a result of a code change. What we have done is common to test the entire area where the code change occurred / or the entire proj, solution.
We were told this could be achieved by TFS build or MTM tools. Could someone share the details?
PM:We are running on TFS 2015 update4,VS2017.
There is a concept of Test Impact Analysis which helps in analysis of impact of development on existing tests. Using TIA, developers know exactly which tests need to be verified as a result of their code change.
The Test Impact Analysis (TIA) feature specifically enables this – TIA
is all about incremental validation by automatic test selection. For a
given code commit entering the pipeline TIA will select and run only
the relevant tests required to validate that commit. Thus, that test
run is going to complete faster, if there is a failure you will get to
know about it faster, and because it is all scoped by relevance,
analysis will be faster as well.
Test Impact Analysis for managed automated tests is available via a checkbox in the 2.* preview version of the VSTest task.
If enabled, only the relevant set of managed automated tests that need to be run to validate a given code change will run. Test Impact Analysis requires the latest version of Visual Studio, and is presently supported in CI for managed automated tests.
However this is only available with TFS2017 update1(need 2.* preview version of VSTS task). More details please refer this blog: Accelerated Continuous Testing with Test Impact Analysis
Try to use python green to control python (2.7.x) code quality, and integrated it within jenkins.
In nose, there is --with-xunit parameter to generate nosetests.xml unit test.
In coverage, it is possible to use xml paramter to generate like coverage.xml coverage report.
Both are very good to be integrated with jenkins.
Do you know how to achieve this in green ?
At the moment, Green does not support any means of producing xml output or integrating with Jenkins.
However, subunit support is planned. Once subunit support is in place, you should be able to hook it up to Jenkins.
Does somebody know about the following scenario:
I need to
calculate an apex code coverage during deployment with Migration tool
give the result on console
after that I need to decide about continuation deployment. If I will give less that 88% (for instance) of code coverage I need to mark build as failed in Jenkins and to stop a deployment, in other case I can go ahead.
So, what I need is just ability to set any value (greater 75) as required test coverage for my project.