After running test cases, I do not want to show 'Failed Test by Reason' and 'Failed Test by analysis' pie charts until they have a value assigned. As soon as i select a 'failure type' and 'resolution' for these two charts should show up. if i don't mark 'failure type' and 'resolution' for a failed test case these pie charts should not be displayed. Yet 'Current state of Test' pie chart should be displayed after each test run.
Is there any way to achieve it?
Also, i need to know what kind of scripts (i.e. js/VB/C# or some other format like batch file) are accepted as setup and cleanup script in automated test setting in test manager.
Hiding the Pie charts is not possible, we cannot change the UI of MTM it is designed to show the charts. These charts are added to give us an overview of the reason for test failure and analysis of failure.
It's a general practice to use batch file scripts as setup and clean script in Test Manager.
Related
I have a few thousand test cases in the Zephyr Jira plugin. Now I have created some custom fields for the test cases via Zephyr and would like to change the content there.
Unfortunately I can't understand the API description of SmartBear. Does anyone know what the exact call is to change the content of a Zephyr custom field?
We run CI tests via Jenkins and review the results via Jenkin's Console Output. If one of the tests doesn't pass, it is printed to the Console in red. Nevertheless, the output is very long and it is tedious to search the entire log to find instances of red output. Are there any good solutions to the issue?
For instance, Gerrit's user interface displays file diffs and a user can use keyboard shortcuts n or p to jump to the next/previous diff block. Is there any such solution/plugin... for Jenkins?
There's the Collapsing Console Sections Plugin:
This plugin allows the creation of sections in build consoles.These sections can be individually collapsed to hide unimportant details. A floating outline widget is available to navigate through all the sections of a build.
and the Console Parser Plugin:
The console parser plugin parses the console log generated by the Jenkins build allowing for:
...
highlighting of errors, warnings , and info
...
separating the comparison log by sections
collapsible sections for faster viewing
You should use a reporting plugin in Jenkins corresponding to the test framework you have used in your project. e.g. If you are using TestNG , then Jenkins has a publish TestNG reports plugin for filtering out test results.
Similarly for Cucumber - there is pretty cucumber reports plugin.
I have multiple jobs running on Jenkins, with some outputs/graphs to measures code quality (warning, unit test result, indicator plots).
How can I create a view that summarize all these informations on one page ?
You can write a script that uses the Jenkins Remote Access API to query the status of various jobs and print a summary.
Then, you can run this script itself as Jenkins job. The summary will appear in the build log of the job.
Another way to do this is to use the dashboard view
The following are fully integrated :
joblist
warning
cppunit test result (each job should publish its result as a post-build action)
cppcheck result (each job should publish its result as a post-build action)
plot display (using the image portlet)
It still dont know how to display Doxygen link...
I have one JMeter script. I want to run it in ant script(as ant provide result in html page). I also want graph result in that or new Html page. I heard i can do with jfreechart but i don't know how to do it.
Please help me!!!
I'd suggest to pick one of the following options:
Each "Graph" listener has `Write results to file/Read from file" section. You can execute JMeter via Ant task with i.e. Graph Results listener enabled saving output to some file, after that open .jtl or .csv file in Listener and you'll get your graph which you can save to image format and add it to HTML report.However using Graph listeners it isn't very recommended from performance perspective.
If you want some HTML charts it's better idea to consider Google Charts instead of JFreeChart, however some basic XSLT/Javascript knowledge will be required to build graph from XML data.
The fastest and easiest way is using Blazemeter Plugin to JMeter, it's free, can be executed via command line, Ant task, GUI, whatever - it's just another listener which provides professional-looking reports and also has feature of comparing test runs.
Rather than trying to do this yourself, have you considered using the Performance Plugin for Jenkins? It may alreay have the charts you need.
If you still need to develop your own charts, you will need to write a plug-in for JMeter; charts are on page 8.
Why not use JMETER PLUGINS great project which contains all graphs you dream about:
http://jmeter-plugins.org/wiki/ActiveThreadsOverTime/
http://jmeter-plugins.org/wiki/ResponseTimesOverTime/
http://jmeter-plugins.org/wiki/TransactionsPerSecond/
http://jmeter-plugins.org/wiki/HitsPerSecond/
http://jmeter-plugins.org/wiki/CompositeGraph/
http://jmeter-plugins.org/wiki/ResponseCodesPerSecond/
http://jmeter-plugins.org/wiki/LatenciesOverTime/
http://jmeter-plugins.org/wiki/BytesThroughput/
Is there a way to mark a fitnesse test such that it will not be run as part of a suite, but it can still be run manually?
We have our FitNesse tests running as part of our continuous integration, so new tests that are not yet implemented cause the build to fail. We'd like a way to allow our testers and BAs to be able to add new tests that will fail while still continuing to validate the existing tests as part of continuous integration.
Any suggestions?
The best way to do this is with suite tags. You can mark tests with a tag from the properties page and then you can filter for the or filter to exclude them.
In this case I would exclude with "NotOnCI" tag. Then add the following argument to the URL:
ExcludeSuiteFilter=NotOnCI
This might look like this then as the full URL:
Http://localhost:8080/FrontPage?test&ExcludeSuiteFilter=NotOnCI
You can select multiple tags by splitting with commas, but they act as "or",Not "and".
Check the FitNesse user guide for more details. http://fitnesse.org/FitNesse.UserGuide.TestSuites.TagsAndFilters
Would it make sense to have multiple Suites, one for regression tests that should always pass, and another one for the tests that are not yet implemented?
Testers and BAs can add tests/suites to the latter suite and the CI server only runs tests in the former suite.
Once a developer believes he has implemented the behavior they can move the test/suite relating to that functionality to the 'regression' suite so that it will be checked in continuous integration.
This might make the status of a test/suite a bit more explicit/obvious than just having a tag. It would also provide a clear handover from development to test/BA to indicate the implementation is finished.
If you just want to have a test/suite not run during an overall run of a suite that contains the particular test/suite you could also just tick 'Skip (Recursive)' in the properties page of that test/suite (below 'Page Type').