I have a large number of test cases in Testlink and I need to export them into either OpenOffice Calc or Writer (and then possibly reimport them), including test steps.
Currently Testlink only supports XML for exports and imports of test cases. To import an XML file into OpenOffice and export it, specifically formatted XSLT files are needed. To create them appears to be rather time consuming if you are not an XML expert.
Did anybody do this already? Does anybody have examples or working XSLT files for Testlink test cases, for Calc or Writer?
My bad. It turns out that it is very easily possible to export test cases with test steps:
Choose Test Reports in the topmost menu of Testlink.
On the Reports and Metrics page, select OpenOffice Writer (or MS Word) as the Report Format.
On the left side, select Test Plan Report. On the right side, select the checkbox Test Scenario (you don't need Test Summary or any other options). Below, click on the test case folder that you wish to export.
You get the document for download. (Note: may require some formatting.)
Alternatively, it is possible to choose HTML in step 2 as the report format and save the frame.
While it is possible to select OpenOffice Calc/MS Excel, there doesn't seem to be a way to include test case steps.
Can you change the test report subjects?
Choose Test Reports in the topmost menu of Testlink.
On the Reports and Metrics page, select OpenOffice Writer (or MS Word) as the Report Format.
On the left side, select Test Plan Report. On the right side, select the checkbox Test Scenario (you don't need Test Summary or
any other options). Below, click on the test case folder that you wish to export.
You get the document for download.
Can i change the download file name?
When i choose the testcase folder, i can download the test report(MS word). but file name is "Test Project name.doc". so I want to change the file name from "Test project name" to "Test Plan name"
Related
I have the Enterprise edition of Visual Studio, but not everyone does.
How can I share the TRX so that other people can see the results? I've seen that there are several Trx to HTML convertors but they're all quite old and either don't work any more or don't show the details of the test.
Most of the results of a load test are stored in a database. The TRX file has only a very small portion of the results - you can easily see what it actually contains by opening the file in Notepad, it is just XML.
Visual Studio trial versions have some support for load tests. (They only support a small number of virtual users and do not support plugins.) They might allow viewing and analysis of the results of runs done on other computers. This limited support of load tests might also be available on licenced Non-Enterprise versions.
The results of individual load test runs, or groups of runs, can be copioed from one computer to another by using the Export and Import commands in the "Open and manage load test results" window. These create and read LTRAR files. See here for more details.
The TRX file contains a connection string to the SQL database and so the file cannot normally be copied and then work properly on another computer. The encoded connection string is located within the resultsRepositoryConnectString=... attribute within the TRX file. Changing the value in a copied file to a value found in a working TRX file on the destination computer allows the copied file to work. See here for more details.
The only other options for viewing results on other computers would be by exporting the results in Excel by using the "Create Excel Report" command. (Access it via the icons on the "Summary - Graphs - Tables - Details - ..." line in the load test results viewer.)
For unit tests, but not load tests, I routinely export trx files from my unit tests, because they ARE shareable, can be placed as attachments into our launch control systems, and DO contain all of the data necessary for the downstream teams to verify the results, including test console output. I do this with a custom task added to the project that runs the vstest.console.exe, with the /logger:trx;logfilename="blah" command line switch. The full task line is:
<Exec Command=""$(DevEnvDir)commonExtensions\Microsoft\TestWindow\vsTest.console.exe" "$(MSBuildProjectDirectory)\$(OutputPath)$(AssemblyName).dll" /logger:trx;LogFileName="$(MSBuildProjectDirectory)\testResults.trx"" />
You can also add ContinueOnError="true" if you don't want a test failure to stop the build.
MSTest still seems to ship with visual studio, supposedly does support load tests, and historically has exported TRX files, but I haven't tried it. Running pre-2019 unit test projects, that still have tasks pointed to mstest, in vs2019 tend to fail with assembly version errors, so I don't know if it even still works at all.
We can import the test cases with their associated steps from excel to TFS.
Is there any way or tool for exporting the test cases with their associated detail steps into excel?
Currently i am using TFS 2015.
You can query the test case work items and select Steps Column in VS, then export the query to Excel directly. But you need to reorganize the format of the Steps:
You can export test artifacts data on TFS Web Access so that they can either be sent via email or printed. The Export capability allows you to customize what data you want to export. https://msdn.microsoft.com/en-us/library/dd380763.aspx
For exporting to Excel, you can choose the Print or Email option from
the Export dialog box, then choose Cancel from the Print dialog box.
The data for the report is displayed. Select the test cases and
associated steps, then you can copy and paste it into a Excel if you
want. All the formatting in the report is retained.
I need to see my test's result in the summary page. The test report in custome format, so I can't use 'in-box' testing task of TFS, but I would like to have the same behaviour as for Unit tests:
statistic in summary page with link leading to Test details tab
Tab with tests details in build summary
TestRun in 'Test->Run' category (optional)
Detail information for each test.
I found how to make it in 'Test->Run' with this link How to create a test run and result using the Team Foundation Server API? (many thanks to author). In this solution Test created base on Test Plan and Test Suite. According to my experience, the Unit test creates Test Run base on Build, and I belive that should be TFS API to do it programmatically.
Could someone suggest me where such API could be found. And if p1 and p2 possible to make and how?
For your requirement, you may have to create a section or a tab by writing your own vsts extension to display the info in build result report and build summary.
You can upload the xml result file as 'testrun-attachment' via REST-interface and got the pieces in place to show the summary values on the build summary tab or on your own extended “test result” tab.
Here is a example with how to display opencover result for your reference: vsts-opencover, finally will get the result as below:
How to write your own extension:
Overview of extensions for Visual Studio Team Services
Some examples :
https://github.com/Microsoft/vsts-tasks
https://github.com/Microsoft/vsts-extension-samples
We have Fortify SCA and we are setting up regular, automated scans of our source code. Our intention is to have an alert if there is an introduced security issue. Is there a way, perhaps using FPRUtility (or some other method) to accomplish this? Ultimately I prefer something that can be easily run from the command line, but if this can also be accomplished using the GUI then I would appreciate knowing how to do that as well.
Use Audit Workbench to run a report. Choose "developer workbook" and disable all except one section. (you can choose any section you want).
In the report section's additional properties, set the filter for the issues to [issue age]:new. This means the report will show ONLY issues in your FPR that were not present in the previous scan, and were introduced in the latest scan. Save the template.
In your scan configuration, make sure to scan to the same FPR every time per project, so that "new" issues can be calculated by the report runner.
After the scan is complete, use the answer by #user1836982 to run the report. Choose the XML template and process it programmatically.
(1) Command for the Fortify report generation to XML FORMAT:
FORTIFY_INSTALL_DIR\bin\ReportGenerator.bat -format xml -f target_file_name.xml -source your_fpr_file_name.fpr -template Detailed-DefaultReportDefinition.xml
(2) you can also use AWB to generate the .pdf/.rtf/.xml report by Report(top menu bar) -> save report -> select format ->save
(3) Just added procedure to create excel sheet here: Export HP Fortify SCA 4.10 results in EXCEL format
(4) If you have access to DB (oracle), you can query with script
If you are using Fortify SCA, you should also have access to Fortify Software Security Center (SSC). SSC can be used to track trending data across builds of a project. SSC has built in capabilities to send out alerts based on user-defined events within SSC; I have never worked with those so can't offer any thoughts other than what the docs say.
The reports generated by Fortify SCA (.fpr files) are zip files XML documents storing all the relevant data; I would suspect some of the data in those files are related to the SCA rulesets that are present in both SCA and SSC instances. I suspect without the rulesets you would be able to determine that new issues have been introduced, but not any good data on what they are, priority level, etc.
Is there any tool exist that transfroms trx file into Excel format?
I am looking for something that will automate unit test result publishing at the time of test suite execution (without requiring microsoft team server)
You can use http://trx2html.codeplex.com/ which provides an easy way to convert an rtx file into an html document.
If you just need the information about the test name and whether passed or failed, you could copy the columns from Test Results UI (in Visual Studio) and paste on excel.
I created a tool for this propose, the trx file will be converted to Excel 2010 format with useful information.
Pls try: https://youtu.be/G7VASkhim7I
Instructions
Download the Trx2Excel application from here and run the application.
Select the option: Get data from one file or from a folder.
Select the result file (for single file option) or result folder (for multiple files option).
Select the location and the saved file (the output file).
Click the Convert button.