Scenario:
We're trying to enable code coverage in our TFS and so far the build summary result was successfully displaying code blocks covered as shown in picture below but the problem is every time we click Coverage Results link (arrow down in the picture) it doesn't shown the details of what has been covered and it display error in the browser even the test results In/Out folder have code coverage file in build server.
Setup:
MSTest Settings (Local.testsetting)
Test Run
Build Definition
Test Results
Build Summary:
After clicking Coverage Results:
Question : Why TFS 2013 project collection Coverage Results link (red arrow down) in build summary doesn't display code coverage details of what has been covered even the code coverage file was generated in build server TFS 2010? What do we missed in the settings to make it possible?
You need to
1). open the build definition, go to Process -> 3.Test -> Automated Test -> on the Add/Edit Test Run dialog, set the Options to be "Enable Code Coverage".
2). Select the .testsettings file in the build definition.
You can also to have a check on Ralph Jansen's reply in this question TFS 2013 - No Code Coverage Results
Moreover, also check could you open this code coverage link in web portal.
Update
add related screenshot
Related
I need to see my test's result in the summary page. The test report in custome format, so I can't use 'in-box' testing task of TFS, but I would like to have the same behaviour as for Unit tests:
statistic in summary page with link leading to Test details tab
Tab with tests details in build summary
TestRun in 'Test->Run' category (optional)
Detail information for each test.
I found how to make it in 'Test->Run' with this link How to create a test run and result using the Team Foundation Server API? (many thanks to author). In this solution Test created base on Test Plan and Test Suite. According to my experience, the Unit test creates Test Run base on Build, and I belive that should be TFS API to do it programmatically.
Could someone suggest me where such API could be found. And if p1 and p2 possible to make and how?
For your requirement, you may have to create a section or a tab by writing your own vsts extension to display the info in build result report and build summary.
You can upload the xml result file as 'testrun-attachment' via REST-interface and got the pieces in place to show the summary values on the build summary tab or on your own extended “test result” tab.
Here is a example with how to display opencover result for your reference: vsts-opencover, finally will get the result as below:
How to write your own extension:
Overview of extensions for Visual Studio Team Services
Some examples :
https://github.com/Microsoft/vsts-tasks
https://github.com/Microsoft/vsts-extension-samples
We are using the Team Foundation Server 2015 and our build task uses Report Generator for generating the Code Coverage view.
Now we are trying to get something similar without any third party libraries.
If I look at the build details, under the section "Code Coverage", I have two charts (Blocks and Lines) and beneath these a link "Download Code Coverage Results". And that's all. If I download the file and open it with Visual Studio I can see the details.
Is there any built-in way to generate a webpage or anything else which shows more details on the TFS itself?
There is no out of box feature for this, you can submit a feature request on VSTS User Voice.
I'm currently trying to run custom code coverage using OpenCover and to generate output in easily readable format using Report Generator. The final output of these two applications is xml files.
I am running these applications using InvokeProcess activity in build definition to get their outputs but the output are currently present only in my build controller's Build binaries folder.(ex:C:\Builds\1\project_collection\build_definition\bin)
I want to save these files per build in some location and to retrieve them back when needed by clicking on a hyperlink present in Build summary similar to how built-in code coverage results are displayed in tfs build summary.
I tried this question but as you can see there is no correct answers listed there.
So how do I copy these output files to some remote location and refer them in my build summary to retrieve them?
Copy outputs to some remote location is similar to copy outputs to drop folder. You can simply specify the drop location in Build Definition--Build Defaults--Staging location. Check the screenshot below:
Refer the binaries in your build summary to retrieve them, you can add a new activity “WriteCustomSummaryInformation” to your build process template, which has several properties that allow you create a new Section and add text to that section. Check this blog for more information: http://blogs.msdn.com/b/jpricket/archive/2012/06/07/tfs11-rc-simple-build-summary-customization.aspx
As I also answered here Display OpenCover results in TFS 2015 build Summary, it is now possible to customize the build results in TFS 2015.
I managed to upload the OpenCover xml result file as 'testrun-attachment' via REST-interface and got the pieces in place to show the summary values on both the build summary tab and on its own extended “build-results-view”.
I have generated my xml coverage file as part of CI build on the TFS server using vNext 2015 build definition. How would one display the results in the TFS 2015 summary ? Either using the xml report or the html generated using the ReportGenerator.
You need to convert the results produced by OpenCover to a format which can be parsed by TFS.
One way to do this would be to use the OpenCoverToCoberturaConverter NuGet package.
Once you have that, you can use the Publish Code Coverage Results build step.
I have described the whole process on my blog.
In “TFS 2015 - Update 2” this is possible by writing your own vsts extension (see here: https://www.visualstudio.com/en-us/docs/integrate/extensions/overview ).
I set up my own 'learning project' for building this as .vsix here: https://github.com/RobertK66/vsts-opencover.
My custom build step uses nunit3 console runner to execute tests under opencover.
I managed to upload the OpenCover xml result file as 'testrun-attachment' via REST-interface and got the pieces in place to show the summary values on both the build summary tab and on its own extended “build-results-view”.
Base for this first version were a lot of examples provided by MS on github: https://github.com/Microsoft/vsts-tasks and https://github.com/Microsoft/vsts-extension-samples
To get a first feeling what places on your TFS Web Portal can be extended/customized you can download and install this extension https://aha.gallery.vsassets.io/items?itemName=ms-samples.samples-contributions-guide from the Visual Studio Marketplace.
Hope this helps to get you started.
Currently, these customizations are not supported. You can't edit the displays in new build summary. However, you can customize Code Coverage Analysis.
You can submit it to User Voice site at:
http://visualstudio.uservoice.com/forums/121579-visual-studio
Instead of using OpenCover extension, it is very convenience for you to generate code coverage result and include it in the build summary page this way:
Select the Visual Studio Test step, check the Code Coverage Enabled option.
Then, the code coverage result shows on the build summary page:
I am having a problem with the TFS SharePoint portals code coverage chart.
We have a .NET 4 solution that is being developed TDD, as a result we have pretty good code coverage, but as a quality check I want to monitor the code coverage rates as the project progresses.
To this end I have a test configuration (a .testsettings file in the solution) which is configured to instrument our soluton assemblies for code coverage and two team build definitions that use that test definition.
Both team builds (one is a CI trigger, the other a nightly shceduled trigger) work and produce code coverage figures
However despite sheduled team builds with code coverage the dashboard "code coverage" excel report always shows 0% coverage, in fact the excel spreadsheet containing the report does not contain any data. This is rather unexpected!
So my question boils down to what steps have I missed to make code coverage data from team builds show up in the TFS database used by the excel code coverage report?
As a side note the SSRS reports are also showing code coverage from the builds, it just seems to be the Excel spreadsheets that fail to see data.
UPDATE
It seems the problem is the filter "Is Build Verification Run" when this filter is removed I see data.
Specifically in my template (MSF for agile v5.0) the version of "code coverage" had a filter applied restricting output to just the "Other" value. Very odd.
When I am back in the office I'll try creating a new project based on the MSF Agile 5.0 tempate and see if this odd filter setting is part of it, or something I did in the past to this project!
I have verified this by creating a new project from the MSF For Agile Software Developmnet v5.0 template.
This turns out to be what I percieve as a bug in the "MSF For Agile Software Developmnet v5.0" template.
When the project is created the excel spreadhseet used for the "code coverage" chart in the dashboards has a filter on it restricting the data to just items whose origin is "other" this excludes code coverage data from a TFS build which has this value set to "true" in the cube.
Simply clearing the filter or including "true" in it resolves the issue and shows you your TFS build code coverage data.