We have upgraded to TFS Update 2 release management recently. Enabled code coverage in Visual Studio Test as show below.
Build runs successfully and show test cases execution result.
But I dont see Code Coverage information. Do we need any other configuration?
UPDATE:
I tries detaching the collection, but no luck.
In your build definition you specify using .runsettings file for Code Coverage. So when Code Coverage is ran it loads the collectionscodecoverage.runsettings file and returns result depending on whats in it. Most likely .runsettings file is not correct.
You can collect Code Coverage without having a settings file. Just leave the "Run Settings File" option empty on build definition.
To validate that your .runsettings file is correct - run Code Coverage locally from Visual Studio having .runsettings file set as default test settings file. See details here.
Another possible problem addressed in this post.
Related
I'm trying to setup our TFS 2015 server to run automated tests. I've got it running, but we need to run our tests in Debug mode (for various reasons I can't really adjust). The problem is that I can't seem to figure out a way to switch the configuration in the Test task.
The help that the task links to (here) says that it's as easy as selecting Platform and Configuration, but the problem is that those options don't exist for me (they exist under Reporting, but the help there suggests that they will simply compare the results to other builds with that configuration).
I've also investigated the vstest.console.exe parameters (help I found was this one) as well as modifying the runsettings file, but these only allow me to modify the platform.
Overall, my question is a)is there a reason why I don't see the Platform/configuration options in TFS, and b) given that I don't see them, how can I modify the configuration that the tests are running under?
If it helps, TFS is reporting the version as Version 14.95.25122.0, which corresponds to Update 2. I checked the logs for 2.1 and 3, but wasn't able to find anything that suggested that this was added in later versions (though I could be wrong).
UPDATE:
I've realized that I misread the Test documentation and that the Platform/Configuration options were always for reporting only.
My question is then if I can actually set this in the tests somehow.
Thank you very much for any help.
Assuming you want to compile your test project in Debug mode. You can add a VS Build step to specify the BuildConfiguration variable, and define debug for variable BuildConfiguration. Check the screenshots below:
Then in VS Test step, specify the Test Assembly as **\$(BuildConfiguration)\*test*.dll to test the assemble under Debug folder:
I have generated my xml coverage file as part of CI build on the TFS server using vNext 2015 build definition. How would one display the results in the TFS 2015 summary ? Either using the xml report or the html generated using the ReportGenerator.
You need to convert the results produced by OpenCover to a format which can be parsed by TFS.
One way to do this would be to use the OpenCoverToCoberturaConverter NuGet package.
Once you have that, you can use the Publish Code Coverage Results build step.
I have described the whole process on my blog.
In “TFS 2015 - Update 2” this is possible by writing your own vsts extension (see here: https://www.visualstudio.com/en-us/docs/integrate/extensions/overview ).
I set up my own 'learning project' for building this as .vsix here: https://github.com/RobertK66/vsts-opencover.
My custom build step uses nunit3 console runner to execute tests under opencover.
I managed to upload the OpenCover xml result file as 'testrun-attachment' via REST-interface and got the pieces in place to show the summary values on both the build summary tab and on its own extended “build-results-view”.
Base for this first version were a lot of examples provided by MS on github: https://github.com/Microsoft/vsts-tasks and https://github.com/Microsoft/vsts-extension-samples
To get a first feeling what places on your TFS Web Portal can be extended/customized you can download and install this extension https://aha.gallery.vsassets.io/items?itemName=ms-samples.samples-contributions-guide from the Visual Studio Marketplace.
Hope this helps to get you started.
Currently, these customizations are not supported. You can't edit the displays in new build summary. However, you can customize Code Coverage Analysis.
You can submit it to User Voice site at:
http://visualstudio.uservoice.com/forums/121579-visual-studio
Instead of using OpenCover extension, it is very convenience for you to generate code coverage result and include it in the build summary page this way:
Select the Visual Studio Test step, check the Code Coverage Enabled option.
Then, the code coverage result shows on the build summary page:
I am creating a build definition in visual studio 2013. As you know, while creating build definition we can also set the assembly (in "Automated Tests" section) which contains the unit test methods to run. I am able to create the definition till this point. Now, what I want is once the test run is over, I want to call an exe that will parse the Trx file generated as a result of test run and send the run results in an email.
Can anyone have an idea about where I can set the path to the exe and pass some arguments to it while creating build definition? If this cannot be done while creating definition, what else is the other way to achieve it?
Thanks!
What you ask involves multiple steps, especially if you want to pull the tool from version control (which is IMHO the best option).
See the ALM Rangers' Team Foundation Build Customization Guide for help and guidance on this topic.
Or you can just setup an email notification when the TFS Build completes. That should include the test run results, and a clickable link to go to the full build report.
I have been using MTM for a time and, so far, so good. I am trying to get Code Coverage over the tests I am executing. I have set my controller with 2 machines: 1 webclient and 1 webserver. For the webserver rule, i have configured code coverage. Of couse the webserver machine is the one that has my application running on IIS.
When I fire my automated tests (that btw were not created using visual studio, it was created using Ranorex) I can see that its generated an .coverage file.
But when I open this file, it is empty and I get this message from Visual Studio:
Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see http://go.microsoft.com/fwlink/?LinkID=253731
Of course I have read
http://go.microsoft.com/fwlink/?LinkID=253731
http://msdn.microsoft.com/en-us/library/vstudio/hh698492.aspx
http://msdn.microsoft.com/library/jj159523.aspx
But I have found nothing wrong in my scenario.
Does anyone have any idea what could be happening?
Go and check if you find one of the following files in your solution: Local.testsettings or TraceAndTestImpact.testsettings and double click them in VS. A new window will open, there you need to select Data and Diagnostics. There you'll see a point Code Coverage, double click it and then select all the assemblies that you want to be code coveraged.
If that does not work, check if your PDB files are present next to your DLL files.
Well. It turned out to be a problem with the permissions of the user I was using to run the test agent on the web server.
After changing the user to another one who has more permissions everything worked fine.
In my TFS 2010 build definition, I call MSBuild on my own custom my.targets MSBuild project. It does all the compilation, testing, etc. that I need. Its log file is specified, as is a log drop location, and the log appears properly in that place. However I can't see it in the TFS Build Summary report. The "View Log File" link is grayed out. It seemed to work when the project I was building was a ".sln" file. What magic voodoo is baked deep in to the heart of the common MSBuild files to make this log link work properly, and how can I reproduce it?
You may find your answer in these posts from Jason Prickett:
TFS2010 – Changing The Way Build Information IS Displayed
TFS2010 – Customizing the Build Details View – Summary View
Important to note is that the MSBuild Log File will only appear if you enabled inside the build definition to copy output to the Drop Folder. If that option is disabled, the link will not appear in the build log view.