I have been using MTM for a time and, so far, so good. I am trying to get Code Coverage over the tests I am executing. I have set my controller with 2 machines: 1 webclient and 1 webserver. For the webserver rule, i have configured code coverage. Of couse the webserver machine is the one that has my application running on IIS.
When I fire my automated tests (that btw were not created using visual studio, it was created using Ranorex) I can see that its generated an .coverage file.
But when I open this file, it is empty and I get this message from Visual Studio:
Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see http://go.microsoft.com/fwlink/?LinkID=253731
Of course I have read
http://go.microsoft.com/fwlink/?LinkID=253731
http://msdn.microsoft.com/en-us/library/vstudio/hh698492.aspx
http://msdn.microsoft.com/library/jj159523.aspx
But I have found nothing wrong in my scenario.
Does anyone have any idea what could be happening?
Go and check if you find one of the following files in your solution: Local.testsettings or TraceAndTestImpact.testsettings and double click them in VS. A new window will open, there you need to select Data and Diagnostics. There you'll see a point Code Coverage, double click it and then select all the assemblies that you want to be code coveraged.
If that does not work, check if your PDB files are present next to your DLL files.
Well. It turned out to be a problem with the permissions of the user I was using to run the test agent on the web server.
After changing the user to another one who has more permissions everything worked fine.
Related
I have the Enterprise edition of Visual Studio, but not everyone does.
How can I share the TRX so that other people can see the results? I've seen that there are several Trx to HTML convertors but they're all quite old and either don't work any more or don't show the details of the test.
Most of the results of a load test are stored in a database. The TRX file has only a very small portion of the results - you can easily see what it actually contains by opening the file in Notepad, it is just XML.
Visual Studio trial versions have some support for load tests. (They only support a small number of virtual users and do not support plugins.) They might allow viewing and analysis of the results of runs done on other computers. This limited support of load tests might also be available on licenced Non-Enterprise versions.
The results of individual load test runs, or groups of runs, can be copioed from one computer to another by using the Export and Import commands in the "Open and manage load test results" window. These create and read LTRAR files. See here for more details.
The TRX file contains a connection string to the SQL database and so the file cannot normally be copied and then work properly on another computer. The encoded connection string is located within the resultsRepositoryConnectString=... attribute within the TRX file. Changing the value in a copied file to a value found in a working TRX file on the destination computer allows the copied file to work. See here for more details.
The only other options for viewing results on other computers would be by exporting the results in Excel by using the "Create Excel Report" command. (Access it via the icons on the "Summary - Graphs - Tables - Details - ..." line in the load test results viewer.)
For unit tests, but not load tests, I routinely export trx files from my unit tests, because they ARE shareable, can be placed as attachments into our launch control systems, and DO contain all of the data necessary for the downstream teams to verify the results, including test console output. I do this with a custom task added to the project that runs the vstest.console.exe, with the /logger:trx;logfilename="blah" command line switch. The full task line is:
<Exec Command=""$(DevEnvDir)commonExtensions\Microsoft\TestWindow\vsTest.console.exe" "$(MSBuildProjectDirectory)\$(OutputPath)$(AssemblyName).dll" /logger:trx;LogFileName="$(MSBuildProjectDirectory)\testResults.trx"" />
You can also add ContinueOnError="true" if you don't want a test failure to stop the build.
MSTest still seems to ship with visual studio, supposedly does support load tests, and historically has exported TRX files, but I haven't tried it. Running pre-2019 unit test projects, that still have tasks pointed to mstest, in vs2019 tend to fail with assembly version errors, so I don't know if it even still works at all.
We have upgraded to TFS Update 2 release management recently. Enabled code coverage in Visual Studio Test as show below.
Build runs successfully and show test cases execution result.
But I dont see Code Coverage information. Do we need any other configuration?
UPDATE:
I tries detaching the collection, but no luck.
In your build definition you specify using .runsettings file for Code Coverage. So when Code Coverage is ran it loads the collectionscodecoverage.runsettings file and returns result depending on whats in it. Most likely .runsettings file is not correct.
You can collect Code Coverage without having a settings file. Just leave the "Run Settings File" option empty on build definition.
To validate that your .runsettings file is correct - run Code Coverage locally from Visual Studio having .runsettings file set as default test settings file. See details here.
Another possible problem addressed in this post.
I'm just getting started with the team build functionality and I'm finding the sheer amount of things required to do something pretty simple a bit overwhelming. My setup at the moment is a solution with a web app, an assembly app and a test app. The web app has a PublishProfile set up which publishes via the filesystem.
I have a TFS build definition set up which currently builds the entire solution nightly and drops it onto a network share as a backup of old builds. All I want to do now is have the PublishProfile I've already setup publish the web app for me. I'm sure this is really simple but I've been playing with MSBuild commands for a full day now with no luck. Help!
Unfortunately sharing of the Publish Profile is not supported or implemented in MSBuild. The logic to publish from the profile is contained in VS itself. Fortunately the profile doesn't contain much information so there are ways to achieve what you are looking for. Our targets do not specifically support the exact same steps as followed by the publish dialog, but to achieve the same result from team build you have two choices, I will outline both here.
When you setup your Team Build definition in order to deploy you need to pass in some values for the MSBuild Arguments for the build process. See image below where I have highlighted this.
Option 1:
Pass in the following arguments:
/p:DeployOnBuild=true;DeployTarget=PipelinePreDeployCopyAllFilesToOneFolder;PackageTempRootDir="\\sayedha-w500\BuildDrops\Publish";AutoParameterizationWebConfigConnectionStrings=false
Let me explain these parameters a bit, show you the result then explain the next option.
DeployOnBuild=true:This tells the project to execute the target(s) defined in the DeployTarget property.
DeployTarget=PipelinePreDeployCopyAllFilesToOneFolder: This specifies the DeployTarget target.
PackageTempRootDir="\\sayedha-w500\BuildDrops\Publish": This specifies the location where the package files will be written. This is the location where the files are written before they are packaged.
AutoParameterizationWebConfigConnectionStrings=false: This tells the Web Publishing Pipeline (WPP) to not parameterize the connection strings in the web.config file. If you do not specify this then your connection string values will be replaced with placeholders like $(ReplacableToken_dummyConStr-Web.config Connection String_0)
After you do this you can kick off a build then inside of the PackageTempRootDir location you will find a PackageTmp folder and this contains the content that you are looking for.
Option 2:
So for the previous option you probably noticed that it creates a folder named PackageTmp and if you do not want that then you can use the following options instead.
/p:DeployOnBuild=true;DeployTarget=PipelinePreDeployCopyAllFilesToOneFolder;_PackageTempDir="\\sayedha-w500\BuildDrops\Publish";AutoParameterizationWebConfigConnectionStrings=false
The difference here is that instead of PackageTempRootDir you would pass in _PackageTempDir. The reason why I don't suggest that to begin with is because MSBuild properties that start with _ signify that the property in essentially "internal" in the sense that in a future version it may mean something else or not exist at all. So use at your own risk.
Option 3
With all that said, you could just use the build to package your web. If you want to do this then use the following arguments.
/p:DeployOnBuild=true;DeployTarget=Package
When you do this in the drop folder for your build you will find the _PublishedWebsites folder as you normally would, then inside of that there will be a folder {ProjectName}_Package where {ProjectName} is the name of the project. This folder will contain the package, the .cmd file, the parameters file and a couple others. You can use these files to deploy your web.
I hope that wasn't information over load.
The ability to publish web sites, configure IIS and push schema changes for the DEV->QA->RELEASE cycle has required either custom configuration to imitate publish or custom code where IIS settings are involved.
As of Visual Studio 2013.2 Microsoft has added a third party product that manages deployment of web sites, configuration changes and database deployment with windows workflow and would be the recommended solution for automating deployment from TFS build.
More information can be found here:
http://www.visualstudio.com/en-us/explore/release-management-vs.aspx
You can use the Publish/Deploy in Visual Studio 2010.
See http://www.ewaldhofman.nl/post/2010/04/12/Auto-deployment-of-my-web-application-with-Team-Build-2010-to-add-Interactive-Testing.aspx for more information
I've got an MVC web application with associated controller tests that run under a TFS build as per normal.
I can see the tests running and passing in the build log and they appear in the "Result details for Any CPU/Release" section of the build
I also have a number of other assemblies with associated tests that are running in the same build. Tests are passing and the details are being shown in the results and logs just fine.
I've enabled code coverage in the build script and the testrunconfig.
The coverage is appearing for all assemblies EXCEPT the web application even though it looks like the tests have been run for it.
Is there anything obvious that I have missed or some sort of work around that I need to do?
I've searched around for a while and haven't found an answer.
Has anyone got code coverage working for MVC web applications using TFS?
After reading the article posted earlier I thought I would try an alternate approach.
looking at the localrun.testconfig in a text editor I saw that the assemblies that contained code coverage data were in the section below
<Regular>
<CodeCoverageItem ... />
<CodeCoverageItem ... />
</Regular>
The only reference to a web project was in the section:
<AspNet>
<AspNetCodeCoverageItem id="..." name="..." applicationRoot="/" url="http://localhost:0/" />
</AspNet>
So to get code coverage working in TFS:
In VS IDE, double click the localrun.testconfig, then click Code Coverage and then manually add the MVC assembly by clicking the Add Assembly... button. You will get a warning message about duplicate assemblies, tell it to use this new file.
What this does behind the scenes in the localrun.testconfig is add a CodeCoverageItem
entry with the MVC assembly and removes the AspNet section from the CodeCoverage section of the file.
This has the added benefit of not starting up a local web server every time you run your tests.
But it may have a detrimental effect on any Watin type tests and coverage.
Team Build, Code Coverage and MVC by Richard Fennell looks a pretty detailed hack.
I have a vcproj file that includes a simple pre-build event along the lines of:
Helpertask.exe $(ProjectDir)
This works fine on developer PCs, but when the solution is built on our TFS 2008 build server under MSBuild, $(ProjectDir) is either blank or points to an unrelated folder on the server!
So far the best workaround I have managed is to hard code the developer and server paths instead:
if exist C:\DeveloperCode\MyProject HelperTask.exe C:\DeveloperCode\MyProject
if exist D:\BuildServerCode\MyProject HelperTask.exe D:\BuildServerCode\MyProject
This hack works in post-build steps but it doesn't work for a pre-build step (the Pre-build task now does nothing at all under MSBuild!)
Do you have any ideas for a fix or workaround? I have very little hair left!
$(MSBuildProjectDirectory) worked for me
I think your problem may be related to how items are initalized. An items include attribute is evaluated at the begining of a build. So if you depend on files that are created in the build process you must declare these as dynamic items. Dynamic items are those defined inside of a target, or by using the CreateItem task. I've detailed this on my blog MSBuild: Item and Property Evaluation.
Sayed Ibrahim Hashimi
My Book: Inside the Microsoft Build Engine : Using MSBuild and Team Foundation Build
I think the problem is that build server's workspace probably isn't initialized properly.
I just kept getting problems with this - I tried many different approaches but they all failed in mysterious ways.
Once $(ProjectDir) started behaving properly again, the pre-build step stopped executing the command (I added echo commands above and below it - they were both executed, but the program in between them was not. No errors or output of any kind were generated to indicate why it failed).
I don't know if this is a dodgy server of if MSBuild is having a laugh.
I've given up now. I gave the build server a big kick and have changed tack: We now run this tool offline (manually) and check in the results for the build server to use. So much for an automated build :-( If only MSBuild would run solutions in the same way as Visual Studio does - it's maddening that it sets up the environment completely differently (different paths coming out of the solution variables, ouptus redirected into different folders so you can't find them where they're supposed to be, etc)
I branched an existing project and $(ProjectDir) kept the old directory in the newly branched code. But that's because I had some compiling errors. Once every project in the solution compiled without errors, $(ProjectDir) changed to the correct path.
Carlos A Merighe