I've got an MVC web application with associated controller tests that run under a TFS build as per normal.
I can see the tests running and passing in the build log and they appear in the "Result details for Any CPU/Release" section of the build
I also have a number of other assemblies with associated tests that are running in the same build. Tests are passing and the details are being shown in the results and logs just fine.
I've enabled code coverage in the build script and the testrunconfig.
The coverage is appearing for all assemblies EXCEPT the web application even though it looks like the tests have been run for it.
Is there anything obvious that I have missed or some sort of work around that I need to do?
I've searched around for a while and haven't found an answer.
Has anyone got code coverage working for MVC web applications using TFS?
After reading the article posted earlier I thought I would try an alternate approach.
looking at the localrun.testconfig in a text editor I saw that the assemblies that contained code coverage data were in the section below
<Regular>
<CodeCoverageItem ... />
<CodeCoverageItem ... />
</Regular>
The only reference to a web project was in the section:
<AspNet>
<AspNetCodeCoverageItem id="..." name="..." applicationRoot="/" url="http://localhost:0/" />
</AspNet>
So to get code coverage working in TFS:
In VS IDE, double click the localrun.testconfig, then click Code Coverage and then manually add the MVC assembly by clicking the Add Assembly... button. You will get a warning message about duplicate assemblies, tell it to use this new file.
What this does behind the scenes in the localrun.testconfig is add a CodeCoverageItem
entry with the MVC assembly and removes the AspNet section from the CodeCoverage section of the file.
This has the added benefit of not starting up a local web server every time you run your tests.
But it may have a detrimental effect on any Watin type tests and coverage.
Team Build, Code Coverage and MVC by Richard Fennell looks a pretty detailed hack.
Related
I have been using MTM for a time and, so far, so good. I am trying to get Code Coverage over the tests I am executing. I have set my controller with 2 machines: 1 webclient and 1 webserver. For the webserver rule, i have configured code coverage. Of couse the webserver machine is the one that has my application running on IIS.
When I fire my automated tests (that btw were not created using visual studio, it was created using Ranorex) I can see that its generated an .coverage file.
But when I open this file, it is empty and I get this message from Visual Studio:
Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see http://go.microsoft.com/fwlink/?LinkID=253731
Of course I have read
http://go.microsoft.com/fwlink/?LinkID=253731
http://msdn.microsoft.com/en-us/library/vstudio/hh698492.aspx
http://msdn.microsoft.com/library/jj159523.aspx
But I have found nothing wrong in my scenario.
Does anyone have any idea what could be happening?
Go and check if you find one of the following files in your solution: Local.testsettings or TraceAndTestImpact.testsettings and double click them in VS. A new window will open, there you need to select Data and Diagnostics. There you'll see a point Code Coverage, double click it and then select all the assemblies that you want to be code coveraged.
If that does not work, check if your PDB files are present next to your DLL files.
Well. It turned out to be a problem with the permissions of the user I was using to run the test agent on the web server.
After changing the user to another one who has more permissions everything worked fine.
My team is trying to integrate Chutzpah into the TFS 2012 build process. We used this blog post as our starting point.
At a high level, the practical issue is that the Visual Studio Test Runner in the build agent context simply isn't finding the Chutzpah hooks. So while we can define **\*.js as a test source, without the Chutzpah bootstrapper actually being found and initialized, the test runner doesn't do anything with these files.
At a more detailed level, we are getting three concerning messages when we check the logs for loading the custom assemblies for the build controller:
Summary: There were 0 failures, 2 errors and 1 warnings loading custom activities and services.
Error: Method 'ToXml' in type 'Chutzpah.VS2012.TestAdapter.ChutzpahAdapterSettings' from assembly 'Chutzpah.VS2012.TestAdapter, Version=2.2.0.171, Culture=neutral, PublicKeyToken=1ca802c37ffe1896' does not have an implementation.
Error: API restriction: The assembly '...\AppData\Local\Temp\VSTFSBuild\8c8e9402-1169-4782-99a9-ce42f83be8f0\A1288811191\Chutzpah\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' has already loaded from a different location. It cannot be loaded from a new location within the same appdomain.
Warning: Could not load file or assembly '...\AppData\Local\Temp\VSTFSBuild\8c8e9402-1169-4782-99a9-ce42f83be8f0\A1288811191\Chutzpah\phantomjs.exe' or one of its dependencies. The module was expected to contain an assembly manifest.
Other than this information, we're more or less stuck. I'd love to hear from someone who has actually got Chutzpah running on a standalone 2012 build server so we can compare configurations.
Error: API restriction: The assembly '....
Indicates that TFS is finding a dll in two different locations. You can change the pattern to Check and see if you have \*\*test.dll as the value set for any test assemblies and change it to *test*.dll. This will prevent it loading it multiple times.
The other error might be because the test projects are not being rebuilt.
Try re-building the test projects.
I hope this helps.
Actually Software Carpenter has a point, I believe what may be happening is your test spec for ordinary unit tests may be test.dll (or something similar) this means Microsoft.VisualStudio.TestPlatform.ObjectModel.dll is loaded as a unit test dll, then TFS tries loading it again when trying to run the chutzpah tests.
Try disable the normal unit test spec and see if it helps, if it does then change your spec to be something else (*test.dll) that doesn't incude TestPlatform.ObjectModel.dll.
Source: I just had the same error when trying to build a project with Test in its name.
This discussion is also taking place on Codeplex. Maybe some help can be provided there.
Javascript Unit Tests on Team Foundation Service with Chutzpah
Our C# solution has a couple of folders that are populated via post-build events (i.e. they are initially empty).
The solution builds fine locally, however when using the TFS build agent, the folders don't show up in the published websites folder.
Any suggestions on how to force TFS to copy the folders over?
This is addressed here: publish empty directories to a web application in VS2010 web project
TFS does not execute the AfterBuild target of your proj file. I believe it will execute the AfterCompile target but this still might not do what you want.
I have used the approach of including dummy files which is simple enough even though its lame.
I've tried the approach of including a powershell script to do some post-publish tasks which works.
More recently I have adopted a convention of including a supplemental MSBuild file that ends in ".package.proj" and added an additional MSBuild execution activity to my Team Build Template that looks for it after the files are dropped to the drop location and then executes it. This provides a simple hook into the Team Build workflow without getting you deep into changing the workflow for a particular build. It's just a single additional activity wrapped in a conditional that looks for the file. If found, execute it.
You could also make a custom build template and use the Workflow activities to perform various cleanup tasks, but it's probably overkill and will increase maintenance on the build templates. Better to keep the customization simple if you can and have it function in a way that doesn't require "opt-out" configuration on builds that don't require the customization. Existing vanilla builds should continue to work as expected after the customization to the template.
I have a vcproj file that includes a simple pre-build event along the lines of:
Helpertask.exe $(ProjectDir)
This works fine on developer PCs, but when the solution is built on our TFS 2008 build server under MSBuild, $(ProjectDir) is either blank or points to an unrelated folder on the server!
So far the best workaround I have managed is to hard code the developer and server paths instead:
if exist C:\DeveloperCode\MyProject HelperTask.exe C:\DeveloperCode\MyProject
if exist D:\BuildServerCode\MyProject HelperTask.exe D:\BuildServerCode\MyProject
This hack works in post-build steps but it doesn't work for a pre-build step (the Pre-build task now does nothing at all under MSBuild!)
Do you have any ideas for a fix or workaround? I have very little hair left!
$(MSBuildProjectDirectory) worked for me
I think your problem may be related to how items are initalized. An items include attribute is evaluated at the begining of a build. So if you depend on files that are created in the build process you must declare these as dynamic items. Dynamic items are those defined inside of a target, or by using the CreateItem task. I've detailed this on my blog MSBuild: Item and Property Evaluation.
Sayed Ibrahim Hashimi
My Book: Inside the Microsoft Build Engine : Using MSBuild and Team Foundation Build
I think the problem is that build server's workspace probably isn't initialized properly.
I just kept getting problems with this - I tried many different approaches but they all failed in mysterious ways.
Once $(ProjectDir) started behaving properly again, the pre-build step stopped executing the command (I added echo commands above and below it - they were both executed, but the program in between them was not. No errors or output of any kind were generated to indicate why it failed).
I don't know if this is a dodgy server of if MSBuild is having a laugh.
I've given up now. I gave the build server a big kick and have changed tack: We now run this tool offline (manually) and check in the results for the build server to use. So much for an automated build :-( If only MSBuild would run solutions in the same way as Visual Studio does - it's maddening that it sets up the environment completely differently (different paths coming out of the solution variables, ouptus redirected into different folders so you can't find them where they're supposed to be, etc)
I branched an existing project and $(ProjectDir) kept the old directory in the newly branched code. But that's because I had some compiling errors. Once every project in the solution compiled without errors, $(ProjectDir) changed to the correct path.
Carlos A Merighe
I've got an existing web site that monitors an SVN repository for changes and when there is a check in, the new web site is built using aspnet_compiler and then a diff of what is on the production site is sftp'd up to a production web server. The entire continuous integration process is built using cruisecontrol.net.
I've refactored the entire site to use ASP.NET MVC and I am now trying to integrate into my build process. It seems that this has changed and I am not getting the same behavior as I used to get with a pre-MVC site. I'm unfortunately not making much progress because it seems that no one has done this yet. Has anyone got something like this to work in any form at all?
Automating the command line using "devenv" like a "regular project" does not work either. There are /Build command line switches but they just build in place. This command line does not expose the "Publish" mechanism that is built into the UI. There is even a {Project Name}.Publish.xml file in the project directory that seems like it is a manifest of what needs to be published. Just no way of automating that.
I'm just trying to emulate the "Publish {Project name}" under the build menu in VS 2008 MVC project from the command line.
If you're using CruiseControl.net, you're 85% of the way to doing this. Are you able to use NAnt? That would definitely be the way to go.
If not, then I'd look into using msbuild from the command line rather than devenv as that wasn't designed to do what you want.
NAnt is definitely a great way to go. Though, with all the changes they've made to msbuild, it is certainly a great contender. Nant just recently got new life breathed back into it, so it should continue to be an excellent option.
When you create a MVC project this is a standard .net project and not a "web project", which are not the same thing. I think that aspnet_compiler is only for web projects, so maybe this is what is causing you problems. I don't know which build tool you are using, but try building it like any other .net project.