I have installed Specflow via Nuget, but .feature files do not appear to be associated with Specflow. The TechTalk.Specflow DLL is present in the references for the DLL containing my tests.
Right clicking on a feature file and selecting Run Tests results in a dialog No tests found in file.
Can anyone help me diagnose this?
The nuget package gives you everything you need to run SpecFlow tests, but there is a separate installation # The Visual Studio Gallery which gives you the design time integration includign file associations
Related
I set up the SpecFlow plug-in for the Visual Studio 2013.
I created new Feature Item and build it.
Result: built test is not shown in the Test Explorer. Why?
I do it according to this video
in order to configure it to generate MSTest tests you need to add this to your app.config:
<specFlow>
<unitTestProvider name="MSTest"></unitTestProvider>
</specFlow>
I experience this same problem, but the solution was different.
Following the advice on a SpecFlow Github issue, the following worked for me:
Close all Visual Studio instances
Go to %TEMP%\VisualStudioTestExplorerExtensions\
Delete all the folders in here
Try again
if the tests are not showing in the test explorer you need to install NUnit test adapter :
go to Tools->Extensions and updates ->Online->Visual Studio Gallery->Tools->Testing and select NUnit Test Adapter
Visual Studio can change the default processor architecture when it loads. While looking endlessly trying to figure out the issue, I finally figured it out, that it wasn't really Specflow causing the issue.
If you have references that require a special architecture, it will prevent the tests from showing up in the test explorer. Change the architecture to x86 or x64 based on your needs.
Another thing you can do to see the build issues is open the Output window and change the dropdown to Tests during build. It will highlight any information of things taking place during your build.
I felt somebody should mention it just in case.
I'm running SpecFlow tests with SpecRunner and had this same problem, a couple of times as a matter of fact.
Having tried all of the above suggestions, I found that the only solution to the problem was to uninstall and reinstall the SpecRun NuGet package. Next build, tests present in Test Explorer window as expected.
In my case the reason why tests weren't discovered were because I was using Resharper build.
As soon as I went to: Resharper -> Options -> Tools -> Build -> General: "Use ReSharper Build"(untick), then all tests were suddenly discovered.
In my case all the approaches above didn't work. My solution: I don't know why, but - I re-installed "Specflow for Visual Studio 2015" in "Tools => Extensions and Updates", then I've created a new feature file (old still didn't work), and this new feature-file works correctly.
For me, I changed from using direct DLL references to using the NuGet references of version 3.1.7.4 of SpecFlow and this solved the problem.
I needed to reference "SpecFlow", "SpecFlow.NUnit" and "SpecFlow.Tools.MsBuild.Generation".
and then clicked the "Show All Files" button in Visual Studio to see the code behind file at a sibling level to it associated Feature file (this used to be an associated child file of the feature file in older versions).
I also need to use the "Include In Project" context menu item on the code behind file, for tests to show up in Visual Studios "Test Explorer".
NOTE: I did try to unzip the Specflow required files from the NuGet package and reference them directly in my project to be consistent with other direct references in my project (non-NuGet references) but this doesn't seem to work and you have to use the NuGet reference. I found an closed issue in github, which states that we have to use the Nuget reference to get the tests to work in Test Explorer:
https://github.com/techtalk/SpecFlow/issues/1617
For me, the issue was the TestAssemblyPath in my srprofile file. It didn't match my actual dll name.
I tried installing the Nugget package
SpecFlow.Tools.MsBuild.Generation
with the same version than specflow and now I can see the test in TestExplorer, other ways not works for me.
I already had installed 1 from
Nugget Package Specflow
Specflow.NUnit
Nunit
Nunit3TestAdapter
and the extension for Visual Studio for Specflow a NUnitTestAdapter
I'm using TFS Server 2010.
In my source I have a folder called BuildProcessTemplates. This has a child folder called Custom Assemblies.
Now, when the build runs, it happily loads the DLLs in the Custom Assemblies folder and does its thing.
I need to modify the .xaml file that defines the workflow of the build. If I double click on the .xaml file VS tries to open it but throws an error saying it can't find certain types that I know are defined in the DLL that is in the Custom Assemblies folder.
The workflow is told where to find these types when the build runs by Build Controller as it is defined as a property in the TFS Admin Console.
So, how on earth do I get VS to open my .xaml file properly? How do I let it know where to look for the DLL? I've tried moving all of the files in Custom Assemblies in to the same folder as the .xaml file, but that didn't help.
Clearly I'm missing something; I just don't have a clue what it might be.
You normally create a C# Project (Unit Test is easiest because it is pretty bare).
When you have you project you then need to:
Add the XAML as a File,
Add the Custom Activities DLL's as references.
Add the relevant TeamFoundation* references from the GAC (make sure the TFS version matches - 2010 = v10, 2012 = v11).
Also, I always open our TFS "Build Project" with VS version that matches our TFS Version, I find it keeps the number of problems down.
There's a great series of posts from Ewald Hofman here, that go through the process - I find it a bit dated, and that the Wrox Professional TFS 2012 book does a better job of explaining it, if you can get a copy.
I have been using MTM for a time and, so far, so good. I am trying to get Code Coverage over the tests I am executing. I have set my controller with 2 machines: 1 webclient and 1 webserver. For the webserver rule, i have configured code coverage. Of couse the webserver machine is the one that has my application running on IIS.
When I fire my automated tests (that btw were not created using visual studio, it was created using Ranorex) I can see that its generated an .coverage file.
But when I open this file, it is empty and I get this message from Visual Studio:
Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see http://go.microsoft.com/fwlink/?LinkID=253731
Of course I have read
http://go.microsoft.com/fwlink/?LinkID=253731
http://msdn.microsoft.com/en-us/library/vstudio/hh698492.aspx
http://msdn.microsoft.com/library/jj159523.aspx
But I have found nothing wrong in my scenario.
Does anyone have any idea what could be happening?
Go and check if you find one of the following files in your solution: Local.testsettings or TraceAndTestImpact.testsettings and double click them in VS. A new window will open, there you need to select Data and Diagnostics. There you'll see a point Code Coverage, double click it and then select all the assemblies that you want to be code coveraged.
If that does not work, check if your PDB files are present next to your DLL files.
Well. It turned out to be a problem with the permissions of the user I was using to run the test agent on the web server.
After changing the user to another one who has more permissions everything worked fine.
My team is trying to integrate Chutzpah into the TFS 2012 build process. We used this blog post as our starting point.
At a high level, the practical issue is that the Visual Studio Test Runner in the build agent context simply isn't finding the Chutzpah hooks. So while we can define **\*.js as a test source, without the Chutzpah bootstrapper actually being found and initialized, the test runner doesn't do anything with these files.
At a more detailed level, we are getting three concerning messages when we check the logs for loading the custom assemblies for the build controller:
Summary: There were 0 failures, 2 errors and 1 warnings loading custom activities and services.
Error: Method 'ToXml' in type 'Chutzpah.VS2012.TestAdapter.ChutzpahAdapterSettings' from assembly 'Chutzpah.VS2012.TestAdapter, Version=2.2.0.171, Culture=neutral, PublicKeyToken=1ca802c37ffe1896' does not have an implementation.
Error: API restriction: The assembly '...\AppData\Local\Temp\VSTFSBuild\8c8e9402-1169-4782-99a9-ce42f83be8f0\A1288811191\Chutzpah\Microsoft.VisualStudio.TestPlatform.ObjectModel.dll' has already loaded from a different location. It cannot be loaded from a new location within the same appdomain.
Warning: Could not load file or assembly '...\AppData\Local\Temp\VSTFSBuild\8c8e9402-1169-4782-99a9-ce42f83be8f0\A1288811191\Chutzpah\phantomjs.exe' or one of its dependencies. The module was expected to contain an assembly manifest.
Other than this information, we're more or less stuck. I'd love to hear from someone who has actually got Chutzpah running on a standalone 2012 build server so we can compare configurations.
Error: API restriction: The assembly '....
Indicates that TFS is finding a dll in two different locations. You can change the pattern to Check and see if you have \*\*test.dll as the value set for any test assemblies and change it to *test*.dll. This will prevent it loading it multiple times.
The other error might be because the test projects are not being rebuilt.
Try re-building the test projects.
I hope this helps.
Actually Software Carpenter has a point, I believe what may be happening is your test spec for ordinary unit tests may be test.dll (or something similar) this means Microsoft.VisualStudio.TestPlatform.ObjectModel.dll is loaded as a unit test dll, then TFS tries loading it again when trying to run the chutzpah tests.
Try disable the normal unit test spec and see if it helps, if it does then change your spec to be something else (*test.dll) that doesn't incude TestPlatform.ObjectModel.dll.
Source: I just had the same error when trying to build a project with Test in its name.
This discussion is also taking place on Codeplex. Maybe some help can be provided there.
Javascript Unit Tests on Team Foundation Service with Chutzpah
I've got a solution with Silverlight projects, a couple Web applications and some Windows service and their small Winforms test applications.
When I build it using TFS it puts all binary files in the Binaries folder. It also creates a _PublishedWebsites folder where it puts the web applications ready for deployment.
What I would like is for a few of the projects to have their ouput in a separate directory in the Binaries folder. I want the files that I would normally find in the bin\Release folder to be there.
I looked long and hard and could only find scarce documentation on how to achieve that and only for TFS 2008. There's a good article there link text and another entry for VS 2010 there link text. However, none of these work :-(
It seems that the build ignores the <OutputPath> property. I tried fiddling with the <AfterBuild> property but unfortunately if I use the $(OutDir) property I get ALL the files in the Binaries directory and not only the ones that just got compiled.
At the moment I'm setting up a separate build that only has the projects I'm interested in that get compiled but I would really like to have all the files needed for the release built all at once.
EDIT: I didn't make it clear enough but I'm using the latest version of TFS (TFS 2010 RTM). I set up <OutputPathDir> on the project level as explained in the links that I provided.
Cheers.
In your build definition (.proj) file, check that
<Project>
<PropertyGroup>
<CustomizableOutDir>true</CustomizableOutDir>
</PropertyGroup>
</Project>
Also, you may run into some problems if you're trying to compile what was once a "website" project type (created from File->New->Website->ASP.NET Webiste). They seem to override the output directory every time. We had to convert some of those to class libraries instead.