Installed TFS 2017 U2 and I can see one of new features is that Visual Studio Test is one of the available tasks in Release management but I can not find any guidance how to integrate Visual Studio webtests into that. Is it possible now properly incoporate WebTest straight up from TFS UI?
No, Visual Studio Test task is using vstest.console.exe to execute tests which is documented at VSTest.Console.exe Command-Line Options.
To run a web test you need to use Visual Studio and MSTest.exe. For more tutorial about the integration of web test and TFS, you could refer below links:
Guide to get started with Visual Studio Web Load Testing and
Automation
Running WebTests as part of a VSTS VNext Release pipeline
Related
In our project we have Unit tests specific to VS2013 and VS2015.We have a requirement of using one build definition (Virtual machine) to run both VS2013 and VS2015 Tests on different timings.
I have installed on that VM Team Foundation Server 2015 Console and created one controller and 2 agents(each for specific version of visual studio).
We are using 2013 build template.
I have ran successfully 2015 UTs on that Virtual Machine(VM) but when I try to run VS2013 specific UTs it is picking up 2015 vstest.exe and throwing exception.
Is there any way to achieve this??
I tried to include MTest Activity build template.Activity is running but it is saying no tests to execute.
I even tried to run throught batch file invoke,this way it is running but through MSTest Activity it is not running scripts.Below is snapshot of TFS log for MSTEST activity
By default, TFS calls the corresponding version of VS to run tests. There is no way to specify the VS version for a test in XAML build with default build process template. You would need to customize your build process template by adding InvokeProcess activity to invoke 2013 vstest.exe to run tests.
Since you are using TFS 2015, I'd like to suggest you use new build system, which is tasks based. In new build system, you can add Visual Studio Test task to run tests. In this task, you can specify VSTest version:
Check this article for more information: https://www.visualstudio.com/en-us/docs/test/continuous-testing/getting-started/getting-started-with-continuous-testing
We have a Visual Studio 2015 solution containing some C++ projects and some tests based on Google Test Framework.
Now I would like to run those tests with the quite new TFS 2015 build features. I know that there is the "Visual Studio Test" build step that is able to run custom test adapters (like the Google Test Adapter?).
Is this the easiest way to setup things? What exactly has to be installed on the (on premise) TFS2015 build server and how to configure the build steps?
Thanks for you help! Sebastian
Yes, the simplest way is just using google test adapter in ""Visual Studio Test" build " task. Just as the feature statement which will using VSTest.Console.exe
You need to install visual studio on your build server(agent). About how to conigure the build steps, there has been a detailed tutorial with Xunit test which also applies to google test. Plesae refer this blog: Running xUnit tests in TFS Build vNext
We just upgraded from TFS/VS2012 to TFS/VS2013 and now no CodedUI tests can be executed as part of a build workflow. I now get the following error when running any test as part of a build workflow:
Error adding test case [<#>] to test run: Unable to load the test
container "\\.dll" or one of its dependencies.
Error details: System.IO.FileNotFoundException: Could not load file or
assembly 'Microsoft.VisualStudio.TestTools.UITesting, Version =
11.0.0.0, Culture=neutral, PublicKeyToken=' or one of its dependencies. The system cannot find the file specified.
Our setup is as follows:
TFS Server:
Windows Server 2012 Standard
TFS 2013 Update 2 (12.0.30324.0)
Visual Studios is NOT installed
Build Server (Used to create the build A):
Windows 7 Professional Service Pack 1
TFS 2013 Update 2 (12.0.30324.0)
Visual Studio Premium 2012 (11.0.50727.1)
Visual Studio Premium 2013 (12.0.21005.1)
Agents for Visual Studio 2013 (12.0.21005.1) (NOTE: These are not
used for this build/deploy/test workflow)
Build/Test Controller (Used to trigger build A on the above server, then execute the deploy/test workflow/build B):
Windows Server 2008 R2 Standard
TFS 2013 Update 2 (12.0.30324.0
Test Controller for VS2013 (12.0.21005.1)
Visual Studio Premium 2013 (12.0.21005.1)
Visual Studio Test Professional 2013 (12.0.21005.1)
Test Agent:
Windows Server 2012 Standard
Agents for Visual Studio 2013 (12.0.21005.1)
Development Environment:
Windows Server 2012 Standard
Visual Studio Premium 2013 (12.0.21005.1)
Visual Studio Test Professional 2013 (12.0.21005.1)
In our environment, Build A is the build that creates the build/setup packages. Build B is what deploys the build/setups to a lab environment and executes the tests.
I can run the tests without issue from the dev environment, on the dev
environment
I cannot run the tests via mstest cmdline from the controller
The DLL Microsoft.VisualStudio.TestTools.UITesting exists at "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PublicAssemblies" on both the Test Controller and the Test Agent, but the version = 12.0.21005.1, not 11.0 which is what the error is complaining about
If Microsoft.VisualStudio.TestTools.UITesting version 12.0 is replaced with version 11.0 from another machine for troubleshooting purposes, the same error displays but for each of the following dll's:
Microsoft.VisualStudio.TestTools.UITest.Common
Microsoft.VisualStudio.TestTools.UITest.Extension
Microsoft.VisualStudio.QualityTools.CodedUITestFramework
Microsoft.VisualStudio.QualityTools.UnitTestFramework
After replacing all of the above version 12.0 DLL's with version 11.0 DLL's, I then get the following error when firing off Build B:
Failed to initialize the unit test extension 'urn:CodedUITest': A unit
test extension is not registered for the following attribute:
Microsoft.VisualStudio.TestTools.UITesting.CodedUITestAttribute
I've tried everything I could think of or find online to troubleshoot this issue, but I'm all out of ideas. Any suggestions on where to go from here to get my CodedUI tests running again? In theory, TFS, Build Controller/Agent, Test Controller/Agent and VS are all upgraded to 2013, so why are 2012(version 11) dll's needed?
We're deploying a set of Build Controllers and Agents throughout our infrastructure to build Web/MVC applications and deploy them using Release Management through TFS 2013.
I read that we need to have Visual Studio installed on the build machines to compile properly, and would like to know if we are able to use Visual Studio Shell (Integrated? Isolated?) or if we need to dedicate a full (Ultimate or Professional) Visual Studio to it.
Thanks in advance--
==Update 10:33 5/5/2014==
Due to the security restrictions placed on our development group, we are unable to use off-premise resources.
==Update 12:47 5/5/2014==
Installing the MSBuild Redistributable (via Erik Funkenbusch) was a good start, and following Jimmy's advice sealed it. Thanks!
The components needed to build Web Applications (of which MVC is a subtype) are in \Program Files (x86)\msbuild\Microsoft\VisualStudio\v12.0\WebApplications (or whichever version of VS you're building against). While these are installed by VS, you can just xcopy them to your build servers without the rest of the VS installation.
No, you don't need Visual Studio installed on the build machine, and wherever you read that... run far away, because they don't know what they're talking about.
In fact, it's explicitly recommended to NOT install Visual Studio on the build machine.
Microsoft has a full MSBuild redistributable with 2013 components. MVC is installed as part of the build via Nuget (along with other components).
http://www.microsoft.com/en-us/download/details.aspx?id=40760
In past years, you had to install the Windows SDK, and do a lot of manual configuration, and that was a pain. With 2013, they've made this much simpler.
I believe the Release Management should also have an installation setup file to install components for build servers....
You don't need Visual Studio for just doing compilation as part of the build process. If you have unit test cases which needs to be run then you need Visual Studio.
We have moved to Visual Studio 2010 but our TFS folks are not ready yet. After some work, we can get the build to work, but we don't get code coverage.
We are running using the MSTest test runner. Does anyone know any tricks to get TFS to report code coverage using Visual Studio 2010 on a build server that has a TFS 2008 Build agent?
There is command line tool that name is VsPerfMon.
Whit this tool you can run your code coverage via command line. You can check this link for more information, http://msdn.microsoft.com/en-us/library/ms182404(VS.80).aspx
So if you can run code coverage via command line, you can invoke this tool via TFS 2008 Build Scripts.