Is there a way to run xUnit tests for UWP with TFS 2015 vNext build?
I try to do it with my local build Agent with it running in interactive mode.
When test needs to be done, the UWP test app runs but no tests discovered.
Is it supported to run this tests?
Yes, assuming you have installed xunit via Nuget and your packages are at the root level of your sources folder, you just need to set the Path to Custom Test Adapters to your packages folder.
In my case: $(Build.SourcesDirectory)\packages - adjust as you need to.
Related
Question:
How do I run NUnit tests on TFS?
Attempts:
I've searched the web and am still struggling to figure out the exact steps required to accomplish (to what I believe is) a trivial task.
I added the NUnit3TestAdapter package to my test project.
However, my tests are still not being discovered.
TFS displays the following message:
No test runs are available for this build. Enable automated tests in
your build pipeline by running your test framework of choice, such as
JUnit, Visual Studio Test, or xUnit. If you choose to run tests using
a custom task or runner, you can publish results using the Publish
Test Results task.
Expectation:
I'm surprised that NUnit is not an option that I can just select from.Hence, it's the standard unit test framework for .Net.
How do I run NUnit tests on TFS?
There is a blog here describes clearly how to run NUnit tests on TFS: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Note：
You do need to edit the Visual Studio Test task’s advanced settings
to point at the NuGet packages folder for your solution (which will be
populated via NuGet restore) so the custom nUnit test adaptor can be
found i.e. usually setting it to $(Build.SourcesDirectory)\packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
In our project we have Unit tests specific to VS2013 and VS2015.We have a requirement of using one build definition (Virtual machine) to run both VS2013 and VS2015 Tests on different timings.
I have installed on that VM Team Foundation Server 2015 Console and created one controller and 2 agents(each for specific version of visual studio).
We are using 2013 build template.
I have ran successfully 2015 UTs on that Virtual Machine(VM) but when I try to run VS2013 specific UTs it is picking up 2015 vstest.exe and throwing exception.
Is there any way to achieve this??
I tried to include MTest Activity build template.Activity is running but it is saying no tests to execute.
I even tried to run throught batch file invoke,this way it is running but through MSTest Activity it is not running scripts.Below is snapshot of TFS log for MSTEST activity
By default, TFS calls the corresponding version of VS to run tests. There is no way to specify the VS version for a test in XAML build with default build process template. You would need to customize your build process template by adding InvokeProcess activity to invoke 2013 vstest.exe to run tests.
Since you are using TFS 2015, I'd like to suggest you use new build system, which is tasks based. In new build system, you can add Visual Studio Test task to run tests. In this task, you can specify VSTest version:
Check this article for more information: https://www.visualstudio.com/en-us/docs/test/continuous-testing/getting-started/getting-started-with-continuous-testing
We have a Visual Studio 2015 solution containing some C++ projects and some tests based on Google Test Framework.
Now I would like to run those tests with the quite new TFS 2015 build features. I know that there is the "Visual Studio Test" build step that is able to run custom test adapters (like the Google Test Adapter?).
Is this the easiest way to setup things? What exactly has to be installed on the (on premise) TFS2015 build server and how to configure the build steps?
Thanks for you help! Sebastian
Yes, the simplest way is just using google test adapter in ""Visual Studio Test" build " task. Just as the feature statement which will using VSTest.Console.exe
You need to install visual studio on your build server(agent). About how to conigure the build steps, there has been a detailed tutorial with Xunit test which also applies to google test. Plesae refer this blog: Running xUnit tests in TFS Build vNext
We have the following setup for unit and functional testing in TFS (under Build Definitions in TFS)
Install our program into a clean VM.
Deploy test agent (default vstf_testagent.exe)
Run tests
Some of the tests are written in NUnit3 and the C++ test in Google Tests. So, they are not seen by the default test runner.
In Visual Studio, the test adapters work great to run the tests.
But, I can't seem to figure out how to install these test adapters to vstf_testagent.exe on the virtual machine so that it sees and runs all the tests.
I found this page on how to install agents but it is with using Visual Studio.
You can install the test adapters to your project via Nuget and add a Nuget restore task in your build definition to restore the packages. "Run Functional Tests" task will search the test adapters in "packages" folder automatically.
Test Adapters: There is no explicit field to specify Test Adapter path in the task. The task automatically searches for "packages"
directory that exists in the same folder as the .sln file (nuget
restored directory structure). If your adapters are in a different
directory or you did not copy over the source files, use a runsettings
file with TestAdaptersPaths as described at
https://msdn.microsoft.com/en-us/library/jj635153.aspx
If the test adapters is in other folder, you can create a runsettings file and run the test with it.
Refer to this link for details: Visual Studio Test using Test Agent Task.
First, you need to install Visual Studio on your VM.
After that the directly way to make NUnit adapters available is downloading them from Visual Studio Gallery, unzip the .vsix extension file, then copy all the extension file in the folder just as the link mentioned in your question.
This operation should be done in every machine where you deployed Test Agents that will be used to run build with nunit tests. This can be annoying if you have lots of agents, if you prefer, Test Runner Task has an options to specify the path where the agent can find needed test runners.
You can also automate all the process by adding Nunit3 Test Adapter nuget package just as Eddie suggest to your test project, adding a standard Nuget Reference. Detail ways you can refer this blog: nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
Nuget Package
NUnit 3 Test Adapter for Visual Studio 2012/13/15
It is also possible to copy the .dlls of your testadapter that you want to use directly from the nuget-package to the folder C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensionsof your computer where you want to run your tests.
Also see:
http://www.codewrecks.com/blog/index.php/2016/06/04/running-unit-tests-on-different-machine-during-tfs-2015-build/
We have come across an issue where TFS build fails to trigger coded UI tests, based on how the Test Settings file is specified in the build definition.
We have been unable to get CodedUI test project to run. The goal is to have it run on a separate machine (virtual machine). We have configured the test controller and agent on the VM. We have also set it to run as interactive process. When we run build it partially succeeds and the following test error is reported
The failure message returned is as below:
MsTestAdapter failed to discover tests in class ‘MVC_Test.MVCTest_CUI’ of assembly ‘C:\Builds\1\Test_COE\MVCAppFramework\bin\MVC_Test.dll’. Reason Could not load file or assembly ‘Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=12.0.0.0. Culture =neutral, PublickeyToken=b03f5f7f11d50a3a’ or one of its dependencies. The system cannot find the file specified..
Setup / Environment Details:
Visual studio 2015 ultimate
TFS server 2015
Agents for Visual studio 2015(Test Controller and Test Agents)
Test agent is installed as interactive process for Coded UI tests
Your CodedUI tests were written in the 2013 version of VS. I believe you have two options:
Install the Agents for Visual Studio 2013 on your test agent
Use binding redirects to automatically use the newer version as described in this TFS 2010 to 2015 Upgrade: CodedUI Feature Test Builds blog post
UPDATE
You could try upgrading your CodedUI DLLS to the VS2015 versions (14.0.0.0).
It cannot find the dll libraries for Version 12.0 which provided by VS2013. Install VS2013 on the test agent and then retry.