I'm possibly in over my head here, but I've been asked to set up a scheduled Team Foundation Build for our team's branch and then after the build completes for our automated tests to be executed using NUnit.
I've had a look at a few online tutorials on setting up the build definition in TFS, but I can't seem to figure out how to call NUnit after the build is successful. I was expecting to see or find some kind of "run this command line on success" option somewhere; the best I could find is "Pre/Post-test script path", but that's related to tests like **\*test*.dll;**\*test*.appx and I'm not sure what that is.
Just knowing what to Google for would be a help, as I am at a loss now.
If you use XAML build:
You can either install the NUnit Test Adapter NuGet package in the unit test project. Or you can check the assemblies into the Build Controller's Custom Assemblies Path.
Useful article:
https://blogs.msdn.microsoft.com/visualstudioalm/2013/06/11/part-3-unit-testing-with-traits-and-code-coverage-in-visual-studio-2012-using-the-tfs-build-and-the-new-nuget-adapter-approach/
https://www.codit.eu/blog/2015/03/18/continuous-integration-with-javascript-nunit-on-tfsbuild-part-3-of-3-/
If you use new task based build:
You can add the NUnit Test Adapter NuGet package to your solution, and specify the path of NUnit Test Adapter NuGet package in the Path to Custom Test Adapters field in VSTest task. Check the screenshot below:
Useful article:
http://bartwullems.blogspot.sg/2015/10/team-foundation-server-2015enable-nunit.html
Related
I am running my test in TFS (Nunit plus Visual Studio with Adapter) and I have set the build definition as below
Build succeeds but no test result file was generated
Does TFS writes this Xml file ?
Log
2017-02-08T08:08:40.8151428Z Executing the powershell script: D:\A1\agent\tasks\PublishTestResults\1.0.20\PublishTestResults.ps1
2017-02-08T08:08:41.0963795Z ##[warning]No test result files were found using search pattern 'D:\A1_work\1\s**\TEST-*.xml'.
If the Nunit plus Visual Studio with Adapter means you have two test steps: one for unit tests and the other one for vs tests.
Please also add two "Publish Test Results" step one for Nunit format.
Also run your test manually on the build server to see if test result file .trx generated on the machine.
we are currently using Wallaby.js for javascript unit testing. Works fine and is great. But within our development pipeline we of course want to run the same tests on the build server - in our case a tfs.
Is it possible to use wallaby on a tfs build server? Anf if yes how?
If not, what is the way to go to run the wallaby configured unit tests on the build server?
As we used the karma test runner earlier, I tried to execute the new test configuration with it but then I get
Can't find variable: wallaby
as in our main/ starting test file it is written
wallaby.delayStart();
require.config({
baseUrl: 'app',
(Originally from a karma/ requirejs configuration)
How to get around this?
Has anyone experience in this scenario?
Wallaby.js main idea is to integrate with editors, run tests for the code that you change and display the results in the editor. You can't use Wallaby.js in a CI build.
You may consider invoking other test runners, or use grunt/gulp task instead for javascript unit testing.
In TFS 2012 and later (might work in 2010 but not sure) you can extend the testing capabilities of the build system.
Check out these posts -
http://www.aspnetperformance.com/post/Unit-testing-JavaScript-as-part-of-TFS-Build.aspx
https://blogs.msdn.microsoft.com/visualstudioalm/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah/
I need to include the output of my failing tests (NUnit) on the final build report of TFS.
I've looked around and found solutions for VSTest, but not NUnit unfortunately.
you can edit the build process template xaml you use and add a custom task. https://tfsbuildextensions.codeplex.com/wikipage?title=How%20to%20integrate%20the%20nUnit%20build%20activity&referringTitle=Documentation
Alternatively (and arguably worse, but it's what I do) is to not modify the template xaml but to target a custom msbuild file (.proj) into which I'd run the nunit tasks. You then get your build def to build this .proj rather that a sln and it's this that runs and outputs your nunit results.
I had this working on a previous project and now on a new project I've setup SpecFlow, got it generating tests from my feature file but I can't run the tests from the feature file and instead have to go to the code behind to run the tests. I've also installed the VS extension "Spec Flow for Visual Studio". What can I try?
as Greg suggested the first thing to check is that your config is set up correctly for ms test. you basically need this:
<specflow>
<unitTestProvider name="MSTest"/>
</specflow>
Also worth checking your generated feature.cs tests to see what unit test language they are in
I'm trying to get TFS to run my unit tests.
The name of the project assembly is Users.SystemTests.dll. It's located in ~/source/Users.SystemTests/bin/debug. The solution file is located in ~/source/Users.sln.
I've included the Nunit.VisualStudio.TestAdapter nuget package the in test assembly project.
The results of the build shows that the tests don't run.
What am I missing? They run fine locally via the Resharper test runner and I can also use nunit-console-x86.exe to run them.
Does this have something to do with the fact that I'm building a solution file? Maybe it's the output location being "AsConfigured?"
Ok, so I tracked this down on my own. It was twofold. The first problem was the the TestAdapter was not being output to the bin directory. The other piece was the Output Location. Setting copy local and then Output Location SingleFolder fixed the issue.
I do believe this is a bug in the tfs build. It works when you use SingleFolder or PerProject, but not AsConfigured. In the latter case the test runner don't find the testassemblies, and this is the same for both NUnit and MSTest, so it is not adapter specific.
The diagnostics log says:
Run VS Test Runner00:00:00
There were no matches for the search pattern C:\a\bin\**\*test*.dll
There were no matches for the search pattern C:\a\bin\**\*test*.appx