We are building a sitecore web application and unit test is based on MSTest. I had a look at link below which has asp.net nunit test runner:
http://www.codeflood.net/testrunner/
However, we don't want to use the nUnit framework for out of box MSTest suit. Could someone please help if there's any existing test runner or how easy it is to convert the nUnit Test runner to MSTest runner?
The reason we want to use the test runner is to run unit test with in HTTPContext.
According to this stackoverflow post the API used by MSTest is sealed and private which is not going to help you in exposing a test runner that mimics the NUnit TestSuiteBuilder functionality.
It could still be done via Reflection if you are determined enough but it might just be simpler to use the technical restrictions around MSTest to build a case to migrate your unit tests that require a Sitecore context to NUnit.
Related
Question:
How do I run NUnit tests on TFS?
Attempts:
I've searched the web and am still struggling to figure out the exact steps required to accomplish (to what I believe is) a trivial task.
I added the NUnit3TestAdapter package to my test project.
However, my tests are still not being discovered.
TFS displays the following message:
No test runs are available for this build. Enable automated tests in
your build pipeline by running your test framework of choice, such as
JUnit, Visual Studio Test, or xUnit. If you choose to run tests using
a custom task or runner, you can publish results using the Publish
Test Results task.
Expectation:
I'm surprised that NUnit is not an option that I can just select from.Hence, it's the standard unit test framework for .Net.
How do I run NUnit tests on TFS?
There is a blog here describes clearly how to run NUnit tests on TFS: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Note：
You do need to edit the Visual Studio Test task’s advanced settings
to point at the NuGet packages folder for your solution (which will be
populated via NuGet restore) so the custom nUnit test adaptor can be
found i.e. usually setting it to $(Build.SourcesDirectory)\packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
I've just literally hit a brick wall trying to configure F# 4.5.0.0 under VS 15.8.1 to run xUnit tests. Here is the repo with the issue: https://github.com/kkkmail/fSharp-xUnit .
When I try to run all tests, VS comes back with this message: No test is available in C:\GitHub\fSharp-xUnit\DummyApp\DummyApp\bin\Debug\DummyApp.exe C:\GitHub\fSharp-xUnit\DummyApp\Tests\bin\Debug\Tests.dll C:\GitHub\fSharp-xUnit\DummyApp\DummyApp\DummyApp.fsproj C:\GitHub\fSharp-xUnit\DummyApp\Tests\Tests.fsproj. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Both the "DummyApp" and tests were created from scratch using VS and I cleared all temp files and all nuget caches prior to creating the solution as advised in various places on the web.
TL;DR Add xunit.runner.visualstudio to your test project.
For discovering tests Visual Studio relies on its test runner. This test runner can discover / run all tests for testing frameworks which implement the corresponding adaptor interfaces ITestDiscoverer and ITestExecutor. For xUnit, these interfaces are implemented in VsTestRunner which is published in xunit.runner.visualstudio.
See also the xUnit docs about running tests in VS
Is there any way to run Nunit tests on deployed project?
Like after project has been deployed I have to run tests against the project on a deployed environment (because part of functionality can be run only on deployed environment due to the special access policy)
Is there any way to run Nunit tests on deployed project?
Normally NUnit is used to write unit tests against separate small functionalities of your application. If you want to run tests against a deployed version of your application then we are talking about integration tests. There are many different tools that can be used to test your running application. One of them is Sikuli. Alternatively if you want to keep using a .NET unit testing platform but firing requests against a real web server you can checkout SpecFlow. But no matter which tool you decide to use, you should be ready to write integration tests which are very different in nature compared to your unit tests.
It is possible to run tests writen for Nunit against a deployed environment.
You need a way to configure your tests according to the environment to test.
The type of test: unit tests, integration tests, functional tests, system tests etc. is independent from the technology you use to run them. NUnit is a technology to execute tests and gather their results. It has probably been architectured thinking about unit tests, but you can even implement your system tests with it.
I have NUnit 2.5.10 and Specflow 1.8.1 installed with VS2010.
Right click on .feature file in VS, there is an item "Debug Specflow Scenarios" in context menu. But I always get "Could not find matching test runner". In Tools/Options/Specflow, there is no test runner option for NUnit.
Am I missing something?
Thanks in advance. :)
I don't know if NUnit has a seperate GUI for running tests as i use MSTests but this should be how u do it.
Setup a config file in your Test Project to tell Specflow what Test framework to use.
https://github.com/techtalk/SpecFlow/wiki/Configuration
The Tools > Options > Specflow > Test Runner tool is the tool that executes the Tests. The fact that nothing is showing here seems like the test runner is not installed. For example i have several, MSTest, Resharper, SpecRun, i normally leave it on Auto and just run the test from the test tool i want to use. I ran installers for these though which should add any VS extensions
You could try using the VS runner, Test > Windows > Test View
VS will just treat a Specflow test as a NUnit as the generated code behind is just an NUnit test (when configured that way). So i think the real question is how to debug an NUnit test in Visual Studio. A quick search suggested attaching the debugger to the NUnit test runner process.
Like Ryan I also have no test runner specifically for Nunit. However since I'm also running resharper I get additional resharper runners, and that is what Auto uses to run it.
Have you actually got nUnit installed on your machine? In which case you can test how successful the process is without VS and Specflow-addin getting in the way. If not, well installing may give you the runner you need.
Alternatively, I use nCrunch to automatically run all my tests. I've had it running nunit/specflow for the last few months on multiple solutions with no issues. No need to right click at all then. :-)
SpecFlow itself does not provide a test runner and if you want to right click on the .feature and run it, you'll have to re-configure SpecFlow to use MSTest, which should use built-in VS runner.
If you want to use NUnit, then additional tools are required. I would recommend Resharper, but it's not free. Check out this article for more options to run NUnit tests from VS.
Good day. To debug i do the following :
Put a break point on the scenario line that you are interested in
Make sure that you have NUnit running
Attach the NUnit process
Go to tools
Select attach process
In the available processes select NUnit agent.exe
Click attach
Run the test you want to debug in NUnit
Hope that helps
I have some automated tests written in nunit and selenium. How can I call the dll to execute from Microsoft test Manager. as I couldnt figure out how to run automated tests from MTM. is it possible.
MTM does allow to automate the test.
We can browse a test case in Visual Studio and attach the executable to the test case (Created in MTM). Later we have to link the build to the test plan as well for the execution of the automated test. Test code should be developed as a test project (CodedUI or Unit test) if not done so the test manager will not detect the method to be tested.
For MTM to run automated tests with Selenium you need to associate the test with a Test Case in MTM. Currently MTM only supports MS Tests so you would need to build a wrapper for each of your selenium tests.
Solution #1: I would build the wrapper with a T4 template that generated the required MS Test stubs that call my Selenium tests.
You would then need to have a Test Case in TFS with the MS Test 'automation' associated with it.
Solution #2: You can use the "tcm import" command line tool to generate and keep your Test Cases in sync with the automation.
Now that you have all of the bits configured you can go ahead and call the generated tests from MTM and you hit the issue that #richard mentioned.
Solution #3: You can configure an Environment in MTM where you are going to collect the data and automate the trigger of the test run. This can be done as part of the Build, or better as part of your binary Pipeline in Release Management (http://nakedalm.com/execute-tests-release-management-visual-studio-2013/)
While there are a lot of hoops to get all setup I have found this to be a fairly robust way to execute the automation. It would be a lot easier if MTM supported other test frameworks, but that is not the case just now.
MTM doesn't run automated tests. It's a tool designed for manual testers. The only automation it offers is the record/replay of actions that a tester has recorded.
If you want to automatically run Selenium tests then look at running NUnit as part of the build process, just make sure the build agent is running as an interactive processes not a service so that Selenium can access the desktop and run a browser.
Alternatively you could look at using a headless browser such as XBrowser or HTMLUnit (though you'll need to use the Selenium2 remote driver)