I'm now somewhat familiar with UI testing within Jetpack Compose thanks to the extensive Android Documentation, but I can't make Android Studio to run Compose tests with coverage. We are currently using Jacoco as coverage tool to generate the reports, but according to this Github issue it does not look like coverage for Compose is a priority for the team (sadly :S).
As we don't want to keep UI tests aside when evaluating code coverage, is there any way to enable code coverage in Android Studio for Compose tests, either using Android Studio or an external tool/library?
Related
Question:
How do I run NUnit tests on TFS?
Attempts:
I've searched the web and am still struggling to figure out the exact steps required to accomplish (to what I believe is) a trivial task.
I added the NUnit3TestAdapter package to my test project.
However, my tests are still not being discovered.
TFS displays the following message:
No test runs are available for this build. Enable automated tests in
your build pipeline by running your test framework of choice, such as
JUnit, Visual Studio Test, or xUnit. If you choose to run tests using
a custom task or runner, you can publish results using the Publish
Test Results task.
Expectation:
I'm surprised that NUnit is not an option that I can just select from.Hence, it's the standard unit test framework for .Net.
How do I run NUnit tests on TFS?
There is a blog here describes clearly how to run NUnit tests on TFS: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Note：
You do need to edit the Visual Studio Test task’s advanced settings
to point at the NuGet packages folder for your solution (which will be
populated via NuGet restore) so the custom nUnit test adaptor can be
found i.e. usually setting it to $(Build.SourcesDirectory)\packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
I've just literally hit a brick wall trying to configure F# 4.5.0.0 under VS 15.8.1 to run xUnit tests. Here is the repo with the issue: https://github.com/kkkmail/fSharp-xUnit .
When I try to run all tests, VS comes back with this message: No test is available in C:\GitHub\fSharp-xUnit\DummyApp\DummyApp\bin\Debug\DummyApp.exe C:\GitHub\fSharp-xUnit\DummyApp\Tests\bin\Debug\Tests.dll C:\GitHub\fSharp-xUnit\DummyApp\DummyApp\DummyApp.fsproj C:\GitHub\fSharp-xUnit\DummyApp\Tests\Tests.fsproj. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Both the "DummyApp" and tests were created from scratch using VS and I cleared all temp files and all nuget caches prior to creating the solution as advised in various places on the web.
TL;DR Add xunit.runner.visualstudio to your test project.
For discovering tests Visual Studio relies on its test runner. This test runner can discover / run all tests for testing frameworks which implement the corresponding adaptor interfaces ITestDiscoverer and ITestExecutor. For xUnit, these interfaces are implemented in VsTestRunner which is published in xunit.runner.visualstudio.
See also the xUnit docs about running tests in VS
We have a CI system using TFS 2015, and all was working well.
However, we added some new tests, and the "Gather Artifacts" stage stopped working.
I tracked this down to the fact that the MSTest command-line gets built thus:
MsTest.exe /testcontainer:blah /test:test1 /test:test2 [...] /resultsfile:"c:\blah.trx"
So when we go above a certain number of tests, the command-line ends up too long and becomes truncated, losing the /resultsfile switch and possibly some of the /test: switches also.
In this instance, TFS reports the suite as having passed, despite the fact that all tests did not run.
I see in the documentation for MSTest.exe that there is a /testlist: parameter that lets you specify which tests run by pointing at a file.
Can anyone help me work out how I switch over to using a testlist, or some other way of solving this issue?
I found the solution, it was to switch from using "Test Plan" to "Test Assembly"
Just as DaveShaw said in the comment, the limitation is not of MSTest or TFS, it's related to Windows Command Line AFAIK.
We can use both VSTEST and MSTEST to run automated unit and coded UI tests from a command line.
VSTest.Console.exe is optimized for performance and is used in place
of MSTest.exe in Visual Studio.
After finish running the test. It won’t save the results and there is no .trx file generated in the target directory.
MSTest.exe You can use the MSTest.exe program to run automated tests
in a test assembly from a command line.
MSTest is used for load tests and for compatibility with Visual Studio 2010 test projects.
MSTest can also be used to view the test results from these test runs, save the results to disk, and save your results to Team Foundation Server.
More detail info you could refer this link: How to choose between tcm.exe, mstest.exe and vstest.console.exe
Since you are using TFS2015, and if you don't have any VS2010 test projects,
we encourage you to use VStest instead of MStest.
I have NUnit 2.5.10 and Specflow 1.8.1 installed with VS2010.
Right click on .feature file in VS, there is an item "Debug Specflow Scenarios" in context menu. But I always get "Could not find matching test runner". In Tools/Options/Specflow, there is no test runner option for NUnit.
Am I missing something?
Thanks in advance. :)
I don't know if NUnit has a seperate GUI for running tests as i use MSTests but this should be how u do it.
Setup a config file in your Test Project to tell Specflow what Test framework to use.
https://github.com/techtalk/SpecFlow/wiki/Configuration
The Tools > Options > Specflow > Test Runner tool is the tool that executes the Tests. The fact that nothing is showing here seems like the test runner is not installed. For example i have several, MSTest, Resharper, SpecRun, i normally leave it on Auto and just run the test from the test tool i want to use. I ran installers for these though which should add any VS extensions
You could try using the VS runner, Test > Windows > Test View
VS will just treat a Specflow test as a NUnit as the generated code behind is just an NUnit test (when configured that way). So i think the real question is how to debug an NUnit test in Visual Studio. A quick search suggested attaching the debugger to the NUnit test runner process.
Like Ryan I also have no test runner specifically for Nunit. However since I'm also running resharper I get additional resharper runners, and that is what Auto uses to run it.
Have you actually got nUnit installed on your machine? In which case you can test how successful the process is without VS and Specflow-addin getting in the way. If not, well installing may give you the runner you need.
Alternatively, I use nCrunch to automatically run all my tests. I've had it running nunit/specflow for the last few months on multiple solutions with no issues. No need to right click at all then. :-)
SpecFlow itself does not provide a test runner and if you want to right click on the .feature and run it, you'll have to re-configure SpecFlow to use MSTest, which should use built-in VS runner.
If you want to use NUnit, then additional tools are required. I would recommend Resharper, but it's not free. Check out this article for more options to run NUnit tests from VS.
Good day. To debug i do the following :
Put a break point on the scenario line that you are interested in
Make sure that you have NUnit running
Attach the NUnit process
Go to tools
Select attach process
In the available processes select NUnit agent.exe
Click attach
Run the test you want to debug in NUnit
Hope that helps
I have some automated tests written in nunit and selenium. How can I call the dll to execute from Microsoft test Manager. as I couldnt figure out how to run automated tests from MTM. is it possible.
MTM does allow to automate the test.
We can browse a test case in Visual Studio and attach the executable to the test case (Created in MTM). Later we have to link the build to the test plan as well for the execution of the automated test. Test code should be developed as a test project (CodedUI or Unit test) if not done so the test manager will not detect the method to be tested.
For MTM to run automated tests with Selenium you need to associate the test with a Test Case in MTM. Currently MTM only supports MS Tests so you would need to build a wrapper for each of your selenium tests.
Solution #1: I would build the wrapper with a T4 template that generated the required MS Test stubs that call my Selenium tests.
You would then need to have a Test Case in TFS with the MS Test 'automation' associated with it.
Solution #2: You can use the "tcm import" command line tool to generate and keep your Test Cases in sync with the automation.
Now that you have all of the bits configured you can go ahead and call the generated tests from MTM and you hit the issue that #richard mentioned.
Solution #3: You can configure an Environment in MTM where you are going to collect the data and automate the trigger of the test run. This can be done as part of the Build, or better as part of your binary Pipeline in Release Management (http://nakedalm.com/execute-tests-release-management-visual-studio-2013/)
While there are a lot of hoops to get all setup I have found this to be a fairly robust way to execute the automation. It would be a lot easier if MTM supported other test frameworks, but that is not the case just now.
MTM doesn't run automated tests. It's a tool designed for manual testers. The only automation it offers is the record/replay of actions that a tester has recorded.
If you want to automatically run Selenium tests then look at running NUnit as part of the build process, just make sure the build agent is running as an interactive processes not a service so that Selenium can access the desktop and run a browser.
Alternatively you could look at using a headless browser such as XBrowser or HTMLUnit (though you'll need to use the Selenium2 remote driver)