I have a visual studio loadTest project and LoadTest2010 result database, when I run the test from visual studio then test work fine and test data save in the LoadTest2010 DB but when run the test from VNext then test run fine (did all the configurations) but data does not save in the DB LoadTest2010.
In VNext VsoAgent agent is running the test.
Did any one face this issue?
vNext is TFS Test Agent and it is VS2015 Test Agent, we ever used Test Controller and Test Agent for VS2013 Update5. vNext is Test Agent for VS2015.
According to your description, whether you have added a web test project in load test, if no, please make sure you have added it.
and then please use MSTest to run load test locally to see the results.
What test steps do you use in your build definition? If you use the steps Visual Studio Test Agent deployment and Run functional test to run your load test, it may get the result like you posted above. Because these 2 steps are using VS Test agent 2015 and if you want to run load test, you need to install VS Test Agent 2013 according to this document.
So about using the VS 2013 test agent in Vnext build, you could follow the step below:
Add a test setting file for your test solution.
Next, specify test controller and Load test result store in the setting file.
Then add a Visual Studio Test step in your build definition and in this step, remember to specify the testsetting file.
this is the only one option right now in the VNext to run the loadtest and data will save to the loadtest database. Below is the snapshot from VNext
Related
Question:
How do I run NUnit tests on TFS?
Attempts:
I've searched the web and am still struggling to figure out the exact steps required to accomplish (to what I believe is) a trivial task.
I added the NUnit3TestAdapter package to my test project.
However, my tests are still not being discovered.
TFS displays the following message:
No test runs are available for this build. Enable automated tests in
your build pipeline by running your test framework of choice, such as
JUnit, Visual Studio Test, or xUnit. If you choose to run tests using
a custom task or runner, you can publish results using the Publish
Test Results task.
Expectation:
I'm surprised that NUnit is not an option that I can just select from.Hence, it's the standard unit test framework for .Net.
How do I run NUnit tests on TFS?
There is a blog here describes clearly how to run NUnit tests on TFS: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Note：
You do need to edit the Visual Studio Test task’s advanced settings
to point at the NuGet packages folder for your solution (which will be
populated via NuGet restore) so the custom nUnit test adaptor can be
found i.e. usually setting it to $(Build.SourcesDirectory)\packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
We are trying to integrate the Coded UI Test automation into TFS 2015. The build definition is created and is working correctly. But Release Definition is having problem.
Below are the tasks added for Release Definition:
1. Windows Machine File Copy
2. Visual Studio Test Agent Deployment
3. Run Functional Tests
Please find the attached screen shots for each task:
WindowsMachineFileCopy
Visual Studio Test Agent Deployment
Run Functional Tests
Please find the attachment for the automated test case which is associated in Run Functional Test Task.
Automated Test Case
The Release runs successfully till Visual Studio Test Agent Deployment. Even the Test Agent pop up is opened. After this, the browser is not getting opened i.e., while running Run functional Tests task, it throws below error:
"Unit Test Adapter threw exception:
Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.."
I have placed all the dll's including the references in bin folder. Please find below screen shot:
DLL's
I am using Visual Studio Premium 2012, Test Agent 2015 Update 3 and TFS 2015.
Installed Test Agent 2013 and now getting attached error in Deploy Test Agent task:
Test Agent Deploy error
Please suggest the solution so that my coded ui starts running through TFS.
You need to install Test Agent for Visual Studio 2013 on the test machine based on the Compatible versions of TFS,and test agent
Also there is a blog introduce the related issue :Test Agents support for Visual Studio 2015
So, just try to use Test Agent for VS 2013 instead of Test Agent 2015 Update 3, then try it again.
I am using a TFS Release to execute .loadtests written in Visual Studio.
I can execute the tests fine using MSTest, this produces the expected .trx file.
What I want is to be able to see the test results within TFS, I have tried using the 'Publish Test Results' task but have experienced issues.
The issue encountered was an Object not set to an instance of an object
Upon analysing the powershell scripts that 'Publish Test Results' task executes, from what I can see, the only results that are read in the TrxResultReader are from 'WebTest' and 'UnitTest'.
Has anyone managed to publish the results of a loadtest within TFS?
For TFS Publish Test Results task. This task supports popular test result formats including JUnit, NUnit 2, NUnit 3, Visual Studio Test (TRX), and xUnit 2. If you use the built-in tasks such as Visual Studio Test to run tests, results are automatically published and you do not need a separate publish test results task.
Based on the powershell script behind the build task, it use TestRunner. Since Vstest cannot run load test, it cannot publish load test result too. Take a look at below similar question:
Publish Test Results task fail when trying to publish a loadtest
It is not possible to use mstest to publish test results to build
made by a vNext. mstest can not identify the correct buildnummer /
buildid. Our solution will probably be that we store the *.trx file a
artifact. An other solution is to use a xaml build to perform
performance tests.
We have a Visual Studio 2015 solution containing some C++ projects and some tests based on Google Test Framework.
Now I would like to run those tests with the quite new TFS 2015 build features. I know that there is the "Visual Studio Test" build step that is able to run custom test adapters (like the Google Test Adapter?).
Is this the easiest way to setup things? What exactly has to be installed on the (on premise) TFS2015 build server and how to configure the build steps?
Thanks for you help! Sebastian
Yes, the simplest way is just using google test adapter in ""Visual Studio Test" build " task. Just as the feature statement which will using VSTest.Console.exe
You need to install visual studio on your build server(agent). About how to conigure the build steps, there has been a detailed tutorial with Xunit test which also applies to google test. Plesae refer this blog: Running xUnit tests in TFS Build vNext
I'm looking to get TFS to build and deploy to our manual and auto TEST environments.
manual TEST - overnight updates
auto TEST - continuous integration
Once deployed, I want TFS to deploy and run a set of web-integration tests (SpecFlow / Selenium / nUnit) on another server (acting pseudo client of the website).
I then want TFS to collate the results of these tests and report back.
What's the best approach to get this working?
You'd normally change the build process template to do what you want. Adding steps to deploy could be one of these customozations. Have a look at the Build Release Deploy build template from the Microsoft ALM Rangers which includes steps to publish the build to another machine.
You can also make use of WebDeploy and add a set of publish profiles to your solution. Team Build will happily execute those steps during build, if told to do so.
In TFS you can use a Test Agent to then execute tests from a remote machine and have the results added to the test results for your build.
Seeing that you'd like to use NUnit ad SpecFlow I suggest using TFS 2012 in combination with Visual Studio 2012, which have built-in support for executing 3rd party based tests. You'll need to install the NUnit adapter and put the assemblies for it in source control so that Team Build will pick them up.