We are using SpecFlow, Selenium WebDriver[C#] with VSTS 2017. In Test Explorer Associate Tests option is disabled and hence not able to do automation association to test case. In Automation Association tab of Test case Choose Tests window is not displaying with any of my tests which are present in Test Explorer.
BDD Frame work : SpecFlow 2,
VSTS 2017 Tool,
Unit Test Provider: SpecRun,
Project Type: Test Project,
Visual Studio default Architecture: X64,
Note: We are using Azure DevOps, only for automation association we are using VSTS.
You should be able to add an association to a test case from the test explorer like this
Right click on the test you want to associate then click on the associate to test case. I have checked and it works in VS 2017 and 2019.
Related
I use Visual Studio Community edition at the moment. I want to create some test cases in a customer's on-prem TFS and they havn't bought the Test Manager extension. When I open web access I cannot add any test plans, test suites nor test cases.
I installed Microsoft Test Manager 2017 and from there I can create everthing I need and also run the test cases. After creating the test cases I can run them from web access.
I would be glad to buy some license for myself but wouldn't want to bother the customer with buying the Test Manager extension.
Question 1: I can't see what license Test Manager is using at the moment, e.g. could I be using a trial license that would expire and thus the above will stop working?
Question 2: Since the above is working, i.e. I can add test cases without paying for anything, is there a chance Microsoft will be plugging this in a future tfs update, i.e. Test Manager will stop working?
The desktop client for Microsoft Test Manager will be retired and only web access will be able to work with test cases.
If you look at https://www.visualstudio.com/vs/pricing/, you see that VSTS Test Manager is included with a Visual Studio Enterprise license. That's one way of purchasing Test.
Another way is to add the Test package to your account through the Marketplace: https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web#pricing. This will cost you $52 per month.
Visual Studio Team Services (VSTS) and Team Foundation Server (TFS) offer both web-based and client-based solutions for manual testing:
The Test Center in Microsoft Test Manager (MTM) client is a
desktop-based manual testing solution, which has traditionally been
used by testers for their Manual testing needs.
The Test hub in VSTS and TFS is a web-based manual testing solution,
which works across all platforms and with all browsers. We have
invested in Test hub over past two years to provide you better
experiences across Plan, Author, Execute and Track phases of Manual
testing.
Because the Test hub is a fully featured Test management solution
which works across all platforms and with all browsers, we recommend
you use the Test hub over Microsoft Test Manager for all your test
management requirements. You can use Microsoft Test Manager to test
your desktop applications by launching the Microsoft Test Runner
(client) from the Test hub.
This answered your second question Test Manager will stop working? For now, even it's not strongly deprecated by Microsoft, but also not been a recommend test management solution in TFS/VSTS. More details you could refer this link.
For question one, there are no trail license for MTM. You need to have Test Manager license. This license will be included in below subscriptions. Otherwise you have to buy this license separately.
Visual Studio Enterprise - monthly
Visual Studio Enterprise - annual
Visual Studio Enterprise with MSDN
MSDN Platforms
Visual Studio Test Professional with MSDN
Source Link: pricing
In your case, there's a strong possibility that you have a Visual Studio Enterprise license, so MTM work properly.
Why is the initial state of a test suite 'In Progress' then you flip it to 'In Planning' then back to 'In progress'? This doesn't make sense to me and I can't find any documentation regarding MTM 2013 states on why this was done? I don't want to edit the states if I don't have to but would like some justification to keep it as is.
You can set the state of a test suite to indicate whether you are ready to run its tests or whether its tests have all completed.
If you don't want to use this feature, leave the state at its default value of In Progress.
Set the state to inform other team members:
In Planning: The test cases in the suite are not yet ready to run.
In Progress: The test suite can be run.
Completed: The tests have completed and do not have to be re-run in this test suite. You might still run the same test cases
in other test suites or test plans.
Note: The tests in a test suite cannot be run unless its state is In Progress.
According to your description, seems you are lacking of the permission "Manage Test Suite" . Then you could not be able to change the state. You could double check this with your TFS Admin. More details please see this link: Test Suite Security Permissions in MTM
Update
In MTM2010, the default state for a test suite in a test plan is still In planning. It's hard to judge a specific period of a product. At that time , one of the top feature requests from Microsoft Test Manager/Test Case Management users across various forums has been the ability to customize the Test Plan and Test Suite artifacts. Based on the feedback, with Team Foundation Server 2013 Update 3, we are adding the functionality to add custom fields and workflows to Test Plans and Test Suites.
With TFS 2013 Update 3, you can customize test suite states, and removed the restriction to only let you run tests if the test suite state is “In Progress”. Some users like you might have their own custom states that they want to use for tests that can be run, so this change allows more flexibility.
Prior to TFS 2013 Update 3, test plans could only have “Active” and “Inactive” states and test suites could only have “In Planning”, “In Progress”, and “Completed” states. With Update 3, a test plan can be customized to have different workflow states. If you use an MTM client with a version of Visual Studio prior to Visual Studio 2013 Update 3, you might get an error when you try to update the state because this version of the client was not designed for custom states. Upgrade your MTM client to Visual Studio 2013 Update 3 and you will be able to change the states. Or you can open the test plan or test suite work item directly from MTM (type Ctrl+g, then enter the id of the work item).
More details please refer this blog: Test Plan and Test Suite Customization with TFS2013 Update3
So in your scenario, the best solution is updating your TFS to TFS2013 update3 or above and also updating VS to VS2013 update3 or above.
My organization uses TFS 2013, with a split of VS 2012 and VS 2013 for the programmers and testers.
When I try to right click --> open in web access on any of our MTM tests or suites, I get this error:
"The test plan with id 97761 does not exist or it’s area path is not owned by the default project team. Include it in the default project team’s owned areas and try again."
Which leads me to believe that this is a configuration/permissions issue, however our IT could not find what the problem is.
Any ideas? I've heard from team mates that this feature used to work before TFS was upgraded from 2012 to 2013.
We had the same problem, and we finally narrowed it down to a change to how Test Plans work in TFS 2013 Update 3.
In the past test plans were not considered work items, but since TFS 2013 Update 3, they are.
We solved the problem by opening the root test plan as a work item in Visual Studio 2013, and saving it.
Since then, test plan web access no longer throws the error "Test plan with id xxxxx does not exist or its area path is not owned by the default project team."
Make sure you are in the Full Access group in the TFS Web Access Control Panel. Also make sure you have permissions to the area that the test case/suite/plan belong to.
I'm trying to setup TFS 2010 Lab Management based Automated Testing system and I have a "Test suite" (Created in TFS Test Manager) with following tests:
1) Start and Login
2) Create group
3) Create user and add to the group
These have order specified properly in Testing Center->Plan->(Test suite)->Order column
However, when they are executed (on TFS Lab Management Build) by Test Agent they run out of order: 2nd then 1st and then 3rd.
Is there a way to make Test Agent run these tests in order?
Thanks.
Create an Ordered Test, it's a simple list of which test methods should run and in what order.
I also had the same issue. I solved it by installing VS2010 Service Pack and TFS 2010 Service Pack in both the Test controller machine and Test agent machines. Please go through the following link for more details
http://social.msdn.microsoft.com/Forums/en-US/vsmantest/thread/29ac3c7b-b0cd-40f3-9992-f5f3c2285331
Unfortunately, There's a blog regarding ordering test cases which says:
Please note this only applies to the manual tests but not automated
tests. For automated tests, the order you set here will not be
respected during test execution.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/11/21/run-your-tests-in-the-order-you-want.aspx
I don't know if it's a new feature but you can create Ordered tests. Just right click on your project in Solution Explorer and choose Add -> Ordered Test. Select you tests in order.
In Test Explorer, run you ordered test.
Details are in the link below but all I had to do is what I'd said above. I am using VS 2015 Enterprise.
https://msdn.microsoft.com/en-us/library/ms182631.aspx
I'm just getting started with the Team Foundation Server SDK and I'm wondering if there is a way to create a new Team Project using the SDK.
My goal in doing this is to create some automated tests to aid in system validation (being an FDA regulated company we are required to validate our in-house tools, to the extent this can be automated we can reduce costs). I would like to validate as much of the process end-to-end as possible (create a team project, check in some items to source control, create some work items, verify everything is stored/recalled correctly, etc). If there is a better approach or any existing work in this area I'm all ears.
Not using the tfs-sdk. If you install Team Explorer 2008, the TFS 2008 power tools and apply VS 2008 SP1 then you can use the tfpt createproject command to create a team project from a script (or execute the tfpt command as a process in your .NET application). The command uses automation hooks added into Visual Studio in 2008 SP1 to automate the team project creation process.
When doing all this in an automated test environment, you'll probably have more luck doing this against a Virtual Machine (either VMWare or Microsoft Virtual PC / Hyper-V) and then you can automate rolling back the TFS instance to a known clean state each time you repeat your testing.