Why is the initial state of a test suite 'In Progress' then you flip it to 'In Planning' then back to 'In progress'? This doesn't make sense to me and I can't find any documentation regarding MTM 2013 states on why this was done? I don't want to edit the states if I don't have to but would like some justification to keep it as is.
You can set the state of a test suite to indicate whether you are ready to run its tests or whether its tests have all completed.
If you don't want to use this feature, leave the state at its default value of In Progress.
Set the state to inform other team members:
In Planning: The test cases in the suite are not yet ready to run.
In Progress: The test suite can be run.
Completed: The tests have completed and do not have to be re-run in this test suite. You might still run the same test cases
in other test suites or test plans.
Note: The tests in a test suite cannot be run unless its state is In Progress.
According to your description, seems you are lacking of the permission "Manage Test Suite" . Then you could not be able to change the state. You could double check this with your TFS Admin. More details please see this link: Test Suite Security Permissions in MTM
Update
In MTM2010, the default state for a test suite in a test plan is still In planning. It's hard to judge a specific period of a product. At that time , one of the top feature requests from Microsoft Test Manager/Test Case Management users across various forums has been the ability to customize the Test Plan and Test Suite artifacts. Based on the feedback, with Team Foundation Server 2013 Update 3, we are adding the functionality to add custom fields and workflows to Test Plans and Test Suites.
With TFS 2013 Update 3, you can customize test suite states, and removed the restriction to only let you run tests if the test suite state is “In Progress”. Some users like you might have their own custom states that they want to use for tests that can be run, so this change allows more flexibility.
Prior to TFS 2013 Update 3, test plans could only have “Active” and “Inactive” states and test suites could only have “In Planning”, “In Progress”, and “Completed” states. With Update 3, a test plan can be customized to have different workflow states. If you use an MTM client with a version of Visual Studio prior to Visual Studio 2013 Update 3, you might get an error when you try to update the state because this version of the client was not designed for custom states. Upgrade your MTM client to Visual Studio 2013 Update 3 and you will be able to change the states. Or you can open the test plan or test suite work item directly from MTM (type Ctrl+g, then enter the id of the work item).
More details please refer this blog: Test Plan and Test Suite Customization with TFS2013 Update3
So in your scenario, the best solution is updating your TFS to TFS2013 update3 or above and also updating VS to VS2013 update3 or above.
Related
Since MTM lab management is deprecated in TFS 2018 and we are setting up test agents as part of build definition ,How can we see test cases results in real time ?
For example if user wants to know how many test cases are in progress ,passed,failed.
In MTM we can see while running test cases using TCM.exe till tfs 2017
Could any one please help
Thanks
Unfortunately real-time output for tests is not supported for now.
There are some similar user voices submitted to suggest the feature:
Provide VStest.console.exe \ Test Explorer Real Time Test Execution
Status DURING Test Execution
That I could see the reports that I see in the MTM tool, from a
Confluence Page (real time updated info on testing status).
You can go and vote them up or submit a new user voice to suggest the feature again to achieve the feature in future release.
Similar threads for your reference:
Real time status/logs of test execution not available on VSTS
console
Real time logs/stats of unit test execution on VSTS console
I'm new to Microsoft Test Manager. I have lot of Test suites in my MTM and want to change the status of each test case from Design to Ready state in a particular test suite.
Is there a way to do it in MTM or TFS without individually changing each and every test case status manually. (There are no tags for the Test Case and I tried to query by test suite name but it doesn't work).
Note:
I know the I could change the state of the test suite to In progress and All the test cases will be available for testing.
Thanks in advance.
If you have latest version of tfs on web test hub you may select several test (screens from vsts):
set new value for the State field
You can write a query in TFS to find all the test cases that meet your criteria.
Export into Excel (or Import using TEAM tab in Excel) and change all the values and publish the results back.
Or on the latest version of VSTS I can select all the WI's in the Query Result, right click and select "Edit", then change the State to Ready and save all.
In the old TFS 2013 days, if you wire up your automated test cases with MTM and run them via MTM, you can view all historical test results for each of those test cases in the Test Plan.
With TFS 2015 Build/Release + Run Functional Tests tasks, in MTM, you can only see the latest result. How can you view all historical runs of a particular test case? In the web portal, you can see "Test Runs", but there's no way to find just the runs pertaining to the particular test case in question.
I bumped into a workaround for this by accident. If I manually mark the test as Pass/Fail, then when I go back to MTM, and "View Results" again for this test case, it will all of a sudden display everything in its history, including the automated test runs generated by builds or releases.
It seems like a bug in the MTM UI, but this works for now!
We are moving our projects to TFS 2010 using the SfTS v3 (Scrum for Team System) template. We need to understand how Microsoft Test Manager is supposed to be used in this Scrum process.
Specific scenario & question:
The QA manager uses Test Manager to create Acceptance Test Work Items (WIs). These new WIs are created and "assigned to" him. The manager doesn't run all the tests, instead he wants to give that responsibility to his staff.
How is a QA engineer supposed to know that he has tests to run? Everything says it is assigned to the manager.
QA Manager has to create Test Suites at his Test Plan and then to assign Testers (QA Engineers) to each Test Suite that his wants to test.
Check this: How to: Assign Who Will Run the Tests in a Test Plan.
I'm trying to setup TFS 2010 Lab Management based Automated Testing system and I have a "Test suite" (Created in TFS Test Manager) with following tests:
1) Start and Login
2) Create group
3) Create user and add to the group
These have order specified properly in Testing Center->Plan->(Test suite)->Order column
However, when they are executed (on TFS Lab Management Build) by Test Agent they run out of order: 2nd then 1st and then 3rd.
Is there a way to make Test Agent run these tests in order?
Thanks.
Create an Ordered Test, it's a simple list of which test methods should run and in what order.
I also had the same issue. I solved it by installing VS2010 Service Pack and TFS 2010 Service Pack in both the Test controller machine and Test agent machines. Please go through the following link for more details
http://social.msdn.microsoft.com/Forums/en-US/vsmantest/thread/29ac3c7b-b0cd-40f3-9992-f5f3c2285331
Unfortunately, There's a blog regarding ordering test cases which says:
Please note this only applies to the manual tests but not automated
tests. For automated tests, the order you set here will not be
respected during test execution.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/11/21/run-your-tests-in-the-order-you-want.aspx
I don't know if it's a new feature but you can create Ordered tests. Just right click on your project in Solution Explorer and choose Add -> Ordered Test. Select you tests in order.
In Test Explorer, run you ordered test.
Details are in the link below but all I had to do is what I'd said above. I am using VS 2015 Enterprise.
https://msdn.microsoft.com/en-us/library/ms182631.aspx