I'm asking this question on behalf of a colleague (which means I may have the terminology incorrect).
When a colleague adds a Test Case to a card, a requirement test suite is created in the wrong test plan.
We don't want a test suite to be created, we just want the test cases to be on the card itself.
How can we change this behaviour in TFS?
I did not see any settings for the test plan creation. The test plan creation works with next steps (tested on vsts):
System tries to find any plan with requirement. If there is no any plan then next.
System tries to find any plan with the iteration path from requirement. If plan exists system will import requirement to that plan. If there is no any plan then next.
System creates plan with iteration path from requirement and with requirement.
So you may use a workaround:
Create iterations for all test plans that you plan to use.
Assign needed iteration for selected test plan.
Assign needed iteration for requirements that will be used for testing.
There is no option to disable this behavior. But you can create a test plan and then configure the VSTS/TFS to place all the test case there:
Click the "Configure team settings" button:
Navigate to "Tests" panel and then choose "Select a test plan".
Related
I am on a development team where we have 2 separate mobile apps. One of the apps is for Users. The other app is for Admin of those users. My main objective is to execute a test case in the Admin app, and then run a test case in the Users app to verify its working properly. How can I approach this?
For example, I want to run a test case in the Admin app that revokes some privilege. I then want to run a test case from the Users app to check to confirm that privilege was revoked.
Maybe this is not a good strategy at all -- but it makes sense for my team because we have 2 apps that work together -- and if we do some function in the Admin app -- we want to see the expected result in the Users app
My plan was to mark each test with a Category, for example, "Privilege"
On Jenkins:
Run "Privilege" Category on the Admin app where I revoke some privilege
Run "Privilege" Category on the User app where I confirm revoked privilege
This seems like an ok test strategy right now. But if I have 20 UITests that means I'll have 20 different Jenkins projects in my dashboard, one for each UITest (per device, per platform). It seems that with 20 UITests I'll end up with over 100 Jenkins projects. Thats not really ideal to me.
Has anybody else come up with a testing strategy where they needed to test 2 separate projects back and forth. I understand that this does not really fall under unit testing - and I may get some vague answers around unit testing and general. But I do believe mobile is a different animal in the UITest world
There are couple of points in your question
do some function in the Admin app -- we want to see the expected result in the Users app
If you need to test such integrations between the two apps, you can go with proper labels for ones that are
mark each test with a Category
in any case, you will need some way to organise your suites. Good way to do so, are test annotations. I think Lazy setup is aplicable in your case. It will set the desired state for all marked tests, when needed.
needed to test 2 separate projects back and forth
End-to-end tests are mandatory, for the most business critical features. My suggestion is to employ Backdoor manipulation. Your other tests should already have covered the simpler cases (e.g. setting a privilege in Admin app), so if you already did exercise this feature, no point of redundancy.
It seems that with 20 UITests I'll end up with over 100 Jenkins projects. Thats not really ideal
You actually don't need a Jenkins project per suite, just configure the tests via CLI arguments and your harness will pick that up for you. What you need is a tag (or platform, or device) to be passed to the runner.
Generally, you do NOT want tests to depend on each other. Have a look at this example:
In the admin app, you set the privilege.
You open the user app.
The privilege should be set, but it's not.
You know that something went wrong, but you don't know whether it's the admin app that's not working or the user app.
Therefore, you should test them independently by mocking (=faking) the backend:
Open the admin app.
Set the privilege.
Ask the mocked backend: Did you receive a call from the admin app to set the privilege?
In an independent UI test for the user app, you do the following:
Set up a fake backend where the privilege is set
Open the user app
See whether the privilege is set.
By separating the tests for both apps, you will know which of the two does not work.
The software that I use at work to do such things is called WireMock but there are others out there, too.
Using the power tool I was able to add a new custom field but I want that it take automatically a specific value if present.
Each test case has a "user story" associated but to associate it we have to go in "Tested User Story Section" so I'd like that in the main page appear automatically the ID of the user story.
Sorry for the confusion maybe with the image will be more understandable:
Someone can advice me of what to introduce in the Visual Studio Power Tool please? because I'm finding different example but none explain how to auto-populate with another dynamic item.
If you are asking me why it's because we want to have the "User Story ID" present in the main page so we know directly at which user story the test case is associated:
There is no way to get the User Story ID as a field. Since a single Test Case can test multiple User Stories this makes little sense anyway.
Have you tried adding Requirement based Suits to Test Professional so that you can see all of the Test Cases under a Story? From a testing perspective it makes lots of sense to look from the Story down to Tests rather than Test up to Stories.
Although my screenshots are from the Web App the features at the same in MTM. You can add a Requirement Based Test Suite that relates back to the User Story.
As #MrHinsh mentioned, there is no direct way to get the User Story ID from a field.
From TFS query, it's easy to get the linked User Story, you can select the Work Item and Direct Links in work item query, check the screenshot below:
Direct Links query can be created two ways in MTM
Plan->Contents->New Query-based suite->(query type)->Work Items and Direct Links
Track->Queries->New Direct-Link Query
First one creates a test suite in the end. Is there any other difference?
Your two techniques are doing different things and have different purposes.
Option 2 is simply doing a Work Item Query, the same way you can do in Team Explorer/Visual Studio and also Team Web Access.
Once you have viewed the information you can discard it or save it in "My Queries" or "Shared Queries". If you save it, you can view the results in Excel for example or perhaps share it with colleagues.
Option 1 is all about creating a Test Suite which is a grouping of Test Cases.
There are static suites which are essentially folders for grouping Test Cases.
There are requirements based suites - for example, add all the PBIs from Sprint 1 and then we can add Test Cases to ensure we have tested against their acceptance criteria.
Then there are query based suites which will add all the test cases which match your specified query. New test cases will be added if they match the query (eg. all high priority test cases)
Organizing Test Cases Using Test Suites(2012)
So, if you want a test plan then use option 1. If you want to simply run queries, use option 2 (or use team explorer/web access to do the same thing)
Have Group Tester and Provider
Group PROVIDER only edit Bug and not access to Test Case related.
Group TESTER Only CReate and Edit BUG and Test Plan and TEst Case.
how in TFS 2012 to remove access to a PROVIDER group of test cases item?
Thank You
First lets state up front, so as to get it out of the way, that this is ridiculously dysfunctional behaviour. If you don't trust people don't give them permission at all.
That out of the way you can kind of hack this by customising the process template. If you edit the Bug to restrict the permission that can move from state "" to "New" then anyone else will get an error. This works for any work item type.
For Test Suit and Plan you need then"Create Test Plan" permission that isbin "settings|permissions" on the web access.
In TFS 2013.3 Test Suit and Test Plan are now work items so ita a little different.
I am looking for a way to track the testing status of user stories in TFS. I noticed a field named Test Phase as an option in the search query, but I don't see how to edit that field. How do I edit the Test Phase of a user story?
I cannot see Test Phase as a field on any Process Templates I have downloaded (that's not to say it's not there), so my advice might not be accurate.
You could try and customise the TFS process template to show this "Test Phase" field on the User Story - depending on what it's purpose is. You can experiment with Microsoft Excel by creating a query and adding that as a Column.
Another option, depending on what you need to track. If it is just to see what needs testing, then you can Customise the Kanban lanes on the Backlog Board. Esteban has an excellent article here describing the process.