If I create a bug in TFS, is it possible to associate it with a unit test and be notified if this bug is fixed/broken?
Not exactly, but the workflow can go the other way: instead of creating a bug up-front, you can let Team Foundation Server handle the work item creation for you. If you create a continuous integration build that runs your unit tests, you can create a bug assigned to the user who submitted the changeset if that build fails (including because the unit tests fail.)
A good article on configuring continuous integration to do this is in MSDN Magazine.
A new feature in TFS 2010 is Gated Check-in, which will prevent this failing code from getting checked-in by requiring the build and test steps to complete successfully before they get committed.
There is, currently, no mechanism to do what you desire. You can force users to check in changes tagged with a work item, which you can then manually examine. Provided the user either alters a test class or creates a new one, and checks in with the fix, you can see whether it is done or not. But, there is no automagic "if this test is changed, alert me".
If you need some type of "change and alert" functionality, I would examine CodePlex and see if there are any tools of this type. It could be facilitated at the database layer, as well, but I am not sure what kind of monkeying that would take. I have not looked at the service endpoints for the latest TFS, so there may be a way to poll to see if there are changes.
Wish I had a better answer, but this is an interesting ASK, so I would consider going to Microsoft Connect for TFS and requesting the feature. It could be included in a future rev if enough people are interested in this feature.
Related
Scope: I need a possibility to re-assign, on a bugs basis, the test case again to the same tester/testers which has reported this bug.
MTM 2015 got this "Verify function" under the testing center menu "Test" - Verify bugs and by right clicking on a bug in the list.
Now when I do this, it'll start the MTM test runner with me as the assigned tester. But thats actually not what I need.
I came as far as this with my researches:
TFS 2015 hasn't any "Verify" function - only available from version 2017.1 or later (more here on section: Verify a bug, re-run tests ...)
So could anybody help me out of this (updating to TFS 2017 or later is no option for the moment)?
The simplest and effective way still is using bug workitem workflow to track the process. Even without the verify function in TFS2015, you still could link your test case in the bug workitem. You need to customize the bug work item type. Add a custom filed such as retest and add a condition (retest filed need to change to pass) during the transform from active to resolve.
Then just set "assign to" filed = work item creator to make sure the test case run again by the same tester/testers which has reported this bug.
However this will create a new test result in a new test run. It will not update the original test result. If you just want to rerun failed test cases from TFS 2015. Unfortunately ,this is not an inbuilt feature. More details please refer this issue in GitHub.
I am investigating a way to automate some of our build processes using Jenkins and HPQC. Currently, we have a process where, once a change to fix a defect has been checked in we set its status to "Fixed" and then reassign defect in HPQC from the individual developers to a team lead.
The team lead is tasked with manually deploying a build for the deliverable to the test environment and when he does this he will then update all of the defects assigned to him this way reassigning them to the test lead, who can assign them to individual testers.
I would like to automate this process where I can. Does HPQC have a web API of some kind? So that a remote system (such as a Jenkins build server) could run a post-build action script to gather a bunch of defect numbers (those included in the build) find each defect in HPQC and then update its status and owner?
There is a REST API for ALM / Quality Center, info is accessible:
http://support.openview.hp.com/selfsolve/manuals
You will have to sign up for an account with HP to access it. Ugh, troglodytes.
Search for "ALM REST API", download and read the newest guide and reference for your version of QC.
(We also use QC at my work. It's pretty damn bad. I should try and convince them to get or build something better.)
The answer above is a good one, I found the reference he mentions, but making use of that is not very intuitive, probably because I am such a newb. For my fellow unenlightened you might want to use another reference I found for how to use the reference :
http://www.consulting-bolte.de/index.php/22-hp-alm/hp-alm-rest-api/115-connect-to-hp-alm-via-java-using-rest-api
The key piece of information for me was that inside all of these class files they give you in the "Example Applications" folder there is a reference to a package :
package org.hp.qc.web.restapi.docexamples.docexamples.infrastructure;
This is just another name for all the files located in the guide in the "infrastructure" subfolder. You do not need to go find this out on github or something.
I want to be able to see the bugs created on MTM in TFS without having to manually create them again. If anybody has any insight on how to automate this process please help.
Thanks
When you are working with MTM you are either executing Test Cases or doing Exploratory testing when you are creating bugs.
If you are executing test cases MTM will automatically associate any Bugs created with the Test Case in context. However, as your Test Case may test multiple PBI's the tool does not know which PBI is in context.
Note: I am not sure if, when using the 'Add Requirement' option to create a Test Suit that I linked to your PBI if the link to the PBI is created.
If you are doing exploratory testing, where you don't have a Test Case, then you can choose to "Explore a requirement". In this case both Bugs and Test Cases created are associated with the PBI in context.
If you run the reporting services Backlog Overview report you will see the number of tests passing and failing against PBI as this report follows the indirect links.
In TFS there is an option in Visual Studio to "Keep Items checked out when checking in" what is the purpose of giving such an option?
I am trying to build a reporting tool to find out the list of files that are checked out, so in the case the user has set the above option would my report be accurate, since the developer can always claim that "Hey, all my changes are checked in"
How do i reflect those kind of checkout in my report?
I use this feature when part of what I'm working on is needed by another developer, but I haven't actually finished the whole task.
Your report would still be accurate, as it's a true reflection of the system - the code really is checked out, and the developer is probably still working on the file. The only way you can truly know if all changes are checked in is by comparing the current version from source control with each developer's locally checked out copy, which isn't going to be feasible for reporting purposes, and is probably of limited value.
In a modern CI environment it's very common to commit changes and simply keep changing the very same modules.On the other hand, once a milestone is reached, the developer will simply commit the changes & start working on something else.So, I think it's very natural for TFS to provide with this configurable flag.Another major feature in the TFS ecosystem is gated-checkin: is this mode the commit is shelved, built & committed once all that has succeeded. If it weren't for this option, the developer would have to stand still & wait until the process has finished.I would disagree with a developer stating "Hey, all my changes are checked in": our principle in the team is that everything that is checked out is being developed, anything else is/should be either shelved or committed.You could consider a rule that all pending changes of each developer get shelved at the end of the working day. I am personally against any such measures, but would surely examine them as a option if my devs wouldn't adapt to the principle.If you would agree to the above, your second question would become rather obsolete: In my opinion there aren't several 'kinds' of checkouts.
Does anyone know how to enforce that all tests pass before a TFS check in?
TFS provides so-called Gated check-ins. That means: the complete build (which includes the test runs) is run and has to pass, otherwise the check-in is rejected. It's quite performance-demanding and somewhat time-consuming, when working in a larger team, but otherwise works like a charm.
HTH.
You can use the default "Testing checkin policy".
If you want to create your own testing policy, you can use this.