I've seen the post about disabling work item creation on all failed builds, but I'd like to have TFS only create a work item on the first failure. We have a very complicated legacy system that involves VB6 COM components and frequently have build failures on the build server that track back to some funkiness VB6 does with binary files (frx, ctl, etc. -- if you haven't had to deal with that in a while, you don't want to). The only way to resolve those issues is to try to make updates on a developer machine, then check in the files and run the build again (since the build doesn't fail on the dev machine). So we may have three or four (or more) failed builds before we get a success, which means we'll have three or four work items to close out.
Ideally, I'd like to have the following:
Joe checks in a change that causes the build to fail
A work item gets created and assigned to Joe
Joe checks in another change and the build still fails
No additional work item creation
Joe checks in a change the build succeeds
The work item assigned to Joe in step 2 above gets marked as Closed
But I'd be happy with just steps 1 through 4.
How would you determine that the second failed build was related to the first one, since there's an additional check-in involved? What happens if the next check-in is actually additional code committed by another developer - you'd want them to know their code broke the build, or that it's still broken, even though according to your steps, nothing would be triggered.
You'd either need to find a way to link the builds - for example, track who the auto-work-item is assigned to and then not create another work-item for checkins from that developer until there's a successful build, and maybe you could somehow queue up the builds for the other developers. I'm not really sure how you'd do it.
Does this move you in the right direction?
Related
I have a custom TFS Build template that includes a procedure that runs a process that involves getting some data from a server and checking it into TFS before moving onto the main build process.
The steps are as follows:
Set build number
Run our custom script - get data, check into TFS
Initialize environment
Get sources from Team Foundation Version Control
Associate the changesets that occurred since the last good build
Compile, Test and Publish
The issue I appear to be having is that all change sets since the last successful build are included, except, the change set associated with the script run at step 2.
Does anyone know what could be going on here? my guess is that the logic that is looking for the change sets since last build is using a cut off that is set at the moment the build is requested (hence why the last change set is ignored) but this is just an uneducated guess.
Thanks in advance for your help.
Trying the steps below:
Edit your build definition and go to "Process" tab.
Enter "T" under "Get version" option.
Queue the build and check the result.
I would have two builds chained: the first build generates and check-in the files the second simply pulls them down.
We just started using Visual Studio Release Management for one of our projects, and we're already having some problems with how we are doing things.
For now, we've created a single release stage, which is responsible for deploying our build artifacts to a dedicated virtual machine for testing. We intend to use this machine to run our integration tests later on.
Right now, we have a gated checkin build process: each checkin fires all the unit tests and we configured the release trigger to happen on this build also. At first, it seemed plausible that, after each checkin, the project was deployed and the integration tests were executed. We noticed that all released builds were polluting the console on Release Management, and that all builds were being marked as "Retain Indefinitely" and our drop folder location was growing fast (after seeing that, it makes sense that the tool automatically does this, since one could promote any build to another stage and the artifacts need to be persisted).
The question then is: what are we doing wrong? I've been thinking about this and it really does not make any sense to "release" every checkin. We should probably be starting this release process when a sprint ends, a point that can be considered a "release candidate".
If we do that though, how and when would we run our automated integration tests? I mean, a deployment process is required for running those in our case, and if we try to use other means to achieve that (like the LabTemplate build process) we will end up duplicating deployment code.
What is the best approach here?
It's tough to say without being inside your organization and looking at how you do things, but I'll take a stab.
First, I generally avoid gated checkin builds unless there's a frequent problem with broken builds. If broken builds aren't a pain point, don't use gated checkin. Why? Simple: If your build/test process takes 10 minutes to run, that's 10 minutes that I have to wait to know whether I can keep working, or if I'm going to get my changes kicked back out at me. It discourages small, frequent checkins and encourages giant, contextless checkins.
It's also 10 minutes that Developer B has to wait to grab Developer A's latest changes. If Developer B needs that checkin to keep working, that's wasted time. Trust your CI process to catch a broken build and your developers to take responsibility and fix them on the rare occasions when they occur.
It's more appropriate (depending on your branching strategy) to do a gated checkin against your trunk, and then CI builds against your dev/feature branches. Of course, that opens up the whole "how do I build once/deploy many when I have multiple branches?" can of worms. :)
If your integration tests are slow and require a deployment to succeed, they're probably not good candidates to run as part of CI. Have a CI/gated checkin build that just:
Builds
Runs fast unit tests
Runs high-priority, non-deployment-based integration tests
Then, have a second build (either scheduled, or rolling) that actually deploys and runs the whole test suite. You can schedule it according to your tastes -- I usually go with one at noon (or whatever passes for "lunch break" among the team), and one at midnight. That way you get a tested build from the morning's work, and one from the afternoon's work.
Using the Release Default Template, you can target your scheduled builds to just go as far as your "dev" (/test/integration/whatever you call it) stage. When you're ready to actually release a build, you can kick off a new release using that specific build that targets Production and let it go through all your stages normally.
Don't get tripped up on the 'Release' word. In MS Release Management (RM), creating a Release does not necessarily mean you will have this code delivered to your customers / not even that it has the quality to move out of dev. It only means you are putting a version of the code on your Release Path. This version/release can stop right in the first stage and that is ok.
Let's say you have a Release Path consisting of Dev, QA, Prod. In the course of a month, you may end up releasing 100 times in Dev, but only 5 times in QA and once in Prod.
You should drive to get each check-in deployed and integration tested. If tests takes a long time, only do the minimal during (gated or not) check-in (for example, unit tests + deployment), and the rest in your second stage of Release Path (which should be automatically triggered after first stage completes). It does not matter if second stage takes a long time. As a dev, check-in, once build completes successfully (and first stage), expect the rest to go smoothly and continue on your next task. (Note that only result of the first stage impacts your TFS build).
Most of the time, deployment and rest will run fine and so there won't be any impact to dev. Every now and then, you will have a failure in first stage, now the dev will interrupt his new work and get a resolution asap.
As for the issue that every build is kept indefinitely, for the time being, that is a side effect of RM. Current customers need to do the clean up manually (or script it). In the coming releases, a new retention policy for releases/builds will be put in place to improve this. This has not been worked on yet, but the intention would be to, for example, instruct RM to keep all releases that went to Prod, keep only the last 5 that went to QA and keep only the last 2 that went to Dev.
This is not a simple question, so also the answer must be articulated.
First of all, you will never keep all of your builds; the older a build, the less interesting to anyone; a build that doesn't get deployed in production is overtaken by builds that reaches that stage.
A team must agree on the criteria that makes a build interesting to keep around and how long to keep it. Define a policy for builds shipped to production or customers: how long do you support them? Until the next release, until the following one, for five years? Potentially shippable builds, still not in your customers' hands, are superseded by newer, so you can use a numeric or a temporal criteria (TFS implements only the first, as the second is more error-prone). Often you have more than one shippable build, when you want a safety net option and being able select from a pool which deliver (the one with more manageable bugs).
The TFS "Retain Indefinitely" should be used when you cannot automate the previous criteria, so you switch to a manually implemented policy. Indefinitely is not forever, means for an unknown time interval.
When I have defined a Gated Build, when somebody checks in code the Integration Build field of a work item changes to the Gated Build number (if the developer associates his check in with work items, of course). Once a CI build is triggered this field changes to the CI build number.
My question is: Is there any way of not changing the Integration Build field of a work item once a Gated Build is triggered?
EDIT
Let me be more clear about how we work.
We have several work itens (some are user stories and some are bugs). When a developer checks in code he or she associates his/her check in with those user stories that gets the Resolved state and a "Gated x.x.x.x" in the integration build field. We never test gated builds. Instead, every night we manually trigger a build and those work itens gets updated again, but this time with a "Release x.x.x.x" in the integration build field. In the next day we test those work itens but the process continues and developers keep check in more US or Bugs (that will have the Gated ...).
Sometime we get confused and we test work itens that should not be tested because they are in the "Gated state".
Even if we have branches that will not solve our problem because the developer associates a check in with work itens and we cant change that.
We do not test gated builds because our QA team is small. The dev team have 20 developers and the QA team have only 2. The process of deploy the application takes about 10 minutes and it can be a pain to wait 10 minutes on every developer check in. Also changing the code while we are testing is never a good idea because it can mess up with our test.
Somebody can think that our process is wrong and suggest a new approach. This will be very welcome, but what we do is working very well besides that small issue.
Jenkins says a build succeeded or failed, but can it identify the exact commit (and author!) that caused a build to fail?
This issue would seem to indicate no.
Edit: From my exchange with Pace:
What I see is "include culprits", which is everyone since the last
build. I don't want that. I want THE culprit, with Jenkins doing the
binary search. If Jenkins does two builds 10 commits apart, I don't
want 10 possible culprits, I want it to find the one.
I haven't yet heard how to do that.
That page was talking about the "find bugs" plugin, not the normal build cycle. Depending on how things are setup Jenkins can identify the exact commit and author that caused a failure. If Jenkins has the appropriate source control plugins installed and is configured to know about the repository the build is tied to then for every build it will list the changes since the last build.
In addition, Jenkins has the capability in many of its reporting plugins to blame the faulty committer. It can, for example, send an e-mail notification on a failed build to the developer that made the faulty commit.
However, many setups make it difficult for Jenkins to know. For example, if Jenkins is configured for daily builds then there are likely many commits which could have caused the issue. It's also possible that Jenkins isn't configured to know about the source control repository, or there is no source control repository. All of these issues could cause Jenkins to be unable to identify the build breaker.
Specifically for e-mailing faulty committers you can use the email-ext plugin which has options to send e-mails to everyone that committed since the last successful build.
For a humorous take on this subject check out this approach.
I think what you're asking for is impossible in some cases. Determining who the culprit is requires insight into conflict resolution that only a human can decide. Even still, sometimes a manager has to be involved in order to arbitrate. Say for instance you get 3 commits (A,B,C) that depend on a preexisting definition. However, another commit (D) modifies the behavior of that function. Which do you revert? Perhaps it's the business plan to keep A,B,C as is and return D to its original state. The opposite, modifying A,B,C to adapt to the changes of D, is also possible.
In the cases where a machine can handle the arbitration, it is the responsibility of unit tests, and static analyzers, to determine the culprit (although still imperfect). Static analyzers sometimes have built in features that email the person who committed a violation. Unit tests can be written that notify teams or team members responsible for a failed test. Both could work in the same way that identifies who was the last committer on a particular line that failed. Still, if it is a problem with linking, then perhaps some members should be associated with the particular makefile.
I've seen the question related to the error message you get from TFS when a workspace is already mapped. The accepted answer for removing the workspace is alright as a workaround, but it's already getting tedious to run a delete command each time this error occurs.
What do I need to change in order to get out of having to use this workaround? I've got two builds (continuous integration and nightly deploy), and need to add at least one more build type. I followed this URL to see if there was a possible resolution there, but I'm not sure I understand it completely.
I am not sure how this is accomplished in TFS 2010, as I have not gotten to work with Team Build in 2010, yet. In 2008, though, if you expand the Builds node in the Team Project and right-right click on either of the builds, you will see a "Manage Build Agents..." option. Click into that, and it will bring up a dialog. One of the things on that dialog is an option called "Working Directory". Do you have the same hard-coded path in both of them?
By default, when you create a new build definition, it provides a calculated folder for this value. This is where the build agent will do the checkout from TFS for the build attempt. The default value is, $(Temp)\$(BuildDefinitionPath), I believe (I am not connected to TFS at the moment).
The article you link to is basically saying that you should include either that $(BuildDefinitionPath) value or the $(BuildDefinitionID) value as part of that path in that dialog so that the two builds do not try to use the same workspace. Changing the working folder to include one of those values should resolve your issue, going forward.