When reviewing the Build History via the browser (servername:8080/tfs/DefaultCollection/MyProj/_Build), I find that there are numerous missing entries for builds which were triggered via gated checkins. When I filter on only that build definition, I find a handful of entries, compared to hundreds of code checkins.
All developers are required to use the gated builds, and all checkins show the telltale signs of success (NO_CI)
Using the API, I find the identical results, which is the root problem...
What would cause this behavior?
check the Retention Policy set on your build definition. By default only the most recent 10 builds will be kept.
Related
I don't understand what Promoted build really is and how it works. Can someone please explain to me like to a 10 years old kid. If you can provide some sample examples would help me a lot.
Thanks
In a typical software developing organization with CI system, there are 10's or 100's of continuous builds daily. Only one of those builds (usually the latest stable) is selected and "promoted" to be a Release Candidate (RC), which goes to the next quality gate - usually the QA department. Then, they select one of those RC's (others are dropped) and again, "promote" it to the next level - either to staging environment, validation etc. Then, finally... one of these builds is again "promoted" to be an official release.
Why is that important?
Visibility: You would want to distinguish many "regular", continuous builds from few, selected "RC" builds.
Retention: If you commit often (which is the best practice), you will likely get lot of daily builds, and would like to implement a retention policy (e.g. only keep last 100 builds or only builds from the last 7 days). You will then want to make sure promoted builds (RCs) are locked against retention. This is mostly important if you deploy binaries to customers, and may need the exact binary to reproduce an escaping bug in the future (though you still have the source code in the repository, I've seen cases where escaping bugs relate to the build process rather than the source code - due to rapid changes to the build process, or time-of-build sensitive data like digital signatures).
Permissions: you may want to prevent access to builds with "half baked" features from non-developers.
Binary Repositories: you may want to publish only meaningful builds to an external binary repository.
Builds in Jenkins can be "promoted" either manually or automatically, using plugins like Promoted Builds Plugin. You can also create your entire "promotion" workflow using pipeline scripts. Here's an example:
a "Continuous" job that polls SCM and builds on every change. It has a retention policy to keep only the last 50 builds. Access is restricted only to developers;
a "Release Candidates" job that copies artifacts from a manually selected build (using parameters). Access is allowed to QA testers;
a "Releases" jobs that copies artifacts from a manually selected RC. Access is allowed to the entire organization. Binaries are released to external/public repository.
I hope this answers your question :-)
We are using TFS and the TFS Build Service. We are considering to migrate the Build service to Jenkins but we came across some issues. According to this site, there are some things that do not work very well with the TFS and Jenkins plugins. All of them we use a lot:
Associated Change sets – Team Build automatically associates a list of change sets that are included in the build
Associated Work Items – Team Build analysis the relationships and also associates Work Items with a build. Indeed it walks the work item tree (parent) and maintains that association in the chain.
Is this still true? We have this scenario:
A developer checks in a code that fix a bug or resolve a User Story. It does that by associating his check in with the work item ID.
His check in triggers a build that will associate the work item with his changeset. For bugs, the build will update the "Integrated in Build" field with the build number. We use this field to know in witch version the bug was fixed.
Is there any way to make Jenkins behave and do what TFS build service does?
Another option is to mix the two using dummy builds on the TFS side that set the records straight and kick-off the Jenkins' builds. Some hints
How to trigger Jenkins builds remotely and to pass parameters and “Fake” a TFS Build.
This approach requires a bit of effort but has many advantages:
No big-bang, use Jenkins opportunistically
Can continue using existing builds
Having a build identifier in TFS allows you an overall monitoring and to use the Test features
I have a VSTS build definition for one of our projects that requires jenkins to build, but we still have all our other products using VSTS natively. To maintain consistency, this build definition triggers a jenkins build. We configured the build definition to not sync code as jenkins will download it (save time) and not to publish the artifacts back to the agent (i have another script for that found here). This allows developers to continue to use git as normal, and the build/release process is consistent with our other products. Along with task tracking and such.
I have TFS continuous integration set up, where every check-in triggers a build. But I also want some scripts to automatically modify specific files as part of the build process. This causes a cascade of builds, because files modified by scripts cause another build to be queued. How do I enable continuous integration, but except some check-ins from triggering the build?
UPDATE Solution is: put the ***NO_CI*** into comments when checking in.
If you look at the way a Gated Build works it bypasses any configured CI by spending ##NOCI## or something similar to the checkin comment. I would think that you could do the same to bypass.
Switch to a Gated Build for one build and see the created comment tag.
We set up TFS 2013 recently and tried to set up gated check-in. In our experiment, it correctly failed the build and rejected the bad check-in. However, both the build notification tray icon and the build tab on the TFS web access show the failure, and it is this way for all users. This will make everyone think "the" build is broken when it's just one person's "gate." It will skew metrics, too.
A) Why would this be the default behavior? It seems very counter-productive and counter-intuitive. [Or maybe this isn't the default behavior and our setup is hosed?]
B) Is there a configuration for the build tab where a rejected gated check-in/build is visible only to the person who broke it?
C) How can we make the build notifier tool ignore gated failures?
a) This is a public build. Why should it be failing. If the gated build is failing it shows a lack of care by the developers. Are they checking in any old thing? Are they running their unit tests first? It really sounds like you have a quality issue there.
That is why the gated-builds are still public builds.
b) no - Anyone can however send a Private build to the build server if they think that there may be an issue that they can't catch locally..
c) Each user can uncheck the monitoring of the gated build. I would however suggest that they should concentrate on not failing the build however rather than sweeping it under the carpet...
A filed build, even a gated one, should be the exception and not the rule...
MrHinsh answered my question from a largely philosophical standpoint, and that led me to this article:
[http://adamstephensen.com/2012/11/01/gated-checkins-mask-dysfunction/]
I agree in principle with the article. However, because we are attacking agile processes in baby steps, we still want this extra safety check for the time being.
Below isn't the exact solution we wanted, but it is a hybrid of the approach in the article and my original question. It doesn't force gated check-ins, but it allows them to be done, giving risky check-ins a safety net. Code quality beyond build-ability will be addressed with other mechanisms outside the scope of this question.
The compromise solution is that "vanilla" changes would be checked in normally (after local testing, of course), and then the CI build would kick in. Fixing broken builds would be high priority to be addressed immediately.
But anything out of the ordinary (configuration changes, referencing new/different external dependencies, or just anything a developer is unsure of) would use a private gated check-in, but not on the main build, because we don't want everyone to be tricked into thinking the build is broken upon rejections. This build definition is a clone of the main build definition, but with the addition that upon successful check-in, it chains a build of the main build definition using this:
[How to chain TFS builds?
While the main build is redundant in most cases (race conditions may apply), it solves the immediate need.
Jenkins says a build succeeded or failed, but can it identify the exact commit (and author!) that caused a build to fail?
This issue would seem to indicate no.
Edit: From my exchange with Pace:
What I see is "include culprits", which is everyone since the last
build. I don't want that. I want THE culprit, with Jenkins doing the
binary search. If Jenkins does two builds 10 commits apart, I don't
want 10 possible culprits, I want it to find the one.
I haven't yet heard how to do that.
That page was talking about the "find bugs" plugin, not the normal build cycle. Depending on how things are setup Jenkins can identify the exact commit and author that caused a failure. If Jenkins has the appropriate source control plugins installed and is configured to know about the repository the build is tied to then for every build it will list the changes since the last build.
In addition, Jenkins has the capability in many of its reporting plugins to blame the faulty committer. It can, for example, send an e-mail notification on a failed build to the developer that made the faulty commit.
However, many setups make it difficult for Jenkins to know. For example, if Jenkins is configured for daily builds then there are likely many commits which could have caused the issue. It's also possible that Jenkins isn't configured to know about the source control repository, or there is no source control repository. All of these issues could cause Jenkins to be unable to identify the build breaker.
Specifically for e-mailing faulty committers you can use the email-ext plugin which has options to send e-mails to everyone that committed since the last successful build.
For a humorous take on this subject check out this approach.
I think what you're asking for is impossible in some cases. Determining who the culprit is requires insight into conflict resolution that only a human can decide. Even still, sometimes a manager has to be involved in order to arbitrate. Say for instance you get 3 commits (A,B,C) that depend on a preexisting definition. However, another commit (D) modifies the behavior of that function. Which do you revert? Perhaps it's the business plan to keep A,B,C as is and return D to its original state. The opposite, modifying A,B,C to adapt to the changes of D, is also possible.
In the cases where a machine can handle the arbitration, it is the responsibility of unit tests, and static analyzers, to determine the culprit (although still imperfect). Static analyzers sometimes have built in features that email the person who committed a violation. Unit tests can be written that notify teams or team members responsible for a failed test. Both could work in the same way that identifies who was the last committer on a particular line that failed. Still, if it is a problem with linking, then perhaps some members should be associated with the particular makefile.