I am told TFS can accept data on build/test metrics from 3rd party continuous integration tools. Does anyone know how this works or have any good links for me? My google-fu seems weak today and I cannot find any info on this. We would like to have a short powershell script or app run at the end of the build and send all known metrics up to TFS so it could show up in certain reports. I actually would like things that (I think) should already have space for in the data warehouse for TFS BUild Server, but I will be using CC.NET. I am thinking build name, result (Pass/Fail), Number of Warnings, Number of Errors, Time, UnitTests Run, UnitTests Passed, Code Coverage, FxCop resultsThanks.
I'm afraid that there is not a ready made integration that does this yet. The plug-in that links CC.NET to TFS is available over at CodePlex but this just lets CC.NET use TFS for version control and doesn't allow the results of the builds to be published back into TFS.
To get the data into TFS from CruiseControl.NET you have a couple of options. You could write your own custom TFS Data Warehouse Adapter which is complex but ultimately flexible or you could use a combination of the Team Build API and a little bit of vodoo to push data in to the TFS Build store that would also get pushed into the TFS Data warehouse. However, this would be limited in TFS2008 as you would only be able to push data about the build and the unit tests but not things like Code Coverage.
That said, pushing data from CC.NET to TFS is something that I originally wanted to do. However in TFS2008 the built in build system was so good that I switched from using CC.NET to trigger the builds to using TFS to trigger and manage the build. This had the advantage that all the stats stuff was taken care of automatically (and the built in UI in Visual Studio). Because I moved to TFS2008 I then lost the motivation to get the CC.NET stuff built.
If anyone wanted to contribute a TFS build result publishing feature to the CC.NET integration then feel free to join the project on CodePlex - I would love to have any help going.
Related
I've been tasked with migrating a build from TFS to VSTS, I've used TFS from a developer point of view, however I've never created builds. Can anyone give me any advice where to start, good websites, tutorials, tools that might help, anything that might catch me out etc, basically my knowledge on this is very limited so anything would help. I've no doubt there must be somebody out there who can say we've done this, here is what we did...the new VSTS build will need to be a copy of the TFS build however some analysis needs to be done as the build process is old and certain parts may not be necessary. Thanks in advance!
Migrate New Vnext build definition from TFS to VSTS
If you want to template your builds or move them from TFS to VSTS, you could also use the REST API's to perform this. Details please refer this blog: HOW TO MOVE BUILD DEFINITIONS IN TFS TO OTHER PROJECTS USING THE REST API
Another example shown here. It shows how to get the JSON response of a build definition and again using the same reference to make a new one in the project you desire.
Migrate Old XAML to New Vnext build definition from TFS to VSTS
Unfortunately, there will not be any automated conversion processes. The new builds are based on a different architecture and run on a completely different system. And you can use both the new builds and agents alongside your XAML builds, controllers, and agents.
If you have heavily customized XAML builds and custom activities, you
can continue using those builds until you are ready to port your
business logic into scripts that can run in the new builds.
You should also take a look at these series of articles by jessehouwing devoted specifically to the case like yours.
About how to create a build, you could refer this tutorial from MSDN.
So here's the setup we have right now
1) We use TFS2013 for our code repository
2) We use TeamCity to do our builds
3) we have our tests created in CodedUI
From what I've read online is that for the CodedUI tests to run automatically, they need a build to reference. However, since we are using Team City to do the builds, our Build Explorer in TFS is empty. Switching off of Team City for builds is a no, so I'm trying to find a way to have the builds appear in the Build Explorer so I can have the automated tests run via MTM.
TL;DR: I need to get the Team City builds to appear in the TFS Build Explorer.
I saw this question, but it was for 2010, so I'm hoping it's changed since How do I make a TeamCity build appear in the TFS Build Explorer?
It can be done using the API. You could use the following blog post as a starting point to create an app that records in TFS the results from your TeamCity build. Then you could run your app as part of your TeamCity build.
http://blogs.msdn.com/b/jpricket/archive/2010/02/23/creating-fake-builds-in-tfs-build-2010.aspx
Update: There are however downsides. Some capabilities of ALM are only available if the build is done in TFS. You loose Test Impact Analysis, relationships between builds and Work Items, identification of Verifiable Bug fixes, roll up for release management of release contents.
TF Build is more than just orchestrating a compilation, it is the glue that knits your ALM strategy together and you will not be able to amortize the realm value in TFS without builds in TF Build or a whole lot of custom work.
Sorry, it's not possible. Team Explorer is designed to show you Team Builds.
I'm starting to dive into TFS 2012 and I have a basic understanding of the tiers and how build servers, controllers and agents work and how different build scripts can have different configurations and projects.
However, one of the things I'm struggling with is a requirement for our source control solution that says that I need to be able to prove a particular changeset or shelfset produced a particular build. That is, given a particular binary, I can point to a release changeset that generated that binary. I should also be able to point to the test changeset that was merged into the release branch. The idea here is not just a separation of duty, but validating that because the release and test changesets are identical, no code was injected into a project by a code reviewer.
I've read one blog post that talks about "Binary promotions" -- would that concept be useful in my situation? I'm having a hard time finding how this binary promotion is set up in TFS.
Deployment
Out of the box TFS doesn't really support deployments, it can deploy to 1 location on build which often is a test server (think lab management). TFS 2012 has built in support for Azure deployments, but they still happen at the end of a build and the build artifacts cannot be automatically deployed to a new location.
You could modify the build template to allow to release to different locations, but that would still be a fresh build for every environment and not true binary promotions.
TFS does, however, have a concept of build quality and actually fires off events when this quality is changed. TFS Deployer is a 3rd party tool that hooks into the quality change event and can execute powershell scripts. This means with a simple change of a dropdown value you can automatically kickoff a script that releases to any environment you want. You can customize the build quality list (per team collection) to be a list of environments (dev, uat, staging, production etc) which the script then figures out where to release the specific build to.
VS2012 also has some nice improvements to web deploy which means deployment configurations are stored in source control with the project, which in theory means they'll be available in the drop folder for TFS Deployer to make use of.
I don't believe TFS keeps a history of build qualities, which means you can't really use the build quality history to maintain a list of what is deployed to which environment. You could fairly easily record this information as part of the deployment script though. Or at the very least add a custom summary node to the build with information about the release.
TFS2012 does have the ability to mark a build as deployed as part of the Azure deployment functionality, you mark tfs deployer builds as deployed using a script but it doesn't feel very useful.
Octopus Deploy is another project that's worth checking out, and could be used instead of TFS Deployer if your build template creates NuGet packages. It requires a bit more control over the production hardware as you need to install agents on each environment to handle releases, but it solves a lot of other issues with deployment.
Versioning
Once you have a nice consistent way of automatically releasing that people don't bypass, you can look at enhancing the build template to inject the build version, or changeset number as the assembly version for anything built as part of that automated build. There's a number of different ways to do it and plenty of blog posts and tools to help you achieve that.
Alternatively you could just use automatic assembly versioning ([assembly: AssemblyVersion("1.0.*")]) to give you the date/time the build occurred, which ends up like 1.0.1234.123 where 1234 is something like the days since jan 1st 2000, and 123 is the minutes since midnight (my specifics may be wrong here).
If you're deploying websites, then I highly recommend injecting the current build version into the html somewhere. This way you can check what version a website is running without needing access to the bin directory. It can also be appended as a querystring to css/js file imports to ensure no browser caching occurs between versions.
Thoughts
Personally I'm hoping Microsoft realise that the xaml build workflows are trying to do too much and that they split the different concerns (build, test, deployment...) into different scriptable parts. Of course that would not be until the next major release of TFS which is years away. Although with Team Foundation Service they are trying to iterate a lot quicker, so they may actually extend the Azure deployment stuff into something more useful in the nearer future.
In 2009, there was a SO question on the same topic.
I'm wondering if later versions of Team Foundation Server are better at longer build pipelines. Refer features of Jenkins, TeamCity, ThoughtWorks' Go (my employer).
The visualizations of the build pipelines are important to me, as well as the notification about individual stages passing or failing. That and the eminent clone-ability of say a 'trunk' pipeline into one for a release branch as that branch leaps into being.
Secondly, a personal holy-grail is the CI server storing its config in the SCM that's holding the buildable thing itself, and even picking up on the creation of branches silently to provision new pipelines; Can TFS be configured to store the CI definitions/scripts in its SCM side rather than its accompanying SqlServer?
TFS build consists of three components:
The build definition - stored on the SQL server data tier.
The build workflow - a XAML file stored in the source control.
The supporting MSBuild scripts - usually contains user defined actions, also stored in the source control.
As the build progresses, you can see visualization of the build steps and you also get a different log for the main build and the MSBuild output.
The build definition in TFS is merely a collection of build settings, similar to CC.Net's config file and TeamCity's build configuration tab which both stored on the file system as well. Assuming there's a backup plan on the database you don't really need to store the build defintions on the source control, but if you must it's possible by exporting the tbl_BuildDefinition table.
The TFS Power Tools adds cloning functionality for build definitions.
There's no OOTB support for provisioning build definitions from a new branch though it's fairly feasible using the TFS-API.
Bit late to the party, but just don't bother with TFS if you want advanced build pipeline automation. It simply doesn't cut it.
I have used Jenkins and TFS both extensively. Tfs is just. pure. crap. Here's why.
No down/up stream build.
No piepline/orchestraion build. (like jenkins)
Obscure ways of adding build steps and falls back to using MsBuild.
Slow and still polls the source control.
Ties you to MsTest.
And please don't point me to "Oh look you can do everything if you write a custom activity". I'm not wasting time doing development for a closed source, sub-par platform. If I am going to contribute something, it's to a FREE. OPEN SOURCE platform.
I am working on an integration approach for integrating Rational (IBM Jazz) Quality Manager (RQM) and Requirement Composer (RRC) with TFS.
As the teams are moving from ClearCase to TFS, what we are looking at is still being able to manage the requirements in RRC and manage testing using RQM.
The flow will be something like:
Requirements are planned and detailed in RRC
Create work items in TFS connected to the Requirements in RRC
Create design and code using VS2010 and managing the version control in TFS
Creating test plans and test cases in RQM (connected to requirements in RRC)
Run test against builds in TFS
Publish test results in RQM against builds in TFS
Run reports in RRC, RQM and TFS that links up the items across the platforms.
I have started looking at TFS Integration platform. But shall like to have your guidance for an early resolution and better solution approach and how to achieve this.
UPDATE:
As I was not getting good response here, I put this on MSDN and I really got good response there. I am not deleting this post here, but give the link here:
http://social.msdn.microsoft.com/Forums/en-US/tfsintegration/thread/fcb20f89-6801-4ad1-8f46-3df68325ca8d
Please follow this.