I'm starting to dive into TFS 2012 and I have a basic understanding of the tiers and how build servers, controllers and agents work and how different build scripts can have different configurations and projects.
However, one of the things I'm struggling with is a requirement for our source control solution that says that I need to be able to prove a particular changeset or shelfset produced a particular build. That is, given a particular binary, I can point to a release changeset that generated that binary. I should also be able to point to the test changeset that was merged into the release branch. The idea here is not just a separation of duty, but validating that because the release and test changesets are identical, no code was injected into a project by a code reviewer.
I've read one blog post that talks about "Binary promotions" -- would that concept be useful in my situation? I'm having a hard time finding how this binary promotion is set up in TFS.
Deployment
Out of the box TFS doesn't really support deployments, it can deploy to 1 location on build which often is a test server (think lab management). TFS 2012 has built in support for Azure deployments, but they still happen at the end of a build and the build artifacts cannot be automatically deployed to a new location.
You could modify the build template to allow to release to different locations, but that would still be a fresh build for every environment and not true binary promotions.
TFS does, however, have a concept of build quality and actually fires off events when this quality is changed. TFS Deployer is a 3rd party tool that hooks into the quality change event and can execute powershell scripts. This means with a simple change of a dropdown value you can automatically kickoff a script that releases to any environment you want. You can customize the build quality list (per team collection) to be a list of environments (dev, uat, staging, production etc) which the script then figures out where to release the specific build to.
VS2012 also has some nice improvements to web deploy which means deployment configurations are stored in source control with the project, which in theory means they'll be available in the drop folder for TFS Deployer to make use of.
I don't believe TFS keeps a history of build qualities, which means you can't really use the build quality history to maintain a list of what is deployed to which environment. You could fairly easily record this information as part of the deployment script though. Or at the very least add a custom summary node to the build with information about the release.
TFS2012 does have the ability to mark a build as deployed as part of the Azure deployment functionality, you mark tfs deployer builds as deployed using a script but it doesn't feel very useful.
Octopus Deploy is another project that's worth checking out, and could be used instead of TFS Deployer if your build template creates NuGet packages. It requires a bit more control over the production hardware as you need to install agents on each environment to handle releases, but it solves a lot of other issues with deployment.
Versioning
Once you have a nice consistent way of automatically releasing that people don't bypass, you can look at enhancing the build template to inject the build version, or changeset number as the assembly version for anything built as part of that automated build. There's a number of different ways to do it and plenty of blog posts and tools to help you achieve that.
Alternatively you could just use automatic assembly versioning ([assembly: AssemblyVersion("1.0.*")]) to give you the date/time the build occurred, which ends up like 1.0.1234.123 where 1234 is something like the days since jan 1st 2000, and 123 is the minutes since midnight (my specifics may be wrong here).
If you're deploying websites, then I highly recommend injecting the current build version into the html somewhere. This way you can check what version a website is running without needing access to the bin directory. It can also be appended as a querystring to css/js file imports to ensure no browser caching occurs between versions.
Thoughts
Personally I'm hoping Microsoft realise that the xaml build workflows are trying to do too much and that they split the different concerns (build, test, deployment...) into different scriptable parts. Of course that would not be until the next major release of TFS which is years away. Although with Team Foundation Service they are trying to iterate a lot quicker, so they may actually extend the Azure deployment stuff into something more useful in the nearer future.
I would like to hear the best practices or know how people perform the following task in TFS 2008.
I am intending on using TFS for building and storing web applications projects. Sometimes these projects can contain 100's of files (*.cs, *.acsx etc)
During the lifetime of the website, a small bug will get raised resulting in say a stylesheet change, and a change to default.aspx.cs for example.
On checking in these changes to TFS, and automated build would be triggered (great!), however for deploying the changes to the target production machine, I only need to deploy for example:
style.css
default.asx
MyWebApplications.dll
So my question is, can MSBuild be customized to generate a "code pack" of only the files which require deploying to the production server based on the changeset which cause the re-build?
You are probably going to have a hard time getting MSBuild itself to do this, but the ideal tool to use in your situation is the Web Deployment Tool, aka MSDeploy. With this tool you can tell it to deploy the changes to the target website. It will determine only the changed files and then just deploy those. Also you can perform customization to the deployment and a whole bunch of other stuff. It's a really great tool.
We are a mostly MS shop at work doing .NET LOB development. We also use MS Dynamics for our CRM app... all the devs are currently using VS/SQL Server 2008. We also use VSS, but everyone hates it at work and that is quickly on its way out.
We are begining our initiative for TDD implementation across the team (~dozen ppl). I've gotten TeamCity setup and have my first automated builds running succesfully using the 2008 sln builder and also using SVN that a co-worker had setup who is doing the source control analysis. When demoing to managament, I think they started to buy into my snake oil and threw out the suggestions of looking into TFS.
This threw a wrench in what I had planned for our TDD architecture; In a good way though, because I had always assumed that TFS was just too expensive and not worth it for our team (and i've seen the same in other shops i've worked at / know of). I do feel like MS is years behind in the TDD/CI area and that the third party products were probably much better and more mature... I still need to do a lot of research, but I figured I'd come here to see if anyone has actually used both systems.
I realize the TFS encompasses a lot more then just a build server... but I didn't want to make this too broad of a question at least on purpose. What are the practical pros/cons of using TFS/TFB instead of TeamCity - e.g. which benefits would we lose/gain? Has anyone here actually used both systems (TFS for TDD/CI and TeamCity/SVN) and can speak from practical standpoint?
I've done some searching on this topic, and one post I found here on SO mentioned that the cons of TFB was it only supported MSBuild. I was planning on using FinalBuilder with TeamCity; and it appears it also supports TFS as well...
Thanks for any advice
EDIT: Has anyone used TFS as their Build/CI server and can tell of success/failure stories?
We are a small development shop, and decided that Team Foundation Server carries too much overhead for us. We used to write custom MSBuild scripts to run from the command line, but after we discovered TeamCity, we moved our entire build process over to it.
We've found TeamCity to be easy to use and configure, and JetBrains provides excellent support and documentation. They are also on a much faster release and update cycle than Microsoft.
Their support for SVN source control is excellent, and we like the fact that they support both MSTest and NUnit for unit testing.
We also liked the fact that the TeamCity Professional edition was free, so we could evaluate it to see if it worked for us. We haven't hit the number of project configurations (20) that would require us to upgrade to the Enterprise edition.
This question has a lot of good answers about TeamCity. It does not compare to TFS but it might shed some light on TeamCity for you.
I have used both, and I have had success with both, but TeamCity was so much easier. TeamCity was a breeze to set up and configure. TFS was not. TeamCity is rock solid, it's easy to maintain and it just plain works. The developers at JetBrains have done a great job responding to the community. They get a release out every 6 to 8 months that adds real value. TFS is on a 2 year or more cycle.
TeamCity gives you more choice in how you build and what source control you use. It's not all in one, but that's sometimes a good thing. It's got a good set of extension points as well. We have also been really happy with the agent model it has.
I've gone through 3 absolutely painles upgrades in TeamCity. The one TFS upgrade we did took our build and source control down for 3 days. I'm the admin for TeamCity on our project and it takes up a couple of hours a month. TFS took a couple of days a week.
TeamCity + SVN + VisualSVN has been the smoothest environment I have ever worked in. TFS was generally smooth on the day to day, but only if someone was there keeping it running.
Hope that helps
The benefits of TFS are one integrated environment that is supported by Microsoft. I personally do not like TFS for source control and have had a number of issues with it. It is clunky, however it had the benefit of having VS integration (which is also available in VisualSVN, but is not as robust).
Personally, I think you would be much better off using SVN/TeamCity. It is just easier to work with and behaves more as you would expect. As with most open source software, both are constantly evolving and will always have the latest and greatest feature before Microsoft. The integration between the 2 is really good and I have found no fatal flaws in the system. I constantly push to go this route in my current company (we use TFS), as I believe it is a much better workflow. As an added benefit, it is significantly cheaper than going the TFS route.
I have also used FinalBuilder with TFS - my question there is what are you really buying with FinalBuilder that you can't do with NANT/MSBuild? The answer at my shop is unfortunately very little IMO.
First off, see this post:
SVN vs. Team Foundation Server
As to your question about which environment better fosters TDD and such, my two cents is that the build management system matters much less than what's in the build file itself. Your Ant or MSBuild file should have the targets that do your testing. With MSBuild or Ant, you don't have to use MS's test suite. You can still use nUnit or whatever else you want. That means it doesn't matter if TFS is calling your MSBuild file, or if CruiseControl is, or if TeamCity is. The smarts are all in the build file and the tools you integrate with it.
My personal choice is not to get locked down into TFS's way of doing things, since you have a lot more freedom for a lot less cost with the wealth open-source testing tools that are out there. TFS is about to receive a major upgrade, as well. If you are going to go with TFS, my advice is to at least wait until 2010 is released. Concentrate on making your MSBuild files as good as they can be right now.
That being said, I must admit that TFS has one of the nicest build systems out there (2005 was terrible, 2008 was nice). Being able to easily customize notifications and the release process all inside .NET code was pretty cool -- you had a lot more central control over build and release policy than we did with CruiseControl.NET.
So I've used TFS and SVN/CCNet. I can't speak much to TeamCity. But IMO a build management system should be fairly agnostic to what is being built and how it's being built. For us, the extra control in the release management process that TFS brought us just wasn't enough of a bonus for us to justify the greatly increased administrative effort of a fully integrated TFS solution. Nor was it enough to justify the extra per-license cost of TFS, which can be significant.
The old TFS Build was XAML based and very cumbersome and and not nice to work with. That said, the new TFS 2015 build system is leaps and bounds better, and is script based with lots of web hooks and 3rd party integrations; very similar to Team City. Also, TFS now supports Git, so you are no longer confined to using Team Foundation Version Control (TFVC). Also, with TFS you can use your own on-prem installation, or can take advantage of a hosted solution through visualstudio.com. TFS is great because it's one completely integrated environment (work items, planning, builds, tests, deployments), whereas Team City is just a build solution. When this question was originally asked in 2010 I would've recommended Team City hands down. Now though, the 2 are very competitive. I would say that it would maybe boil down to if you want an all-in-one solution, then go with TFS, but if you are looking for purely just a build system, then Team City.
Comparing TeamCity to Visual Studio Team Services (the latest cloud-based offering from Microsoft):
Both work great for implementing a continuous integration process
TeamCity is more mature and everything just works.
Visual Studio Team Services by contrast is constantly evolving to catch up with TeamCity and some things just don't work well (e.g. try triggering builds based on paths that have changes from Git - the documentation is weak and the feature itself just doesn't work (as of August 2016))
Visual Studio Team Services makes it easy to have only cloud-based agents running your build (the downside however is that each has to do a clean pull of your repository for each build which may add a minute or more to the build). Both can also support local build agents which do not need to wipe the working directory for each fresh build.
But in either case I would highly recommend you also look at CakeBuild which moves most of the configuration information about how to do a build out of the CI system and into C# code that is in your Git repository along with all your other source code. With CakeBuild you can run the same build locally as you will run in the CI system and if you need to go back a month to build a specific version you have the source code and the build script to do it.
With CakeBuild in place you are now free to easily switch between TeamCity and Visual Studio Team Services.
The only downside to CakeBuild is that all your build steps are bundled into a single task in the CI system which makes reporting slightly less nice and may involve some extra work to get the test results out into a format that the CI reporting system can use.
MS is years behind in the TDD/CI area
Being one who has TDD'd for 4 years now you are correct. MS is still not even promoting it nor do they offer tools that work well with the TDD flow.
Don't get stuck dealing with Visual Studio for any kind of automation, source control, or agile workflow period (stop using TFS please!!). That stuff even though they say is "new" is monolithic and always comes with weird issues and bloat. It is always painful.
I've used Team City and it's simply amazing, things work, it's designed for usability, and it's simply designed well and compatible with most all test tools, etc. Fine use Visual Studio for code, nothing else. Look for external and open source tools to help build a better CI. The "you can do everything right in VS" sell is not selling, and it's not working. People nowdays are used to and always combining different tools from the outside to get things done. Relying on all MS toolsets is just not the way to go IMO for .NET. MS likes to sell "hey you can just do everything right here". But you end up with nothing but pain when you go that route and drink their koolade (TFS, MS Fakes, etc.).
If you plan on doing TDD, you definitely don't want to be using all MS tools. You'll either be pushed down "their way" of doing things which is often proprietary and/or bloated when you try to TDD with their tools or be totally restrictive. For TDD you need to be able to have some flexibility and choices when you decide to layer in different test frameworks, assertion libraries, etc.
Add Octopus on top of Team City, and it's stellar...you will simply fall in love with it as developer or for anyone doing DevOps.
Stop trying to rely on Microsofts continued failure at agile tool offerings
Start looking outside the box and try new things is what I keep repeating to the .NET world, me being a .NET developer in the past and who has tried new things outside the MS world.
I am told TFS can accept data on build/test metrics from 3rd party continuous integration tools. Does anyone know how this works or have any good links for me? My google-fu seems weak today and I cannot find any info on this. We would like to have a short powershell script or app run at the end of the build and send all known metrics up to TFS so it could show up in certain reports. I actually would like things that (I think) should already have space for in the data warehouse for TFS BUild Server, but I will be using CC.NET. I am thinking build name, result (Pass/Fail), Number of Warnings, Number of Errors, Time, UnitTests Run, UnitTests Passed, Code Coverage, FxCop resultsThanks.
I'm afraid that there is not a ready made integration that does this yet. The plug-in that links CC.NET to TFS is available over at CodePlex but this just lets CC.NET use TFS for version control and doesn't allow the results of the builds to be published back into TFS.
To get the data into TFS from CruiseControl.NET you have a couple of options. You could write your own custom TFS Data Warehouse Adapter which is complex but ultimately flexible or you could use a combination of the Team Build API and a little bit of vodoo to push data in to the TFS Build store that would also get pushed into the TFS Data warehouse. However, this would be limited in TFS2008 as you would only be able to push data about the build and the unit tests but not things like Code Coverage.
That said, pushing data from CC.NET to TFS is something that I originally wanted to do. However in TFS2008 the built in build system was so good that I switched from using CC.NET to trigger the builds to using TFS to trigger and manage the build. This had the advantage that all the stats stuff was taken care of automatically (and the built in UI in Visual Studio). Because I moved to TFS2008 I then lost the motivation to get the CC.NET stuff built.
If anyone wanted to contribute a TFS build result publishing feature to the CC.NET integration then feel free to join the project on CodePlex - I would love to have any help going.
This seems like it should be obvious so maybe I am not understanding something. I have TFS setup. My web apps code is in source control. I can check-in/out and create builds but I don't quite understand what the recommended method for publishing the code to the production environment is? I was looking for something like "publish release to web server" but can't find anything. I think my larger problem is that I can't find a good work-flow diagram for how a team is supposed to move from editing code to testing and then to deploying to the production servers all within the TFS system. If anyone has any thoughts or links to good walkthroughs on publishing a website from within TFS that would be fantastic.
For the workflow I would suggest the code promotion model.
Within TFS you have the idea of build quality. SO when you initially check code in the build will kick off with a quality of say "ready for test" if the build passes the testers can decide when they want to take a build for testing, when they have tested a build they can mark it as "ready for deployment", you can then use that realeas to deploy.
Unless things have changed in TFS 2010 you have to roll your own implementation for deployment. We used MS build to build MSIs using Wix, then had the testers test the deployment process for us.
Have a look at the following msbuild extensions to help with deploying to test/live environments:
http://msbuildtasks.tigris.org/
http://msbuildextensionpack.codeplex.com/
I would really recommend you spend a little time on your build process and get it water tight, there is nothing more embarrassing that deploying a release to live that doesn't work and the client loosing faith over it.
I don't believe there is a built-in mechanism in Team Build to publish websites, however this can be achieved by overriding AfterDropBuild or AfterEndToEndIteration in your build definition to include necessary tasks for copying the build output to your web servers.