We have a very long running build and we are trying to break it down into incremental builds. The code is c#/.net with jenkins as the build manager and TFS (visualstudio online as the source control)
So the architecture would look like:
Area 1 (1 or more solutions)
Area 2 (1 or more solutions)
Area 3 (1 or more solutions)
So the idea is that if code is checked in in Area 1, the associated solutions for only that area are built and the tests for that area are run.
We would then also have a nightly build that builds everything.
My question is, is it possible for Jenkins/TFS to detect which area of code has been checked in and pass a variable (or list of variables) into the build script which tells it which items to build/test?
Visual Studio Team Service (Visual Studio Online) supports Git and Team Foundation Version Control(TFVC).
If you are using TFVC for version control, it is possible for VSTS to detect which area of code has been checked in. However, it cannot tell Jenkins which area is updated and to build. So the alternative way would be create three Jenkins jobs for the three area and triggered by three VSTS service hooks.
For example, you have three area in "Test" project, so the folder structure will be:
$/Test
---/Area1
------/Porjects&Solutions
---/Area2
------/Porjects&Solutions
---/Area3
------/Porjects&Solutions
You can create a Jenkins Service Hooks for Area1 as following to trigger the Jenkins job for Area1:
And in the Jenkins, update the setting for Area1 job to just get the source file for Area1:
Now, only when the code in Area1 is checked in, the Jenkins job for Area1 can be triggered. And repeat the steps above to configure the settings for Area2 and Area3.
If you are using Git for version control. There isn't any way to trigger the build by the area folders in the Git repository, you need to separate the three area and put them into three Git repository.
Related
I maintain our build process on our TFS 2012 system.
Up till now I've been making changes locally to a solution that generates our custom activity DLLs. These DLLs are checked into TFS and are used by the build controllers and agents for the custom activities in our custom build template.
For all of my code and workflow activities I have a unit test library that I can locally run to put in some base test values. While this gives me a good idea at how my custom tasks will run it is not optimal.
What I would like to be able to do is run the entire build definition. Is there a way to test the whole build process locally so that I can be sure that my changes to the workflow are acceptable before checking in and kicking off a build? Right now my builds take between 30 and 40 minutes to complete and I would like to be able to start, monitor, debug, and stop the build locally without occupying one of our agents.
Edit
Another method for me to solve this problem is to come up with some manner to execute the workflow that is the build template locally. Has anyone been able to do this?
Take a look at the Ranger Build Customization Guidance. There is a HOL for doing a lot of this. It doesn't look like it is in v2 so take a look at v1.2.
http://vsarbuildguide.codeplex.com/releases/view/87995
If I am understanding the issue correctly, one way to do this may be:
Install a build agent locally and register it with the TFS Build Controller.
Give the new agent a custom tag so it doesn't get assigned other builds from the queue
Shelve your changeset
In VS, right click the build definition you want, and select queue new build. A "Queue Build" window will open
Under the general tab, "What do you want to build", select your shelveset.
Under Parameters tab ->Advanced -> agent settings, enter your agent's tag.
Click queue.
I accomplish this using a modified version of #malexander recommendation.
Create a code branch in TFS, commit your changes there, especially the changes to your custom activity DLL's (compiled binaries, not just source).
Install a TFS Build Controller and Build Agent locally, registering the controller with your TFS server.
Configure your new local controller to use your modified custom activity DLL's, stored in your new TFS code branch. Team Explorer > Builds > Actions > Manage Build Controllers >
(select your new local controller) > Properties > Version control
path to custom assemblies
4.
Queue a new build, from an existing TFS build definition: change its controller to use your new local controller.
The new build will execute on your local controller/agent, using your modified custom activity DLL's, without interfering/restarting your production agents.
Also: since you're in your own code branch, you can commit debug versions of your custom activity DLL's, without impacting your production agents.
I have created 3 continuous integration builds under same source control folder i.e under one TFS folder, there are 22 projects and 3 solutions under this folder, each solution contains some shared projects. I have created a continuous integration build for each of the 3 solution files. The problem is whenever anyone checks in under any solution all 3 builds are triggering. Is there any way to trigger the related build only if there is any check-in under the any solution ? Please let me know how to configure 'Items to build' and source setting tabs in TFS 2012 ?
TFS 2015u1 and older
The Source Control Folders in the mapping govern when a CI build triggers, nothing else. A custom Activity might be able to check that no changes were made in specific folders and prematurely stop the build, but that requires you to create a custom activity and customize the workflow.
Normally you'd place each solution and its projects in their own folder in Source Control, that way you can create a source control mapping for each specific CI build.
TFS 2015 update 2 and onwards:
The new build engine supports separate definitions for workspaces and triggers.
I am working with Visual Studio 2012 .NET 4.5 ASP.NET MVC 4 project that uses TFS for source control and TFS Build for continuous integration (CI).
I want to create functionality that on each check in the build number gets updated prior to the CI build is kicked off.
From research it seems that a custom activity can be created and integrated in TFS 2010 build template.
I have also seen examples of this can be achieved with MSBuild task.
I haven't done work in this area before, so I am wondering which is the better approach or the recommended approach based upon the options open to me? In general when would I use MSBuild tasks as oppose to custom activity? For example, I will be looking to run FxCop and StyleCop against check ins also in the future, so I would like a common approach to this.
In the case of incrementing the build number, I'd vote for the TFS Build Activity so that the implementation is not tied to your msbuild implementation. This allows you to easily apply the TFS workflow activity to any number of branches without tying it to the branches directly. In addition, it keeps your MSBuild project files clean of the task so that it isn't mistakenly executed on developer machines.
Holistically, I'd say that you need to take a variety of factors into account when deciding between MSBuild and Workflow activities:
1 - Does MSBuild support the functionality out of the box (like Code Analysis/FxCop)?
2 - Does the build step need to run on developer boxes as well as servers (StyleCop/FxCop)?
3 - Does the build step need to interact with the TFS API or source control directly (checking out/in a version file for incrementing)?
4 - Are you going to change build job schedulers later to something free (for example, Jenkins)?
It's the combination of these things that determines the implementation of any given tool integration in my book. I'd implement FxCop, StyleCop and any other tool that should be run on a developer box build via MSBuild. I'd implement build steps such as version incrementing, bin-placing and CI deployment invocation (for example, deployment of a SharePoint webpart as a post-build step) via a Code Activity or some scriptware.
In 2009, there was a SO question on the same topic.
I'm wondering if later versions of Team Foundation Server are better at longer build pipelines. Refer features of Jenkins, TeamCity, ThoughtWorks' Go (my employer).
The visualizations of the build pipelines are important to me, as well as the notification about individual stages passing or failing. That and the eminent clone-ability of say a 'trunk' pipeline into one for a release branch as that branch leaps into being.
Secondly, a personal holy-grail is the CI server storing its config in the SCM that's holding the buildable thing itself, and even picking up on the creation of branches silently to provision new pipelines; Can TFS be configured to store the CI definitions/scripts in its SCM side rather than its accompanying SqlServer?
TFS build consists of three components:
The build definition - stored on the SQL server data tier.
The build workflow - a XAML file stored in the source control.
The supporting MSBuild scripts - usually contains user defined actions, also stored in the source control.
As the build progresses, you can see visualization of the build steps and you also get a different log for the main build and the MSBuild output.
The build definition in TFS is merely a collection of build settings, similar to CC.Net's config file and TeamCity's build configuration tab which both stored on the file system as well. Assuming there's a backup plan on the database you don't really need to store the build defintions on the source control, but if you must it's possible by exporting the tbl_BuildDefinition table.
The TFS Power Tools adds cloning functionality for build definitions.
There's no OOTB support for provisioning build definitions from a new branch though it's fairly feasible using the TFS-API.
Bit late to the party, but just don't bother with TFS if you want advanced build pipeline automation. It simply doesn't cut it.
I have used Jenkins and TFS both extensively. Tfs is just. pure. crap. Here's why.
No down/up stream build.
No piepline/orchestraion build. (like jenkins)
Obscure ways of adding build steps and falls back to using MsBuild.
Slow and still polls the source control.
Ties you to MsTest.
And please don't point me to "Oh look you can do everything if you write a custom activity". I'm not wasting time doing development for a closed source, sub-par platform. If I am going to contribute something, it's to a FREE. OPEN SOURCE platform.
I am preparing to move my team's source control from VSS over to TFS 2008.
This is for an asp.net website, and I am currently using a combination of nant scripts and Cruise Control to do all of the builds and deployments.
I've been trying to wrap my head around the best way to architect TFS build to do the same thing I'm doing with NANT and Cruise Control, but I can't determine the best approach.
Here are my requirements:
When code reaches a certain point, I manually apply a label to it.
This labeled code needs to be built and deployed to any of our 25 different Dev, QA, or production environments.
Any of these 25 environments can be on any current or past labeled version of the application.
I need to be able to deploy any labeled version of the application to any of the environments.
I'm currently accomplishing the above using NANT to perform the build, and using Cruise Control to just pass in command line options for which environment(s) to build and deploy. I have a Nant config file with a list of all of my environments, and an associated label each environment should currently be using. This file gets manually updated whenever a new label is created.
I know the approach I'm using for NANT probably won't be the same as with Team Build, but has anyone done something similar with Team Build and could share how you accomplished it?
Labeling in TFS is much more robust than in VSS. When you create a label, you can create based on a changeset, date, workspace version, heck even a different label. (BTW, I was grabbing a link and came across this post that you might find relevant.)
By default, a Team Build will build from the latest version of code in source, but you can override the "CoreGet" target in a build to build a specific version. Aaron Hallberg (a.k.a. TFS's John Skeet) shows an example here.
see 4.
I haven't personally had this difficult of a requirement, but I've done something similar. When you queue the build, you can pass in any number of parameters in a couple important ways, 1) through the response file and 2) at queue time (simple example here). In either case, two parameters could be which environment and which label/version number. At my current project, I have continuous integration turned on, so when code in a workspace is checked in, the current code is automatically labeled, pull the specifics for my drop location from the response file, then deploy to the respective location.
Given the fact that you have ~25 environments and n number of versions/labels, you could build a simple GUI that reads the current labels through the TFS API and lets you pick which version to build to a particular environment.
To answer the question, the way I addressed this was to use a combination of a custom build task, cruise control, and msbuild.
The custom build task allowed me to get the latest version from a specific branch and label.
Cruise control allowed me to pass in specific information for a specific build to MSbuild, using a config file, but initiate the build from a UI.
msbuild was used like normal, however it was called from cruise control, and the custom build task did most of the work.