TFS - build definition with frequently changing branch path - tfs

My team uses a per-release and per-sprint branching approach. So we typically have a fresh branch off of Main (integration) for the current sprint and a branch off of Main for each release.
Main Branch
|
-- Development Folder
| |
| -- Sprint 2.10_1 Branch
| -- Sprint 2.10_2 Branch *current*
|
-- Release Folder
| |
| -- Release 2.8.0 Branch
| -- Release 2.9.0 Branch *current*
There are two build definitions. One points at the current dev branch and the other points at the current release branch.
This setup is working well, except that it has become a bit time-consuming and error-prone to update all the branch paths in the builds every time we branch for a new sprint and branch for a new release. Each build has branch paths in these places:
Source Settings > source control folders (multiple active and cloaked paths)
Process parameters > Build > projects to build (paths to multiple
projects)
A build only ever points to one branch location in tfs, and the only part of that branch path that changes each time is the number associated with the current sprint or release. So for instance a build might switch from pointing from /developement/2.10_1/ to /developement/2.10_2/.
Is there a way to define a base-path once in your build definition and then use it throughout the definition? That way each time we switch branches we only have to specify the branch path in one place? Even better, could that variable's value be managed outside of the build definition so that it could be used by several build definitions? Could the variable value possibly be dynamic based on the active Iteration for the project?
Or can the path entries in the build definition be defined in such a way that they are relative to the branch?
Any suggestions? Thanks!

I've set up our builds so that they use a custom $(BranchToBuild) parameter which is inserted into all the build paths within the build. This eliminates the problem you have in the 'projects to build' section of the definition.
This parameter can then be passed into the build by adding a /p:BranchToBuild=2.10_2 to the build parameters in the Queue Build dialog, so you can manually select any branch to build from with every build you queue up.
You can also set the default parameters within your build definition so that it defaults to /p:BranchToBuild=2.10_2 for every build - each time you make a new branch the "current" one you can just change this default and all subsequent builds will automatically use the correct branch (but you can still go back and do a build off any previous branch easily, for example if you have to merge a bug fix back into a previous release)
The only hassle with this is (as you've spotted) that you have to also map the code for the branch onto the build server for it to be got from source control. There is a shortcut for this though - in the build definition, select all the mappings for your old branch (10.1_1) and copy them. Paste into a text editor and you'll see that each just becomes a simple line of text. Now you can globally search and replace 10.1_1 with 10.1_2, then copy and paste the entire set of mappings back to the build definition. Miles faster and less error prone than manually editing every line in the mappings.
All the above means that setting up a new branch takes me about 30 seconds.
The caveat is that the build definition points at the vcproj file that controls the build, and it gets this file before it runs the build itself. It is therefore problematic to put your build definition inside the branch. Generally this is not a problem, but occasionally when you need to update the build definition it can therefore break your branches unless you also manually point the build definition at the correct variant of the vcproj. Generally I get around this by avoiding breaking changes in the build, so it has only bitten me once in the last 7 years.

Related

When my TFS build is triggered by a branch-specific check-in, why doesn't it set that branch as its source?

My goal is to create a TFS CI Build that is triggered by check-ins on two specific branches of a project, hosted in TFVC. When I queue the build manually and type a specific branch in the "Shelveset name" field of the queue prompt, the build works as intended. However, when I check in changes from one specific branch, the build attempts to use the parent folder of the two branches as its default source, resulting in incorrect file paths throughout the build definition wherever I use the $(Build.SourceBranchName) variable.
My folder structure is as follows:
- $/MyRepo
|
| -Integration Suite
|
| -CurrentVersion
|
| -Dev
I have CI triggers turned on with path filters to include $/MyRepo/Integration Suite/CurrentVersion and $/MyRepo/Integration Suite/Dev
In the "Get Sources" task, my repository is set to $/MyRepo, and I map these server paths to the CurrentVersion and Dev local paths, respectively.
I would expect, for example, that checking in changes under the CurrentVersion branch would set the source of the build to $/MyRepo/Integration Suite/CurrentVersion, but instead it is set to $/MyRepo/Integration Suite. This means that any paths I build in the definition with $(Build.SourceBranchName) have Integration Suite where they should instead have CurrentVersion.
Am I doing something wrong? Or is my desired behavior not supported?
TFVC relies on workspace mappings to know what to download. The workspace mappings can encompass multiple TFVC repos across different team projects, multiple branches within a single repository, basically, whatever your heart desires. As a result, there's no way for it to understand how to dynamically change workspace mappings to be for a specific branch.
The sad reality of TFVC builds is that you need one build for every branch. This makes TFVC... er... not great for many common continuous delivery scenarios.

How does TFS choose which check-ins to associate with a build?

Our builds generally have a mish mash of work items and commits associated with them and I cannot tell how TFS determines what to add. We are using TFS 2015 update 3 and TFVC.
When a build runs, it gets code from a location somewhere in the branching and folder of TFVC. Typically, something like "root\dev\src\component name" in this way we avoid getting all of the code in our repository and we have CI set up to run so that any changes in this folder will result in a CI build running.
We also run daily builds which run more tests and create a release package that is used by TFS Release Management. I would expect that any changes to code inside of the folder defined in setting up the repository for this build to be included in the associated change-sets of a build. I also expect that any changes checked-in outside of these branches would not be associated. But this is not the case. We see commits from across the entire project.
Does anyone know how this is supposed to work?
I am not sure if this should go in the question or the answer but I have found some additional information, thanks to the hints provided in the answers below.
It appears that the source settings will take the common root between mapped folders of the repository settings, so if I have 2 folders $/Relo/Dev/B1/src/Claims.Services and $/Relo/Dev/B1/src/PSScripts it will take the common root $/Relo/Dev/B1/src as the source settings and include any changes from that folder down within the build. Can anyone confirm this? Of course thats not what I want to have happen. In the History tab of the build definition if I looked at the diff I can see a field "defaultBranch" in the json which seems to be the value that controls this, is there any way to update this field directly?
TFS determines what changesets should be mapped to a build based on the Source Repository Mappings (Build vNext) in the build definition and the last successful build.
So, you will see a list of the changesets with files committed in the lowest common base of any of the mapped folders including all their descendents, since the latest successful build. Whenever you get a successful build (I hope that it happens more often than failing ones ;-)) the list will shorten and only show the last check-in.
Example mappings below will result in any changeset made to anything below $/Relo/Dev/B1/src (because it is the lowest common base):
$/Relo/Dev/B1/src/Claims.Services
$/Relo/Dev/B1/src/PSScripts
Similar it will pick up all the related work items to the above changesets.
This is what should happen. If you see something else, I would have a closer look at the Repository Mappings or Source Settings of the build definition.
#Noel - I guess you are using vNext build and not XAML builds. Or are you using a mix of XAML and vNext?
In general a scheduled TFS build will associate all changes which were not associated in the last successful run of the same build.
I suggest you check once again if the source folder locations are the same for CI build and Daily build?

Copying and Editing Jenkins Jobs

I have been using Jenkins for a couple of months now, and have been able to set up a simple CI system.
I currently have a build tab - that will build and deploy 25 different components successfully, based on building from an SVN trunk.
I'm now taking my first branch - some people will develop on the trunk and some will develop fixes on the branch.
I would now like to have CI up and running for both branch and trunk - so would create a second tab - with a repeat of the jobs from the first tab, but this time changing the SVN path to check out from the branch.
As I have rather a lot of jobs and the task is quite repetitive, is there an easy way to do this ? I'm hoping that each job tab might be a single xml that I can edit / rename to give me a second tab ?
Yes, each job is as single xml file, located under $JENKINS_HOME/jobs/$JOB_NAME/config.xml. However there are a number of places in the config.xml that reference it's location, so simply copy-pasting the actual file isn't the best option.
Jenkins UI itself has "copy job" function.
Click "New Item" where you want it
Select "Copy existing Item"
Specify the name of existing item to copy
Specify the name for the new job
Then go to configure the new job and change what you need.
Another tip: may not apply to your setup, but more often than not, there is a "unifying difference" between multiple similar jobs, most often a branch name, or project name, or similar. The value would be that only one that needs to change between otherwise identical jobs, however that changed value may appear several times within the job configuration.
I usually make a "choice" parameter, with a single choice, and put that different value there. The rest of the job references the choice param as a variable. So when I copy identical jobs, I just need to change that 1 value at the top of job configuration page. Everything else falls into place.

Jenkins schedule multiple versions of the same build

I have a project that has 3-5 different mercurial branches going at all times. I want to schedule a weekly Jenkins test to run our tests on all relevant branches.
What I want, I think, is a parameterized build, with the branch name as the parameter, and then to have a list of branches, and once a week, run the parameterized build with each of the parameters in the list.
However, I see that you can't send parameters into a triggered build. I assume that there is a plugin for this. Is job Generator the correct plugin? Is there something better?
I should mention that currently, we are doing this with multiple SCMs, and having the body of the build have a sh loop that runs through each directory and runs the tests. This is really inefficient, and a pain to maintain...
I can suggest one solution but it couldn't be called elegant.
Firstly, you need create multi-configuration project (aka Matrix project).
In this project you need declare one node (it can be already existed master node)
And one type of axis (for example BRANCH - be careful don't use Jenkins Set Environment Variables variables) with values corresponding for each branch (for example default, testing, devel, etc).
After you need add in your project build action in which you need check environment variable (previously declared $BRANCH) and discover for which branch this build was launched (the main idea is illustrated by example with using bash).
And finally you need manually get sources from corresponding branch.
Next build steps can be the same for all branches.
This approach have set of drawbacks:
1. You can not triggered this project by changes in repository (you can check using Mercurial plugin only one branch).
2. All subprojects will be rebuilt even if they have not changed.
3. Appropriate only for statically defined branches.
4. Not elegant.
But it has one advantage versus parameterized build:
1. All artifacts (and build logs) of branches is stored in separated directories (because they are separate subprojects).

How to set up gated check in for a branch

I have two build definitions for my project which has two branches.
Development and Live.
I have set up manual build for development branch and this is running just fine.
Now I wanted to set up gated check in for live branch, that is triggered only when developers are committing into live branch. this is mapped to root of the project not only a specific branch and after i take map the project for build.
However my set up is triggered everytime when developers are committing into development branch too.
Is there anything what i am doing wrong?
My project layout:
$/KCTC/Lib/ (Contains all referenced dlls)
$/KCTC/Projects/ (contains branches)
$/KCTC/Projects/Development
$/KCTC/Projects/Live
How ever the branch does not see Lib referenced files:
Considered
"........\Lib\fluentnhibernate-NH3.1-1.2\Iesi.Collections.dll", but
it didn't exist.
Setup of my live gated build:
Also i have unit test created in NUnit in project and this is failing because
Queries\StarMetrics\20110613\StageTestSuite.cs (2): The type or
namespace name 'NUnit' could not be found (are you missing a using
directive or an assembly reference?)
workspace definition:
and my process defition
Gated Check-ins will be triggered for any attempted check in of a source control item that is present under any entry for your workspace mapping defined for the build definition. In your case, you have
$/KCTC/Projects/ (contains branches) <-- remove this
$/KCTC/Projects/Development <-- remove this as well
$/KCTC/Projects/Live <-- this should contain everything you need for the Live branch correct?
Which basically says, "perform gated checkins for anything contained under this folder"... Youll need to remove the indicated line to ensure you don`t fire gated check-ins when checking in code from the development branch or the parent folder containing all branches.
As for your missing Lib binaries, Id bet the relative path is just slightly different. I`d check physically on the build machine to confirm.
Looking at your workspace definition screenshot, it appears you're breaking the relative paths on the build server by including "\Moose" in your build agent folder.
You want:
$/KCTC/Lib | $(SourceDir)\Lib
$/KCTC/Projects/Live | $(SourceDir)\Projects\Live
Ok and solutions is:
I have went and added every item i needed for gated check in by hand. Where i can specify version control path to custom assemblies.
NOTE
if you have multiple dlls with the same name, you will get error in tfs (this will not affect the build but its error)
Or the alternative is add the dlls requited for project into build resources.

Resources