I have a TFS 2010 with some projects and a common library used in 5 of them. We use VS 2013 and we have Rolling Builds enabled in most if not all build definitions. When the common library is checked in, all of the projects referencing it are recompiled - but the order is poor, the most commonly used project is compiled as the last one. Is there a way to change that so it gets compiled first?
This question hints at a lot of problems and possible solutions. The simplest answer is probably to just add more build servers to run all the builds in parallel.
Otherwise you need to consider turning off rolling builds and writing your own build scheduler. That or other strategies such as building the DLL once and checking it in as source to the other builds or running the build on a branch that is outside of the other 5 builds and merging the source into those builds when they want to pick up the change.
If you are otherwise happy with how things are now and don't want to do a lot of work to solve the problem.... then just scale out your build farm with additional agents.
There's a "Priority in queue" on the build definition, but it sounds like you might want to change your solution's > Project Dependencies' > Build Order?
Related
I'm struggling to get Azure Devops Server 2019 (on prem) to build a complicated project setup of mine.
I have multiple solutions that build various BizTalk apps. Some of these solutions references some of the projects in other solutions/repos. This works fine in Visual Studio (providing everybody names their repos as the default, which they do).
To automate this build, I've created a multi-stage build pipeline that builds each solution in order.
Whilst solution 1 will build successfully, when it comes to solution 2, the msbuild tasks will not reference the outputs for the projects that have been included as a project reference in the solution.
This seems to be because these referenced projects aren't marked for build - because they wouldn't be able to build themselves without them in turn referencing their other projects in the main solutions etc.
Bundling everything into one big super solution file is not feasible.
I'm wanting msbuild to /reference the projects that have already been built in the previous step, as per being included as project references. But msbuild is a complicated beast and I cannot figure out a way of achieving this.
Is what I'm trying to achieve even possible? Can anybody point me in the right direction?
TFS / msbuild, building project references
This is a known issue about project reference in different Repos, but unfortunately it doesn't have a best answer. Because the best solution is always to have a single repository.
Git thinks of files as the content of the whole repository, not as a collection of files. Therefore this is quite hard to do. As workaround, you can consider to use Git Submodules or Git Subtree:
Check the details info from here:
Git and Visual Studio project references
Besides, the solution we are using now is to put the shared code in the NuGet package. Then, you can use the package from any repository, and you don't have to perform any unstable git settings and keep access control as they should be.
Hope this helps.
It seems that msbuild was changed to only reference project references, and that you can revert that to reference all references by setting the property OnlyReferenceAndBuildProjectsEnabledInSolutionConfiguration to true when running msbuild.
Source: https://stackoverflow.com/a/25144169
Have confirmed this works.
I've finally revisited this and done some more experimentation.
The only way, that I can see, to get TFS/msbuild to /reference other projects outputs is to mark them in the solution build configuration to build.
Now if you do this in Visual Studio and try and build a solution that has projects from another solution then the build will fail if the previous solution hasn't been fully built yet (as the other chained dependencies won't have been built). This makes sense.
But with TFS/msbuild, the build will succeed. From what I can tell there is some magic going on that ensures the dependencies across all solutions are somehow resolved. This might be luck, it might be specific behaviour (it's working for me so far).
The problem of course is that it's incompatible with Visual Studio. So I have an extra "Build" definition setup in each Solution file that has all the projects set to build.
This seems the easiest way to manage lots of interrelated solutions without having one big one.
I have a large solution, with many projects and many files, and only one build configuration, Release. I am using TFS, and the complete rebuild takes like 2 hours.
Is it possible to distribute the build across several agents, so that they will compile different projects, or, even better, different files? Something like dictcc? I can distribute the build on up to 10+ different machines, but the build only works on one.
For now, my impression is that agents can only have specialized jobs, like build, run tests, etc, but not split and distribute only one build.
I already tried optimizing the build, but still the project is big and can benefit on parallel build
You can but but must roll-up your sleeves: there is no built-in template that helps, but Jim explains how to make one.
Do not forget that you can also leverage multi-CPU/Core as explained in Building Multiple Projects in Parallel with MSBuild.
Your best option would be to break your solution down into defunct components that can be built separately.
If you seperate each bit and build and test before publishing as Nuget you can distribute easily across build servers and even only build the bits that have changed.
This process will also work in the new build system coming in 2015 that does not use XAML.
My team has TFS build machine building checkins.
Today, we have more than a dozen build definitions to prevent building projects/running test in unaffected areas. This was when we weren't using incremental build.
Now that we enabled incremental sync/build, I am thinking about creating one giant gate definition that include all my team's source code. Since incremental sync/build is enabled, unchanged files don't get built anyway. But TFS is still running all tests.
Is there a way to dynamically pick just test assemblies that were built recently?
I can do it by modifying build template to filter out test assemblies that are more x hrs old but before I go that route, I wanted to check if there is something already available.
thanks
Customizing the template is the right path. Beside the official documentation, take a look at ALM Rangers' Build Customization Guide.
The FindMatchingFiles activity has no date filtering, so you have to roll your own.
Is there a way in TFS 2012 to set Project (not .csproj projects but TFS projects) build priority?
Currently we have many projects in TFS, one of which is a core project that many other projects reference. If someone checks in changes to both core and another project at once, the core project build doesn't always kick off first (I think they build in alphabetical order). It would be nice to be able to set the core project as the highest priority so that its build always runs first when changes to multiple projects are checked in. Is this possible?
There is no way to set the default build priority for a Build Definition. Since a Build Definition isn't bound to a Team project, but to a collection, there is also no way to configure the Queue to pick up builds from one Team Project before any others.
What you can do is to set a tag on an agent and then assign that tag to the build you want to take precedence. This will exclusively reserve the Build Agent to that specific build definition, causing it to jump in the queue.
Is there a way in TFS 2012 to set Project (not .csproj projects but TFS projects) build priority?
A project in TFS is quite a loose term which does not correspond to a VS project, it's actually used to refer to a product which can contain many branches. The high level project management tools then operate on the TFS "project".
If someone checks in changes to both core and another project at once, the core project build doesn't always kick off first (I think they build in alphabetical order).
Nope, assuming you don't have a custom build then they (the VS projects) don't build in alphabetic order at all, they build in order of dependency. Solutions are built in the order they are returned from the evaluation of the $(SolutionsToBuild) property (this does a wildcard search of the filesystem for .sln files, so maybe that's where your alphabetic observation comes from). Note that dependencies are evaluated on projects within a solution, there is no evaluation of dependencies across solutions.
For the following statements I'm going to assume that you have standard CI style builds gated upon checkin.
If a developer checks in everything at once then the checkin will happen as an atomic unit of work and the build will commence once the checkin is successfully committed. If this is not your experience then I would suggest that you either have funky stuff happening in a custom build, or the developer is checking blocks of work in separately rather than all at once.
Note that this happens per checkin per workspace - if two different developers check stuff in at the same time then whoever checks-in first will trigger the build, and the submission of the second developer will miss the build.
I'm starting to dive into TFS 2012 and I have a basic understanding of the tiers and how build servers, controllers and agents work and how different build scripts can have different configurations and projects.
However, one of the things I'm struggling with is a requirement for our source control solution that says that I need to be able to prove a particular changeset or shelfset produced a particular build. That is, given a particular binary, I can point to a release changeset that generated that binary. I should also be able to point to the test changeset that was merged into the release branch. The idea here is not just a separation of duty, but validating that because the release and test changesets are identical, no code was injected into a project by a code reviewer.
I've read one blog post that talks about "Binary promotions" -- would that concept be useful in my situation? I'm having a hard time finding how this binary promotion is set up in TFS.
Deployment
Out of the box TFS doesn't really support deployments, it can deploy to 1 location on build which often is a test server (think lab management). TFS 2012 has built in support for Azure deployments, but they still happen at the end of a build and the build artifacts cannot be automatically deployed to a new location.
You could modify the build template to allow to release to different locations, but that would still be a fresh build for every environment and not true binary promotions.
TFS does, however, have a concept of build quality and actually fires off events when this quality is changed. TFS Deployer is a 3rd party tool that hooks into the quality change event and can execute powershell scripts. This means with a simple change of a dropdown value you can automatically kickoff a script that releases to any environment you want. You can customize the build quality list (per team collection) to be a list of environments (dev, uat, staging, production etc) which the script then figures out where to release the specific build to.
VS2012 also has some nice improvements to web deploy which means deployment configurations are stored in source control with the project, which in theory means they'll be available in the drop folder for TFS Deployer to make use of.
I don't believe TFS keeps a history of build qualities, which means you can't really use the build quality history to maintain a list of what is deployed to which environment. You could fairly easily record this information as part of the deployment script though. Or at the very least add a custom summary node to the build with information about the release.
TFS2012 does have the ability to mark a build as deployed as part of the Azure deployment functionality, you mark tfs deployer builds as deployed using a script but it doesn't feel very useful.
Octopus Deploy is another project that's worth checking out, and could be used instead of TFS Deployer if your build template creates NuGet packages. It requires a bit more control over the production hardware as you need to install agents on each environment to handle releases, but it solves a lot of other issues with deployment.
Versioning
Once you have a nice consistent way of automatically releasing that people don't bypass, you can look at enhancing the build template to inject the build version, or changeset number as the assembly version for anything built as part of that automated build. There's a number of different ways to do it and plenty of blog posts and tools to help you achieve that.
Alternatively you could just use automatic assembly versioning ([assembly: AssemblyVersion("1.0.*")]) to give you the date/time the build occurred, which ends up like 1.0.1234.123 where 1234 is something like the days since jan 1st 2000, and 123 is the minutes since midnight (my specifics may be wrong here).
If you're deploying websites, then I highly recommend injecting the current build version into the html somewhere. This way you can check what version a website is running without needing access to the bin directory. It can also be appended as a querystring to css/js file imports to ensure no browser caching occurs between versions.
Thoughts
Personally I'm hoping Microsoft realise that the xaml build workflows are trying to do too much and that they split the different concerns (build, test, deployment...) into different scriptable parts. Of course that would not be until the next major release of TFS which is years away. Although with Team Foundation Service they are trying to iterate a lot quicker, so they may actually extend the Azure deployment stuff into something more useful in the nearer future.