I want to use the same targets defined in an external file in multiple build definitions. This external target file should be stored on the source control.
The bootstrap phase explained in TFSBuild.proj and Importing External Targets does not seem to allow downloading the same file for multiple build definition as the build file is always named TFSBuild.proj and consequently sits in a separate folder for each build definition.
The question was somewhat aked in a comment of the previouly mentionned thread but it seems motre appropriate to create a new thread.
Quoting that comment by David Keaveny:
Is it possible to get TFS to download
from other folder paths during
bootstrap? I have a Common.targets
file that contains all the custom
tasks that my TFSBuild.proj should be
executing, but given that I several
build definitions (one folder per
definition), it sits in a separate
folder at the same level as the build
definitions. Without it, of course,
the builds fail immediately. – David
Keaveny Aug 17 '10 at 23:58
I'm not sure if there is an easy way to do this. You could have a master copy of the targets file, and branch that in to each of your build definition folders. When you update the targets file you'd need to remember to merge it to all of the folders. You could probably write a script that automates the merging, the script should be able to work out what the branching relationships are using tf branches and then iterate around them and call tf merge then tf checkin. This feels like a lot of work though.
You could just install the target on your build server(s). We have a number of custom targets that we use and we manage them as if they were a product, they have their own area for source control and are versioned independently of the software they are used to build. We package them using wix and once they have been tested they are installed on all of the build machines. This seems to work for us and it means we can control the rollout of new targets files, and the targets install is just part of a build server basline build.
I'd be very interested if someone has an easier way of managing this though.
Related
I am new to the world of scripting with TFS2015. I created a script that builds all of the projects within my solution (it is a rather large solution) and puts it out in a shared folder (where each project has its own subfolder).
I would like to create a separate script for each project that simply copies the bin folder from the shared and pastes it out on my Test environment. I rarely need to deploy everything, so the idea is one build...multiple deploys.
However, when I run my deploy script using the Copy Files step it is doing another build. Although it copies the files that I expect, it is after a full build that creates the folder structure for the build.
Am I able to make the Copy Files step NOT do a Build?
Here is the steps that my script is curently doing:
As you can see, there is only one step (Copy Files) but it still does the Get sources and copies everything into a new folder on the build box like so (where the number keeps incrementing up with each run of the script):
I just want to copy the files from the Source to the Target and not do a build or Get Sources.
It looks like you're still on TFS 2015 RTM or Update 1. Which is already pretty old technology if you compare it to the lifetime of the new build system which was introduced with this version.
TFS 2015 update 2 has introduced a similar system to the Build pipelines to orchestrate Releases. This doesn't require you to map any workspaces or git repositories and can act on the artefacts of your builds or simply on the contents of file shares.
It makes sense that a Build has to build something and in order to build something, it has to get the things to build. If you're actually not building something, then you're probably deploying or releasing or packaging something else. Hence the distinction between Build and Release pipelines.
TFS 2017+ has an option to disable the syncing of sources. Primarily to allow people to get the sources themselves in creative ways (e.g. a custom powershell script that invokes git.exe).
My primary advice would be to upgrade to TFS 2018 update 3 or at least TFS 2017 update 3.1, worst case TFS 2015 update 4.1. The fact that versions older than update 2015.4.1 have a known XSS scripting security bug may be reason enough to convince your organisation to perform this update.
Barring that option you're left with one solution:
Link your build definition either to a git repository with only a single commit (If I remember correctly the 2015 agent still crashes when syncing an empty Git repo) or link it to a TFVC repository and set the workspace settings to cloak everything. This essentially causes the build to sync an empty folder, which it can cache, before calling your powershell script.
Related to TFS 2017 release management artifact files from version control
I'm asking a new question because I believe I have an edge case the answers don't directly address and I don't want to derail that OP. Specifically, how do I allow an independent, offsite team building required supporting scripts in a separate TFS Team Project supply their scripts as an artifact in the Release definition of a separate TFS Team Project? The separate team projects are built by independent customers and we are not allowed to append content to their source control. Further, updates to the scripts must automatically spread to all Release definitions using them on the TFS.
We have about 40 team projects in TFS all running on different schedules. A separate operations team handles all build and release management tasks in TFS.
Because of the constant bouncing between team projects and because ops also wanted to use the version control and work item tracking features in TFS, we created a separate team project for them to store scripts, installers, and license files. These are referenced in other projects' RM tasks for automatic installation/execution. There is also a separate version control folder tree for tracking project specific scripts - like this:
Common
Applications
App1
App2
...
App43
This makes it significantly easier for them to manage their scripts and associate them with work items themselves without having to shuffle across all the other team projects. The dev teams do not have access to the ops project.
However, when linking a version control artifact in RM from their project, it will only bind to the root and appears to copy the entirety of the version control structure to the agent, even though most of this content is not relevant to the app being deployed.
Is there a way to add specific, not all, folders from their project in version control as artifacts to a release definition in a separate project? We have our QA release start the process to production and it pulls in the artifacts from the ops project and the project being released. All subsequent releases reuse the artifacts that succeeded in the QA build instead of going back to the server for new versions of the artifacts.
Build definitions don't let us pick workspace paths outside of the team project so I don't see a way to pull in their scripts in a build step, either.
Is there a way to do this? How are other organizations handling this issue?
No.
The same answer I provided in the other answer applies here: Don't. Publish them as NuGet packages or as separate build artifacts; a release definition can have multiple artifacts linked to it.
I appreciate Daniel's answer and I believe what he is stating is best practice. However, I believe I found a more direct technical answer to my question through the use of additional repositories.
Release Management allows you to reference Git repos and branches independently, like I had originally hoped to do with folders under the TFVC repo already in a separate project. In this way, we configured the TFVC repo to handle large binaries (installers), license files, etc. which we version and put in a Team Project Nuget feed for reference from RM. To address the folder issue, we created separate Git repos for our operations team project in the same TFS project. Like this:
Binaries (TFVC-based repo)
Git Repositories
CommonDeploymentScripts
Environment Scripts
App1 Scripts
App2 Scripts
etc.
This way TFS RM from any other project can be configured to pull in any one or multiple of these repos as artifacts for use by the agents, bringing down only those scripts that were placed in them.
Also, the ops team doesn't have to cross reference app-specific scripts while bouncing around in a bunch of independent team projects. Note: Daniel is right when he says app-specific stuff should really be versioned and stored with the app project itself. However, some environments may not yet have that luxury so this can fill that need.
RM lets you reference branches under a single Git repo as well so this might be overkill. However, we didn't like the idea of branches under a repo not really having any business ever being merged up into the master - felt like too much room for error.
I have a drop folder created by TFS build which contains all the produced artifacts:
.dll \ .config \ .pdb files from all projects
_PublishedWebsites folder
*.msi files produces by Wix projects
All I really care about is *.msi files as everything I deploy is in them.
How can I specify for the TFS build not to bother with all the other files in the Drop folder?
I know I can customize the build to delete files after the solution build, but maybe there is a clever way of disabling them at all?
Personally I use named platforms such as Application and Setup in my SLNs so that when TFS archives the drop folder one class of files goes into one directory structure and another class of files goes into another. This way it's easy to find the MSI and it's also easy to see what the application code looked like before then. (Perhaps the contents of a web.config or an HTML that was added to the application sln but not the installer sln.
IF you want to suppress the application sln from archiving you have to look at that. It's not an MSI / WiX thing.
If you are using TFS 2013 (or VSO) you can easily have a PowerShell executed post build to do whatever clean up and rearranging you need.
Note: The build used the files in the root to do testing, code analysis, test impact analysis, and other automatic actions. Make sure that you only remove files after all of the checks. A better idea is to leave the files be and just push theb*.msi files to a "/_PublishedApplications/* folder.
There is a PublishedApplication Nuget package that can make this easyer. Take a look...
If you are using TFS 2010/2012 you can use the "TFS Community Build Tools" to call PowerShell and do other things.
I'm using TFS 2013 on premises. I have four build agents configured on a Build machine. Several build definitions compile ASP .NET websites. I configured the msbuild parameters to deploy the IIS application to the integration server, which sits out there in Rackspace.
By default webdeploy does differential deployments by comparing file dates. In my case that's a big plus because copying files from our network to Rackspace takes quite some time. Now, in order to preserve file dates the build agent has to compile the same base set of source code. On every build only the differential source code yields a new DLL, minimizing the number of files deployed.
All of that works fine, with a caveat: a given build definition has to be assigned to a build agent (by agent name or tag). The problem is I create a lot of contingency when all builds assigned to the same agent are queued up. They wait in line until the previous build is done.
In an ideal world any agent should be able to take care of any build, but the source code being compiled has to be the same, regardless of the agent.
I tried changing the working folder of all agents to point to the same location but I get an error because two agents can't be mapped to the same folder. I guess there is one workspace per agent.
Any ideas?
Finally I found a way to do this. Here are all the changes you need to do:
By default the working folder of each agent is $(SystemDrive)\Builds\$(BuildAgentId)\$(BuildDefinitionPath). That means there's one working folder per BuildAgentId. I changed it so that all Agents share the same folder: $(SystemDrive)\Builds\WorkingFolder\$(BuildDefinitionPath)
By default at runtime the workflow creates a workspace that looks like "[BuildDefinitionId][AgentId][MachineName]". Because all agents share the same working folder there's an error trying to create each separate workspace. The solution to this is in the build definition: Edit the xaml and look for an activity called "Get sources from Team Foundation Version Control". There's a property called WrokspaceName. Since I want to have one workspace per build definition I set that property to the BuildDetail.BuildDefinition.Name.
Save your customized build template and create a build that uses it.
Make sure the option "1. TF VersionControl/1. Clean workspace" is set to False. Otherwise the build will wipe out all the source code on every build.
Make sure the option "2. Build/3. Clean build" is set to false. Otherwise the build will wipeout the output binaries on every build.
With this setup you can queue up the same build on any agent, and all of them will point to the same source code and bin output. When the source code changes only the affected binaries are recompiled. I have a custom step in the template that deploys the output files to IIS, to all the servers in our webfarm, using msdeploy.exe. Now my builds+deployments take one or two minutes, because only the dlls or content that changed during the build are synchronized to the servers.
You can't run two build agents in the same folder. The point of build agents is to run multiple builds in parallel, usually on separate PCs. If you try to run them on the same source code, then (a) it's pointless as two build of exactly the same source should produce identical results, and (b) they are almost certainly going to trip over each other and cause the builds to fail or produce unexpected results.
If you want to be able to build and then deploy a series of versions of your codebase, then there are two options:
if you queue up multiple builds, then the last one will "win", so the intermediate builds are of no real value. So if you check in New code before your first build completes, you may as well stop the active build and start a new one. you should be asking yourself why the build is so slow, or why you are checking in changes so often that this is necessary.
if each build produces an incremental update to the deployed result, then you need to pass the output of your builds to some deployment agent that is able to diff it against the deployed version and send only the changes to be deployed. This could be set up to gather results from multiple build agents if that would be beneficial.
but I wonder if perhaps your build Is slow because you are doing a complete build each time (which cleans the build folder, gets all the sources, and does a full rebuild), when what you want is an incremental build (which gets the latest changes, compiles only what is affected, and complete quickly). perhaps you should investigate making your build incremental.
I have some Libraries added in my TFS 2010 project in a Lib folder.
Each time I check In a dll in the Lib folder, all the build that reference that Lib folder trig automatically (more than 20 Build).
I can use the ***NO_CI*** to prevent that. But I prefer to have something automatic.
Is it possible to have an event handler on TFS to prevent that.
Ex. If the check in is on that folder, do not scan all the CI Build Workspaces to trig the build.
The workspace mapping in the build definitions defines which files/folders are part of that build. You can use cloaked folders to explicitly exclude folders from the workspace.
However, the workspace mapping is used for 2 purposes: to determine which files get downloaded to the build server as part of the build, and to determine which files trigger CI/Rolling/Gated builds.
If you exclude folders from the workspace (e.g. using cloaked folders) it will stop the builds from being triggered but it will also stop those files from being downloaded as part of the build.
If you want to have a build download a certain folder as part of the build but not have builds triggered by check-ins to that folder I don't believe it is possible without customizing the build workflow.
When you include a folder in a workspace, you're saying that changes to that folder affect your project. This is the same for changes to shared source code as for shared binary. If it changes, then the affected applications should be built (and their automated tests should be run) so taht it can be determined whether the changed files have broken anything.
Make sure each library in the lib folder has it's own folder + version folders beneath
lib
EntityFramework
4.1
EntityFramework.dll
4.2
EntityFramework.dll
Then modify your builds to only reference specific folders for the library a project references.
It takes a lot of work to setup builds, but will ensure the build only triggers when a file needed for the build is changed.
Alternatively a copy of each library in the projects directory, which you could manage with a package manager like NuGet or OpenWrap