I'm trying to work with MSBuild and TFS.
I've managed to create my own MSBuild script, that works great from the command-line. The script works with csproj files, and compiles, obfuscate, sign and copies everything that's needed.
However, looking at the documentation of TFS & Team Build, it appears that it expect solutions as the "input" for the script.
Also, I haven't found an easy/intuitive way of performing a "Get Latest Version" from the TFS as part of the script. I'm assuming that the Team Build automatically do a "Get Latest" on the solutions it's suppose to compile, but again - I don't (want to) work with solutions...
Any insights? any pointers? any links?
Team Build defines about 25 targets of its own. When you queue a Team Build, they are automatically run for you in the predefined order listed # MSDN. Don't modify this process. Instead, simply set a couple of these properties that determine how the tasks behave. For example, set <IncrementalGet> to "true" if you want ordinary Get behavior, or "false" if you want something closer to tf get /force.
As far as running your own MSBuild script, again this shouldn't be necessary. Start with the TFSBuild.proj file that's provided for you. It should only require minimal modifications to do everything you describe. Call your obfuscation & signing code by overriding a task like AfterCompile or AfterTest. Put your auto-deploy code in AfterDropBuild. Etc.
Even really complex scenarios are possible if you refactor appropriately. See past answers #1 #2.
As far as the actual compile, you're right that Team Build operates on solutions. I recommend giving it what it wants. I'll be the first to admit that *.sln files are ugly and largely undocumented, but at least you're offloading the work to a well tested & supported product.
If you really wanted to, you could give it a blank/dummy solution and override the CoreCompile task with your custom compiler logic. But this is really asking for trouble. At bare minimum, you lose all of Team Build's flexibility WRT building multiple platforms and flavors. More practically, you're bound to spend a lot of time debugging something that's designed to "just work" -- and there are no good MSBuild debuggers yet (that I know of). Not worth it, IMO.
BTW, the solution files do not affect the Get process. As you can see in the 1st link, the Get is done very early on, long before Team Build even reads the solution file(s). Apart from a few options like <IncrementalGet>, this is not controlled from MSBuild at all -- in particular, the paths to be downloaded are determined by the workspace mappings associated with the build definition. I.e., they are stored in the Team Build SQL database, not the filesystem, and managed with tools (like Team Explorer) that call the TFS webservice API.
Related
I have a configured CI with TFS. What are the best ways to organize post-build (or even better post-test) deployment. My binaries are some libraries with single executable file.
Here is what I need:
Build on each commit. (This is configured and done)
When build is successful (or tests), grep binaries and drop it to some specific folder on the same build machine with full replacement of previous files and folders. (I`d like to be able to configure somehow the folder location)
Launch the application with some parameters and I need to have standart output redirection. For example: App.exe param=paramValue > log.txt
And before starting the application I need to kill the previous instance of it. (This is some kind of server instance that is alive all the time)
The most obvious solution that I tried was to do this with post-build script. But this try failed. See here
Use Release Management in conjunction with PowerShell (or better still, Desired State Configuration) scripts. Depending on your MSDN licensing, it could be free for you, and it's specifically designed from the ground up to handle managing releases.
Overextending the build process to also do deployment is an awful idea. The build tools were designed to build, and they're good at it! They're not good at the types of considerations you have when you're trying to do deployments.
The problem is that most CI solutions (TFS included) would get you to the point where you had binaries, then say "Welp, you're on your own! Have fun figuring out how to deploy this stuff!" This never ends well -- you end up with something inflexible and very difficult to troubleshoot and maintain.
The modern "devops" approach here is to have your application's requirements in source control, treated as code (in this case, as a DSC script or scripts).
One other consideration: It sounds like you're trying to treat a console application as a service. This is going to be a big, big pain for you, since most software that handles releases will not run in an interactive session. Turn it into a true Windows service and your life will be easier.
Currently I'm in the progress of updating TFS 2012 to 2013. My plan is to use the default build process template and stick to it, if possible. The existing build definitions use the old TFS method with TFSBuild.proj files. In thse proj files everything happens. Initializing the solution directory, build, clean, run unit test, drop files, etc. Compared to the 2013 build process template this is incorrect, since the workflow has activities for build, clean, run tests, drop files, etc. As well it seems that the targets file used in the TFSBuild.proj files is of a previous TFS version and hasn't been updated to a 2013 version.
The problem is that besides the build, clean, run test an drop files activities there are other activities needed. Version numbers are changed in certain files, obfuscation for dlls, check procedures if the source doesn't contain any unwanted files, zipping of pdb files, etc.
Of course it is possible to execute these tasks/activities with PowerShell scripts. On the other hand working with tasks in the a project file also seems logical. My concern with performing extra tasks in a project files is that TFS is running the activities for test, dropping files, etc. after calling the MSBuild activity.
Can someone point me in the right direction? Do I need custom developed activities that I can use in the build process template? Or is working with PowerShell scripts best practice?
I think you've answered your own question! Poweshell is the way forward!
I would say that modifying the XAML and writing a custom activity should only be considered if powershell isn't working for you, or if your scripts are becoming so big and difficult to debug that it makes sense to turn them in to code. As you mention, some of these tasks will need to be performed well after the MSbuild activity has completed so that's not going to work.
It looks like you want to do some fairly simple steps at defined points in the process and the powershell extensibility points are perfect for this.
A simple script for versioning in the pre-build step. Checking the source for unwanted files could happen here as well.
The obfuscation, zipping and copying stuff can happen post test.
From a Maintainability point of view, I would say that powershell has a much bigger user base than Windows Workflow and MSBuild so that's another advantage to this approach.
I am in the process of setting up continuous integration in our TFS system. One major part of our system are the development of about 50 DotNetNuke modules to support our CMS infrastructure. Right now, each of those projects have their own solution since their code bases are mostly siloed (with common code in 1 or 2 common projects). Keeping them in their own solution is done because it makes the development process faster (loading, compiling, etc....)
However, this has proven difficult to maintain when setting up TFS team build as each solution has to be manually added to the build definition and MSBuild seems unable to take advantage of parallel compiling due to each project being in its own solution. This causes about 5 minute full build times, which while isn't horrible isn't ideal. Mostly though, it's not ideal from a build definition maintenance aspect.
To solve this I creating a global solution that included all projects. The idea being that if you want your project to be automatically compiled and deployed by TFS you will have to include your project in the global solution. This seems works well, as it's easy to maintain from a build definition standpoint and brings the total build time down to 70 seconds.
The one problem is that the displayed TFS build log groups all warnings and errors together under the solution instead of separating them out by project. This makes it difficult to quickly see what project caused which errors and warnings.
Is there a good way to see project level error/warning messages in the build log summary view without delving into the cluttered build log?
To answer your direct question, I believe the answer is no (at least not without some heavy customization).
For me this is never a big concern as I am pretty aggressive about getting my teams to bring errors/warnings down to zero, then enforcing it via TFS Build (/p:TreatWarningsAsErrors=true). This means you should never have to wade through hundreds of warnings in the build summary.
If you add all your individual solutions to the build definition, you can always use the TFS Power Tools to "clone" a build def to make maintenance easier. You could also modify the Build Template to build the solutions/projects in parallel, although this runs the risk of having file contention issues.
We have an automated build and QA process for our software, using tfs/teambuild and msbuild, and we want to be able to know (for audit purposes) whether a component has gone through that process or not.
For example, if a library is installed on a user's machine, I'd like to be able to inspect it in some way to tell that it went through the build. In particular, I want to be able to distinguish it from components built directly on a developer's machine, and then manually installed.
What is the best way to do this? Code signing as part of the build process seems closest to these requirements, but presumably this would not cover any 3rd-party libraries that might be used? I also read about the ILMerge tool to merge all assemblies into one, but then I don't know enough to work out whether they can then be signed or not?
I'm sure we're not the first people to have the requirement, so casting around for any ideas or hints from others who might have done such a thing
Thanks!
Our developer builds are set to keep the versions at "0.0.0.0", but our build server marks the build based on a pre-configured version and automagically generated build string. "1.0.3.xxx". Your build server doesn't allow for this?
Your build process should be updating each of your projects assemblyinfo.cs files (or a global linked equivalent), you can do this with the TFS changeset number, so like the previous poster indicated you end up with the property on each dll of 1.0.changeset.buildno or something similar. You can do this easily in msbuild.
You could have the values of each assembly info file set in source control to be something obvious like 0 or 999.
A lot of what your asking is about process and training as well though.
If your using installers or zips to package your deliverables then you can also label them with the build number as part of your build process.
But if you have changeset you have the link from dll to code, so traceable, coupled with links to third party dll references as defined in each csproj.
We have a TFS 2008 project with two branches ("Main" and "NewFeature").
Each is a complete, independent "copy" (variant) of the source code.
By changing the workspace mappings, we can map either variant onto our local PCs and have been working with both branches with no problems.
However, if I set up the mappings to switch our build server on to the NewFeature branch (which should simply swap in the NewFeature source code without changing anything else as far as the build server is concerned) I get errors:
There is no working folder mapping for $/Main/Product.sln
i.e. when it is building from the NewFeature branch, something is still looking in the Main branch, even though there are no references anywhere in the source code to this branch. It appears to be caching some reference to Main?!
I have done a completely clean build (deleted the build folder from the server and run the build with /p:ForceGet=true to make sure the mapping is flushed through to the server, and there are no files on the server that might cache the workspace bindings), but this doesn't help.
Any suggestions?
Verify that:
$(SolutionToBuild) uses a relative path when referencing Product.sln
the relative path between $/NewFeature/.../TFSBuild.proj and $/NewFeature/Product.sln is the same as it is in the Main branch.
/ EDIT /
Note, however, it's not important that $/Main and $/Branches/Feature live at the same level in the tree hierarchy. Nor should the local path on the build server matter.* All that matters is what's underneath each branch. If the contents is internally consistent then all of your existing build scripts should work without modification.
For concrete examples of how I like to tie everything together, see my past answers, e.g.:
Modular TeamBuilds
SDLC Mangement for TFS Build Scripts
Where to put my database project in TFS?
How do you share external dependencies between Visual Studio solutions?
My way is not the only way, but I can attest that it works better than all the other variations I've encountered over the years :)
*Frankly, trying to micromanage Team Build can become a lot more painful than the proposed restructuring to your MSBuild scripts. For reliability you have to place your tfsbuildservice.exe.config customizations under version control somewhere...if you own >1 build server (or might in the future) then you have to consider a change deployment strategy...you end up needing a meta-SCM process to manage your SCM process!
I also had this problem when running a build from a branch in TFS 2010. TFS was reporting that "There is no working folder mapping for $/Main/Product.sln" The solution turned out to be to edit the build definition as follows (I am using the "Default Template" build process template—I have not tried this with a custom template):
Go to the Process section/tab of the build definition.
Expand 1. Required and look for Projects to Build. Make sure this entry is pointing to the solution file inside the branch you are building.
Expand 2. Basic and look for Automated Tests. Point this to the correct test settings file in the branch being built.
OK, the results are in - I've found a workaround.
Due to our legacy build processes (build, copy, obfuscate, build custom installers, copy to drop folder), I can't easily place the branch alongside the main branch. It needs to replace it.
So, if I have Main and NewFeature, I wish to unmap Main and map NewFeature in its place (i.e. use "c:\Main" on the build server, and simply change the source code that appears there)
Solution #1 (the most simple, obvious and logical solution) is to use these mappings:
$/NewFeature -> c:\Main
Expected result: NewFeature code structure simply replaces Main, and the build server doesn't know it's on a different branch.
Actual Result: Failure with a "you haven't mapped $/Main even though you're not using it" error.
Solution #2 is to do this:
$/Main -> c:\IgnoreThisFolder
$/NewFeature -> c:\Main
This works (it suppresses the warning and thus allows the build to proceed with MSBuild unaware that it is building in a branch). However, it's ugly and the build gets all the Main branch source code unnecessarily.
Solution #3 (untested, too expensive to try unless I know it'll work much better than #2) is:
Move all the source code (from $/Main, $/Branches/Feature) to $/Branches/Main and $/Branches/Feature to get a consistent hierarchy depth, and rewrite the MSBuild script to work with these new paths.
Hope that I can then map in only the branch I need and edit TFSBuild.proj to redirect it to build in that branch.
(Edit: Yes, this works well. We have now reorganised our entire code structure so that everything (all branches) is under a common root in a single Team Project, and branching/building is no longer a problem - it's easy to do whatever we need now. The trick is to insert a root folder into the hierarchy so that you can branch at any level you like. I've added a small tweak to the build script so that we can pass the branch to build as a parameter to MSBuild, so it's easy to build any variant now. Any branches we don't want to work on can just be cloaked and the build server remains happy.)
Summary
All these solutions (to use the technical term) suck. You have to remap the workspace (in this case, it's not simple: 9 mapping entries are required so it's an error prone and tedious thing to do), edit the TFSBuild.proj, delete all the source code, and run a build with /p:ForceGet=true to switch the build between branches. So it takes about an hour to switch branches. Unbelievable - it should take a few minutes at most!
I realise our project is far from ideally set up, but I can't believe it should be this difficult to branch in TFS (It was a piece of cake in SourceSafe, Accurev, and Perforce, so why so painful in TFS?).
How does everyone else organise their TFS branches? How do you switch developers between branches? How do you switch server builds between branches? Does it really have to be this painful?
When you Edit the build definition there are two places that need to be changed.
Source Settings - Point to your new project location
Process - (This sometimes takes a while to load so be patient) Under Required, change the "items to build" location to the new solution.
Hope this helps.
New update:
As reported in the other answer, I found a workaround that was ok for a short-lived feature branch, but it really didn't work very well. I've since come back to the problem, and the full solution is ridiculously simple:
In the TFSBuild.proj, the path was based on $(BuildProjectFolderPath). This path resolves to a server-side (source control path) like $/Main - not a local path (D:\ServerBuildFolder\Main).
Unfortunately, for historical reasons our source code is split across several team projects, which means the one "branch" is fragmented into several branched folders in Source Control (i.e. $/Main/Code and $/Libraries/Code. You can't create a branch that contains $/Main and $/Libraries). We thus have to reassemble these disparate fragments from Source Control back into a coherent hierarchy using workspace mappings.
This means that Richard was spot on - the relative path from the TFSBuild.proj file to the .sln file was incorrect, because MSBuild/TFS is assuming that the .sln lies within the same Team Project and source control hierarchy (so was looking for $/Main/Libraries.sln instead of $/Libraries/Libraries.sln).
The solution is simple: I replaced $(BuildProjectFolderPath) with a local path (e.g. D:\ServerBuildFolder\Main) for the files, so that the relative reference was resolved in "local space" (after the mappings had been applied), and MSBuild is now running sweetly.
The moral of the story:
1) NEVER use more than one Team Project if there is any chance that you will ever wish to have any kind of reference between those code-bases. Don't be fooled into thinking that a new Team Project will offer some kind of painless logical distinction between applications/libraries. Extra projects have proven to just be an administration nightmare - loads of extra problems for absolutely zero benefit. (It's all one big shared pile under the bonnet, so all the work items and source control folders are still visible in all the projects (!), but it adds a few brick walls that make inter-project links very problematic)
2) Always create a single root-level folder in your Team Project source control, and then put everything else underneath that folder. e.g. For the project "$/Main", create "$/Main/Root" and then put everything from your source hierarchy inside Root.
By following these rules, you will be able to branch the single 'Root' folder in future, and will then only need a single branch and a single extra workspace mapping. This will help you avoid premature baldness.
(In my defence, I would have done it this way to begin with - I'm working with a legacy setup. In defence of the legacy setup, it sounds good on paper but just isn't a Microsoft-supported approach!)
I got this error and all I can fathom is that the definition became corrupt or something. I just redid the process stuff (re-added the solution I was trying to build) and remapped the workspaces and it started working again. HTH.