QUERY:
I want to create a build definition template on TFS that would keep a check on the files being checked in. I want to retrieve the path of the files that are being checked in and these files should be saved to the SQL server.
WHY DOING THIS:
I want to search the files (contents of the file also) on TFS and show the relevant file(s) to the end user based on the search term. As we know TFS saves Data to SQL server but doesn't stores in a way that we can use SQL full text index. So, I am planning to save the files to the SQL server using the TFS Build definition template that would watch the files being checked in and save the file to the SQL Server.
Please help with any pointers on TFS build definition template on how get the path of files on TFS and how to trigger the save event to SQL server.
Thanks in Advance.
Install a build controller and register at least one build agent (prerequisite for the TFS build system).
Create a new build definition and set the trigger to "Gated check-in".
Download the default process template and modify it. Follow the instructions over here: Customize your build process template.
Remove everything that belongs to calling MSBuild from the workflow (inside the "Run on agent" activity).
Create your own activity and put it there. The activity should use the downloaded sources, inspect them and write build errors if something is not to your liking. (see Use and develop custom build process activities)
I’m using TFS 2012 Build and running into an error
Access to the path is denied
The solution being built contains about 15 projects of which a number are using the Castle.Components.Validator.2.5.0 assembly. I have seen other posts that talk about the TFS Build Access Denied errors, but they generally refer to having simultaneous builds running. In this case only one build runs at a time. Also, the error occurs when the server is restarted or the build has not run for some time.Once a build is run and fails, the next one succeeds and each one after that succeeds again until the build hasn’t been run for a while or the server is restarted. Although we can get around this, it is a manual headache. Here is the error:
C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets (3513): Unable to copy file "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll". Access to the path 'D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll' is denied.
When looking at the log file you can see that the build is trying to copy the file twice. Because the first one has a lock on the file, the second one fails and thus the build fails. Here is a snippet of the log file that shows what is happening:
2>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll".
5>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll".
2>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\MvcContrib.Mvc3.FluentHtml-ci.3.0.96.0\lib\MvcContrib.FluentHtml.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\MvcContrib.FluentHtml.dll".
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\RhinoMocks.3.6\lib\Rhino.Mocks.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Rhino.Mocks.dll".
Any help on how to fix this would be greatly appreciated.
As others mentioned, this happens when performing multithreaded builds with a common destination directory and the file copy task happens to encounter a simultaneous conflict with a copy task running for a different project.
Normally this should result in a "file used by another process" exception (which is handled and retried by the file copy task) but sometimes the file operation results in an "Access is denied" exception instead. (I'm still not sure why)
Some suggest that you should "solve the duplication", but I don't see that as being feasible for cases where all the projects need to directly reference a library like log4net.
Obviously one way to prevent the issue is to explicitly run msbuild with /p:BuildInParallel=false or /m:1 or /maxcpucount:1 (or omit the argument entirely) to force single-threaded mode.
However, in TFS 2013, the default build template automatically always passes /m (use all cores) to msbuild, which silently overrides any single-thread setting you can manually pass in. (Determined by my own experimentation and examining diagnostic logs)
Another workaround I attempted was to manually pass /p:AllowedReferenceRelatedFileExtensions=none to msbuild, which prevents all pdb and xml files from being copied from referenced libraries. (Since for a while I only ever saw xml files having this issue.) But then I kept having problems with log4net.dll.
The ultimate workaround that I used was one I discovered by decompiling the source code for Microsoft.Build.Tasks.Copy:
if (hrForException == -2147024891)
{
if (!Copy.alwaysRetryCopy)
throw;
else
this.LogDiagnostic("Retrying on ERROR_ACCESS_DENIED because MSBUILDALWAYSRETRY = 1", new object[0]);
}
If error -2147024891 (0x80070005 access is denied) occurs, the Copy task will check a special variable to see if it should retry. That value is set via an environment variable:
Copy.alwaysRetryCopy = Environment.GetEnvironmentVariable("MSBUILDALWAYSRETRY") != null;
After setting the environment variable MSBUILDALWAYSRETRY = 1 (and restarting the build server), the problem went away. And I also periodically started seeing "Retrying on ERROR_ACCESS_DENIED..." as warnings in the build logs, proving that the setting was taking effect, (instead of the builds merely coincidentally succeeding).
(Note that this environment variable is not well documented, use as appropriate.)
Update: Apparently TFS 2015 no longer overrides your /m:1 with /m (even on legacy/XAML build definitions), which should make /m:1 a valid fix again.
It looks like there are two projects copying the same file. Depending on the timing, they sometimes happen at the same time, resulting in the failure. You have to trace the node id back to find the source project. See http://blogs.msdn.com/b/buckh/archive/2012/01/21/a-tool-to-find-duplicate-copies-in-a-build.aspx for more details and code that may track it down for you.
As Buck Hodges and Nimblejoe have rightly said, this is mostly due to TFS running multiple MSBuild processes by default to build your projects.
You can override it in the build definition in Process -> 3. Advanced -> MSBuild Arguments by adding the MSBuild argument /p:BuildInParallel=false
This can also happen if you have a build agent's folder open.
I also had same problem. I got error messages that related to cannot copy since access to path denied. In my case all my dll's and xml files and so on are place at
D:\TFS\Example\Bin\Debug folder.
I right clicked on Bin folder and clicked Properties and saw that Read-only check box is checked under Attributes.
I un-checked Read only check box and cliked Apply and clicked OK on the new popup that is shown.
I went back to Visual Studio and build my solution which was giving me error messages.
Voilaa.. This time it build successfully without errors.
I donot know whether this is perfect but I did this to solve my issue.
To work around this problem I had to remove the "ReadOnly" flag on the source directory
Then in the build definition set Clean Workspace
to None
Like Ziggler, I solved this problem with building a project by removing the 'read only' property of the bin folder in my project. It is only happening to XML files stored in a /packages/ directory that is common to the solution that contains this project. The 'bin' folder is not checked into source control. I am still stumped as to the root cause of the problem.
I found the same problem which occurred after the build tried to overwrite files in the "Working Directory" it had created in a previous attempt to build. (set in the Agent)
I resolved this by manually deleting the output folder it created (in my case [Working Directory]\Binaries) before attempting the build.
This can be done automatically by changing the Build Definition. Under Process---2.Basic---Clean Workspace set this to the Outputs option
Here's a variation of this problem which I had to deal with:
I couldn't figure out why my build kept failing on an "Access to the path is denied" error, even though I had added things like /p:BuildInParallel=false and /p:OverwriteReadOnlyFiles=true to the MSBuild Arguments of my XAML build. The cause turned out to be a "Post-build event command line" in my Project's properties.
After changing
%WinDir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe[SNIP]
/P:Configuration=$(ConfigurationName);[SNIP]
;AutoParameterizationWebConfigConnectionStrings=false
to
%WinDir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe[SNIP]
/P:Configuration=$(ConfigurationName);[SNIP]
;AutoParameterizationWebConfigConnectionStrings=false;OverwriteReadOnlyFiles=true
the error went away.
One possible cause is if you have the bin or obj folders for class libraries checked-in into TFS. Deleting the bin or obj folders of the projects from TFS will resolve this issue if that is the case.
I was having this problem and chose to ignore it because I didn't want to sacrifice build performance for the sake of getting rid of some benign error messages by NuGet. However, I seem to have stumbled across a solution while trying to solve another problem, and I think it is related. I think the order of fetching of NuGet packages is related to the build order of projects in the solution. So if this has somehow become disjointed, then NuGet may be the first casualty before you run into build errors where you start getting "Metadata file 'XXX.dll' could not be found" errors which annoyingly require you to build again until the build succeeds (as described here).
So, I believe the solution is to follow the steps described in the accepted answer to the aforementioned question. Or, follow the more comprehensive steps in one of the alternative answers. In other words, disable building of all projects, restart VS, then re-enable building of all projects. This will (normally) resolve build order. And that should hopefully resolve the NuGet issue. Please let me know if this fixes it for anyone.
I had this issue, with TFS 2015.
It turned out to be because the build Agent was running under the default (NETWORK SERVICE) credentials, which didn't have write permissions on the target folder.
Once I'd removed the Agent and reinstalled it with credentials it worked.
It did have me trawling through the logs for a while, checking and unchecking the multi-proc box and even restarting the build server in my hunt.
Check the obvious stuff first...
For me, it was that the build agent wasn't started in an administrator powershell.
MSBuild arguments:- /tv:14.0 /t:Rebuild /m:1 /p:RunCodeAnalysis=false /p:TreatWarningsAsErrors=false /p:OverwriteReadOnlyFiles=true /p:BuildInParallel=false /p:AllowedReferenceRelatedFileExtensions=none
strong text
Set false to Clean workspace
Go to build agent and remove read only from mapped folder.
as a lot of people have already stated before, this happens when building projects in parallel. Project A and B both referencing 3rd Party Library C (Copy Local) will cause this when they are build at the same Time - side by side.
The real problem is, that TFS Build 2012 and below are configured that when building a solution, the whole output of the solution is copied to a single folder. Thats where the pains of parallel builds are having their origins.
Since TFS 2013 you can easily solve this by setting the "Output location" in the build definition to "PerProject". This forces the build services to behave like a local msbuild run where the setings regarding the output locations are read from the corresponding project files. So the output is written to the bin folders under each project.
For TFS 2012 and below this article (+linked articles) will help you getting the same result as with TFS 2013:
http://blog.stangroome.com/2012/05/10/override-the-tfs-team-build-outdir-property-net-4-5/
I resolved a very similar issue by closing all open instances of Visual Studio, re-opening the solution and building it again.
We are currently setting up Team Build 2010 for our company, and I am trying to use workflow activities to retrieve the exact local path to the current solution being built. I haven't found a way to get this value, does anybody know how (without writing a custom activity)?
Either one of server or local path would suffice (i.e $/TeamProject/Branch/OurProject or C:\TeamBuild\src\path\to\branch\OurProject) since we can use the conversion activities on the server item.
The reason we need this path is for updating version info files, and that needs to be done for only the current solution being built, and the files have the same names (AssemblyInfo.cs, for example).
In this similar question, the solution is to define a parameter, but since this information is particular to the solution being built and not some external path, we were hoping that this info would retrievable.
You can retrieve this particular info without adding anything. If you navigate within your Build Process Template to the position where MSBuild breaks out, you will see that the solution that shall be build is set as a string named localProject. This will contain the local path where TFS has downloaded your SLN, something like C:\TeamBuild\src\path\to\branch\OurProject\OurProject.sln.Open the XAML and navigate to:
Run On Agent
Try Compile, Test, and Associate Changesets and Work Items
Compile, Test, and Associate Changesets and Work Items
Try Compile and Test
Compile and Test
For Each Configuration in BuildSettings.PlatformConfigurations
Compile and Test for Configuration
If BuildSettings.HasProjectsToBuild
For Each Project in BuildSettings.ProjectsToBuild
Try to Compile the Project
Compile the Project
Run MSBuild for Project
if you select Run MSBuild for Project & hit F4 you see it.
In order to retrieve what you are after you can define another string-variable solutionPath in your Build Process Template & insert under the Run MSBuild for Project a new Assign activity withTo : solutionPath andValue : Path.GetDirectoryName(localProject)
Have you looked at the TFS Community Build Extensions, they give you a assembly versioning out of the box?
You can use the variable called SourcesDirectory to get the current Source Directory on the Build Server. You can also use an ConvertWorkspaceItem activity to convert between server and local paths.
There's also a blog post that cover's all of this here.
In a customer project, I need to copy a built dll to another place where it will be checked in and shared amongst different solutions.
I am using a post build step to checkout the target file specifying the /lock:none parameter so that others will be able to create local release builds as well and then copy my new dll file over the old one.
However, when I use tf.exe checkout /lock:none on a console prompt, everything works as expected. When used within a post build script, the file gets locked and nobody can check it out anymore.
How can I solve this?
First question is why do a "checkout" instead of just a "get"? If you're not going to be changing the original file, there's no reason to do a checkout.
To answer the question specifically, though, the reason this is happening is that by default, executable files are set to not allow merging. That means that-- no matter what-- a checkout on a DLL is going to be an exclusive checkout.
To change this behavior, in Visual Studio 2010:
Go to the Team menu
Select Team Project Collection Settings, then
select Source Control File Types
Find Executable Files in the file list, and Edit it to enable File Merging
Click on OK to commit your changes, and you should be good to go.
I've seen the question related to the error message you get from TFS when a workspace is already mapped. The accepted answer for removing the workspace is alright as a workaround, but it's already getting tedious to run a delete command each time this error occurs.
What do I need to change in order to get out of having to use this workaround? I've got two builds (continuous integration and nightly deploy), and need to add at least one more build type. I followed this URL to see if there was a possible resolution there, but I'm not sure I understand it completely.
I am not sure how this is accomplished in TFS 2010, as I have not gotten to work with Team Build in 2010, yet. In 2008, though, if you expand the Builds node in the Team Project and right-right click on either of the builds, you will see a "Manage Build Agents..." option. Click into that, and it will bring up a dialog. One of the things on that dialog is an option called "Working Directory". Do you have the same hard-coded path in both of them?
By default, when you create a new build definition, it provides a calculated folder for this value. This is where the build agent will do the checkout from TFS for the build attempt. The default value is, $(Temp)\$(BuildDefinitionPath), I believe (I am not connected to TFS at the moment).
The article you link to is basically saying that you should include either that $(BuildDefinitionPath) value or the $(BuildDefinitionID) value as part of that path in that dialog so that the two builds do not try to use the same workspace. Changing the working folder to include one of those values should resolve your issue, going forward.