TFS Build missing .targets files intermittently - tfs

Question

Our builds are failing on the TFS build server with the same errors from time to time.
I'm getting this message on one of our projects (the error refers to a WebApi project):
The imported project "C:\Program Files
(x86)\MSBuild\12.0\bin\Microsoft.CompactFramework.CSharp.targets" was
not found. Confirm that the path in the declaration is
correct, and that the file exists on disk.
And this one on another project (on smart device projects):
The imported project "C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\v12.0\WebApplications\Microsoft.WebApplication.targets"
was not found. Confirm that the path in the declaration is
correct, and that the file exists on disk.
Usually this means that the file is not there and it should be a matter of either installing the tool that contains the targets files or manually copy them from a developer machine. I've seen it before in that situation and solved it easily since it is a trivial problem.
Problem is, the files are there! The builds seem to fail randomly, which is extremely obscure to me. If we select 'Retry Build' a few times, the build goes through normally. Some times the build just works the first time, and then it starts showing this error. I failed to detect any pattern as to when it happens.
There is absolutely nothing special about the files themselves. I thought they were missing some permissions somehow, but they have the same security settings as other accompanying files.
I have no idea what to even investigate here. The message points out that a file is missing but the file is there. The fact that it occurs intermittently is even more problematic.
This is happening on a TFS 2013 build machine. Visual Studio 2013 Ultimate was installed on it after installing TFS. We have another build VM in the company using TFS2012 and I've never seen such odd behavior.
EDIT:
I forgot to mention it, but there are also a bunch of random warnings on the build containing the web project. They only appear when the error about the .targets file occurs, and do not show up on the other build. Perhaps it is related with the fact that one is .Net4.5 (the web) and the other is .Net3.5cf:
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (989): The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (989): The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (1635): There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "C:\Windows\Microsoft.NET\Framework\v4.0.30319\\mscorlib.dll", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (1635): There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "System.Data", "AMD64". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (989): The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (1635): There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "C:\Windows\Microsoft.NET\Framework\v4.0.30319\\mscorlib.dll", "x86". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (1635): There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "System.Data", "AMD64". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets (989): The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
EDIT 2:
Ok. After Dylan's comment, I went to the build logs to "confirm" that there was only one agent running the builds, and to my surprise:
Successful Build: Run on agent (reserved build agent Build2014 - Agent1)
Failed Build: Run on agent (reserved build agent TFS2013 - Agent1)
Build2014 is our build VM, and TFS2013 is our source control one.
When TFS 2013 was installed on TFS2013, a build agent was installed alongside it on the same machine. The controller that was installed on a separate machine also had an agent with the same name. This means that the controller was picking one of them and whenever they hit the TFS machine the build failed (because there is no dev environment there on purpose).
Removing the wrongly installed agent on the TFS machine is the way to go here.

Answer 1

Are you sure your builds are always running on the same build agent/server? Is it possible you have multiple build agents setup and it only works on certain ones?