I'm wondering if anybody else has seen the following issue with catkin build. It's rare, but every now and then when I rebuild my workspace (literally just calling catkin build again after changing a .cpp file) it seems like changes to certain files are not reflected (i.e. new print statements don't show up, even though I can see the change on git).
Has anybody else seen something like this? I suspect what's happening is that catkin build is (for whatever reason) not registering that the file in question has been modified, and as such does not rebuild that unit. Cleaning and rebuilding the workspace completely of course seems to fix the issue, but I'm wondering if others have encountered this and if there's a specific reason this happens.
FWIW I have two Ubutnu 18.04 machines each running ROS Melodic and I've observed this behavior on each one (albeit only a few times). I'm also using VS Studio Code on each machine (in the unlikely case that this matters).
As always, TYSM for the help!
This is a known issue(at least to me) and I’ve observed the behavior in Kinetic, Melodic, and Noetic. As of right now there isn’t a specific reason that has been singled out inside of cstkin_make as to what causes it. However I do suspect it’s a caching issue.
As you stated it can easily be solved by cleaning the build space and forcing a new build; which is probably good practice in most circumstances regardless.
Related
The last time I deployed the project, the build worked perfectly.
In the meantime I have changed nothing that would affect the pip requirements, yet I get an error when building:
Could not find a version that matches Django<1.10,<1.10.999,<1.11,
<1.12,<1.9.999,<2,<2.0,==1.9.13,>1.3,>=1.11,>=1.3,>=1.4,>=1.4.10,
>=1.4.2,>=1.5,>=1.6,>=1.7,>=1.8
I get the same error when building the project locally with docker-compose build web.
What could be the problem?
The problem here is that although you may not have modified any requirements, the dependencies of a project can sometimes change on their own.
You may even have pinned all of your own requirements (which is generally a good idea) but that still won't help if one of them itself has an unpinned dependency.
Anywhere an unpinned dependency exists, you can run into this.
Here's an example. Suppose your requirements.in contains super-django==1.2.4. That's better than simply specifying super-django, as you won't be taken by surprised if a new, incompatible version of the Super Django package is released.
But suppose that in turn Super Django 1.2.4, in its requirements, lists:
Django==1.11
django-super-admin
If a new version of Django Super Admin is released, that requires say Django>=2.0, your next build will fail because of the mutually exclusive requirements.
To track down the culprit when you run into such a failure, you need to examine the build logs. You'll see there something like:
Could not find a version that matches Django==1.11,>=2.0 [etc].
So now you know to look back through the logs to find what is wanting to install Django>=2.0, and you'll find:
adding Django>=2.0
from django-super-admin==1.7.0
So now you know that it's django-super-admin==1.7.0 that is the key. Since you can't trust super-django to pin the correct version of django-super-admin, you'll have to do it yourself, by adding django-super-admin<1.7.0 to the requirements.in of your project.
There's more information about this at How to identify and resolve a dependency conflict.
You can also Pin all of your project’s Python dependencies to ensure this never happens again with any other dependency, though you sacrifice some flexibility for the guarantee.
Note: I am a member of the Divio team. This question is one that we see quite regularly via our support channels.
I have setup a build controller etc and the builds were failing, I have fixed these now and the build failed properly - as in because of an error.
I have fixed the error and checked the code back in but now the code is not being extracted, although sometimes one folder of many is.
I have deleted the code from the build machine and requeued a build but it keeps failing. It complains that it cannot find the solution that I specified as the build solution.
I have checked the check box to build even if nothing has changed.
Have I missed a setting somewhere for extracting the code?
TFS version is 2012 Express
Visual Studio version is 2010 Professional
I had this issue recently with TFS 2012. I think it boils down to this:
In the lastest build definition files, it appears that it performs a Clean task before updating the workspace. This means that if you do something that causes the Clean part of the build to fail, it will never download the new files in order to fix it.
Recently, I was making big changes to my build file and inevitably made a lot of mistakes, I found that if one of these mistakes caused the Clean to break, I had to go onto the Build server and change the file manually to get it working again.
Does this sound like it might be the same issue?
There are several properties in your build definition you can check. I would start with setting the "Clean Workspace" to All to ensure the correct code is being pulled down and built.
There are other checks you can look at as well like the agent set for the build and the "GetVersion" property. Check the below link out. It should be able to help you in more detail.
Define a Build Process that is Based on the Default Template
I’m using TFS 2012 Build and running into an error
Access to the path is denied
The solution being built contains about 15 projects of which a number are using the Castle.Components.Validator.2.5.0 assembly. I have seen other posts that talk about the TFS Build Access Denied errors, but they generally refer to having simultaneous builds running. In this case only one build runs at a time. Also, the error occurs when the server is restarted or the build has not run for some time.Once a build is run and fails, the next one succeeds and each one after that succeeds again until the build hasn’t been run for a while or the server is restarted. Although we can get around this, it is a manual headache. Here is the error:
C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets (3513): Unable to copy file "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll". Access to the path 'D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll' is denied.
When looking at the log file you can see that the build is trying to copy the file twice. Because the first one has a lock on the file, the second one fails and thus the build fails. Here is a snippet of the log file that shows what is happening:
2>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll".
5>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\Castle.Components.Validator.2.5.0\lib\NET40\Castle.Components.Validator.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Castle.Components.Validator.dll".
2>_CopyFilesMarkedCopyLocal:
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\MvcContrib.Mvc3.FluentHtml-ci.3.0.96.0\lib\MvcContrib.FluentHtml.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\MvcContrib.FluentHtml.dll".
Copying file from "D:\Builds\12\Foo\Check-In Build\Sources\packages\RhinoMocks.3.6\lib\Rhino.Mocks.dll" to "D:\Builds\12\Foo\Check-In Build\Binaries\Rhino.Mocks.dll".
Any help on how to fix this would be greatly appreciated.
As others mentioned, this happens when performing multithreaded builds with a common destination directory and the file copy task happens to encounter a simultaneous conflict with a copy task running for a different project.
Normally this should result in a "file used by another process" exception (which is handled and retried by the file copy task) but sometimes the file operation results in an "Access is denied" exception instead. (I'm still not sure why)
Some suggest that you should "solve the duplication", but I don't see that as being feasible for cases where all the projects need to directly reference a library like log4net.
Obviously one way to prevent the issue is to explicitly run msbuild with /p:BuildInParallel=false or /m:1 or /maxcpucount:1 (or omit the argument entirely) to force single-threaded mode.
However, in TFS 2013, the default build template automatically always passes /m (use all cores) to msbuild, which silently overrides any single-thread setting you can manually pass in. (Determined by my own experimentation and examining diagnostic logs)
Another workaround I attempted was to manually pass /p:AllowedReferenceRelatedFileExtensions=none to msbuild, which prevents all pdb and xml files from being copied from referenced libraries. (Since for a while I only ever saw xml files having this issue.) But then I kept having problems with log4net.dll.
The ultimate workaround that I used was one I discovered by decompiling the source code for Microsoft.Build.Tasks.Copy:
if (hrForException == -2147024891)
{
if (!Copy.alwaysRetryCopy)
throw;
else
this.LogDiagnostic("Retrying on ERROR_ACCESS_DENIED because MSBUILDALWAYSRETRY = 1", new object[0]);
}
If error -2147024891 (0x80070005 access is denied) occurs, the Copy task will check a special variable to see if it should retry. That value is set via an environment variable:
Copy.alwaysRetryCopy = Environment.GetEnvironmentVariable("MSBUILDALWAYSRETRY") != null;
After setting the environment variable MSBUILDALWAYSRETRY = 1 (and restarting the build server), the problem went away. And I also periodically started seeing "Retrying on ERROR_ACCESS_DENIED..." as warnings in the build logs, proving that the setting was taking effect, (instead of the builds merely coincidentally succeeding).
(Note that this environment variable is not well documented, use as appropriate.)
Update: Apparently TFS 2015 no longer overrides your /m:1 with /m (even on legacy/XAML build definitions), which should make /m:1 a valid fix again.
It looks like there are two projects copying the same file. Depending on the timing, they sometimes happen at the same time, resulting in the failure. You have to trace the node id back to find the source project. See http://blogs.msdn.com/b/buckh/archive/2012/01/21/a-tool-to-find-duplicate-copies-in-a-build.aspx for more details and code that may track it down for you.
As Buck Hodges and Nimblejoe have rightly said, this is mostly due to TFS running multiple MSBuild processes by default to build your projects.
You can override it in the build definition in Process -> 3. Advanced -> MSBuild Arguments by adding the MSBuild argument /p:BuildInParallel=false
This can also happen if you have a build agent's folder open.
I also had same problem. I got error messages that related to cannot copy since access to path denied. In my case all my dll's and xml files and so on are place at
D:\TFS\Example\Bin\Debug folder.
I right clicked on Bin folder and clicked Properties and saw that Read-only check box is checked under Attributes.
I un-checked Read only check box and cliked Apply and clicked OK on the new popup that is shown.
I went back to Visual Studio and build my solution which was giving me error messages.
Voilaa.. This time it build successfully without errors.
I donot know whether this is perfect but I did this to solve my issue.
To work around this problem I had to remove the "ReadOnly" flag on the source directory
Then in the build definition set Clean Workspace
to None
Like Ziggler, I solved this problem with building a project by removing the 'read only' property of the bin folder in my project. It is only happening to XML files stored in a /packages/ directory that is common to the solution that contains this project. The 'bin' folder is not checked into source control. I am still stumped as to the root cause of the problem.
I found the same problem which occurred after the build tried to overwrite files in the "Working Directory" it had created in a previous attempt to build. (set in the Agent)
I resolved this by manually deleting the output folder it created (in my case [Working Directory]\Binaries) before attempting the build.
This can be done automatically by changing the Build Definition. Under Process---2.Basic---Clean Workspace set this to the Outputs option
Here's a variation of this problem which I had to deal with:
I couldn't figure out why my build kept failing on an "Access to the path is denied" error, even though I had added things like /p:BuildInParallel=false and /p:OverwriteReadOnlyFiles=true to the MSBuild Arguments of my XAML build. The cause turned out to be a "Post-build event command line" in my Project's properties.
After changing
%WinDir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe[SNIP]
/P:Configuration=$(ConfigurationName);[SNIP]
;AutoParameterizationWebConfigConnectionStrings=false
to
%WinDir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe[SNIP]
/P:Configuration=$(ConfigurationName);[SNIP]
;AutoParameterizationWebConfigConnectionStrings=false;OverwriteReadOnlyFiles=true
the error went away.
One possible cause is if you have the bin or obj folders for class libraries checked-in into TFS. Deleting the bin or obj folders of the projects from TFS will resolve this issue if that is the case.
I was having this problem and chose to ignore it because I didn't want to sacrifice build performance for the sake of getting rid of some benign error messages by NuGet. However, I seem to have stumbled across a solution while trying to solve another problem, and I think it is related. I think the order of fetching of NuGet packages is related to the build order of projects in the solution. So if this has somehow become disjointed, then NuGet may be the first casualty before you run into build errors where you start getting "Metadata file 'XXX.dll' could not be found" errors which annoyingly require you to build again until the build succeeds (as described here).
So, I believe the solution is to follow the steps described in the accepted answer to the aforementioned question. Or, follow the more comprehensive steps in one of the alternative answers. In other words, disable building of all projects, restart VS, then re-enable building of all projects. This will (normally) resolve build order. And that should hopefully resolve the NuGet issue. Please let me know if this fixes it for anyone.
I had this issue, with TFS 2015.
It turned out to be because the build Agent was running under the default (NETWORK SERVICE) credentials, which didn't have write permissions on the target folder.
Once I'd removed the Agent and reinstalled it with credentials it worked.
It did have me trawling through the logs for a while, checking and unchecking the multi-proc box and even restarting the build server in my hunt.
Check the obvious stuff first...
For me, it was that the build agent wasn't started in an administrator powershell.
MSBuild arguments:- /tv:14.0 /t:Rebuild /m:1 /p:RunCodeAnalysis=false /p:TreatWarningsAsErrors=false /p:OverwriteReadOnlyFiles=true /p:BuildInParallel=false /p:AllowedReferenceRelatedFileExtensions=none
strong text
Set false to Clean workspace
Go to build agent and remove read only from mapped folder.
as a lot of people have already stated before, this happens when building projects in parallel. Project A and B both referencing 3rd Party Library C (Copy Local) will cause this when they are build at the same Time - side by side.
The real problem is, that TFS Build 2012 and below are configured that when building a solution, the whole output of the solution is copied to a single folder. Thats where the pains of parallel builds are having their origins.
Since TFS 2013 you can easily solve this by setting the "Output location" in the build definition to "PerProject". This forces the build services to behave like a local msbuild run where the setings regarding the output locations are read from the corresponding project files. So the output is written to the bin folders under each project.
For TFS 2012 and below this article (+linked articles) will help you getting the same result as with TFS 2013:
http://blog.stangroome.com/2012/05/10/override-the-tfs-team-build-outdir-property-net-4-5/
I resolved a very similar issue by closing all open instances of Visual Studio, re-opening the solution and building it again.
This morning I ran my tests and there are 2 failures. But I haven't changed any code for a few days and all tests were passing.
According to Git, there are no changes (except for coverage.data, which is the test output). gitk shows no other changes.
How does Git know when code changes? Could this be caused by an SSD failure/error?
What is the best way to figure out what happened?
EDIT: Working in Ruby on Rails with Unit Test framework.
I'd start by figuring out why the tests failed, and that might give you some clues as to how they might have passed before. Sometimes it's a timing issue, intermittent failure, something external to the test harness, data changing, a change of date or time, all sorts of stuff.
I see Mike found your problem (ticky the little answer box please).
Yes, it is possible for code to change without Git knowing about it. The file which caused the failure, perhaps a temporary testing file or fixture, could be ignored either in .gitignore or .git/info/exclude. Doing a git clean -dxf will wipe the checkout clean of anything not known to git. git status --ignored will show the files ignored by git. If that's the case, you want to add better test cleanup as part of your test runner.
For posterity, here's the short list of ways tests could fail without there being any code change visible to git:
"Temporary" test files and fixtures might be dirty.
"Temporary" databases and tables might be dirty.
It is sensitive to time or date.
It uses network resources and they changed.
The compiler was changed.
The installed libraries used were changed.
The libraries the libraries use were changed.
The kernel was changed.
Any servers used (databases, web servers, etc...) were changed.
It uses parallel processing and a subtle bug only occurs sometimes.
The disk (or the filesystem where temp files go) is full.
The disk (or the filesystem where temp files go) is broken.
Your memory/disk/process/filehandle quotas were reduced.
The machine has run out of memory.
The machine has run out of filehandles.
It uses fixtures with randomly generated data and generated some that tickled a bug.
Your repository can't have any changes without Git knowing about it and logging it somewhere since it's necessary to retrieve the logs to be able to get the repo files anyways. I would check to see that the failures aren't caused by something on the machine that shouldn't be running at the same time as the tests or it could possibly be a race condition that you're not expecting.
I'm noticing some solutions in Team Foundation Server 2008 won't build completely. As in, some number of the projects in the solution succeed but then one fails. The particular failing project says I’m missing an assembly reference. But I'm not - the project has the reference. It builds fine on my PC. I'm looking at the .csproj file that the build agent pulled down and it has the reference, too (it's line-for-line identical to the project I'm building).
The reference in question is to another project in the solution. At first I thought it wasn't building projects in the right order but the build log is telling me that it did (i.e., the project which the reference is in reference to built successfully). So my guess is that somehow on this project (and I’d say about 10-20% of the projects I’m having it build are failing) it’s looking at the wrong folder for assemblies, but I have no idea.
Has anyone ever seen this before?
I did have one Solution which was building things in the wrong order and some Googling seemed to indicate that this was an occasional side-effect of converting a VS2003 SLN file to a VS2008 SLN file (and it was happening in Visual Studio as well), so in that case I made a new SLN from scratch, checked that in "on top" of the problematic one, and it worked fine. But I tried that in the problem above and it didn’t make a difference.
I've seen it on multicore machines where the project dependencies haven't been setup correctly, meaning that tfs starts a project compiling before it's dependencies are finished compiling.
At first I thought it wasn't building projects in the right order but the build log is telling me that it did (i.e., the project which the reference is in reference to built successfully).
Define "built successfully." In particular, make sure the CoreCompile target was invoked and ran to completion. I've seen cases where a different target on the referenced project was built, but that target was insufficient to generate the output the dependent project needed. For example, a web project that includes a Silverlight control will call the GetXapOutputFile target on the Silverlight project -- which is fine & dandy, but it's no replacement for CoreCompile.
If this clue isn't enough to resolve your issue, you should probably post a link to the log and/or the msbuild makefiles.