I have three builds.
Build A
Build B (cloned from A)
Build C (cloned from B)
Build A builds our entire environment, Builds B and C were cloned, but then stripped to only build specific parts and from different branches. Since doing this, we've seen the build time on Build A triple. It is running tf vc workspace /delete on every build.
Build C also deletes it's workspace, but build B does NOT. On the repository tab, all three have Clean set to false.
Is there anywhere to view/set options for the Get Sources task? What would cause one build not to do this and the other two to do it?
There are only three TFVC options in get source step of TFS build pipeline.
No any other extra option to control the get source task.
First suggest you check whether you set Clean = true in Visual Studio Build task, you need to uncheck it here.
If you do not check Clean option and the build always refreshed workspace on local build server and build agent always cleans TFVC Repository. Try this solution: completely wipe the agent directory and start again, this may fix the problem. Another way is reconfigure your build agent and trigger the build again.
Besides, you could also disable the default get source steps in the build definition. And use your own script to do the get source/pull files to bypass workspace mapping issue. How to, please follow: Is it able to ignore/disable the first step Get source in vNext Build?
I found my answer. By adding the System.debug = true I was able to get a view of the actual mappings in the workspace. Then I matched that up against a list of all the mappings in the repository of the build. What I found was that when we map a folder and one of its sub-folders as well, only the parent folder mapping is created in the workspace definition.
When Get Sources runs, it looks for a workspace name that matches and then first does a simple count check. When the number of mappings in the repository doesn't match the number of mappings in the workspace it fails out and immediately deletes and rebuilds the workspace.
By removing the sub-folder mappings, my Get Sources task dropped from 11+ minutes to seconds.
Related
I have our Solution on CI build. That works.
When dev's check in changes, the solution builds, but only for changes to that solution.
How do I get the build definition to build on changes to OTHER folder changes outside of the solution?
Yes, I can add a workspace in the workspace sources tab. But that means all that code is downloaded on every build.
Our solution has over a dozen dependencies. I would like to trigger a build if any of those dependencies change. We don't need the dependency source code to download into the build workspace at all. That's just pointless.
Yes, we have a folder in TFS for our Nuget Packages. We check them in automatically on build (Thank you TFS).
I could just add the NugetPackage workspace to the solution's workspace list, BUT that would result in every version of every dependency getting downloaded into the build workspace.
How can I trigger a build on a change that I do NOT list in the workspaces list of the build definition?
Btw, we are using TFS 2012
I was hoping the Cloaking feature would allow for this, but if it's cloaked, the automated build does not trigger. The automated build only fires if the workspace folder is set to active. Which also means download every NugetPackage in that same folder!
It's not able to trigger a CI build on a change which not list in the workspaces list of the build definition.
A few other things to know
Make sure the folders you include in your trigger are also included in your mappings on the Repository tab(the same to
workspace mappings).
Source Link
As a workaround you could set clean workspace to false, which will not get other unchanged files every time.
If your build process does not require a clean workspace or
repository, you can significantly reduce the time that is required to
run the build setting this parameter value to False.
I have renamed a Jenkins Job from the Jenkins GUI. I changed the Project name in the Configure menu and hitting Save afterwards.
However the workspace name for this Jenkins job has not been changed. What I am finding is upon the job execution a new workspace is getting created with this given new name and none of the contents of the old workspace is getting copied.
So the issue is contents of the old workspace is not copied to the new workspace.
What should I do instead?
I know there are several questions in SO in this area. However those do not answer my question.
Renaming job in jenkins/hudson
Rename a job in Jenkins
So please check this before marking this question as a duplicate.
I was able to workaround this is issue using the Use custom workspace option.
To change this location, I need to choose configure job and click on the Advanced button in the Advanced Project Options section.
After opening the settings, you will find some more configuration options for your job. Look for the Use custom workspace option on the right hand side and check the box.
Reference: Jenkins: Change Workspaces and Build Directory Locations
Workspaces are volatile by nature and may reside on a build node which has gone offline, therefore your build job should not rely on files being present in the workspace. However sometimes you will benefit from a speed-up by reusing unchanged files existing in workspace and decide not to clean them.
When you start a build, a new workspace is (as you noted) created, this is the correct behaviour, you should not need to store files in your workspace between builds but set up your system to load all sources from your vcs. This way you will always be able to make a fresh build from source, there are also a few options available to clear the workspace from old files.
If you do not want to populate the workspace from a source code addon you can always use the custom shell script feature to run a few shell commands to copy the needed files.
I'm using TFS 2013 on premises. I have four build agents configured on a Build machine. Several build definitions compile ASP .NET websites. I configured the msbuild parameters to deploy the IIS application to the integration server, which sits out there in Rackspace.
By default webdeploy does differential deployments by comparing file dates. In my case that's a big plus because copying files from our network to Rackspace takes quite some time. Now, in order to preserve file dates the build agent has to compile the same base set of source code. On every build only the differential source code yields a new DLL, minimizing the number of files deployed.
All of that works fine, with a caveat: a given build definition has to be assigned to a build agent (by agent name or tag). The problem is I create a lot of contingency when all builds assigned to the same agent are queued up. They wait in line until the previous build is done.
In an ideal world any agent should be able to take care of any build, but the source code being compiled has to be the same, regardless of the agent.
I tried changing the working folder of all agents to point to the same location but I get an error because two agents can't be mapped to the same folder. I guess there is one workspace per agent.
Any ideas?
Finally I found a way to do this. Here are all the changes you need to do:
By default the working folder of each agent is $(SystemDrive)\Builds\$(BuildAgentId)\$(BuildDefinitionPath). That means there's one working folder per BuildAgentId. I changed it so that all Agents share the same folder: $(SystemDrive)\Builds\WorkingFolder\$(BuildDefinitionPath)
By default at runtime the workflow creates a workspace that looks like "[BuildDefinitionId][AgentId][MachineName]". Because all agents share the same working folder there's an error trying to create each separate workspace. The solution to this is in the build definition: Edit the xaml and look for an activity called "Get sources from Team Foundation Version Control". There's a property called WrokspaceName. Since I want to have one workspace per build definition I set that property to the BuildDetail.BuildDefinition.Name.
Save your customized build template and create a build that uses it.
Make sure the option "1. TF VersionControl/1. Clean workspace" is set to False. Otherwise the build will wipe out all the source code on every build.
Make sure the option "2. Build/3. Clean build" is set to false. Otherwise the build will wipeout the output binaries on every build.
With this setup you can queue up the same build on any agent, and all of them will point to the same source code and bin output. When the source code changes only the affected binaries are recompiled. I have a custom step in the template that deploys the output files to IIS, to all the servers in our webfarm, using msdeploy.exe. Now my builds+deployments take one or two minutes, because only the dlls or content that changed during the build are synchronized to the servers.
You can't run two build agents in the same folder. The point of build agents is to run multiple builds in parallel, usually on separate PCs. If you try to run them on the same source code, then (a) it's pointless as two build of exactly the same source should produce identical results, and (b) they are almost certainly going to trip over each other and cause the builds to fail or produce unexpected results.
If you want to be able to build and then deploy a series of versions of your codebase, then there are two options:
if you queue up multiple builds, then the last one will "win", so the intermediate builds are of no real value. So if you check in New code before your first build completes, you may as well stop the active build and start a new one. you should be asking yourself why the build is so slow, or why you are checking in changes so often that this is necessary.
if each build produces an incremental update to the deployed result, then you need to pass the output of your builds to some deployment agent that is able to diff it against the deployed version and send only the changes to be deployed. This could be set up to gather results from multiple build agents if that would be beneficial.
but I wonder if perhaps your build Is slow because you are doing a complete build each time (which cleans the build folder, gets all the sources, and does a full rebuild), when what you want is an incremental build (which gets the latest changes, compiles only what is affected, and complete quickly). perhaps you should investigate making your build incremental.
I am just starting with Jenkins 1.487 and wanted to integrate Jenkins in my Ant project. But while configuring it, I can't find any way to make Jenkins re-use an already checked out codebase, instead of downloading a fresh copy relative to its workspace root. Is there a way to do that ?
I tried to specify a custom workspace manually (where my codebase was already checked out), and clicked on 'Build now'. The result was that it wiped out my current checked out code saying
"Checking out a fresh workspace because there's no workspace at /home/daud/Work
Cleaning local Directory ."
Not even a warning..
If you really want to build from an existing checkout somewhere on the file system, then do not use "Source Code Management" section of Jenkins. Leave it as "none"
Go straight to the "Build" section
Click "Add Build Step"
Select Invoke Ant"
Click Advanced
And under "Build File", provide a full path to the ant build file on your file system. You would have to include the drive letter (if on Windows) or a leading / (if on Linux) to break from the Workspace (by default, this path is relative to Workspace). Or use a lot of ../../../.. if needed.
But like others have said, this is not the way a CI system is supposed to be used
The idea behind Jenkins and CI is that it works on a fresh copy of the codebase. Every build done by Jenkins should not depend on any external preconditions and it should be reproducible.
You might want to try using the Clone Workspace SCM Plugin for Jenkins. It will allow you to zip up the workspace from one job and use it to create the workspace for another one. I've used this for downstream jobs that need to act on the work from a previous job.
This is also helpful if you're using something like Git for source control and want to avoid a second Git clone (or SVN checkout). Furthermore, you can limit the content of zip file that is used to recreate the workspace, for example to avoid carrying unnecesary files (e.g. the .git or .svn directories) downstream.
My build agent working directories are starting to take too much space on the disk. I wonder if it is okay for me to delete some old folders, or if I should back them up.
What is the impact of deleting a TFS build agent working directory?
Are the labels affected?
Is the build history affected?
You can do it if your builds are not incremental (incremental get). If your builds always get all source files for every build it is OK to delete working directories. Build history and labels are not affected. Your build logs are in the drop location and it shouldn't be a problem either.
It is a problem for incremental (at least from incremental get) builds. These builds are getting only the latest changes from the TFS source control before every build, not the whole workspace as defined in the build definition's workspace.
Check your build definition's Process → CleanWorkspace settings. If it is set to All, it should not be a problem to delete the build directory.
Assuming I understand your question correctly, make sure you delete the builds through the TFS interface. Don't just delete the folders off the disk if you can help it. The TFS 2010 Build Deletion dialog gives you some options about what to keep and what to delete.
http://blogs.msdn.com/b/jpricket/archive/2009/12/09/tfs-2010-how-about-those-build-delete-options.aspx
You can also set up build retention policies so that old/unused builds automatically get deleted.