I'm facing the following issue. We have a Gated Check-In setup which means changes are not directly checked-in, but a Shelveset is created and a build is run against that shelve. Only if the build is successful, the changes are checked in.
Now we setup the build definition to only map as few things as needed to build, as we want to keep the time it needs to run at a very minimum. The problem now is that if you include in your change anything that is not mapped, the build will fail in the GetSources task, as it fails to unshelve and gives a folder mapping error.
For a better understnanding, imagine the following structure:
- MyRepository
-- Sources
--- MyCodeFiles
-- Documents
--- SomeDocuments
So in the BuildDefinition we only map the Sources directory, as this contains the files we need to build. You now have to make sure to never include any change outside of Sources, for example within Documents, otherwise the Build will immediately fail.
I already searched and found the description of the unshelve command: https://learn.microsoft.com/en-us/vsts/tfvc/unshelve-command?view=vsts
According to this description it should be possible to specifiy the itemSpec that
Identifies the file or folder revisions to unshelve into the current workspace
So it seemes like the unshelve command would actually support to "partially" unshelve. However I did not see any way on how to achieve that with a vNext build, as I could not find any infos on how the GetSources task is working.
Is there something I missed or is this with the current implementation simply not possible to achieve?
I'm afraid to say you haven't missed anything, it's just not possible in the current implementation. There is a more detailed discussion here:
https://github.com/Microsoft/vsts-agent/issues/1087
Related
Our builds generally have a mish mash of work items and commits associated with them and I cannot tell how TFS determines what to add. We are using TFS 2015 update 3 and TFVC.
When a build runs, it gets code from a location somewhere in the branching and folder of TFVC. Typically, something like "root\dev\src\component name" in this way we avoid getting all of the code in our repository and we have CI set up to run so that any changes in this folder will result in a CI build running.
We also run daily builds which run more tests and create a release package that is used by TFS Release Management. I would expect that any changes to code inside of the folder defined in setting up the repository for this build to be included in the associated change-sets of a build. I also expect that any changes checked-in outside of these branches would not be associated. But this is not the case. We see commits from across the entire project.
Does anyone know how this is supposed to work?
I am not sure if this should go in the question or the answer but I have found some additional information, thanks to the hints provided in the answers below.
It appears that the source settings will take the common root between mapped folders of the repository settings, so if I have 2 folders $/Relo/Dev/B1/src/Claims.Services and $/Relo/Dev/B1/src/PSScripts it will take the common root $/Relo/Dev/B1/src as the source settings and include any changes from that folder down within the build. Can anyone confirm this? Of course thats not what I want to have happen. In the History tab of the build definition if I looked at the diff I can see a field "defaultBranch" in the json which seems to be the value that controls this, is there any way to update this field directly?
TFS determines what changesets should be mapped to a build based on the Source Repository Mappings (Build vNext) in the build definition and the last successful build.
So, you will see a list of the changesets with files committed in the lowest common base of any of the mapped folders including all their descendents, since the latest successful build. Whenever you get a successful build (I hope that it happens more often than failing ones ;-)) the list will shorten and only show the last check-in.
Example mappings below will result in any changeset made to anything below $/Relo/Dev/B1/src (because it is the lowest common base):
$/Relo/Dev/B1/src/Claims.Services
$/Relo/Dev/B1/src/PSScripts
Similar it will pick up all the related work items to the above changesets.
This is what should happen. If you see something else, I would have a closer look at the Repository Mappings or Source Settings of the build definition.
#Noel - I guess you are using vNext build and not XAML builds. Or are you using a mix of XAML and vNext?
In general a scheduled TFS build will associate all changes which were not associated in the last successful run of the same build.
I suggest you check once again if the source folder locations are the same for CI build and Daily build?
I've used jenkins for quite a few years but have never set it up myself which I did at my new job. There are a couple questions and issues that I ran into.
Default workspace location - It seems like the latest Jenkins has the default workspace in Jenkins\jobs[projectName]\workspace and is overwritten (or wiped if selected) for every build. I thought that it should instead be in Jenkins\jobs[projectName]\builds[build_id]\ so that it would be able to store the workspace state for every build for future reference?
Displaying workspace on the project>Build_ID page - This goes along with the previous as I expected each 'workspace' for previous builds to show here. Currently in my setup this individual page gives you nothing except the Git revision and what repo changes triggered the build. As well as console output. Where are the artifacts? Where is the link to this build's workspace that was used?
Archiving Artifacts in builds - When choosing artifacts, the filter doesn't seem to work. My build creates a filestructure with the artifacts in it inside workspace. I want to store this and the artifacts filter says it starts at workspace. So I put in 'artifacts' and nothing gets stores (also where would this get stored?). I have also tried '/artifacts' and 'artifacts/*'.
Any help would be great! Thanks!
It does seem like you are confused about several aspects of Jenkins.. I think your question basically boils down to the following.
What is a difference between a workspace and a build?
So, here are some thoughts on this topic:
Builds are historical data. They (usually) don't change like a workspace does during building/checkout.
Builds contain information about a run (e.g. its status, build number, change log, etc) and any artifacts that you tell it to archive (logs, test results, etc). They (usually) don't contain source code like a workspace.
Builds are stored in the Jenkins\jobs\[projectName]\builds\[build_id]\ directory. This is a directory managed by Jenkins and you (usually) do not need to modify anything in this directory. However, workspaces are directories meant for the build and you can do pretty much anything with them and place them anywhere (it does not need to be in the default Jenkins\jobs\[projectName]\workspace directory.
Workspaces should be able to be wiped at any given time. To restore it, just rebuild the job with the same parameters/revision. If you need to keep something after a build, tell Jenkins to archive it before the build is done.
In regard to saving the entire state, I don't think you need to do that. As mentioned in #4, you should be able to reproduce the same build by kicking off the same revision/parameters as the build in question. If you cannot get back to the original state from the same revision/parameters, then that might be something to strive for as debugging is going to be a nightmare. :)
A workspace is an aspect of the project and not a build and that is why there is no link to the workspace from that page. Again, a build is just saved data from a previous run. A project uses the workspace to build stuff and that is why you can get to the workspace from that page.
In regard to how to save artifacts, you must specify the names of the files you want to save. Unless you are trying to save a file called "artifacts", then you should probably use something else. How about **/*.log for all log files? or **/*.xml for all xml files?
Hope this helps.
we have setup TFS Build for our project, but on every build the system copies the whole repository and then compiles our solution. How can we make sure TFS Build only downloads the files needed for the solution without having to cloak each un-needed directory manually ? Now it downloads over 2GIGs of data just to compile a project that is less then 100mb in size (source files). The other data are test databases and files that are not needed for the automatic build.
EDIT:
some further investigation let me to some keywords for searching. These posts are helping out:
Team Build - Get Workspace - get latest from specific paths, NOT everything
TFS Build and workspace
still investigating though. Any comments are welcome.
EDIT:
An option is to replace CreateWorkspace in the Build process definition with my own extended activity. I'm hoping to find out that somebody already did.. basically you would use the VersionControlServer object to download the necessary files instead of the whole workspace.
EDIT
There is currently no real good answer / solution to this. I gave some options and the people that responded gave some alternatives, but you can't easily change the TFS Build process to just download the data that is part of the solution instead of the whole repository. So be aware when you are building your repository.
You want to set the Build Definition mapping to only include the source you wish to compile. This means that you don't have to cloak any thing.
Edit Build Configuration
Click on Source Settings (VS 2012), Workspace (VS 2010)
An example specific mapping would look like this:
StatusSource Control Folder Build Agent Folder
Active$/Path/To/The/SolutionOrProjectFolder $(SourceDir)\
This will make the workspace for this build be limited to the solution that you wish to build. Therefore only AssemblyInfo files under that will be visible to your build activity.
If you cannot do this due to how your source control is setup, then I would suggest restructuring your folders within your Source Control.
If you have more than one Build Agent, you should limit the number of agents that the build definition can run against. That will stop multiple copies of the same source been downloaded on to the build machine(s).
The next part you have already answered in you question, by changing the "Clean Workspace" option in your Build Definition to None the build agent will only download the changesets between the current and last build.
Is there a way to avoid the automated build when we checkin only js or css files?
It doesn't need to build anything, I would like to skip the build for these files.
I don't think it is possible to do what you are wanting to do- I don't think gated check-ins is that mature yet. There are quite a few outstanding enhancement requests for gated check-ins. It might be worth adding this as a request. Even cloaked folders are at the folder level and not the file level.
From MSDN:
Automatically Run Gated Check-in
Builds
A gated check-in build is run
automatically when either of the
following events occur:
A build has been defined with the Gated Check-in check box selected on
the Trigger tab of the build
definition.
Someone attempts to check in one or more changes that intersect with
any of the mapped folders in the
Workspace tab of the build definition.
Even if you add a ***NO_CI*** tag and try to checkin, it will stll fire the check to build changes.
The obvious workaround is of course to check the checkbox ignoring the need to do a build (a path I am sure you are trying to avoid).
If we take a look at the UI around editing a build definition, there aren't optional parameters for rules etcetera. All that matters is the mapped folders.
You can however set specific folders to not build by cloaking a workspace folder. However, if you do that, it will not pick up those files for builds at all. So if you had all your JS files in one folder and excluded it from the build using the cloaked approach builds would fail when you actually wanted to run the build.
In the TFS 2010 build definition window, under “Process” there are two required items. They are “Configurations to build” and “Projects to build”. Under projects to build, it will allow me to enter something like:
$/TeamProject/Area1/Area2/*
However, this doesn’t seem to do what I expect. The build fails because it’s looking for:
$/TeamProject/Area1/Area2/Sources/*
What I am trying to achieve by this is to build all the solutions held under this area. For example, I have:
$/TeamProject/Area1/Area2/Solution1/Solution1.sln
$/TeamProject/Area1/Area2/Solution2/Solution2.sln
$/TeamProject/Area1/Area2/Solution3/Solution3.sln
There are many more solutions than this, which is why I’m looking for a way to build all solutions under the specified path recursively. Is there a way to do this in TFS 2010?
You can modify the process template. Expand it with the Matching files (I don't have the exact naming now) activity. Add a parameter that passes the information you set in the build defintion to the MachingFiles actvity. Then pass into the build solution activity instead of the argument that you enter in the build definition the files that is found by the MatchingFiles activity.
Now add a dummy solution in the build definition for the solution to build (it is not used anymore).
See the blog post series on the build customization for more information on customizing the build process template.
FWIW,
I've got: "configurations to build" blank
and under "projects to build" I've added my solutions via the ellipsis button
I would setup mappings for
$/TeamProject/Area1/Area2/Solution1/
$/TeamProject/Area1/Area2/Solution2/
$/TeamProject/Area1/Area2/Solution3/
Then in the build definitions enter the three projects to build
$/TeamProject/Area1/Area2/Solution1/Solution1.sln
$/TeamProject/Area1/Area2/Solution2/Solution2.sln
$/TeamProject/Area1/Area2/Solution3/Solution3.sln
You can leave the configurations to build as blank, or if you want to do a certain build you can set it to (for example) something like Debug|Mixed Platforms (check your Configuration Manager... for the solutions you are building to see what is valid)
Alternatively, you can just map the following (depending on how much you have in this folder, if you have Solutions 4+ that you don't want to trigger builds on, don't do it at this level)
$/TeamProject/Area1/Area2
And have one solution which contains the Solution1, Solution2 and Solution3, and build that instead.
By default building your Solution1 which is mapped to
$/TeamProject/Area1/Area2
On a build agent with a working directory that is going to looks something like:
$(SystemDrive)\Builds\$(BuildAgentId)\$(BuildDefinitionPath)
You'll end up with it being build under
C:\Builds\1\Solution1\Binaries
C:\Builds\1\Solution1\Sources
C:\Builds\1\Solution1\TestResults
Which is why you want to make sure that your OutDir's etc are all correct and not hard coded!
If you have a look while building, you'll see the build agent populating the Sources folder, and it should (if configured correctly) put all outputs into the Binaries folder (and then copy them to the Drop Folder configured under Build Defaults in TFS.