I've used jenkins for quite a few years but have never set it up myself which I did at my new job. There are a couple questions and issues that I ran into.
Default workspace location - It seems like the latest Jenkins has the default workspace in Jenkins\jobs[projectName]\workspace and is overwritten (or wiped if selected) for every build. I thought that it should instead be in Jenkins\jobs[projectName]\builds[build_id]\ so that it would be able to store the workspace state for every build for future reference?
Displaying workspace on the project>Build_ID page - This goes along with the previous as I expected each 'workspace' for previous builds to show here. Currently in my setup this individual page gives you nothing except the Git revision and what repo changes triggered the build. As well as console output. Where are the artifacts? Where is the link to this build's workspace that was used?
Archiving Artifacts in builds - When choosing artifacts, the filter doesn't seem to work. My build creates a filestructure with the artifacts in it inside workspace. I want to store this and the artifacts filter says it starts at workspace. So I put in 'artifacts' and nothing gets stores (also where would this get stored?). I have also tried '/artifacts' and 'artifacts/*'.
Any help would be great! Thanks!
It does seem like you are confused about several aspects of Jenkins.. I think your question basically boils down to the following.
What is a difference between a workspace and a build?
So, here are some thoughts on this topic:
Builds are historical data. They (usually) don't change like a workspace does during building/checkout.
Builds contain information about a run (e.g. its status, build number, change log, etc) and any artifacts that you tell it to archive (logs, test results, etc). They (usually) don't contain source code like a workspace.
Builds are stored in the Jenkins\jobs\[projectName]\builds\[build_id]\ directory. This is a directory managed by Jenkins and you (usually) do not need to modify anything in this directory. However, workspaces are directories meant for the build and you can do pretty much anything with them and place them anywhere (it does not need to be in the default Jenkins\jobs\[projectName]\workspace directory.
Workspaces should be able to be wiped at any given time. To restore it, just rebuild the job with the same parameters/revision. If you need to keep something after a build, tell Jenkins to archive it before the build is done.
In regard to saving the entire state, I don't think you need to do that. As mentioned in #4, you should be able to reproduce the same build by kicking off the same revision/parameters as the build in question. If you cannot get back to the original state from the same revision/parameters, then that might be something to strive for as debugging is going to be a nightmare. :)
A workspace is an aspect of the project and not a build and that is why there is no link to the workspace from that page. Again, a build is just saved data from a previous run. A project uses the workspace to build stuff and that is why you can get to the workspace from that page.
In regard to how to save artifacts, you must specify the names of the files you want to save. Unless you are trying to save a file called "artifacts", then you should probably use something else. How about **/*.log for all log files? or **/*.xml for all xml files?
Hope this helps.
Related
In our builds there is sometimes a need (for one or another reason) to copy/create some files that are required during the build, but are not actually artifacts.
As it is a bit problematic to guarantee cleanup/deletion of those files after the build in all cases, they should obviously be stored in some directory that is managed by the build pipeline, so when the build completes or thereafter that directory will be automatically cleaned up.
So far we have been using Build.ArtifactStagingDirectory to get the expected behavior as it is the only one that is
purged before each new build, so you don't have to clean it up yourself.
Still, as such files are not really artifacts, it seems to be slightly inappropriate.
Perhaps there is a better alternative?
$(Build.BinariesDirectory) can also be configured to be purged before each build. That may be more appropriate for temporary build outputs.
However, there's no right answer to this.
I have a TFS 2017 instance running on windows server 2012 R2 with a test box running windows 10.
I am running into a very odd issue. Most of my automated CodedUI tests are running pretty well. However, I have noticed that code changes are not always picked up by TFS when it performs a new build, at least not in the testing code area.
The builds themselves work well, and new code always gets incorporated for those. However, when the latest build triggers a release containing CodedUI tests, those tests do not always grab the latest build.
I have noticed this primarily in my App.config file which contains connection strings that are not being updated. In one case I had three tests that ran apparently successfully, but then they ran again using the values from the old App.config file.
I also have found that changes to the [TestCategory()] attribute are not always picked up either. I use that category to specify which tests I want run in a particular release build. I use variations on the same word for my categories: CodedUI, CodedUIExtended, CodedUIStage. At first I thought the system was doing some sort of StartsWith and picking up the other names, but when I tell it to run CodedUI it is running both the CodedUI and CodedUIStage categories.
[TestCategory("CodedUI"), TestMethod]
public void UI_Login_AdminAuthenticate()
{
...
}
Because the CodedUIStage categories were recently changed and used to be CodedUI that has led me in the direction of suspecting some sort of caching being used in TFS.
Can anyone shed some light on why my category and app.config changes are not being picked up correctly? What is causing this, and could it be happening to the code itself as well when I attempt to deploy a fix/correction?
EDIT:
As suggested, I tried checking the clean option on my TFS build configuration, however it had no affect.
The release, which is triggered immediately upon build does show it is using the newly finished build number.
And looking at the artifacts, the test dll has the current date modified, so it looks like it was just created.
The test.dll.config seems to be the issue. In the artifacts it has a very old Date modified and is not the current version that is checked into TFS. Typically this would feel like a clean issue, however TFS always deletes and re-copies all destination files when doing a build or Release. I have verified that by watching the files being deleted and re-created on the file system during the process.
C:\agent_work\r6\a\ [artifact_name]\bin
EDIT2:
With a little more exploration, the build artifacts are correct. It is when the Release copies those artifacts into the release process that the problem happens.
A week ago I renamed the folder in TFS containing my test project inside the solution. The old folder name is showing up in the artifacts the Release copies into itself. The new folder name is also showing up, which means I now have two dlls, and that is causing problems.
I am not sure where the Release is finding this copy of the old folder. I explicitly deleted it from the Release's copy of the build artifacts and re-ran the build and release and it showed back up.
Thanks to the suggestions from Daniel, I eventually figured out that after creating the artifacts, my build process was then publishing those to a separate place on the file system.
Unfortunately, the Copy and Publish Build Artifacts task does not have a clean feature like the basic Copy Files task does.
As such, whenever files are removed from the build they still exist in that location when the Release process goes to try and grab what it thinks the artifacts are. And so it ends up grabbing extra files.
Manually deleting the old files from that secondary artifact destination location solved the issue.
Our builds generally have a mish mash of work items and commits associated with them and I cannot tell how TFS determines what to add. We are using TFS 2015 update 3 and TFVC.
When a build runs, it gets code from a location somewhere in the branching and folder of TFVC. Typically, something like "root\dev\src\component name" in this way we avoid getting all of the code in our repository and we have CI set up to run so that any changes in this folder will result in a CI build running.
We also run daily builds which run more tests and create a release package that is used by TFS Release Management. I would expect that any changes to code inside of the folder defined in setting up the repository for this build to be included in the associated change-sets of a build. I also expect that any changes checked-in outside of these branches would not be associated. But this is not the case. We see commits from across the entire project.
Does anyone know how this is supposed to work?
I am not sure if this should go in the question or the answer but I have found some additional information, thanks to the hints provided in the answers below.
It appears that the source settings will take the common root between mapped folders of the repository settings, so if I have 2 folders $/Relo/Dev/B1/src/Claims.Services and $/Relo/Dev/B1/src/PSScripts it will take the common root $/Relo/Dev/B1/src as the source settings and include any changes from that folder down within the build. Can anyone confirm this? Of course thats not what I want to have happen. In the History tab of the build definition if I looked at the diff I can see a field "defaultBranch" in the json which seems to be the value that controls this, is there any way to update this field directly?
TFS determines what changesets should be mapped to a build based on the Source Repository Mappings (Build vNext) in the build definition and the last successful build.
So, you will see a list of the changesets with files committed in the lowest common base of any of the mapped folders including all their descendents, since the latest successful build. Whenever you get a successful build (I hope that it happens more often than failing ones ;-)) the list will shorten and only show the last check-in.
Example mappings below will result in any changeset made to anything below $/Relo/Dev/B1/src (because it is the lowest common base):
$/Relo/Dev/B1/src/Claims.Services
$/Relo/Dev/B1/src/PSScripts
Similar it will pick up all the related work items to the above changesets.
This is what should happen. If you see something else, I would have a closer look at the Repository Mappings or Source Settings of the build definition.
#Noel - I guess you are using vNext build and not XAML builds. Or are you using a mix of XAML and vNext?
In general a scheduled TFS build will associate all changes which were not associated in the last successful run of the same build.
I suggest you check once again if the source folder locations are the same for CI build and Daily build?
we have setup TFS Build for our project, but on every build the system copies the whole repository and then compiles our solution. How can we make sure TFS Build only downloads the files needed for the solution without having to cloak each un-needed directory manually ? Now it downloads over 2GIGs of data just to compile a project that is less then 100mb in size (source files). The other data are test databases and files that are not needed for the automatic build.
EDIT:
some further investigation let me to some keywords for searching. These posts are helping out:
Team Build - Get Workspace - get latest from specific paths, NOT everything
TFS Build and workspace
still investigating though. Any comments are welcome.
EDIT:
An option is to replace CreateWorkspace in the Build process definition with my own extended activity. I'm hoping to find out that somebody already did.. basically you would use the VersionControlServer object to download the necessary files instead of the whole workspace.
EDIT
There is currently no real good answer / solution to this. I gave some options and the people that responded gave some alternatives, but you can't easily change the TFS Build process to just download the data that is part of the solution instead of the whole repository. So be aware when you are building your repository.
You want to set the Build Definition mapping to only include the source you wish to compile. This means that you don't have to cloak any thing.
Edit Build Configuration
Click on Source Settings (VS 2012), Workspace (VS 2010)
An example specific mapping would look like this:
StatusSource Control Folder Build Agent Folder
Active$/Path/To/The/SolutionOrProjectFolder $(SourceDir)\
This will make the workspace for this build be limited to the solution that you wish to build. Therefore only AssemblyInfo files under that will be visible to your build activity.
If you cannot do this due to how your source control is setup, then I would suggest restructuring your folders within your Source Control.
If you have more than one Build Agent, you should limit the number of agents that the build definition can run against. That will stop multiple copies of the same source been downloaded on to the build machine(s).
The next part you have already answered in you question, by changing the "Clean Workspace" option in your Build Definition to None the build agent will only download the changesets between the current and last build.
Having real problems creating artifacts in teamcity 6.5 (using TFS & MSBuild as the buildrunner if it makes any odds, which it probably does as any examples I find seem to use SVN...).
The Build works, so long as I enter no checkout rules.
If I understand it, I'll need to set up some artifacts, that themselves rely on checkout rules(?).
I have two builds that are identical other than the way they are kicked off.
One is initiated on check-in
One is initiated manually from within TC. This build is the Test Build
Assembly version numbers come from a single versioninfo.cs file that is a linked file in all projects in the solution. This method is detailed here : http://www.codeproject.com/Articles/328977/The-Right-Way-to-Version-Your-Assemblies and holds the version number thus:
[assembly: AssemblyFileVersion("9.1.0.0")]
Ultimately, I'm unable to copy the output of the test build to another location.
As it stands, the only output of a build is in the teamcity data directory, for example :
C:\TeamCity\buildAgent\work\ceaaf65dc87ff856\Project1\bin\Debug
C:\TeamCity\buildAgent\work\ceaaf65dc87ff856\Project2\bin\Debug
etc
I'd like to copy the output files (exes and DLLs) to an output folder which has the build number of the build on it
For arguments sake, lets say for the version number above, this would be to
c:\BuildServer_Output\SolutionName\9.1.0.0
Currently I have not been able to create artifact paths that actually do anything - i.e. to copy anything anywhere.
For instance I have acoupe of artifact paths, but nothing ever gets put into C:\BuildServer_TestBuilds -
+:Accounts\bin\debug* => C:\BuildServer_TestBuilds
+:BackOffice\bin\debug* => C:\BuildServer_TestBuilds
Am I getting no artifacts (and my artifact paths therefore ignored) because I have no checkout rules?
Any help would be appreciated.
I am pretty sure artifacts and checkout rules are completely independent. Artifacts just deal with what has been built. Checkout rules tell teamcity how to react to and checkout changes in the VCS.
It looks like your artifact paths are beginning with absolute paths. I have always found it easier to use relative paths with wildcards. That way I don't need to worry about where teamcity put the build. We use the following to get all dlls and exes to one folder
**\bin\Debug\*.*=>deploymentdir
Our build configuration page has an artifacts link and when we open it it will have things like
deploymentdir\common\bin\debug\common.dll
deploymentdir\common\bin\debug\common.pdb
deploymentdir\runner\bin\debug\runner.exe
In one of our other builds we use an msbuild script to flatten our output before putting it through the artifact process.
We do use checkout rules but we have not had to change our artifact paths to accommodate them.