I have a repo that has 2 subfolders $/Repo/project and $/Repo/thirdparty. I need to pull both of those into Jenkins for a single build. Naturally I tried just pulling $/Repo, but this gives me a bunch of other projects along with false polls (it will build every time ANYTHING is checked into $/Repo). I have tried using the multi-scm plugin which works, but does not save the configuration (annoying, but not unusable). I tried using the regular tfs plugin and manually putting the calls for the other repo into a windows command (this did not work even through i bound them them to different folders).
What is the best way to approach this? Some sort of subjob that pulls third party? Fix the multiple scm plugin? Is there some tfs command or trigger to pull a different repo when you pull a project?
I was able to get this working with a job pipeline. It's kinda hacky, but it works.
The program I'm trying to build uses $/Department/Framework/Main (as workspace\Framework), and $/Department/Products/TheProgram/Main (as workspace\TheProgram).
I created three jobs in Jenkins, each "downstream" of the other:
Framework-Get: normal source code triggering on TFS' Project Path of $/Department/Framework/Main. No build step.
TheProgram-Get: normal source code triggering on TFS' Product Path of $/Department/Products/TheProgram. No build step.
TheProgram-Build: No source code control. But the build steps xcopy's the source from the above two steps. Then, you can run a normal build step.
TheProgram-Build's first build step is a windows batch command:
REM ====================================
REM First Get the Framework folder:
rmdir /s/q Framework
mkdir Framework
xcopy /y /q /e ..\..\Framework-Get\Workspace\Framework Framework
REM ====================================
REM Then Get the TheProgram Folder:
rmdir /s/q TheProgram
mkdir TheProgram
xcopy /y /q /e ..\..\TheProgram-Get\Workspace\TheProgram TheProgram
The second build step was a simple call to ant. But you could use msbuild or whatever you like here.
The TFS pluging for Jenkins currently does not support checking out the sources from multiple locations. multiple-scm-plugin might be the answer, but as You pointed out in the question - it's really not an option at this point. There are really, as far I can see it, only to possible solutions for you to test out:
Create a workspace within TFS that will include all the neccesary imports. I use this functionality in my every-day encounters with TFS, although I have never a chance to use that with Jenkins plugin. It might work, it might not.
You can use, and please - this is a quite serious option, at least for me - git. There is a git-tfs and import all of the required projects into git repository. And having them in git will open a bunch of possibilities for you, including using separate repos for every folder, using git modules, git externals... and so on. So, at least for me, it is a valid option, although it seems like an ugly workaround at the first look...
The TFS plugin supports the ability to cloak folders in your $\Repo that you are not interested in. Checkins to cloaked folders will not trigger a build. Unfortunately that may be a lot of folders and you are only interested in two - you would need to maintain the list of cloaked folders as new ones are added.
We avoid the TFS plugin and instead scripted the setup of our TFS workspaces via powershell step using the tfs commandline. Each build specifies the folders it wants and the script takes care to cloak/uncloak the remainder.
My solution to this is to create two Jobs, one that just download your dependency and another to make the build.
In my case I managed the build with Maven properties, for example:
pom.xml
<properties>
<my.dir>../MyDir</wsdl.dir>
</properties>
Jenkins Build
Goals clean package -U -Dmy.dir=${WORKSPACE}/../../another-build/workspace/MyDir
I had to create a workaround myself for Jenkins. This was achieved using both TF and the powershell Snapin Microsoft.TeamFoundation.PowerShell.
Basically the workflow is as follows :
Get-TFsWorkspace (Powershell : To check for the workspace)
TF Workspace /new (To Create a workspace)
TF Workfold /unmap (use this to remove the default $/ mapping which is made during workspace creation)
TF Workfold /map (To map specific locations, ie $/Repo/project)
TF Scorch (to remove any artifacts if there are any)
TF Get (To get the code)
There may be other methods that people have, but this would allow you to use the tf Workfold /cloak functionality as well.
Cheers,
Hope this helps.
Can confirm that that Multiple SCM 0.5 works with the Team Foundation Server plug-in 4.0
The polling does seem to break however.
Related
I am new to the world of scripting with TFS2015. I created a script that builds all of the projects within my solution (it is a rather large solution) and puts it out in a shared folder (where each project has its own subfolder).
I would like to create a separate script for each project that simply copies the bin folder from the shared and pastes it out on my Test environment. I rarely need to deploy everything, so the idea is one build...multiple deploys.
However, when I run my deploy script using the Copy Files step it is doing another build. Although it copies the files that I expect, it is after a full build that creates the folder structure for the build.
Am I able to make the Copy Files step NOT do a Build?
Here is the steps that my script is curently doing:
As you can see, there is only one step (Copy Files) but it still does the Get sources and copies everything into a new folder on the build box like so (where the number keeps incrementing up with each run of the script):
I just want to copy the files from the Source to the Target and not do a build or Get Sources.
It looks like you're still on TFS 2015 RTM or Update 1. Which is already pretty old technology if you compare it to the lifetime of the new build system which was introduced with this version.
TFS 2015 update 2 has introduced a similar system to the Build pipelines to orchestrate Releases. This doesn't require you to map any workspaces or git repositories and can act on the artefacts of your builds or simply on the contents of file shares.
It makes sense that a Build has to build something and in order to build something, it has to get the things to build. If you're actually not building something, then you're probably deploying or releasing or packaging something else. Hence the distinction between Build and Release pipelines.
TFS 2017+ has an option to disable the syncing of sources. Primarily to allow people to get the sources themselves in creative ways (e.g. a custom powershell script that invokes git.exe).
My primary advice would be to upgrade to TFS 2018 update 3 or at least TFS 2017 update 3.1, worst case TFS 2015 update 4.1. The fact that versions older than update 2015.4.1 have a known XSS scripting security bug may be reason enough to convince your organisation to perform this update.
Barring that option you're left with one solution:
Link your build definition either to a git repository with only a single commit (If I remember correctly the 2015 agent still crashes when syncing an empty Git repo) or link it to a TFVC repository and set the workspace settings to cloak everything. This essentially causes the build to sync an empty folder, which it can cache, before calling your powershell script.
I have two different TFS instances. Both use TFVC as a source control.
I want to set up a build definition on TFS1, so that it gets sources from TFS2. Is it possible to do it?
There is no option "Remote TFVC", only "Remove Git".
Thank you.
As far as I can tell, it isn't supported in either TFS On-Prem, or VSTS. Depending on your needs, circumstances and limitations, you might consider one of the following options (all are more or less trade-offs):
Migrate sources from TFS2 to TFS1 (the one to run builds)
This seems to be the correct thing to do, but it is a time-consuming and error-prone process
Fake the Get Sources action and do the real get/checkout in the first build step
You can configure the Get Sources step to address any Git repo in the same team project, and turn the Don't sync sources flag ON. Then, in the first real build step, run command-line Get (something like this)
You may use custom build step (cmd or bat file) and download files from remote server. Examples:
Copy files from tfs versioncontrol to directory with PowerShell
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/9559f7a5-405a-456c-a66b-8123d52ed23a/how-to-copy-a-folder-from-tfs-source-control-to-shared-location-with-powershell-script?forum=tfsgeneral
You could just add the extension to get them from an external vsts/tfs.
Go to the marketplace and search for them (see image).
You will have to configure a endpoint to your external vsts/tfs, but it is quit easy to do.
You can install this extension: TFS artifacts for Release Management.
Then add a Download Artifacts-External TFVC task from Utility.
I have a problem with Jenkins building my source code. I'm using a lib repo and a repo for my code. I want Jenkins to build the project if anything in one of the repos changes.
Does anyone has some pointer how to solve this? I managed to get it working in the case that everything is in only one repo, but I want to separate the lib and the project code.
Unfortunately,the TFS pluging for Jenkins currently does not support checking out the sources from multiple locations.
However, as a work around you could use the command line to create any sort of workspace that you like, and even copy a template workspace that you have lying around.
To achieved this use both TF and the powershell Snapin Microsoft.TeamFoundation.PowerShell.
Basically the workflow is as follows :
Get-TFsWorkspace (Powershell : To check for the workspace)
TF Workspace /new (To Create a workspace)
TF Workfold /unmap (use this to remove the default $/ mapping which is
made during workspace creation)
TF Workfold /map (To map specific locations, ie $/Repo/project)
TF Scorch (to remove any artifacts if there are any)
TF Get (To get the code)
More details please refer this answer in a similar question.
I have a build job in Jenkins, which is based on a git project.
Looking at the changes page for each build, I can see the commits that are new to this version, who contributed them, and the list of files modified.
What I'm missing is the option to see the actual diff - what lines of code were changed.
How can this be achieved?
I'm using a local git repo hosted on the same PC as the jenkins server
You can use Last Changes Plugin to accomplish that.
Just install it and make sure to choose from Post-build Actions drop down menu the option Publish Last Changes.
I am just starting with Jenkins 1.487 and wanted to integrate Jenkins in my Ant project. But while configuring it, I can't find any way to make Jenkins re-use an already checked out codebase, instead of downloading a fresh copy relative to its workspace root. Is there a way to do that ?
I tried to specify a custom workspace manually (where my codebase was already checked out), and clicked on 'Build now'. The result was that it wiped out my current checked out code saying
"Checking out a fresh workspace because there's no workspace at /home/daud/Work
Cleaning local Directory ."
Not even a warning..
If you really want to build from an existing checkout somewhere on the file system, then do not use "Source Code Management" section of Jenkins. Leave it as "none"
Go straight to the "Build" section
Click "Add Build Step"
Select Invoke Ant"
Click Advanced
And under "Build File", provide a full path to the ant build file on your file system. You would have to include the drive letter (if on Windows) or a leading / (if on Linux) to break from the Workspace (by default, this path is relative to Workspace). Or use a lot of ../../../.. if needed.
But like others have said, this is not the way a CI system is supposed to be used
The idea behind Jenkins and CI is that it works on a fresh copy of the codebase. Every build done by Jenkins should not depend on any external preconditions and it should be reproducible.
You might want to try using the Clone Workspace SCM Plugin for Jenkins. It will allow you to zip up the workspace from one job and use it to create the workspace for another one. I've used this for downstream jobs that need to act on the work from a previous job.
This is also helpful if you're using something like Git for source control and want to avoid a second Git clone (or SVN checkout). Furthermore, you can limit the content of zip file that is used to recreate the workspace, for example to avoid carrying unnecesary files (e.g. the .git or .svn directories) downstream.