I'm trying to write a build step within TFS that relies on knowing where the Build agent has nuget.exe stored (the standard nuget-install step mucks around with the order of arguments in a way that breaks build execution, so I want to run the exe myself using one of the batch/shell/ps steps).
It would seem that setting up a capability on the Build Agent with that path would make sense, but I cannot seem to reference the value in any of my build steps, and I cannot find anything helpful on MSDN.
I'm expecting it to be something like $(Env.MyUserCapability), but it never resolves to the value.
Is it possible to retrieve a capability value within a build step? And if so, how do you do it? And if not, what is a viable alternative?
The user-defined capabilities are metadata only. But you can set a global environment variable (e.g. NUGET) and set that to a path to a nuget.exe, when you restart the agent, the machine-wide environment is then discovered as capability and you can then use it.
If you are writing a custom task, you can also add a nuget.exe to the task that will be downloaded to the executing agent.
UPDATE: I made a public extension out of this.
UPDATE: this works in Azure DevOps 2019.
In TFS 2018u1, the following works:
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Common"
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"
Add-Type -Assembly "Microsoft.TeamFoundation.DistributedTask.WebApi"
$VSS = Get-VssConnection -TaskContext $distributedTaskContext
$AgentCli = $VSS.GetClient([Microsoft.TeamFoundation.DistributedTask.WebApi.TaskAgentHttpClient])
$AgentConfig = Get-Content "$Env:AGENT_HOMEDIRECTORY\.agent" -Raw | ConvertFrom-Json
$Agent = $AgentCli.GetAgentAsync($AgentConfig.PoolId, $Env:AGENT_ID, $TRUE, $FALSE, $NULL, $NULL, [System.Threading.CancellationToken]::None).GetAwaiter().GetResult()
if($Agent.UserCapabilities.MyCapability)
{
Write-Host "Got the capability!";
}
The long string of default arguments ending with CancellationToken::None is for compatibility with Powershell 4. PS4 doesn't support default values for value-typed method parameters, PS5 does.
This snippet does something very questionable - it relies on the location and the structure of the agent configuration file. This is fragile. The problem is that the GetAgentAsync method requires both pool ID and the agent ID, and the former is not exposed in the environment variables. A slightly less hackish approach would check all pools and find the right one by the agent ID:
$Pools = $AgentCli.GetAgentPoolsAsync($NULL, $NULL, $NULL, $NULL, $NULL, [System.Threading.CancellationToken]::None).GetAwaiter().GetResult()
$Demands = New-Object 'System.Collections.Generic.List[string]'
foreach($Pool in $Pools)
{
$Agent = $AgentCli.GetAgentsAsync($Pool.ID, $Env:AGENT_NAME, $TRUE, $FALSE, $NULL, $Demands, $NULL, [System.Threading.CancellationToken]::None).Result
if($Agent -and $Agent.Id -eq $Env:AGENT_ID)
{
Break
}
}
This relies on another undocumented implementation detail, specifically that agent IDs are globally unique. This seems to hold as late as TFS 2018, but who knows.
When you employ the $distributedTaskContext, the task is connecting back to TFS with an artificial user identity, "Project Collection Build Service" (not with the agent service account). There's one user like that in each collection, they're distinct. In order to allow tasks running in releases in a collection to query the agent for user capabilities, you need to grant the Reader role to the relevant pool(s) (or all pools) to the user account called "Project Collection Build Service (TheCollectionName)" from that collection.
It also looks like some actions also grant an implicit Reader role on a pool to the task identity.
Alternatively, you can construct a VssConnection from scratch with Windows credentials, and grant the agent account(s) Reader role on the pool(s).
Related
We were building our solution without any "Pipeline" in Jenkins until recently, so I'm currently in the progress to move our build to multibranch pipelines.
The issue that I'm running into is that we have a lot of structure une our solution(lot of subfolder, and sometimes some big names).
Currently, the jenkins pipeline extract everything in a folder that looks like:
D:\ws\ght-build_feature_pipelines-TMQ33LB5OQIQ5VXVMFKFDG2HWCD4MUOGEGUWJUOMZ5D2GI42BIQA
Which is very-long, and now we are reaching the 260 characters limit of MSBuild:
C:\Program Files (x86)\Microsoft Visual
Studio\2017\Professional\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(2991,5):
error MSB3553: Resource file
"obj\Release\xx.aaaaaaaaaa.yyy.bbbbbb.dddddddddddddd.yyyyyyy.vvv.dddddddddd.Resources.resources"
has an invalid name. The item metadata "%(FullPath)" cannot be applied
to the path
"obj\Release\xx.aaaaaaaaaa.yyy.bbbbbb.dddddddddddddd.yyyyyyy.vvv.dddddddddd.Resources.resources".
The specified path, file name, or both are too long. The fully
qualified file name must be less than 260 characters, and the
directory name must be less than 248 characters.
[D:\ws\ght-build_feature_pipelines-TMQ33LB5OQIQ5VXVMFKFDG2HWCD4MUOGEGUWJUOMZ5D2GI42BIQA\Src\bbbbbb\dddddd\dddddddddddddd\yyyyyyy\xx.aaaaaaaaaa.yyy.bbbbbb.dddddddddddddd.yyyyyyy.vvv\xx.aaaaaaaaaa.yyy.bbbbbb.dddddddddddddd.yyyyyyy.vvv.csproj]
We have so much cases where the length is big that it's really a big job to refactor everything, so I'm looking on how to specify to jenkins a smaller path?
What I finally did:
pipeline {
agent {
node{
label 'windows-node'
customWorkspace "D:\\ws\\${env.BRANCH_NAME}"
}
}
options{
skipDefaultCheckout()
}
...
}
And I've a step that does the checkout. It was easier for me to have a "per-job" behavior, without touching jenkins global settings.
Update (for any recent Jenkins instances)
Turns out that with recent Jenkins versions PATH_MAX seems to be ignored.
The only thing it does: Issue a warning in the Jenkins log when smaller than a certain value, which actually does not matter - as the setting itself will anyways be ignored (as seen on Jenkins 2.249.3). See also: JENKINS-2111
As far as I can tell - the new setting was introduced in jenkins-branch-api 2.0.21:
There's a new property introduced: MAX_LENGTH.
This defaults to 32 characters by default.
You can set it the same way like PATH_MAX:
As a java property - to ensure that Jenkins will start using the right setting, e.g.:
-Djenkins.branch.WorkspaceLocatorImpl.MAX_LENGTH=40
or during run-time, using the script console:
jenkins.branch.WorkspaceLocatorImpl.MAX_LENGTH=40
For older Jenkins instances
Actually there's a java property you can set to specify the length of the directory name, e.g.:
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=20
To make it permanent you have to specify this property in the Jenkins java startup configuration file.
You may also read and write this property using the Jenkins script console for temporary changes or to just give it a try as it takes effect immediately, e.g.
println jenkins.branch.WorkspaceLocatorImpl.PATH_MAX
jenkins.branch.WorkspaceLocatorImpl.PATH_MAX = 20
println jenkins.branch.WorkspaceLocatorImpl.PATH_MAX
Setting this value to 0 changes the path generation behavior.
For details please check:
https://issues.jenkins-ci.org/browse/JENKINS-34564
https://issues.jenkins-ci.org/browse/JENKINS-38706
I am using VSTS build & release feature to build the .net project. After building the project, I want build number to be updated in the Global List so that I can select the same build number to resolve the defects (Integrated in Build) and while executing the test cases through MTM.
This feature was present when we were using in-premise TFS 2013.
Please let me know if there is a way to update the build no. in the Global List using VSTS Rest API.
Thanks in Advance!!
There isn’t the Rest API to update the Global List, you can update the Global List by using VSTS/TFS client API (WorkItemStore.ImportGlobalLists Method) or witadmin commands.
A sample by using PowerShell: Get TfsCollection and TFS Services and Adding to a GlobalList.
function Add-TfsGlobalListItem {
Param(
[parameter(Mandatory=$true)][Microsoft.TeamFoundation.Client.TfsTeamProjectCollection] $TfsCollection,
[parameter(Mandatory=$true)][String] $GlobalListName,
[parameter(Mandatory=$true)][String] $GlobalEntryValue
)
# Get Global List
$store = Get-TfsWorkItemStore $TfsCollection
[xml]$export = $store.ExportGlobalLists();
$globalLists = $export.ChildNodes[0];
$globalList = $globalLists.SelectSingleNode("//GLOBALLIST[#name='$GlobalListName']")
# if no GL then add it
If ($globalList -eq $null)
{
$globalList = $export.CreateElement("GLOBALLIST");
$globalListNameAttribute = $export.CreateAttribute("name");
$globalListNameAttribute.Value = $GlobalListName
$globalList.Attributes.Append($globalListNameAttribute);
$globalLists.AppendChild($globalList);
}
#Create a new node.
$GlobalEntry = $export.CreateElement("LISTITEM");
$GlobalEntryAttribute = $export.CreateAttribute("value");
$GlobalEntryAttribute.Value = $GlobalEntryValue
$GlobalEntry.Attributes.Append($GlobalEntryAttribute);
#Add new entry to list
$globalList.AppendChild($GlobalEntry)
# Import list to server
$store.ImportGlobalLists($globalLists)
On the other hand, you can refer to this article to update related work item of build: Build association with work Items in vNext
Update:
Simple code to connect to VSTS:
param(
[string]$address,
[string]$username,
[string]$password
)
$credentials = New-Object System.Net.NetworkCredential($username, $password)
$tfsCollection = New-Object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection((New-Object System.URI($address)))
$wis = $tfsCollection.GetService([Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore])
Then, you can specify the arguments in PowerShell task (build or release) Arguments textbox -address XXX -username XXX -password XXX. (The username can be empty and use personal access token as password)
On the other hand, you can import the assemblies files that in Microsoft Team Foundation Server Extended Client package.
How can I get build time stamp of the latest build from Jenkins?
I want to insert this value in the Email subject in post build actions.
Build Timestamp Plugin will be the Best Answer to get the TIMESTAMPS in the Build process.
Follow the below Simple steps to get the "BUILD_TIMESTAMP" variable enabled.
STEP 1:
Manage Jenkins -> Plugin Manager -> Installed...
Search for "Build Timestamp Plugin".
Install with or without Restart.
STEP 2:
Manage Jenkins -> Configure System.
Search for 'Build Timestamp' section, then Enable the CHECKBOX.
Select the TIMEZONE, TIME format you want to setup with..Save the Page.
USAGE:
When Configuring the Build with ANT or MAVEN,
Please declare a Global variable as,
E.G. btime=${BUILD_TIMESTAMP}
(use this in your Properties box in ANT or MAVEN Build Section)
use 'btime' in your Code to any String Variables etc..
NOTE: This changed in Jenkins 1.597, Please see here for more info regarding the migration
You should be able to view all the global environment variables that are available during the build by navigating to https://<your-jenkins>/env-vars.html.
Replace https://<your-jenkins>/ with the URL you use to get to Jenkins webpage (for example, it could be http://localhost:8080/env-vars.html).
One of the environment variables is :
BUILD_ID
The current build id, such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss)
If you use jenkins editable email notification, you should be able to use ${ENV, var="BUILD_ID"} in the subject line of your email.
One way this can be done is using shell script in global environment section, here, I am using UNIX timestamp but you can use any shell script syntax compatible time format:
pipeline {
agent any
environment {
def BUILDVERSION = sh(script: "echo `date +%s`", returnStdout: true).trim()
}
stages {
stage("Awesome Stage") {
steps {
echo "Current build version :: $BUILDVERSION"
}
}
}
}
Try use Build Timestamp Plugin and use BUILD_TIMESTAMP variable.
Generate environment variables from script (Unix script) :
echo "BUILD_DATE=$(date +%F-%T)"
I know its late replying to this question, but I have recently found a better solution to this problem without installing any plugin. We can create a formatted version number and can then use the variable created to display the build date/time.
Steps to create: Build Environment --> Create a formatted version number:
Environment Variable Name: BUILD_DATE
Version Number Format String: ${BUILD_DATE_FORMATTED}
thats it. Just use the variable created above in the email subject line as ${ENV, var="BUILD_DATE"} and you will get the date/time of the current build.
You can use the Jenkins object to fetch the start time directly
Jenkins.getInstance().getItemByFullName(<your_job_name>).getBuildByNumber(<your_build_number>).getTime()
also answered it here:
https://stackoverflow.com/a/63074829/1968948
BUILD_ID used to provide this information but they changed it to provide the Build Number since Jenkins 1.597. Refer this for more information.
You can achieve this using the Build Time Stamp plugin as pointed out in the other answers.
However, if you are not allowed or not willing to use a plugin, follow the below method:
def BUILD_TIMESTAMP = null
withCredentials([usernamePassword(credentialsId: 'JenkinsCredentials', passwordVariable: 'JENKINS_PASSWORD', usernameVariable: 'JENKINS_USERNAME')]) {
sh(script: "curl https://${JENKINS_USERNAME}:${JENKINS_PASSWORD}#<JENKINS_URL>/job/<JOB_NAME>/lastBuild/buildTimestamp", returnStdout: true).trim();
}
println BUILD_TIMESTAMP
This might seem a bit of overkill but manages to get the job done.
The credentials for accessing your Jenkins should be added and the id needs to be passed in the withCredentials statement, in place of 'JenkinsCredentials'. Feel free to omit that step if your Jenkins doesn't use authentication.
This answer below shows another method using "regexp feature of the Description Setter Plugin" which solved my problem as I could not install new plugins on Jenkins due to permission issues:
Use build timestamp in setting build description Jenkins
If you want add a timestamp to every request from browser to jenkins server.
You can refer to the jenkins crumb issuer mechanism, and you can hack the /scripts/hudson-behavior.js add modify here. so it will transform a timestamp to server.
/**
* Puts a hidden input field to the form so that the form submission will have the crumb value
*/
appendToForm : function(form) {
// add here. ..... you code
if(this.fieldName==null) return; // noop
var div = document.createElement("div");
div.innerHTML = "<input type=hidden name='"+this.fieldName+"' value='"+this.value+"'>";
form.appendChild(div);
}
The product I work on comprises 3/4 seperate (non-dependant) TFS builds.
I would like to create a single TFS build which queues the other 3/4 builds from within the ProcessTemplate AND, critically, pass process parameters to them. This build would wait for them all to complete and return an overall success/failure of the build.
So my questions are:
Can this be achieved by any existing 'standard' Workflow activities (my manager has had bad experiences with custom workflow activities)?
If not, I am able to 'shell out' to powershell. Can I achieve what I want from within Powershell (accessing the API)?
Maybe using TFSBuild.exe? But I can't find a way of passing the custom process parameters I need.
Any assistance or guidance would be appreciated.
UPDATE
The following powershell script will execute the build, but I'm still at a loss to be able to pass my custom process parameters :-(
function Get-BuildServer
{
param($serverName = $(throw 'please specify a TFS server name'))
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName ("Microsoft.TeamFoundation.Build.Client")
$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($serverName)
return $tfs.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
}
$buildserver = Get-BuildServer "http://tfsserver:8080/tfs/My%20Project%20Collection"
$teamProject = "ESI"
$buildDefinition = "iPrl_BuildMaster"
$definition = $buildserver.GetBuildDefinition($teamProject, $buildDefinition)
$request = $definition.CreateBuildRequest()
$buildserver.QueueBuild($request, "None")
Now after googling, I have found the following C# code to update the verbosity and, assuming it's the same for my custom process parameters, I need to convert this to work with the above powershell script. Any ideas?
IDictionary<String, Object> paramValues = WorkflowHelpers.DeserializeProcessParameters(processParameters);
paramValues[ProcessParameterMetadata.StandardParameterNames.Verbosity] = buildVerbosity;
return WorkflowHelpers.SerializeProcessParameters(paramValues);
In order to retrieve the information which Changeset was included in which Build, we use "Label Sidekick" of Team Foundation Sidekicks, where we place the Label of the Build & expect to find the newly built Changeset.
Our development process in TFS 2010 is making use of 'Gated' checkins, so we are faced with the situation that the latest checkins are not presented in Sidekicks (we actually receive the changeset of the previous build). This is explainable, since at the time the labeling takes place, the latest changes have not yet been committed.
The BuildLog does report the associated Changeset correctly.
I 've made several experiments in our Build Process Template but can't seem to get what we need.
Placing, for example, the Labeling activity out of the "Run On Agent" scope, lead me to a build that fails at the very start with an "Object reference not set to an instance of an object." (I suppose this is related with fact I had to widen the scope for 'Label' & 'Workspace' variables to get the second part running).
The 'before' state of the build process template for this attempt is here (this works), the 'after' state ("Object ref not set..") is here.
So, to summarize, two different types of input could help me out:
How should I change our build process template so that the labeling happens after the Gated checkins have been committed? (-- This would rationalize the display in Sidekicks)
or
How can I programmatically retrieve the associated Changeset of each Build? (-- This would enable me to write a small app that could obsolete the Sidekicks angle)
You can use the TFS API to get this done.
public static void GetBuild()
{
var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://tfsdevlonuk:8080/tfs/gazprom.mt"), new UICredentialsProvider());
tfs.EnsureAuthenticated();
var buildServer = tfs.GetService<IBuildServer>();
// Get Project Name
var versionControl = tfs.GetService<VersionControlServer>();
var teamProjects = versionControl.GetAllTeamProjects(true);
// Get Builds for a team project
var buildDetails = buildServer.QueryBuilds(teamProjects[0].Name);
// For each build
foreach (IBuildDetail buildDetail in buildDetails)
{
// Get the build details
var buildInfor = buildDetail.Information;
// More build infor like shelveset, etc
Debug.Write(buildDetail.LabelName + buildDetail.ShelvesetName);
}
The above code will help you get the build details programatically. I have some blog posts on how to connect to tfs programmatically and use the tfs api. http://geekswithblogs.net/TarunArora/archive/2011/06/18/tfs-2010-sdk-connecting-to-tfs-2010-programmaticallyndashpart-1.aspx