TFS 2017 - Retrieve Trigger information build definitions - tfs

Im trying to retrieve the schedule of the vNext BuildDefinitions through my C# Application.
I can retrieve the BuildDefinition objects that i need.
List<BuildDefinitionReference> builddefs = buildClient.GetDefinitionsAsync().Result;
and
var buildDef = buildClient.GetFullDefinitionsAsync(project: project,
name: buildDefName).Result.ToArray();
But when looking at the BuildDefinition.Trigger information, this does not seem to contain what i need.
Simular to the XAML builds, i want to be able to read the BuildDefinition.Schedules, to see when each build is scheduled.
Who knows where this information is currently kept ?
Update:
Fixed the issue by converting the BuildTrigger to a ScheduleTrigger
if (buildDefinition.Triggers != null)
{
foreach (BuildTrigger trigger in buildDefinition.Triggers)
{
if (trigger is ScheduleTrigger)
{
ScheduleTrigger scheduleTrigger = trigger as ScheduleTrigger;
foreach (var schedule in scheduleTrigger.Schedules)
{
//Do magic
}
}
}
}

Update the Microsoft.TeamFoundationServer.ExtendedClient package to the latest version and the GetFullDefinitionsAsync method should return the schedules information, that are stored in Triggers property instead of Schedules property.

You could use Rest API to Get a build definition, a sample:
GET https://tfsserver:8080/tfs/DefaultCollection/Fabrikam-Fiber-Git/_apis/build/definitions/29?api-version=2.0
Then you will get a response as below which include trigger information:
"triggers": [
{
"schedules": [
{
"branchFilters": [
"+$/MyFirstProject/Test"
],
"timeZoneId": "UTC",
"startHours": 3,
"startMinutes": 0,
"daysToBuild": "all",
"scheduleJobId": "18316444-edbb-479d-950d-7714ba39d3d6",
"scheduleOnlyWithChanges": true
}
],
"triggerType": "schedule"
}
And the each member for your reference:
Members
branchFilters: string[].
daysToBuild: ScheduleDays. Days for a build (flags enum for days of the week)
scheduleJobId: string. The Job ID of the Scheduled job that will queue the scheduled build. Since a single trigger can have multiple
schedules and we want a single job to process a single schedule (since
each schedule has a list of branches to build), the schedule itself
needs to define the Job Id. This value will be filled in when a
definition is added or updated. The UI does not provide it or use it.
startHours: number. Local timezone hour to start
startMinutes: number. Local timezone minute to start
timeZoneId: string. Time zone of the build schedule (string representation of the time zone id)
Details please refer Schedule Rest API.

Related

Ideas to implement dynamic parallel build using jenkins pipeline plugin

I have a requirement to run a set of tasks for a build in parallel, The tasks for the build are dynamic it may change. I need some help in the implementation of that below are the details of it.
I tasks details for a build will be generated dynamically in an xml which will have information of which tasks has to be executed in parallel/serial
example:
say there is a build A.
Which had below task and the order of execution , first task 1 has to be executed next task2 and task3 will be executed in parallel and next is task 4
task1
task2,task3
task4
These details will be in an xml dynamically generated , how can i parse that xml and schedule task accordingly using pipeline plugin. I need some idea to start of with.
You can use Groovy to read the file from the workspace (readFile) and then generate the map containing the different closures, similar to the following:
parallel(
task2: {
node {
unstash('my-workspace')
sh('...')
}
},
task3: {
node {
unstash('my-workspace')
sh('...')
}
}
}
In order to generate such data structure, you simply iterate over the task data read using XML parsing in Groovy over the file contents you read previously.
By occasion, I gave a talk about pipelines yesterday and included very similar example (presentation, slide 34ff.). In contrast, I read the list of "tasks" from another command output. The complete code can be found here (I avoid pasting all of this here and instead refer to this off-site resource).
The kind of magic bit is the following:
def parallelConverge(ArrayList<String> instanceNames) {
def parallelNodes = [:]
for (int i = 0; i < instanceNames.size(); i++) {
def instanceName = instanceNames.get(i)
parallelNodes[instanceName] = this.getNodeForInstance(instanceName)
}
parallel parallelNodes
}
def Closure getNodeForInstance(String instanceName) {
return {
// this node (one per instance) is later executed in parallel
node {
// restore workspace
unstash('my-workspace')
sh('kitchen test --destroy always ' + instanceName)
}
}
}

Quartz.net scheduler

We have installed quartz.net scheduler service and configured a (memory)job to run daily # 10 pm. In case the server hosting this service is restarted, is there a way to force the job to run as soon as the service comes up? In normal scenario job should fire at 10pm as scheduled, but whenever the server/service is restarted, we want the job to run immediately even if it is not scheduled to run at that time. If there's some configuration value to achieve this, that would be the best option.
Write a little code that reads a small xml file (custom one of your own doing)....
and put it in your startup code.
foreach( xmlElement in yourXmlFile)
{
string someJobName= ""; /* read xml for jobName */
String someJobGroup= ""; /* read xml for job group name */
Trigger trigger = TriggerBuilder.newTrigger()
.withIdentity(someJobName, someJobGroup)
.startNow()
.build();
}
You can perform this with the help of WithMisfireHandlingInstructionFireAndProceed method of CronScheduleBuilder as shown below:
ITrigger trigger = TriggerBuilder.Create()
.WithIdentity("trigger1", "group1")
.StartNow()
.WithSchedule(CronScheduleBuilder
.WeeklyOnDayAndHourAndMinute(DayOfWeek.Monday, 09, 00)
.WithMisfireHandlingInstructionFireAndProceed()
//MISFIRE_INSTRUCTION_FIRE_NOW
.InTimeZone(TimeZoneInfo.FindSystemTimeZoneById("GTB Standard Time"))
)
//.ForJob(myJobKey)
.Build();
scheduler.ScheduleJob(job, trigger);

How to set up a one-off job trigger at a specified time using Grails Quartz2 plugin

I am using Quartz2 Plugin, and am trying to dynamically trigger a very simple job. When the user performs a certain action, the job should be triggered some certain number of minutes in the future, and only run once.
I have tried using the simple 'schedule' method that takes a date and job data:
def sendTime = new Date()
use(groovy.time.TimeCategory) {
sendTime = sendTime + (connectionInstance.timeout).minutes
println "I will send the job at $sendTime"
}
ReportSmssyncTimeoutJob.schedule(sendTime, [connectionId:params.id])
In this setup, I find that the job actually triggers immediately instead of waiting until 'sendTime'.
My second attempt, after looking at the plugin source, was to use a SimpleTrigger
def sendTime = new Date()
use(groovy.time.TimeCategory) {
sendTime = sendTime + (connectionInstance.timeout).minutes
println "I will send the job at $sendTime"
}
// arguments here are: jobKey='test', startTime=sendTime, repeatCount=0, repeatInterval=1 (zero not allowed), job-arguments)
def trigger = TriggerHelper.simpleTrigger(new JobKey("test"), sendTime, 0, 1, [connectionId:params.id])
ReportSmssyncTimeoutJob.schedule(trigger)
In this setup, the job also triggers immediately. Is there something wrong with the SimpleTrigger implementation which prevents it from waiting until startDate?
Unfortunately, switching to the main 'quartz' plugin (which now has support for Quartz 2) is not an option as I am working on a project that has loads of jobs set up to work with the quartz2 plugin.
I asked this on the Grails mailing list and got the answer: this is a bug in the quartz2 plugin. It should be fixed in the next release (bug was noted in 0.2.3).
Update: tested this in v2.1.6.2 of the quartz2 plugin, and can confirm that both of the approaches in my question now work.

From Jenkins, how do I get a list of the currently running jobs in JSON?

I can find out just about everything about my Jenkins server via the Remote API, but not the list of currently running jobs.
This,
http://my-jenkins/computer/api/json
or
http://my-jenkins/computer/(master)/api/json
Would seem like the most logical choices, but they say nothing (other than the count of jobs) about which jobs are actually running.
There is often confusion between jobs and builds in Jenkins, especially since jobs are often referred to as 'build jobs'.
Jobs (or 'build jobs' or 'projects') contain configuration that describes what to run and how to run it.
Builds are executions of a job. A build contains information about the start and end time, the status, logging, etc.
See https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project for more information.
If you want the jobs that are currently building (i.e. have one or more running builds), the fastest way is to use the REST API with XPath to filter on colors that end with _anime, like this:
http://jenkins.example.com/api/xml?tree=jobs[name,url,color]&xpath=/hudson/job[ends-with(color/text(),%22_anime%22)]&wrapper=jobs
will give you something like:
<jobs>
<job>
<name>PRE_DB</name>
<url>http://jenkins.example.com/job/my_first_job/</url>
<color>blue_anime</color>
</job>
<job>
<name>SDD_Seller_Dashboard</name>
<url>http://jenkins.example.com/job/my_second_job/</url>
<color>blue_anime</color>
</job>
</jobs>
Jenkins uses the color field to indicate the status of the job, where the _anime suffix indicates that the job is currently building.
Unfortunately, this won't give you any information on the actual running build. Multiple instances of the job maybe running at the same time, and the running build is not always the last one started.
If you want to list all the running builds, you can also use the REST API to get a fast answer, like this:
http://jenkins.example.com/computer/api/xml?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds
Will give you something like:
<builds>
<url>http://jenkins.example.com/job/my_first_job/1412/</url>
<url>http://jenkins.example.com/job/my_first_job/1414/</url>
<url>http://jenkins.example.com/job/my_second_job/13126/</url>
</builds>
Here you see a list off all the currently running builds. You will need to parse the URL to separate the job name from the build number. Notice how my_first_job has two builds that are currently running.
I have a view defined using View Job Filters Plugin that filters just currently running jobs, then you can use /api/json on the view page to see just the jobs that are running. I also have one for aborted, unstable, etc.
UPDATE
Select Edit View → Job Filters → Add Job Filter ▼ → Build Statuses Filter
Build Statuses: ☑ Currently Building
Match Type: Exclude Unmatched - ...
Bit of a hack but I think you can infer what jobs are currently running by looking at the color key in the job objects when you do a GET at /jenkins/api/json?pretty=true. If the 'ball' icon for a given job in Jenkins is animated, we know it's running.
Have a look at the array of job objects in the JSON response:
{
...
"jobs" : [
{
"name" : "Test Job 1",
"url" : "http://localhost:8000/jenkins/job/Test%20Job%201/",
"color" : "blue"
},
{
"name" : "Test Job 2",
"url" : "http://localhost:8000/jenkins/job/Test%20Job%202/",
"color" : "blue_anime"
}
...
}
In this case "color" : "blue_anime" indicates that the job is currently running, and "color" : "blue" indicates that the job is not running.
Hope this helps.
Marshal the output and filter for "building: true" from the following call to json api on a job with tree to filter out the extraneous stuff (hope this helps):
http://jenkins.<myCompany>.com/job/<myJob>/api/json?pretty=true&depth=2&tree=builds[builtOn,changeSet,duration,timestamp,id,building,actions[causes[userId]]]
will give you something like:
{
"builds" : [
{
"actions" : [
{
},
{
"causes" : [
{
"userId" : "cheeseinvert"
}
]
},
{
},
{
},
{
},
{
}
],
"building" : true,
"duration" : 0,
"id" : "2013-05-07_13-20-49",
"timestamp" : 1367958049745,
"builtOn" : "serverA",
"changeSet" : {
}
}, ...
You can do this with the jenkins tree api, using an endpoint like this:
http://<host>/api/json?tree=jobs[name,lastBuild[building,timestamp]]
You can see what attributes from lastBuild you can use if you access <job-endpoint>/lastBuild/api/json.
I had a similar problem where some pipeline builds get stuck in the building state after I restart jenkins (piepline jobs are supposed to be durable and resume but most of the time they get stuck indefinitely).
These builds do not use an executor so the only way to find them is to open every job.
All of the other answers seem to work when the project is considered building, i.e.: the last build is building. But they ignore past builds still building.
The following query works for me and gives me all the currently running builds, i.e.: they do not have a result.
http://localhost:8080/api/xml?tree=jobs[name,builds[fullDisplayName,id,number,timestamp,duration,result]]&xpath=/hudson/job/build[count(result)=0]&wrapper=builds
Nothing worked me properly. I copied and modified code form python-jenkins. Since Master node name changed , it was giving exception. Did'nt want to rely on plugin.
def get_running_builds():
builds = []
nodes = server.get_nodes()
for node in nodes:
# the name returned is not the name to lookup when
# dealing with master :/
if node['name'] == 'Built-In Node':
continue
if node['name'] == 'master':
node_name = '(master)'
else:
node_name = node['name']
try:
info = server.get_node_info(node_name, depth=2)
except server.JenkinsException as e:
# Jenkins may 500 on depth >0. If the node info comes back
# at depth 0 treat it as a node not running any jobs.
if ('[500]' in str(e) and
server.get_node_info(node_name, depth=0)):
continue
else:
raise
for executor in info['executors']:
executable = executor['currentExecutable']
if executable and 'number' in executable:
#print(f'{executable}')
executor_number = executor['number']
build_number = executable['number']
url = executable['url']
m = re.search(r'/job/([^/]+)/.*', urlparse(url).path)
job_name = m.group(1)
builds.append({'name': executable['fullDisplayName'],
'number': build_number,
'url': url,
'node': node_name,
'executor': executor_number,
'timestamp': executable['timestamp']})
return builds
timestamp gives time in millisecs.

TFS2010 - Wrong changeset appearing at SourceGetVersion

I am currently setting up a Team Foundation Server 2010 and I found a very strange behavior when performing a build:
The situation explained:
We have 2 Branches
Development
Main
All developers check in code into the Development branch only. Once per day, the build manager merges some changesets over to the Main branch. On the Development brach, a continuous build at each check in is running. On the Main branch, once per day (in the night) a build is triggered.
Now suppose that the changesets 1-100 are being merged into the Main brach at 5pm, giving changeset 101 as the merge operation. Some developers check in changesets 102-106 after 5 o'clock into the Development branch. Now at 11pm the daily build is automatically triggered and runs on the Main branch. The last changeset of the Main branch is changeset 101. However, the Build details shows changeset 106:
I could imagine that this behavior is intended, because if you check out changeset 106 on the Main branch, you will in fact get the content of changeset 101. But it would be much more readable if this Build summary showed the correct number.
Question 1: Is there a way of manipulating the ouput of the SourceGetVersion information? Maybe through the Build Process Template?
The second scenario, where the TFS behaves strange is even worse:
When queuing a new build, there is the option of entering the "Get Version" Parameter, as shown in the following picture:
If I now click on "queue", the build is triggered and AGAIN the build detail outputs the changeset 106 although I specifically set it to get changeset 76.
Question 2: Is this a bug? Is there a hotfix or something to fix this? Or is there any option flag that has to be set?
I hope someone knows more about this. I don't really believe that this is a bug, because it is such a vital functionality that other people must have encountered it before.
Thanks for any help!!
Christian
EDIT 1
The folder structure of the Team Project is:
$ProjectName
BuildProcessTemplates
Documentation
SourceCode
Development <-- this is a branch
3rdParty
Source
Main <-- this is a branch
3rdParty
Source
The build only pulls the Main branch and everything below it.
EDIT 2
Here is a picture of the Workspace tab in the build definition:
Finally I found out what is going on:
Basically The changeset that can be seen in my picture 1 is always the latest changeset of the entire Team Project Collection. It is the property "SourceGetVersion" on the object "BuildDetails" of type "IBuildDetails".
I think this is a bug which can be worked around:
If you change the BuildDetails.SourceGetVersion (which is a string) to some other value, then the build summary will show the updated string. Furthermore, it is then saved correctly to the collection database.
What I have done in order to add the correct changeset number is I have created a custom build activity that takes the branch which should be build as input parameter. It outputs the correct changeset. The activity finds out the correct changeset by connecting to the TFS and downloading the History. Then it looks at all the items in the history and outputs the largest changeset number. Here is the code of that activity:
using System.Activities;
using System.Collections;
using System.Net;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.VersionControl.Client;
using Microsoft.TeamFoundation.Build.Client;
namespace SourceGetVersionActivity
{
[BuildActivity(HostEnvironmentOption.All)]
public sealed class SourceGetVersionActivity : CodeActivity<string>
{
// Define an activity input argument of type string
public InArgument<string> Branch { get; set; }
// If your activity returns a value, derive from CodeActivity<TResult>
// and return the value from the Execute method.
protected override string Execute(CodeActivityContext context)
{
// Obtain the runtime value of the Text input argument
string branch = context.GetValue(this.Branch);
ICredentials account = new NetworkCredential("Useranme", "password", "domain");
// connect / authenticate with tfs
TeamFoundationServer tfs = new TeamFoundationServer("http://tfs:8080/tfs/CollectionName", account);
tfs.Authenticate();
// get the version control service
VersionControlServer versionControl = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));
IEnumerable changesets = versionControl.QueryHistory(branch, VersionSpec.Latest, 0, RecursionType.Full,
null, null, null, int.MaxValue, false, false, false, false);
int maxVersion = 0;
foreach (Changeset c in changesets)
{
if (c.ChangesetId > maxVersion)
{
maxVersion = c.ChangesetId;
}
}
return string.Concat('C', maxVersion.ToString());
}
}
}
I call this activity as soon as possible (after the GetBuild activity).
Basically in the BuildProcessTemplate I have added an Argument (string) "Branch" which needs to be filled with a string that points to the top folder that is being build. The custom activity takes that as input and outputs a string which is the correct changeset id. The BuildDetail.SourceGetVersion property will then be overriden by the correct changeset id.
I find it really strange that no-one else seems to have encountered this problem. I could not find any person on the internet with the same problem. Anyway, I hope this answer helps someone else in the future as well.
EDIT - Writing the above code directly in Workflow Foundation:
To get the correct changeset using more compact code and avoiding custom activites, it is also possible to use Workflow Foundation directly. Below is the "code" (doing exactly what is done in above C# code):
(1) The GetTeamProjectCollection activity gets the current collection. I am saving it inside the TeamProjectCollection variable (see bottom of the picture). Important: The variable needs to be defined inside this sequence, if you define it in outer scope, an error will occur: "Unable to serialize type 'Microsoft.TeamFoundation.Client.TfsTeamProjectCollection'. Verify that the type is public and either has a default constructor or an instance descriptor."
(2) Foreach "changeset" in "TeamProjectCollection.GetService(Of VersionControlServer).QueryHistory(Branch, VersionSpec.Latest, 0, RecursionType.Full, Nothing, Nothing, Nothing, Integer.MaxValue, False, False, False).Cast(Of Changeset)()"
The TypeArgument of the Foreach loop is "Microsoft.TeamFoundation.VersionControl.Client.Changeset".
This expression gets the version control object from the collection, calls it "QueryHistory" method which returns an IEnumerable with all changesets.
(3) So we are iterating over all changesets and looking at the ChangesetId. Then saving the maximum ChangesetId to the variable "maxId".
(4) At the end, BuildDetails.SourceGetVersion = "C" + maxId.ToString(). The "C" indicates, that the version is a changeset.
I hope someone finds this piece of "Code" useful!
Christian

Resources