I'm looking for a ADO extension that would provide a dashboard contain data that can be use to create metrics. We have 150+ pipelines that I want to track.
I found a "Build Metrics Widget" which may fit your requirements.
For more relevant extensions, you can check out this search results.
What's more, in Azure DevOps, there are some built-in widgets for pipelines.
Chart for build history: Shows the build history of a selected build definition.
Deployment status: Shows the depolyment and test status of a branch across the enviornments in your release definitions.
Release pipeline overview: Shows the status of enviornments in a release definition.
Requirements quality: Shows the quality of requirement(s) associated to automated tests.
For detailed information of these widgets, please click Azure Pipelines widgets.
Related
We are trying to use Microsoft Azure DevOps API and retrieve following information for each build under each project in Azure DevOps organization. Ultimate goal is to retrieve these datils using C# code and use as a data to make some decisions in one of our dashboard. As we can see, these details available in the summary of the build once build completed (in Azure DevOps build summary page).
Unit Test Status
Code Coverage
Build Status
Deployment Status
Automated Test Execution Status
We have referred this Microsoft official documentation in our research. And found that below GET methods has some information that may help with our research, but unfortunately these does not contains all the information that we are looking for.
Build
Builds
History
Could someone help us by recommending possible candidate API's to efficiently extract this data?
We have TFS 2017 installed on our server and JMeter performance tests which i want to run through non-gui mode on TFS build agent.
I can't find way how to publish jmeter results to TFS?
There is no approved plugin for JMeter (at least none I could find) that incorporate your results into VSTS test results. You would need to dump your JMeter results locally, add a task that captures the output and puts it with the $(build.artifactstagingdirectory) path. You will then need to make sure that your final task is a Publish Build Artifacts. This will ensure that your JMeter results will get published as artifacts as part of the build.
Now the fun part, you obviously want to view the results so you are left with two options. The first is to create a web app that contacts VSTS via the API, pulls the JMeter artifact and displays the results as desired. There may actually be
Last option is to create a hub page for VSTS. This could query the build, get the JMeter artifacts and render them as desired.
These are the best options I could find for you at present. You can always request such a feature in the VSTS User Voice Forum.
There is not any build-in tool to integrate the JMeter with TFS for now.
You can have a try for Cloud-based Apache JMeter Load Test task. However it's cloud based, and need Visual Studio Team Services account.
There are detailed reports and analysis tool when using could based jmeter test such as this tutorial.
Please reference this article for detail steps : Running Apache JMeter based load tests in the cloud – how to
In your case, the simplest way is as virusstorm suggested using custom extensions to catch your test results and publish test results. If you are not familiar with this area, there are detail tutorials here. Moreover you could also ask help of some ALM company and team.
I've recently developed custom build runners (plugins) to TeamCity and Jenkins. The plugins enable the users to start automated load tests as part of the build process.
To give you some idea here's the Jenkins plugin page with a lot of description:
https://wiki.jenkins-ci.org/display/JENKINS/Apica+Loadtest+Plugin
Here are a couple of screenshots of the TeamCity plugin:
The setup GUI: https://i.imgsafe.org/5221a01.png
Build log: https://i.imgsafe.org/c93f7f9.png
Custom tab with load test summary: https://i.imgsafe.org/f5b6937.png
Setup validation example: https://i.imgsafe.org/52cc9a2.png
These Continuous Integration frameworks allow a high level of customisation for plugin development: the UI, the output, client and server code etc. can all be tailored.
I've received a question whether we can develop a similar build runner for MS Team Foundation Build Service. I've completely new to TFS, I've never used it for anything. I've read through a couple of tutorials on how to install and set up TFS but there seems to be very little material available regarding custom build runners. The closest I've got are the following pages:
https://msdn.microsoft.com/en-us/library/bb130146.aspx
http://blogs.msdn.com/b/jimlamb/archive/2010/02/12/how-to-create-a-custom-workflow-activity-for-tfs-build-2010.aspx
However, they don't provide any example on GUI extensions, validation, customisation etc.
Therefore I'd like to get the opinion of experienced TFS users before we get any more serious with the framework:
is it possible to develop plugins for TFS build?
what are the limitations? E.g. can I build a custom GUI with custom client/server actions?
can I include custom pages, show graphs etc?
Any advice is welcome.
Thanks for your help,
Andras
When it comes to TFS Build, you first need to know that there are two build systems: XAML Build and a new, now default, build system.
XAML Builds are based on Windows Workflow Foundation. You can create custom activities and add these to a Build Definition Template. XAML Build only run on Windows and extending them is not very easy.
The new Build system is based on Node, runs cross-platform and is very easily extendable. Microsoft has open sourced all the tasks they have for TFS Build (see GitHub for the repository).
Targeting the new build system means that you support on-premises installations of TFS 2015 and the cloud hosted version of TFS: Visual Studio Team Services (see visualstudio.com for more info).
The easiest way to get started is by creating a new VSTS account (free!), adding some code and running a build. If you have that working, you can start exploring the existing build tasks and learn what's possible. You can then easily create your own task and start experimenting.
One thing that might be of interest to you is that TFS/VSTS already support load testing. You can run a very simple load test with a couple of mouse clicks or configure more complex Web Tests and use these in TFS in combination with Application Insights. I'm not sure if that's what your customers are looking for but it's worth checking out (see Cloud-based Load Testing for more information)
We are using TFS and the TFS Build Service. We are considering to migrate the Build service to Jenkins but we came across some issues. According to this site, there are some things that do not work very well with the TFS and Jenkins plugins. All of them we use a lot:
Associated Change sets – Team Build automatically associates a list of change sets that are included in the build
Associated Work Items – Team Build analysis the relationships and also associates Work Items with a build. Indeed it walks the work item tree (parent) and maintains that association in the chain.
Is this still true? We have this scenario:
A developer checks in a code that fix a bug or resolve a User Story. It does that by associating his check in with the work item ID.
His check in triggers a build that will associate the work item with his changeset. For bugs, the build will update the "Integrated in Build" field with the build number. We use this field to know in witch version the bug was fixed.
Is there any way to make Jenkins behave and do what TFS build service does?
Another option is to mix the two using dummy builds on the TFS side that set the records straight and kick-off the Jenkins' builds. Some hints
How to trigger Jenkins builds remotely and to pass parameters and “Fake” a TFS Build.
This approach requires a bit of effort but has many advantages:
No big-bang, use Jenkins opportunistically
Can continue using existing builds
Having a build identifier in TFS allows you an overall monitoring and to use the Test features
I have a VSTS build definition for one of our projects that requires jenkins to build, but we still have all our other products using VSTS natively. To maintain consistency, this build definition triggers a jenkins build. We configured the build definition to not sync code as jenkins will download it (save time) and not to publish the artifacts back to the agent (i have another script for that found here). This allows developers to continue to use git as normal, and the build/release process is consistent with our other products. Along with task tracking and such.
In 2009, there was a SO question on the same topic.
I'm wondering if later versions of Team Foundation Server are better at longer build pipelines. Refer features of Jenkins, TeamCity, ThoughtWorks' Go (my employer).
The visualizations of the build pipelines are important to me, as well as the notification about individual stages passing or failing. That and the eminent clone-ability of say a 'trunk' pipeline into one for a release branch as that branch leaps into being.
Secondly, a personal holy-grail is the CI server storing its config in the SCM that's holding the buildable thing itself, and even picking up on the creation of branches silently to provision new pipelines; Can TFS be configured to store the CI definitions/scripts in its SCM side rather than its accompanying SqlServer?
TFS build consists of three components:
The build definition - stored on the SQL server data tier.
The build workflow - a XAML file stored in the source control.
The supporting MSBuild scripts - usually contains user defined actions, also stored in the source control.
As the build progresses, you can see visualization of the build steps and you also get a different log for the main build and the MSBuild output.
The build definition in TFS is merely a collection of build settings, similar to CC.Net's config file and TeamCity's build configuration tab which both stored on the file system as well. Assuming there's a backup plan on the database you don't really need to store the build defintions on the source control, but if you must it's possible by exporting the tbl_BuildDefinition table.
The TFS Power Tools adds cloning functionality for build definitions.
There's no OOTB support for provisioning build definitions from a new branch though it's fairly feasible using the TFS-API.
Bit late to the party, but just don't bother with TFS if you want advanced build pipeline automation. It simply doesn't cut it.
I have used Jenkins and TFS both extensively. Tfs is just. pure. crap. Here's why.
No down/up stream build.
No piepline/orchestraion build. (like jenkins)
Obscure ways of adding build steps and falls back to using MsBuild.
Slow and still polls the source control.
Ties you to MsTest.
And please don't point me to "Oh look you can do everything if you write a custom activity". I'm not wasting time doing development for a closed source, sub-par platform. If I am going to contribute something, it's to a FREE. OPEN SOURCE platform.