We've been using Team City as our build deployment server. We'd like to now update the metrics in tfs like "Code Churn", "Code Coverage". Is it possible to do this with TeamCity or do we just need to start using the tfs build server?
It is possible to publish builds results (including metrics) to the TFS server but only when those are collected via MSTest. The MSTest /publish command might be helpful here. It looks like publishing custom results like this will be easier in TFS2010 but there are no details as of yet.
Related
We have TFS 2017 installed on our server and JMeter performance tests which i want to run through non-gui mode on TFS build agent.
I can't find way how to publish jmeter results to TFS?
There is no approved plugin for JMeter (at least none I could find) that incorporate your results into VSTS test results. You would need to dump your JMeter results locally, add a task that captures the output and puts it with the $(build.artifactstagingdirectory) path. You will then need to make sure that your final task is a Publish Build Artifacts. This will ensure that your JMeter results will get published as artifacts as part of the build.
Now the fun part, you obviously want to view the results so you are left with two options. The first is to create a web app that contacts VSTS via the API, pulls the JMeter artifact and displays the results as desired. There may actually be
Last option is to create a hub page for VSTS. This could query the build, get the JMeter artifacts and render them as desired.
These are the best options I could find for you at present. You can always request such a feature in the VSTS User Voice Forum.
There is not any build-in tool to integrate the JMeter with TFS for now.
You can have a try for Cloud-based Apache JMeter Load Test task. However it's cloud based, and need Visual Studio Team Services account.
There are detailed reports and analysis tool when using could based jmeter test such as this tutorial.
Please reference this article for detail steps : Running Apache JMeter based load tests in the cloud – how to
In your case, the simplest way is as virusstorm suggested using custom extensions to catch your test results and publish test results. If you are not familiar with this area, there are detail tutorials here. Moreover you could also ask help of some ALM company and team.
I've been tasked with migrating a build from TFS to VSTS, I've used TFS from a developer point of view, however I've never created builds. Can anyone give me any advice where to start, good websites, tutorials, tools that might help, anything that might catch me out etc, basically my knowledge on this is very limited so anything would help. I've no doubt there must be somebody out there who can say we've done this, here is what we did...the new VSTS build will need to be a copy of the TFS build however some analysis needs to be done as the build process is old and certain parts may not be necessary. Thanks in advance!
Migrate New Vnext build definition from TFS to VSTS
If you want to template your builds or move them from TFS to VSTS, you could also use the REST API's to perform this. Details please refer this blog: HOW TO MOVE BUILD DEFINITIONS IN TFS TO OTHER PROJECTS USING THE REST API
Another example shown here. It shows how to get the JSON response of a build definition and again using the same reference to make a new one in the project you desire.
Migrate Old XAML to New Vnext build definition from TFS to VSTS
Unfortunately, there will not be any automated conversion processes. The new builds are based on a different architecture and run on a completely different system. And you can use both the new builds and agents alongside your XAML builds, controllers, and agents.
If you have heavily customized XAML builds and custom activities, you
can continue using those builds until you are ready to port your
business logic into scripts that can run in the new builds.
You should also take a look at these series of articles by jessehouwing devoted specifically to the case like yours.
About how to create a build, you could refer this tutorial from MSDN.
I've recently developed custom build runners (plugins) to TeamCity and Jenkins. The plugins enable the users to start automated load tests as part of the build process.
To give you some idea here's the Jenkins plugin page with a lot of description:
https://wiki.jenkins-ci.org/display/JENKINS/Apica+Loadtest+Plugin
Here are a couple of screenshots of the TeamCity plugin:
The setup GUI: https://i.imgsafe.org/5221a01.png
Build log: https://i.imgsafe.org/c93f7f9.png
Custom tab with load test summary: https://i.imgsafe.org/f5b6937.png
Setup validation example: https://i.imgsafe.org/52cc9a2.png
These Continuous Integration frameworks allow a high level of customisation for plugin development: the UI, the output, client and server code etc. can all be tailored.
I've received a question whether we can develop a similar build runner for MS Team Foundation Build Service. I've completely new to TFS, I've never used it for anything. I've read through a couple of tutorials on how to install and set up TFS but there seems to be very little material available regarding custom build runners. The closest I've got are the following pages:
https://msdn.microsoft.com/en-us/library/bb130146.aspx
http://blogs.msdn.com/b/jimlamb/archive/2010/02/12/how-to-create-a-custom-workflow-activity-for-tfs-build-2010.aspx
However, they don't provide any example on GUI extensions, validation, customisation etc.
Therefore I'd like to get the opinion of experienced TFS users before we get any more serious with the framework:
is it possible to develop plugins for TFS build?
what are the limitations? E.g. can I build a custom GUI with custom client/server actions?
can I include custom pages, show graphs etc?
Any advice is welcome.
Thanks for your help,
Andras
When it comes to TFS Build, you first need to know that there are two build systems: XAML Build and a new, now default, build system.
XAML Builds are based on Windows Workflow Foundation. You can create custom activities and add these to a Build Definition Template. XAML Build only run on Windows and extending them is not very easy.
The new Build system is based on Node, runs cross-platform and is very easily extendable. Microsoft has open sourced all the tasks they have for TFS Build (see GitHub for the repository).
Targeting the new build system means that you support on-premises installations of TFS 2015 and the cloud hosted version of TFS: Visual Studio Team Services (see visualstudio.com for more info).
The easiest way to get started is by creating a new VSTS account (free!), adding some code and running a build. If you have that working, you can start exploring the existing build tasks and learn what's possible. You can then easily create your own task and start experimenting.
One thing that might be of interest to you is that TFS/VSTS already support load testing. You can run a very simple load test with a couple of mouse clicks or configure more complex Web Tests and use these in TFS in combination with Application Insights. I'm not sure if that's what your customers are looking for but it's worth checking out (see Cloud-based Load Testing for more information)
So here's the setup we have right now
1) We use TFS2013 for our code repository
2) We use TeamCity to do our builds
3) we have our tests created in CodedUI
From what I've read online is that for the CodedUI tests to run automatically, they need a build to reference. However, since we are using Team City to do the builds, our Build Explorer in TFS is empty. Switching off of Team City for builds is a no, so I'm trying to find a way to have the builds appear in the Build Explorer so I can have the automated tests run via MTM.
TL;DR: I need to get the Team City builds to appear in the TFS Build Explorer.
I saw this question, but it was for 2010, so I'm hoping it's changed since How do I make a TeamCity build appear in the TFS Build Explorer?
It can be done using the API. You could use the following blog post as a starting point to create an app that records in TFS the results from your TeamCity build. Then you could run your app as part of your TeamCity build.
http://blogs.msdn.com/b/jpricket/archive/2010/02/23/creating-fake-builds-in-tfs-build-2010.aspx
Update: There are however downsides. Some capabilities of ALM are only available if the build is done in TFS. You loose Test Impact Analysis, relationships between builds and Work Items, identification of Verifiable Bug fixes, roll up for release management of release contents.
TF Build is more than just orchestrating a compilation, it is the glue that knits your ALM strategy together and you will not be able to amortize the realm value in TFS without builds in TF Build or a whole lot of custom work.
Sorry, it's not possible. Team Explorer is designed to show you Team Builds.
I am told TFS can accept data on build/test metrics from 3rd party continuous integration tools. Does anyone know how this works or have any good links for me? My google-fu seems weak today and I cannot find any info on this. We would like to have a short powershell script or app run at the end of the build and send all known metrics up to TFS so it could show up in certain reports. I actually would like things that (I think) should already have space for in the data warehouse for TFS BUild Server, but I will be using CC.NET. I am thinking build name, result (Pass/Fail), Number of Warnings, Number of Errors, Time, UnitTests Run, UnitTests Passed, Code Coverage, FxCop resultsThanks.
I'm afraid that there is not a ready made integration that does this yet. The plug-in that links CC.NET to TFS is available over at CodePlex but this just lets CC.NET use TFS for version control and doesn't allow the results of the builds to be published back into TFS.
To get the data into TFS from CruiseControl.NET you have a couple of options. You could write your own custom TFS Data Warehouse Adapter which is complex but ultimately flexible or you could use a combination of the Team Build API and a little bit of vodoo to push data in to the TFS Build store that would also get pushed into the TFS Data warehouse. However, this would be limited in TFS2008 as you would only be able to push data about the build and the unit tests but not things like Code Coverage.
That said, pushing data from CC.NET to TFS is something that I originally wanted to do. However in TFS2008 the built in build system was so good that I switched from using CC.NET to trigger the builds to using TFS to trigger and manage the build. This had the advantage that all the stats stuff was taken care of automatically (and the built in UI in Visual Studio). Because I moved to TFS2008 I then lost the motivation to get the CC.NET stuff built.
If anyone wanted to contribute a TFS build result publishing feature to the CC.NET integration then feel free to join the project on CodePlex - I would love to have any help going.