I would like to create a report from our TFS database which selects all source code repository names and commit history for these repositories. We are using git and it is integrated with TFS, but I am not sure how to access commit history from the TFS database.
I'd like to create something like this
Repository 1 - Name
Commit 1- author - date
Commit 2- author - date
Commit 3- author - date
Repository 2 - Name
Commit 1- author - date
Commit 2- author - date
Commit 3- author - date
I was reading this document:
https://learn.microsoft.com/en-us/azure/devops/report/sql-reports/table-reference-relational-warehouse-database?view=azure-devops-2019#SourceCode
Though that's not quite what I want, it got me started. Looking at the schema I found these tables:
DimChangeset (changeset data)
DimFile (individual files changed)
DimPerson (user data)
DimTeamProject (Teams project)
That seems to cover most of what I need, however I am not seeing where to get the repository names that are within the teams projects. Any suggestions? The repository names are in the TFS UI, so I assume they are stored in the database somewhere.
The SQL data warehouse and analysis cubes are deprecated. They haven't received updates for new features for many, many years -- that includes Git repos and JSON/YAML build/releases. What you're seeing is for TFVC repos, not Git repos.
The modern implementation is the OData analytics feed. However, it's not available in versions prior to Azure DevOps Server 2019. Unfortunately, repos aren't yet supported. There's not really a supported way to get the information you're after right now. You can query repo information using the REST APIs, but it's not in a format that's really intended to be used for tracking trends.
Related
I found articles on GIT back up and restore but I did not find any on Microsoft TFS Scheduled backup for GIT version control.
This document describes on how TFS's traditional versioning can be scheduled for automatic back up. Configure a backup schedule and plan for Team Foundation Server
However, I did not find any link where I can read that "GIT repository is also included in schedule". I am not a TFS expert and I do not have access to TFS server to verify in person. So any help is appreciated.
The tutorial you are refer which is Scheduled Backups tool. It' a build-in tool on the Scheduled Backups page in the TFS Administration console.
It' the whole database back up not only referring to source control, but also work items, pull requests, builds, test plans or anything else that the service offers.
As a limitation, you need both an administrator for TFS and a member of the SQL Server System Administrators group.
And no need to to verify in person, it's under SQL server, when you want to use the backup, just restore the database.
If you are actually using the online VSTS . For now there is no build-in tool and got a uservoice:
Provide a backup service for Visual Studio Team Services
https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/5339461-provide-a-backup-service-for-visual-studio-team-se
However if you just want a git repository(source code) back up, you could use some 3rd-party tool to achieve this:
We use the VSO Rest API to query our VSO account and get all the data
we need. Since in VSO you can only have one Team Project Collection,
we retrieve all the team projects of the default collection. Each of
these team projects can have multiple repositories that need to be
backed up. A folder is created for each team project and saved to a
location on disk that can be configured in the app.config. When the
team project folder is created, the task loops over each repository in
the team project and creates folders for each repository.
Source Link
You can also fork it on GitHub here. Certainly, you should also be able to use this for on-premise TFS also use the API, just need to change some part.
TFS is implemented the way to keep Git repositories in the SQL Server database, too. Thus, when you follow the instructions from the article you reference, the source code will also be included in the backup, no difference whether it is TFVC or Git.
Is it possible to filter changesets in TFS 2012 to exclude the ones from a single user?
The commit history of the solution I currently work on contains quite a lot of noise due to an automated process making a commit each time a build is queued.
I'd like to filter it out to make the commit history more readable.
An alternative, try to install Team Foundation Sidekicks, and check History Sidekick. You can export the records to a .csv file and then filter the user in it:
There is no single query can achieve what you need. You may have to use TFS API.
If you want to get the changesets in a solution via TFS API. The folder history will be the simplest approach. Details please refer this: TFS client C# API - get all changesets of an Item You can exclude those changesets from the special user using Changeset.Committer Property which gets the user who committed the changeset.
Can we get release information from checkedIN history.
we are using TFS 2012.
we have frequent release mostly on each week or on each 14 days.
Can we generate release document from TFS ?
(something like, based on TFS checked IN comments OR some how )
One way to do this would be to use Work Items (e.g. User Stories and Bugs) to generate your release notes.
But this requires your team to have the discipline to enter the information into TFS that allows it to associate work items with a release. For example:
Require developers to associate change sets with work items
Use fields such as "Iteration" or "Integrated in build" in work items to identify the release to which a work item belongs.
You could then create a suitable Team Query and export it to Excel to generate release notes.
I tend to use the Tf history command line tool to get information of changesets between two particular dates (or changesets). You can pass the /format:detailed parameter to get details such as changes involeed.
For example, the following will list all the changes between the 01/02/2014 & 24/04/2014 and write it to the history file.
tf history /noprompt * /recursive /v:D01/02/2014~D24/04/2014
/format:detailed > C:\history.txt
I like the TFS Changelog tool. Ready to use and easy to integrate.
TFS ChangeLog allows Team Foundation Server (TFS) users to extract information related to Changesets and associated WorkItems into XML format that is transformed into HTML.
All of these are nice tools, but there are a few things still missing I think.
These are technical notes (more like build / history notes) but actual release notes contain more customer friendly items
Not all of the information in TFS are wanted on the customer release notes (internally found bugs vs reported bugs)
As of now I use the TFS Community build manager to generate the technical build notes. It uses OpenXML and works fine for you build notes. It's even easy to create a custom word document for it.
For release notes we actually use the TFS User stories / tasks to be able to query on it. And for now, we create the document manually with the query results. We're looking into creating some tools that might do the following:
Run queries corresponding to the document paragrahs
We have a TFS template field indicating that the work item must be on the customer release notes (but we should change this to use 'special' iterations or so to eliminate the need of a custom field.
Automatically fill in the OpenXml placeholders in the document the same way as the build notes generator does.
Like the TFS Community Build Manager you could create a TFS Community Release Notes Manager that asks for query names and runs these queries to get to the required data.
The main question here is whether there are already tools that do this?
I know this is an old question, but perhaps you can use this task extension Generate Release Notes Build Task (for local TFS or as an extension to VisualStudio.com).
This extension is a build task you can use in build steps. This task generates a markdown release notes file based on a template passed into the tool. Here is an example of release notes output:
Release notes for build SampleSolution.Master
Build Number: 20160229.3 Build started: 29/02/16 15:47:58 Source
Branch: refs/heads/master
Associated work items
Task 60 [Assigned by: Bill ] Design WP8 client Associated change
sets/commits
ID bf9be94e61f71f87cb068353f58e860b982a2b4b Added a template ID
8c3f8f9817606e48f37f8e6d25b5a212230d7a86 Start of the project
I'm building a tool to integrate with TFS and it needs to properly parse TFS logs (from the tf.exe history command) and checkout different revisions (again using tf.exe). It works great on the test TFS server I have, but I want to test it on a broad range of large repositories to make sure my parsing works properly.
I'd hoped to use Codeplex to get access to TFS repositories, but it seems you only get TFS access to Codeplex projects if you're a project member.
Are there any collections of open source code hosted on public TFS servers? Are there any other publicly available servers I could use for testing?
I would suggest using svn2tfs and choose any relatively active project on SourceForge. There are plenty of projects on SF to choose from that use SVN and not CVS. You might even get a bonus out of it and help the svn2tfs project work out any kinks.
Since you mention tf history command, I assume you want to collect/parse logs on the project's (and its files) history of checkins.
So in addition to large repository, you also need a good amount of history, am I right? If yes, then here's your set of problems:
Most projects on codeplex use Mercurial, not TFS. So even if you get access, you cannot use TFS with them.
As you mentioned, they require you to a be a member for you to access the source.
Even if you get access or find a public server (unlikely), you still would need good amount of history.
If I'm correct in my assumptions so far, here's the easiest (bit tedious though) way out:
Go to any large projects's such as Nuget or Wix
revisions
Download any old revision (go back as far as you want the history for). You can download zipped src files without being a member.
In your test server, checkin the code (src) to create the baseline.
Download the next revision.
Checkout files in your server and overwrite them with the newer revision's files.
While checkin, use the history.txt (sample) to create checkin comments
Repeat this process few times.
Voila!! You now have a large repository with lot of history!
Hope this helps.
Have you tried some of the larger projects on Codeplex?
http://www.codeplex.com
If you only need read access you should be able to play around with the various repositories.
I don't have a huge amount of tfs experience, but I would assume there are migration tools that let you ingest code repositories from other products (e.g svn or hit).
If so, you might want to find a svn/git repo for a sizable foss project, and try importing that.
"I'd hoped to use Codeplex to get access to TFS repositories, but it seems you only get TFS access to Codeplex projects if you're a project member."
This solution appears to be the general consensus amoung SO'rs. I've read some of the Codeplex TFS connection problem threads (you linked to below) and I hope the comments in this thread resolves the issue:
Connecting to Codeplex TFS as a Coordinator or Developer.
I'm wondering if you can use git-tfs project to import an existing Git project into TFS.
Download and install git-tfs
Create a new TFS project
Clone the TFS project to a Git project using git-tfs ("git tfs clone http://tfs:8080/tfs/DefaultCollection $/some_project")
Import a existing Git project of your choice into your fresh new Git project (I don't know the command but I think it's possible).
Use git-tfs to checkin to TFS Server ("git tfs checkintool")
=> Do it makes sense ? And works ?
For more information:
http://lostechies.com/jimmybogard/2011/09/20/git-workflows-with-git-tfs/
I am attempting to understand how TFS 2008 (and 2010 if it's different) store and communicate details of a set of changes in a changeset.
Now when I commit to a Subversion hosted project, the client sends diffs to the server. So if I have added a file and changed a few lines in another it sends something like "Added file A.txt, put 2 lines "A" "B" into B.txt" . This means I can back out a revision as diffs are nicely reversible - "Delete file A.txt and take out two lines from B.txt". Pretty sure it is the same with git too.
I have a feeling that TFS does not store changesets in this way (based on rumour and the fact that I cannot easily roll back revisions etc.).
What I believe occurs is this:
If I changed files A, B and C within a changeset, what is actually stored in the changeset is a whole new copy of files A, B, C. Not just the diffs.
Am I correct in this assumption? As commercial software, I could well believe this information is not available but I thought I would ask as googling produced nothing!
As a corollary: Where are TFS changesets stored? I know that SQL server is used for some storage... changsets too?
We are currently using TFS 2008 but it would also be useful to know the answer for 2010 as I think there are plans to upgrade.
TFS stores all changeset data in a SQL Server database. It stores the code from the most recent changeset in its entirety. It then stores a diff to the previous version. One of the benefits of storing it all in SQL Server is that it gains the "all or none" saving capability that is provided by transactions.
As for WHERE it's stored, you can check out this blog entry http://blogs.msdn.com/b/bharry/archive/2009/04/19/team-foundation-server-2010-key-concepts.aspx