I'm trying to migrate a large ClearCase stream (it has a large history set too) to TFS 2010 using "TFS Integration Tools".
The tool hangs at TfsMigrationShell.exe Information: 0 : VersionControl: ClearCase history command: 'lshistory -minor -since 01-Jan-0001.00:00:00 -eventid -recurse -fmt "*%n*%o*%m*%e*%d*%Nc*%l##" -pname \IB_FE'
and then it gives the following run time error:
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.String.Split(String[] separator, Int32 count, StringSplitOptions options)
at Microsoft.TeamFoundation.Migration.ClearCaseDetailedHistoryAdapter.ClearCaseCommandSpec.ParseHistoryTable(String cmdOutput)
at Microsoft.TeamFoundation.Migration.ClearCaseDetailedHistoryAdapter.ClearCaseServer.GetHistoryRecords(ReadOnlyCollection'1 filters, DateTime since, Boolean writeHistoryRecordsFound)
at Microsoft.TeamFoundation.Migration.ClearCaseDetailedHistoryAdapter.ClearCaseAnalysisProvider.queryHistory(ChangeGroupService changeGroupService)
at Microsoft.TeamFoundation.Migration.ClearCaseDetailedHistoryAdapter.ClearCaseAnalysisProvider.GenerateDeltaTable()
at Microsoft.TeamFoundation.Migration.Toolkit.AnalysisEngine.GenerateDeltaTables(Guid sourceId)
Please advise.
Thanks in advance.
The issue is that:
a cleartool lshistory -minor can be huge for a repo with a large history
it also can be incomplete, since Vob scrubbing is run every week, unless those jobs have been modified to keep them. See "Keep minor event records after scrubbing a VOB database"
Since you cannot modify the tool in order to import only up to a certain data "d1", the "d1" to "d2", and so on until "present day", I would really consider:
importing only a very short history of ClearCase into TFS (the last 5 baselines for instance, if you were using ClearCase UCM)
keeping ClearCase in read-only mode, if you need to go back to older history (for archive).
You have a System.OutOfMemoryException raised, looks like you need more free RAM/Paging for the tool to run.
Close all the running applications you can, make sure the System Paging file is big enough (three times the amount of RAM, for instance).
Then try again.
Related
I am using the vNext build system of TFS 2015.
I currently have the my builds versioning in the traditional format. Major.Minor-rev.RevisionNumber. So, if I have a build for Major 1, Minor 12, the build version would look like 1.12-rev.1 when I start. I would like to know if it is possible to have the build version start at a number other than one, say 55. Such that the build version would look like 1.12-rev.55, and then increment by one as usual after that.
Actually, it is possible to effect this in a vNext build without hacking the database.
There are 2 steps.
First, you will need to implement a powershell build step (as the first step of the build) with the following inline script:
#Set the BuildNumberOffset. (Change this to the difference between the TFS build number,
#and the number that your build needs.)
$BuildNumberOffset = 543
#Don't change
$BuildNumberParts = $($env:BUILD_BUILDNUMBER) -split '\.'
$TFSRevision = [int]$BuildNumberParts[$BuildNumberParts.Length-1]
$BuildNumberParts[$BuildNumberParts.Length-1] = ($TFSRevision + $BuildNumberOffset).ToString()
$BuildNumber = [string]::Join(".", $BuildNumberParts)
Write-Host "##vso[build.updatebuildnumber]$BuildNumber"
Second, on the Label format field of the repository tab set the label format to "$(Build.BuildNumber)" instead of "$(Build.DefinitionName)$(rev:.r)". This is important so that your label will be the same as your updated build number.
There is a way to do this. It isn't pretty, but it works.
Assuming you have a build number format of something like $(Major).$(Minor)-rev$(rev:.r)
To do this queue up a build which will get the number 1.12-rev.1. Then go to the TFS database and into a table called tbl_Build. Find that last build you did and change the value in the BuildNumberRevision column to 54.
The next build that fires off will now be 1.12-rev.55
Unfortunately, it's impossible.
Every build definition has a build number format field where you can use some macros to dictate what the resulting build number should look like. In this format we are using $(Rev:.rr) Its start by one.
What is $(Rev:.rr)?
To ensure that every completed build has a unique name. When a build
is completed, if nothing else in the build number has changed, the Rev
integer value is incremented by one.
Source:MSDN
Moreover, if you want to generating a custom build number without increment.
Here is a blog with detailed procedures:Generate custom build numbers in TFS Build vNext
Couldn't we just manually set this in the format for one build i.e:
$(Major).$(Minor)-rev$(rev:.54)
and then afterwards revert back to:
$(Major).$(Minor)-rev$(rev:.r)
Not tried it, but if it works it'll save hacking around in the database.
You can do this easily, but only if you are using a Git repository in conjunction with a tool called GitVersion
It is a wonderful tool that I always use with my git repos. For your use-case: When you have version 1.2.3 and you want to jump to version 1.2.55, you just add a git tag 'v1.2.55' and it will start the versioning from there. GitVersion is a lot more complicated and does a lot more, but that is one of the features. You don't have to mess with special PowerShell scripts or anything, it instead reads your git repo history and git tags overrides the calculated versioning. There is already a TFS/VSTS/Azure Devops extension called GitVersion that works great from the same developer.
The Answer of #Steve Sims works still with TFS 2017 vNext. Thanks a lot!
I had only to do the first step "to add the script as an inline powershell script in my build." Thanks to #PainElemental
With this "Build number format" in the Options-Tab it works:
$(BuildDefinitionName)_1.2.0$(rev:.r)
I didn't label my sources with the build, so I don't checked:
Second, on the Label format field of the repository tab set the label
format to "$(Build.BuildNumber)" instead of
"$(Build.DefinitionName)$(rev:.r)". This is important so that your
label will be the same as your updated build number.
I think, you can edit the Label Format on the "Advanced" GetSources Options. (usually hidden).
This was also very painful for us, migrating from a existing CI system with it's own build numbering, we needed build numbers to increment from a specific value. Hacking databases wasn't allowed in the organisation, offsets seemed a cludge.
In the end, we used an AutoIt script to start and stop builds and delete the build result using the WebUI and left it running. Not nice, but it did the job.
Needs tweaking for screen resolutions and such, timings also perhaps. Use AutoIt Window Info to find the button locations, make sure browser is fullscreen (not windowed) Run it for a few cycles to ensure it robust before setting the loop larger.
#include <AutoItConstants.au3>
;Increment TFS build count (Chrome browser buttons locations). Start from build result page.
For $i = 1 To 15 Step 1
ConsoleWrite ( "Loop " & $i & " of 5" & #CRLF )
Sleep(200)
MouseClick($MOUSE_CLICK_PRIMARY, 1800, 200, 2)
Sleep(500)
MouseClick($MOUSE_CLICK_PRIMARY, 1150, 881, 2)
Sleep(2000)
MouseClick($MOUSE_CLICK_PRIMARY, 1800, 200, 2)
Sleep(500)
MouseClick($MOUSE_CLICK_PRIMARY, 1055, 165, 2)
Sleep(10000)
Next
TFS/Devops is a really immature CI system, and it's not a patch on what we were running. Unfortunately corporate policy said we move to TFS, as nobody ever got fired for buying Microsoft products (but plenty of people should have got fired for buying them/forcing them where they don't belong).
We are currently using Scrum 2.0 process template from microsoft in new TFS 2012, however we don't use hours as task estimates, we simply count tasks. This is easily archieved by setting each task value as 1 as default and set that field read only in task property form.
However original template uses hours as unit with tasks, so there are are mark "h" all around template. Is there way to get rid of this hour mark since it causes constant confusion? Especially in management.
You can configure how the remaining work is displayed, by modifying the format attribute in the following row in the commonconfiguration.xml file:
<TypeField refname="Microsoft.VSTS.Scheduling.RemainingWork" type="RemainingWork" format="{0} h" />
By default this attribute is "{0} h", so you can simply set it to "{0}" to fit your needs.
You can download the commonconfiguration.xml file from the server, using the witadmin command:
witadmin exportcommonprocessconfig /collection:http:<your collection> /p:<your project> /f:<the file name>
After editing the file you must import it back into TFS using witadmin command:
witadmin importcommonprocessconfig ...the same parameters as above
Note: with the RC it looks like you must leave a space after the placeholder, like "{0} ", otherwise you will get a validation error, when importing the file. I haven't tried yet with the RTM to see if this has been fixed.
As far as I can tell, all of those h's are hardcoded straight into web access. So the only way to remove them would be to find them (I used Chrome's dev tools), isolate them, and then go into the Web Access pages located on your TFS Server (application tier) and manually remove them. This is because they aren't actually part of any template, so there isn't a way to remove them all at once. The path to the files will be something like
c:\Program Files\Microsoft Team Foundation Server 11.0\Application Tier\Web Services
Good luck, I've had a heck of a time trying to alter them.
You can use TFPT or the WITADMIN command line tool to remove the remaining effort field.
I would suggest however not to remove it, but rather to remove it from the form itself (so that it just doesn't show), and add a rule so that it defaults to 1 and is read-only. Furthermore, in the workflow, add a rule that changes the value to 0 when you reach done.
Since there's no actual meaning to the number itself, the units can be actual hours, ideal hours, story points or whatever you want them to mean. By making each task 1 or 0 (when done), you can make use of TFS' built in tools such as the burndown reports to track your progress. Each task done will reduce the remaining work, and you will be able to keep track of velocity by summing up the effort completed in each iteration (which is the same as counting).
I was wondering if any of you guys had any experience generating code coverage reports in TFS Build Server 2010 while running NUnit tests.
I know it can be easily done with the packaged alternative (MSTest + enabling coverage on the testrunconfig file), but things are a little more involved when using NUnit. I've found some info here and there pointing to NCover, but it seems outdated. I wonder if there are other alternatives and whether someone has actually implemented this or not.
Here's more info about our environment/needs:
- TFS Build Server 2010
- Tests are in plain class libraries (not Test libraries - i.e., no testrunconfig files associated), and are implemented in NUnit. We have no MSTests.
- We are interested in running coverage reports as part of each build and if possible setting coverage threshold requirements for pass/fail criteria.
We 've done it with NUnit-NCover and are pretty happy with our results. NUnit execution is followed by NUnitTfs execution in order to get our testing results published in the Build Log. Then NCover kicks in, generating our code coverage results.
One major thing that poses as a disadvantage is fact that setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it.
Two things could pose as disadvantages:
NUnitTfs doesn't work well with NCover (at least I couldn't find a way to execute both in the same step, so (since NCover invokes NUnit) I have to run Unit tests twice: (1) to get the test results and (2) to get coverage results over NCover. Naturally, that makes my builds last longer.
Setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it .
In any case, the resulting reporting (especially the Trend aspect) is very useful in monitoring how our code evolves within time. Especially if you 're working on a Platform (as opposed to short-timed Projects), Trend reports are of great value.
EDIT
I 'll try to present in a quick & dirty manner how I 've implemented this, I hope it can be useful. We currently have NCover 3.4.12 on our build server.
Our simple naming convention regarding our NUnit assemblies is that if we have a production assembly "123.dll", then another assembly named "123_nunit.dll" exists that implements its tests. So, each build has several *_nunit.dll assemblies that are of interest.
The part in the build process template under "If not disable tests" is the one that has been reworked in order to achieve our goals, in particular the section that was named "Run MSTest for Test Assemblies". The whole implementation is here, after some cleanups to make the flow easier to be understood (pic was too large to be directly inserted here).
At first, some additional Arguments are implemented in the Build Process Template & are then available to be set in each build definition:
We then form the NUnit args in "Formulate nunitCommandLine":
String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL))
This is then used in the "Invoke NUnit"
In case this succeeds & we have set coverage for this build we move to "Generate NCover NCCOV" (the coverage file for this particular assembly). For this we invoke NCover.Console.exe with the following as Args:
String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""",
NUnitPath,
Path.GetFileName(nunitDLL),
Path.GetDirectoryName(nunitDLL),
Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)),
Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"),
Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""),
BuildDetail.BuildNumber)
All these run in the foreach loop "For all nunit dlls". When we exit the loop, we enter "Final NCover Activities" & at first the part "Merge NCCovs", where NCover.Console.exe is executed again - this time with different args:
String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject
)
When this has run, we have reached the point where all NCCOV files of this build are merged into one NCCOV-file named after the build + the Trend file (that monitors the build throughout its life) has been updated with the elements of this current build.
We now have to only generate the final HTML report, this is done in "Generate final NCover rep" where we invoke NCover.reporting with the following args:
String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ",
Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
BuildDetail.BuildNumber,
PathForNCoverResults,
NCoverDropLocation,
BuildDetail.BuildDefinition.TeamProject,
BuildType
)
I have a task to create reports about various work items from a Team Foundation Server 2010 instance. They are looking for more information than the query tools seem to expose which is why I am not using the OOB reporting capabilities. The documentation on creating custom reports against TFS identify the Tfs_Analysis cube and the Tfs_Warehouse database as the intended sources for reporting.
They have created a custom work item, "Deployment Requests", to track requests for code migrations. This work item has custom urgency levels (critical, medium, low).
According to Manually Process the Data Warehouse and Analysis Services Cube for Team Foundation Server, every two minutes my ODS (Tfs_DefaultCollection) should sync with the Tfs_Warehouse and every 2 hours it hits the Tfs_Analysis cube. The basic work items correctly show up in my Tfs_Warehouse except not all of the data makes it over, in particular, the urgency isn't getting migrated.
As a concrete example, work item 19301 was a deployment request. This is what they can see using the native query tool from the web front-end.
I can find it in the Tfs_DefaultCollection and the "Urgency" is mapped to Fld10176.
SELECT
Fld10176 AS Urgency
, *
FROM Tfs_DefaultCollection.dbo.WorkItemsAre
WHERE ID = 19301
trimmed results...
Urgency Not A Field Changed Date
1 - Critical - (Right Away) 58 2011-09-07 15:52:29.613
If I query the warehouse, I see the deployment request and the "standard" data (people, time, area, etc)
SELECT
DWI.System_WorkItemType
, DWI.Microsoft_VSTS_Common_Priority
, DWI.Microsoft_VSTS_Common_Severity
, *
FROM
Tfw_Warehouse.dbo.DimWorkItem DWI
WHERE
DWI.System_Id = 19301
Trimmed results
System_WorkItemType Microsoft_VSTS_Common_Priority Microsoft_VSTS_Common_Severity
Deployment Request NULL NULL
I am not the TFS admin (first exposure to TFS is at this new gig) and thus far, they've been rather ...unhelpful.
Is there be a way to map that custom field over to an existing field in the Tfs_Warehouse? (Backfilling legacy values would be great but fixing current/future is all I need)
Is there a different approach I should be using?
Did you mark the field as reportable? See http://msdn.microsoft.com/en-us/library/ee921481.aspx for more information about this topic.
Based on Ewald Hofman's link, I ran
C:\Program Files\Microsoft Visual Studio 10.0\VC>witadmin listfields /collection:http://SomeServer/tfs > \tmp\witadmin.txt
and discovered a host of things not configured
Reportable As: None
At this point, I punted the ticket to the TFS admins and indicated they needed to fix things. In particular, examine these two fields
Field: Application.Changes
Name: ApplicationChanges
Type: PlainText
Use: Project1, Project2
Indexed: False
Reportable As: None
or
Field: Microsoft.VSTS.Common.ApplicationChanges
Name: Application Changes
Type: Html
Use: Project1, Project2
Indexed: False
Reportable As: None
It will be a while before the TFS Admins do anything but I'm happy to accept Edwald's answer.
I'm trying to figure out a way to find out which files were affected by a work item in TFS 2008.
I realize that this is a duplication of a question already asked by someone else here - View a list of all files changed as part of a Workitem in TFS but it went unanswered and I've been, off and on, looking for this for a while.
I understand can view the links tab of the work item and then view each changeset to see the files that have been changed. But, the work item very likely will end up with many changesets linked to it, and I would like to review the files modified as part of the work item, but I feel like the likelihood of missing a file or two is very high if I have to rely on looking at each of the 100+ changesets individually.
Does anyone know of a way to accomplish this? Thanks in advance for any help or guidance.
Sounds like a job for Powershell...
function Get-TfsItem([int] $workItemNumber)
{
Get-TfsServer njtfs -all |
foreach { $_.wit.GetWorkItem($workItemNumber) } |
foreach { $_.Links } |
foreach { ([regex]'vstfs:///VersionControl/Changeset/(\d+)').matches($_.LinkedArtifactUri) } |
foreach { $_.groups[1].value } |
Get-TfsChangeset |
Select-TfsItem |
Sort Path -Unique
}
The first several lines are kind of ugly. We have to hit the webservice API directly since the TFS cmdlets don't cover the bug tracking system. And the objects we get back require some regular expression love before they'll do what we need. Piping to "foreach" over & over is an unfortunate Powershell idiom that arises when you mate an unfriendly API to a lame projection operator. (I use my own replacement, personally, but you can't rely on that.)
The last 3 lines should be self explanatory if my TFS Power Cmdlets are installed & doing their job.
I just found Scrum Power Tools plugin for VS 2010 that does this with a button click in VSS, installed and it worked. http://visualstudiogallery.msdn.microsoft.com/3f261226-530e-4e9c-b7d7-451c2f77f262
I am just trying to make the powershell version 2010 work. http://msdn.microsoft.com/en-us/vstudio/bb980963
First problem is that the pwoer shell option is not installed by default, use custom install and select that option. When completed there is a powershell prompt in the TFS powertools 2010 menu, the commands only work in there.
The get server I had to replace njtfs with the url http://tfsserver:8080/tfs and remove -all. The script still fails.
Ultimately I need a detail report that lists:
source 'work item' 'change set'
For example:
xyz.cs 'work item 1' 'C397'
xyz.cs 'work item 2' 'C399'
Eventually I have to then work out that work item 1 is dependant on work item 2. I also have to track back to work item 1 to check the status.
Can someone assist with a 2010 version script? I have never written PS before.
I needed the exact same thing and I wrote a TFS utility for myself, using TFS API. It allows you to see all changes a work item triggered over time, and some things more. I've put it on codeplex. You can get it from:
tfshelper.codeplex.com