We have been using the old style (XAML) of builds until a month ago and then started using vNext builds. After that I noticed that tbl_Content table in TFS database started growing really fast. For instance, in the last 8 hours it grew 10 GB and I can't figure out what why it is doing that. Does any one know what it is?
The reason I am saying it is the vNext build is because I noticed it started growing after we upgraded, but I way be wrong. I hope it is not trying to store my build output or something like that. Is there way to know what is in the tbl_Content table? Can they be deleted? Or is there way to delete some things from that table without messing up TFS?
TFS Databases grows for all sorts of reasons. Some common sources of
runaway growth include using build or release automation without
configuring appropriate retention policies, creation of a
small number of very large items (work items, version control files,
etc.) by humans or tooling, and so forth. We have work on the backlog
to make the distribution of space in TFS databases more discoverable.
You could try to use a sql script such as below to show a increase of the tbl_Content over the last months:
select DATEPART(yyyy, CreationDate) as [year],
DATEPART(mm, CreationDate) as [month],
count(*) as [count],
SUM(DATALENGTH(Content)) / 1048576.0 as [Size in Mb],
(SUM(DATALENGTH(Content)) / 1048576.0) / count(*) as [Average Size]
from tbl_Content
group by DATEPART(yyyy, CreationDate),
DATEPART(mm, CreationDate)
order by DATEPART(yyyy, CreationDate),
DATEPART(mm, CreationDate)
It's able to know to look at the distribution of "owners" for the data in tbl_Content through a SQL query. Detail steps please refer Aaron Hallberg's reply in this similar question: TFS Database size
To reduce the size of the tbl_Content table, you could refer to this blog: TFS tbl_Content Table and Database growth out of control,it can be summed up in three steps in general:
Clean some old workspaces that you doesn't need any more.
Run the tf destory command to delete those unnecessary source files permanently.
Using TFS power tool to clean Test attachments and test results.
I was googling around and tried doing all kinds of things to see what is making the problem. Other problem is there is no documentation about the TFS database schema. After spending almost 10 hours with playing with TFS Database I almost become a TFS DB pro. Querying around the tbl_Content, tbl_FileMetadata and tbl_FileType helped me little to figure what exactly is making the problem.
So at the end removing the /v:diag from MSBuild arguments fixed my issue. I am not sure what all it was putting to the database, but each build was adding close to 1GB data to the database.
Related
We develop every US and Bug on a branch to be able to review the code related with that element so we generate a nice amount of branches. Every month we delete the old ones and traditionally this process has been way too slow.
We can only commit the deletion of 6 to 10 branches at once, otherwise we get a timeout. The deletion of 10 branches takes more that 5 minutes.
We have migrated recently from TFS 2010 and VS 2010 to TFS 2015 and VS2015 and nothing changed.
Is this normal? Is there a way to speed up this? Looking for an answer in google all I found is something related with local workspaces which is not the case, the workspace is remote.
Regards.
First, please check if other operations also delay so long, like adding a file, are fast.
And you could use tf delete command instead of GUI in VS. Which may do the trick. Deleting the branch will only actually perform a "soft delete". The branch will still exist complete with all of it's history, except it will be hidden. If you want to permanently deletes version-controlled files from Team Foundation version control, need to perform a tf destroy command.
Also check if it's a client issue, like a conflicting Visual Studio add-in. Try to clear TFS and VS cache.Besides, you could use another account and machine to do the delete operation next month.
Must to say there is some delay of deleting branch in TFS using VS GUI. I have created 20 empty branches for test. It took about 30 seconds to complete the whole delete operation. So if your branch have a number of folders with a large number of files, then the deletion of 10 branches takes about 5 minutes looks like acceptable. After all in TFVC, we don't create and delete branches frequently like GIT.
I have the following problem with Team Foundation Server Express 2015.
I ran out of SQL Database Space. So I cleaned up a littlebit and used TFS Destroy to permanently delete some larger files from source control.
I expected that the Size auf the table dbo.tbl_Content would get less after some time. But the opposite happened. 3000 Entries where added to the table and it got larger.
How can I cleanup TBL_Content from old files that have been destroyed already?
The TBL_Content Table has now 8.8 GB.
Executing stored procedures like prc_DeleteUnusedContent and [dbo].prc_DeleteUnusedFiles did not help at all.
Orphan data still leaves after running the tf destroy command, and will be deleted later by a TFS Background job agent. That is the reason why table dbo.tbl_Content size doesn't drops.
Here are some steps you can try to reduce the size of TFS databases:
1). Download and install TFS2015 Power Tool to get the Test Attachment Cleaner to remove large test attachments:
Tcmpt.exe attachmentcleanup /collection: TeamProjectCollectionUrl /teamProject: TeamProject /settingsFile: SettingsFilePath [/outputFile: ReportFilePath] [/mode: Preview/Delete]
2). Remove old workspaces which is no longer used for long time.
3). Delete old unused team projects: https://msdn.microsoft.com/en-us/library/ms181482(v=vs.140).aspx
We have a 50GB on-premises TFS collection, which we want to have migrated to TFS online. We are now using the OpsHub Visual Studio Online Migration Utility, to test with. The conversion of some separate test projects seem to work fine, so now we are trying to measure the time it will take for a full migration. It looks like the migration of all 78K changesets will take about a month. That is way too long for us. The tool is using only one Core of our 8 available Cores. I don't see other resource bottlenecks, other than the CPU (the Core in use is over 50% most of the time).
Are there any tips for a faster migration? Is there any way to speed up things?
Besides asking you all for a possible solution, I am now also trying to do several smaller migrations in parallel. As soon as I have results of that, I will share them.
Which version you are using?
There is performance issue with label which is fixed in version 1.0.1.005.
If your projects are independent in commit (Branch and Merge operation) then you can create migration, project wise which run parallel.
Last week I upgraded our TFS 2012 Server to TFS 2013. I read the MSDN documentation first and I also followed the documentation as I performed the upgrade. Everything seemed to go ok.
After the upgrade I ended up with 7 or so Team Projects that the wizard couldn't configure, for whatever reason, and needed manual configuration.
I noticed this week that ALL of the work items under one of my Team Projects are missing. Gone. Like even if I select Team|Go To Work Item and enter in a known Work Item Id, I receive an error that the item is either missing or I do not have permission to view it. I'm an Administrator on the TFS server and I'm the TFS Admin, so I highly doubt permissions are the issue.
I remoted into the server and launched SSMS to explore the raw data. I know for a fact Work Item 450 is missing (it's the only Id I remember at this point). I selected the TOP 1000 from WorkItemsAre, which seems to be the table that holds the Work Items (?). There is a gap in the Ids, I see 1-448, then the numbering picks up again at 457. So, somehow my Work Items appear to have been deleted. I stopped there, I assume there are more gaps since I'm missing more than 9 items.
Now I haven't gone through every one of our Team Projects. I've only touched 3 of them since the upgrade. Thankfully the largest, most active Team Project, with the most work items/version history seems to be intact. I'm not sure if any other Team Projects are missing their Work Items too.
Has anyone else experienced this? Does anyone know if there's some "secret squirrel" way to recover these missing work items, or have they been hard deleted and are gone for good (other than looking through tape backups of the server).
Any advice would be appreciated.
I already migrated to TFS 2013 from TFS 2012.
The problem of manual configuring the project may occur when you have customized work item types in TFS Project Templates. Did you customize your project templates?
Although, I can hardly believe that work-items getting hard deleted from TFS. This issue may occur probably because of archiving during migration. The workitems that TFS upgrade wizard may not have "understood" during migration/upgrade, might be archived and moved to another table in database Tfs_DefaultCollection.
You may want to consider that. I am not sure if that may be the case, but this happened when we migrated from TFS 2010 to TFS 2012 because we had many custom work items in TFS 2010. Hence we had to standardize templates before migration using powershell. But we lost some amount of history.
Hope that sparks some idea.
I'm trying to perform a baseless merge between two branches using tf.exe.
When I run the command, it fails and tells me that there 15-20 files that are locked by another user. I've looked at the other user's pending changes and he only has three files checked out in the target branch and none of them are locked.
Has anyone experienced something similar to this before?
Edit
With help from Robaticus, I've found that the locks are in the tbl_lock table, but the pending changes they are associated with do not exist any more. It looks like there are multiple orphaned locks other than the ones that are giving us the current merge issue.
Does anyone know what causes these or a way of fixing them that does not involved touching the DB directly? I'm off to do a little research to see if I can find anything.
Edit 2
First Google search for 'tfs orphaned locks' returns this: Orphaned locks in TFS
The TFS team recommends deleting them directly from the table, so I guess I'll have to do that.
I didn't see any posts indicating that it had been fixed yet.
You may want to check to see if they are locked in an abandoned workspace. To do this, install the TFS Power Tools (October edition) for TFS 2008. Then you can right click on a folder in the source explorer and "Find in source control." Use the "Status..." and you can find all files that are locked by that user.
If you have administrative rights, you can selectively remove the locks from any locked files.
* EDIT: Dangerous Code Ahead *
The code below can be executed against the TFSVersionControl database. This will show you all files that are locked. This might tell you if you have an orphaned file out there somewhere.
I wouldn't recommend that you make any changes within that database, and, of course, always make sure you have a current backup of your repository before you ever do something like this.
select
lck.FullPath,
wspc.WorkSpaceName,
id.DisplayName
from tbl_lock lck
inner join tbl_workspace wspc on wspc.WorkSpaceId = lck.WorkSpaceId
inner join tbl_identity id on id.IdentityId = wspc.OwnerId
Did you use the Attrice Sidekicks to look into this issue. It is a nice UI around the commandline tools of TFS.