We are hosting a new Umbraco 7 site with the database on SQL server 2012 Express. The database seems to be growing rapidly. It's currently about 5Gb and SQL 2012 express has a maximum database size of 10Gb so we are starting to get a little concerned. The cmsPropertyData and cmsPreviewXml seem to be taking up the most space at about 2.5Gb each. Is there any general housekeeping that needs to be done to keep these tables under control. Have tried shrinking the database but there is no unused space.
Any Advice?
I don't know for sure this is the problem in your case, but Umbraco creates a new version of the content node each time the node is saved. This can cause your database to grow rapidly. There's a cool package that automatically removes older versions called "UnVersion". https://our.umbraco.org/projects/website-utilities/unversion/
Related
I have a server with 48 GB memory and a sql server analysis service (tabular mode), 2016 standard version SP1 CU7 installed on it.
I can deploy a tabular model from visual studio.
I can manually run a XMLA script:
{
"refresh": {
"type": "full",
"objects": [
{
"database": "MyCube"
}
]
}
}
But when i run that script from sql agent job, i get this error :
the JSON DDL request failed with the following error: Failed to execute XMLA. Error returned: 'There's not enough memory to complete this operation. Please try again later when there may be more memory available.'.. at Microsoft.AnalysisServices.Xmla.XmlaClient.CheckForSoapFault
The memory before porcessing is about 4GB, it increases during processing the cube, but when it hits about 18.5 GB, it fails.
Does anybody know a solution?
Analysis Services Tabular instances in SQL Server 2016 are limited to 16GB of RAM as documented here if you are running Standard Edition. Enterprise Edition removes that cap.
When you do a process full you keep a working copy of the cube and in the background you process a shadow copy. When the shadow copy is ready then it will replace the working copy. Basically this means that at processing time you need twice the amount of memory as the size of your cube. This can be an issue when you have the 16 GB limitation per instance with SSAS Standard edition.
One solution is to do a process with clearValues first, this empties the cube, and then to do the full process. More details here http://byobi.com/2016/12/how-much-ram-do-i-need-for-my-ssas-tabular-server/
Or another one is to play with the Memory \ VertiPaqPagingPolicy settings of the SSAS server. See more details here https://www.jamesserra.com/archive/2012/05/what-happens-when-a-ssas-tabular-model-exceeds-memory/ and here https://www.sqlbi.com/articles/memory-settings-in-tabular-instances-of-analysis-services/
And of course another solution is to upgrade to Enterprise Edition.
To follow up on Greg comments, i am facing similar issue at work and the workaround was instead of doing a database refresh, i did table refresh instead. I created 2 SQL jobs. My tabular model had 40 tables. So based on the sizes of the tables, i refresh x amount of tables in one job and y amount of table in the other jobs. You can create more than 2 SQL jobs and has less tables per job if you wish. This will put less load on the memory.
You can process small subsets of your data by partitioning your tables, this can be handled in SSMS. This Article provides a nice overview on how to achieve this.
We're about to deploy TFS 2012 - mainly for source control at this stage but will hopefully ultimately provide a full work-flow for us.
Can anybody point me towards a sizing guide for the database aspect ?
The short answer is "how long is a piece of string?".
To qualify that short answer a bit, there is obviously an overhead to begin with. TFS is much better than SourceSafe in that only changes are stored, so you don't get a different version of the file in the database for each check-in. This is a good thing.
That said the answer to this question really depends on how often you're going to be checking in, the amount of changes there are between those check-ins and the overall size of all the projects and their related files.
To give you some metric, on our TFS server, the supporting TFS databases plus our "collection" database which has been running for 6 months now, with regular daily check-ins, is hitting 800mb.
Now, unless you head a massive project, I can't see you going over a half a TB anytime soon. That said, given that TFS is SQL Server based - should you need to upgrade in the future it's not as much of a nightmare as you may think.
According to Microsoft's documentation:
Fewer than 250 users: 1 disk at 7.2k rpm (125 GB)
250 to 500 users: 1 disk at 10k rpm (300 GB)
500 to 2,200 users: 1 disk at 7.2k rpm (500 GB)
2,200 to 3,600 users:1 disk at 7.2k rpm (500 GB)
However, as Moo-Juice said, the real-world numbers are dependent on how you actually use TFS.
Also keep in mind that you'll want to also create, store, and maintain backups of your TFS databases.
Firebird 2.1.3 database seems to be creating a lot of garbage from uncompleted transactions this is causing the database to run very slowly until its garbage is removed via a database sweep or server restart. My database size its 30gb+.
Have you any idea what could be causing this?
Do any of the new stored procedures create excess garbage?
Please Help me.?
A Firebird database getting slow after a period of time is usually a sign of bad client transaction management. This can be easily checked by inspecting various transaction counters from the header page, which can be queried by running:
gstat -h <yourdatabase>
when your database becomes slow. For example: Pretty much all access libraries, when running transactions in auto commit mode (basically when you don't care about starting explicit transactions in your client application), are using COMMIT RETAINING, which basically blocks moving OIT/OAT forward.
Beside the gstat command-line tool, with Firebird 2.1 you also have the monitoring tables, in particular MON$TRANSACTIONS, to identify long-running transactions.
Hi I am using neo4j in my application and my structure is as following:
I am using Embedded Graph API
I have several databases that I point to using a pool that I maintain in my application eg-> db1, db2, db3, ..... db100
When I want to access a particular database I point to it using new EmbeddedGraphDatabase("Path to db(n)")
The problem is that when the connection pool count increases the RAM size being consumed by the application keep increasing and breaks down the application at a point of limit.
So I am Thinking of migrating from Neo4j to some other Database.
Additionally only a small part of my database is utilizing the graph structure.
One way for migration is that I write a script for it. Is there any better option?
My another question is what is the best Database so that my structure can be maintained.
Other view-point that I am thinking about is I can keep part of my data into Neo4j and shift another part to some other database.
If anything is unclear I can clarify.
Thanks in advance.
An EmbeddedGraphDatabase instance is not the equivalent of a "connection" in SQL. It's designed to run a long time (days, months). Hence starting/stopping is costly.
What is the use case for having hundreds of separate databases in the same JVM?
Your lots of small databases will perform poorly as the graphdb is designed to hold the whole datamodel on a single host.
Do you run a single JVM per database?
You can control the amount of memory used by neo4j by providing the correct properties for memory mapping and also use the gcr cache from neo4j-enterprise and control the cache size-property variables.
I think it still makes sense to keep the graph part in Neo4j and only move the non-graphy part.
We're trying to reload the TFS 2010 SSAS cube, but when the warehouse is processing, we get an exception in the log. It is important to note that the cube does not fail completely, but loads incompletely. For example, we have data up to June 2011, but not beyond.
Microsoft.TeamFoundation.Server.WarehouseException: OLE DB error: OLE
DB or ODBC error: Snapshot isolation transaction failed in database
'Tfs_Warehouse' because the object accessed by the statement has been
modified by a DDL statement in another concurrent transaction since
the start of this transaction. It is disallowed because the metadata
is not versioned. A concurrent update to metadata can lead to
inconsistency if mixed with snapshot isolation.; 42000
This is our future production system, and contains data migrated over from a TFS 2008 system. The database size of the version control repository is close to 200GB, so we're dealing with a relatively large instance of TFS.
We could remove snapshot isolation from our warehouse, but I'm a little concerned about doing this, as I can't find anything that tells me whether snapshot isolation is required on the TFS_Warehouse database. Any insight would be appreciated.
From this source (see the TempDB and RCSI section), it would seem that removing snapshot isolation could be a big mistake.
Here are some other options ordered easy to difficult from and implementation standpoint...
increase the size of TempDB to accomodate longer running SELECTs
decrease the partition size the measure groups in the cube. You may
want to run a profiler trace durring SSAS processing first to
determine which measure groups are taking the longest and chop those
partitions up first
implement an incremental processing strategy
Here's a link providing more information on cube-partitioning...