i using reportviewer v.10.0.0.0, runtime v2.0.50727. when i fill about 1 million records, memory being up to 2G or than.
It very bad, and if you know solution how to fix it, pls tell me know.
May be it's just that it's not designed well. E.g. there would be large object heap fragmentation.
GCSettings.LargeObjectHeapCompactionMode
Related
My Delphi6 program crashes because the TotalAddrSpace (THeapStatus) at some point hits the 2GB level, upon which it crashes. I have been able to increase the limit the 4GB level (using {$SetPEFlags $20}), but that only delays the eventual crash.
The problem is that the TotalUncommitted memory keeps increasing for some reason, while the TotalCommitted memory and also the TotalAllocated memory nicely stabilize at an acceptable level (about 550 MB).
I cannot quite figure out WHY the TotalUncommitted memory keeps increasing and eventually makes the TotalAddrSpace hit the 2GB (now: 4GB) level and the program crashes.
In the program I use many dynamic arrays, whose length I increase or reduce regularly with a simple adjustment via the SetLength command. Does this regular increasing/decreasing of dynamic arrays in this way effectively lead to an increase-beyond-bounds of the TotalUncommitted memory?
Any advice or insight is very much appreciated.
Also if you know of a general mechanism to somehow actively decrease the TotalUncommitted memory ...
Thanks for all help!!
My problem turned out to be one of Heap Fragmentation (or my understanding of it).
I used Setlength to increase/decrease, upon need, a dynamic array always in steps of 5 positions. Given the size of each array element, this apparently lead the OS to reserve much more memory than actually needed, which made the Heap.TotalUncommited grow without bounds, as did the Heap. TotalAddrSpace.
I tried different step sizes to see the impact. With a somewhat bigger step size the problem vanished.
I most strongly suggest you run a special build that has MemCheck included. It's a really great tool to detect memory leaks in your application. More modern Delphi versions have some of this built in (in part thanks to FastMem), but this one has been around since the first Delphi versions and works great on versions 5,6,7.
This is a follow-up to this question: What could explain the difference in memory usage reported by FastMM or GetProcessMemoryInfo?
My Delphi XE application is using a very large amount of memory which sometimes lead to an out of memory exception. I'm trying to understand why and what is causing this memory usage and while FastMM is reporting low memory usage, when requesting for TProcessMemoryCounters.PageFileUsage I can clearly see that a lot of memory is used by the application.
I would like to understand what is causing this problem and would like some advise on how to handle it:
Is there a way to know what is contained in that memory and where it has been allocated ?
Is there some tool to track down memory usage by line/procedure in a Delphi application ?
Any general advise on how to handle such a problem ?
EDIT 1 : Here are two screenshots of FastMMUsageTracker indicating that memory has been allocate by the system.
Before process starts:
After process ends:
Legend: Light red is FastMM allocated and dark gray is system allocated.
I'd like to understand what is causing the system to use that much memory. Probably by understanding what is contained in that memory or what line of code or procedure did cause that allocation.
EDIT 2 : I'd rather not use the full version of AQTime for multiple reasons:
I'm using multiple virtual machines for development and their licensing system is a PITA (I'm already a registered user of TestComplete)
LITE version doesn't provide enough information and I won't waste money without making certain the FULL version will give me valuable information
Any other suggestions ?
Another problem might be heap fragmentation. This means you have enough memory free, but all the free blocks are to small. You might see it visually by using the source version of FastMM and use the FastMMUsageTracker.pas as suggested here.
You need a profiler, but even that won't be enough in lots of places and cases. Also, in your case, you would need the full featured AQTime, not the lite version that comes with Delphi XE and XE2. (AQTIME is extremely expensive, and annoyingly node-locked, so don't think I'm a shill for SmartBear software.)
The thing is that people often mistake AQTime Allocation Profiler as only a way to find leaks. It can also tell you where your memory goes, at least within the limits of the tool. While running, and consuming lots of memory, I click Run -> Get Results.
Here is one of my applications being profile in AQTime with its Allocation Profiler showing exactly what class is allocating how many instances on the heap and how much memory those use. Since you report low Delphi heap usage with FastMM, that tells me that most of AQTime's ability to analyze by delphi class name will also be useless to you. However by using AQTime's events and triggers, you might be able to figure out what areas of your application are causing you a "memory usage expense" and when those occur, what the expense is. AQTime's real-time instrumentation may be sufficient to help you narrow down the cause even though it might not find for you what function call is causing the most memory usage automatically.
The column names include "Object Name" which includes things like this:
* All delphi classes, and their instance count and heap usage.
* Virtual Memory blocks allocated via Win32 calls.
It can detect Delphi and C/C++ library allocations on the heap, and can see certain Windows-API level memory allocations.
Note the live count of objects, the amount of memory from the heap that is used.
I usually try to figure out the memory cost of a particular operation by measuring heap memory use before, and just after, some expensive operation, but before the cleanup (freeing) of the memory from that expensive operation. I can set event points inside AQTime and when a particular method gets hit or a flag gets turned on by me, I can measure before, and after values, and then compare them.
FastMM alone can not even detect a non-delphi allocation or an allocation from a heap that is not being managed by FastMM. AQTime is not limited in that way.
I've just been analyzing an ipad app I'm developing using Instruments. In particular I was interested in the memory usage, as I have been receiving some memory warnings.
First of all the Activity monitor reports overall some 40MB of memory used just after starting the application. This really seams like a lot to me. Especially as after the startup nothing really fancy is going on.
So I have been analyzing the app in the VM tracker.
First of all can somebody explain how to interpret the dirty memory? I mean the ipad doesn't really have virtual memory, in the sense that there is no swapping etc.
Ok the really weird thing is that I have some 40 MB of dirty memory, that is resident! Some 38MB are listed under IOKit. Under IOKit there is no further information, what that actually means.
So what exactly does IOKit do?
What could be causing this insane those huge values?
Any kind of hint is appreciated! :)
Try Heapshot Analysis, bbum has a great tutorial here.
Basically you take a Heapshot, run some procedure, take another Heapshot for several iterations. This will help find memory that lost but not a leak. I use this method often,
I have used Heapshot many times to great advantage, many thanks to bum.
What is dirty memory?
According to this session.
memory written by an app
all heap allocations
decoded image buffers
VM profile shows some info of the dirty memory
like dirty memory size. They are anonymous.
vmmap --summary App.memgraph
In this session, this Apple dev uses heap to get more info about the object sizes.
heap App.memgraph
In a previous post ( My program never releases the memory back. Why? ) I show that FastMM can cache (read as hold for itself) pretty large amounts of memory. If your application just loaded a large data set in RAM, after releasing the data, you will see that impressive amounts of RAM are not released back to the memory pool.
I looked around and it seems that calling the SetProcessWorkingSetSize API function will "flush" the cache to disk. However, I cannot decide when to call this function. I wanted to call it at the end of the OnClick event on the button that is performing the RAM intensive operation. However, some people are saying that this may cause AV.
If anybody used this function successfully, please let me (us) know.
Many thanks.
Edit:
1. After releasing the data set, the program still takes large amounts of RAM. After calling SetProcessWorkingSetSize the size returns to few MB. Some argue that nothing was released back. I agree. But the memory foot print is now small AND it is NOT increasing back after using the program normally (for example when performing normal operation that does not involves loading large data sets). Unfortunately, there is no way to demonstrate that the memory swapped to disk is ever loaded back into memory, but I think it is not.
2. I have already demonstrated (I hope) this is not a memory leak:
My program never releases the memory back. Why?
How to convince the memory manager to release unused memory
If SetProcessWorkingSetSize would solve your problem, then your problem is not that FastMM is keeping hold of memory. Since this function will just trim the workingset of your application by writing the memory in RAM to the page file. Nothing is released back to Windows.
In fact you only have made accessing the memory again slower, since it now has to be read from disc. This method has the same effect as minimising your application. Then Windows presumes you are not going to use the application again soon and also writes the workingset in RAM to the pagefile. Windows does a good job of deciding when to write RAM to the pagefile and tries to keep the most used memory in RAM as long as it can. It will make the workinset size smaller (write to pagefile) when there is little RAM left. I would not mess with it just to give the illusion that you program is using less memory while in fact it is using just as much as before, only now it is slower to access. Memory that is accessed again will be loaded into RAM again and make the workinset size grow again. Touching less memory keeps the workingset size smaller.
So no, this will not help you forcing FastMM to release the memory. If your goal is for your application to use less memory you should look elsewhere. Look for leaks, look for heap fragmentations look for optimisations and if you think FastMM is keeping you from doing so you should try to find facts to support it. If your goal is to keep your workinset size small you could try to keep your memory access local. Maybe FastMM or another memory manager could help you with it, but it is a very different problem compared to using to much memory. And maybe this function does help you solve the problem you are having, but I would use it with care and certainly not use it just to keep the illusion that your program has a low memory usage.
I agree with Lars Truijens 100%, if you don't than you can check the FasttMM memory usage via FasttMM calls GetMemoryManagerState and GetMemoryManagerUsageSummary before and after calling API SetProcessWorkingSetSize.
Are you sure there is a problem? Working sets might only decrease when there really is a memory shortage.
Problem solved:
I don't need to use SetProcessWorkingSetSize. FastMM will eventually release the RAM.
To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I crated a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.
I used this function just once, when I implemented TWebBrowser. This component took me so much memory even if I destroyed the instance.
i'm using delphi 7 to run the hiercube version 4.32. When i retrieve the data, the pc memory keep increasing until occur 'Out of Memory' bug. I suspect the cache of memory did not clear when retrieve the data from hiercube, so when i retrieve data for 2nd or third times then it occur 'Out of Memory' error. Any idea to solve this 'Out of Memory' problem?
The obvious fix (work-around?) is to get more RAM. 1 GB is a bit on the low side these days, especially when 4 GB is at around the $100 mark.
Use FastMM as your memory manager, then either (or both):
make it report memory leaks on shutdown
use the memory monitor example form at run-time
Then start investigating the memory leak.
--jeroen