Delphi 7, DUnit and FastMM reporting Strings incorrectly - delphi

Question

I'm using DUnit and FastMM to catch unfinalized memory blocks but there seems to be a Bug. I dunno if its in FastMM, DUnit or in Delphi itself, but here goes:
When my Test Case has internal strings, the test fails with memory leaks. If I run the same test again without closing the DUnit GUI, the test passes OK. The same occours with DUnit GUI Testing, I believe for the same reason. There are no Leaks in my app, the proof is that FastMM doesn't generate the leak report in those cases.
Question 1: Is there a way to ignore them without setting the AllowedMemoryLeakSize
Question 2: I'm using Delphi 7, any news if this fix in Delphi XE?
My actual test configuration:
test.FailsOnNoChecksExecuted := True;
test.FailsOnMemoryLeak := True;
test.FailsOnMemoryRecovery := False;
test.IgnoreSetUpTearDownLeaks := True;
Here's a sample code (implementation only)
procedure TTest.Setup;
begin
A := 'test';
end;
procedure TTest.TearDown;
begin
// nothing here :)
end;
procedure TTest.Test;
begin
CheckTrue(True);
end;
Thanks!!!!
UPDATE: The problem i'm facing is documented in http://members.optusnet.com.au/mcnabp/Projects/HIDUnit/HIDUnit.html#memoryleakdetection
But the same link doesn't present a solution other than running the same test again.

Answer 1

Actually, strictly speaking your test is leaking memory on the first run.
It's not a bug in FastMM, DUnit or in Delphi, the bug is in your test.
Let's start by clearing up misconceptions, and explaining some inner workings:
Misconception: FastMM proves there are no leaks in my app
The problem here is that FastMM can give you a false sense of security if it doesn't detect leaks. The reason is that any kind of leak detection has to look for leaks from checkpoints. Provided all allocations done after the Start checkpoint are recovered by the End checkpoint - everything's cool.
So if you create a global object Bin, and send all objects to the Bin without destroying them, you do have a memory leak. Keep running like and your application will run out of memory. However, if the Bin destroys all its objects before the FastMM End checkpoint, FastMM won't notice anything untoward.
What's happening in your test is FastMM has a wider range on its checkpoints than DUnit leak detection. Your test leaks memory, but that memory is later recovered by the time FastMM does its checks.
Each DUnit test gets its own instance for multiple runs
DUnit creates a separate instance of your test class for each test case. However, these instances are reused for each run of the test. The simplified sequence of events is as follows:
Start checkpoint
Call SetUp
Call the test method
Call TearDown
End checkpoint
So if you have a leak between those 3 methods - even if the leak is only to the instance, and will be recovered as soon as the object is destroyed - the leak will be reported. In your case, the leak is recovered when the object is destroyed. So if DUnit had instead created & destroyed the test class for each run, no leak would be reported.
NOTE This is by design, so you can't really call it a bug.
Basically DUnit is being very strict about the principle that your test must be 100% self contained. From SetUp to TearDown, any memory you allocate (directly/indirectly) must be recovered.
Constant strings are copied whenever they are assigned to a variable
Whenever you code StringVar := 'SomeLiteralString' or StringVar := SomeConstString or StringVar := SomeResourceString the value of the constant is copied (yes, copied - not reference counted)
Again, this is by design. The intention is that if the string was retrieved from a library, you don't that string to be trashed if the library is unloaded. So it's not really a bug, just an "inconvenient" design.
So the reason your test code leaks memory on the first run is that A := 'test' is allocating memory for a copy of "test". On the subsequent runs, another copy of "test" is made, and the previous copy is destroyed - but the net memory allocation is the same.
Solution
The solution in this particular case is trivial.
procedure TTest.TearDown;
begin
A := ''; //Remove the last reference to the copy of "test" and presto leak is gone :)
end;
And in general, you shouldn't have to do much more than that. If your test creates child objects that reference copies of constant strings, those copies will be destroyed when the child objects are destroyed.
However, if any of your tests pass references to strings to any global objects / singletons (naughty, naughty, you know you shouldn't be doing that), then you'll have leaked a reference and hence some memory - even if it is recovered later.
Some further observations
Going back to the discussion about how DUnit runs tests. It is possible for separate runs of the same test to interfere with each other. E.g.
procedure TTestLeaks.SetUp;
begin
FSwitch := not FSwitch;
if FSwitch then Fail('This test fails every second run.');
end;
Expanding on the idea, you can get your test to "leak" memory on the first and every second(even) run.
procedure TTestLeaks.SetUp;
begin
FSwitch := not FSwitch;
case FSwitch of
True : FString := 'Short';
False : FString := 'This is a long string';
end;
end;
procedure TTestLeaks.TearDown;
begin
// nothing here :( <-- note the **correct** form for the smiley
end;
This doesn't really result in overall consumption of memory increasing because each alternate run recovers the same amount of memory that is leaked on every second run.
The string copying results in some interesting (and perhaps unexpected) behaviour.
var
S1, S2: string;
begin
S1 := 'Some very very long string literal';
S2 := S1; { A pointer copy and increased ref count }
if (S1 = S2) then { Very quick comparison because both vars point to the same address, therefore they're obviously equal. }
end;
However....
const
CLongStr = 'Some very very long string literal';
var
S1, S2: string;
begin
S1 := CLongStr;
S2 := CLongStr; { A second **copy** of the same constant is allocated }
if (S1 = S2) then { A full comparison has to be done because there is no shortcut to guarantee they're the same. }
end;
This does suggest an interesting, though extreme and probably ill-advised workaround just due to the sheer absurdness of the approach:
const
CLongStr = 'Some very very long string literal';
var
GlobalLongStr: string;
initialization
GlobalLongStr := CLongStr; { Creates a copy that is safely on the heap so it will be allowed to be reference counted }
//Elsewhere in a test
procedure TTest.SetUp;
begin
FString1 := GlobalLongStr; { A pointer copy and increased ref count }
FString2 := GlobalLongStr; { A pointer copy and increased ref count }
if (FString1 = FString2) then { Very efficient compare }
end;
procedure TTest.TearDown;
begin
{... and no memory leak even though we aren't clearing the strings. }
end;
Finally / Conclusion
Yes, apparently this lengthy post is going to end.
Thank you very much for asking the question.
It gave me a clue as to a related problem I remember experiencing a while back. After I've had a chance to confirm my theory, I'll post a Q & A; as others might also find it useful.

Answer 2

I would try the current release from Subversion first (but this version does not work with Delphi 7, only 2007 and newer):
In the commit log, one version has a comment about a fix in the area
Revision 40 Modified Fri Apr 15 23:21:27 2011 UTC (14 months ago)
move JclStartExcetionTracking and JclStopExceptionTracking out of
DUnit recursion to prevent invalid memory leak reporting

Answer 3

I found a way to lessen the problem: instead of working with Strings, I used ShortStrings and WideStrings in the Test Classes. No leaks poped from them.
It's not the solution, which by the way seems to be solved in the newest Delphi versions.

Answer 4

Bottom line is that the detected leak may be irrelevant to the test case being executed but it is a legitimate leak at the time it is detected. The memory for the string was unallocated prior to entry into the SetUp procedure and it is not deallocated prior to exiting from the TearDown procedure. So it is a memory leak until either the string variable is reassigned or the test case is destroyed.
For strings and dynamic arrays you can use SetLength(<VarName>, 0) in the TearDown procedure.