This is not a major leak, but more of a would be nice to tidy I think, but I have found that my Delphi XE code can leak a String. This is because it is defined as a threadvar as it needs to be, but when the thread terminates, it is apparently not tidying up such variables.
Is there a way for me to manually tidy a string on termination of the thread? Do I just assign an empty string to it, or set it to nil or something?
Assign an empty string to it, set it to nil or call Finalize() on it. They are all equivalent and they will deallocate the storage thus removing your memory leak.
In response to Marco's comment, the documentation is explicit on this:
Dynamic variables that are ordinarily
managed by the compiler (long strings,
wide strings, dynamic arrays,
variants, and interfaces) can be
declared with threadvar, but the
compiler does not automatically free
the heap-allocated memory created by
each thread of execution. If you use
these data types in thread variables,
it is your responsibility to dispose
of their memory from within the
thread, before the thread terminates.
For example:
threadvar
S: AnsiString;
S := 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
...
S := ; // free the memory used by S
Rather bizarrely the documentation contains a clear error in the final line which should read S := nil;
It is of course easy to see for yourself that thread local variables are not disposed automatically:
program LeakMe;
{$APPTYPE CONSOLE}
threadvar
s: string;
begin
ReportMemoryLeaksOnShutdown := True;
s := 'Leak me';
end.
Some other small solution - you can use ShortString instead of String here (or any other kind of fixed length array), then memory leak disappear
Related
I am having a memory problem that cannot explain root cause, please help me!
The problem is as follows:
My program throws exception when i exit.
I investigated and thought that the cause of the error was the use Copymemory function to copy the record containing the variable have string data type.
Below is my demo program by delphi 2009:
In *.dpr file, I added ShareMem unit for use BorlndMM.dll instead of using FastMM memory management.
I define a struct containing a String variable.
Allocate 1 array of 2048 PByte elements.
Use Memory Copy function to copy 2 struct.
finally, Free array 2048.
when i exit program, my program threw exception.
type
TMyStructure = record
F1: TMyStructure;
str1: string;
unit 1
procedure TForm9.FormCreate(Sender: TObject);
var
i: Integer;
begin
F1.str1 := 'This is a string to demo for copying the String data type
using the Copymemory method';
end;
unit 2
procedure TForm8.btn1Click(Sender: TObject);
var
Form9: TForm9;
i: Integer;
s1: array[0..2048-1] of PByte;
begin
// Alloc 1st 2048 segments, each segment gain 1KB
for i := 0 to Length(s1) - 1 do
begin
s1[i] := AllocMem(1024);
end;
// Create Form9
Form9 := TForm9.Create(Owner);
// -> ERROR
CopyMemory(#Self.F1, #Form9.F1, SizeOf(TMyStructure));
// -> OK
// Self.F1 := Form9.F1;
// Free memory s1 array
for i := 0 to Length(s1) - 1 do
begin
FreeMem(s1[i]);
end;
end;
Note:
I have known management in memory of string is reference counting.
I've been investigating because of the use of Memorycopy, so reference count of the string is not incremented although there are 2 references to it.
I investigated that when the string was free for the first time, its reference count dropped to zero, but that memory was not returned to the OS.However, it seems that when another free variable has a nearby address, that memory loses the reference, so when accessing the second free will cause an exception.
Since no source code can be found as well as documentation describing the operation mechanism of BorlndMM.dll should read the spec as well as GetMem.inc source code file to understand the mechanism of operation FastMM and speculate the root cause of the bug. I'm guessing when a free variable, BorlndMM find ahead and after that free space and the combination leads to the memory area that can not be referenced anymore.Of course, using Memorycopy to copy two string variables is a action wrong and must fix. However, I would like to understand how the Memory Management mechanism works to explain the root cause of the above phenomenon.
Expect for help!
If possible please explain the root cause of the above phenomenon for me. And, If there is a document / source explaining how the operation of Memory Management 2005-BorlndMM , please send it to me. Thanks you so much!
The problem with your code is that it bypasses string reference counting. By copying the structure using CopyMemory you end up with two variables, both referring to the same string object. But that object still has a reference count of one.
What happens after that is unpredictable. You may encounter runtime errors. Or you may not. The behaviour is not well defined.
From your question you appear to be trying to understand why different memory managers lead to different behaviours. The thing about undefined behaviour is that it is, well, not defined. There's no point trying to reason about the behaviour. It's not possible to reason about it.
You must fix your code by replacing the memory copy with simple assignment:
F1 := Form9.F1;
Let's start with this simple Delphi 2010 snippet:
var
StringList: TStringList;
begin
ReportMemoryLeaksOnShutdown := True;
StringList := TStringList.Create;
StringList.LoadFromFile('c:\fateh.txt');
RegisterExpectedMemoryLeak(StringList);
FastMM4 report the memory leak again and again even with Addr(StringList) as parameter
so how to Register Expected MemoryLeak and why the methods sitting above doesn't work
thank's in advance.
You only registered the leak of the string list object. You also need to register that you are leaking all the objects owned by the string list. In this case it owns StringList.Count instances of string objects. The memory manager does not know that those strings are owned by the string list object and so will also be leaked.
And that's much easier said than done. Because you need to find the start of the memory block that represents a string. That's at a fixed offset from the string's first character, and the offset depends on which Delphi version you use.
In Unicode Delphi, in 32 bit code, the offset is 12 bytes. So the following will register the leaked strings:
for i := 0 to StringList.Count-1 do
if StringList[i]<>'' then
RegisterExpectedMemoryLeak(PByte(StringList[i])-12);
Even when you do that you'll still get two reported memory leaks. At least one of those is explained by the dynamic array that is owned by the string list, TStringList.FList. If you wanted to register that leak, then you'll need to do some more hacking because again you'll have to rely on implementation details as to where that array is stored.
I think that C++ supports something on the lines of :
Object objects[100];
This would instantiate a 100 objects, right? Is it possible to do this in Delphi (specifically 2007)? Something other than:
for i:=0 to 99 do
currentObject = TObject.Create;
or using the Allocate function, with a passed size value a hundred times the TObject size, because that just allocates memory, it doesn't actually divide the memory and 'give' it to the objects.
If my assumption that the c++ instantiation is instant rather than under-the-hood-iterative, I apologize.
What you are looking for is impossible because
Delphi does not support static (stack allocated) objects.
Delphi objects do not have default constructors that can be automatically invoked by compiler.
So that is not a lack of 'sugar syntax'.
For the sake of complete disclosure:
Delphi also supports legacy 'old object model' (Turbo Pascal object model) which allows statically allocated objects;
Dynamic object allocation itself does not prevent automatic object instantiation syntax, but makes such a syntax undesirable;
Automatic object instantiation syntax is impossible because Delphi does not have default constructors: Delphi compiler never instantiate objects implicitly because it does not know what constructor to call.
While you can't do what you want using objects, if your objects are relatively simple, you may be able to get what you want by using an array of records.
Records in Delphi can have properties (including setters and getters), and class and instance methods. They are created automatically when declared, so declaring an array of them will create them all without iterating.
For more info: http://docwiki.embarcadero.com/RADStudio/XE3/en/Structured_Types#Records_.28advanced.29.
(I'm not sure when the new functionality was added to Delphi, it may well be after the 2007 version).
I don't know of any non-hacky way to do this besides iterating:
var
MyObjects: array[0..99] of TMyObject;
i: Integer;
begin
for i := 0 to 99 do
MyObjects[i] := TMyObject.Create;
end;
That declaration wouldn't create 100 objects, it would just give you an array of 100 object references that point to nothing useful.
Creating an object is a two step process. The first step is allocating memory (which your code also doesn't), the second step is calling the constructor (Create method) to initialize that memory, create additional objects, etc, etc.
The allocation part can be done without the loop, but the constructor needs to be called to intialize each instance.
Many VCL classes don't have an additional constructor. They just have the empty constructor that does nothing. In that case, there is no need to call it.
For instance, to fetch an array of stringlists, you can use the following code, adjusted from this example:
type
TStringListArray = array of TStringList;v
var
Instances: array of Byte;
function GetStringLists(Number: Integer): TStringListArray;
var
i: Integer;
begin
// Allocate instance memory for all of them
SetLength(Instances, Number * TStringList.InstanceSize);
// Zero the memory.
FillChar(Instances[0], Length(Instances), 0);
// Allocate array for object references.
SetLength(Result, Number);
for i := 0 to High(Result) do
begin
// Store object reference.
Result[i] := #Instances[i * TStringList.InstanceSize];
// Set the right class.
PPointer(Result[i])^ := TStringList;
// Call the constructor.
Result[i].Create;
end;
end;
And to get an array of 100 stringlists:
var
x: TStringListArray;
begin
x := GetStringLists(100);
So while this procedure may save you a neglectable amount of time, and may theoretically be more memory-efficient (less fragmentation), you will still need the loop. No easy way out.
It is somewhat possible in Delphi (but is not very practical in a Delphi environment, unless you are writing a custom memory manager). Creating an object instance is a two-step process - allocating memory for the object, and constructing the object's members inside of that memory. There is nothing requiring the memory of a given object to be allocated individually. You can allocate a larger block of memory and construct multiple objects inside of that block, taking advantage of a feature of Delphi that calls a constructor like a normal method if it is called from an instance variable instead of a class type, eg:
var
objects: array of Byte;
obj: TSomeClass;
begin
SetLength(objects, 100 * TSomeClass.InstanceSize);
FillChar(objects[0], 0, Length(objects));
for i := 0 to 99 do
begin
obj := TSomeClass.InitInstance(#objects[i * TSomeClass.InstanceSize]));
obj.Create;
end;
...
for i := 0 to 99 do
begin
obj := TSomeClass(#objects[i * TSomeClass.InstanceSize]);
obj.CleanupInstance;
end;
SetLength(objects, 0);
end;
This is not much different than what C++ does behind the scenes when declaring an array of object instances, only C++ supports declaring the array statically and it will call the constructors and destructors automatically, whereas Delphi does not support that.
There are a number of third-party implementations that allow you to allocate objects on the stack or in user-defined buffers, such as:
Objects on the Stack: A Delphi Relic
Allocate objects on stack, at specified memory address, or through any memory manager
Just to name a couple.
I'm working with Delphi XE, and writing an application that is using RemObjects SDK to communicate (in case that may be relevant). I have FastMM debug on, and sometimes (not always) when I close it gives a warning about a single "Unexpected Memory Leak". "An unexpected memory leak has occurred. The unexpected small block leaks are: 117-124 bytes: UnicodeString x 1". Very occasionally, I get x2 reported.
Now, my understanding is that Strings are reference counted, and since there is no other object involved to cause the leak, what might be the situation that could cause this to happen? In this StackOverflow question people cannot find a way to make a leak.
If there is no obvious way, then I will download the latest FastMM source (it appears not to be included with the XE source).
[Edit once resolved] The solution to finding this was to install FastMM source, and enable the FullDebugMode to get the stack trace.
When using typed constants and depending on finalization order, it used to be possible for FastMM to report a leak when there really isn't.
FastMM: Leaked memory reported where I believe it shouldn't.
In short, when the FinalizedFirst unit
get's finalized, the SString constant
get's freed. After finalization of the
unit is done, the finalization of
FinalizedLast get's called. In it is
finalization, it call's the method
LeakMemory of the FinalizedFirst
method. The SString variable gets
initialized again and doesn't get
freed as the finalization of
FinalizedFirst has already run.
FinalizedLast Unit
unit FinalizedLast;
interface
uses FinalizedFirst;
implementation
initialization LeakMemory;
finalization LeakMemory;
end.
FinalizedFirst Unit
unit FinalizedFirst;
interface
procedure LeakMemory;
implementation
uses FinalizedLast;
procedure LeakMemory;
const
SString: string = '';
begin
//***** SString will get initialized once or twice depending on the
// finalization order of units. If it get's initialized twice,
// a memory leak is reported.
if SString = '' then
SString := 'FooBar';
end;
end.
Project LeakMemory
program LeakMemory;
uses
FastMM4 in 'FastMM4.pas',
Forms,
FinalizedFirst in 'FinalizedFirst.pas',
FinalizedLast in 'FinalizedLast.pas';
{$R *.RES}
begin
Application.Initialize;
Application.Run;
end.
You can leak strings by freeing records on the heap using FreeMem instead of Dispose or if you overwrite a record using System.Move or FillChar. In the first case the finalization code isn't run and in the second if the string field was filled with a nil it will think it's already cleared it.
If you want to find the location for the leak download FastMM and turn on FullDebugMode. It will include a stack trace of where the leak occurred.
The only way that comes to mind where you can leak a string without deliberately breaking it (like manually incrementing the ref count or doing some messy pointer operation) is by using threadvar.
Like the help file states,
Dynamic variables that are ordinarily
managed by the compiler (long strings,
wide strings, dynamic arrays,
variants, and interfaces) can be
declared with threadvar, but the
compiler does not automatically free
the heap-allocated memory created by
each thread of execution. If you use
these data types in thread variables,
it is your responsibility to dispose
of their memory from within the
thread, before the thread terminates.
Beside that, there's nothing that comes to mind that wasn't already stated.
[Edit by questioner] This was indeed the issue, and the specific code was as follows:
threadvar
g_szAuthentication : String;
procedure TMyBase.SetAuthentication(szUserName, szPassword: String);
begin
g_szAuthentication := '?name=' + szUserName + '&pass=' + szPassword;
end;
I usually see string leaks when they are contained inside other objects that have not been destroyed properly. For example an object that has not been freed. However, you would expect to see that object reported too.
The way to resolve this is to download and use the full version of FastMM and configure it to report stack traces when it detects leaks. When you do that you will get a full stack trace of the code that allocated the leaked object and at that point it is usually clear what the problem is.
I was talking to a co-worker the other day about how you can leak a string in Delphi if you really mess things up. By default strings are reference counted and automatically allocated, so they typically just work without any thought - no need for manual allocation, size calculations, or memory management.
But I remember reading once that there is a way to leak a string directly (without including it in an object that gets leaked). It seems like it had something to do with passing a string by reference and then accessing it from a larger scope from within the routine it was passed to. Yeah, I know that is vague, which is why I am asking the question here.
I don't know about the issue in your second paragraph, but I was bitten once by leaked strings in a record.
If you call FillChar() on a record that contains strings you overwrite the ref count and the address of the dynamically allocated memory with zeroes. Unless the string is empty this will leak the memory. The way around this is to call Finalize() on the record before clearing the memory it occupies.
Unfortunately calling Finalize() when there are no record members that need finalizing causes a compiler hint. It happened to me that I commented out the Finalize() call to silence the hint, but later when I added a string member to the record I missed uncommenting the call, so a leak was introduced. Luckily I'm generally using the FastMM memory manager in the most verbose and paranoid setting in debug mode, so the leak didn't go unnoticed.
The compiler hint is probably not such a good thing, silently omitting the Finalize() call if it's not needed would be much better IMHO.
No, I don't think such a thing can happen. It's possible for a string variable to obtain a value that you didn't expect, but it won't leak memory. Consider this:
var
Global: string;
procedure One(const Arg: string);
begin
Global := '';
// Oops. This is an invalid reference now. Arg points to
// what Global used to refer to, which isn't there anymore.
writeln(Arg);
end;
procedure Two;
begin
Global := 'foo';
UniqueString(Global);
One(Global);
Assert(Global = 'foo', 'Uh-oh. The argument isn''t really const?');
end;
Here One's argument is declared const, so supposedly, it won't change. But then One circumvents that by changing the actual parameter instead of the formal parameter. Procedure Two "knows" that One's argument is const, so it expects the actual parameter to retain its original value. The assertion fails.
The string hasn't leaked, but this code does demonstrate how you can get a dangling reference for a string. Arg is a local alias of Global. Although we've changed Global, Arg's value remains untouched, and because it was declared const, the string's reference count was not incremented upon entry to the function. Reassigning Global dropped the reference count to zero, and the string was destroyed. Declaring Arg as var would have the same problem; passing it by value would fix this problem. (The call to UniqueString is just to ensure the string is reference-counted. Otherwise, it may be a non-reference-counted string literal.) All compiler-managed types are susceptible to this problem; simple types are immune.
The only way to leak a string is to treat it as something other than a string, or to use non-type-aware memory-management functions. Mghie's answer describes how to treat a string as something other than a string by using FillChar to clobber a string variable. Non-type-aware memory functions include GetMem and FreeMem. For example:
type
PRec = ^TRec;
TRec = record
field: string;
end;
var
Rec: PRec;
begin
GetMem(Rec, SizeOf(Rec^));
// Oops. Rec^ is uninitialized. This assignment isn't safe.
Rec^.field := IntToStr(4);
// Even if the assignment were OK, FreeMem would leak the string.
FreeMem(Rec);
end;
There are two ways to fix it. One is to call Initialize and Finalize:
GetMem(Rec, SizeOf(Rec^));
Initialize(Rec^);
Rec^.field := IntToStr(4);
Finalize(Rec^);
FreeMem(Rec);
The other is to use type-aware functions:
New(Rec);
Rec^.field := IntToStr(4);
Dispose(Rec);
Actually, passing string as CONST or non const are the same in term of reference count in Delphi 2007 and 2009. There was a case that causing access violation when string is passed as CONST. Here is the problem one
type
TFoo = class
S: string;
procedure Foo(const S1: string);
end;
procedure TFoo.Foo(const S1: string);
begin
S:= S1; //access violation
end;
var
F: TFoo;
begin
F:= TFoo.create;
try
F.S := 'S';
F.Foo(F.S);
finally
F.Free;
end;
end.
Another way to leak a string is to declare it as a threadvar variable. See my question for details. And for the solution, see the solution on how to tidy it.
I think this might have been similar to what I was thinking of. It is the reverse of a string leak, a string that gets collected early:
var
p : ^String;
procedure InitString;
var
s, x : String;
begin
s := 'A cool string!';
x := s + '. Append something to make a copy in' +
'memory and generate a new string.';
p := #x;
end;
begin
{ Call a function that will generate a string }
InitString();
{ Write the value of the string (pointed to by p) }
WriteLn(p^); // Runtime error 105!
{ Wait for a key press }
ReadLn;
end.