My question is about debugging memory leaks which seem to be a nightmare.
In my app there is a simple class derived from TObject. All objects of that class are stored in a collection/list of of the class derived from TObjectList:
type
TOffer = class(TObject)
Item: string;
Price: string;
Id: string;
end;
TOffers = class(TObjectList<TOffer>)
protected
procedure SetOffer(I: Integer; AOffer: TOffer);
function GetOffer(I: Integer): TOffer;
public
property Offers[I: Integer]: TOffer read GetOffer write SetOffer
end;
The usage scenario:The crawler downloads the offers, parses them and saves to objects collection. This approach seems to be quite convenient as I can refer to the objects later (fill grids/lists, write them to file, etc.)
The problem is the proper disposal of the objects to avoid memory leaks. The app allocates ~4Mb memory on start but after processing ~12k offers it devours 32Mb. The leaks caused by not properly disposed objects/variables after the process finishes.
ReportMemoryLeaksOnShutdown shows horrible digits, but the crucial is -- I have no idea where to look and how to properly debug the damn thing.
Another example is the variable var MyString: string which also needs a proper disposal!! It was sorta insight for me :) I thought each procedure/function automatically manages garbage collection of the out-of-scope variables.
The list of offers is created by a function:
function GetOffersList: TOffers;
begin
Result := TOffers.Create;
while not rs.EOF do
begin
Offer := TOffer.Create;
try
// here come collected offer attributes as variables of type string:
Order.Item := CollectedOfferItem;
Order.Price := CollectedOfferPrice;
Order.Id := CollectedOfferId;
Result.Add(Offer);
finally
Offer := nil;
end;
end;
end;
Then I address those offers directly as a collection. The key thing is that I want this app to run 24/7, so the correct resource disposal is a must.
How to properly dispose object(s) of the above types?
Shall I consider the other techniques to manage object/object lists?
How to properly dispose variables of type string?
Can you please advise the good reading on fighting memory leaks in Delphi?
Thank you.
By default, when you create an object, you become its owner. So long as you are the owner, you are responsible for freeing it. Here are some of the common patterns:
1. Local variable
For an object that is created in a method and only referred to locally, you use the try/finally pattern:
Obj := TMyClass.Create;
try
... use Obj
finally
Obj.Free;
end;
2. Object owned by another object
Commonly created in the constructor and destroyed in the destructor. Here you have a member field of the owning object that holds the reference to the owned object. All you need to do is call Free on all owned objects in the owning class destructor.
3. Owned TComponent
If a TComponent or a derived class is created with an Owner, then that owner destroys the component. You do not need to.
4. TObjectList or similar with OwnsObjects set to True
You show this pattern in your question. You create a TObjectList<T> and by default OwnsObjects is True. This means that when you add a member to the container, the container assumes ownership. From that point on the container assume responsibility for destroying its members and you do not have to. However, somebody still has to destroy the container.
5. Reference counted interfaced objects
Common examples are objects derived from TInterfacedObject. The interface reference counting manages lifetime. You don't need to destroy the object.
6. Function that creates and returns a new instance
This is towards the more tricky end of the spectrum. Thankfully it's a rather rarer pattern. The idea is that the function returns a newly instantiated and initialized object to the caller, who then assumes ownership. But while the function is still executing it is the owner and must defend against exceptions. Typically the code goes like this:
function CreateNewObject(...): TMyClass;
begin
Result := TMyClass.Create;
try
Result.Initialize(...);
except
Result.Free;
raise;
end;
end;
This has to be an exception handler with a call to Free and a re-raise because the code is not in a position to use a finally. The caller will do that:
Obj := CreateNewObject(...);
try
....
finally
Obj.Free;
end;
Looking at the code in the question, that appears to be using both items 4 and 6 from my list. However, do note that your implementation of GetOffersList is not exception safe. But there's no indication that is the problem. It seems plausible that the code that calls GetOffersList is failing to destroy up the container.
Why are you leaking strings? Well, strings are managed objects. They are referenced counted and you need to take no explicit action to destroy them. However, if they are contained in other classes, instances of which are leaked, the contained strings are also leaked. So concentrate on fixing the leaks of objects, and you'll take care of the string leaks.
For what it is worth, TOffer feels more like a value type than a reference type to me. It has no method and contains three simple scalar values. Why not make it a record and use TList<TOffer>?
So, how do you proceed? The FastMM leak report is what you need. You'll want the full FastMM rather than the cut down Embarcadero version. It will identify the allocations that were not matched with deallocations. Deal with them one by one.
In parallel with this, study good quality code. Good open source Delphi libraries will demonstrate all the patterns above, and many more. Learn from them.
String is auto-managed by the compiler, you do not need to free it manually (except in rare corner cases that do not apply to this situation). TObjectList has an OwnsObjects property that you can set to True so the list will free the objects automatically for you. Its constructor has an optional AOwnsObjects parameter to initialize the OwnsObjects property.
Related
I know of these three techniques:
Manually
uses
System.Classes;
procedure DoSomething;
var
sl: TStringList;
begin
sl := TStringList.Create;
try
finally
sl.Free; // Invoking destructor
end;
end;
Reference counting / interface (TInterfacedObject)
Fastest example from the top of my head in the standard library:
uses
Xml.XMLDoc, Xml.XMLIntf;
procedure DoSomething;
var
xmldoc: IXMLDocument;
begin
xmldoc := TXMLDocument.Create(nil) as IXMLDocument;
end; // No reference to xmldoc anymore, freed automatically
Ownership
Like almost the whole VCLibrary or this one:
uses
System.Generics.Collections;
procedure DoSomething;
var
ol: TObjectList<TObject>;
i: Integer;
o: TObject;
begin
ol := TObjectList<TObject>.Create(true); // the list takes ownership of the objects
try
for i := 0 to 9 do begin
o := TObject.Create;
ol.Add(o);
end;
finally
ol.Free; // does not only free the list but all objects in the list, too
end;
end;
Are there more?
When it comes to memory management models for managing object instances in Delphi, there are two: manual memory management and automatic reference counting. All object instances will be released either manually or through reference counting mechanism.
But when it comes to actual coding patterns, there is number of ways we can write the code in order to trigger the release of an object instance and it is almost impossible to list and categorize them all.
The best way to illustrate the complexity involved is by asking additional question:
What do you consider as a memory management technique?
For instance, manually releasing object instance requires invoking the destructor. But there are several commonly used ways to do so: by calling Free, Destroy, or FreeAndNil. But, Free and FreeAndNil eventually will call Destroy. So the question is, should we consider that those different methods of invoking the destructor are the same technique or different techniques? What about other custom written methods that will trigger destruction of an object instance?
When it comes to releasing reference counted object instance, your example has shown indirect way of release - just letting reference go out of scope. But there is an additional way to release such object instance, and that is by explicitly assigning nil to such reference.
procedure DoSomething;
var
xmldoc: IXMLDocument;
begin
xmldoc := TXMLDocument.Create(nil) as IXMLDocument;
...
xmldoc := nil;
...
end;
Again, the question is whether we consider those two different examples as the same or different?
When it comes to ownership, it is just a way to delegate releasing an object instance to some other entity. At the end, in case of manually managed object instances some code at some point will directly invoke destructor on such object. While this is clearly a different coding pattern than directly invoking destructor on object reference without going through additional layers of indirection, at the end the instance will be released manually.
We can also transfer ownership of reference counted object instances. If you have a collection that holds interface references, then this can also be considered as ownership transfer, as release of those instances will depend on the release of the collection itself, even though involved code will not directly call destructor, but will rely on automatic reference counting to do so.
The next question that arises is: What about fields? Your first example shows construction and destruction of local object instance. If we have an object field in a class and manually call Free to such field in its destructor, should we consider that as a manual technique or ownership transfer, because actual release of that inner object instance will depend on the release of its outer, owning object.
There is additional aspect to reference counting. While compiler automatically inserts reference counting code (calls to _AddRef and _Release methods) in appropriate places, the _Release method itself will have to directly call the destructor to actually free the instance. In a way this is just another example of ownership transfer, with some help of the compiler.
From one perspective, we can say that those three techniques you have mentioned are the (two) three basic techniques to release an object instance. But one the other hand, there is an infinite number of them.
I am struggling to learn delphi and memory management, coming from C#.
The current incarnation of that struggle is that I don't know the right way to dispose of the objects when I am done with them. From reading and my experiments it seems that if I have an object that is cast as an interface, then my ONLY choice is set the reference to nil.
If I go an call
FreeAndNil()
I end up getting an access violation, EX:
var
foo: IFoo;
begin
foo := TFoo.Create();
FreeandNil(foo);
end;
Sure, all I need to do it change that foo:IFoo; to foo:TFoo; and it is happy. OR simply set the pointer to nil, NOT call freeandNil.
foo := nil;
So, on one level, I don't understand in the least where the AV is.
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way, but I can't seem to write a method that can deal with something that is a Class or an interface. Well, that is not true, I do have something, but it is so ugly I hesitate to post it.
But I guess I should also be asking, what is everyone else doing? Mentally keeping track of what is an interface and just nil those pointers? otherwise calling FreeAndNil?
I am going to want to implement things the first time around as a concrete class, but later come back and change that to an interface when I find some way that the code can do from 2 different ways. And I am not going to want to go through the code and change how it was dealing with that reference, that is the last thing on my mind at that point.
But for the sake of discussion, the best (almost only) idea I have is this class:
interface
type
TMemory = class(TObject)
class procedure Free(item: TObject); overload; static;
class procedure Free<T: IInterface>(item: T); overload; static;
end;
implementation
uses
System.SysUtils;
{ TMemory }
class procedure TMemory.Free(item: TObject);
begin
FreeandNil(item);
end;
class procedure TMemory.Free<T>(item: T);
begin
//don't do anything, it is up the caller to always nil after calling.
end;
Then I can consistently call:
TMemory.Free(Thing);
Thing := nil;
Test code:
procedure TDoSomething.MyWorker;
var
foo: IFoo;
fooAsClass: TFoo;
JustAnObject: TObject;
begin
foo := TFoo.Create();
fooAsClass := TFoo.Create();
JustAnObject := TObject.Create();
TMemory.Free(foo);
foo := nil;
TMemory.Free(fooAsClass);
fooAsClass := nil;
TMemory.Free(JustAnObject);
JustAnObject := nil;
end;
runs with no leaks or access violations. (using MadExcept)
But a big thank you to the Delphi community on SO. You guys have been the best thing out there for learning!
The reason for the access violation is that FreeAndNil takes an untyped parameter, but expects it to be an object. So the method operates on the object.
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj); //Obj must be a TObject otherwise all bets are off
Pointer(Obj) := nil; //Will throw an AV if memory violation is detected
Temp.Free; //Will throw an AV if memory violation is detected
end;
A memory violation in the above might (NB not guaranteed) be detected if you destroy an object that has either been previously destroyed or never created. It's also likely to be detected if Obj doesn't reference an object at all but something else (such as an interface, record, Integer because these don't implement Free and if they did, it wouldn't be located in the same way as TObject.Free).
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way.
This is like saying you want to use your car in exactly the same way that you use your shower.
Ok, maybe the difference is not quite that extreme. But the point is that interfaces and objects (and for that matter records) use different memory management paradigms. You cannot manage their memory in the same way.
Objects need to be explicitly destroyed. You can use an ownership model, but destruction is still an explicit external action.
Interfaces are reference counted. The compiler injects code to track the number of fields and variables referencing (looking at) the underlying instance. Typically the object destroys itself when the last reference is released. (There are ways beyond the scope of this answer to change this.)
If we access some object by interface variable, it doesn't always mean that object is destroyed the moment reference counter drops to zero. For example, TComponent methods _AddRef and _Release implementations are 'dummy': no reference counting is implemented and TComponent is never destroyed because interface variables are out of scope.
To behave as we expect from 'real' interfaces, all your objects should be descendants from TInterfacedObject or you need to implement _AddRef / _Release yourself.
Yes, there are 2 different approaches to memory management which usually co-exist in a program, but confusion (and AV) arises only when the same object is treated in both ways. If we destroyed object and only then the interface variables have gone out of scope, they call _Release method of destroyed object which causes access violation. That's some risky business, though with some attention it is doable.
Classic Delphi components are not reference-counted, the concept of ownership is used instead. Each component has an owner whose responsibility is to free all the memory when it itself is destroyed. So each component has an owner, but it may also have a lot of pointers to another components, like when Toolbar has ImageList variable. If such components were refcounted, they would never be destroyed because of circular reference, so in order to break this circle you'd need 'weak' references as well which don't 'count'. They are here, too, but that's very recent feature of Delphi.
If there is some hierarchy in your objects, so you know that 'bigger' objects need all of 'smaller' ones to function, then use this good old approach, it's pretty simple and has very good implementation in Delphi, which is: you can make a code which will be leak-free no matter where exception could arise. There are all these little things like using .Free instead of .Destroy, because if exception happened in constructor, destructor is called automatically, and so on. Very clever solution in fact.
I'd use refcounted interfaces only if you don't know for how long some object is needed for you and there is no suitable 'owner' for it. I did it with scanned image which I saved to file in one thread, while converting to smaller image to show on screen on another thread. When all is done, image is no more needed in RAM and can be destroyed, but I have no idea which happens first. In this case using refcounting is best thing to do.
I am new to Delphi and am trying to better understand object creation / freeing as I am used to the luxury of .NET's GC. I have two questions specifically:
Let's assume I am setting a TDataSource as below. In .NET I wouldn't explicitly destroy the object as I am with adoQuery.Free. But I am assuming that with Delphi I need to free these objects. However, by destroying the adoQuery I am also setting the dataset to null. In this way adoQuery is meant to be a locally scoped variable to the function only with the datsource being retuned from the function. Therefore, how can I best handle this?
dataSrc := TDataSource.Create(nil);
dataSrc.DataSet := adoQuery;
dataSrc.Enabled := true;
{ adoQuery.Free; }
cnt := DataSrc.DataSet.RecordCount;
I've been reading several suggestions when returning a variable from a function that the best thing to do is create the variable within the caller and pass it to the subroutine. Therefore, the signature to a function would look like:
AdoConnectionManager.GetResult(query : String; dataSrc: TDataSource) : TDataSource;
Result := dataSrc;
This is unattractive to me. I'd prefer to have a new variable created within the subroutine and then returned back to the caller. However, this is something again I never really had to worry about with .NET GC and here I have to explicitly destroy the variable, right?
Thanks!
You've asked two questions. One concerns these database classes, and I'm going to ignore that question since I don't know anything about those classes. Instead I will answer the other questions. Do note that this sort of answer is why the site policy is for questions to be asked one at a time.
Regarding a function that returns a new object, that is certainly viable. However, it is sometimes more flexible to let the caller supply the object. That allows them to re-use instance, or supply objects that are derived from a base class. A classic example would be a function that populated a TStrings instance.
In this scenario you'd probably use a procedure rather than a function. It might look like this:
procedure PopulateList(List: TMyList);
If you want to have a function that returns a newly minted instance that would be done like so:
function CreateAndPopulateList: TMyList;
begin
Result := TMyList.Create;
try
// code to populate Result goes here, and may raise exceptions
except
Result.Free; // in case of exceptions, we must destroy the instance to avoid leaks
raise;
end;
end;
Note the naming. I use create to imply to the caller that a new instance is created. The calling code would look like this:
List := CreateAndPopulateList;
try
// do stuff with list
finally
List.Free;
end;
And this pattern is the standard object creation pattern. So you use CreateAndPopulateList just as you could a constructor.
It should also be mentioned here, that Delphi also provides Reference-Counting (but different to .NET).
The very short explanation to Reference-Counting in Delphi:
In difference to other Languagues, Reference-Counting in Delphi is only available by using Interfaces. Further, there is no Garbage-Collector: a reference-counted Object gets instantly destroyed when its Referencecount reaches 0.
So as an Delphi Developer, there are the following "global" Rules for destroying Instances:
- you do destroy an Object manually, whenever it's declared as a ClassType (e.g. var m: TMyClass)
- you never destroy an Object manually, whenever it's declared as a InterfaceType (e.g. var m: IMyClass)
With Delphi when you create an object, you should decide how it would be freed. There are several ways:
You can free it manually
It can be freed together with it's owner
It can be freed as a part of TObjectList or similar container
It can be freed because it's interfaced and reference counter for it became zero
And so on...
About first question: you should understand that with Delphi object variable is a pointer to object. When leaving function, you can lost locally scoped (pointer) variable, but you don't harm object itself. For example, you can do something like this:
function GetDataSource: TDataSource;
var Query: TADOQuery;
begin
Result := TDataSource.Create(nil);
Query := TADOQuery.Create(Result);
Query.SQL.Text := ' ... ';
Result.DataSet := Query;
end;
It'll give you datasource you want with background query. When you free this datasource, query also would be freed.
About second question: intrafunction creation of return object is a normal practise, part of good design. Yes, you should decide who will free this object and how. You can use many strategies, there are no silver bullet here. Just for example, you can decide to add parameter 'datasource's owner' to function above and controls it's lifetime this way.
Delphi 2010 - I have a Treeview with approx 2000 nodes. Each node has a Data Object, which points to a record. When I exit my program, does the memory associated with the Data objects get automatically freed, or do I need to traverse each node a DISPOSE of them?
The Data property of the TTreeNode class is intended for you to use for your private needs. It is an untyped pointer and that gives you the freedom to put whatever you like in there.
As a corollary to the property being an untyped pointer, the control cannot know what type of thing you put in there. And hence it cannot destroy it, even if it wanted to.
So yes, you do need to make sure that the memory you allocated is destroyed. Essentially you are the owner of that memory, the control does not own it.
How you go about doing that is of course entirely up to you. One option would be to have a separate container that owns the data associated with each node. Then you can delete the container and let it delete the data.
That might be inconvenient if you have a complex tree structure that is hard to replicate in the containers available to you. In that case you might choose to customise the tree view to create nodes of a type that you define. That custom node type could take charge of allocating and deallocating the data. The OnCreateNodeClass is the key to making this happen.
Having found this question via a Google search and not finding the solution I have just used, I have decided to put an answer even though the question is a couple of years old.
TTreeView has an OnTreeDeletion method which is called as each node in the tree is deleted giving the opportunity to free/dispose of any associated data.
Here is my code:
procedure TfmMyForm.MyTreeDeletion(Sender: TObject; Node: TTreeNode);
var
rci: TRecentCodeItem;
begin
rci := node.data;
if assigned(rci) then
freeandnil(rci);
end;
I store a pointer to an object of type TRecentCodeItem with the node.
One difference between my situation and the questioner's is that my tree view was not on the main form so I couldn't just leave it to windows to mop up the memory after the application closes. Not that I would do that anyway - I do like to have zero memory leaks.
Yes, the data gets freed once you exit your program since the objects reside in the same memory space of the process your application runs in. It is good practice to free them manually anyway using Object.free or better yet FreeAndNil(object).
I use this soulution, maybe it helps someone in future:
TTreeViewItemEx<TData: class> = class(TTreeViewItem)
private
_dataObject: TData;
public
destructor Destroy; override;
property dataObject: TData read _dataObject write _dataObject;
end;
destructor TTreeViewItemEx<TData>.Destroy;
begin
if Assigned(_dataObject) then
_dataObject.Free;
inherited;
end;
usage:
procedure UsageExample;
var
node: TTreeViewItemEx<TData>;
begin
node := TTreeViewItemEx<TData>.Create(treeView);
node.DataObject := TData.Create;
node.Parent := treeView;
end;
This is similar to this question. I asked "Why?" to the most popular response but I don't know that anyone would ever look at it again. At least not in any timely manner.
Anyway, my question is about best practices for delegating responsibility for creation of objects to functions or procedures, without causing memory leaks. It seems that this:
procedure FillObject(MyObject: TMyObject; SomeParam: Integer);
begin
//Database operations to fill object
end;
procedure CallUsingProcedure();
var
MyObject: TMyObject;
begin
MyObject = TMyObject.Create();
try
FillObject(MyObject, 1);
//use object
finally
MyObject.Free();
end;
end;
is preferred over this:
function CreateMyObject(DBID: Integer): TMyObject;
begin
Result := TMyObject.Create();
try
//Database operations to fill object
except on E: Exception do
begin
Result.Free();
raise;
end;
end;
end;
procedure CallUsingFunction();
var
MyObject: TMyObject;
begin
MyObject = CreateMyObject(1);
try
//use object
finally
MyObject.Free();
end;
end;
Why?
I'm relatively new to Delphi, having previously worked most with Java and PHP, as well as C++, though to a lesser extent. Intuitively, I lean toward the function method because:
It encapsulates the object creation code in the function, rather than create the object separately whenever I want to use the procedure.
I dislike methods that alter their parameters. It's often left undocumented and can make tracing bugs more difficult.
Vague, but admittedly it just "smells" bad to me.
I'm not saying I'm right. I just want to understand why the community chooses this method and if there is good reason for me to change.
Edit:
References to #E-Rock in comments are to me(Eric G). I changed my display name.
One problem is what Ken White wrote: you hand the user of the function an object he or she must free.
Another advantage of procedures is that you can pass several objects of a hierarchy, while a function that creates such an object always generates the same. E.g.
procedure PopulateStrings(Strings: TStrings);
To that procedure, you can pass any kind of TStrings, be it the Lines of a TMemo, the Items of a TListBox or TComboBox or a simple standalone TStringList. If you have a function:
function CreateStrings: TStrings;
You always get the same kind of object back (which object exactly is not known, as TStrings is abstract, so you probably get a TStringList), and must Assign() the contents to the TStrings you want to modify. The procedure is to be preferred, IMO.
Additionally, if you are the author of the function, you can't control whether the object you create is freed, or when. If you write a procedure, that problem is taken off your hands, since the user provides the object, and its lifetime is none of your concern. And you don't have to know the exact type of the object, it must just be of the class or a descendant of the parameter. IOW, it is also much better for the author of the function.
It is IMO seldom a good idea to return an object from a function, for all the reasons given. A procedure that only modifies the object has no dependency on the object and creates no dependency for the user.
FWIW, Another problem is if you do that from a DLL. The object returned uses the memory manager of the DLL, and also the VMT to which it points is in the DLL. That means that code that uses as or is in the user code does not work properly (since is and as use the VMT pointer to check for class identity). If the user must pass an object of his, to a procedure, that problem does not arise.
Update
As others commented, passing an object to a DLL is not a good idea either. Non-virtual functions will call the functions inside the DLL and use its memory manager, which can cause troubles too. And is and as will not work properly inside the DLL either. So simply don't pass objects into or out of a DLL. That goes with the maxime that DLLs should only use POD type parameters (or compound types -- arrays, records -- that only contain POD types) or COM interfaces. The COM interfaces should also only use the same kind of parameters.
Creating the object instance and passing it into another procedure makes it clear which code is responsible for freeing the instance.
In the first case (using a procedure to fill it):
MyObj := TMyObject.Create;
try
// Do whatever with MyObj
finally
MyObj.Free;
end;
This is clear that this block of code is responsible for freeing MyObj when it's finished being used.
MyObj := CreateMyObject(DBID);
What code is supposed to free it? When can you safely free it? Who is responsible for exception handling? How do you know (as a user of someone else's code)?
As a general rule, you should create, use, and free object instances where they're needed. This makes your code easier to maintain, and definitely makes it easier for someone who comes along later and has to try and figure it out. :)
I use a combination of both idioms. Pass the object as an optional parameter and if not passed, create the object. And in either case return the object as the function result.
This technique has (1) the flexibility of the creation of the object inside of the called function, and (2) the caller control of the caller passing the object as a parameter. Control in two meanings: control in the real type of the object being used, and control about the moment when to free the object.
This simple piece of code exemplifies this idiom.
function MakeList(aList:TStrings = nil):TStrings;
var s:TStrings;
begin
s:=aList;
if s=nil then
s:=TSTringList.Create;
s.Add('Adam');
s.Add('Eva');
result:=s;
end;
And here are three different ways to use it
simplest usage, for quick and dirty code
var sl1,sl2,sl3:TStrings;
sl1:=MakeList;
when programmer wants to make more explicit ownership and/or use a custom type
sl2:=MakeList(TMyStringsList.create);
when the object is previously created
sl3:=TMyStringList.Create;
....
MakeList(sl3);