I'm trying to create a generic method to get all the references to an object from an object.
For example:
TTest2 = class(TObject);
TTest = class(TObject)
Test2: TTest2;
end;
I want to create a method like:
var
Local: TTest;
LinkedObjects: TList;
begin
Local := TTest.Create;
LinkedObjects := Local.GetChildren;
//blah
end;
I'd like to create a method that says to me that on offset X, there is a reference for an object. The objective is to be able to list any object in any kind of field, so, published field's (that are listed on object header - vmtFieldTable) won't solve, Rtti (As it's not default for every classes) won't solve too.
It's probably not possible without some help from compiler (providing some information), but if you have some idea, please let me know.
I'm researching the possibility to develop a GC for Delphi. Everything on a GC is very mature, the technology is not a problem. But how to have access for some information is what make things complicated. At this point, I'm thinking a way to deal with the Mark step.
Some thoughts
Overload the assign operator of TObject ?
It's not possible just on NextGen compilers. Full answer.
Go through all the allocated memory and search for valid pointers on its space ?
Slow, but is it possible ? initialize and finalize all the object's memory clear, and then go through the memory looking for pointer with a valid object header ? or can I create a parity bit on objects to make it easier to identify?
Update: I found an interesting link! Talking about the same problem we discussed here.
I'll try do it.
I'm putting some information together here.
Thanks you,
Related
tl;dr:
Trying to pass a list of objects to a function, which expects list of objects implementing an interface. Said objects implement that interface. Compiler will not allow it.
Looking for an alternative workaround or my mistake.
The setup
I did not use the actual code, and that should not matter IMO. Seems like a conceptional problem to me.
The class TObjectWithInterface implements the interface ISomeInterface and extends the class TCustomInterfacedObject, which simply works around the reference counting of IInterface as given in the documentation.
TObjectWithInterface = class(TCustomInterfacedObject, ISomeInterface)
If have a procedure which takes a list of objects implementing that interface:
procedure SomeFunction(List: TList<ISomeInterface>)
The issue
Inside of a function of TObjectWithInterface, I try to call that function, using a list of objects of TObjectWithInterface:
procedure TObjectWithInterface.DoTheStuff(ListOfObjects: TList<TObjectWithInterface>)
begin
// ...
SomeFunction(ListOfObjects); // <-- Compiler error: Incompatible types
// ...
end;
The compiler tells me the following:
E2010: Incompatible types: 'System.Generics.Collections.TList' and 'System.Generics.Collections.TList'
The dumb workaround
I really dislike my current workaround, which consists of creating a new list and type casting each TObjectWithInterface to ISomeInterface:
procedure TObjectWithInterface.DoTheStuff(ListOfObjects: TList<TObjectWithInterface>)
var
ListOfInterfaceObjects: TList<ISomeInterface>;
begin
// ...
ListOfInterfaceObjects := TList<ISomeInterface>.Create;
for var Object in ListOfObjects do
ListOfInterfaceObjects.Add(Objects as ISomeInterface);
SomeFunction(ListOfInterfaceObjects)
// ...
end;
This seems very hacky to me. I may have done something stupid, or do not understand something correctly, as this is the first time, that I am trying to use Interfaces in Delphi. Please don't be mad.
Either way, I hope someone can point out my mistake or, if this is a language limitation, has an alternative workaround.
Your "workaround" to copy the list of objects into a separate list of interfaces is actually the correct and proper solution. You can't use a TList<X> where a TList<Y> is expected, they are different and unrelated types. Just because you are disabling reference counting on the interfaces doesn't change the memory layout of the objects in relation to their interfaces, so you still have to pass around proper memory pointers, which means performing necessary conversions from one to the other.
I am struggling to learn delphi and memory management, coming from C#.
The current incarnation of that struggle is that I don't know the right way to dispose of the objects when I am done with them. From reading and my experiments it seems that if I have an object that is cast as an interface, then my ONLY choice is set the reference to nil.
If I go an call
FreeAndNil()
I end up getting an access violation, EX:
var
foo: IFoo;
begin
foo := TFoo.Create();
FreeandNil(foo);
end;
Sure, all I need to do it change that foo:IFoo; to foo:TFoo; and it is happy. OR simply set the pointer to nil, NOT call freeandNil.
foo := nil;
So, on one level, I don't understand in the least where the AV is.
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way, but I can't seem to write a method that can deal with something that is a Class or an interface. Well, that is not true, I do have something, but it is so ugly I hesitate to post it.
But I guess I should also be asking, what is everyone else doing? Mentally keeping track of what is an interface and just nil those pointers? otherwise calling FreeAndNil?
I am going to want to implement things the first time around as a concrete class, but later come back and change that to an interface when I find some way that the code can do from 2 different ways. And I am not going to want to go through the code and change how it was dealing with that reference, that is the last thing on my mind at that point.
But for the sake of discussion, the best (almost only) idea I have is this class:
interface
type
TMemory = class(TObject)
class procedure Free(item: TObject); overload; static;
class procedure Free<T: IInterface>(item: T); overload; static;
end;
implementation
uses
System.SysUtils;
{ TMemory }
class procedure TMemory.Free(item: TObject);
begin
FreeandNil(item);
end;
class procedure TMemory.Free<T>(item: T);
begin
//don't do anything, it is up the caller to always nil after calling.
end;
Then I can consistently call:
TMemory.Free(Thing);
Thing := nil;
Test code:
procedure TDoSomething.MyWorker;
var
foo: IFoo;
fooAsClass: TFoo;
JustAnObject: TObject;
begin
foo := TFoo.Create();
fooAsClass := TFoo.Create();
JustAnObject := TObject.Create();
TMemory.Free(foo);
foo := nil;
TMemory.Free(fooAsClass);
fooAsClass := nil;
TMemory.Free(JustAnObject);
JustAnObject := nil;
end;
runs with no leaks or access violations. (using MadExcept)
But a big thank you to the Delphi community on SO. You guys have been the best thing out there for learning!
The reason for the access violation is that FreeAndNil takes an untyped parameter, but expects it to be an object. So the method operates on the object.
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj); //Obj must be a TObject otherwise all bets are off
Pointer(Obj) := nil; //Will throw an AV if memory violation is detected
Temp.Free; //Will throw an AV if memory violation is detected
end;
A memory violation in the above might (NB not guaranteed) be detected if you destroy an object that has either been previously destroyed or never created. It's also likely to be detected if Obj doesn't reference an object at all but something else (such as an interface, record, Integer because these don't implement Free and if they did, it wouldn't be located in the same way as TObject.Free).
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way.
This is like saying you want to use your car in exactly the same way that you use your shower.
Ok, maybe the difference is not quite that extreme. But the point is that interfaces and objects (and for that matter records) use different memory management paradigms. You cannot manage their memory in the same way.
Objects need to be explicitly destroyed. You can use an ownership model, but destruction is still an explicit external action.
Interfaces are reference counted. The compiler injects code to track the number of fields and variables referencing (looking at) the underlying instance. Typically the object destroys itself when the last reference is released. (There are ways beyond the scope of this answer to change this.)
If we access some object by interface variable, it doesn't always mean that object is destroyed the moment reference counter drops to zero. For example, TComponent methods _AddRef and _Release implementations are 'dummy': no reference counting is implemented and TComponent is never destroyed because interface variables are out of scope.
To behave as we expect from 'real' interfaces, all your objects should be descendants from TInterfacedObject or you need to implement _AddRef / _Release yourself.
Yes, there are 2 different approaches to memory management which usually co-exist in a program, but confusion (and AV) arises only when the same object is treated in both ways. If we destroyed object and only then the interface variables have gone out of scope, they call _Release method of destroyed object which causes access violation. That's some risky business, though with some attention it is doable.
Classic Delphi components are not reference-counted, the concept of ownership is used instead. Each component has an owner whose responsibility is to free all the memory when it itself is destroyed. So each component has an owner, but it may also have a lot of pointers to another components, like when Toolbar has ImageList variable. If such components were refcounted, they would never be destroyed because of circular reference, so in order to break this circle you'd need 'weak' references as well which don't 'count'. They are here, too, but that's very recent feature of Delphi.
If there is some hierarchy in your objects, so you know that 'bigger' objects need all of 'smaller' ones to function, then use this good old approach, it's pretty simple and has very good implementation in Delphi, which is: you can make a code which will be leak-free no matter where exception could arise. There are all these little things like using .Free instead of .Destroy, because if exception happened in constructor, destructor is called automatically, and so on. Very clever solution in fact.
I'd use refcounted interfaces only if you don't know for how long some object is needed for you and there is no suitable 'owner' for it. I did it with scanned image which I saved to file in one thread, while converting to smaller image to show on screen on another thread. When all is done, image is no more needed in RAM and can be destroyed, but I have no idea which happens first. In this case using refcounting is best thing to do.
I need to be able to pass the same set of structures (basically arrays of different records) over two different interfaces
The first (legacy) which is working requires a pointer to a record and the record size
The second, which I am attempting to develop, is type-safe and requires individual fields to be set using Get/Set methods for each field
Existing code uses records (probably around 100 or so) with memory management being handled in a 3rd party DLL (i.e. we pass the record pointer and size to it and it deals with memory management of new records).
My original thought was to bring the memory management into my app and then copy over the data on the API call. This would be easy enough with the old interface, as I just need to be able to access SizeOf() and the pointer to the record structure held in my internal TList. The problem comes when writing the adapter for the new type-safe interface
As these records are reliant on having a known size, there is heavy use of array 0..n of char static arrays, however as soon as I try to access these via 2010-flavour RTTI I get error messages stating 'Insufficient RTTI information available to support this operation'. Standard Delphi strings work, but old short-strings don't. Unfortunately, fixing string lengths is important for the old-style interface to work properly. I've had a look at 3rd party solutions such as SuperObject and the streaming in MorMot, though they can't do anything out of the box which doesn't give me too much hope of a solution not needing significant re-work.
What I want to be able to do is something like the following (don't have access to my Delphi VM at the moment, so not perfect code, but hopefully you get the gist):
type
RTestRec = record
a : array [0..5] of char;
b : integer;
end;
// hopefully this would be handled by generic <T = record> or passing instance as a pointer
procedure PassToAPI(TypeInfo: (old or new RTTI info); instance: TestRec)
var
Field: RTTIField;
begin
for Field in TypeInfo.Fields do
begin
case Field.FieldType of
ftArray: APICallArray(Field.FieldName, Field.Value);
ftInteger: APICallInteger(Field.FieldName, Field.Value.AsInteger);
...
end;
end;
Called as:
var
MyTestRec: RTestRec;
begin
MyTestRec.a := 'TEST';
MyTestRec.b := 5;
PassToAPI(TypeInfo(TestRec), MyTestRec);
end;
Can the lack of RTTI be forced by a Compiler flag or similar (wishful thinking I feel!)
Can a mixture of old-style and new-style RTTI help?
Can I declare the arrays differently to give RTTI but still having the size constraints needed for old-style streaming?
Would moving from Records to Classes help? (I think I'd need to write my own streaming to an ArrayOfByte to handle the old interface)
Could a hacky solution using Attributes help? Maybe storing some of the missing RTTI information there? Feels like a bit of a long-term maintenance issue, though.
My question is about debugging memory leaks which seem to be a nightmare.
In my app there is a simple class derived from TObject. All objects of that class are stored in a collection/list of of the class derived from TObjectList:
type
TOffer = class(TObject)
Item: string;
Price: string;
Id: string;
end;
TOffers = class(TObjectList<TOffer>)
protected
procedure SetOffer(I: Integer; AOffer: TOffer);
function GetOffer(I: Integer): TOffer;
public
property Offers[I: Integer]: TOffer read GetOffer write SetOffer
end;
The usage scenario:The crawler downloads the offers, parses them and saves to objects collection. This approach seems to be quite convenient as I can refer to the objects later (fill grids/lists, write them to file, etc.)
The problem is the proper disposal of the objects to avoid memory leaks. The app allocates ~4Mb memory on start but after processing ~12k offers it devours 32Mb. The leaks caused by not properly disposed objects/variables after the process finishes.
ReportMemoryLeaksOnShutdown shows horrible digits, but the crucial is -- I have no idea where to look and how to properly debug the damn thing.
Another example is the variable var MyString: string which also needs a proper disposal!! It was sorta insight for me :) I thought each procedure/function automatically manages garbage collection of the out-of-scope variables.
The list of offers is created by a function:
function GetOffersList: TOffers;
begin
Result := TOffers.Create;
while not rs.EOF do
begin
Offer := TOffer.Create;
try
// here come collected offer attributes as variables of type string:
Order.Item := CollectedOfferItem;
Order.Price := CollectedOfferPrice;
Order.Id := CollectedOfferId;
Result.Add(Offer);
finally
Offer := nil;
end;
end;
end;
Then I address those offers directly as a collection. The key thing is that I want this app to run 24/7, so the correct resource disposal is a must.
How to properly dispose object(s) of the above types?
Shall I consider the other techniques to manage object/object lists?
How to properly dispose variables of type string?
Can you please advise the good reading on fighting memory leaks in Delphi?
Thank you.
By default, when you create an object, you become its owner. So long as you are the owner, you are responsible for freeing it. Here are some of the common patterns:
1. Local variable
For an object that is created in a method and only referred to locally, you use the try/finally pattern:
Obj := TMyClass.Create;
try
... use Obj
finally
Obj.Free;
end;
2. Object owned by another object
Commonly created in the constructor and destroyed in the destructor. Here you have a member field of the owning object that holds the reference to the owned object. All you need to do is call Free on all owned objects in the owning class destructor.
3. Owned TComponent
If a TComponent or a derived class is created with an Owner, then that owner destroys the component. You do not need to.
4. TObjectList or similar with OwnsObjects set to True
You show this pattern in your question. You create a TObjectList<T> and by default OwnsObjects is True. This means that when you add a member to the container, the container assumes ownership. From that point on the container assume responsibility for destroying its members and you do not have to. However, somebody still has to destroy the container.
5. Reference counted interfaced objects
Common examples are objects derived from TInterfacedObject. The interface reference counting manages lifetime. You don't need to destroy the object.
6. Function that creates and returns a new instance
This is towards the more tricky end of the spectrum. Thankfully it's a rather rarer pattern. The idea is that the function returns a newly instantiated and initialized object to the caller, who then assumes ownership. But while the function is still executing it is the owner and must defend against exceptions. Typically the code goes like this:
function CreateNewObject(...): TMyClass;
begin
Result := TMyClass.Create;
try
Result.Initialize(...);
except
Result.Free;
raise;
end;
end;
This has to be an exception handler with a call to Free and a re-raise because the code is not in a position to use a finally. The caller will do that:
Obj := CreateNewObject(...);
try
....
finally
Obj.Free;
end;
Looking at the code in the question, that appears to be using both items 4 and 6 from my list. However, do note that your implementation of GetOffersList is not exception safe. But there's no indication that is the problem. It seems plausible that the code that calls GetOffersList is failing to destroy up the container.
Why are you leaking strings? Well, strings are managed objects. They are referenced counted and you need to take no explicit action to destroy them. However, if they are contained in other classes, instances of which are leaked, the contained strings are also leaked. So concentrate on fixing the leaks of objects, and you'll take care of the string leaks.
For what it is worth, TOffer feels more like a value type than a reference type to me. It has no method and contains three simple scalar values. Why not make it a record and use TList<TOffer>?
So, how do you proceed? The FastMM leak report is what you need. You'll want the full FastMM rather than the cut down Embarcadero version. It will identify the allocations that were not matched with deallocations. Deal with them one by one.
In parallel with this, study good quality code. Good open source Delphi libraries will demonstrate all the patterns above, and many more. Learn from them.
String is auto-managed by the compiler, you do not need to free it manually (except in rare corner cases that do not apply to this situation). TObjectList has an OwnsObjects property that you can set to True so the list will free the objects automatically for you. Its constructor has an optional AOwnsObjects parameter to initialize the OwnsObjects property.
This is similar to this question. I asked "Why?" to the most popular response but I don't know that anyone would ever look at it again. At least not in any timely manner.
Anyway, my question is about best practices for delegating responsibility for creation of objects to functions or procedures, without causing memory leaks. It seems that this:
procedure FillObject(MyObject: TMyObject; SomeParam: Integer);
begin
//Database operations to fill object
end;
procedure CallUsingProcedure();
var
MyObject: TMyObject;
begin
MyObject = TMyObject.Create();
try
FillObject(MyObject, 1);
//use object
finally
MyObject.Free();
end;
end;
is preferred over this:
function CreateMyObject(DBID: Integer): TMyObject;
begin
Result := TMyObject.Create();
try
//Database operations to fill object
except on E: Exception do
begin
Result.Free();
raise;
end;
end;
end;
procedure CallUsingFunction();
var
MyObject: TMyObject;
begin
MyObject = CreateMyObject(1);
try
//use object
finally
MyObject.Free();
end;
end;
Why?
I'm relatively new to Delphi, having previously worked most with Java and PHP, as well as C++, though to a lesser extent. Intuitively, I lean toward the function method because:
It encapsulates the object creation code in the function, rather than create the object separately whenever I want to use the procedure.
I dislike methods that alter their parameters. It's often left undocumented and can make tracing bugs more difficult.
Vague, but admittedly it just "smells" bad to me.
I'm not saying I'm right. I just want to understand why the community chooses this method and if there is good reason for me to change.
Edit:
References to #E-Rock in comments are to me(Eric G). I changed my display name.
One problem is what Ken White wrote: you hand the user of the function an object he or she must free.
Another advantage of procedures is that you can pass several objects of a hierarchy, while a function that creates such an object always generates the same. E.g.
procedure PopulateStrings(Strings: TStrings);
To that procedure, you can pass any kind of TStrings, be it the Lines of a TMemo, the Items of a TListBox or TComboBox or a simple standalone TStringList. If you have a function:
function CreateStrings: TStrings;
You always get the same kind of object back (which object exactly is not known, as TStrings is abstract, so you probably get a TStringList), and must Assign() the contents to the TStrings you want to modify. The procedure is to be preferred, IMO.
Additionally, if you are the author of the function, you can't control whether the object you create is freed, or when. If you write a procedure, that problem is taken off your hands, since the user provides the object, and its lifetime is none of your concern. And you don't have to know the exact type of the object, it must just be of the class or a descendant of the parameter. IOW, it is also much better for the author of the function.
It is IMO seldom a good idea to return an object from a function, for all the reasons given. A procedure that only modifies the object has no dependency on the object and creates no dependency for the user.
FWIW, Another problem is if you do that from a DLL. The object returned uses the memory manager of the DLL, and also the VMT to which it points is in the DLL. That means that code that uses as or is in the user code does not work properly (since is and as use the VMT pointer to check for class identity). If the user must pass an object of his, to a procedure, that problem does not arise.
Update
As others commented, passing an object to a DLL is not a good idea either. Non-virtual functions will call the functions inside the DLL and use its memory manager, which can cause troubles too. And is and as will not work properly inside the DLL either. So simply don't pass objects into or out of a DLL. That goes with the maxime that DLLs should only use POD type parameters (or compound types -- arrays, records -- that only contain POD types) or COM interfaces. The COM interfaces should also only use the same kind of parameters.
Creating the object instance and passing it into another procedure makes it clear which code is responsible for freeing the instance.
In the first case (using a procedure to fill it):
MyObj := TMyObject.Create;
try
// Do whatever with MyObj
finally
MyObj.Free;
end;
This is clear that this block of code is responsible for freeing MyObj when it's finished being used.
MyObj := CreateMyObject(DBID);
What code is supposed to free it? When can you safely free it? Who is responsible for exception handling? How do you know (as a user of someone else's code)?
As a general rule, you should create, use, and free object instances where they're needed. This makes your code easier to maintain, and definitely makes it easier for someone who comes along later and has to try and figure it out. :)
I use a combination of both idioms. Pass the object as an optional parameter and if not passed, create the object. And in either case return the object as the function result.
This technique has (1) the flexibility of the creation of the object inside of the called function, and (2) the caller control of the caller passing the object as a parameter. Control in two meanings: control in the real type of the object being used, and control about the moment when to free the object.
This simple piece of code exemplifies this idiom.
function MakeList(aList:TStrings = nil):TStrings;
var s:TStrings;
begin
s:=aList;
if s=nil then
s:=TSTringList.Create;
s.Add('Adam');
s.Add('Eva');
result:=s;
end;
And here are three different ways to use it
simplest usage, for quick and dirty code
var sl1,sl2,sl3:TStrings;
sl1:=MakeList;
when programmer wants to make more explicit ownership and/or use a custom type
sl2:=MakeList(TMyStringsList.create);
when the object is previously created
sl3:=TMyStringList.Create;
....
MakeList(sl3);