I have a graph like structure. I don't know exactly when to destroy the objects in traditional Delphi manner, instead I would like to implement something like reference counted objects. I know that I can use something like object.GetReference and object.Release instead of Free, and use a private variable for reference counting, but is there any better way?
Thanks
If you have problems determining the correct way (place, order and so on) of destroying standard objects in a Delphi program, then using reference counted objects or interfaces instead will not help you at all.
I understand that you want the nodes in a graph to keep references to each other, and when there are no references left to an object, it should be destroyed automatically. But consider the fact that two nodes can each have a reference to the other node, and the ref count will never reach 0 again, so that those objects will never be freed. You will need to break at least one of the references / dependencies manually. And if you have to do this anyway, then you can as well skip reference counting altogether. For some more information see the Wikipedia article about weak references. Interfaces can be used in Delphi for reference counting, but weak references can only be maintained with clever typecasting. For an example see this source code and its comments.
One idea that you might want to explore is to keep the standard lifetime management for your objects, and let the graph objects keep track of the dependencies instead.
Let each graph object have a list of other objects that it has connections with. Now you can simply free any of the objects, and the housekeeping code for the list will remove all references to the object being destroyed from all other objects. If you want to modify the graph you simply free the nodes that you want removed, and the dependencies will be updated. If you want to destroy the whole graph, just destroy all nodes. Basically you have a list of nodes (ownership, lifetime management) and another data structure for describing the graph.
You can use interfaces instead of objects, underlaying objects will be automatically reference counted, and the best thing is that delphi will destroy objects for you, when refcount = 0.
If you are using D2009 you can use Barry Kelly's implementation of Smart Pointers.
http://barrkel.blogspot.com/2008/11/somewhat-more-efficient-smart-pointers.html
Otherwise using interfaces is your best bet. Create an interface with ALL the functionality you require, create an object that implements that interface and descends from TInterfacedObject and then use interfaces everywhere, not the object.
As you probably know, Delphi does not have a garbage collector.
You're best bet would be to have your objects descend from TInterfacedObject.
Something like
type
IMyObject = interface
[Guid]
end;
MyObject = class(IMyObject, TInterfacedObject)
end;
var
myObject: IMyObject;
begin
myObject := TMyObject.Create;
end
The myObject variable is reference counted and will be freed when its reference count drops to zero.
Related
Can anyone point me toward tutorials or best practice for using object interfaces in a TList?
I started implementing this model to take advantage of the reference counting as an alternative to TObjectList (that I use extensively). While I had no problems implementing the list it appears that doing this adds complexity in other ways.
Specifically because the list is only holding an interface I don't have access to the class fields (obvious). So I end up littering the code with "MyInterface as TMyClass" or creating a lot of functions to access the fields. Is there some trick that I am not aware of that makes an interface container more usable?
If you need to access certain methods, why are you hiding them with interfaces? An interface is just that: A description of the ways that others can interact [interface] with that object: If you want other classes to interact with your class as an interfaced object, those methods should be part of the interface.
Using an interface as a shortcut to simplify finalization and then hacking it with casting violates the entire concept of interfaces, and can also cause memory leaks and hard to track down AV's, because of the way you are mixing your references: sometimes interface, sometimes TObject.
Can anyone point me toward tutorials or best practice for using object
interfaces in a TList?
Your problem has nothing to do with TList. All TList does is hold references to your objects, however they are designed - it knows nothing about the specifics of your object's interface or implementation. Your problem is that you are not designing your classes and interfaces correctly, as #ArnaudBouchez has pointed out in the comments.
I started implementing this model to take advantage of the reference
counting as an alternative to TObjectList (that I use extensively)
Why is that necessary? To simplify finalization? TObjectList owns its objects by default. (And you can also set the OwnsObjects property ) . You don't need to use interfaces to automate your clean-up if you're using TObjectList. You just have to free the TObjectList instance, just as you have to free a TList. Unless you need interfaces for a specific reason, using Objects and TObjectList is arguably preferable, since as opposed to interfaces, your finalization is deterministic.
From XE Help:
Create TObjectList instance.
This overloaded method creates a TObjectList instance.
The OwnsObjects parameter is a boolean that indicates whether object
entries are owned by the list. If the object is owned, when the entry
is removed from the list, the object is freed. The OwnsObjects
property is set from the value of this parameter. The default is true.
When you free TObjectList, the objects are also deallocated when OwnObjects=True. See:
Clear a TList or a TObjectList (accepted answer there)
Know: Normally there is no reason for you as a Delphi developer to get involved at all with details of reference counting. If you feel you need to do so, something is wrong with your design at the fundamental level. Reference counting is implemented in TInterfacedObject, and works transparently. All of your Delphi interfaced objects should be derived from TInterfacedObject (or one of its sibling or derivative classes), as follows:
interface
...
type
IMyInterface=interface (IUnknown)
[GUID]
function GetValue:integer;
end;
TMyInterface=class(TInterfacedObject,IMyInterface)
protected
function GetValue:integer;
end;
implementation
...
TMyInterface.GetValue:integer;
begin
result := fValue;
end;
end.
Then use it like this:
procedure useValue;
var im:IMyInterface;
x:integer;
begin
im:=TMyInterface.create;
x:=im.GetValue;
showMessage(inttstr(x));
end;
When useValue returns, im is cleaned up - you don't have to do anything or worry about reference counting. TInterfacedObject takes care of all that.
Note that in useValue, you declare the reference as IMyInterface but call the class constructor for TMyInterface - that is how you must initialize the interface - an interface has no constructor. However, since your reference type is IMyInterface, that is how it is treated by the compiler - only methods exposed through IMyInterface will be accessible.
I'm fairly new to Delphi and have been doing all my memory management manually, but have heard references to Delphi being able to use interfaces to do reference counting and providing some memory management that way. I want to get started with that, but have a few questions.
Just generally, how do I use it. Create the interface and the class implementing it. Then anytime I need that object, have the variable actually be of the Interface type, but instantiate the object and presto? No nee to think about freeing it? No more try-finallys?
It seems very cumbersome to create a bunch of interfaces for classes that really don't need them. Any tips on auto generating those? How do I best organize that? Interface and class in the same file?
What are common pitfalls that might cause me grief? Ex: Does casting the interfaced object to the an object of its class break my reference counting? Or are there any non-obvious ways Delphi would create reference loops? (meaning besides A uses B uses C uses A)
If there are tutorials that cover any of this, that would be great, but I didn't come up with anything in my searches. Thanks.
I am currently working with a very large project that takes advantage of the "side affect" of interface reference counting for the purpose of memory management.
My own personal conclusion is that you end up with a lot of code that is overly complex for no better reason than, "I don't have to worry about calling free"
I would strongly advise against this course of action for some very basic reasons:
1) You are using a side affect that exists for the purpose of COM compatibility.
2) You are making your object footprint and efficiency heavier. Interfaces are pointers to lists of pointers.. or something along those lines.
3) Like you stated... you now have to make piles of interfaces for the sole purpose of avoiding freeing memory yourself... this causes more trouble than it's worth in my opinion.
4) Most common bug that will be a HUGE pain to debug will become when an object gets freed, before it's reference. We have special code in our own reference counting to try and test for this problem before software goes out the door.
Now to answer your questions.
1) Given TFoo and interface IFoo you can have a method like the following
function GetFoo: IFoo;
begin
Result := (TFoo.Create as IFoo);
end;
...and presto, you don't need the finally to free it.
2) Yes like I said, you think it's a great idea, but it turns into a huge pain in the bupkis
3) 2 problems.
A) you have Object1.Interface2 and Object2.Interface1... these objects will never be freed due to the circular reference
B) Freeing the object before all the references are released, I cannot stress how dificult these bugs are to track down...
The most common complaint leading to the desire for "automatic garbage collection" in Delphi is the way that even short-lived temporary objects have to be disposed of manually and that you have to write a fair amount of "boiler-plate" code to ensure that this takes place when exceptions occur.
For example, creating a TStringList for some temporary sorting or other algorithmic purpose within a procedure:
procedure SomeStringsOperation(const aStrings: TStrings);
var
list: TStringList;
begin
list := TStringList.Create;
try
:
// do some work with "list"
:
finally
list.Free;
end;
end;
As you mentioned, objects that implement the COM protocol of reference counted lifetime management avoid this by cleaning themselves up when all references to them have been released.
But since TStringList isn't a COM object, you cannot enjoy the convenience this offers.
Fortunately there is a way to use COM reference counting to take care of these things without have to create all new, COM versions of the classes you wish to use. You don't even need to switch to an entirely COM based model.
I created a very simple utility class to allow me to "wrap" ANY object inside a lightweight COM container specifically for the purpose of getting this automatic cleanup behaiour. Using this technique you can replace the above example with:
procedure SomeStringsOperation(const aStrings: TStrings);
var
list: TStringList;
begin
AutoFree(#list);
list := TStringList.Create;
:
// do some work with "list"
:
end;
The AutoFree() function call creates an "anonymous" interfaced object that is Release()'d in the exit code generated by the compiler for the procedure. This autofree object is passed a pointer to the variable that references the object you wish to be free'd. Among other things this allows us to use the AutoFree() function as a pseudo-"declaration", placing any and ALL AutoFree() calls at the top of the method, as close as possible to the variable declarations that they reference, before we have even created any objects.
Full details of the implementation, including source code and further examples, are on my blog in this post.
The memory management of interfaces is done through implementation of _AddRef and _Release which are implemented by TInterfacedObject.
In general using interfaces to make memory management less cumbersome can be a nice idea, but you need to take care of these things:
Make sure the classes that implement interfaces are derived from TInterfacedObject or roll your own ancestor class that provides good implementations for _AddRef and _Release
Use either/or: so either user interfaces references, or use object instance references, don't mix them. That can be problematic when implementing interfaces in components (as those derive from TComponent, not TInterfacedObject)
Don't go the TInterfacedComponent way as that mixes Owner based memory management and _AddRef/_Release based memory management
Watch circular interface references (you can go around implementing "weak interface references" mentioned here and implemented here)
You need to maintain extra code as you need to define interfaces for the parts your classes that you want to expose, and keep those two in sync (you could Model Maker Code Explorer for this; it allows you to extract interfaces and in general boost your development because it manages the interface/implementation parts of code in single-actions)
You need some extra plumbing to create instances of the underlying classes. You can use the factory pattern for that.
That is not always effectively, but does answer a few of your underlying questions.
Shortest possible answer: The default delphi memory model is that owners free the objects they own. All other references are weak references and must let go before the owner does. "Sharing" an object that has a lifetime shorter than the entire lifetime of the app is rarely done. Reference counting is rarely done, and when it is done, it is only done by experts, or else it adds more bugs and crashes than it solves.
Learn idiomatic delphi style and try to imitate it, don't fight the grain. Sadly, people think that "program against interfaces, not implementations" means "Use IUnknown everywhere". That's not true. I recommend you don't use COM IUnknown interfaces, and use abstract base classes instead. The only thing you can't do is implement two abstract base classes in a single class, and the need for that is rare.
Update: I've recently found it helpful to use COM Interfaces (IUnknown based) to help me separate out my model and controller implementations from my UI classes. So I do find using IUnknown based interfaces useful. But there is not a lot of documentation and prior art out there to base your efforts on. I'd like to see a "cookbook" style recipe that lays all this out for people, so they can work without the usual problem of combining interface and non-interface based lifetime management, and all the trouble that comes while you get used to that extra complexity.
Switching to interfaces only for avoiding manual Free's is senseless. Little economy in Free/try-finally lines will hardly compensate the necessity of declaring both g/setters and properties in the interface not mentioning the necessity of keeping the intf/class declarations in sync. Interfaces also bring performance loss due to implicit finalize code and reference counting. If performance is not the main point and all you want to achieve is autofreeing, I'd recommend using some universal interface wrappers like the one Deltics suggested.
I have a following definition.
type
TOmniTaskDelegate = reference to procedure(const task: IOmniTask);
What type of container should I use (should be supported in D2009) to store a list of TOmniTaskDelegate instances? Currently I'm using array of TOmniTaskDelegate but I'm not really happy with that.
I would use TList<TOmniTaskDelegate>. Since this is typesafe due to the use of generics, it will correctly handle the lifetime issues of its members.
Edit: Delphi 2009 includes the generic TList<T>, I assume it's implemented using array of, just as the one in Delphi 2010. That makes the TList<T> the optimal choice! My original answer stays because it explains why array of is a great data structure and why not using it is a lot of trouble.
Your choice of array of Anonym looks very good to me because:
Anonymous methods are managed entities (implemented using interfaces). They need to be properly finalized.
The dynamic array is itself a managed type, making sure the anonymous method references are properly finalized.
Delphi 2010 generic containers are implemented using dynamic arrays, so they're up to the task. But make sure you don't grow your arrays one-by-one, grow in chunks.
If you use anything else for the implementation you'll need to take care of finalizing the references yourself. Examples:
If you use plain blocks of memory you'll need an destructor that deliberately sets each item to nil (ie: not ZeroMemory or FillChar) so the compiler gets a chance to generate finalization code.
Records are managed objects, and they could hold references to dynamic methods, but they can only hold a finite number of references, if you need more you'll need to implement a sort of linked list and then you'll need to carefully manage there life cycle.
Classes suffer all the deficiencies of records, and they add their own layer of overhead on top of that.
I have implemented the FlyWeight pattern in my Delphi application. Everything has worked great, everything is a lot faster and takes less memory, but there is one thing I am worried about.
My implementation will only work as long as client code never calls Free() on the shared objects. In the Flyweight pattern, the FlyweightFactory itself is supposed to "maintain a reference to flyweights" i.e. to the shared objects.
My problem is that there is no (obvious) way to stop other code from destroying the objects once they have a reference. I could live with this, but it would be a "big win" if I could pass these objects round freely without worrying about accidental freeing.
To show a (contrived) example:
flyweight1:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
flyweight1.Free;
flyweight2:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
// Object has already been Freed!; behaviour is undefined
I have considered overriding the destructor as shown here to stop the flyweight object being freed altogether. This is not an option in my case as
a) I only want to stop cached objects from being Freed, not objects that aren't part of the cache. There is a lot of legacy code that doesn't use the cache; they still need to create and free objects manually.
b) I do want the FlyweightFactory to Free the objects during finalization; I agree with Warren P that a "zero leaked memory" policy is best.
I'll leave with a quote from the Flyweight chapter of GoF
Sharability implies some form of
reference counting or garbage
collection to reclaim storage when
it's no longer needed. However,
neither is necessary if the number of
flyweights is fixed and small. In that
case, the flyweights are worth keeping
around permanently.
In my case the flyweights are "fixed" and (sufficiently) small.
[UPDATE See my answer for details of how I solved this problem]
My answer to the question you link to still applies. The objects must know by means of a private boolean flag that they are cached objects. Then they can elect not to destroy themselves in Destroy and FreeInstance. There really is no alternative if you want to allow Free to be called.
To deal with finalization you would want to add the cached objects to a list of cached objects. That list of objects can be freed at finalization time. Of course the flag to disable freeing would have to be reset whilst you walked the list.
Having made this point regarding finalization, I would advise you to register an expected memory leak and just leak this memory. It makes the code much simpler and there's nothing to lose. Any memory you don't free will be reclaimed by the OS as soon as your executable closes. One word of caution: if your code is compiled into a DLL then leaking could be troublesome if your DLL is loaded, unloaded, loaded again etc.
What all this is telling you is that you are swimming against the current. Is it possible that you could achieve your goals with a different solution that fitted better with the way Delphi is steering you?
I suggest to add a reference count in order to known if your shared object is still used.
Every client should use the pattern AddRef / Release (AddRef increases the count; Release decrements it; if count reaches zero Free is called)
The AddRef may be called directly by your GetFlyweight method; Release has to be used instead of Free.
If you refactor your class and extract an interface from it the AddRef/Release pattern in naturally implemented in then interface implementation. (You could derive from TInterfacedObject or implement IInterface by your self)
Ideally you seldom want 2 ways of using the same things. It just complicates matters in the long run. In 6 months time, you might not be sure whether a particular piece of code is using the new flyweight paradigm or the old paradigm.
The best way to prevent someone calling Free or Destroy is to make sure it's not even there. And within the Delphi world, the only way to do that is to use interfaces.
To expand on your contrived example:
type
TFlyweightObject = class
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): TFlyweightObject;
end;
This being an object can easily be destoyed by a rogue client. You could make the following changes:
type
IFlyweight = interface
//place guid here
function Description: string;
end;
TFlyweightObject = class(TInterfacedObject, IFlyweight)
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): IFlyweight;
end;
Now any code that is updated to use the flyweight paradigm is forced to use it as intended. It's also easier to recognise the old code that still needs to be refactored because it doesn't use the interface. Old code would still construct the "flyweight" object directly.
You could also hide a destructor by making it protected or private. Programmers won't see it outside the scope of the unit in which it is declared in.
But I am posting this answer more like a curiosity because this will not prevent freeing an object by using FreeAndNil or by using a "Protected Hack"
I managed to get around the problems I cited in my original question using the following techniques, suggested by David Heffernan in his answer.
a) I only want to stop cached objects
from being Freed, not objects that
aren't part of the cache. There is a
lot of legacy code that doesn't use
the cache; they still need to create
and free objects manually.
I fixed this by subclassing the Flyweight class and overriding destroy, BeforeDestruction and FreeInstance in the subclass only. This left the parent class as is. The cache contains instances of the subclass (which can't be freed), whereas objects outside the cache can be freed as per usual.
b) I do want the FlyweightFactory to
Free the objects during finalization;
I agree with Warren P that a "zero
leaked memory" policy is best.
To solve this, I added a private boolean flag that has to be set to true before the object can be freed. This flag can only be set from the Cache Unit, it is not visible to other code. This means that the flag cannot be set outside by code outside the cache.
The destructor just looks like this:
destructor TCachedItem.destroy;
begin
if destroyAllowed then
inherited;
end;
If client code trys to Free a cached object, the call will have no effect.
I'm still something of a newbie, and I know my thinking is incorrect; I just don't know where ...
Just about everything in Delphi is descended from TObject. What if everything instead descended from a TInterfaceObject that implemented some trivial interface (e.g., "INamable," with a single method that returned a class's name string)? Since TObject already has a property that returns a name string, you wouldn't need to add anything to additional classes.
In other words, a TInterfacedObject would inherit from TObject (or something high up in the hierarchy), and everything currently descending from TObject would now descend from this new class. Wouldn't this mean everything was now reference counted?
If you can spot where my knowledge is lacking, I'd love to learn. Thanks, as always -- Al C.
It's not clear whether you're asking:
Why didn't Borland do this, when they originally developed Delphi?
Why don't Embarcadero do this, in a future version of Delphi?
Why don't I do this, with my own user data types?
Wouldn't this mean everything was now reference counted?
Yes it would.
However, you don't necessarily want everything to be ref-counted: every little integer, every string, every boolean, every element in an array ... if for no other reason that the implementation of ref-counting adds some overhead, e.g. a little extra memory per object, perhaps insignificant for large objects but proportionally more significant if applied to every tiny object.
Also, see also Garbage Collector For Delphi Objects and Components which says (quote),
Delphi provides three ways of object management :
Create/destroy the objects using try..finally.
Use TComponent descendants - create a component and let its owner free it.
Interfaces - when the reference count for an interface becomes 0 the
object which implements it is
destroyed.
The Delphi help says you shouldn't mix
the TComponent owner approach with the
interface memory management, but ...
Would this be garbage collection?
Not quite; mere reference-counting isn't as robust as garbage-collection:
With reference-counting, if you have two reference-counted instances each holding a reference to the other, then they're not released automatically. To release them you would need to break this 'circular reference' (i.e. explicitly tell one of them to release its reference to the other).
With true garbage-collection, the garbage-collector would notice that those two istance aren't referenced from anywhere else, and release them both.
Update
If you annotate your potentially circular references as [weak] references, then they will get destroyed ok. But prior to Delphi 10.1 Berlin this only works in the NexGen compilers (i.e. those that use LLVM under the hood). From 10.1 Berlin onwards these [weak] references work everywhere.
It wouldn't be working garbage collection because interfaces use a very simple reference-counting system, and circular references, which are very common in Delphi code, break simple ref-counting.
No, because of two things:
Even if a class implements an interface it does not automatically make it reference counted. Only if you actually use it to implement that interface the reference counting will have any effect.
As others already said: Reference counting in interfaces will result in the class instance to be freed immediately when the reference count reaches 0. It is an implicit call to the Free method at that point in code. This will fail e.g. if two objects reference each other. True garbage collection will free the objects not when they go out of scope but when memory is needed, so there is no performance impact every time the reference count reaches 0 because the object will just continue to exist. In addition a good garbage collector will detect the isolated circular references (e.g. A references B references C references A but nothing else references any of these objects) and will free these objects as well.
Garbage collection is different from simple ref counting. You can have automatic deletion when a ref count reaches 0, but that too is not garbage collection. Garbage collection means letting go of your ability to control when things are deleted from memory, and allowing the underlying language's implementation to optimize the behviours. You stop paying attention to reference counts, and trust in the dynamic behaviours of a particular implementation of garbage collection.
Naturally, garbage collection uses a system of reference counting to know when something is no longer referenced, but that is a small piece of the puzzle.
Reference counting is a form of garbage collection, but not a very good one. It is used by some languages (python I think) although often with cycle detection.
Even if you descended from TInterfaceObject, the object is not reference counted and thus garbage collected unless you only use the interface reference and not the object reference.
I.e. you would need to use
Var
nameable: IMyInterface;
begin
nameable:= IMyInterface.Create();
nameable.x(y);
etc
end;
This implies that your interface needs to support already the methods and properies that you require which quickly becomes tedious as you need to create an interface for each class.
It can be done reasonably easily in D2009 or later though. See Barry Kelly's implimentation of smart pointers. The usual reference countng cavets apply though.