How can I allocate class using HeapCreate and HeapAlloc in Delphi? The following example will just crash.
program Project2;
uses
System.SysUtils,
System.Classes,
Winapi.Windows;
var
SL: TStringList;
Handle: THandle;
P: ^TStringList;
begin
Handle := HeapCreate(0, SizeOf(TStringList), SizeOf(TStringList));
P := HeapAlloc(Handle, 0, SizeOf(TStringList));
P.Add('some random string'); // crash
HeapFree(Handle, 0, #P);
end.
Delphi provides only limited facility for allocating objects from anywhere other than the main memory manager. To do so, you need to override the class's NewInstance method. That's the method the constructor calls to allocate a new instance of the class. The destruction counterpart is FreeInstance.
Override those methods and call HeapAlloc and HeapFree. The number of bytes to allocate is not given by SizeOf, though. SizeOf tells you the size of the object reference, which is always SizeOf(Pointer). You need the size of the instance, which is given by the class's InstanceSize method.
Although you can override the methods to use your chosen memory-allocation strategy, you're probably not going to be satisfied with the results because there are a few problems:
The NewInstance method doesn't accept any parameters, so there's no way for you to tell the class which heap to allocate from. You could have a single global heap, or a heap per class, but you can't choose which heap to use for each instance individually.
The NewInstance method is shared by all instances of a given class, so there's no way to allocate only some instances on your special heap and allocate the rest from the default memory manager. This is mainly another way of wording the previous problem.
You can't retrofit this into existing classes, so you can allocate TMyStringList, but not TStringList. (Well, you can, but it requires patching each class's VMT, which is generally never recommended.)
There are a few problems with the code you tried. First is that you never actually allocated a TStringList. You allocated a pointer to one, which, as I mentioned above, is SizeOf(Pointer) bytes. That's not enough memory to hold a TStringList instance.
You never really needed a ^TStringList at all. You could have assigned your allocated memory directly to the TStringList variable SL:
SL := HealAlloc(Handle, 0, TStringList.InstanceSize);
Notice how I've adjusted the size.
That's still not enough, though, because while this has allocated some memory, it hasn't constructed the object. You'd need to call the constructor.
SL.Create;
Note that the constructor will allocate more memory to hold the list of strings, and allocate still more memory to hold the string contents. Those memory allocations won't go on the heap. They'll go to the default memory manager, just as they always do.
Destroying the object will be another issue. You need to call the destructor to free the string of strings, but if you do that, the destructor will attempt to free the object's memory, too. It will use the default memory manager to free it, but since you didn't use the default memory manager to allocate the TStringList object, the memory manager will throw an EInvalidPointer exception.
Delphi provides no way to destroy an object without also releasing the associated memory. That is, you cannot call Destroy without also calling FreeInstance, but TStringList.FreeInstance will call FreeMem, not HeapFree.
I recommend taking a step back from your current task and re-examining the problem you were aiming to solve with your idea of a separate heap for certain objects. There might be better solutions that don't involve as much effort to fight against what your tools are designed to do.
Related
I am struggling to learn delphi and memory management, coming from C#.
The current incarnation of that struggle is that I don't know the right way to dispose of the objects when I am done with them. From reading and my experiments it seems that if I have an object that is cast as an interface, then my ONLY choice is set the reference to nil.
If I go an call
FreeAndNil()
I end up getting an access violation, EX:
var
foo: IFoo;
begin
foo := TFoo.Create();
FreeandNil(foo);
end;
Sure, all I need to do it change that foo:IFoo; to foo:TFoo; and it is happy. OR simply set the pointer to nil, NOT call freeandNil.
foo := nil;
So, on one level, I don't understand in the least where the AV is.
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way, but I can't seem to write a method that can deal with something that is a Class or an interface. Well, that is not true, I do have something, but it is so ugly I hesitate to post it.
But I guess I should also be asking, what is everyone else doing? Mentally keeping track of what is an interface and just nil those pointers? otherwise calling FreeAndNil?
I am going to want to implement things the first time around as a concrete class, but later come back and change that to an interface when I find some way that the code can do from 2 different ways. And I am not going to want to go through the code and change how it was dealing with that reference, that is the last thing on my mind at that point.
But for the sake of discussion, the best (almost only) idea I have is this class:
interface
type
TMemory = class(TObject)
class procedure Free(item: TObject); overload; static;
class procedure Free<T: IInterface>(item: T); overload; static;
end;
implementation
uses
System.SysUtils;
{ TMemory }
class procedure TMemory.Free(item: TObject);
begin
FreeandNil(item);
end;
class procedure TMemory.Free<T>(item: T);
begin
//don't do anything, it is up the caller to always nil after calling.
end;
Then I can consistently call:
TMemory.Free(Thing);
Thing := nil;
Test code:
procedure TDoSomething.MyWorker;
var
foo: IFoo;
fooAsClass: TFoo;
JustAnObject: TObject;
begin
foo := TFoo.Create();
fooAsClass := TFoo.Create();
JustAnObject := TObject.Create();
TMemory.Free(foo);
foo := nil;
TMemory.Free(fooAsClass);
fooAsClass := nil;
TMemory.Free(JustAnObject);
JustAnObject := nil;
end;
runs with no leaks or access violations. (using MadExcept)
But a big thank you to the Delphi community on SO. You guys have been the best thing out there for learning!
The reason for the access violation is that FreeAndNil takes an untyped parameter, but expects it to be an object. So the method operates on the object.
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj); //Obj must be a TObject otherwise all bets are off
Pointer(Obj) := nil; //Will throw an AV if memory violation is detected
Temp.Free; //Will throw an AV if memory violation is detected
end;
A memory violation in the above might (NB not guaranteed) be detected if you destroy an object that has either been previously destroyed or never created. It's also likely to be detected if Obj doesn't reference an object at all but something else (such as an interface, record, Integer because these don't implement Free and if they did, it wouldn't be located in the same way as TObject.Free).
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way.
This is like saying you want to use your car in exactly the same way that you use your shower.
Ok, maybe the difference is not quite that extreme. But the point is that interfaces and objects (and for that matter records) use different memory management paradigms. You cannot manage their memory in the same way.
Objects need to be explicitly destroyed. You can use an ownership model, but destruction is still an explicit external action.
Interfaces are reference counted. The compiler injects code to track the number of fields and variables referencing (looking at) the underlying instance. Typically the object destroys itself when the last reference is released. (There are ways beyond the scope of this answer to change this.)
If we access some object by interface variable, it doesn't always mean that object is destroyed the moment reference counter drops to zero. For example, TComponent methods _AddRef and _Release implementations are 'dummy': no reference counting is implemented and TComponent is never destroyed because interface variables are out of scope.
To behave as we expect from 'real' interfaces, all your objects should be descendants from TInterfacedObject or you need to implement _AddRef / _Release yourself.
Yes, there are 2 different approaches to memory management which usually co-exist in a program, but confusion (and AV) arises only when the same object is treated in both ways. If we destroyed object and only then the interface variables have gone out of scope, they call _Release method of destroyed object which causes access violation. That's some risky business, though with some attention it is doable.
Classic Delphi components are not reference-counted, the concept of ownership is used instead. Each component has an owner whose responsibility is to free all the memory when it itself is destroyed. So each component has an owner, but it may also have a lot of pointers to another components, like when Toolbar has ImageList variable. If such components were refcounted, they would never be destroyed because of circular reference, so in order to break this circle you'd need 'weak' references as well which don't 'count'. They are here, too, but that's very recent feature of Delphi.
If there is some hierarchy in your objects, so you know that 'bigger' objects need all of 'smaller' ones to function, then use this good old approach, it's pretty simple and has very good implementation in Delphi, which is: you can make a code which will be leak-free no matter where exception could arise. There are all these little things like using .Free instead of .Destroy, because if exception happened in constructor, destructor is called automatically, and so on. Very clever solution in fact.
I'd use refcounted interfaces only if you don't know for how long some object is needed for you and there is no suitable 'owner' for it. I did it with scanned image which I saved to file in one thread, while converting to smaller image to show on screen on another thread. When all is done, image is no more needed in RAM and can be destroyed, but I have no idea which happens first. In this case using refcounting is best thing to do.
FastMM reports a memory leak (UnicodeString) for the following code snippet that uses a record thread variable with a string:
program Project10;
{$APPTYPE CONSOLE}
{$R *.res}
uses
FastMM4,
System.SysUtils;
type
TContext = record
Value : String;
end;
threadvar
Context : TContext;
begin
Context.Value := 'asdfsdfasfdsa';
end.
Is this a real memory leak or is the cleanup of thread variables just happening after FastMM checks for memory leaks?
Even more important: how can I suppress these "memory leaks" being reported as they clutter up any other memory leaks that might be found?
That's a real leak. Thread local variables are not finalized when they go out of scope. Because your record contains a field that is managed, the string field, the heap allocated memory associated with that string is leaked if the record is not finalized.
The documentation calls this out explicitly:
Dynamic variables that are ordinarily managed by the
compiler (long strings, wide strings, dynamic arrays, variants,
and interfaces) can be declared with threadvar, but the compiler does not automatically free the heap-allocated
memory created by each thread of execution. If you use
these data types in thread variables, it is your responsibility
to dispose of their memory from within the thread, before
the thread terminates.
If you want to plug the leak, you'll need to finalize the variable as the scope ends. That is, as the thread is terminating.
Finalize(Context);
Note that you must execute this code from the thread which owns the variable since, obviously, only that thread has access to it.
If you want to suppress reporting of these leaks then call RegisterExpectedMemoryLeak.
If you cannot execute code when threads terminate then it may be better to avoid heap allocation and use a fixed length character array. Quite possibly that you meet your needs.
It would seem odd that you claim not to be able to execute code when the threads terminate. If you cannot do that, how are you able to execute any code in the context of these threads. In other words, in order for there to be a leak, you must have executed your code in these threads.
Create global array of TContext, and then store index of the element, that belongs to your thread, in a threadvar.
I have implemented the FlyWeight pattern in my Delphi application. Everything has worked great, everything is a lot faster and takes less memory, but there is one thing I am worried about.
My implementation will only work as long as client code never calls Free() on the shared objects. In the Flyweight pattern, the FlyweightFactory itself is supposed to "maintain a reference to flyweights" i.e. to the shared objects.
My problem is that there is no (obvious) way to stop other code from destroying the objects once they have a reference. I could live with this, but it would be a "big win" if I could pass these objects round freely without worrying about accidental freeing.
To show a (contrived) example:
flyweight1:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
flyweight1.Free;
flyweight2:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
// Object has already been Freed!; behaviour is undefined
I have considered overriding the destructor as shown here to stop the flyweight object being freed altogether. This is not an option in my case as
a) I only want to stop cached objects from being Freed, not objects that aren't part of the cache. There is a lot of legacy code that doesn't use the cache; they still need to create and free objects manually.
b) I do want the FlyweightFactory to Free the objects during finalization; I agree with Warren P that a "zero leaked memory" policy is best.
I'll leave with a quote from the Flyweight chapter of GoF
Sharability implies some form of
reference counting or garbage
collection to reclaim storage when
it's no longer needed. However,
neither is necessary if the number of
flyweights is fixed and small. In that
case, the flyweights are worth keeping
around permanently.
In my case the flyweights are "fixed" and (sufficiently) small.
[UPDATE See my answer for details of how I solved this problem]
My answer to the question you link to still applies. The objects must know by means of a private boolean flag that they are cached objects. Then they can elect not to destroy themselves in Destroy and FreeInstance. There really is no alternative if you want to allow Free to be called.
To deal with finalization you would want to add the cached objects to a list of cached objects. That list of objects can be freed at finalization time. Of course the flag to disable freeing would have to be reset whilst you walked the list.
Having made this point regarding finalization, I would advise you to register an expected memory leak and just leak this memory. It makes the code much simpler and there's nothing to lose. Any memory you don't free will be reclaimed by the OS as soon as your executable closes. One word of caution: if your code is compiled into a DLL then leaking could be troublesome if your DLL is loaded, unloaded, loaded again etc.
What all this is telling you is that you are swimming against the current. Is it possible that you could achieve your goals with a different solution that fitted better with the way Delphi is steering you?
I suggest to add a reference count in order to known if your shared object is still used.
Every client should use the pattern AddRef / Release (AddRef increases the count; Release decrements it; if count reaches zero Free is called)
The AddRef may be called directly by your GetFlyweight method; Release has to be used instead of Free.
If you refactor your class and extract an interface from it the AddRef/Release pattern in naturally implemented in then interface implementation. (You could derive from TInterfacedObject or implement IInterface by your self)
Ideally you seldom want 2 ways of using the same things. It just complicates matters in the long run. In 6 months time, you might not be sure whether a particular piece of code is using the new flyweight paradigm or the old paradigm.
The best way to prevent someone calling Free or Destroy is to make sure it's not even there. And within the Delphi world, the only way to do that is to use interfaces.
To expand on your contrived example:
type
TFlyweightObject = class
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): TFlyweightObject;
end;
This being an object can easily be destoyed by a rogue client. You could make the following changes:
type
IFlyweight = interface
//place guid here
function Description: string;
end;
TFlyweightObject = class(TInterfacedObject, IFlyweight)
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): IFlyweight;
end;
Now any code that is updated to use the flyweight paradigm is forced to use it as intended. It's also easier to recognise the old code that still needs to be refactored because it doesn't use the interface. Old code would still construct the "flyweight" object directly.
You could also hide a destructor by making it protected or private. Programmers won't see it outside the scope of the unit in which it is declared in.
But I am posting this answer more like a curiosity because this will not prevent freeing an object by using FreeAndNil or by using a "Protected Hack"
I managed to get around the problems I cited in my original question using the following techniques, suggested by David Heffernan in his answer.
a) I only want to stop cached objects
from being Freed, not objects that
aren't part of the cache. There is a
lot of legacy code that doesn't use
the cache; they still need to create
and free objects manually.
I fixed this by subclassing the Flyweight class and overriding destroy, BeforeDestruction and FreeInstance in the subclass only. This left the parent class as is. The cache contains instances of the subclass (which can't be freed), whereas objects outside the cache can be freed as per usual.
b) I do want the FlyweightFactory to
Free the objects during finalization;
I agree with Warren P that a "zero
leaked memory" policy is best.
To solve this, I added a private boolean flag that has to be set to true before the object can be freed. This flag can only be set from the Cache Unit, it is not visible to other code. This means that the flag cannot be set outside by code outside the cache.
The destructor just looks like this:
destructor TCachedItem.destroy;
begin
if destroyAllowed then
inherited;
end;
If client code trys to Free a cached object, the call will have no effect.
I haven't been able to find the answers to a couple of my Delphi memory management questions. I could test different scenarios (which I did to find out what breaks the FreeAndNil method), but its takes too long and its hard! But seriously, I would also like to know how you all (Delphi developers) handle these memory management issues.
My Questions (Feel free to pose your own I'm sure the answers to them will help me too):
Does FreeAndNil work for COM objects? My thoughts are I don't need it, but if all I need to do is set it to nil than why not stay consistent in my finally block and use FreeAndNil for everything?
Whats the proper way to clean up static arrays (myArr : Array[0..5] of TObject). I can't FreeAndNil it, so is it good enough to just set it to nil (do I need to do that after I've FreeAnNil'd each object?)?
Thanks Guys!
COM objects are referenced via Interfaces, which you don't need to do anything to free. The compiler takes care of the necessary reference-counting logic to make sure the COM object will be disposed of at the right time.
As for static arrays, (or dynamic arrays, for that matter,) they don't need to be freed by you either. If they contain objects then the objects have to be freed at the appropriate time, but the arrays don't.
Also, never use FreeAndNil on anything that's not an object reference. Using it with interfaces or other variables can corrupt memory. It's best to never use it (use Free instead) unless you're dealing with an object that you need to free and then reuse later.
First, in most situation, FreeAndNil is a bit of overkill. It's handy when you free and object's field outside it's destructor, or on a global(ugly) variable. But most of the time, just calling free is enough.
As you should know, an object variable is actually a pointer to the object's data. When you call Free, that buffer is freed (after the destructor is ran, of course), but the Object variable still points to the memory position that was just freed. It's called a "Dangling pointer". Having a dangling pointer is not a problem as long as you KNOW it's dangling in that context. For exemple:
Procedure Myproc;
var vString : TStringList;
begin
//Here, vString is "dangling"
vString := TStringList.Create;
//Here, vString is valid
try
//Do some stuff
finally
vString.Free;
end;
//Here, vString is "dangling"... But who care, it's about to go out of scope and we won't use it again.
end;
Calling FreeAndNil makes more sense on global variable where you don't know exactly when or how the variable can be freed. With that being said, there is nothing wrong in calling FreeAndNil all the time (except in very tight loops where you try to get every oz of performance).
Now, for the COM objects... Like Mason stated, they are reference counted. So if you hold the only reference to that interface, calling MyInterface := nil; will free it. But when/if the variable goes out of scope, the compiler take care of adding cleanup code to make sure the interface reference is decremented. So if you are trying to keep the memory requirement to a minimum, set the interface to nil. Otherwise, it doesn't matter that much.
As for your array... You can just call Free on every items in the list... Optionnaly set them to nil after.
Regarding static arrays, if you created the contents of the array, just free those objects you created. You don't need to do anything special to clean up the space used by myArr itself.
COM objects are automatically reference counted; as soon as variable goes out of scope or the object that has the interface pointer as a field is deleted Delphi will call _Release and the object will delete itself. You don't need to set anything to nil explicitly.
For static arrays you need to loop over them and free each object explicitly.
I inherited an Intraweb app that had a 2MB text file of memory leaks as reported by FastMM4. I've got it down to 115 instances of one class leaking 52 bytes.
A brief description of the bad actor is:
TCwcBasicAdapter = class(TCwcCustomAdapter)
protected
FNavTitleField: TField;
function GetAdapterNav(aDataSet: TDataSet): ICwcCDSAdapterNav; override;
public
constructor Create(aDataSource: TDataSource; aKeyField, aNavTitleField: TField; aMultiple: boolean);
end;
and the interface is:
ICwcCDSAdapterNav = interface(IInterface)
Am I barking up the wrong tree, since the property is reference counted? Are there any circumstances where the interface property could keep the class from being destroyed?
Here is the implementation of the method above:
function TCwcBasicAdapter.GetAdapterNav(aDataSet: TDataSet): ICwcCDSAdapterNav;
var
AdapterNav: TCwcCDSAdapterNavBase;
begin
result := nil;
if Assigned(aDataSet) then begin
AdapterNav := TCwcCDSAdapterNavBasic.Create(aDataSet, FKeyField.Index, FNavTitleField.Index);
try
AdapterNav.GetInterface(ICwcCDSAdapterNav, result);
except
FreeAndNil(AdapterNav);
raise;
end;
end;
end;
with the class declared as:
TCwcCDSAdapterNavBase = class(TInterfacedObject, ICwcCDSAdapterNav)
FastMM should give you what is leaked and where it was created.
That would help narrowing it down to the real culprit: who is leaking what?
I'm not sure what really your question is?
Your code is incomplete or not the one in question: your class does not have an Interface property nor an Interface private Field, just a method that returns an Interface, which is harmless.
Edit: Without seeing the code of your Object implementing ICwcCDSAdapterNav, we can't tell if it is indeed reference counted.
If you don't descend from TInterfacedObject, chances are that it's not reference counted and that you cannot rely on this automagically freeing...
You may want to give a look at this CodeRage 2 session: Fighting Memory Leaks for Dummies. It mainly shows how to use FastMM to prevent/detect memory leaks in Delphi. Was for D2007 but still relevant for other versions.
You've got some good answers so far about how FastMM works. But as for your actual question, yes, interfaced objects can leak in two different ways.
Interfaces are only reference-counted if the objects they belong to have implemented reference counting in their _AddRef and _Release methods. Some objects don't.
If you have circular interface references, (Interface 1 references interface 2, which references interface 1,) then the reference count will never fall to 0 without some special tricks on your part. If this is your problem, I'll refer you to Andreas Hausladen's recent blog post on the subject.
If you are leaking 115 instances of that class, then it is that class that is being leaked. The memory occupied by that class, not the memory occupied by the things it refers to, is being leaked. Somewhere, you have 115 instances of TCwcBasicAdapter that you're not freeing.
Furthermore, properties don't store data, no matter they're interfaces or some other type. Only fields occupy memory (along with some hidden space the compiler allocates on the class's behalf).
So, yes, you are barking up the wrong tree. Your memory leak is somewhere else. When FastMM tells you that you have a memory leak, doesn't it also tell you where each leaked instance was allocated. It has that capability; you might need to adjust some conditional-compilation symbols to enable that feature.
Surely it's not only instances of that class that are leaking, though. FastMM should also report some other things leaking, such as instances of the class or classes that implement the interface.
Based on the function you added, I've begun to suspect that it's really TCwcCDSAdapterNavBase that's leaking, and that could be because of the atypical way you use for creating it. Does the exception handler in GetAdapterNav ever run? I doubt it; TObject.GetInterface never explicitly raises an exception. If the object doesn't support the interface, it returns False. All that exception handler could catch are things like access violation and illegal operations, which you really shouldn't be catching there anyway.
You can implement that function more directly like this:
if Assigned(FDataSet) then
Result := TCwcCDSAdapterNavBase.Create(...);