I am looking for hints on how to debugging a crash in an application that uses the MS XML wrappers in the Delphi VCL. I suspect memory corruption, or some kind of obscure evil thing happening between objects and interfaces, such as reference counting bugs, or heap corruption. The question is, in effect: how do I debug such a crash?
This particular code makes heavy internal use of and extends on the base XmlIntf Interfaces (IXMLNode). ISomethingCustom is an interface that extends IXMLNode. THe problem happens where we crash somewhere in a recursive function that is passed an ISomethingCustom which is also (or supports also, in interface terms) IXMLNode.
boolean UtilityFunction( aNode: ISomethingCustom ):Boolean;
begin
if not Assigned(aNode) then exit; // this works. great.
if not Assigned(aNode.ParentNode) then exit; // this DOES NOT WORK.
// code that blows up if aNode.ParentNode is not assigned.
end;
The situation is that the aNode is also IXMLNode, and IXMLNode.ParentNode value is assigned (not nil), and yet it points to a COM object that may have been freed, destroyed, or corrupted somehow. I am trying to figure out WHAT is going on when an interface pointer can appear to be valid, but the object behind it has been nuked somehow.
Checking Assigned(aNode.ParentNode) returns TRUE, even when, if you were to attempt a cast in the debugger (at runtime only, not in the code), like this:
inspect/evaluate aNode
inspect/evaluate TInterfacedObject(aNode).ClassName
(works in Delphi 2010, at least!)
now cast TWhateverClassNameYouGotBefore(aNode).
In the debugger I now see that this is NIL. WHich may mean that
the magic "casting interface back to
the object" feature that is new in
delphi 2010, is failing.
I believe I am trying to debug a problem where heaps are corrupted, or COM objects are corrupt on the heap, because of a reference counting problem.
I really think that nobody should ever have the situation arise where an interface appears valid, but the object underneath has been deleted. I really would like to know what to do, and what is going on.
Although you haven't shown it in your code, your comments seem to indicate that you're type-casting the interface variable to a class type. That's not allowed. I've described why:
Why can’t I cast an interface reference to an object reference?
Interface references and object references don't point to the same things. Therefore, calling a method on one when the compiler thinks you have the other will yield unexpected results. You were unlucky because the code continued to run instead of crashing with an access violation, which would have been a bigger indication that you were doing something wrong.
My article above concludes by suggesting you use the JclSysUtils.GetImplementorOfInterface function from the JCL if you have a Delphi-implemented interface and the interface offers no function of its own for revealling the underlying object.
Wild guess: Have you tried to put aNode.ParentNode in a local variable and use it in the rest of the Utilityfunction:
function UtilityFunction(aNode: ISomethingCustom): Boolean;
var
lParentNode: INode;
begin
if not Assigned(aNode) then exit; // this works. great.
lParentNode := aNode.ParentNode;
if not Assigned(lParentNode) then exit;
// code that uses lParentNode.
end;
My suggestion is to make sure that the ParentNode function is actually called in Assigned(aNode.ParentNode). There are some nasty corner-cases in Delphi where a procedure/function without arguments doesn't get called, but rather it's reference is taken when you omit the parenthesis's.
Try to change it to Assigned(Anode.ParentNode()) (which should have the same effect as François suggestion).
Related
I am having some difficulty understanding typecasting when using a class that is a passed parameter. I tried searching for this but couldn't find other answers.
I am working with some legacy Delphi code, using Delphi 2006, which doesn't support Generics (introduced in Delphi 2009).
The code is using TLists to store pointers to instantiated classes of particular types. When clearing the list, they use this:
procedure ClearList(AList: TList);
var i: Integer;
begin
for i := 0 to AList.Count - 1 do
TObject(AList[i]).Free;
AList.Clear;
end;
And it is called like this:
ClearList(FExtraVisitTypes);
ClearList(FDiagnoses);
ClearList(FProcedures);
ClearList(FImmunizations);
ClearList(FSkinTests);
ClearList(FPatientEds);
ClearList(FHealthFactors);
ClearList(FExams);
My understanding of this may be off, but I am concerned that if the pointed-to objects are freed as TObject, that the destructor of the descendant object won't be called, potentially leading to a memory leak. (My polymorphisim kung-fu is a bit rusty, which may be causing my confusion.)
So I tried to change the clear function as below:
procedure ClearList(AList: TList; ItemClass: TPCEItemClass); //mod to add ItemClass
var i: Integer;
begin
for i := 0 to AList.Count - 1 do begin
(AList[i] as ItemClass).Free;
end;
AList.Clear;
end;
TPCEItemClass is defined like this:
TPCEItemClass = class of TPCEItem;
I then changed the clear calls like this:
ClearList(FExtraVisitTypes, TPCEProc);
ClearList(FDiagnoses, TPCEDiag);
ClearList(FProcedures, TPCEProc);
ClearList(FImmunizations, TPCEImm);
ClearList(FSkinTests, TPCESkin);
ClearList(FPatientEds, TPCEPat);
ClearList(FHealthFactors, TPCEHealth);
ClearList(FExams, TPCEExams);
But the compiler won't allow this and gives this error:
[Pascal Error] uPCE.pas(1730): E2015 Operator not applicable to this operand type
For this erroneous line:
(AList[i] as ItemClass).Free;
Questions:
Does the original way of coding, where the item is freed by simply calling the great-great-great (etc) ancestor Free method end up effecting the descendant's destructor method? As I write this, I'm now thinking that it actually does. But I don't know why. So any answers to help me keep this in my head would be great.
Why does my method of trying to typecast via the parameter which is of type class not work? Is this just not allowed? Or is my syntax wrong? Is there another way to do this?
Am I going about this all wrong? Is there a better way?
Thanks
I am concerned that if the pointed-to objects are freed as TObject, that the destructor of the descendant object won't be called, potentially leading to a memory leak.
That is not the case for classes that are properly implemented.
All classes derive from TObject. TObject.Free() calls the TObject.Destroy() destructor, which is virtual. Any descendant that requires destruction logic must override that destructor (if it doesn't, it has a flaw that needs fixing).
So, in properly written code, the original code will work perfectly fine as shown. Calling Free() on any valid and correctly implemented object will invoke its most-derived destructor.
Now, that being said, there have been plenty of cases over the years of people forgetting to override the destructor when their classes require it, thus causing the kinds of memory leaks you are worried about. So, make sure you pay attention to what your classes are doing, and you will be fine.
So I tried to change the clear function as below ... But the compiler won't allow this and gives this error
Correct, because you can't perform a type-cast on an object using a variable to a metaclass type, like you are trying to do. Type-casts require the target type to be specified at compile-time, but metaclass variables are not assigned until runtime.
Does the original way of coding, where the item is freed by simply calling the great-great-great (etc) ancestor Free method end up effecting the descendant's destructor method?
The original code will work just fine 99% of the time, yes. Most Delphi coders are good about override'ing the destructor when it is appropriate. But that other 1% is only when you are dealing with classes that are not implemented correctly, in which case it is their author's responsibility to fix them, not your responsibility to fix the code that is calling Free() on them.
As I write this, I'm now thinking that it actually does. But I don't know why.
Polymorphic dispatch of the virtual destructor, just like when calling any other virtual method.
Why does my method of trying to typecast via the parameter which is of type class not work? Is this just not allowed?
Correct. It is illegal.
Is there another way to do this?
No (well, yes, but it involves walking an object's class structure's manually at runtime, but that requires a deep understanding of how the compiler lays out objects in memory, so I'm not going to get into that here).
I am struggling to learn delphi and memory management, coming from C#.
The current incarnation of that struggle is that I don't know the right way to dispose of the objects when I am done with them. From reading and my experiments it seems that if I have an object that is cast as an interface, then my ONLY choice is set the reference to nil.
If I go an call
FreeAndNil()
I end up getting an access violation, EX:
var
foo: IFoo;
begin
foo := TFoo.Create();
FreeandNil(foo);
end;
Sure, all I need to do it change that foo:IFoo; to foo:TFoo; and it is happy. OR simply set the pointer to nil, NOT call freeandNil.
foo := nil;
So, on one level, I don't understand in the least where the AV is.
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way, but I can't seem to write a method that can deal with something that is a Class or an interface. Well, that is not true, I do have something, but it is so ugly I hesitate to post it.
But I guess I should also be asking, what is everyone else doing? Mentally keeping track of what is an interface and just nil those pointers? otherwise calling FreeAndNil?
I am going to want to implement things the first time around as a concrete class, but later come back and change that to an interface when I find some way that the code can do from 2 different ways. And I am not going to want to go through the code and change how it was dealing with that reference, that is the last thing on my mind at that point.
But for the sake of discussion, the best (almost only) idea I have is this class:
interface
type
TMemory = class(TObject)
class procedure Free(item: TObject); overload; static;
class procedure Free<T: IInterface>(item: T); overload; static;
end;
implementation
uses
System.SysUtils;
{ TMemory }
class procedure TMemory.Free(item: TObject);
begin
FreeandNil(item);
end;
class procedure TMemory.Free<T>(item: T);
begin
//don't do anything, it is up the caller to always nil after calling.
end;
Then I can consistently call:
TMemory.Free(Thing);
Thing := nil;
Test code:
procedure TDoSomething.MyWorker;
var
foo: IFoo;
fooAsClass: TFoo;
JustAnObject: TObject;
begin
foo := TFoo.Create();
fooAsClass := TFoo.Create();
JustAnObject := TObject.Create();
TMemory.Free(foo);
foo := nil;
TMemory.Free(fooAsClass);
fooAsClass := nil;
TMemory.Free(JustAnObject);
JustAnObject := nil;
end;
runs with no leaks or access violations. (using MadExcept)
But a big thank you to the Delphi community on SO. You guys have been the best thing out there for learning!
The reason for the access violation is that FreeAndNil takes an untyped parameter, but expects it to be an object. So the method operates on the object.
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj); //Obj must be a TObject otherwise all bets are off
Pointer(Obj) := nil; //Will throw an AV if memory violation is detected
Temp.Free; //Will throw an AV if memory violation is detected
end;
A memory violation in the above might (NB not guaranteed) be detected if you destroy an object that has either been previously destroyed or never created. It's also likely to be detected if Obj doesn't reference an object at all but something else (such as an interface, record, Integer because these don't implement Free and if they did, it wouldn't be located in the same way as TObject.Free).
On a differently level, I want to write the code such that it does not need to know if it is an interface or an object. I want to be able to write all of my memory management the same exact way.
This is like saying you want to use your car in exactly the same way that you use your shower.
Ok, maybe the difference is not quite that extreme. But the point is that interfaces and objects (and for that matter records) use different memory management paradigms. You cannot manage their memory in the same way.
Objects need to be explicitly destroyed. You can use an ownership model, but destruction is still an explicit external action.
Interfaces are reference counted. The compiler injects code to track the number of fields and variables referencing (looking at) the underlying instance. Typically the object destroys itself when the last reference is released. (There are ways beyond the scope of this answer to change this.)
If we access some object by interface variable, it doesn't always mean that object is destroyed the moment reference counter drops to zero. For example, TComponent methods _AddRef and _Release implementations are 'dummy': no reference counting is implemented and TComponent is never destroyed because interface variables are out of scope.
To behave as we expect from 'real' interfaces, all your objects should be descendants from TInterfacedObject or you need to implement _AddRef / _Release yourself.
Yes, there are 2 different approaches to memory management which usually co-exist in a program, but confusion (and AV) arises only when the same object is treated in both ways. If we destroyed object and only then the interface variables have gone out of scope, they call _Release method of destroyed object which causes access violation. That's some risky business, though with some attention it is doable.
Classic Delphi components are not reference-counted, the concept of ownership is used instead. Each component has an owner whose responsibility is to free all the memory when it itself is destroyed. So each component has an owner, but it may also have a lot of pointers to another components, like when Toolbar has ImageList variable. If such components were refcounted, they would never be destroyed because of circular reference, so in order to break this circle you'd need 'weak' references as well which don't 'count'. They are here, too, but that's very recent feature of Delphi.
If there is some hierarchy in your objects, so you know that 'bigger' objects need all of 'smaller' ones to function, then use this good old approach, it's pretty simple and has very good implementation in Delphi, which is: you can make a code which will be leak-free no matter where exception could arise. There are all these little things like using .Free instead of .Destroy, because if exception happened in constructor, destructor is called automatically, and so on. Very clever solution in fact.
I'd use refcounted interfaces only if you don't know for how long some object is needed for you and there is no suitable 'owner' for it. I did it with scanned image which I saved to file in one thread, while converting to smaller image to show on screen on another thread. When all is done, image is no more needed in RAM and can be destroyed, but I have no idea which happens first. In this case using refcounting is best thing to do.
Delphi doesn't have a garbage collector, so coming from a Java background this is a real pain in the neck.
Usually, to destroy some memory I won't use anymore I'd use:
if (SomeMemory <> nil) then
SomeMemory.Free
What would happen if I don't check for nil before deleting?
Additionally, why someone would want to deal with all this "garbage collecting" by himself? Why among all the compiling options Delphi has, there is no Garbage Collector = true
The code for TObject.Free looks like this :
procedure TObject.Free;
begin
if Self <> nil then
Destroy;
end;
So there is no need to check for nil. You can still get into trouble if you try to free an uninitialized variable, however, as this will result in an AV. Checking for nil (or Assigned) obviously doesn't help you there either. The compiler will warn you if you try to do this, however.
To answer your second question
Why among all the compiling options Delphi has, there is no Garbage Collector = true
The simple answer is that Delphi does not have such a garbage collector. Certain managed types (like strings, dynamic arrays, etc) implement compiler-managed automatic reference counting, and those objects are automatically freed when their reference counts fall to zero. For all other unmanaged objects it is the responsibility of the developer to appropriately clean up object instances when they are no longer needed.
It's not a question of wanting to have to manage your application's memory, it's just a matter of course that you need to.
Checking for nil before Free is redundant. If the reference is nil, then it's already safe to call Free on it. If the reference isn't nil, then the safety of calling Free on it is unchanged and depends entirely on whether the variable contains a valid reference.
For example:
SomeMemory := nil;
SomeMemory.Free; // This is safe.
SomeMemory := TObject.Create;
SomeMemory.Free; // This is safe
Assert(SomeMemory <> nil);
SomeMemory.Free; // This is an error (EInvalidOperation)
In the first block, we don't check whether the variable is null, and yet the call to Free is perfectly safe. It does nothing. In the second block, we see that the variable remains non-null, but calling Free on it yields an exception.
There is a pretty straight-forward way in Delphi to have (something like) garbage collection: use interface pointers. By using variables of a type ultimately derived from IInterface (or IUnknown which is basically the same thing), Delphi will keep close count of the references, and free/destroy/release the instance when the last reference is removed. Using interface pointers, apart from instance creation and a few other things, is almost identical to using object references.
In Nick Hodges' recent work Coding In Delphi, there's quite a bit on this programming technique, and its relations to abstraction, generics, unit testing and dependency injection.
Also new versions of Delphi for other platforms will have ARC (automatic reference counting) on object references as well, operating like interfaces do now. (Obsoleting the Free method, but that's another story.)
You can always consult the documentation in order to answer such a question. Here it is, with my emphasis:
Use Free to destroy an object. Free automatically calls the destructor if the object reference is not nil. Any object instantiated at run time that does not have an owner should be destroyed by a call to Free, so that it can be properly disposed of and its memory released. Unlike Destroy, Free is successful even if the object is nil; if the object was never initialized, Free would not result in an error.
Find a more complete discussion of the issue here: Why should I not use "if Assigned()" before using or freeing things?
I have read A case against FreeAndNil but still don't understand why I cannot use this method in a class destructor ? Can anyone explain.
Update: I think the comment from Eric Grange was most useful for me. The link show that this is not obvious how to deal with it and it is mainly a matter of taste. Also the method FreeAndInvalidate was useful.
The problem with that is that many
seem to use FreeAndNil as some magic
bullet that will slay that mysterious
crash dragon. If using FreeAndNil() in
the destructor seems to solve a crash
or other memory corruption problems,
then you should be digging deeper into
the real cause. When I see this, the
first question I ask is, why is the
instance field being accessed after
that instance was destroyed? That
typically points to a design problem.
It argues that it hides the real problem you have. It must mean your code is accessing properties/fields/methods of an object that is already destroyed (the destructor is called). So instead of hiding the real problem with FreeAndNil you should really be solving the underlying problem.
This code below would not crash if you would use FreeAndNil PropertyA in the destructor of SomeObject. But it hides the real problem that SomeObject is used after it is destroyed. It is better to solve this design problem (accessing destroyed objects) instead of hiding it.
SomeObject.Free; // Destructor called
if Assigned(SomeObject.PropertyA) then // SomeObject is destroyed, but still used
SomeObject.PropertyA.Method1;
EDIT
On the other case, one could argue that if FreeAndNil is not used, the code would not crash either. Since even though the object is destroyed, the memory might not be reused and all structures might be in tact. The code above might even run without problems if Free is used to destroy PropertyA instead of FreeAndNil.
And if FreeAndNil was used to destroy SomeObject, you would also see the real problem no matter what the code in the destructor is.
So although I agree with the argument that it could hide the real design flaw and personally do not use FreeAndNil in destructors, it is not some magic bullet to discover such design flaws.
The issue is fairly easy to explain, and the contention around this issue is more subjective than objective. The use of FreeAndNil is simply unnecessary if the variable reference to the object being freed will go out of scope:
procedure Test;
var
LObj: TObject;
begin
LObj := TObject.Create;
try
{...Do work...}
finally
//The LObj variable is going out of scope here,
// so don't bother nilling it. No other code can access LObj.
//FreeAndNil(LObj);
LObj.Free;
end;
end;
In the above code snippet, nilling the LObj variable would be pointless, for the reason given. However, if an object variable can be instantiated and freed several times during the lifetime of an app, then it becomes necessary to check whether the object is indeed instantiated or not. The easy way to check this is whether the object reference has been set to nil. In order to facilitate that setting to nil, the FreeAndNil() method will both free the resources, and set nil for you. Then, in code you can check to see whether the object is instantiated with either LObj = nil or Assigned(LObj).
The case of whether to use .Free or FreeAndNil() in object destructors is a grey area, but for the most part, .Free should be safe, and nilling the references to sub-objects in the destructor should be unnecessary. There are various arguments around how to deal with exceptions in constructors and destructors.
Now pay attention: if you prefer to pick and choose whether to use .Free or FreeAndNil() depending on the specific circumstances outlined above, that's fine, but note that the cost of a bug due to not nilling a freed object reference that is subsequently accessed can be very high. If the pointer is subsequently accessed (object freed but reference not set to nil), it can happen that you are unlucky and the detection of memory corruption occurs many lines of code away from the access to the freed-but-unnilled object reference. This kind of bug can take a very long time to fix, and yes, I know how to use FastMM.
Therefore for some people, including me, it has become habit (a lazy one, perhaps) to simply nil all object pointers when they're freed, even when the nilling is not strictly necessary.
I tended to use FreeAndNil fairly often (for whatever reason) but not anymore. What made me stop doing this is not related to whether the variable needs to be nil afterwards or not. It is related to code changes, especially type changes of variables.
I got bitten several times after changing the type of a variable from TSomething to an interface type ISomething. FreeAndNil doesn't complain and happily continues doing its job on the interface variable. This sometimes lead to mysterious crashes which could not be immediately followed back to the place where it happened and took some time to find.
So I switched back to calling Free. And when I deem it necessary I set the variable to nil afterwards explicitly.
i hunted down a stackoverflow question talking about FreeAndNil and FreeAndInvalidate to mention that the Microsoft Security Development Lifecycle now recommends something similar to FreeAndInvalidate:
In light of the SDL recommendation above – and a number of real bugs related to reuse of stale references to deleted C++ objects...
The obvious choice of sanitization value is NULL. However there are downsides to that: we know that a large number of application crashes are due to NULL pointer dereferences. Choosing NULL as a sanitization value would mean that new crashes introduced by this feature may be less likely to stand out to a developer as needing a proper solution – i.e. proper management of C++ object lifetimes – rather than just a NULL check that suppresses the immediate symptom.
Also checks for NULL are a common code construct meaning that an existing check for NULL combined with using NULL as a sanitization value could fortuitously hide a genuine memory safety issue whose root cause really does needs addressing.
For this reason we have chosen 0x8123 as a sanitization value – from an operating system perspective this is in the same memory page as the zero address (NULL), but an access violation at 0x8123 will better stand out to the developer as needing more detailed attention.
So basically:
procedure FreeAndInvalidate(var obj);
var
temp : TObject;
const
INVALID_ADDRESS = $8123; //address on same page as zero address (nil)
begin
//Note: Code is public domain. No attribution required.
temp := TObject(obj);
Pointer(obj) := Pointer(INVALID_ADDRESS);
temp.Free;
end;
Is there a way to be sure we hold a useable reference to an object i.e. being sure it has not been already freed leaving that non nil reference dangling.
If you're using FastMM4 as your Memory Manager, you can check that the class is not TFreeObject.
Or, in a more standard case, use a routine that will verify that your object is what it says it is by checking the class VMT.
There have been such ValidateObj functions hannging around for some time (by Ray Lischner and Hallvard Vassbotn: http://hallvards.blogspot.com/2004/06/hack-6checking-for-valid-object.html)
Here's another:
function ValidateObj(Obj: TObject): Pointer;
// see { Virtual method table entries } in System.pas
begin
Result := Obj;
if Assigned(Result) then
try
if Pointer(PPointer(Obj)^) <> Pointer(Pointer(Cardinal(PPointer(Obj)^) + Cardinal(vmtSelfPtr))^) then
// object not valid anymore
Result := nil;
except
Result := nil;
end;
end;
Update: A bit of caution... The above function will ensure that the result is either nil or a valid non nil Object. It does not guarantee that the Obj is still what you think it is, in case where the Memory Manager has already reallocated that previously freed memory.
No. Unless you use something like reference counting or a garbage collector to make sure no object will be freeed before they have zero references.
Delphi can do reference counting for you if you use interfaces. Of course Delphi for .Net has a gargage collector.
As mentioned you could use the knowledege of Delphi or the memory manager internals to check for valid pointers or objects, but they are not the only ones that can give you pointers. So you can't cover all pointers even with those methods. And there also is a chance that your pointer happens to be valid again, but given to somebody else. So it is not the pointer you are looking for. Your design should not rely on them. Use a tool to detect any reference bugs you make.
Standard, no...
That's why VCL components can register themselves to be notified of the destruction of an object, so that they can remove the reference from there internal list of components or just reset their property.
So if you'd want to make sure you haven't got any invalid references their are two options:
Implement a destruction notification handler which every class can subscribe to.
Fix your code in a way that the references aren't spread around trough different object. You could for instance only provide the access to the reference via a property of another object. And instead of copying the reference to a private field you access the property of the other object.
As others have said, no definitive way, but if you manage the ownership well, then the FreeAndNil routine will ensure that your variable is nil if it doesn't point to anything.
It's usually not a good idea to check a reference is valid anyway. If a reference is not valid, your program will crash at the place where it is using the invalid reference. Otherwise the invalid reference might survive longer and debugging becomes harder.
Here are some references to why it's better to crash on an invalid reference. (They talk about pointers in Win32, but the ideas are still relevant):
IsBadXxxPtr should really be called CrashProgramRandomly
Should I check the parameters to my function?
Unfortunately there is no way to 100% guarantee that a pointer to anything is still valid, except by meticolously writing the correct code.
With the usage of interface references (instead of object references) it is possible to avoid these invalid pointer problems because there is no explicit call to Free in your code anymore.