In Delphi 10.3, I had written some Excel automation code. I used variants. When the routine was finished, I cleared and freeAndNil the Variant...
VarClear(arrData);
FreeAndNil(arrData);
This compiled and ran fine. I have just upgraded to D11, i.e. Alexandria. This code now gives an error. ... Incompatible Type: TObject and Variant.
I rewrote this portion of code to be:
VarClear(arrData);
arrData.Free;
This compiles, and at first glance, seems to run fine. Is this the proper way to clear/dispose of variants in Alexandria?
The signature of FreeAndNil() was indeed changed, in Delphi 10.4, to be exact:
https://blog.marcocantu.com/blog/2020-may-delphi-104-rtl.html
We updated the signature of the FreeAndNil procedure, to avoid its use with interface references and other unsupported data types. It is now declared to require a reference to a TObject:
procedure FreeAndNil(const [ref] Obj: TObject); inline;
This means that incorrect usage of FreeAndNil will now cause a compiler error. In the past, incorrect usage would not be caught, leading to difficult bugs. Note that although the parameter is declared as const, the by-reference variable is indeed modified.
It is wrong to try to Free() a Variant in the first place. That has never worked, and should never have been in your code in the first place. Under the old definition, FreeAndNil() would have called TObject.Free() on the Variant, which is wrong. Under your new code, arrData.Free; will compile, but will invoke method dispatching at runtime to call a non-existent Free() method on the Excel object, which will fail.
So, just remove the Free altogether, it doesn't belong here. The correct solution, in all Delphi versions, to manually reset a Variant to a default state is to assign Variants.Null or Variants.Unassigned to it, eg:
arrData := Null;
arrData := Unassigned;
Both of these code snippets are wrong. You don't call Free on a variant. You call Free on an object instance. Simply remove that line.
Related
I am having some difficulty understanding typecasting when using a class that is a passed parameter. I tried searching for this but couldn't find other answers.
I am working with some legacy Delphi code, using Delphi 2006, which doesn't support Generics (introduced in Delphi 2009).
The code is using TLists to store pointers to instantiated classes of particular types. When clearing the list, they use this:
procedure ClearList(AList: TList);
var i: Integer;
begin
for i := 0 to AList.Count - 1 do
TObject(AList[i]).Free;
AList.Clear;
end;
And it is called like this:
ClearList(FExtraVisitTypes);
ClearList(FDiagnoses);
ClearList(FProcedures);
ClearList(FImmunizations);
ClearList(FSkinTests);
ClearList(FPatientEds);
ClearList(FHealthFactors);
ClearList(FExams);
My understanding of this may be off, but I am concerned that if the pointed-to objects are freed as TObject, that the destructor of the descendant object won't be called, potentially leading to a memory leak. (My polymorphisim kung-fu is a bit rusty, which may be causing my confusion.)
So I tried to change the clear function as below:
procedure ClearList(AList: TList; ItemClass: TPCEItemClass); //mod to add ItemClass
var i: Integer;
begin
for i := 0 to AList.Count - 1 do begin
(AList[i] as ItemClass).Free;
end;
AList.Clear;
end;
TPCEItemClass is defined like this:
TPCEItemClass = class of TPCEItem;
I then changed the clear calls like this:
ClearList(FExtraVisitTypes, TPCEProc);
ClearList(FDiagnoses, TPCEDiag);
ClearList(FProcedures, TPCEProc);
ClearList(FImmunizations, TPCEImm);
ClearList(FSkinTests, TPCESkin);
ClearList(FPatientEds, TPCEPat);
ClearList(FHealthFactors, TPCEHealth);
ClearList(FExams, TPCEExams);
But the compiler won't allow this and gives this error:
[Pascal Error] uPCE.pas(1730): E2015 Operator not applicable to this operand type
For this erroneous line:
(AList[i] as ItemClass).Free;
Questions:
Does the original way of coding, where the item is freed by simply calling the great-great-great (etc) ancestor Free method end up effecting the descendant's destructor method? As I write this, I'm now thinking that it actually does. But I don't know why. So any answers to help me keep this in my head would be great.
Why does my method of trying to typecast via the parameter which is of type class not work? Is this just not allowed? Or is my syntax wrong? Is there another way to do this?
Am I going about this all wrong? Is there a better way?
Thanks
I am concerned that if the pointed-to objects are freed as TObject, that the destructor of the descendant object won't be called, potentially leading to a memory leak.
That is not the case for classes that are properly implemented.
All classes derive from TObject. TObject.Free() calls the TObject.Destroy() destructor, which is virtual. Any descendant that requires destruction logic must override that destructor (if it doesn't, it has a flaw that needs fixing).
So, in properly written code, the original code will work perfectly fine as shown. Calling Free() on any valid and correctly implemented object will invoke its most-derived destructor.
Now, that being said, there have been plenty of cases over the years of people forgetting to override the destructor when their classes require it, thus causing the kinds of memory leaks you are worried about. So, make sure you pay attention to what your classes are doing, and you will be fine.
So I tried to change the clear function as below ... But the compiler won't allow this and gives this error
Correct, because you can't perform a type-cast on an object using a variable to a metaclass type, like you are trying to do. Type-casts require the target type to be specified at compile-time, but metaclass variables are not assigned until runtime.
Does the original way of coding, where the item is freed by simply calling the great-great-great (etc) ancestor Free method end up effecting the descendant's destructor method?
The original code will work just fine 99% of the time, yes. Most Delphi coders are good about override'ing the destructor when it is appropriate. But that other 1% is only when you are dealing with classes that are not implemented correctly, in which case it is their author's responsibility to fix them, not your responsibility to fix the code that is calling Free() on them.
As I write this, I'm now thinking that it actually does. But I don't know why.
Polymorphic dispatch of the virtual destructor, just like when calling any other virtual method.
Why does my method of trying to typecast via the parameter which is of type class not work? Is this just not allowed?
Correct. It is illegal.
Is there another way to do this?
No (well, yes, but it involves walking an object's class structure's manually at runtime, but that requires a deep understanding of how the compiler lays out objects in memory, so I'm not going to get into that here).
NOTE: Bear with me, I feel a little "flame grilled" due to some discussions over here and here and some issues I reported here and here.
Some background
Ye olde (pre 10.4) FreeAndNil looked like this:
FreeAndNil(var SomeObject)
The new and fresh FreeAndNil looks like this:
FreeAndNil(const [ref] SomeObject: TObject);
IMO both have their downsides:
The old one doesn't do any type checking, so calling FreeAndNil on pointers, records and interfaces compiles just fine, but produces interesting but usually unwanted effects during runtime. (Goes completely berserk or if you are lucky it halts with EAccessViolation, EInvalidOperation etc.)
The new one accepts a const parameter, and therefore any object. But then the provided object pointer is actually changed using some hacky-wacky code.
You can now call the new FreeAndNil like this: FreeAndNil(TObject.Create) and it will compile and even run just fine. I liked the old FreeAndNil that warned me when I went wrong and provided e.g. a property instead of a field. Unsure what happens if you provide a object type property to this FreeAndNil implementation. Didn't try.
If we would change the signature into FreeAndNil(var SomeObject:TObject) then it will not allow us to pass any other variable type then exactly the TObject type. Which also makes sense, as if it weren't FreeAndNil, one could easily change a variable provided as type TComponent in the routine change the var variable into an object of a completely different type, e.g. TCollection. Of course FreeAndNil will do no such thing, as it always changes the var parameter to nil.
So this makes FreeAndNil a special case.
Maybe even special enough to convince delphi to add a compiler magic FreeAndNil implementation? Votes anyone?
Potential work-around
I came up with the code below as an alternative (here as a helper method, but could as well be part of TObject implementation) which kind-a combines both worlds. The Assert will help finding invalid calls during runtime.
procedure TSGObjectHelper.FreeAndNilObj(var aObject);
begin
if Assigned(self) then
begin
Assert(TObject(aObject)=self,ClassName+'.FreeAndNil Wrong parameter provided!');
pointer(aObject):=nil;
Destroy;
end;
end;
Usage would be something like this:
var MyObj:=TSOmeObject.Create;
...
MyObj.FreeAndNilObj(MyObj);
I have actually tested this routine, and it even is slightly faster than the 10.4 FreeAndNil implementation. I guess because I do the assignment check first and call Destroy directly.
What I do not like so much is that:
the type checking takes place during runtime, and then only if Assertions are ON.
it feels like having to pass the same variable twice. Which isn't necessarily true/required. It has to be the same object, and the parameter has to be a variable.
Another investigation
But wouldn't it be great if one could call without the parameter
var MyObj:=TSomeObject.Create;
...
MyObj.FreeAndNil;
So I messed around with the self pointer and managed to set it to nil using the same Hacky-Wacky code that 10.4 utilizes in their FreeAndNil. Well... that worked inside the method, self pointed to nil. But after calling FreeAndNil like this, the MyObj variable wasn't nil, but a stale pointer. (This was what I expected.) Moreover, MyObj could be a property or (the result of) a routine, constructor etc.
so nope over here as well...
And finally the question:
Can you think of a cleaner/better solution or trick that would:
FreeAndNil(var aObject:TObject) with not-so-strict type checking compile time (maybe a Compiler directive?) so it allows compiling and calling for variables of any object type.
Complains compile time when something is passed that is not a variable/field of some object type
Help describing what is the best solution/requirement in RSP-29716
The only proper solution to FreeAndNil that is both type safe and does not allow freeing function results and properties would be generic var parameter:
procedure FreeAndNil<T: class>(var Obj: T); inline;
But, currently Delphi compiler does not allow generics on standalone procedures and functions https://quality.embarcadero.com/browse/RSP-13724
Still, that does not mean you cannot have generic FreeAndNil implementation, only that it will be a bit more verbose than necessary.
type
TObj = class
public
class procedure FreeAndNil<T: class>(var Obj: T); static; inline;
end;
class procedure TObj.FreeAndNil<T>(var Obj: T);
var
Temp: TObject;
begin
Temp := Obj;
Obj := nil;
Temp.Free;
end;
Type inference introduced in Rio will allow you to call it without specifying generic signature:
TObj.FreeAndNil(Obj);
Calling (and using) generic FreeAndNil in older Delphi versions is also possible but even more verbose
TObj.FreeAndNil<TFoo>(Obj);
Because we cannot create a global procedure FreeAndNil<T:class>(var aObject:T) I would suggest the code below as a method to the TObject class. (rtl change to be made by embarcadero, but does not need a compiler change)
class procedure TObject.InternalFreeAndNil(var Object:TObject); static; // strict private class method
begin
if Assigned(Object) then
begin
var tmp:=Object;
Object:=nil;
tmp.Destroy;
end;
end;
class procedure TObject.FreeAndNil<T:class>(var Object:T); inline; // public generic class method
begin
InternalFreeAndNil(TObject(Object));
end;
and to have the current (10.4 and earlier) FreeAndNil removed from the sysutils unit to avoid ambiguity.
When the new generic FreeAndNil method is called from within any other method, one can simply call:
FreeAndNil(SomeObjectVariable)
and 10.3+ type inference avoids having to write:
FreeAndNil<TMyClassSpec>(SomeObjectVariable)
which is nice because most of your code will compile nicely without a change.
In some other spots, eg global routines and initialization / finalization sections one would have to call:
TObject.FreeAndNil(SomeObjectVariable)
Which to me would be acceptable, and a lot better than the current and historical half-way solutions with a FreeAndNil(const [ref] aObject:TObject) or an untyped FreeAndNil(var aObject)
And since the routine is so utterly simple and performance appears to be an issue, one could argue to have an assembler implementation for it. Though I am not sure if this is allowed/possible for generic, (and preferably inline) methods.
FTM: One could also just keep FreeAndNil(var aObject:TObject) and tell people to do a typecast like below, which also avoids the compiler complaining about the var type. But in this case, probably a lot of source code has to be adjusted. On the other hand it saves on code bloat, still avoids Invalid use of function results, properties or invalid types like records and pointers as parameter to FreeAndNil, and is utterly simple to change/implement.
...
var Obj:=TSomeObject.Create;
try
DoSOmethingUseFulWithObj(Obj);
finally
FreeAndNil(TObject(Obj)); // typecast avoids compiler complaining. Compiler wont allow invalid typecasts
end;
...
The following code should compile and does compile with many other types.
However, the compiler reports a "Constant object cannot be passed as var parameter" error - despite the variable quite obviously being a variable.
program CurrencyConstant;
{$APPTYPE CONSOLE}
var
GVar: Currency;
begin
FillChar(GVar, SizeOf(GVar), 0);
end.
Similarly, the same problem occurs with a local variable in a procedure.
procedure TestCurrency;
var
LVar: Currency;
begin
FillChar(LVar, SizeOf(LVar), 0);
end;
I suspect it has something to do with the fact that FillChar is a compiler magic procedure, and that Dest is an untyped var parameter. FillChar is the only routine I've found with this problem.
What causes this problem?
Are any other types affected?
In response to the inevitable "Why would you do that comments": We have a code generator that uses FillChar to generically initialise record structures & primitive types. It works with everything else, but unexpectedly failed with Currency. We do have workarounds, but it would be nice to understand the root cause, and know whether anything else is likely to cause us trouble.
Edit
From Jeroen's answer it is reasonable to conclude that the issue exists in all vesions of Delphi. Furthermore array's of Currency apparently exhibit a similar problem.
David's answer provides some nice workarounds.
One final workaround to consider is, modifying the generator to deal with Currency as a special case and simply set the Value := 0.
What causes the problem?
A compiler bug. Please submit a QC report.
Are any other types affected?
Maybe. Try some to find out.
As for a work around I would write it like this:
FillChar(Pointer(#LVar)^, SizeOf(LVar), 0);
or perhaps like this:
ZeroMemory(#LVar, SizeOf(LVar));
or even like this:
LVar := Default(Currency);
Personally I regard ZeroMemory as being more descriptive than FillChar.
As requested by Craig Young:
It still occurs in Delphi XE4.
Report No: 118866 Status: Reported
Cannot perform FillChar on Currency variables
https://web.archive.org/web/20150322021442/http://qc.embarcadero.com/wc/qcmain.aspx?d=118866
It is similar to
http://qc.embarcadero.com/wc/qcmain.aspx?d=87168 (not archived)
The workaround for this compiler bug for Delphi < 2009: use ZeroMemory or FillMemory from the Windows unit which works just as well as FillChar.
On the Delphi side, ZeroMemory and FillMemory use FillChar underneath which might be inlined as of Delphi 2006.
On the C++ side both use compiler macros.
It might be that this issue only happens with Currency because that is the only numeric compiler type that is scaled.
The issue does not reproduce with ordinal types, regular floating point types, and Comp.
Edit: The issue has been fixed in XE5 Update 2
I have discovered (the hard way) that if a file has a valid UTF-8 BOM but contains any invalid UTF8 encodings, and is read by any of the Delphi (2009+) encoding-enabled methods such as LoadFromFile, then the result is a completely empty file with no error indication. In several of my applications, I would prefer to simply lose a few bad encodings, even if I get no error report in this case either.
Debugging reveals that MultiByteToWideChar is called twice, first to get the output buffer size, then to do the conversion. But TEncoding.UTF8 contains a private FMBToWCharFlags value for these calls, and this is initialized with a MB_ERR_INVALID_CHARS value. So the call to get the charcount returns 0 and the loaded file is completely empty. Calling this API without the flag would 'silently drop illegal code points'.
My question is how best to weave through the nest of classes in the Encoding area to work around the fact that this is a private value (and needs to be, because it is a class var for all threads). I think I could add a custom UTF8 encoding, using the guidance in Marco Cantu's Delphi 2009 book. And it could optionally raise an exception if MultiByteToWideChar has returned an encoding error, after calling it again without the flag. But that does not solve the problem of how to get my custom encoding used instead of Tencoding.UTF8.
If I could just set this up as a default for the application at initialization, perhaps by actually modifying the class var for Tencoding.UFT8, this would probably be sufficient.
Of course, I need a solution without waiting to lodge a QC report asking for a more robust design, getting it accepted, and seeing it changed.
Any ideas would be very welcome. And can someone confirm this is still an issue for XE4, which I have not yet installed?
I ran into the MB_ERR_INVALID_CHARS issue when I first updated Indy to support TEncoding, and ended up implementing a custom TEncoding-derived class for UTF-8 handling to avoid specifying MB_ERR_INVALID_CHARS. I didn't think to use a class helper.
However, this issue is not just limited to UTF-8. Any decoding failure of any of the TEncoding classes will result in a blank result, not an exception being raised. Why Embarcadero chose that route, when most of the RTL/VCL uses exceptions instead, is beyond me. Not raising an exception on error caused a fair amount of issues in Indy that had to be worked around.
This can be done pretty simply, at least in Delphi XE5 (have not checked earlier versions). Just instantiate your own TUTF8Encoding:
procedure LoadInvalidUTF8File(const Filename: string);
var
FEncoding: TUTF8Encoding;
begin
FEncoding := TUTF8Encoding.Create(CP_UTF8, 0, 0);
// Instead of CP_UTF8, MB_ERR_INVALID_CHARS, 0
try
with TStringList.Create do
try
LoadFromFile(Filename, FEncoding);
// ...
finally
Free;
end;
finally
FEncoding.Free;
end;
end;
The only issue here is that the IsSingleByte property for the newly instantiated TUTF8Encoding is then incorrectly set to False, but this property is not currently used anywhere in the Delphi sources.
A partial workaround is to force the UTF8 encoding to suppress MB_ERR_INVALID_CHARS globally. For me, this avoids the need for raising an exception, because I find it makes MultiByteToWideChar not quite 'silent': it actually inserts $fffd characters (Unicode 'replacement character') which I can then find in the cases where this is important. The following code does this:
unit fixutf8;
interface
uses System.Sysutils;
type
TUTF8fixer = class helper for Tmbcsencoding
public
procedure setflag0;
end;
implementation
procedure TUTF8fixer.setflag0;
{$if CompilerVersion = 31}
asm
XOR ECX,ECX
MOV Self.FMBToWCharFlags,ECX
end;
{$else}
begin
Self.FMBToWCharFlags := 0;
end;
{$endif}
procedure initencoding;
begin
(Tencoding.UTF8 as TmbcsEncoding).setflag0;
end;
initialization
initencoding;
end.
A more useful and principled fix would require changing the calls to MultiByteToWideChar not to use MB_ERR_INVALID_CHARS, and to make an initial call with this flag so that an exception could be raised after the load is complete, to indicate that characters will have been replaced.
There are relevant QC reports on this issue, including 76571, 79042 and 111980. The first one has been resolved 'as designed'.
(Edited to work with Delphi Berlin)
Your "global" approach is not really global - it relies upon the assumption that all the code would only use one and the same instance of TUTF8Encoding. The same instance where you hacked the flags field.
But it would not work if one obtain TUTF8Encoding object(s) by other means than TEncoding.GetUTF8, for example in XE2 another method - TEncoding.GetEncoding(CP_UTF8) - would create a new instance of TUTF8Encoding instead of re-using FUTF8 shared one. Or some function might run TUTF8Encode.Create directly.
So i'd suggest two more approaches.
Approach with patching the class implementation, somewhat hacky. You introduce your own class for the sake of obtaining new "fixes" constructor body.
type TMyUTF8Encoding = class(TUTF8Encoding)
public constructor Create; override;
end;
This constructor would be the copycat of TUTF8Encoding.Create() implementation, except for setting the flag as you want it ( in XE2 it is done by calling another, inherited Create(x,y,z) so u would not need an access to the private field ) instead.
Then you can patch the stock TUTF8Encoding VMT overriding its virtual constructor to that new constructor of yours.
You may read Delphi documentation about "internal formats" and so forth, to get the VMT layout. You would also need calling VirtualProtect (or other platform-specific function) to remove protection from VMT memory area before patching and then to restore it.
Examples to learn from
How to change the implementation (detour) of an externally declared function
https://stackoverflow.com/a/1482802/976391
Or you may try using Delphi Detours library, hopefully it can patch virtual constructors. Then... it might be an overkill here to use that rather complex lib for that single goal.
After you hacked the TUTF8Encoding class do call the TEncoding.FreeEncodings to remove the already created shared instances (if any) if any and thus trigger recreating the UTF8 instances with your modifications.
Then, if you compile your program as a single monolithic EXE , without using runtime BPL modules, you just can copy the SysUtils.pas sources to your application folder and then to include that local copy into your project explicitly.
How to patch a method in Classes.pas
There you would change the very TUTF8Encoding implementation as you see fit in the sources and Delphi would use it.
This brain-deadly simplistic (hence - equally reliable) approach would not work though if your projects would be built to reuse rtlNNN.bpl runtime package instead of being monolithic.
I am looking for hints on how to debugging a crash in an application that uses the MS XML wrappers in the Delphi VCL. I suspect memory corruption, or some kind of obscure evil thing happening between objects and interfaces, such as reference counting bugs, or heap corruption. The question is, in effect: how do I debug such a crash?
This particular code makes heavy internal use of and extends on the base XmlIntf Interfaces (IXMLNode). ISomethingCustom is an interface that extends IXMLNode. THe problem happens where we crash somewhere in a recursive function that is passed an ISomethingCustom which is also (or supports also, in interface terms) IXMLNode.
boolean UtilityFunction( aNode: ISomethingCustom ):Boolean;
begin
if not Assigned(aNode) then exit; // this works. great.
if not Assigned(aNode.ParentNode) then exit; // this DOES NOT WORK.
// code that blows up if aNode.ParentNode is not assigned.
end;
The situation is that the aNode is also IXMLNode, and IXMLNode.ParentNode value is assigned (not nil), and yet it points to a COM object that may have been freed, destroyed, or corrupted somehow. I am trying to figure out WHAT is going on when an interface pointer can appear to be valid, but the object behind it has been nuked somehow.
Checking Assigned(aNode.ParentNode) returns TRUE, even when, if you were to attempt a cast in the debugger (at runtime only, not in the code), like this:
inspect/evaluate aNode
inspect/evaluate TInterfacedObject(aNode).ClassName
(works in Delphi 2010, at least!)
now cast TWhateverClassNameYouGotBefore(aNode).
In the debugger I now see that this is NIL. WHich may mean that
the magic "casting interface back to
the object" feature that is new in
delphi 2010, is failing.
I believe I am trying to debug a problem where heaps are corrupted, or COM objects are corrupt on the heap, because of a reference counting problem.
I really think that nobody should ever have the situation arise where an interface appears valid, but the object underneath has been deleted. I really would like to know what to do, and what is going on.
Although you haven't shown it in your code, your comments seem to indicate that you're type-casting the interface variable to a class type. That's not allowed. I've described why:
Why can’t I cast an interface reference to an object reference?
Interface references and object references don't point to the same things. Therefore, calling a method on one when the compiler thinks you have the other will yield unexpected results. You were unlucky because the code continued to run instead of crashing with an access violation, which would have been a bigger indication that you were doing something wrong.
My article above concludes by suggesting you use the JclSysUtils.GetImplementorOfInterface function from the JCL if you have a Delphi-implemented interface and the interface offers no function of its own for revealling the underlying object.
Wild guess: Have you tried to put aNode.ParentNode in a local variable and use it in the rest of the Utilityfunction:
function UtilityFunction(aNode: ISomethingCustom): Boolean;
var
lParentNode: INode;
begin
if not Assigned(aNode) then exit; // this works. great.
lParentNode := aNode.ParentNode;
if not Assigned(lParentNode) then exit;
// code that uses lParentNode.
end;
My suggestion is to make sure that the ParentNode function is actually called in Assigned(aNode.ParentNode). There are some nasty corner-cases in Delphi where a procedure/function without arguments doesn't get called, but rather it's reference is taken when you omit the parenthesis's.
Try to change it to Assigned(Anode.ParentNode()) (which should have the same effect as François suggestion).