I need a way to write a generic procedure to act upon an object type or any of its descendants.
My first attempt was to declare
procedure TotalDestroy(var obj:TMyObject);
but when using it with a descendant object
type TMyNewerObject = class(TMyObject);
var someNewerObject: TMyNewerObject;
TotalDestroy(someNewerObject);
I get the infamous error "types of formal and actual parameters must be identical"
So, while strugling to find a solution, I looked at the source code of Delphi system FreeAndNil procedure. And I found this awesome declaration, along with this astonishing comment
{ FreeAndNil frees the given TObject instance and
sets the variable reference to nil.
Be careful to only pass TObjects to this routine. }
procedure FreeAndNil(var Obj);
It avoids the type checking error, but it uses no safety net.
My question is ... is there any safe way to check the type of an untyped var parameter?
or in other words, can you improve this Delphi source code so that the warning would not be needed?
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj);
Pointer(Obj) := nil;
Temp.Free;
end;
Let's examine what you want to do.
You want to call a method that takes X, passing in an object of type Y, where Y is a descendant of X. The snag, the parameter is a "var" parameter.
Let's analyze what you could do if that was possible.
type
TBase = class
end;
TDescendant = class(TBase)
end;
procedure Fiddle(var x: TBase);
begin
x := TDescendant.Create;
end;
type
TOtherDescendant = class(TBase)
end;
var a: TOtherDescendant;
a := TOtherDescendant.Create;
Fiddle(a);
Uh-oh, now a no longer contains an instance of TOtherDescendant, it contains an instance of TDescendant. That probably comes as a surprise to the code that follows the call.
You must not only consider what you intend to do with the syntax you propose, but effectively what you could do with the syntax.
You should read Eric Lipperts excellent blog post about similar issues in .NET, found here: Why do ref and out parameters not allow type variation?.
I've written about this before, using an example very similar to Lasse's:
Delphi Q&A: Why must the types of actual and formal var parameters be identical?
Unless you're writing an assignment statement to change the value of the input parameter itself, and not just one of its properties, you shouldn't pass a parameter by reference in the first place.
If you are writing an assignment statement to change the parameter's value, then the compiler message really is true, and you should heed it.
One reason for needing to by-pass the error is when you're writing a function like TApplication.CreateForm. Its job is to change the input parameter's value, and the type of the new value varies and cannot be determined at compile time. If you're writing such a function, then your only option with Delphi is to use an untyped var parameter, and then there is extra burden on both the caller and the receiver to make sure everything goes right. The caller needs to make sure it passes a variable that is capable of holding values of whatever type the function will put in it, and the function needs to make sure it stores a value of a type compatible with what the caller requested.
In the case of CreateForm, the caller passes in a class-reference literal and a variable of that class type. The function instantiates the class and stores the reference in the variable.
I don't think very highly of either CreateForm or FreeAndNil, largely because of the way their untyped parameters sacrifice type safety in return for comparatively little extra convenience. You haven't shown the implementation of your TotalDestroy function, but I suspect its var parameter will ultimately provide the same low utility as in those other two functions. See my articles on both:
When should I use FreeAndNil?
Why shouldn't I call Application.CreateForm?
In addition to what Lasse wrote, which is quite correct, most of the time you don't want to pass an object to a var parameter anyway.
An object is a reference type. What you see as the object is actually a reference to it. You would only want to pass an object reference to a var parameter if you wanted to change your object out for a new object. If you just want to be able to modify the members of the object, then you can do that by simply passing it to a normal parameter. Make method call take a TMyObject parameter instead of a var TMyObject parameter and it should work.
Of course, if you really are replacing the object, then feel free to disregard all this, and see Lasse's answer.
can you improve this Delphi source code so that the warning would not be needed?
Yes, you can get a type safe way to avoid the compiler error.
In the newest Delphi 10.4 Sidney, the FreeAndNil procedure has been changed into this:
procedure FreeAndNil(const [ref] Obj: TObject);
var
Temp: TObject;
begin
Temp := Obj;
TObject(Pointer(#Obj)^) := nil;
Temp.Free;
end;
It is type safe for objects and will catch errors when passing an interface reference for example.
The way to pass a parameter by const [ref] means that the parameter is passed by reference. Without the [ref] attribute, parameters with size equal and smaller than a pointer would otherwise be passed by value.
Here, even though the object is passed as a constant, the reference will be modified.
In that sense, it is not a perfect declaration, but will do its job better than the former implementation.
From New features in Delphi 10.4:
This means that incorrect usage of FreeAndNil will now cause a compiler error. In the past, incorrect usage would not be caught, leading to difficult bugs. Note that although the parameter is declared as const, the by-reference variable is indeed modified.
A new, but ‘not as bad’, class of incorrect calling is possible with this declaration of FreeAndNil: the method can be called passing in properties or a method result, as well as cast expressions, a type’s implicit conversion to TObject, etc. The nil-ed value will then be the temporary variable in the expression.
Related
NOTE: Bear with me, I feel a little "flame grilled" due to some discussions over here and here and some issues I reported here and here.
Some background
Ye olde (pre 10.4) FreeAndNil looked like this:
FreeAndNil(var SomeObject)
The new and fresh FreeAndNil looks like this:
FreeAndNil(const [ref] SomeObject: TObject);
IMO both have their downsides:
The old one doesn't do any type checking, so calling FreeAndNil on pointers, records and interfaces compiles just fine, but produces interesting but usually unwanted effects during runtime. (Goes completely berserk or if you are lucky it halts with EAccessViolation, EInvalidOperation etc.)
The new one accepts a const parameter, and therefore any object. But then the provided object pointer is actually changed using some hacky-wacky code.
You can now call the new FreeAndNil like this: FreeAndNil(TObject.Create) and it will compile and even run just fine. I liked the old FreeAndNil that warned me when I went wrong and provided e.g. a property instead of a field. Unsure what happens if you provide a object type property to this FreeAndNil implementation. Didn't try.
If we would change the signature into FreeAndNil(var SomeObject:TObject) then it will not allow us to pass any other variable type then exactly the TObject type. Which also makes sense, as if it weren't FreeAndNil, one could easily change a variable provided as type TComponent in the routine change the var variable into an object of a completely different type, e.g. TCollection. Of course FreeAndNil will do no such thing, as it always changes the var parameter to nil.
So this makes FreeAndNil a special case.
Maybe even special enough to convince delphi to add a compiler magic FreeAndNil implementation? Votes anyone?
Potential work-around
I came up with the code below as an alternative (here as a helper method, but could as well be part of TObject implementation) which kind-a combines both worlds. The Assert will help finding invalid calls during runtime.
procedure TSGObjectHelper.FreeAndNilObj(var aObject);
begin
if Assigned(self) then
begin
Assert(TObject(aObject)=self,ClassName+'.FreeAndNil Wrong parameter provided!');
pointer(aObject):=nil;
Destroy;
end;
end;
Usage would be something like this:
var MyObj:=TSOmeObject.Create;
...
MyObj.FreeAndNilObj(MyObj);
I have actually tested this routine, and it even is slightly faster than the 10.4 FreeAndNil implementation. I guess because I do the assignment check first and call Destroy directly.
What I do not like so much is that:
the type checking takes place during runtime, and then only if Assertions are ON.
it feels like having to pass the same variable twice. Which isn't necessarily true/required. It has to be the same object, and the parameter has to be a variable.
Another investigation
But wouldn't it be great if one could call without the parameter
var MyObj:=TSomeObject.Create;
...
MyObj.FreeAndNil;
So I messed around with the self pointer and managed to set it to nil using the same Hacky-Wacky code that 10.4 utilizes in their FreeAndNil. Well... that worked inside the method, self pointed to nil. But after calling FreeAndNil like this, the MyObj variable wasn't nil, but a stale pointer. (This was what I expected.) Moreover, MyObj could be a property or (the result of) a routine, constructor etc.
so nope over here as well...
And finally the question:
Can you think of a cleaner/better solution or trick that would:
FreeAndNil(var aObject:TObject) with not-so-strict type checking compile time (maybe a Compiler directive?) so it allows compiling and calling for variables of any object type.
Complains compile time when something is passed that is not a variable/field of some object type
Help describing what is the best solution/requirement in RSP-29716
The only proper solution to FreeAndNil that is both type safe and does not allow freeing function results and properties would be generic var parameter:
procedure FreeAndNil<T: class>(var Obj: T); inline;
But, currently Delphi compiler does not allow generics on standalone procedures and functions https://quality.embarcadero.com/browse/RSP-13724
Still, that does not mean you cannot have generic FreeAndNil implementation, only that it will be a bit more verbose than necessary.
type
TObj = class
public
class procedure FreeAndNil<T: class>(var Obj: T); static; inline;
end;
class procedure TObj.FreeAndNil<T>(var Obj: T);
var
Temp: TObject;
begin
Temp := Obj;
Obj := nil;
Temp.Free;
end;
Type inference introduced in Rio will allow you to call it without specifying generic signature:
TObj.FreeAndNil(Obj);
Calling (and using) generic FreeAndNil in older Delphi versions is also possible but even more verbose
TObj.FreeAndNil<TFoo>(Obj);
Because we cannot create a global procedure FreeAndNil<T:class>(var aObject:T) I would suggest the code below as a method to the TObject class. (rtl change to be made by embarcadero, but does not need a compiler change)
class procedure TObject.InternalFreeAndNil(var Object:TObject); static; // strict private class method
begin
if Assigned(Object) then
begin
var tmp:=Object;
Object:=nil;
tmp.Destroy;
end;
end;
class procedure TObject.FreeAndNil<T:class>(var Object:T); inline; // public generic class method
begin
InternalFreeAndNil(TObject(Object));
end;
and to have the current (10.4 and earlier) FreeAndNil removed from the sysutils unit to avoid ambiguity.
When the new generic FreeAndNil method is called from within any other method, one can simply call:
FreeAndNil(SomeObjectVariable)
and 10.3+ type inference avoids having to write:
FreeAndNil<TMyClassSpec>(SomeObjectVariable)
which is nice because most of your code will compile nicely without a change.
In some other spots, eg global routines and initialization / finalization sections one would have to call:
TObject.FreeAndNil(SomeObjectVariable)
Which to me would be acceptable, and a lot better than the current and historical half-way solutions with a FreeAndNil(const [ref] aObject:TObject) or an untyped FreeAndNil(var aObject)
And since the routine is so utterly simple and performance appears to be an issue, one could argue to have an assembler implementation for it. Though I am not sure if this is allowed/possible for generic, (and preferably inline) methods.
FTM: One could also just keep FreeAndNil(var aObject:TObject) and tell people to do a typecast like below, which also avoids the compiler complaining about the var type. But in this case, probably a lot of source code has to be adjusted. On the other hand it saves on code bloat, still avoids Invalid use of function results, properties or invalid types like records and pointers as parameter to FreeAndNil, and is utterly simple to change/implement.
...
var Obj:=TSomeObject.Create;
try
DoSOmethingUseFulWithObj(Obj);
finally
FreeAndNil(TObject(Obj)); // typecast avoids compiler complaining. Compiler wont allow invalid typecasts
end;
...
Unlike the case with common objects, it is impossible to directly assign generics of different related types in Delphi as follows:
Possible (normal objects):
var
var_1 : TObject;
var_2 : MyTObjectSubClass;
var_1 := var_2; //Works
Not possible (generics):
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := var_2; //Does not compile
It is possible to use casts to accomplish this though, as follows:
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := TList<TObject>(var_2); //Works
This creates the need to be able to dynamically cast generics (i.e. to dynamically parameterize their generic type specification) somehow, but I have not been able to find a way to do this, so my question is: Is this in any way possible?
I'm indeed aware of the covariance/contravariance problems related to this, but in some cases it would indeed both be useful and "correct" to do such a thing.
One example of such a situation is the current code I'm writing for generic streaming of Delphi objects over a TStream, where the receiving end knows the exact type of the object that is incoming over the stream, e.g. TList<MyTObjectSubClass>. This type information is extracted by means of RTTI though (from a provided target variable to which the loaded object should be written), so I cannot explicitly mention the exact generics type in my stream-loading code in advance, but rather have to detect it by means of RTTI (which is possible, although somewhat hacky) and then write it to a target variable that I only at that run-time point will know the exact type of.
Thus, the load-object-from-stream code has to be fully generic, and thus, it would need to dynamically cast an existing TList<TObject> variable (which is defined explicitly in the code) to the exact type of TList<MyTObjectSubClass> (which I at that point have just learned about, through the use of RTTI), in order to be able to pass this object loaded from the stream to its final destination variable.
So again, is there ANY way whatsoever to accomplish this, or is it on the contrary actually completely impossible to assign to a not-in-advance-known generics collections using generic code (i.e. code that does not explicitly have some kind of "if [type of xxx is TList<TMyObject1>] then ... else if [type of xxx is TList<TMyObject2>] then ... else ..." test, containing explicit mentions of every single generics type that should be supported by it)?
PS.
The generics type of the stream-loaded object obviously already exists somewhere in the program (since it is concluded by means of RTTI on the target variable that the stream-loaded object should be written to), so I'm not asking about full run-time dynamic creation of generics types, but rather just about how to be able to dynamically pick the right one of those generics types already defined at compile-time in the program, and then cast a variable to that type.
EDIT:
By request from #RemyLebeau , here comes some more example code from my application, from its stream-loading function:
var
source_stream : TStream;
field_to_process : TRttiField;
field_type : TRttiType;
loaded_value : TValue;
temp_int : integer;
//...
//The fields of any object given to the streaming function are
//enumerated and sorted here
//...
//Then, for each field (provided in field_to_process),
//the following is done:
case field_to_process.FieldType.TypeKind of
//...
tkInteger:
begin
source_stream.ReadBufferData(temp_int);
loaded_value := TValue.From(temp_int);
end;
tkString,
tkLString,
tkWString,
tkUString:
begin
source_stream.ReadBufferData(noof_raw_bytes_in_string_data);
SetLength(raw_byte_buf, noof_raw_bytes_in_string_data + 4);
source_stream.ReadBuffer(raw_byte_buf[0], noof_raw_bytes_in_string_data);
temp_str := used_string_encoding.GetString(raw_byte_buf, 0, noof_raw_bytes_in_string_data);
loaded_value := TValue.From(temp_str);
end;
tkClass:
begin
is_generics_collection_containing_TObject_descendants := <does some hacky detection here>; //Thanks Remy :-)
if is_generics_collection_containing_TObject_descendants then
begin
<magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type>
end;
end;
//...
end;
field_to_process.SetValue(self, loaded_value);
That should hopefully give a somewhat better overview of my problem. The superfluous code for strings and integers are just for context, by showing how some simple types are handled.
For more info about the (necessarily) "hacky detection" mentioned in the code, please see this question. After doing that, I will know the exact type of the generics collection and its subitems, for example TList<TSomeTObjectDescendant>.
So, as you hopefully can see now, the question is about the <magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type> part. How can it be implemented?
NOTE: My problem is not to understand how to serialize/deserialize contents of an enumerable through a stream (which can of course be done by simply iterating over the items in the enumerable and then recursing the stream saving/loading code for each of them, where the number of items is given first of all in the stream). The problem is rather how to create generic code that will be able to recreate/populate any kind of generics collection of TObject descentants, whose type you only get to know at runtime, and then to finally get this into the object field that was originally enumerated by RTTI at the beginning of the stream-loading code. As an example, assume that the processed field has the type TList<TSomeTObjectDescendant>, and that you can easily load its subobjects from the stream using a call like function load_list_TSomeTObjectDescendant_subitems(input_stream : TStream) : array of TSomeTObjectDescendant. How could I then get these subitems into the TList<TSomeTObjectDescendant> field?
Type-casts and variable declarations are parsed at compile-time (though is and as casts are executed at runtime based on compiler-provided RTTI). The type being casted to, and the type of the variable being assigned to, must be known to the compiler. So what you are asking for is simply not possible with Generics. Not the way you have described it, anyway.
What's the difference between parameters declared with var and those declared with out? How does the compiler treat them differently (e.g., by generating different code, or by changing which diagnostics it issues)? Or do the different modifiers merely allow the programmer to document intended use of the parameters? What effect do the types of the parameters have on the matter?
A var parameter will be passed by reference, and that's it.
An out parameter is also passed by reference, but it's assumed that the input value is irrelevant. For managed types, (strings, Interfaces, etc,) the compiler will enforce this, by clearing the variable before the routine begins, equivalent to writing param := nil. For unmanaged types, the compiler implements out identically to var.
Note that the clearing of a managed parameter is performed at the call-site and so the code generated for the function does not vary with out or var parameters.
There is not much difference, for the compiler that is. See Mason's answer for that.
Semantically, there is a big difference:
var tells the programmer that the routine could work with its current value,
out tells the programmer that the routine will ignore/discard its current value.
Slightly late but just for the record, I came across a case where var or out made a big difference.
I was working on a SOAP web service which exported the following method:
function GetUser( out User :TUser ) :TResult;
which was getting imported into C# as the equivalent of
function GetUser( out Result :TResult) :TUser;
when I changed the out to a var it it imported correctly.
I'm guessing that the Delphi SOAP invoker treats the function result as an out parameter and that having two out parameters confuses the Delphi SOAP routines. I'm not sure if there is a workaround to allow you to use out parameters.
I read earlier that out parameter is set to default by called function, but today I realized that it is not completely true. Value of out parameter is discarded by called routine, but if that routine does not change its value, caller can still get it initial value, which was assigned before passing to called thread.
For example:
procedure JustNothing(out x : integer);
begin
// do nothing
end;
procedure TestOutVar;
var i : Integer;
begin
i := 100;
JustNothing(i); // after this call, i will still be 100
end;
It looks like the Delphi compiler does not honor const record parameters when
"records-with-methods" are involved.
Having not tried to abuse the const convention previously, I was a little surprised
to find the compiler accepted code like that:
type
TTest = record
Field : String;
procedure Update;
end;
procedure TTest.Update;
begin
Field := Field + '+1';
end;
procedure DoStuff(const t : TTest);
begin
ShowMessage(t.Field);
t.Update;
ShowMessage(t.Field);
end;
While if you try to do a
t.Field:='doh'; in DoStuff f.i., the compiler will properly complain, but you're allowed to call methods that modify the "const" record without even a hint or warning. So this is different behavior than for reference types (such as classes or dynamic arrays), where direct field writes are allowed (as const only restricts changes to the parameter itself).
Addendum: this allows to modify declared compile-time constants this way too, as in:
const
cTest : TTest = (Field : '1');
...
cTest.Update; // will show '1' then '1'+'1'
ShowMessage(cTest.Field); // will show '1' (because optimized at compile-time)
Is that an accepted/documented behavior? or just a compiler shortcoming?
const never places any restrictions on method calls in Delphi, be they on records or instances of classes. So I don't think there is anything inconsistent with the treatment of method calls.
If methods could not be called on record passed as a const parameter, then that would pretty much render records with methods useless. It would mean, for example, that a property getter could not be called. In order to place restrictions on such records passed as const, there would need to be an equivalent concept to the const member functions of C++. That would allow the compiler to know that certain methods were non-mutating.
David analyzed the restriction pretty well. If the compiler was to check out such details it could really do it with some penalty. Additionally I don't see anything wrong with the compiler's behaviour. The method which gets the record can't directly alter its data, but only when using the method it contains. The record in this case works like an object: you can in the same way an object as a const and still have the same problem you described, ie. the object's methods can be used to alter its data.
The benefit of the object is, that such methods can be declared to be private, which enables you to protect its data. You could even create an inherited class which does just that, namely hiding all possibility to alter its data. Maybe you want to try this approach?
I'm effectively trying to deserialize a form.
One of the objects on the serialized form has a method which takes a series of events as parameters.
Now since I don't have the class type of the object when I'm deserializing, I have a method on the object doing the deserialization called AddMethod which is declared like this:
procedure TMyDeserializer.AddMethod(ControlName, EventName: String;
MethodAddr: Pointer);
var
TargetControl : TControl;
Method : TMethod;
begin
if Not Assigned(TempForm) then
Exit;
if TempForm.Name = ControlName then
TargetControl := TempForm
else
TargetControl := TempForm.FindChildControl(ControlName);
if Assigned(TargetControl) then
begin
Method.Code := MethodAddr;
Method.Data := TargetControl;
SetMethodProp(TargetControl, EventName, Method);
end;
end;
So that I can poke subroutines into the various controls as I deserialize them, The problem is I need to add events as a list of parameters (not to a control). e.g.
SetUpEvents(EventHandler1:TNotifyEvent;EventHandler2:TNotifyEvent);
Where EventHandler1 and EventHandler2 are defined somewhere in code as
Procedure EventHandler1(Sender:TNotifyEvent);
begin
// Do something
end;
These are not methods but stand alone subroutines.
When I'm assigning these to objects the subroutine doesn't need to be part of an object as the AddMethod procedure handles it with a call like
MyDeserializerInstance.AddMethod('Button1','OnClick',#EventHandler1);
This works for standard event handlers, such as Button1.OnClick but not if I want to do
Procedure SetUpButton1Click(Method: TNotifyEvent)
begin
TButton(MyDeserializerInstance.TempForm.FindChildControl('Button1')).OnClick = Method;
end;
The problem is I can't pass the subroutine as a method to the example Set Up Procedure.
The form being created isn't declared in an interface and is entirely defined by the file it is read from as well as a few stand alone routines in code.
So I suppose the question is how do turn a subroutine into a method at run time (after creating the object it is supposed to be part of), and if I can't do that how do I pass the subroutines in code as parameters in another method?
So far I've tried casting a TMethod as the correct event type and filling in the .Data as the TempForm. It called the correct method but it scrambled the parameters.
Delphi version is 2007
Non-static class methods have a hidden Self input parameter that is filled in when the method is called. That is what the TMethod.Data field corresponds to. In order to use a standalone procedure as a handler for an event that expects a class method, the procedure must have an extra parameter defined to represent the Self parameter so the value of TMethod.Data has somewhere to go, ie:
procedure Button1ClickHandler(Self: Pointer; Sender: TObject);
begin
// Do something
end;
MyDeserializerInstance.AddMethod('Button1', 'OnClick', #Button1ClickHandler);
Your AddMethod() implementation is assigning the TargetControl as the TMethod.Data value, so the Self and Sender parameters above will end up pointing at the same object at runtime, but that is OK.
Without the explicit Self parameter defined, that explains why your parameters are getting "scrambled" when the procedure called at runtime. The hidden Self value is being assigned to the Sender parameter, and the real Sender value is being ignored.
I'm sure someone will correct me if I'm wrong but I don't believe there is a way to create a type definition at runtime in native Delphi. Delphi's RTTI just doesn't handle this yet.
The two scenarios that come to mind for object serialization are persistence and IPC. (There may be more that I haven't thought of).
Delphi's DFM serialization would be an example of persistence. If you look at a dfm you'll notice it isn't defining methods at all. It's simply assigning event handlers to properties expecting an event handler. Both the handlers and the properties are defined at design time using normal type definitions.
If your intent is IPC(whether on the same machine or a remote one) there are already existing frameworks for accomplishing this. (RemObjects comes to mind)
You don't "make a method" at run time. That would amount to compiling new code. The methods that you assign to various event properties need to already exist.
Furthermore, you can't "add events." The object you're deserializing already has events. You defined them when you wrote the class declaration in your Delphi code. You can't add new event properties to a class after it's been compiled.
It appears that what you're really saying is that you have a standalone procedure that you happen to have named Method1, and you want to pass it as a TNotifyEvent parameter when you call SetUpMethods.
That's the wrong way to go. That Method1 procedure isn't a method, despite its name, so you mustn't use it where a method is required. Change that declaration so it belongs to a class, and then it will be a method.
If you don't want to have to instantiate the class that the method belongs to, that's fine — you can declare it as a class method instead:
class procedure TSomeClass.Method1(Sender: TNotifyEvent);
I encourage you to change the declaration of AddMethod so that the last parameter is of type TMethod. Then you're sure to have both the code and data portions of the method pointer. Right now, you're assigning the data portion based on the object whose event property you're assigning, but as I mentioned in my comment, it's rare for that relationship to exist, especially now that the method belongs to an entirely unrelated class (TSomeClass in my example). The value of the TMethod.Data field becomes the Self value when the method gets called. It's your responsibility to ensure that the value you store in that field is of a compatible type for the class the code belongs to.