Unlike the case with common objects, it is impossible to directly assign generics of different related types in Delphi as follows:
Possible (normal objects):
var
var_1 : TObject;
var_2 : MyTObjectSubClass;
var_1 := var_2; //Works
Not possible (generics):
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := var_2; //Does not compile
It is possible to use casts to accomplish this though, as follows:
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := TList<TObject>(var_2); //Works
This creates the need to be able to dynamically cast generics (i.e. to dynamically parameterize their generic type specification) somehow, but I have not been able to find a way to do this, so my question is: Is this in any way possible?
I'm indeed aware of the covariance/contravariance problems related to this, but in some cases it would indeed both be useful and "correct" to do such a thing.
One example of such a situation is the current code I'm writing for generic streaming of Delphi objects over a TStream, where the receiving end knows the exact type of the object that is incoming over the stream, e.g. TList<MyTObjectSubClass>. This type information is extracted by means of RTTI though (from a provided target variable to which the loaded object should be written), so I cannot explicitly mention the exact generics type in my stream-loading code in advance, but rather have to detect it by means of RTTI (which is possible, although somewhat hacky) and then write it to a target variable that I only at that run-time point will know the exact type of.
Thus, the load-object-from-stream code has to be fully generic, and thus, it would need to dynamically cast an existing TList<TObject> variable (which is defined explicitly in the code) to the exact type of TList<MyTObjectSubClass> (which I at that point have just learned about, through the use of RTTI), in order to be able to pass this object loaded from the stream to its final destination variable.
So again, is there ANY way whatsoever to accomplish this, or is it on the contrary actually completely impossible to assign to a not-in-advance-known generics collections using generic code (i.e. code that does not explicitly have some kind of "if [type of xxx is TList<TMyObject1>] then ... else if [type of xxx is TList<TMyObject2>] then ... else ..." test, containing explicit mentions of every single generics type that should be supported by it)?
PS.
The generics type of the stream-loaded object obviously already exists somewhere in the program (since it is concluded by means of RTTI on the target variable that the stream-loaded object should be written to), so I'm not asking about full run-time dynamic creation of generics types, but rather just about how to be able to dynamically pick the right one of those generics types already defined at compile-time in the program, and then cast a variable to that type.
EDIT:
By request from #RemyLebeau , here comes some more example code from my application, from its stream-loading function:
var
source_stream : TStream;
field_to_process : TRttiField;
field_type : TRttiType;
loaded_value : TValue;
temp_int : integer;
//...
//The fields of any object given to the streaming function are
//enumerated and sorted here
//...
//Then, for each field (provided in field_to_process),
//the following is done:
case field_to_process.FieldType.TypeKind of
//...
tkInteger:
begin
source_stream.ReadBufferData(temp_int);
loaded_value := TValue.From(temp_int);
end;
tkString,
tkLString,
tkWString,
tkUString:
begin
source_stream.ReadBufferData(noof_raw_bytes_in_string_data);
SetLength(raw_byte_buf, noof_raw_bytes_in_string_data + 4);
source_stream.ReadBuffer(raw_byte_buf[0], noof_raw_bytes_in_string_data);
temp_str := used_string_encoding.GetString(raw_byte_buf, 0, noof_raw_bytes_in_string_data);
loaded_value := TValue.From(temp_str);
end;
tkClass:
begin
is_generics_collection_containing_TObject_descendants := <does some hacky detection here>; //Thanks Remy :-)
if is_generics_collection_containing_TObject_descendants then
begin
<magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type>
end;
end;
//...
end;
field_to_process.SetValue(self, loaded_value);
That should hopefully give a somewhat better overview of my problem. The superfluous code for strings and integers are just for context, by showing how some simple types are handled.
For more info about the (necessarily) "hacky detection" mentioned in the code, please see this question. After doing that, I will know the exact type of the generics collection and its subitems, for example TList<TSomeTObjectDescendant>.
So, as you hopefully can see now, the question is about the <magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type> part. How can it be implemented?
NOTE: My problem is not to understand how to serialize/deserialize contents of an enumerable through a stream (which can of course be done by simply iterating over the items in the enumerable and then recursing the stream saving/loading code for each of them, where the number of items is given first of all in the stream). The problem is rather how to create generic code that will be able to recreate/populate any kind of generics collection of TObject descentants, whose type you only get to know at runtime, and then to finally get this into the object field that was originally enumerated by RTTI at the beginning of the stream-loading code. As an example, assume that the processed field has the type TList<TSomeTObjectDescendant>, and that you can easily load its subobjects from the stream using a call like function load_list_TSomeTObjectDescendant_subitems(input_stream : TStream) : array of TSomeTObjectDescendant. How could I then get these subitems into the TList<TSomeTObjectDescendant> field?
Type-casts and variable declarations are parsed at compile-time (though is and as casts are executed at runtime based on compiler-provided RTTI). The type being casted to, and the type of the variable being assigned to, must be known to the compiler. So what you are asking for is simply not possible with Generics. Not the way you have described it, anyway.
Related
tl;dr:
Trying to pass a list of objects to a function, which expects list of objects implementing an interface. Said objects implement that interface. Compiler will not allow it.
Looking for an alternative workaround or my mistake.
The setup
I did not use the actual code, and that should not matter IMO. Seems like a conceptional problem to me.
The class TObjectWithInterface implements the interface ISomeInterface and extends the class TCustomInterfacedObject, which simply works around the reference counting of IInterface as given in the documentation.
TObjectWithInterface = class(TCustomInterfacedObject, ISomeInterface)
If have a procedure which takes a list of objects implementing that interface:
procedure SomeFunction(List: TList<ISomeInterface>)
The issue
Inside of a function of TObjectWithInterface, I try to call that function, using a list of objects of TObjectWithInterface:
procedure TObjectWithInterface.DoTheStuff(ListOfObjects: TList<TObjectWithInterface>)
begin
// ...
SomeFunction(ListOfObjects); // <-- Compiler error: Incompatible types
// ...
end;
The compiler tells me the following:
E2010: Incompatible types: 'System.Generics.Collections.TList' and 'System.Generics.Collections.TList'
The dumb workaround
I really dislike my current workaround, which consists of creating a new list and type casting each TObjectWithInterface to ISomeInterface:
procedure TObjectWithInterface.DoTheStuff(ListOfObjects: TList<TObjectWithInterface>)
var
ListOfInterfaceObjects: TList<ISomeInterface>;
begin
// ...
ListOfInterfaceObjects := TList<ISomeInterface>.Create;
for var Object in ListOfObjects do
ListOfInterfaceObjects.Add(Objects as ISomeInterface);
SomeFunction(ListOfInterfaceObjects)
// ...
end;
This seems very hacky to me. I may have done something stupid, or do not understand something correctly, as this is the first time, that I am trying to use Interfaces in Delphi. Please don't be mad.
Either way, I hope someone can point out my mistake or, if this is a language limitation, has an alternative workaround.
Your "workaround" to copy the list of objects into a separate list of interfaces is actually the correct and proper solution. You can't use a TList<X> where a TList<Y> is expected, they are different and unrelated types. Just because you are disabling reference counting on the interfaces doesn't change the memory layout of the objects in relation to their interfaces, so you still have to pass around proper memory pointers, which means performing necessary conversions from one to the other.
I have a list of type TList<TForm>. I need to cast it and use it as TList<TObject> like this:
procedure mainForm.testCast;
var
listT: TList<TForm>;
listW: TList<TObject>;
obj: TObject;
begin
listT := TList<TForm>.create;
listT.add(form1);
listT.add(form2);
listW := TList<TObject>(listT); // Casting is OK
// This works, but is this fine?
for obj in listW do
memo1.lines.add(obj.className);
end;
The sample works as expected, but is it ok to cast like this between generic lists? Will this cause some data structure corruption etc?
I use it only for looping (DoGetEnumerator) purposes and some string checks i.e. I'll not add/remove items.
The real function is little more complicated. It gets reference to listT using RTTI in a TValue.
The main goal is not to link FMX.Forms in my unit.
Update:
Why are TGeneric<Base> and TGeneric<Descendant> incompatible types?
Well, your code will work, but it somewhat dubious in my view. Simply put the cast is not legal because
TList<TForm>.InheritsFrom(TList<TObject>)
is false. So a TList<TForm> object is not a TList<TObject>. If it were, then the cast would not be needed.
That this is so is because Delphi's generic types are invariant. More details can be found here:
Why is a class implementing an interface not compatible with the interface type when used in generics?
If you have any difficulty understanding why the designers made generic types invariant, consider for a moment the effect of writing listW.Add(TObject.Create) in your code. Think what it means to the true underlying object of type TList<TForm>.
So the language promises you nothing. You are venturing outside its guarantees. It so happens that the implementation of these two un-related types is compatible enough for your code to work. But that is really just an accident of implementation.
Since you are already using RTTI, then I suggest that you iterate over the list with RTTI. You can call GetEnumerator and so on using RTTI. That way you will call the actual methods of the object.
What's the difference between parameters declared with var and those declared with out? How does the compiler treat them differently (e.g., by generating different code, or by changing which diagnostics it issues)? Or do the different modifiers merely allow the programmer to document intended use of the parameters? What effect do the types of the parameters have on the matter?
A var parameter will be passed by reference, and that's it.
An out parameter is also passed by reference, but it's assumed that the input value is irrelevant. For managed types, (strings, Interfaces, etc,) the compiler will enforce this, by clearing the variable before the routine begins, equivalent to writing param := nil. For unmanaged types, the compiler implements out identically to var.
Note that the clearing of a managed parameter is performed at the call-site and so the code generated for the function does not vary with out or var parameters.
There is not much difference, for the compiler that is. See Mason's answer for that.
Semantically, there is a big difference:
var tells the programmer that the routine could work with its current value,
out tells the programmer that the routine will ignore/discard its current value.
Slightly late but just for the record, I came across a case where var or out made a big difference.
I was working on a SOAP web service which exported the following method:
function GetUser( out User :TUser ) :TResult;
which was getting imported into C# as the equivalent of
function GetUser( out Result :TResult) :TUser;
when I changed the out to a var it it imported correctly.
I'm guessing that the Delphi SOAP invoker treats the function result as an out parameter and that having two out parameters confuses the Delphi SOAP routines. I'm not sure if there is a workaround to allow you to use out parameters.
I read earlier that out parameter is set to default by called function, but today I realized that it is not completely true. Value of out parameter is discarded by called routine, but if that routine does not change its value, caller can still get it initial value, which was assigned before passing to called thread.
For example:
procedure JustNothing(out x : integer);
begin
// do nothing
end;
procedure TestOutVar;
var i : Integer;
begin
i := 100;
JustNothing(i); // after this call, i will still be 100
end;
It looks like the Delphi compiler does not honor const record parameters when
"records-with-methods" are involved.
Having not tried to abuse the const convention previously, I was a little surprised
to find the compiler accepted code like that:
type
TTest = record
Field : String;
procedure Update;
end;
procedure TTest.Update;
begin
Field := Field + '+1';
end;
procedure DoStuff(const t : TTest);
begin
ShowMessage(t.Field);
t.Update;
ShowMessage(t.Field);
end;
While if you try to do a
t.Field:='doh'; in DoStuff f.i., the compiler will properly complain, but you're allowed to call methods that modify the "const" record without even a hint or warning. So this is different behavior than for reference types (such as classes or dynamic arrays), where direct field writes are allowed (as const only restricts changes to the parameter itself).
Addendum: this allows to modify declared compile-time constants this way too, as in:
const
cTest : TTest = (Field : '1');
...
cTest.Update; // will show '1' then '1'+'1'
ShowMessage(cTest.Field); // will show '1' (because optimized at compile-time)
Is that an accepted/documented behavior? or just a compiler shortcoming?
const never places any restrictions on method calls in Delphi, be they on records or instances of classes. So I don't think there is anything inconsistent with the treatment of method calls.
If methods could not be called on record passed as a const parameter, then that would pretty much render records with methods useless. It would mean, for example, that a property getter could not be called. In order to place restrictions on such records passed as const, there would need to be an equivalent concept to the const member functions of C++. That would allow the compiler to know that certain methods were non-mutating.
David analyzed the restriction pretty well. If the compiler was to check out such details it could really do it with some penalty. Additionally I don't see anything wrong with the compiler's behaviour. The method which gets the record can't directly alter its data, but only when using the method it contains. The record in this case works like an object: you can in the same way an object as a const and still have the same problem you described, ie. the object's methods can be used to alter its data.
The benefit of the object is, that such methods can be declared to be private, which enables you to protect its data. You could even create an inherited class which does just that, namely hiding all possibility to alter its data. Maybe you want to try this approach?
I need a way to write a generic procedure to act upon an object type or any of its descendants.
My first attempt was to declare
procedure TotalDestroy(var obj:TMyObject);
but when using it with a descendant object
type TMyNewerObject = class(TMyObject);
var someNewerObject: TMyNewerObject;
TotalDestroy(someNewerObject);
I get the infamous error "types of formal and actual parameters must be identical"
So, while strugling to find a solution, I looked at the source code of Delphi system FreeAndNil procedure. And I found this awesome declaration, along with this astonishing comment
{ FreeAndNil frees the given TObject instance and
sets the variable reference to nil.
Be careful to only pass TObjects to this routine. }
procedure FreeAndNil(var Obj);
It avoids the type checking error, but it uses no safety net.
My question is ... is there any safe way to check the type of an untyped var parameter?
or in other words, can you improve this Delphi source code so that the warning would not be needed?
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj);
Pointer(Obj) := nil;
Temp.Free;
end;
Let's examine what you want to do.
You want to call a method that takes X, passing in an object of type Y, where Y is a descendant of X. The snag, the parameter is a "var" parameter.
Let's analyze what you could do if that was possible.
type
TBase = class
end;
TDescendant = class(TBase)
end;
procedure Fiddle(var x: TBase);
begin
x := TDescendant.Create;
end;
type
TOtherDescendant = class(TBase)
end;
var a: TOtherDescendant;
a := TOtherDescendant.Create;
Fiddle(a);
Uh-oh, now a no longer contains an instance of TOtherDescendant, it contains an instance of TDescendant. That probably comes as a surprise to the code that follows the call.
You must not only consider what you intend to do with the syntax you propose, but effectively what you could do with the syntax.
You should read Eric Lipperts excellent blog post about similar issues in .NET, found here: Why do ref and out parameters not allow type variation?.
I've written about this before, using an example very similar to Lasse's:
Delphi Q&A: Why must the types of actual and formal var parameters be identical?
Unless you're writing an assignment statement to change the value of the input parameter itself, and not just one of its properties, you shouldn't pass a parameter by reference in the first place.
If you are writing an assignment statement to change the parameter's value, then the compiler message really is true, and you should heed it.
One reason for needing to by-pass the error is when you're writing a function like TApplication.CreateForm. Its job is to change the input parameter's value, and the type of the new value varies and cannot be determined at compile time. If you're writing such a function, then your only option with Delphi is to use an untyped var parameter, and then there is extra burden on both the caller and the receiver to make sure everything goes right. The caller needs to make sure it passes a variable that is capable of holding values of whatever type the function will put in it, and the function needs to make sure it stores a value of a type compatible with what the caller requested.
In the case of CreateForm, the caller passes in a class-reference literal and a variable of that class type. The function instantiates the class and stores the reference in the variable.
I don't think very highly of either CreateForm or FreeAndNil, largely because of the way their untyped parameters sacrifice type safety in return for comparatively little extra convenience. You haven't shown the implementation of your TotalDestroy function, but I suspect its var parameter will ultimately provide the same low utility as in those other two functions. See my articles on both:
When should I use FreeAndNil?
Why shouldn't I call Application.CreateForm?
In addition to what Lasse wrote, which is quite correct, most of the time you don't want to pass an object to a var parameter anyway.
An object is a reference type. What you see as the object is actually a reference to it. You would only want to pass an object reference to a var parameter if you wanted to change your object out for a new object. If you just want to be able to modify the members of the object, then you can do that by simply passing it to a normal parameter. Make method call take a TMyObject parameter instead of a var TMyObject parameter and it should work.
Of course, if you really are replacing the object, then feel free to disregard all this, and see Lasse's answer.
can you improve this Delphi source code so that the warning would not be needed?
Yes, you can get a type safe way to avoid the compiler error.
In the newest Delphi 10.4 Sidney, the FreeAndNil procedure has been changed into this:
procedure FreeAndNil(const [ref] Obj: TObject);
var
Temp: TObject;
begin
Temp := Obj;
TObject(Pointer(#Obj)^) := nil;
Temp.Free;
end;
It is type safe for objects and will catch errors when passing an interface reference for example.
The way to pass a parameter by const [ref] means that the parameter is passed by reference. Without the [ref] attribute, parameters with size equal and smaller than a pointer would otherwise be passed by value.
Here, even though the object is passed as a constant, the reference will be modified.
In that sense, it is not a perfect declaration, but will do its job better than the former implementation.
From New features in Delphi 10.4:
This means that incorrect usage of FreeAndNil will now cause a compiler error. In the past, incorrect usage would not be caught, leading to difficult bugs. Note that although the parameter is declared as const, the by-reference variable is indeed modified.
A new, but ‘not as bad’, class of incorrect calling is possible with this declaration of FreeAndNil: the method can be called passing in properties or a method result, as well as cast expressions, a type’s implicit conversion to TObject, etc. The nil-ed value will then be the temporary variable in the expression.