What's the difference between parameters declared with var and those declared with out? How does the compiler treat them differently (e.g., by generating different code, or by changing which diagnostics it issues)? Or do the different modifiers merely allow the programmer to document intended use of the parameters? What effect do the types of the parameters have on the matter?
A var parameter will be passed by reference, and that's it.
An out parameter is also passed by reference, but it's assumed that the input value is irrelevant. For managed types, (strings, Interfaces, etc,) the compiler will enforce this, by clearing the variable before the routine begins, equivalent to writing param := nil. For unmanaged types, the compiler implements out identically to var.
Note that the clearing of a managed parameter is performed at the call-site and so the code generated for the function does not vary with out or var parameters.
There is not much difference, for the compiler that is. See Mason's answer for that.
Semantically, there is a big difference:
var tells the programmer that the routine could work with its current value,
out tells the programmer that the routine will ignore/discard its current value.
Slightly late but just for the record, I came across a case where var or out made a big difference.
I was working on a SOAP web service which exported the following method:
function GetUser( out User :TUser ) :TResult;
which was getting imported into C# as the equivalent of
function GetUser( out Result :TResult) :TUser;
when I changed the out to a var it it imported correctly.
I'm guessing that the Delphi SOAP invoker treats the function result as an out parameter and that having two out parameters confuses the Delphi SOAP routines. I'm not sure if there is a workaround to allow you to use out parameters.
I read earlier that out parameter is set to default by called function, but today I realized that it is not completely true. Value of out parameter is discarded by called routine, but if that routine does not change its value, caller can still get it initial value, which was assigned before passing to called thread.
For example:
procedure JustNothing(out x : integer);
begin
// do nothing
end;
procedure TestOutVar;
var i : Integer;
begin
i := 100;
JustNothing(i); // after this call, i will still be 100
end;
Related
Unlike the case with common objects, it is impossible to directly assign generics of different related types in Delphi as follows:
Possible (normal objects):
var
var_1 : TObject;
var_2 : MyTObjectSubClass;
var_1 := var_2; //Works
Not possible (generics):
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := var_2; //Does not compile
It is possible to use casts to accomplish this though, as follows:
var
var_1 : TList<TObject>;
var_2 : TList<MyTObjectSubClass>;
var_1 := TList<TObject>(var_2); //Works
This creates the need to be able to dynamically cast generics (i.e. to dynamically parameterize their generic type specification) somehow, but I have not been able to find a way to do this, so my question is: Is this in any way possible?
I'm indeed aware of the covariance/contravariance problems related to this, but in some cases it would indeed both be useful and "correct" to do such a thing.
One example of such a situation is the current code I'm writing for generic streaming of Delphi objects over a TStream, where the receiving end knows the exact type of the object that is incoming over the stream, e.g. TList<MyTObjectSubClass>. This type information is extracted by means of RTTI though (from a provided target variable to which the loaded object should be written), so I cannot explicitly mention the exact generics type in my stream-loading code in advance, but rather have to detect it by means of RTTI (which is possible, although somewhat hacky) and then write it to a target variable that I only at that run-time point will know the exact type of.
Thus, the load-object-from-stream code has to be fully generic, and thus, it would need to dynamically cast an existing TList<TObject> variable (which is defined explicitly in the code) to the exact type of TList<MyTObjectSubClass> (which I at that point have just learned about, through the use of RTTI), in order to be able to pass this object loaded from the stream to its final destination variable.
So again, is there ANY way whatsoever to accomplish this, or is it on the contrary actually completely impossible to assign to a not-in-advance-known generics collections using generic code (i.e. code that does not explicitly have some kind of "if [type of xxx is TList<TMyObject1>] then ... else if [type of xxx is TList<TMyObject2>] then ... else ..." test, containing explicit mentions of every single generics type that should be supported by it)?
PS.
The generics type of the stream-loaded object obviously already exists somewhere in the program (since it is concluded by means of RTTI on the target variable that the stream-loaded object should be written to), so I'm not asking about full run-time dynamic creation of generics types, but rather just about how to be able to dynamically pick the right one of those generics types already defined at compile-time in the program, and then cast a variable to that type.
EDIT:
By request from #RemyLebeau , here comes some more example code from my application, from its stream-loading function:
var
source_stream : TStream;
field_to_process : TRttiField;
field_type : TRttiType;
loaded_value : TValue;
temp_int : integer;
//...
//The fields of any object given to the streaming function are
//enumerated and sorted here
//...
//Then, for each field (provided in field_to_process),
//the following is done:
case field_to_process.FieldType.TypeKind of
//...
tkInteger:
begin
source_stream.ReadBufferData(temp_int);
loaded_value := TValue.From(temp_int);
end;
tkString,
tkLString,
tkWString,
tkUString:
begin
source_stream.ReadBufferData(noof_raw_bytes_in_string_data);
SetLength(raw_byte_buf, noof_raw_bytes_in_string_data + 4);
source_stream.ReadBuffer(raw_byte_buf[0], noof_raw_bytes_in_string_data);
temp_str := used_string_encoding.GetString(raw_byte_buf, 0, noof_raw_bytes_in_string_data);
loaded_value := TValue.From(temp_str);
end;
tkClass:
begin
is_generics_collection_containing_TObject_descendants := <does some hacky detection here>; //Thanks Remy :-)
if is_generics_collection_containing_TObject_descendants then
begin
<magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type>
end;
end;
//...
end;
field_to_process.SetValue(self, loaded_value);
That should hopefully give a somewhat better overview of my problem. The superfluous code for strings and integers are just for context, by showing how some simple types are handled.
For more info about the (necessarily) "hacky detection" mentioned in the code, please see this question. After doing that, I will know the exact type of the generics collection and its subitems, for example TList<TSomeTObjectDescendant>.
So, as you hopefully can see now, the question is about the <magic code goes here that loads data from the stream into the currently processed field, whose type has been detected to be of some specific generics collection type> part. How can it be implemented?
NOTE: My problem is not to understand how to serialize/deserialize contents of an enumerable through a stream (which can of course be done by simply iterating over the items in the enumerable and then recursing the stream saving/loading code for each of them, where the number of items is given first of all in the stream). The problem is rather how to create generic code that will be able to recreate/populate any kind of generics collection of TObject descentants, whose type you only get to know at runtime, and then to finally get this into the object field that was originally enumerated by RTTI at the beginning of the stream-loading code. As an example, assume that the processed field has the type TList<TSomeTObjectDescendant>, and that you can easily load its subobjects from the stream using a call like function load_list_TSomeTObjectDescendant_subitems(input_stream : TStream) : array of TSomeTObjectDescendant. How could I then get these subitems into the TList<TSomeTObjectDescendant> field?
Type-casts and variable declarations are parsed at compile-time (though is and as casts are executed at runtime based on compiler-provided RTTI). The type being casted to, and the type of the variable being assigned to, must be known to the compiler. So what you are asking for is simply not possible with Generics. Not the way you have described it, anyway.
It looks like the Delphi compiler does not honor const record parameters when
"records-with-methods" are involved.
Having not tried to abuse the const convention previously, I was a little surprised
to find the compiler accepted code like that:
type
TTest = record
Field : String;
procedure Update;
end;
procedure TTest.Update;
begin
Field := Field + '+1';
end;
procedure DoStuff(const t : TTest);
begin
ShowMessage(t.Field);
t.Update;
ShowMessage(t.Field);
end;
While if you try to do a
t.Field:='doh'; in DoStuff f.i., the compiler will properly complain, but you're allowed to call methods that modify the "const" record without even a hint or warning. So this is different behavior than for reference types (such as classes or dynamic arrays), where direct field writes are allowed (as const only restricts changes to the parameter itself).
Addendum: this allows to modify declared compile-time constants this way too, as in:
const
cTest : TTest = (Field : '1');
...
cTest.Update; // will show '1' then '1'+'1'
ShowMessage(cTest.Field); // will show '1' (because optimized at compile-time)
Is that an accepted/documented behavior? or just a compiler shortcoming?
const never places any restrictions on method calls in Delphi, be they on records or instances of classes. So I don't think there is anything inconsistent with the treatment of method calls.
If methods could not be called on record passed as a const parameter, then that would pretty much render records with methods useless. It would mean, for example, that a property getter could not be called. In order to place restrictions on such records passed as const, there would need to be an equivalent concept to the const member functions of C++. That would allow the compiler to know that certain methods were non-mutating.
David analyzed the restriction pretty well. If the compiler was to check out such details it could really do it with some penalty. Additionally I don't see anything wrong with the compiler's behaviour. The method which gets the record can't directly alter its data, but only when using the method it contains. The record in this case works like an object: you can in the same way an object as a const and still have the same problem you described, ie. the object's methods can be used to alter its data.
The benefit of the object is, that such methods can be declared to be private, which enables you to protect its data. You could even create an inherited class which does just that, namely hiding all possibility to alter its data. Maybe you want to try this approach?
I need a way to write a generic procedure to act upon an object type or any of its descendants.
My first attempt was to declare
procedure TotalDestroy(var obj:TMyObject);
but when using it with a descendant object
type TMyNewerObject = class(TMyObject);
var someNewerObject: TMyNewerObject;
TotalDestroy(someNewerObject);
I get the infamous error "types of formal and actual parameters must be identical"
So, while strugling to find a solution, I looked at the source code of Delphi system FreeAndNil procedure. And I found this awesome declaration, along with this astonishing comment
{ FreeAndNil frees the given TObject instance and
sets the variable reference to nil.
Be careful to only pass TObjects to this routine. }
procedure FreeAndNil(var Obj);
It avoids the type checking error, but it uses no safety net.
My question is ... is there any safe way to check the type of an untyped var parameter?
or in other words, can you improve this Delphi source code so that the warning would not be needed?
procedure FreeAndNil(var Obj);
var
Temp: TObject;
begin
Temp := TObject(Obj);
Pointer(Obj) := nil;
Temp.Free;
end;
Let's examine what you want to do.
You want to call a method that takes X, passing in an object of type Y, where Y is a descendant of X. The snag, the parameter is a "var" parameter.
Let's analyze what you could do if that was possible.
type
TBase = class
end;
TDescendant = class(TBase)
end;
procedure Fiddle(var x: TBase);
begin
x := TDescendant.Create;
end;
type
TOtherDescendant = class(TBase)
end;
var a: TOtherDescendant;
a := TOtherDescendant.Create;
Fiddle(a);
Uh-oh, now a no longer contains an instance of TOtherDescendant, it contains an instance of TDescendant. That probably comes as a surprise to the code that follows the call.
You must not only consider what you intend to do with the syntax you propose, but effectively what you could do with the syntax.
You should read Eric Lipperts excellent blog post about similar issues in .NET, found here: Why do ref and out parameters not allow type variation?.
I've written about this before, using an example very similar to Lasse's:
Delphi Q&A: Why must the types of actual and formal var parameters be identical?
Unless you're writing an assignment statement to change the value of the input parameter itself, and not just one of its properties, you shouldn't pass a parameter by reference in the first place.
If you are writing an assignment statement to change the parameter's value, then the compiler message really is true, and you should heed it.
One reason for needing to by-pass the error is when you're writing a function like TApplication.CreateForm. Its job is to change the input parameter's value, and the type of the new value varies and cannot be determined at compile time. If you're writing such a function, then your only option with Delphi is to use an untyped var parameter, and then there is extra burden on both the caller and the receiver to make sure everything goes right. The caller needs to make sure it passes a variable that is capable of holding values of whatever type the function will put in it, and the function needs to make sure it stores a value of a type compatible with what the caller requested.
In the case of CreateForm, the caller passes in a class-reference literal and a variable of that class type. The function instantiates the class and stores the reference in the variable.
I don't think very highly of either CreateForm or FreeAndNil, largely because of the way their untyped parameters sacrifice type safety in return for comparatively little extra convenience. You haven't shown the implementation of your TotalDestroy function, but I suspect its var parameter will ultimately provide the same low utility as in those other two functions. See my articles on both:
When should I use FreeAndNil?
Why shouldn't I call Application.CreateForm?
In addition to what Lasse wrote, which is quite correct, most of the time you don't want to pass an object to a var parameter anyway.
An object is a reference type. What you see as the object is actually a reference to it. You would only want to pass an object reference to a var parameter if you wanted to change your object out for a new object. If you just want to be able to modify the members of the object, then you can do that by simply passing it to a normal parameter. Make method call take a TMyObject parameter instead of a var TMyObject parameter and it should work.
Of course, if you really are replacing the object, then feel free to disregard all this, and see Lasse's answer.
can you improve this Delphi source code so that the warning would not be needed?
Yes, you can get a type safe way to avoid the compiler error.
In the newest Delphi 10.4 Sidney, the FreeAndNil procedure has been changed into this:
procedure FreeAndNil(const [ref] Obj: TObject);
var
Temp: TObject;
begin
Temp := Obj;
TObject(Pointer(#Obj)^) := nil;
Temp.Free;
end;
It is type safe for objects and will catch errors when passing an interface reference for example.
The way to pass a parameter by const [ref] means that the parameter is passed by reference. Without the [ref] attribute, parameters with size equal and smaller than a pointer would otherwise be passed by value.
Here, even though the object is passed as a constant, the reference will be modified.
In that sense, it is not a perfect declaration, but will do its job better than the former implementation.
From New features in Delphi 10.4:
This means that incorrect usage of FreeAndNil will now cause a compiler error. In the past, incorrect usage would not be caught, leading to difficult bugs. Note that although the parameter is declared as const, the by-reference variable is indeed modified.
A new, but ‘not as bad’, class of incorrect calling is possible with this declaration of FreeAndNil: the method can be called passing in properties or a method result, as well as cast expressions, a type’s implicit conversion to TObject, etc. The nil-ed value will then be the temporary variable in the expression.
In one of the Delphi demo applications, I've stumbled upon some syntax that I didn't know the Delphi compiler accepted:
// ......\Demos\DelphiWin32\VCLWin32\ActiveX\OleAuto\SrvComp\Word\
// Main.pas, line 109
Docs.Add(NewTemplate := True); // note the assignment
I can't seem to reproduce this type of parameter passing in my own code, and I never see anyone use it. So these are my questions:
Can i use this in "normal" methods and is it part of "the Delphi Language", or is this some compiler hack for automation objects?
What's needed in order to be able to use this?
Is this anything like C#4's named and optional parameters?
Additional information: I usually pass
records or simple classes when there
are many optional parameters to
methods, but it looks like I wouldn't
need that with this syntax. I'm aware
of default parameter values, but their
usefulness is limited because you
cannot provide any parameters to the
right of an omitted one. In JavaScript
I'm using this named parameter style
all the time (be it with different
syntax), and it's powerful.
Clearly the Delphi language supports named parameters since they appear right there in sample Delphi code. Delphi supports named parameters on automation objects, which are objects that implement the IDispatch interface. There are restrictions on the types the parameters and return types can have; in particular, they can't be Delphi classes.
I don't think the convenience you seek from named parameters would outweigh the performance hit you'd take by having every method call routed through the IDispatch.Invoke method. A call may also need to use GetIDsOfNames first. You don't see this in more code because late binding is usually something people try to avoid. Use early binding whenever possible to avoid the cost of looking up dispatch IDs and indirect method invocations.
Delphi supports optional parameters in non-automation code by allowing default values. You can omit the actual parameters for any parameter with a default value as long as you also omit the actual parameters of all subsequent parameters — the compiler ensures that a function's declaration allows for that.
I think optional parameters are overrated. They save time for the (one) person writing the code, but not for the (many) people reading the code. Whoever's reading it needs to know what the default values will be of any unspecified parameters, so you may as well just provide all the values explicitly anyway.
If you declare your procedure like so:
procedure DoSomething(AParam : integer = 0);
... it will assume a value of 0 for the parameter if it isn't given. As I recall, parameters with default values have to be at the end of the call, so like this:
procedure DoSomething(AFirstParam : string; AParam : integer = 0);
not like this:
procedure DoSomething(AParam : integer = 0; ASecondParam : string);
It is basically "some compiler hack for automation objects". I sometimes have to use it for Excel and Word automation.
e.g.
MSExcel.Application.Cells.Replace(What:='', Replacement:='', LookAt:=xlPart,
SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=True, ReplaceFormat:=True);
Is equivalent to VBA
Application.Cells.Replace(What='', Replacement='', LookAt=xlPart, _
SearchOrder=xlByRows, MatchCase=False, SearchFormat=True, ReplaceFormat=True)
One nice thing about anonymous methods is that I can use variables that are local in the calling context. Is there any reason why this does not work for out-parameters and function results?
function ReturnTwoStrings (out Str1 : String) : String;
begin
ExecuteProcedure (procedure
begin
Str1 := 'First String';
Result := 'Second String';
end);
end;
Very artificial example of course, but I ran into some situations where this would have been useful.
When I try to compile this, the compiler complains that he "cannot capture symbols". Also, I got an internal error once when I tried to do this.
EDIT I just realized that it works for normal parameters like
... (List : TList)
Isn't that as problematic as the other cases? Who guarantees that the reference is still pointing to an alive object whenever the anonymous method is executed?
Var and out parameters and the Result variable cannot be captured because the safety of this operation cannot be statically verified. When the Result variable is of a managed type, such as a string or an interface, the storage is actually allocated by the caller and a reference to this storage is passed as an implicit parameter; in other words, the Result variable, depending on its type, is just like an out parameter.
The safety cannot be verified for the reason Jon mentioned. The closure created by an anonymous method can outlive the method activation where it was created, and can similarly outlive the activation of the method that called the method where it was created. Thus, any var or out parameters or Result variables captured could end up orphaned, and any writes to them from inside the closure in the future would corrupt the stack.
Of course, Delphi does not run in a managed environment, and it doesn't have the same safety restrictions as e.g. C#. The language could let you do what you want. However, it would result in hard to diagnose bugs in situations where it went wrong. The bad behaviour would manifest itself as local variables in a routine changing value with no visible proximate cause; it would be even worse if the method reference were called from another thread.
This would be fairly hard to debug. Even hardware memory breakpoints would be a relatively poor tool, as the stack is modified frequently. One would need to turn on the hardware memory breakpoints conditionally upon hitting another breakpoint (e.g. upon method entry). The Delphi debugger can do this, but I would hazard a guess that most people don't know about the technique.
Update: With respect to the additions to your question, the semantics of passing instance references by value is little different between methods that contain a closure (and capture the paramete0 and methods that don't contain a closure. Either method may retain a reference to the argument passed by value; methods not capturing the parameter may simply add the reference to a list, or store it in a private field.
The situation is different with parameters passed by reference because the expectations of the caller are different. A programmer doing this:
procedure GetSomeString(out s: string);
// ...
GetSomeString(s);
would be extremely surprised if GetSomeString were to keep a reference to the s variable passed in. On the other hand:
procedure AddObject(obj: TObject);
// ...
AddObject(TObject.Create);
It is not surprising that AddObject keeps a reference, since the very name implies that it's adding the parameter to some stateful store. Whether that stateful store is in the form of a closure or not is an implementation detail of the AddObject method.
The problem is that your Str1 variable is not "owned" by ReturnTwoStrings, so that your anonymous method cannot capture it.
The reason it cannot capture it, is that the compiler does not know the ultimate owner (somewhere in the call stack towards calling ReturnTwoStrings) so it cannot determine where to capture it from.
Edit: (Added after a comment of Smasher)
The core of anonymous methods is that they capture the variables (not their values).
Allen Bauer (CodeGear) explains a bit more about variable capturing in his blog.
There is a C# question about circumventing your problem as well.
The out parameter and return value are irrelevant after the function returns - how would you expect the anonymous method to behave if you captured it and executed it later? (In particular, if you use the anonymous method to create a delegate but never execute it, the out parameter and return value wouldn't be set by the time the function returned.)
Out parameters are particularly difficult - the variable that the out parameter aliases may not even exist by the time you later call the delegate. For example, suppose you were able to capture the out parameter and return the anonymous method, but the out parameter is a local variable in the calling function, and it's on the stack. If the calling method then returned after storing the delegate somewhere (or returning it) what would happen when the delegate was finally called? Where would it write to when the out parameter's value was set?
I'm putting this in a separate answer because your EDIT makes your question really different.
I'll probably extend this answer later as I'm in a bit of a hurry to get to a client.
Your edit indicates you need to rethink about value types, reference types and the effect of var, out, const and no parameter marking at all.
Let's do the value types thing first.
The values of value types live on the stack and have a copy-on-assignment behaviour.
(I'll try to include an example on that later).
When you have no parameter marking, the actual value passed to a method (procedure or function) will be copied to the local value of that parameter inside the method. So the method does not operate on the value passed to it, but on a copy.
When you have out, var or const, then no copy takes place: the method will refer to the actual value passed. For var, it will allow to to change that actual value, for const it will not allow that. For out, you won't be able to read the actual value, but still be able to write the actual value.
Values of reference types live on the heap, so for them it hardly matters if you have out, var, const or no parameter marking: when you change something, you change the value on the heap.
For reference types, you still get a copy when you have no parameter marking, but that is a copy of a reference that still points to the value on the heap.
This is where anonymous methods get complicated: they do a variable capture.
(Barry can probably explain this even better, but I'll give it a try)
In your edited case, the anonymous method will capture the local copy of the List. The anonymous method will work on that local copy, and from a compiler perspective everything is dandy.
However, the crux of your edit is the combination of 'it works for normal parameters' and 'who guarantees that the reference is still pointing to an alive object whenever the anonymous method is executed'.
That is always a problem with reference parameters, no matter if you use anonymous methods or not.
For instance this:
procedure TMyClass.AddObject(Value: TObject);
begin
FValue := Value;
end;
procedure TMyClass.DoSomething();
begin
ShowMessage(FValue.ToString());
end;
Who guarantees that when someone calls DoSomething, that the instance where FValue points to still exists?
The answer is that you must guarantee this yourself by not calling DoSomething when the instance to FValue has died.
The same holds for your edit: you should not call the anonymous method when the underlying instance has died.
This is one of the areas where reference counted or garbage collected solutions make life easier: there the instance will be kept alive until the last reference to it has gone away (which might cause instance to live longer than you originally anticipated!).
So, with your edit, your question actually changes from anonymous methods to the implications of using reference typed parameters and lifetime management in general.
Hopefully my answer helps you going in that area.
--jeroen