I want to fix a design flaw in a record TMyValue that has been in use for years, and I desperately want to fix it in the record itself - to avoid changing the public interface - and not require the code using the record to be changed.
Essentially, the flaw is that a TMyValue must be initialised to zero when it is created, otherwise calling the Clear method on it can cause a crash. This is because it contains a field that is a pointer to dynamically-allocated memory if the pointer is non-nil, and Clear causes the dynamic memory to be freed.
This is a problem if the TMyValue is created on the stack, because stack variables are not zeroed out automatically when they are created.
I thought I could use a record constructor to zero out the record, but record constructors cannot be parameterless. Presumably this means that you cannot force a record constructor to be executed automatically when a record is created on the stack.
I suspect the answer to my question is "it cannot be done". Please prove me wrong!
Add a dummy string member into your record.
Since a string is a managed type it will be initialized to an empty string when the record comes in scope.
So when calling your Clear method, test if the dummy string is empty first.
Set the string to a value when appropriate to the use logic to mark the record as initialized.
Related
I was debugging a complicated bug recently. It was caused by accessing a non-existing Form.Handle (garbaged pointer). The bug revealed itself in rather unexpected way for me - accessing Forms Handle caused resizes and repaints.
I would expect accessing Form.Handle by a garbage pointer would just return some garbage THandle. Expecting that the Handle is created once on form creation and stays the same till the Form is destroyed.
The question
Why is it so, that TForm.Handle is not a field that gets initialized on form creation and is accessed via
property Handle: Integer read FHandle;
, but is a getter
property Handle: Integer read GetHandle;
that creates the Handle and even the Window (CreateWnd) on first access?
The form object can exist even when the underlying OS window doesn't. During those times, the Handle field would be 0, which isn't helpful to code that needs a valid window handle. To ensure you get a valid handle each time you need one, you'd need to call HandleNeeded prior to referring to the Handle field. As a property with a getter, the property can call HandleNeeded for you automatically, making it easier to use the Handle property.
I use Delphi Seattle.
My problem occurs when I attempt to free an object I created.
I searched in this site (and other sites as well) for answers already posted for this question, but they all are a bit different. According to those discussions, my code should work, but obviously something isn't quite right.
So, I need help...
Flow of execution:
a) in form fmLoanRequest, I create an object based on Class TStorageLoan (a Sub-class of TLoan). The Constructor loads all kinds of values into some of the object's attributes (now shown here).
b) Later, I pass the object's address to another form (fmLoan) to an appropriate public variable. fmLoan is the form where all the user's dealings with the contents of Loan happen. Note that fmLoanRequest remains as is while we're in fmLoan. We'll return to fmLoanrequest when fmLoan closes.
c) The fmLoan form is displayed (and shows the data in the object - all that is working well).
d) When closing fmLoan, a procedure is called to Free the Loan object - if it is assigned (see line 10 of second code snippet). That seems to work OK (no error).
e) The 'Invalid Pointer Operation' error occurs when the code in line 14 below is executed: ( if Assigned(oLoan) then oLoan.Free; ).
I had added this line to make sure the object would be freed if fmLoan didn't for some reason deal with it. I realize that the object has been Freed by this time, but shouldn't the 'if Assgned()' prevent unnecessary freeing of the object?
Partial code from form fmLoanRequest (I added some line numbers for reference)
1 // In form fmLoanRequest
2 // Create new Loan Object (from a Loan sub-class as it happens)
3 // Create the object here; Object address will be passed to fmLoan later for handling.
4 oLoan := TStorageLoan.Create(iNewLoanID);
5 ...
6 ...
7 fmLoan.oLoan := oLoan; // pass the address to the other form
8 fmLoan.show;
9 // User would click the 'btnClose' at this point. See event code below.
10 ...
11 ...
12 procedure TfmLoanRequests.btnCloseClick(Sender: TObject);
13 begin
14 if Assigned(oLoan) then oLoan.Free; // <--- ERROR HERE
15 fmLoanRequests.Close;
16 end;
Partial code from form fmLoan (I added some line numbers for reference)
1 //Form fmLoan
2 ...
3 public
4 oLoan : TLoan;
5 ...
6 // In form fmLoan, I call the following upon closing the Form
7 // in the OnClick event of the 'btnClose' button.
8 Procedure TfmLoan.Clear_Loan_Object;
9 begin
10 if Assigned(oLoan) then oLoan.Free; // <-- THIS WORKS FINE
11 end;
Should I try a different approach?
Should I just remove that line (line 14 - first code snippet) and hope for the best. That's not at all my philosophy on proper coding!
Am I going at it the wrong way?
Note: I obviously don't use pointers.
Any help would be appreciated!
It is clear that you are freeing the Loan object twice, that is why you are getting the error. You need to free it only once. fmLoanRequests creates the object, but you say it can be closed before fmLoan is closed, so fmLoan should take ownership of the object and free it when fmLoan is closed. Don't free the object at all when fmLoanRequest is closed.
The alternative is to define an ILoan interface that TLoan and descendants implement, and then pass ILoan around instead of TLoan directly. Interfaces are reference counted, so the Loan object will be freed automatically, and only once, after both fmLoanRequests and fmLoan have released their references to it.
I had added this line to make sure the object would be freed if fmLoan didn't for some reason deal with it. I realize that the object has been freed by this time, but shouldn't the if Assigned() prevent unnecessary freeing of the object?
This is a key misunderstanding. Consider the following program:
{$APPTYPE CONSOLE}
var
obj: TObject = nil;
begin
Writeln(Assigned(obj));
obj := TObject.Create;
Writeln(Assigned(obj));
obj.Free;
Writeln(Assigned(obj));
Readln;
end.
This outputs the following:
FALSE
TRUE
TRUE
Note that the final line of output is TRUE. In other words, when you destroy an object be calling its Free method, the reference variable is not set to nil.
Your mistake is that you believe that Assigned tests whether or not the object has been destroyed. It does not do that. It merely tests whether or not the reference variable is nil or not. Let's look at the code again in more detail.
obj := TObject.Create;
Here we create a new object, allocated on the heap, with a call to TObject.Create. We also assign to obj the address or reference of that object. After this line executes, obj is a reference variable that contains the address of a valid object.
obj.Free;
This destroys the object to which obj refers. The destructor is run, and then the memory is destroyed. After this line executes, the object has been destroyed, but obj still refers to that destroyed and now invalid piece of memory. That is why Assigned(obj) yields true.
Note: I obviously don't use pointers.
That's an interesting point. In fact, you are using pointers whenever you use a reference variable. Although the language hides this fact, an object reference variable is nothing more than a pointer to memory allocated on the heap. We use the terminology reference rather than pointer but really these are the same things. They behave identically, the assignment operator has identical semantics, you are still subject to potential for leaking, double freeing, access after free, and all the other pitfalls of pointers. So although you do not explicitly use pointers, it still pays to think about object reference variables as though they are pointers.
I wrote a detailed answer on this whole topic for a different question. I suggest that you read that answer: https://stackoverflow.com/a/8550628/505088.
One of the points that you will take away is that code like
if Assigned(oLoan) then
oLoan.Free;
is pointless. The Free method also checks whether or not the object reference is nil. That line of code is in effect expanded to:
if Assigned(oLoan) then
if Assigned(oLoan) then
oLoan.Destroy;
So, instead of
if Assigned(oLoan) then
oLoan.Free;
you should simply write
oLoan.Free;
Now, back to the access violation. I think that it should now be obvious that you are attempting to destroy an object that has already been destroyed. You must not do that. You'll need to re-examine your lifetime management. Reasoning like "if fmLoan didn't for some reason deal with it" really are not good enough. You need to be 100% sure about lifetime management. You need to make sure that your objects are destroyed exactly once. Not being able to see all your code I don't want to make specific recommendations.
One pattern that is sometimes useful is to set the object reference to nil when you destroy the object. If the object may be destroyed in multiple places then this technique can be used to make sure that that you don't attempt to destroy it twice. You may even use the FreeAndNil helper function. However, it is worth stressing that if you are unsure of whether or not you have already destroyed the object, then that is usually indicative of poor design. If you find yourself want to add calls to Free to act "just in case" then you are almost certainly doing something seriously wrong.
I read many posts on forum about pointers, Assigned function, Free function, FreeAndNil function, etc... I already know Free function don't remove the pointer reference to an object assigned and FreeAndNil does it... All posts I read treat this subject considering Create method already was executed, or in other words, considering an object already created.
My question is: Why Assigned function returns true for a uninitialized object variable ?
Follow an example:
procedure TForm1.FormCreate(Sender: TObject);
var
Qry: TADOQuery;
begin
if Assigned(Qry) then
ShowMessage('Assigned')
else
ShowMessage('Unassigned');
Qry := TADOQuery.Create(nil);
if Assigned(Qry) then
ShowMessage('Assigned')
else
ShowMessage('Unassigned');
end;
That example displays 'Assigned' twice!
Conclusion: Immediately after Qry has been declared and before its create method has been executed the pointer to Qry isn't NIL !
If I put Qry := nil; at the first line into procedure above everything works fine... it displays 'Unassigned' and 'Assigned'.
Why??
Is there any safe way to know if a class variable already has its create method executed?
Your variable is a local variable and so is not initialized. It could contain any value.
The documentation says:
On the Win32 platform, the contents of
a local variable are undefined until a value is assigned to
them.
Note that, as an implementation detail, some types are managed and even local variables of managed types are initialized. Examples of managed types include: strings, interfaces, dynamic arrays, anonymous types and variants.
You ask:
Is there any safe way to know if a class variable already has its create method executed?
If that variable is a local variable, the answer is no. The onus falls to you the programmer. In practice it is seldom an issue because good code has short procedures which makes it harder for you to slip up. And even if you do the compiler will invariably warn you.
Other types of variables like class fields and global variables are initialized.
Because when creating a pointer, it cames with whatever garbage value was in that memory position. If you want to write NIL in it, it takes some CPU cycles, and I think it's not automatically done by Delphi because you may want something faster. In your example, why assign NIL to a variable, if soon afterwards you're going to put another value in it?
From the documentation of the Assigned function (emphasis mine):
Use Assigned to determine whether the pointer or procedure referenced by P is nil. P must be a variable reference of a pointer or procedural type. Assigned(P) corresponds to the test P<> nil for a pointer variable, and #P <> nil for a procedural variable.
Assigned returns false if P is nil, true otherwise.
Note: Assigned can't detect a dangling pointer--that is, one that isn't nil but no longer points to valid data. For example, in the code example for Assigned, Assigned won't detect the fact that P isn't valid.
The Assigned function is effectively implemented as:
function Assigned(const P): Boolean;
begin
Result := Pointer(P) <> nil;
end;
So the function isn't really checking whether the value truly is assigned. Rather it's checking a side-effect of being assigned.
As a result the function is guaranteed to return True if it is assigned.
But behaviour is undefined if the value is uninitialised. Basically since an uninitialised value has a garbage value left over from previous operations, it might be nil, or if might not.
Another thing to note is that Assigned has no way to determine the validity of its value. E.g. The following call to Assigned returns True even though the underlying object is no longer valid.
var
LObject: TObject;
begin
LObject := TObject.Create;
LObject.Free;
if Assigned(LObject) then ShowMessage('Still assigned!?');
end;
EDIT: Addendum
In response to the second part of your question.
Is there any safe way to know if a class variable already has its create method executed?
There is no safe way to determine if an object instance has been created. (There's also no way to reliably confirm that it hasn't already been destroyed.)
However, there are conventions (and good practices) you can follow to help you on the way.
First note that you should only be "unsure" if something was created if it's a deliberate feature of that piece of code. E.g. If you intend an object to be "lazy initialised".
What I'm trying to say here is: Never check Assigned just because you're worried that there might be a bug that prevents it from being assigned.
Not only is this impossible to do reliably, but you overcomplicate your code... Which increases the chance of bugs.
Also if you find something is unexpectedly not Assigned, then what can you do about it? Ignoring it would simply be pointless. Also, it's no good saying: "Ok, then I'll create the object". Because then you're duplicating creation logic in multiple places.
Basically you should try to make every part of your program correct - not have your program try to double-check itself everywhere.
So now that we're (hopefully) agreed that you only check if something is created if you've deliberately chosen that being created is optional. You do this as follows:
At first opportunity, ensure the variable/field reference is initialised to nil. So then it's guranteed to be assigned a value which means the object is not created. (Yes, the naming is a bit warped.)
You can set the vairable/field reference to a new instance of an object or set it by copying another reference of an already existing object. (Note the existing refernce might also be nil, but that doesn't cause any problems.)
If you ever destroy the object (or even just want to stop using it from that reference), set your variable/field reference to nil again.
NOTE: Delphi already initialises the member fields of a new class. So those won't need special attention.
One nice thing about anonymous methods is that I can use variables that are local in the calling context. Is there any reason why this does not work for out-parameters and function results?
function ReturnTwoStrings (out Str1 : String) : String;
begin
ExecuteProcedure (procedure
begin
Str1 := 'First String';
Result := 'Second String';
end);
end;
Very artificial example of course, but I ran into some situations where this would have been useful.
When I try to compile this, the compiler complains that he "cannot capture symbols". Also, I got an internal error once when I tried to do this.
EDIT I just realized that it works for normal parameters like
... (List : TList)
Isn't that as problematic as the other cases? Who guarantees that the reference is still pointing to an alive object whenever the anonymous method is executed?
Var and out parameters and the Result variable cannot be captured because the safety of this operation cannot be statically verified. When the Result variable is of a managed type, such as a string or an interface, the storage is actually allocated by the caller and a reference to this storage is passed as an implicit parameter; in other words, the Result variable, depending on its type, is just like an out parameter.
The safety cannot be verified for the reason Jon mentioned. The closure created by an anonymous method can outlive the method activation where it was created, and can similarly outlive the activation of the method that called the method where it was created. Thus, any var or out parameters or Result variables captured could end up orphaned, and any writes to them from inside the closure in the future would corrupt the stack.
Of course, Delphi does not run in a managed environment, and it doesn't have the same safety restrictions as e.g. C#. The language could let you do what you want. However, it would result in hard to diagnose bugs in situations where it went wrong. The bad behaviour would manifest itself as local variables in a routine changing value with no visible proximate cause; it would be even worse if the method reference were called from another thread.
This would be fairly hard to debug. Even hardware memory breakpoints would be a relatively poor tool, as the stack is modified frequently. One would need to turn on the hardware memory breakpoints conditionally upon hitting another breakpoint (e.g. upon method entry). The Delphi debugger can do this, but I would hazard a guess that most people don't know about the technique.
Update: With respect to the additions to your question, the semantics of passing instance references by value is little different between methods that contain a closure (and capture the paramete0 and methods that don't contain a closure. Either method may retain a reference to the argument passed by value; methods not capturing the parameter may simply add the reference to a list, or store it in a private field.
The situation is different with parameters passed by reference because the expectations of the caller are different. A programmer doing this:
procedure GetSomeString(out s: string);
// ...
GetSomeString(s);
would be extremely surprised if GetSomeString were to keep a reference to the s variable passed in. On the other hand:
procedure AddObject(obj: TObject);
// ...
AddObject(TObject.Create);
It is not surprising that AddObject keeps a reference, since the very name implies that it's adding the parameter to some stateful store. Whether that stateful store is in the form of a closure or not is an implementation detail of the AddObject method.
The problem is that your Str1 variable is not "owned" by ReturnTwoStrings, so that your anonymous method cannot capture it.
The reason it cannot capture it, is that the compiler does not know the ultimate owner (somewhere in the call stack towards calling ReturnTwoStrings) so it cannot determine where to capture it from.
Edit: (Added after a comment of Smasher)
The core of anonymous methods is that they capture the variables (not their values).
Allen Bauer (CodeGear) explains a bit more about variable capturing in his blog.
There is a C# question about circumventing your problem as well.
The out parameter and return value are irrelevant after the function returns - how would you expect the anonymous method to behave if you captured it and executed it later? (In particular, if you use the anonymous method to create a delegate but never execute it, the out parameter and return value wouldn't be set by the time the function returned.)
Out parameters are particularly difficult - the variable that the out parameter aliases may not even exist by the time you later call the delegate. For example, suppose you were able to capture the out parameter and return the anonymous method, but the out parameter is a local variable in the calling function, and it's on the stack. If the calling method then returned after storing the delegate somewhere (or returning it) what would happen when the delegate was finally called? Where would it write to when the out parameter's value was set?
I'm putting this in a separate answer because your EDIT makes your question really different.
I'll probably extend this answer later as I'm in a bit of a hurry to get to a client.
Your edit indicates you need to rethink about value types, reference types and the effect of var, out, const and no parameter marking at all.
Let's do the value types thing first.
The values of value types live on the stack and have a copy-on-assignment behaviour.
(I'll try to include an example on that later).
When you have no parameter marking, the actual value passed to a method (procedure or function) will be copied to the local value of that parameter inside the method. So the method does not operate on the value passed to it, but on a copy.
When you have out, var or const, then no copy takes place: the method will refer to the actual value passed. For var, it will allow to to change that actual value, for const it will not allow that. For out, you won't be able to read the actual value, but still be able to write the actual value.
Values of reference types live on the heap, so for them it hardly matters if you have out, var, const or no parameter marking: when you change something, you change the value on the heap.
For reference types, you still get a copy when you have no parameter marking, but that is a copy of a reference that still points to the value on the heap.
This is where anonymous methods get complicated: they do a variable capture.
(Barry can probably explain this even better, but I'll give it a try)
In your edited case, the anonymous method will capture the local copy of the List. The anonymous method will work on that local copy, and from a compiler perspective everything is dandy.
However, the crux of your edit is the combination of 'it works for normal parameters' and 'who guarantees that the reference is still pointing to an alive object whenever the anonymous method is executed'.
That is always a problem with reference parameters, no matter if you use anonymous methods or not.
For instance this:
procedure TMyClass.AddObject(Value: TObject);
begin
FValue := Value;
end;
procedure TMyClass.DoSomething();
begin
ShowMessage(FValue.ToString());
end;
Who guarantees that when someone calls DoSomething, that the instance where FValue points to still exists?
The answer is that you must guarantee this yourself by not calling DoSomething when the instance to FValue has died.
The same holds for your edit: you should not call the anonymous method when the underlying instance has died.
This is one of the areas where reference counted or garbage collected solutions make life easier: there the instance will be kept alive until the last reference to it has gone away (which might cause instance to live longer than you originally anticipated!).
So, with your edit, your question actually changes from anonymous methods to the implications of using reference typed parameters and lifetime management in general.
Hopefully my answer helps you going in that area.
--jeroen
In my Delphi7 this code
var MStr: TMemoryStream;
...
FreeAndNil(MStr);
MStr.Size:=0;
generates an AV: Access violation at address 0041D6D1 in module 'Project1.exe'. Read of address 00000000.
But somebody insists that it should not raise any exception, no matter what. He also says that his Delphi 5 indeed raises no exceptions. He calls this a “stale pointer bug”.
In other words he says that FreeAndNil cannot be used as debugger to detect a double attempt to free an object or to use a freed object.
Can anybody enlighten me? Should this raise and error (always/randomly) or the program should run over this bug without problems?
Thanks
I ask this because I believe I have a "double free object" or "free and re-access" bug in my program. How can I fill the memory allocated to an object with zeros AFTER I freed the object? I want this way to detect where the bug is, by getting and AV.
Initially, I hoped that if I set the object to FreeAndNil, I will ALWAYS get an AV when trying to re-access it.
It's always wrong to use methods or properties of a null reference, even if it appears to work sometimes.
FreeAndNil indeed cannot be used to detect double frees. It is safe to call FreeAndNil on an already-nil variable. Since it's safe, it doesn't help you detect anything.
This is not a stale-pointer bug. This is a null-reference bug. A stale-pointer bug is when you have freed an object but not cleared all variables that referenced it. Then the variable still holds the old address of the object. Those are very hard to detect. You can get such a bug like this:
MStr := TMemoryStream.Create;
MStr.Free;
MStr.Size := 0;
You can also get one like this:
MStr := TMemoryStream.Create;
OtherStr := MStr;
FreeAndNil(MStr);
OtherStr.Size := 0;
Using MStr.Size after you have freed the object MStr referenced is an error, and it should raise an exception. Whether it does raise an exception depends on the implementation. Maybe it will, and maybe it won't. It's not random, though.
If you're searching for a double-free bug, you can use the debugging aides that FastMM provides, as others have suggested as well. It works by not actually releasing the memory back to the operating system, or even back to Delphi's internal free-memory pool. Instead, it writes known-bad data into the object's memory space, so when you see those values, you'll know you're reading from something that you already freed. It also modifies the object's VMT so that the next time you call a virtual method on that object reference, you'll get a predictable exception, and it will even tell you which supposedly freed object you tried to use. When you attempt to free the object again, it can tell you not only that you already freed it, but also where it was freed the first time (with a stack trace), and where it was allocated. It also collects that information to report about memory leaks, where you freed an object less than one time instead of more.
There are also habits you can use to avoid the issue for future code:
Reduce the use of global variables. A global variable could be modified by any code throughout the program, forcing you to wonder whenever you use it, "Is this variable's value still valid, or did some other code free it already?" When you limit the scope of a variable, you reduce the amount of code you have to consider in your program when looking for reasons a variable doesn't have the value you expect.
Be clear about who owns an object. When there are two pieces of code that have access to the same object, you need to know which of those pieces of code owns the object. They might each have a different variable for referencing the object, but there's still just one object there. If one piece of code calls FreeAndNil on its variable, that still leave's the other code's variable unchanged. If that other code thinks it owns the object, then you're in trouble. (This concept of owner is not necessarily tied to the TComponent.Owner property. There doesn't need to be an object that owns it; it could be a general subsystem of your program.)
Don't keep persistent references to objects you don't own. If you don't keep long-lived references to an object, then you don't have to worry about whether those references are still valid. The only persistent reference should be in the code that owns the object. Any other code that needs to use that object should receive a reference as an input parameter, use the object, and then discard the reference when it returns its result.
From what I am seeing, this code should always result in an error. FreeAndNil explicitly sets that passed value to Nil (aka 0), so you should absolutely get an access violation when trying to dereference the object.
Just to complicate the issue:
If the method you call is a static (not virtual) method and it does not call any virtual methods itself nor does it access any fields of the object, you will not get an access violation even if the object reference has been set to NIL.
The reason for this is that the access violation is caused by dereferencing the self pointer (in this case NIL), but that only happens when accessing a field or the object's VMT for calling a virtual method.
This is just an exception to the rule that you cannot call methods of an NIL object reference that I'd like to mention here.
If you set a pointer to nil, you shouldn't be able to use it any more. But if you have another pointer to the same object, you can use it without getting an AV, because this pointer still points to the object address and not to nil.
Moreover, freeing an object do not clear the memory used by the that object. It just marks it as not in use. Thats the reason you want get an AV. If the freed memory is allocated for another object, you will get a AV, because it no longer contains data that seems valid.
FastMM4 has some settings that you can use while debugging, that will detect such conditions. From the FsatMM4Options.inc:
{Set the following option to do extensive checking of all memory blocks. All
blocks are padded with both a header and trailer that are used to verify the
integrity of the heap. Freed blocks are also cleared to to ensure that they
cannot be reused after being freed. This option slows down memory operations
dramatically and should only be used to debug an application that is
overwriting memory or reusing freed pointers. Setting this option
automatically enables CheckHeapForCorruption and disables ASMVersion.
Very important: If you enable this option your application will require the
FastMM_FullDebugMode.dll library. If this library is not available you will
get an error on startup.}
{$define FullDebugMode}
Another quote from the same file:
FastMM always catches attempts to free the same memory block twice...
As delphi uses FastMM from Delphi 2007 (2006 ?), you should get an error if you try to doublefree an object.
Thomas Mueller: have you tried virtual class methods? A constructor is sort of a virtual method but you call it against the type - not the instance. This means that even some specific virtual methods will not cause AV on a null-reference :D
Vegar: You couldn't be more right! FastMM is the best ever ever ever tool that helped me tracking down this kind of bugs.
The EurekaLog Blog had a great post on this in April 2009:
Why should you always use FreeAndNil instead of Free.