There's a bug in delphi 6 which you can find some reference online for when you import a tlb the order of the parameters in an event invocation is reversed. It is reversed once in the imported header and once in TServerEventDIspatch.Invoke.
you can find more information about it here:
http://cc.embarcadero.com/Item/16496
somewhat related to this issue there appears to be a memory leak in TServerEventDispatch.Invoke with a parameter of a Variant of type Var_Array (maybe others, but this is the more obvious one i could see). The invoke code copies the args into a VarArray to be passed to the event handler and then copies the VarArray back to the args after the call, relevant code pasted below:
// Set our array to appropriate length
SetLength(VarArray, ParamCount);
// Copy over data
for I := Low(VarArray) to High(VarArray) do
VarArray[I] := OleVariant(TDispParams(Params).rgvarg^[I]);
// Invoke Server proxy class
if FServer <> nil then FServer.InvokeEvent(DispID, VarArray);
// Copy data back
for I := Low(VarArray) to High(VarArray) do
OleVariant(TDispParams(Params).rgvarg^[I]) := VarArray[I];
// Clean array
SetLength(VarArray, 0);
There are some obvious work-arounds in my case: if i skip the copying back in case of a VarArray parameter it fixes the leak. to not change the functionality i thought i should copy the data in the array instead of the variant back to the params but that can get complicated since it can hold other variants and seems to me that would need to be done recursively.
Since a change in OleServer will have a ripple effect i want to make sure my change here is strictly correct.
can anyone shed some light on exactly why memory is being leaked here? I can't seem to look up the callstack any lower than TServerEventDIspatch.Invoke (why is that?)
I imagine that in the process of copying the Variant holding the VarArray back to the param list it added a reference to the array thus not allowing it to be release as normal but that's just a rough guess and i can't track down the code to back it up.
Maybe someone with a better understanding of all this could shed some light?
Interestingly enough, i think the solution was in the link i provided in the question, but i didn't understand the implication until digging into it a bit more.
A few things to clarify:
When a variant containing an array is assigned from the VarArray back to the Params, a copy is made. This is explained within the delphi help pages.
Assigning over an existing variant will definitely free the memory associated with the previous value of the Variant, so the array contained by the variant prior to the assignment would have been freed on assignment.
VarClear will free the memory associated with the variant and tests show that a VarClear on the variant hold in the Params after the assignment will in fact remove the memory leak.
It appears that the issue has to do with the indiscriminate write back of param values. The particular event i'm dealing with does not have any parameters marked as var, so the COM object is not expecting changes to invocation param to free new memory that's been allocated.
Roughly the COM object allocated an array, invokes the event and then frees it's own memory after the event. The Oleserver however allocates some new memory when it copied the array param back to the param list which the COM object wouldn't even know about since it didn't pass anything by reference and is not expecting changes to its params. There must be some additional marshalling magic there that i'm neglecting, if anyone knows the details i'd definitely be curious.
The TVariantArg's vt field has a flag to indicate whether it is passed by value or reference. As far as i can discern, we should only be copying the value back if the param is marked as being passed by reference.
Furthermore it may be necessary to do more than just assign the variant if this is in fact pass by reference, although that could be taken care of by the marshalling, still not sure about this part.
The solution for now is to change the code to:
if ((TDispParams(Params).rgvarg^[I].vt and VT_BYREF) <> 0) then begin
OleVariant(TDispParams(Params).rgvarg^[I]) := VarArray[I];
end;
Related
I use Delphi Seattle.
My problem occurs when I attempt to free an object I created.
I searched in this site (and other sites as well) for answers already posted for this question, but they all are a bit different. According to those discussions, my code should work, but obviously something isn't quite right.
So, I need help...
Flow of execution:
a) in form fmLoanRequest, I create an object based on Class TStorageLoan (a Sub-class of TLoan). The Constructor loads all kinds of values into some of the object's attributes (now shown here).
b) Later, I pass the object's address to another form (fmLoan) to an appropriate public variable. fmLoan is the form where all the user's dealings with the contents of Loan happen. Note that fmLoanRequest remains as is while we're in fmLoan. We'll return to fmLoanrequest when fmLoan closes.
c) The fmLoan form is displayed (and shows the data in the object - all that is working well).
d) When closing fmLoan, a procedure is called to Free the Loan object - if it is assigned (see line 10 of second code snippet). That seems to work OK (no error).
e) The 'Invalid Pointer Operation' error occurs when the code in line 14 below is executed: ( if Assigned(oLoan) then oLoan.Free; ).
I had added this line to make sure the object would be freed if fmLoan didn't for some reason deal with it. I realize that the object has been Freed by this time, but shouldn't the 'if Assgned()' prevent unnecessary freeing of the object?
Partial code from form fmLoanRequest (I added some line numbers for reference)
1 // In form fmLoanRequest
2 // Create new Loan Object (from a Loan sub-class as it happens)
3 // Create the object here; Object address will be passed to fmLoan later for handling.
4 oLoan := TStorageLoan.Create(iNewLoanID);
5 ...
6 ...
7 fmLoan.oLoan := oLoan; // pass the address to the other form
8 fmLoan.show;
9 // User would click the 'btnClose' at this point. See event code below.
10 ...
11 ...
12 procedure TfmLoanRequests.btnCloseClick(Sender: TObject);
13 begin
14 if Assigned(oLoan) then oLoan.Free; // <--- ERROR HERE
15 fmLoanRequests.Close;
16 end;
Partial code from form fmLoan (I added some line numbers for reference)
1 //Form fmLoan
2 ...
3 public
4 oLoan : TLoan;
5 ...
6 // In form fmLoan, I call the following upon closing the Form
7 // in the OnClick event of the 'btnClose' button.
8 Procedure TfmLoan.Clear_Loan_Object;
9 begin
10 if Assigned(oLoan) then oLoan.Free; // <-- THIS WORKS FINE
11 end;
Should I try a different approach?
Should I just remove that line (line 14 - first code snippet) and hope for the best. That's not at all my philosophy on proper coding!
Am I going at it the wrong way?
Note: I obviously don't use pointers.
Any help would be appreciated!
It is clear that you are freeing the Loan object twice, that is why you are getting the error. You need to free it only once. fmLoanRequests creates the object, but you say it can be closed before fmLoan is closed, so fmLoan should take ownership of the object and free it when fmLoan is closed. Don't free the object at all when fmLoanRequest is closed.
The alternative is to define an ILoan interface that TLoan and descendants implement, and then pass ILoan around instead of TLoan directly. Interfaces are reference counted, so the Loan object will be freed automatically, and only once, after both fmLoanRequests and fmLoan have released their references to it.
I had added this line to make sure the object would be freed if fmLoan didn't for some reason deal with it. I realize that the object has been freed by this time, but shouldn't the if Assigned() prevent unnecessary freeing of the object?
This is a key misunderstanding. Consider the following program:
{$APPTYPE CONSOLE}
var
obj: TObject = nil;
begin
Writeln(Assigned(obj));
obj := TObject.Create;
Writeln(Assigned(obj));
obj.Free;
Writeln(Assigned(obj));
Readln;
end.
This outputs the following:
FALSE
TRUE
TRUE
Note that the final line of output is TRUE. In other words, when you destroy an object be calling its Free method, the reference variable is not set to nil.
Your mistake is that you believe that Assigned tests whether or not the object has been destroyed. It does not do that. It merely tests whether or not the reference variable is nil or not. Let's look at the code again in more detail.
obj := TObject.Create;
Here we create a new object, allocated on the heap, with a call to TObject.Create. We also assign to obj the address or reference of that object. After this line executes, obj is a reference variable that contains the address of a valid object.
obj.Free;
This destroys the object to which obj refers. The destructor is run, and then the memory is destroyed. After this line executes, the object has been destroyed, but obj still refers to that destroyed and now invalid piece of memory. That is why Assigned(obj) yields true.
Note: I obviously don't use pointers.
That's an interesting point. In fact, you are using pointers whenever you use a reference variable. Although the language hides this fact, an object reference variable is nothing more than a pointer to memory allocated on the heap. We use the terminology reference rather than pointer but really these are the same things. They behave identically, the assignment operator has identical semantics, you are still subject to potential for leaking, double freeing, access after free, and all the other pitfalls of pointers. So although you do not explicitly use pointers, it still pays to think about object reference variables as though they are pointers.
I wrote a detailed answer on this whole topic for a different question. I suggest that you read that answer: https://stackoverflow.com/a/8550628/505088.
One of the points that you will take away is that code like
if Assigned(oLoan) then
oLoan.Free;
is pointless. The Free method also checks whether or not the object reference is nil. That line of code is in effect expanded to:
if Assigned(oLoan) then
if Assigned(oLoan) then
oLoan.Destroy;
So, instead of
if Assigned(oLoan) then
oLoan.Free;
you should simply write
oLoan.Free;
Now, back to the access violation. I think that it should now be obvious that you are attempting to destroy an object that has already been destroyed. You must not do that. You'll need to re-examine your lifetime management. Reasoning like "if fmLoan didn't for some reason deal with it" really are not good enough. You need to be 100% sure about lifetime management. You need to make sure that your objects are destroyed exactly once. Not being able to see all your code I don't want to make specific recommendations.
One pattern that is sometimes useful is to set the object reference to nil when you destroy the object. If the object may be destroyed in multiple places then this technique can be used to make sure that that you don't attempt to destroy it twice. You may even use the FreeAndNil helper function. However, it is worth stressing that if you are unsure of whether or not you have already destroyed the object, then that is usually indicative of poor design. If you find yourself want to add calls to Free to act "just in case" then you are almost certainly doing something seriously wrong.
I have this example method below
procedure ReadData(var data:TDataSet)
begin
if Assigned(data) then
data.Free;
data:=TDataSet.Create(nil);
....
end;
.....
procedure SomethingProcedure()
var
dataSet:TDataset;
begin
ReadData(dataSet);
end;
if I debugged and I place breakpoint on Assigned checking, data.Free always executed, and I saw on watch list, data always inaccessible value
My point is SomethingProcedure is access for many other procedure, and I want data parameter if it assigned (already created TDataset object), free it first, but if not assigned (not created object), free command doesn't execute, but free statement always executed, either "data" object created or not
How I can check my object already created or not
You have some issues with your code example
You declare dataset but pass data in your Init procedure
The if statement doesn't have a then in your ReadData procedure
All in all, you can not have been debugging the example you've given so I'm going to make some assumptions here.
I believe your actual problem is coming from the fact that local, not finalized variables do not get initialized to zero/nil. The dataset variable in your Init procedure contains whatever garbage happens to be at the location the variable points to.
Which variables are initialized when in Delphi?
As you don't initialize the local variable dataset (something you should always do with local variables), Assigned will return true (all it does is check for nil) and free will get called.
Most of the time and if you are lucky, the call to free will throw an AV. Worst case, it will succeed and you will have a really hard time figuring out why something is going wrong.
Edit
I assume by your edit that you mean that ReadData is called in many other procedures?
If that's the case, there really is not a lot you can (or for that matter should) do to protect you from callers passing in garbage. What you should do is fix the callers.
Fix the root cause, not the symptoms
First of all, you do not have to check your data object for assignment. It is perfectly safe to call Free method when data is pointing to nil.
As for the reason why you can not see on what the data is pointing to: you probably need to turn off the optimization for your project. To do this go to: Project > Options > Delphi Compiler > Compiling (this may vary depending on your Delphi version) and switch Optimization to False.
I haven't been able to find the answers to a couple of my Delphi memory management questions. I could test different scenarios (which I did to find out what breaks the FreeAndNil method), but its takes too long and its hard! But seriously, I would also like to know how you all (Delphi developers) handle these memory management issues.
My Questions (Feel free to pose your own I'm sure the answers to them will help me too):
Does FreeAndNil work for COM objects? My thoughts are I don't need it, but if all I need to do is set it to nil than why not stay consistent in my finally block and use FreeAndNil for everything?
Whats the proper way to clean up static arrays (myArr : Array[0..5] of TObject). I can't FreeAndNil it, so is it good enough to just set it to nil (do I need to do that after I've FreeAnNil'd each object?)?
Thanks Guys!
COM objects are referenced via Interfaces, which you don't need to do anything to free. The compiler takes care of the necessary reference-counting logic to make sure the COM object will be disposed of at the right time.
As for static arrays, (or dynamic arrays, for that matter,) they don't need to be freed by you either. If they contain objects then the objects have to be freed at the appropriate time, but the arrays don't.
Also, never use FreeAndNil on anything that's not an object reference. Using it with interfaces or other variables can corrupt memory. It's best to never use it (use Free instead) unless you're dealing with an object that you need to free and then reuse later.
First, in most situation, FreeAndNil is a bit of overkill. It's handy when you free and object's field outside it's destructor, or on a global(ugly) variable. But most of the time, just calling free is enough.
As you should know, an object variable is actually a pointer to the object's data. When you call Free, that buffer is freed (after the destructor is ran, of course), but the Object variable still points to the memory position that was just freed. It's called a "Dangling pointer". Having a dangling pointer is not a problem as long as you KNOW it's dangling in that context. For exemple:
Procedure Myproc;
var vString : TStringList;
begin
//Here, vString is "dangling"
vString := TStringList.Create;
//Here, vString is valid
try
//Do some stuff
finally
vString.Free;
end;
//Here, vString is "dangling"... But who care, it's about to go out of scope and we won't use it again.
end;
Calling FreeAndNil makes more sense on global variable where you don't know exactly when or how the variable can be freed. With that being said, there is nothing wrong in calling FreeAndNil all the time (except in very tight loops where you try to get every oz of performance).
Now, for the COM objects... Like Mason stated, they are reference counted. So if you hold the only reference to that interface, calling MyInterface := nil; will free it. But when/if the variable goes out of scope, the compiler take care of adding cleanup code to make sure the interface reference is decremented. So if you are trying to keep the memory requirement to a minimum, set the interface to nil. Otherwise, it doesn't matter that much.
As for your array... You can just call Free on every items in the list... Optionnaly set them to nil after.
Regarding static arrays, if you created the contents of the array, just free those objects you created. You don't need to do anything special to clean up the space used by myArr itself.
COM objects are automatically reference counted; as soon as variable goes out of scope or the object that has the interface pointer as a field is deleted Delphi will call _Release and the object will delete itself. You don't need to set anything to nil explicitly.
For static arrays you need to loop over them and free each object explicitly.
In my Delphi7 this code
var MStr: TMemoryStream;
...
FreeAndNil(MStr);
MStr.Size:=0;
generates an AV: Access violation at address 0041D6D1 in module 'Project1.exe'. Read of address 00000000.
But somebody insists that it should not raise any exception, no matter what. He also says that his Delphi 5 indeed raises no exceptions. He calls this a “stale pointer bug”.
In other words he says that FreeAndNil cannot be used as debugger to detect a double attempt to free an object or to use a freed object.
Can anybody enlighten me? Should this raise and error (always/randomly) or the program should run over this bug without problems?
Thanks
I ask this because I believe I have a "double free object" or "free and re-access" bug in my program. How can I fill the memory allocated to an object with zeros AFTER I freed the object? I want this way to detect where the bug is, by getting and AV.
Initially, I hoped that if I set the object to FreeAndNil, I will ALWAYS get an AV when trying to re-access it.
It's always wrong to use methods or properties of a null reference, even if it appears to work sometimes.
FreeAndNil indeed cannot be used to detect double frees. It is safe to call FreeAndNil on an already-nil variable. Since it's safe, it doesn't help you detect anything.
This is not a stale-pointer bug. This is a null-reference bug. A stale-pointer bug is when you have freed an object but not cleared all variables that referenced it. Then the variable still holds the old address of the object. Those are very hard to detect. You can get such a bug like this:
MStr := TMemoryStream.Create;
MStr.Free;
MStr.Size := 0;
You can also get one like this:
MStr := TMemoryStream.Create;
OtherStr := MStr;
FreeAndNil(MStr);
OtherStr.Size := 0;
Using MStr.Size after you have freed the object MStr referenced is an error, and it should raise an exception. Whether it does raise an exception depends on the implementation. Maybe it will, and maybe it won't. It's not random, though.
If you're searching for a double-free bug, you can use the debugging aides that FastMM provides, as others have suggested as well. It works by not actually releasing the memory back to the operating system, or even back to Delphi's internal free-memory pool. Instead, it writes known-bad data into the object's memory space, so when you see those values, you'll know you're reading from something that you already freed. It also modifies the object's VMT so that the next time you call a virtual method on that object reference, you'll get a predictable exception, and it will even tell you which supposedly freed object you tried to use. When you attempt to free the object again, it can tell you not only that you already freed it, but also where it was freed the first time (with a stack trace), and where it was allocated. It also collects that information to report about memory leaks, where you freed an object less than one time instead of more.
There are also habits you can use to avoid the issue for future code:
Reduce the use of global variables. A global variable could be modified by any code throughout the program, forcing you to wonder whenever you use it, "Is this variable's value still valid, or did some other code free it already?" When you limit the scope of a variable, you reduce the amount of code you have to consider in your program when looking for reasons a variable doesn't have the value you expect.
Be clear about who owns an object. When there are two pieces of code that have access to the same object, you need to know which of those pieces of code owns the object. They might each have a different variable for referencing the object, but there's still just one object there. If one piece of code calls FreeAndNil on its variable, that still leave's the other code's variable unchanged. If that other code thinks it owns the object, then you're in trouble. (This concept of owner is not necessarily tied to the TComponent.Owner property. There doesn't need to be an object that owns it; it could be a general subsystem of your program.)
Don't keep persistent references to objects you don't own. If you don't keep long-lived references to an object, then you don't have to worry about whether those references are still valid. The only persistent reference should be in the code that owns the object. Any other code that needs to use that object should receive a reference as an input parameter, use the object, and then discard the reference when it returns its result.
From what I am seeing, this code should always result in an error. FreeAndNil explicitly sets that passed value to Nil (aka 0), so you should absolutely get an access violation when trying to dereference the object.
Just to complicate the issue:
If the method you call is a static (not virtual) method and it does not call any virtual methods itself nor does it access any fields of the object, you will not get an access violation even if the object reference has been set to NIL.
The reason for this is that the access violation is caused by dereferencing the self pointer (in this case NIL), but that only happens when accessing a field or the object's VMT for calling a virtual method.
This is just an exception to the rule that you cannot call methods of an NIL object reference that I'd like to mention here.
If you set a pointer to nil, you shouldn't be able to use it any more. But if you have another pointer to the same object, you can use it without getting an AV, because this pointer still points to the object address and not to nil.
Moreover, freeing an object do not clear the memory used by the that object. It just marks it as not in use. Thats the reason you want get an AV. If the freed memory is allocated for another object, you will get a AV, because it no longer contains data that seems valid.
FastMM4 has some settings that you can use while debugging, that will detect such conditions. From the FsatMM4Options.inc:
{Set the following option to do extensive checking of all memory blocks. All
blocks are padded with both a header and trailer that are used to verify the
integrity of the heap. Freed blocks are also cleared to to ensure that they
cannot be reused after being freed. This option slows down memory operations
dramatically and should only be used to debug an application that is
overwriting memory or reusing freed pointers. Setting this option
automatically enables CheckHeapForCorruption and disables ASMVersion.
Very important: If you enable this option your application will require the
FastMM_FullDebugMode.dll library. If this library is not available you will
get an error on startup.}
{$define FullDebugMode}
Another quote from the same file:
FastMM always catches attempts to free the same memory block twice...
As delphi uses FastMM from Delphi 2007 (2006 ?), you should get an error if you try to doublefree an object.
Thomas Mueller: have you tried virtual class methods? A constructor is sort of a virtual method but you call it against the type - not the instance. This means that even some specific virtual methods will not cause AV on a null-reference :D
Vegar: You couldn't be more right! FastMM is the best ever ever ever tool that helped me tracking down this kind of bugs.
The EurekaLog Blog had a great post on this in April 2009:
Why should you always use FreeAndNil instead of Free.
Is there a way to be sure we hold a useable reference to an object i.e. being sure it has not been already freed leaving that non nil reference dangling.
If you're using FastMM4 as your Memory Manager, you can check that the class is not TFreeObject.
Or, in a more standard case, use a routine that will verify that your object is what it says it is by checking the class VMT.
There have been such ValidateObj functions hannging around for some time (by Ray Lischner and Hallvard Vassbotn: http://hallvards.blogspot.com/2004/06/hack-6checking-for-valid-object.html)
Here's another:
function ValidateObj(Obj: TObject): Pointer;
// see { Virtual method table entries } in System.pas
begin
Result := Obj;
if Assigned(Result) then
try
if Pointer(PPointer(Obj)^) <> Pointer(Pointer(Cardinal(PPointer(Obj)^) + Cardinal(vmtSelfPtr))^) then
// object not valid anymore
Result := nil;
except
Result := nil;
end;
end;
Update: A bit of caution... The above function will ensure that the result is either nil or a valid non nil Object. It does not guarantee that the Obj is still what you think it is, in case where the Memory Manager has already reallocated that previously freed memory.
No. Unless you use something like reference counting or a garbage collector to make sure no object will be freeed before they have zero references.
Delphi can do reference counting for you if you use interfaces. Of course Delphi for .Net has a gargage collector.
As mentioned you could use the knowledege of Delphi or the memory manager internals to check for valid pointers or objects, but they are not the only ones that can give you pointers. So you can't cover all pointers even with those methods. And there also is a chance that your pointer happens to be valid again, but given to somebody else. So it is not the pointer you are looking for. Your design should not rely on them. Use a tool to detect any reference bugs you make.
Standard, no...
That's why VCL components can register themselves to be notified of the destruction of an object, so that they can remove the reference from there internal list of components or just reset their property.
So if you'd want to make sure you haven't got any invalid references their are two options:
Implement a destruction notification handler which every class can subscribe to.
Fix your code in a way that the references aren't spread around trough different object. You could for instance only provide the access to the reference via a property of another object. And instead of copying the reference to a private field you access the property of the other object.
As others have said, no definitive way, but if you manage the ownership well, then the FreeAndNil routine will ensure that your variable is nil if it doesn't point to anything.
It's usually not a good idea to check a reference is valid anyway. If a reference is not valid, your program will crash at the place where it is using the invalid reference. Otherwise the invalid reference might survive longer and debugging becomes harder.
Here are some references to why it's better to crash on an invalid reference. (They talk about pointers in Win32, but the ideas are still relevant):
IsBadXxxPtr should really be called CrashProgramRandomly
Should I check the parameters to my function?
Unfortunately there is no way to 100% guarantee that a pointer to anything is still valid, except by meticolously writing the correct code.
With the usage of interface references (instead of object references) it is possible to avoid these invalid pointer problems because there is no explicit call to Free in your code anymore.