I am wondering what is the lifespan/moment where the memory is reclaimed for anonymous object like this one:
type Cell_Type is abstract tagged limited record
Where : Coordinates_Array; -- 2D array
Contents : access String;
end CELl_TYPE;
Cell_type is meant to created through allocators, too.
I know the rule (or think so, correct me if I'm wrong) for named access types, that is that the pool is reclaimed when the type(not subtype)'s scope of definition ends. But what about anonymous access objects ?
The rules says:
The storage pool used for an allocator of an anonymous access type should be determined as follows:
25.1/2 : [coextensions, I got that one]
25.2/2: For other access discriminants and access parameters, the storage pool should be created at the point of the allocator, and be reclaimed when the allocated object becomes inaccessible; [ I don't think it matters here... My case doesn't seem to be an access discriminant, even though the syntax seems similar]
25.3/3 [access returning functions, doesn't matter]
25.4/2: Otherwise, a default storage pool should be created at the point where the anonymous access type is elaborated; such a storage pool need not support deallocation of individual objects.
So ?
When is "Contents" deallocated, then ?
A: At the program very end ?
Or B: when the type "cell_type" is out of scope ? Which, for a library level definition, would mean at the end of the scope of instantiation of library... which would mean A, for my case.
Related
In another question, I found out that the Assigned() function is identical to Pointer <> nil. It has always been my understanding that Assigned() was detecting these dangling pointers, but now I've learned it does not. Dangling Pointers are those which may have been created at one point, but have since been free'd and haven't been assigned to nil yet.
If Assigned() can't detect dangling pointers, then what can? I'd like to check my object to make sure it's really a valid created object before I try to work with it. I don't use FreeAndNil as many recommend, because I like to be direct. I just use SomeObject.Free.
Access Violations are my worst enemy - I do all I can to prevent their appearance.
If you have an object variable in scope and it may or may not be a valid reference, FreeAndNil is what you should be using. That or fixing your code so that your object references are more tightly managed so it's never a question.
Access Violations shouldn't be thought of as an enemy. They're bugs: they mean you made a mistake that needs fixed. (Or that there's a bug in some code you're relying on, but I find most often that I'm the one who screwed up, especially when dealing with the RTL, VCL, or Win32 API.)
It is sometimes possible to detect when the address a pointer points to resides in a memory block that is on the heap's list of freed memory blocks. However, this requires comparing the pointer to potentially every block in the heap's free list which could contain thousands of blocks. So, this is potentially a computationally intensive operation and something you would not want to do frequently except perhaps in a severe diagnostic mode.
This technique only works while the memory block that the pointer used to point to continues to sit in the heap free list. As new objects are allocated from the heap, it is likely that the freed memory block will be removed from the heap free list and put back into active play as the home of a new, different object. The original dangling pointer still points to the same address, but the object living at that address has changed. If the newly allocated object is of the same (or compatible) type as the original object now freed, there is practically no way to know that the pointer originated as a reference to the previous object. In fact, in this very special and rare situation, the dangling pointer will actually work perfectly well. The only observable problem might be if someone notices that the data has changed out from under the pointer unexpectedly.
Unless you are allocating and freeing the same object types over and over again in rapid succession, chances are slim that the new object allocated from that freed memory block will be the same type as the original. When the types of the original and the new object are different, you have a chance of figuring out that the content has changed out from under the pointer. However, to do that you need a way to know the type of the original object that the pointer referred to. In many situations in native compiled applications, the type of the pointer variable itself is not retained at runtime. A pointer is a pointer as far as the CPU is concerned - the hardware knows very little of data types. In a severe diagnostic mode it's conceivable that you could build a lookup table to associate every pointer variable with the type allocated and assigned to it, but this is an enormous task.
That's why Assigned() is not an assertion that the pointer is valid. It just tests that the pointer is not nil.
Why did Borland create the Assigned() function to begin with? To further hide pointerisms from novice and occasional programmers. Function calls are easier to read and understand than pointer operations.
The bottom line is that you should not be attempting to detect dangling pointers in code. If you are going to refer to pointers after they have been freed, set the pointer to nil when you free it. But the best approach is not to refer to pointers after they have been freed.
So, how do you avoid referring to pointers after they have been freed? There are a couple of common idioms that get you a long way.
Create objects in a constructor and destroy them in the destructor. Then you simply cannot refer to the pointer before creation or after destruction.
Use a local variable pointer that is created at the beginning of the function and destroyed as the last act of the function.
One thing I would strongly recommend is to avoid writing if Assigned() tests into your code unless it is expected behaviour that the pointer may not be created. Your code will become hard to read and you will also lose track of whether the pointer being nil is to be expected or is a bug.
Of course we all do make mistakes and leave dangling pointers. Using FreeAndNil is one cheap way to ensure that dangling pointer access is detected. A more effective method is to use FastMM in full debug mode. I cannot recommend this highly enough. If you are not using this wonderful tool, you should start doing so ASAP.
If you find yourself struggling with dangling pointers and you find it hard to work out why then you probably need to refactor the code to fit into one of the two idioms above.
You can draw a parallel with array indexing errors. My advice is not to check in code for validity of index. Instead use range checking and let the tools do the work and keep the code clean. The exception to this is where the input comes from outside your program, e.g. user input.
My parting shot: only ever write if Assigned if it is normal behaviour for the pointer to be nil.
Use a memory manager, such as FastMM, that provides debugging support, in particular to fill a block of freed memory with a given byte pattern. You can then dereference the pointer to see if it points at a memory block that starts with the byte pattern, or you can let the code run normallly ad raise an AV if it tries to access a freed memory block through a dangling pointer. The AV's reported memory address will usually be either exactly as, or close to, the byte pattern.
Nothing can find a dangling (once valid but then not) pointer. It's your responsibility to either make sure it's set to nil when you free it's content, or to limit the scope of the pointer variable to only be available within the scope it's valid. (The second is the better solution whenever possible.)
The core point is that the way how objects are implemented in Delphi has some built-in design drawbacks:
there is no distinction between an object and a reference to an object. For "normal" variables, say a scalar (like int) or a record, these two use cases can be well told apart - there's either a type Integer or TSomeRec, or a type like PInteger = ^Integer or PSomeRec = ^TSomeRec, which are different types. This may sound like a neglectable technicality, but it isn't: a SomeRec: TSomeRec denotes "this scope is the original owner of that record and controls its lifecycle", while SomeRec: PSomeRec tells "this scope uses a transient reference to some data, but has no control over the record's lifecycle. So, as dumb it may sound, for objects there's virtually no one who has denotedly control over other objects' lifecycles. The result is - surprise - that the lifecycle state of objects may in certain situations be unclear.
an object reference is just a simple pointer. Basically, that's ok, but the problem is that there's sure a lot of code out there which treats object references as if they were a 32bit or 64bit integer number. So if e.g. Embarcadero wanted to change the implementation of an object reference (and make it not a simple pointer any more), they would break a lot of code.
But if Embarcadero wanted to eliminate dangling object pointers, they would have to redesign Delphi object references:
when an object is freed, all references to it must be freed, too. This is only possible by double-linking both, i.e. the object instance must carry a list with all of the references to it, that is, all memory addresses where such pointers are (on the lowest level). Upon destruction, that list is traversed, and all those pointers are set to nil
a little more comfortable solution were that the "one" holding such a reference can register a callback to get informed when a referenced object is destroyed. In code: when I have a reference FSomeObject: TSomeObject I would want to be able to write in e.g. SetSomeObject: FSomeObject.OnDestruction := Self.HandleDestructionOfSomeObject. But then FSomeObject can't be a pointer; instead, it would have to be at least an (advanced) record type
Of course I can implement all that by myself, but that is tedious, and isn't it something that should be addressed by the language itself? They also managed to implement for x in ...
I am trying to understand precisely how Delphi manages the memory and I have read from Marco Cantu that it uses references like Java does.
I have understood that a variable (let's say var theObj: TSomeClass;) holds a reference (= a pointer) to indicate the memory location where the object has been stored. So when I call the constructor of theObj I am creating the class on the heap and then theObj is a pointer to the newly created location.
So far so good but now I have the doubt. He says that:
In some OOP languages, declaring a variable of a class type creates an
instance of that class.
Does this mean that a pointer to a memory location is not involved? My guess is that here you declare a variable and it directly creates the object without using the heap. So is the variable only create on the stack and it holds all the data of the class?
Note. The guess and the question above are made because I made a comparison with C++.
Delphi: theObj := TSomeClass.Create
C++: TSomeClass *theObj = new TSomeClass;
Not going off topic and talking of C++ (it was just as example, I know only this language to make the comparison) but here in both cases I create the object in the heap and I have a pointer to it. The second question I made above came out because of this: in C++ I can also create an object like this TSomeClass theObj; and it will live until the end of the scope of the function.
In Delphi I cannot. I think that marco refers so this when he says "declaring a variable of a class type creates an instance of that class". Am I correct?
Marco is thinking of C++ where classes and structs are essentially the same with different default accessibility. In C++
SomeClass obj;
creates the object.
There is nothing for you to worry about. Your understanding, as expressed in the second paragraph, is impeccable. A Delphi variable of class type is, under the hood, just a pointer to the instance.
So I am exposing complicated data objects that are both created entirely within a C++ environment but more recently I have made it so Lua can also create an instances of these objects. I use the UserData to track where the allocation happened so when __gc is called it knows what it is supposed to do.
Is Lua smart enough to know a reference to a user data it created is stored in a user data block that it did not create?
Code Sample
function AddNewSubjObject(containerId, subObjectType)
local containerObject = meta_ContainerObject:getContainer(containerId)
local newSubObject = meta_SubObject:CreateNew(subObjectType)
containerObject.AddChild(newSubObject)
return containerObject;
end
We get a local variable to the userdata object that can contain other objects. We then create a new object that its only tracking is in Lua. Finally we call a method that will store the sub object in the container object.
When this function returns, to me it looks like it has removed all references to the newSubObject so I fear it will delete the memory allocated for it even though it is still being referenced..
Will this either leak memory by not cleaning up the objects properly or will it corrupt memory by deleting it when it is still be potentially used? Basically I am wondering if this is a situation that has to be considered and handled when exposing userdata via Lua.
The Lua garbage collector does not look inside userdata.
Lua provides weak tables to manage these kinds of references. See PiL Chapter 17. You can use a table with weak keys of subObjects and values of the container of the subObject. When the container is collected so will the subObjects.
According to the Guice wiki page, unscoped binding is preferred to Singleton for stateless objects.
I do not agree with the statement though because:
Singleton scope gives more information to developers. Developer can assume singleton objects are thread-safe and they can be injected without using Provider most of the time.
Creating an unscoped object can be expensive because of dependencies to other unscoped objects.
Cyclic dependency of unscoped objects can only be resolved with Providers, which could lead to weird behavior.
e.g. A depends on B, B depends on Provider, and some method f() in A invokes B which invokes some method in A, then a new instance of A and B will be created each time f() is called.
When injecting a Request Scoped object into unscoped objects, it is hard to tell if you need a provider or not because the life cycle of unscoped object is unknown.
Could someone explain why is default scoped preferred to singleton scope for stateless objects?
There's one big and important reason for the preference to unscoped objects instead of singleton objects: Singleton objects can never be garbage collected, nor can the entire transitive tree of objects to which they hold references. Their construction needs to be synchronized (to prevent race conditions during creation), and then they remain in memory for the entire lifetime of the Injector.
This is particularly an issue because, in a production environment, all Guice singletons are eager regardless of whether they're bound asEagerSingleton. This means that in production your memory commitment of your singleton tree starts when the Injector is created, and often lasts until the application closes.
Using Singletons for everything may cause slower startup and larger memory requirements than judiciously choosing singletons.
Of course, if the object and its dependencies are all stateless, the memory footprint of each object is pretty light. However, if your singleton depends on an object that depends on an object that depends on a database, then that database and its memory will be required on startup and never freed.
injector ___________________TIME_________________________\
creation /
|---------------------SINGLETON-------------------------->
|----------REQUEST A------------| |---REQUEST B----->
|-UNSCOPED C-| |-UNSCOPED D-| |-UNSCOPED E-|
Scopes track roughly to the lifetime of the object, and the goal is to avoid injecting a narrower-scoped object into a wider-scoped object (say, keeping around unscoped object C, which might be relevant to request A but not request B). To do so would be a scope-widening injection.
When you refer to unscoped objects, think of them as ephemeral or disposable, and having the narrowest scope possible. This makes it particularly easy to make stateless objects unscoped, because it doesn't matter which instance you have.
Those unscoped objects are also free to inject any object in the graph without using providers, on the assumption that the unscoped object will always have the shorter lifetime compared to any other dependency you pick. By contrast, a singleton must use providers for all of its non-singleton dependencies, because it will absolutely outlive any non-singleton it depends on.
This should make it clear when you need Providers, as you allude to in #1 and #4 above.
In summary, I can only direct you back to the section you linked: Unless you have a good reason (state, expensive construction, or resource management) you should probably leave your injectors unscoped.
In my Delphi7 this code
var MStr: TMemoryStream;
...
FreeAndNil(MStr);
MStr.Size:=0;
generates an AV: Access violation at address 0041D6D1 in module 'Project1.exe'. Read of address 00000000.
But somebody insists that it should not raise any exception, no matter what. He also says that his Delphi 5 indeed raises no exceptions. He calls this a “stale pointer bug”.
In other words he says that FreeAndNil cannot be used as debugger to detect a double attempt to free an object or to use a freed object.
Can anybody enlighten me? Should this raise and error (always/randomly) or the program should run over this bug without problems?
Thanks
I ask this because I believe I have a "double free object" or "free and re-access" bug in my program. How can I fill the memory allocated to an object with zeros AFTER I freed the object? I want this way to detect where the bug is, by getting and AV.
Initially, I hoped that if I set the object to FreeAndNil, I will ALWAYS get an AV when trying to re-access it.
It's always wrong to use methods or properties of a null reference, even if it appears to work sometimes.
FreeAndNil indeed cannot be used to detect double frees. It is safe to call FreeAndNil on an already-nil variable. Since it's safe, it doesn't help you detect anything.
This is not a stale-pointer bug. This is a null-reference bug. A stale-pointer bug is when you have freed an object but not cleared all variables that referenced it. Then the variable still holds the old address of the object. Those are very hard to detect. You can get such a bug like this:
MStr := TMemoryStream.Create;
MStr.Free;
MStr.Size := 0;
You can also get one like this:
MStr := TMemoryStream.Create;
OtherStr := MStr;
FreeAndNil(MStr);
OtherStr.Size := 0;
Using MStr.Size after you have freed the object MStr referenced is an error, and it should raise an exception. Whether it does raise an exception depends on the implementation. Maybe it will, and maybe it won't. It's not random, though.
If you're searching for a double-free bug, you can use the debugging aides that FastMM provides, as others have suggested as well. It works by not actually releasing the memory back to the operating system, or even back to Delphi's internal free-memory pool. Instead, it writes known-bad data into the object's memory space, so when you see those values, you'll know you're reading from something that you already freed. It also modifies the object's VMT so that the next time you call a virtual method on that object reference, you'll get a predictable exception, and it will even tell you which supposedly freed object you tried to use. When you attempt to free the object again, it can tell you not only that you already freed it, but also where it was freed the first time (with a stack trace), and where it was allocated. It also collects that information to report about memory leaks, where you freed an object less than one time instead of more.
There are also habits you can use to avoid the issue for future code:
Reduce the use of global variables. A global variable could be modified by any code throughout the program, forcing you to wonder whenever you use it, "Is this variable's value still valid, or did some other code free it already?" When you limit the scope of a variable, you reduce the amount of code you have to consider in your program when looking for reasons a variable doesn't have the value you expect.
Be clear about who owns an object. When there are two pieces of code that have access to the same object, you need to know which of those pieces of code owns the object. They might each have a different variable for referencing the object, but there's still just one object there. If one piece of code calls FreeAndNil on its variable, that still leave's the other code's variable unchanged. If that other code thinks it owns the object, then you're in trouble. (This concept of owner is not necessarily tied to the TComponent.Owner property. There doesn't need to be an object that owns it; it could be a general subsystem of your program.)
Don't keep persistent references to objects you don't own. If you don't keep long-lived references to an object, then you don't have to worry about whether those references are still valid. The only persistent reference should be in the code that owns the object. Any other code that needs to use that object should receive a reference as an input parameter, use the object, and then discard the reference when it returns its result.
From what I am seeing, this code should always result in an error. FreeAndNil explicitly sets that passed value to Nil (aka 0), so you should absolutely get an access violation when trying to dereference the object.
Just to complicate the issue:
If the method you call is a static (not virtual) method and it does not call any virtual methods itself nor does it access any fields of the object, you will not get an access violation even if the object reference has been set to NIL.
The reason for this is that the access violation is caused by dereferencing the self pointer (in this case NIL), but that only happens when accessing a field or the object's VMT for calling a virtual method.
This is just an exception to the rule that you cannot call methods of an NIL object reference that I'd like to mention here.
If you set a pointer to nil, you shouldn't be able to use it any more. But if you have another pointer to the same object, you can use it without getting an AV, because this pointer still points to the object address and not to nil.
Moreover, freeing an object do not clear the memory used by the that object. It just marks it as not in use. Thats the reason you want get an AV. If the freed memory is allocated for another object, you will get a AV, because it no longer contains data that seems valid.
FastMM4 has some settings that you can use while debugging, that will detect such conditions. From the FsatMM4Options.inc:
{Set the following option to do extensive checking of all memory blocks. All
blocks are padded with both a header and trailer that are used to verify the
integrity of the heap. Freed blocks are also cleared to to ensure that they
cannot be reused after being freed. This option slows down memory operations
dramatically and should only be used to debug an application that is
overwriting memory or reusing freed pointers. Setting this option
automatically enables CheckHeapForCorruption and disables ASMVersion.
Very important: If you enable this option your application will require the
FastMM_FullDebugMode.dll library. If this library is not available you will
get an error on startup.}
{$define FullDebugMode}
Another quote from the same file:
FastMM always catches attempts to free the same memory block twice...
As delphi uses FastMM from Delphi 2007 (2006 ?), you should get an error if you try to doublefree an object.
Thomas Mueller: have you tried virtual class methods? A constructor is sort of a virtual method but you call it against the type - not the instance. This means that even some specific virtual methods will not cause AV on a null-reference :D
Vegar: You couldn't be more right! FastMM is the best ever ever ever tool that helped me tracking down this kind of bugs.
The EurekaLog Blog had a great post on this in April 2009:
Why should you always use FreeAndNil instead of Free.