What is best method to free up memory when my console application terminates?
I would like to free up any related loaded DLLs and any other resources consumed by my application.
I want to free all things in memory :)
I have searched Google but could not find a way to do this.
I'm not using Units in my application it just console application
program MyAPP;
{$APPTYPE CONSOLE}
uses
Windows, SysUtils;
/// functions
// procedures
begin
end.
begin
try
// Your entire program goes here.
finally
SomeFunction;
end;
end.
When your application terminates all memory allocated by it either directly or by Delphi internal allocator (TObject.Create / InitInstance) or by modules is freed by operating system.
Even memory leaks within the application are not an issue after program was terminated. You can check for memory leaks by using FastMM4 library.
Windows should also free it's objects for which you have obtained handles by initialization calls to specialized libraries like GDIPlus yet these should be freed manually after object instance became obsolete in the local scope of the code using them.
For all in-code deallocations you should use try..finally. So your program.dpr might ideally look like:
program YourProgram;
{$APPTYPE CONSOLE}
uses
MainUnit;
var
main: TProgramMain
begin
main := TProgramMain.Create;
try
main.Execute;
finally
main.Free;
end;
end.
You have not showed what objects you created, but whatever you did to create those objects, the reverse is how you free them, if you wish to.
Note that if your objects are simply consuming memory in the heap, you could just let Windows reclaim all memory used by your process, and the side effects of this (if there are none) might include that your tiny application shuts down faster.
Nevertheless, most conscientious developers choose to fully free every object they create. If you created an object like this:
x := TMyObject.Create;
.. then you free it by calling Free:
x.Free;
If the object is a component, owned by its parent, you don't need to free it.
If the object is an interfaced object (reference counted) you simply let go of your reference like this:
x := nil;
If the type in question is a value-type (Double, Integer) then you don't have to free it.
If the type in question is automatically managed (record, string) you also don't have to free it.
If your object allocates more objects, the correct place to free them is in the destructor of that object.
These are the rules. Just follow them, and you will have no leaks. Your console application terminates in a manner you have not specified. Were you expecting some magic answer to your unspecified question? If it were me, I would ensure my application had an orderly shutdown, and that during normal shutdown, it frees its resources. If it terminates abnormally, then any "try..finally" blocks you have written will not be executed. I recommend you single step through your shutdown code, to see that it executes at all.
If you had posted some examples of your code, then more specific answers would have been possible.
Related
I'm working with an old and complex system that shares memory between dozens (sometimes hundreds) of Win32 processes. The code is mostly very old Pascal that has been ported to Delphi a few years ago.
(Almost) all of the code is in a single DLL, which all of the processes load. At the moment, we have forced a fixed loading address of that DLL. Image base is defined and ASLR is disabled in linker settings. Each process checks the DLL loading addresses at startup and the entire system refuses to work if the DLL cannot be loaded at the exact same address in all of the processes. This is of course a problematic solution. Sometimes customers have all sorts of 3rd party gadgets which affect the address space and prevents our product from having the address it wants for the DLL.
The reason for the fixed DLL loading address is below. I'm wondering if there is a way to work around this problem.
I've been trying to introduce object-oriented programming. The problem is, if I instantiate a Delphi class in the shared memory, that instance now seems to be dependent on the DLL loading address. For example, if another process tries to destroy that object, it will crash, unless the two processes happen to have the same DLL address. Delphi runtime seems to save function addresses in the object instance, assuming they will stay fixed for the lifetime of the object.
One possible solution might be to copy the DLL contents to the shared memory, then do some sort of magic trickery on DLL_PROCESS_ATTACH to make the process run that copy of the code instead of the loaded DLL address. The shared memory we have is always mapped at the same addresses. (Yes, this is also a problem sometimes but very rarely since the shared memory can be mapped at high (above 2 GB) addresses which are easily available.)
Or is there a way to tell Delphi compiler "please do not assume that the addresses of the functions related to this class are fixed"? I'm using Delphi 11.1.
I was able to figure out a a solution that seems to work well, so let me answer my own question.
The issue is that in order for dynamic dispatch to work, the object instance must be 'tagged' with type information. In the case of Delphi in Win32, this tag is in the first 32 bits of the object instance, and it is a memory address into the DLL where the the code of the class in question is.
If you shift this address to match the variable (process-specific) address of the DLL, the dynamically dispatched methods work fine. In order to do this, you need to compare this address to the loading address of the DLL (or any other reference address inside the DLL) and save the offset, when creating the object.
Then, before calling the object's methods in another process, "localize" the object by taking the actual address of the DLL, adding the offset, and writing this sum to the first 32 bits of the object.
Now you can use the object in any process, as long as you localize it first:
Obj.Localize;
Obj.Do_Something;
This can be neatly wrapped in a class.
Offset is simply a private 32-bit UInt32.
constructor Global_Object.Create;
begin
Self.Offset := PUInt32(Self)^ - Self.Reference_Address;
end;
procedure Global_Object.Localize;
begin
PUInt32(Self)^ := Self.Reference_Address + Self.Offset;
end;
destructor Global_Object.Destroy;
begin
inherited Destroy;
end;
function Global_Object.Reference_Address
: Cardinal;
begin
// Anything in the DLL can be used as a reference,
// such as the address of this function.
Result := Cardinal(#Global_Object.Reference_Address);
end;
Suppose an application with some forms and only one data module are created at start.
In the DM1.OnCreate event, a TStringList is created to be used at runtime.
We know that when the application is being terminated, all things will be destroyed and memory will automatically freed. Freeing something can take some time, and so is not always advised to worry about memory leaks on shutdown. See for example this answer from Barry Kelly or this post from Raymond Chen.
Beside that, FastMM reports the memory leak if I don't add TStringList.Free to DM1.OnDestroy. This turns out to be a problem when searching for any other memory leaks that I should really worry about.
So basically I am asking if/why/when I should free object instances that will be freed by application or OS (Windows in this specific case). Is there any other valid case that is not when hunting for memory leaks?
NOTE: In this specific case, the data module is not created or recreated more times. There will not be any memory leak at all, besides the one. The data module scrap source:
unit UDM1;
interface
uses SysUtils, Classes, ...;
type
TDM1 = class(TDataModule)
procedure DataModuleCreate(Sender: TObject);
procedure DataModuleDestroy(Sender: TObject);
procedure DoStuffWithStringList1(Sender: TObject);
private
internalStL: TStringList;
end;
var
DM1: TDM1;
implementation
procedure TDMInterfacePAFECF.DataModuleCreate(Sender: TObject);
begin
internalStL := TStringList.Create();
end;
procedure TDMInterfacePAFECF.DataModuleDestroy(Sender: TObject);
begin
internalStL.Free; //<-- IS THIS NECESSARY OR ADVISED?
end;
procedure DoStuffWithStringList(Sender: TObject);
begin
//Place some code using internalStL here...
end;
For the same reason I strongly advocates (understatement) for not leaving any Compiler Hint or Warning in a project, clean after yourself and DO NOT LEAVE A REPORTED MEMORY LEAK!
EVER!
Now, that does not necessarily means that you have to Free everything in the Destructor of your DataModule if you have a strong case for not doing it, but in that case, you need to register your Memory Leak so that it will not be reported. (And put there a very visible comment to justify and explain why)
But consider the fact that you may leave this project and a year from now, someone else is maintaining it and has a new business requirement to create multiple DataModules... Chances are that if they do not know the inside of your code well enough, they will trust your code to be clean and problems are likely to follow.
So I would strongly advise against not freeing unless in a very special and expected and documented case...
PS: Seen that and had to mop up the memory dripping all over the place so many times that I even did some CodeRage sessions on fighting memory leaks...
Updayte: Here's the link to download that CodeRage session...
My answer can be considered philosophical, but the main reason is that any action (or absence of it) has consequences. I thought about your example and probably other examples and I see one good reason to free the object. Every time I think I can ignore freeing object increases the probability of not doing this in other, maybe more serious situation. Another example is a habit of doing "try finally end" everywhere something is allocated or freed. I don't care about freeing in case of exception, but this habit helps me avoid leaking
Use RegisterExpectedMemoryLeak for the intentional memory leaks. The routine has a few overloaded versions of which one can be fead with an object class:
begin
RegisterExpectedMemoryLeak(TStringList);
FStringList := TStringList.Create;
...
or better, register the pointer itself:
begin
FStringList := TStringList.Create;
RegisterExpectedMemoryLeak(FStringList);
...
In this case the unexpected memory leaks will show up normally and can not be confused with this particular string list.
Let me answer by asking you a question.
Can you say for certain that the life cycle of the data module will always be tied to the lifetime of the application or that you will never need to create additional instances of it?
If you answer yes then feel free to ignore standard memory management practices.
If you answer no then you should make sure object's clean up after themselves.
Stuff gets done when you free an object, possibly more then only deallocating the memory.
Comes to mind a database object that does transactions and ends all started transactions in the ondestroy.
If you don't call free, ondestroy will not get fired and you may end up with locked tables.
I have implemented the FlyWeight pattern in my Delphi application. Everything has worked great, everything is a lot faster and takes less memory, but there is one thing I am worried about.
My implementation will only work as long as client code never calls Free() on the shared objects. In the Flyweight pattern, the FlyweightFactory itself is supposed to "maintain a reference to flyweights" i.e. to the shared objects.
My problem is that there is no (obvious) way to stop other code from destroying the objects once they have a reference. I could live with this, but it would be a "big win" if I could pass these objects round freely without worrying about accidental freeing.
To show a (contrived) example:
flyweight1:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
flyweight1.Free;
flyweight2:=FlyweightFactory.GetFlyweight(42);
WriteLn('Description is '+flyweight.Description);
// Object has already been Freed!; behaviour is undefined
I have considered overriding the destructor as shown here to stop the flyweight object being freed altogether. This is not an option in my case as
a) I only want to stop cached objects from being Freed, not objects that aren't part of the cache. There is a lot of legacy code that doesn't use the cache; they still need to create and free objects manually.
b) I do want the FlyweightFactory to Free the objects during finalization; I agree with Warren P that a "zero leaked memory" policy is best.
I'll leave with a quote from the Flyweight chapter of GoF
Sharability implies some form of
reference counting or garbage
collection to reclaim storage when
it's no longer needed. However,
neither is necessary if the number of
flyweights is fixed and small. In that
case, the flyweights are worth keeping
around permanently.
In my case the flyweights are "fixed" and (sufficiently) small.
[UPDATE See my answer for details of how I solved this problem]
My answer to the question you link to still applies. The objects must know by means of a private boolean flag that they are cached objects. Then they can elect not to destroy themselves in Destroy and FreeInstance. There really is no alternative if you want to allow Free to be called.
To deal with finalization you would want to add the cached objects to a list of cached objects. That list of objects can be freed at finalization time. Of course the flag to disable freeing would have to be reset whilst you walked the list.
Having made this point regarding finalization, I would advise you to register an expected memory leak and just leak this memory. It makes the code much simpler and there's nothing to lose. Any memory you don't free will be reclaimed by the OS as soon as your executable closes. One word of caution: if your code is compiled into a DLL then leaking could be troublesome if your DLL is loaded, unloaded, loaded again etc.
What all this is telling you is that you are swimming against the current. Is it possible that you could achieve your goals with a different solution that fitted better with the way Delphi is steering you?
I suggest to add a reference count in order to known if your shared object is still used.
Every client should use the pattern AddRef / Release (AddRef increases the count; Release decrements it; if count reaches zero Free is called)
The AddRef may be called directly by your GetFlyweight method; Release has to be used instead of Free.
If you refactor your class and extract an interface from it the AddRef/Release pattern in naturally implemented in then interface implementation. (You could derive from TInterfacedObject or implement IInterface by your self)
Ideally you seldom want 2 ways of using the same things. It just complicates matters in the long run. In 6 months time, you might not be sure whether a particular piece of code is using the new flyweight paradigm or the old paradigm.
The best way to prevent someone calling Free or Destroy is to make sure it's not even there. And within the Delphi world, the only way to do that is to use interfaces.
To expand on your contrived example:
type
TFlyweightObject = class
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): TFlyweightObject;
end;
This being an object can easily be destoyed by a rogue client. You could make the following changes:
type
IFlyweight = interface
//place guid here
function Description: string;
end;
TFlyweightObject = class(TInterfacedObject, IFlyweight)
public
constructor Create(ANumber: Integer);
function Description: string;
end;
TFlyweightFactory = class
public
function GetFlyweight(ANumber: Integer): IFlyweight;
end;
Now any code that is updated to use the flyweight paradigm is forced to use it as intended. It's also easier to recognise the old code that still needs to be refactored because it doesn't use the interface. Old code would still construct the "flyweight" object directly.
You could also hide a destructor by making it protected or private. Programmers won't see it outside the scope of the unit in which it is declared in.
But I am posting this answer more like a curiosity because this will not prevent freeing an object by using FreeAndNil or by using a "Protected Hack"
I managed to get around the problems I cited in my original question using the following techniques, suggested by David Heffernan in his answer.
a) I only want to stop cached objects
from being Freed, not objects that
aren't part of the cache. There is a
lot of legacy code that doesn't use
the cache; they still need to create
and free objects manually.
I fixed this by subclassing the Flyweight class and overriding destroy, BeforeDestruction and FreeInstance in the subclass only. This left the parent class as is. The cache contains instances of the subclass (which can't be freed), whereas objects outside the cache can be freed as per usual.
b) I do want the FlyweightFactory to
Free the objects during finalization;
I agree with Warren P that a "zero
leaked memory" policy is best.
To solve this, I added a private boolean flag that has to be set to true before the object can be freed. This flag can only be set from the Cache Unit, it is not visible to other code. This means that the flag cannot be set outside by code outside the cache.
The destructor just looks like this:
destructor TCachedItem.destroy;
begin
if destroyAllowed then
inherited;
end;
If client code trys to Free a cached object, the call will have no effect.
We are trying to figure out if we have memory leaks in our software. So, I have been using various tools and programs to help me find possible memory leaks. One of the software I used was AQTime. As it came with Delphi XE, it was only a demo. So, I was not really able to get any useful information from it. Then, I decided to use free software, MemProof. So far, it has shown me many issues with our software that requires attention. One of which is an error.
As soon as I start my program through MemProof, it lists 2 errors, which is attempting to destroy non-existent object from the unit file, system.pas. So, when I actually put a break point within TObject.Free procedure, it breaks even before my program started all the way. Stepping through the procedure Free in system.pas, I found out that TIconimage is trying to destroy or free itself. In other word, free procedure is not invoked from within my program prior to actually starting up.
Here is the actual Free procedure:
procedure TObject.Free;
begin
if Self <> nil then
Destroy;
end;
After that observation, I removed the breakpoint and let the program run all the way. My programs main window popped up ready for user input. However, I also found out that TObject.Free procedure is invoked non-stop if any part of my program's WINDOW is displayed on the screen. I don't understand that at all. Why is that? Can anyone explain? How is TForm is related to TObject.Free in any shape or form as the procedure is constantly invoked when the TForm is displayed on the screen?
Thanks in advance.
Regarding why TObject.Free executes a lot, every single time an object is destroyed, any object, that method will be called. All classes derive from TObject, it's the common ancestor, so almost any action in a Delphi program involves large numbers of object create/destroy pairs and consequently will hit TObject.Free.
Regarding detection of memory leaks, you have all you need built in to Delphi to solve this. The FastMM memory manager can be run in "report memory leaks" mode and it will give you loads of diagnostics of any memory that you leak.
Consider the following trivial program:
program Leaker;
begin
ReportMemoryLeaksOnShutdown := True;
TObject.Create;
end.
This results in the following output:
You just need to set ReportMemoryLeaksOnShutdown to True somewhere in your app (the start of the .dpr file is as good a place as any).
If you wish to receive more information in the report then you can download the full version of FastMM and configure it to your heart's content.
Then you get output like this:
A memory block has been leaked. The size is: 84
This block was allocated by thread 0x1304, and the stack trace (return addresses) at the time was:
40455E [System][System.#GetMem]
405A2F [System][System.TObject.NewInstance]
40602E [System][System.#ClassCreate]
4474C2 [Classes][Classes.TStringList.Create]
C275A3 [Main.pas][Main][Main.TMainForm.CreateAuxiliaryForms][997]
C84C8A [OrcaFlex.dpr][OrcaFlex][OrcaFlex.OrcaFlex][351]
75E633CA [BaseThreadInitThunk]
77519ED2 [Unknown function at RtlInitializeExceptionChain]
77519EA5 [Unknown function at RtlInitializeExceptionChain]
The block is currently used for an object of class: TStringList
It's truly wonderful. It tells me that the leaking memory was allocated in Main.pas line 997, and that's precisely where I put my intentional leak!
As you know, TApplication has an Icon property which you can f.i. assign in the application settings in project options. This property is reflected with an FIcon field of TApplication which is created in the constructor of the Application object. TIcon has a TIconImage field representing the actual image which gets created in its constructor. When the Application object loads and assigns the icon from the project resource file, this initial 'TIconImage' has to be freed in order to prevent a leak. All this happens even before Application.Initialize is called in the project source, because the Application object is constructed from the initialization section of 'controls.pas'.
Lots of things are happening when an application is launching or running. When launching, the streaming mechanism creates objects (resource streams, readers, class finders, component lists ..) and then frees them. Even a blank VCL form (with no controls on it) when running, creates a list each time it gets activated to find a control to put the focus on, and then frees this list. With complex GUI applications, a variety of graphics objects can be created and freed even if you hover the mouse on something. Or the alignment/arrangement code can create/free objects even if you press your mouse on to something.
To debug leaks you can take the course outlined by David's answer, or when using a 3rd party product concentrate on what it says leaked, not on every object which gets created/freed. :)
TObject.Free will be called whenever ANY instance of a class in Delphi is Free'd.
This includes a whole host of objects that are created and destroyed simply as part of the normal execution of a Delphi program, including in response to events processed automatically by a TForm object in response to the messages generated by the system simply to maintain the window object itself in existence in the Windows own Window Manager.
For example, consider this snipped fragment of code from the TCustomForm WndProc:
WM_MEASUREITEM:
begin
:
Canvas := TControlCanvas.Create;
with Canvas do
try
:
finally
Canvas.Free;
end;
:
end;
The key here being that in response to a WM_MEASUREITEM message, a custom form (and therefore a standard TForm derived class, since this ultimately derives from TCustomForm) creates a temporary TControlCanvas, which it then Free's when it is finished with it.
This may not necessarily be the source of the TObject.Free calls that you are seeing in your particular form's case, it is just an example, but shows how a TForm merely existing can result in other objects being brought into existence and destroyed in response to automatic, system generated messages.
I'm currently working on porting an existing Delphi 5 application to Delphi 2010.
It's a multithreaded DLL (where the threads are spawned by Outlook) that loads into Outlook. When compiled through Delphi 2010, whenever I close a form I run into an "invalid pointer operation" inside TMonitor.Destroy... the one in system.pas, that is.
As this is an existing and kinda complex application, I have a lot of directions to look into, and the delphi help doesn't even document barely documents this particular TMonitor class to begin with (I traced it to some Allen Bauer posts with additional information) ... so I figured I'd first ask around if anyone had encountered this before or had any suggestions on what could cause this problem.
For the record: I am not using the TMonitor functionality explicitly in my code, we are talking a straight port of Delphi 5 code here.
Edit Callstack at the moment the problem occurs:
System.TMonitor.Destroy
System.TObject.Free
Forms.TCustomForm.CMRelease(???)
Controls.TControl.WndProc(???)
Controls.TWinControl.WndProc((45089, 0, 0, 0, 0, 0, 0, 0, 0, 0))
Forms.TCustomForm.WndProc(???)
Controls.TWinControl.MainWndProc(???)
Classes.StdWndProc(15992630,45089,0,0)
Forms.TApplication.ProcessMessage(???)
The pointer to the System.Monitor instance of each object is stored after all the data fields. If you write too much data to the last field of an object it could happen that you write a bogus value to the address of the monitor, which would most probably lead to a crash when the destructor of the object attempts to destroy the bogus monitor. You could check for this address being nil in the BeforeDestruction method of your forms, for a straight Delphi 5 port there shouldn't be any monitors assigned. Something like
procedure TForm1.BeforeDestruction;
var
MonitorPtr: PPMonitor;
begin
MonitorPtr := PPMonitor(Integer(Self) + InstanceSize - hfFieldSize + hfMonitorOffset);
Assert(MonitorPtr^ = nil);
inherited;
end;
If this is a problem in your original code you should be able to detect it in the Delphi 5 version of your DLL by using the FastMM4 memory manager with all checks activated. OTOH this could also be caused by the size increase of character data in Unicode builds, and in that case it would only manifest in DLL builds using Delphi 2009 or 2010. It would still be a good idea to use the latest FastMM4 with all checks.
Edit:
From your stack trace it looks like the monitor is indeed assigned. To find out why I would use a data breakpoint. I haven't been able to make them work with Delphi 2009, but you can do it easily with WinDbg.
In the OnCreate handler of your form put the following:
var
MonitorPtr: PPMonitor;
begin
MonitorPtr := PPMonitor(Integer(Self) + InstanceSize - hfFieldSize + hfMonitorOffset);
MessageDlg(Format('MonitorPtr: %p', [pointer(MonitorPtr)]), mtInformation,
[mbOK], 0);
DebugBreak;
// ...
Now load WinDbg and open and run the process that calls your DLL. When the form is created a message box will show you the address of the monitor instance. Write down the address, and click OK. The debugger will come up, and you set a breakpoint on write access to that pointer, like so:
ba w4 A32D00
replacing A32D00 with the correct address from the message box. Continue the execution, and the debugger should hit the breakpoint when the monitor gets assigned. Using the various debugger views (modules, threads, stack) you may get important information about the code that writes to that address.
An invalid pointer operation means your program attempted to free a pointer, but there was one of three things wrong with it:
It was allocated by some other memory manager.
It had already been freed once before.
It had never been allocated by anything.
It's unlikely that you'd have multiple memory managers allocating TMonitor records, so I think we can rule out the first possibility.
As for the second possibility, if there's a class in your program that either doesn't have a custom destructor or that doesn't free any memory in its destructor, then the first actual memory deallocation for that object could be in TObject, where it frees the object's monitor. If you have an instance of that class and you attempt to free it twice, that problem could appear in the form of an exception in TMonitor. Look for double-free errors in your program. The debugging options in FastMM can help you with that. Also, when you get that exception, use the call stack to find out how you got to TMonitor's destructor.
If the third possibility is the cause, then you have memory corruption. If you have code that makes assumptions about the size of an object, then that could be the cause. TObject is four bytes larger as of Delphi 2009. Always use the InstanceSize method to get an object's size; don't just add up the size of all its fields or use a magic number.
You say the threads are created by Outlook. Have you set the IsMultithread global variable? Your program normally sets it to True when it creates a thread, but if you're not the one creating threads, it will remain at its default False value, which affects whether the memory manager bothers to protects its global data structures during allocation and deallocation. Set it to True in your DPR file's main program block.
After a lot of digging it turns out I was doing a nice (read: horrifying, but it has been properly doing its job in our delphi 5 apps for ages)
PClass(TForm)^ := TMyOwnClass
somewhere deep down in the bowels of our application framework. Apparently Delphi 2010 has some class initialization to initialize the "monitor field" that now didn't happen, causing the RTL to try and "free the syncobject" upon form destruction because getFieldAddress returned a non-nil value. Ugh.
The reason why we were doing this hack in the first place was because I wanted to automatically change the createParams on all form instances, to achieve an iconless resizable form. I will open up a new question on how to do this without rtl-breaking hacks (and for now will simply add a nice shiny icon to the forms).
I will mark Mghie's suggestion as the answer, because it has provided me (and anyone reading this thread) with a very large amount of insight. Thanks everyone for contributing!
There are two TMonitor in Delphi:
System.TMonitor; which is a record, and is used for thread synchronization.
Forms.TMonitor; which is a class representing an attached monitor (display device).
System.TMonitor is added to Delphi since Delphi 2009; so if you are porting a code from Delphi 5, what your code was using was Forms.TMonitor, not System.TMonitor.
I think the class name is referenced without unit name in your code, and that is making the confusion.