IDR is a good tool for decompiling Delphi applications, but hHow do I know the ID number of Windows messages assigned to message handlers?
For example, from IDR decompiling, I see:
procedure sub_004D2398(var Msg: TMsg);dynamic;
The original source code is:
procedure Name_procedure(var Msg: TMsg); message 1028;
How do I know the message number 1028 while reverse-engineering the code in IDR?
A given procedure doesn't know whether it is a message handler or not, because that information is not stored in the procedure itself where a decompiler can easily access it (it in available in RTTI, though).
Every class has its own dispatch table that the compiler generates to hold pointers to all of the class's dynamic and message methods (and in the case of message, also their message IDs). So, when you declare a class procedure as a message handler, the compiler inserts an entry for that message ID in that class's dispatch table along with a pointer to the handler.
When a UI control receives a message, the message first goes to the control's WindowProc (which the app can subclass directly). If the message is not handled, it goes to the control's WndProc() method. If the message is still not handled, it goes to the TObject.Dispatch() method, which looks up the message ID in the control's dispatch table and calls the associated procedure if one is found. Otherwise, the message goes to the control's DefaultHandler() method.
So, unless IDR is smart enough to decompile a procedure, determine which class it belongs to, and find and decompile that class's dispatch table or RTTI to determine the message ID belonging to the procedure, then you will have to do this manually while you are analyzing the decompiled output. A class's VMT contains pointers to the class's dispatch table and RTTI (amongst other things).
Once you are able to determine the message ID that belongs to a given message handler, only then can you research and figure out what kind of message that ID might refer to, as described in Deltics' answer.
Normally to find the message declaration corresponding to a given message number you would just look in any header file (C/C++) or unit (Delphi) that declares message constants. From memory I think in (older?) Delphi versions this is in the Windows unit, or possibly Messages.
In the case of Delphi you will find a bunch of declarations similar to:
const
WM_LBUTTONDOWN = 513;
Or if in hex:
const
WM_LBUTTONDOWN = $0201;
Just find the declaration for the WM_ constant with a value of 1028 (or the hex equivalent, $0404).
However you almost certainly will not find one!
Private Messages
1028 is greater than 1024, and 1024 is the constant for the "special message": WM_USER. This is not intended to be used as a message directly but rather indicates the first message number that an application can use for its own, custom/private messages.
i.e. the message with value 1028 has no standard meaning across all Windows applications.
Rather it is the 4th "custom" or private message (or possibly 5th - some people start with WM_USER+1 as documented, others start with WM_USER) used by, and meaningful to, only that application.
In the absence of the original declared constant name for the message, whatever it means and is used for can only be determined by inspecting the code and identifying its use within that code.
Related
I want to introduce some new methods into a custom control I am deriving from TCustomListBox.
What I want is a method that can be used when a item is added/inserted into the listbox, and a method for when an item is removed from the listbox.
What would be a good place to start with this? I know controls like TListView for example have an OnInsert event but I cannot see anything for listbox?
I wanted to introduce into my control for example:
OnInsert
OnRemove
Would I need to use some kind of API or Messages to detect when items are added/removed and then take it from there? Is there an easier way to do this or does it require some difficulty?
I tried looking at some of the VCL source but most of it is confusing for me.
Thanks in advance.
The API you need already exists.
If you inspect the source of a TCustomListBox you will see that the mechanism by which items are added, inserted or removed from a list is implemented using window messages. For example in TListBoxString.Add() as well as house-keeping code you will see that the string is eventually added by sending a message to the control:
Result := SendTextMessage(ListBox.Handle, LB_ADDSTRING, 0, S);
Delphi provides various mechanisms for providing handlers on control and window classes that respond to specific messages. Perhaps the most straightforward and appropriate, for adding a simple notification mechanism such as your require, is to implement a specific message handler method.
You provide a message handler procedure and declare which message it responds to. In your case, for example, you could add your own handling of the LB_ADDSTRING message:
TChattyList = class(TCustomListbox)
procedure LBAddString(var aMessage: TMessage); message LB_ADDSTRING;
end;
The parameters of the message (wParam and lParam) are packaged up inside the TMessage record passed as the by reference parameter to your handler). You will need to consult the Windows API documentation for the message in question to determine the use of these parameters.
You can do pretty much whatever you want in your message handler although you should always pay close attention to what a window is expected to do in response to documented messages, including any return values (set in the Result field of the TMessage parameter, which is why it is passed by reference, as var.
In this trivial example, the new handler calls inherited to ensure that the inherited implementation is allowed to respond by actually adding the new item string and then crudely pops up a message box to let us know that an item was added:
procedure TChattyList.LBAddString(var aMessage: TMessage);
begin
inherited;
ShowMessage('item added');
end;
In essence your event mechanism will do exactly the same, but instead of presenting a message box you would trigger your new event after allowing the inherited implementation to do it's work (and checking the result code set, to ensure that it was successful, according to the expected return values for the message in question):
procedure TChattyList.LBAddString(var aMessage: TMessage);
begin
inherited;
if (aMessage.Result = LB_ERR) or (aMessage.Result = LB_ERRSPACE) then
EXIT;
if Assigned(fOnInsert) then
fOnInsert(self);
end;
If the inherited handler failed to add an item then according to the documentation it should set the result to LB_ERR or LB_ERRSPACE, so we test for these values and exit if they are found. Otherwise we call the appropriate event handler, if one is assigned.
This assumes that for your purposes a simple TNotifyEvent is sufficient and that you do not discriminate between an item being inserted vs an item being added. You could of course have separate events or provide some indication in parameters to a specialised event type.
Which messages you choose to handle and expose as which sorts of events is then a question of exactly what your requirements are, but based on what you have stated in your question, at a minimum I think you will need message handlers for LB_ADDSTRING, LB_INSERTSTRING and LB_DELETESTRING.
You may need to handle additional messages and should consult the Windows API documentation for listbox controls for further information.
How to find out the list of messages that a certain VCL component can accept ???/
For example if i want to scroll the Memo1 by sending message to it
I probably will write the following lines of code knowing that the memo can accept EM_LINESCROLL
SendMessage(Memo1->Handle,EM_LINESCROLL,-1,0);
//Memo1->Perform(EM_SCROLL,SB_LINEUP,0);
Memo1->Perform(EM_SCROLL,SB_LINEDOWN,0);
How to find to find out if certain VCL comps can accept or do not accept messages???
All components accept all messages, but if a component have no assigned message handler, it just does nothing
If you want to discover if VCL component have special handler to certain Windows message, you have to look into VCL sources, which are usually provided with C++Builder (except Starter Edition's of XE and XE2).
VCL Sources are located in %CBuilderDir%\Sources\VCL (looking at my CBuilder5/6)
Sources are written in delphi, but it won't be difficult to find all what we need.
First, You'll have to find defininition of your target class. You can search through whole VCL source dir for file with line looking like
TMemo = Class (for your example with TMemo)
Open file where you found your class, (usually it will be stdctrls.pas or controls.pas - most useful components are located there), go to the line with class definition and scroll a little down until you find a group of procedures, looking like
procedure WMLButtonDown(var Message: TWMLButtonDown); message WM_LBUTTONDOWN;
procedure WMRButtonDown(var Message: TWMRButtonDown); message WM_RBUTTONDOWN;
...and so on. These procedures are called in response to certain messages, which id's are provided after procedure definition.
If a class have procedure for certain message, then it provides some response to it.
Message handlers are inherited in delphi, so if you didn't find handler for your message, you can look into base classes and their message handlers. To discover full class hierarchy you can simply look into the help file, or look at class definition again TMemo = class (TCustomMemo) and take parent class name from braces.
Then you can repeat search for message handler for all parent classes untill you reach TObject :-)
By the way. Simpy searching through VCL source dir of my CBuilder5 for any presense of EM_LINESCROLL I've figured than no VCL component processes it.
If you only need to provide special interaction for certain message, not trying to figure if a component already have or not have message handlers your can simply override WindowProc method of your component. All descendants of TControl have this method.
This Method process all messages received by component, and you can add response to additional system or user messages here.
void __fastcall TMyForm::NewWndProc(Messages::TMessage &Message)
{
if (Message.Msg == EM_LINESCROLL)
// Do something special for this message
else OldWndProc(Message);
}
Only thing that you'll need to do is to preserve value of old WindowProc, to call it in NewWndProc after you do all your stuff.
Its better to define and assign NewWndProc and store old WindowProc for TMemo in the form which holds your component, so you won't need to mess with making new inherited component from TMemo. So, define TWndMethod OldWndProc in form and put following, for example, in form OnCreate() handler
TWndMethod OldWndProc = MyMemo->WindowProc;
MyMemo->WindowProc = NewWndProc;
Also your can prevent firing of predefined handlers, by not passing certain messages to OldWndProc. Be careful, if you prevent processing of sensible system messages (like WM_CREATE) you'll get errors.
TMemo is a thin wrapper around a standard Win32 API multiline EDIT control. You have to read the MSDN documentation to see which messages an EDIT control natively handles. TMemo does not process EM_LINESCROLL directly, but Windows does.
We are trying to figure out if we have memory leaks in our software. So, I have been using various tools and programs to help me find possible memory leaks. One of the software I used was AQTime. As it came with Delphi XE, it was only a demo. So, I was not really able to get any useful information from it. Then, I decided to use free software, MemProof. So far, it has shown me many issues with our software that requires attention. One of which is an error.
As soon as I start my program through MemProof, it lists 2 errors, which is attempting to destroy non-existent object from the unit file, system.pas. So, when I actually put a break point within TObject.Free procedure, it breaks even before my program started all the way. Stepping through the procedure Free in system.pas, I found out that TIconimage is trying to destroy or free itself. In other word, free procedure is not invoked from within my program prior to actually starting up.
Here is the actual Free procedure:
procedure TObject.Free;
begin
if Self <> nil then
Destroy;
end;
After that observation, I removed the breakpoint and let the program run all the way. My programs main window popped up ready for user input. However, I also found out that TObject.Free procedure is invoked non-stop if any part of my program's WINDOW is displayed on the screen. I don't understand that at all. Why is that? Can anyone explain? How is TForm is related to TObject.Free in any shape or form as the procedure is constantly invoked when the TForm is displayed on the screen?
Thanks in advance.
Regarding why TObject.Free executes a lot, every single time an object is destroyed, any object, that method will be called. All classes derive from TObject, it's the common ancestor, so almost any action in a Delphi program involves large numbers of object create/destroy pairs and consequently will hit TObject.Free.
Regarding detection of memory leaks, you have all you need built in to Delphi to solve this. The FastMM memory manager can be run in "report memory leaks" mode and it will give you loads of diagnostics of any memory that you leak.
Consider the following trivial program:
program Leaker;
begin
ReportMemoryLeaksOnShutdown := True;
TObject.Create;
end.
This results in the following output:
You just need to set ReportMemoryLeaksOnShutdown to True somewhere in your app (the start of the .dpr file is as good a place as any).
If you wish to receive more information in the report then you can download the full version of FastMM and configure it to your heart's content.
Then you get output like this:
A memory block has been leaked. The size is: 84
This block was allocated by thread 0x1304, and the stack trace (return addresses) at the time was:
40455E [System][System.#GetMem]
405A2F [System][System.TObject.NewInstance]
40602E [System][System.#ClassCreate]
4474C2 [Classes][Classes.TStringList.Create]
C275A3 [Main.pas][Main][Main.TMainForm.CreateAuxiliaryForms][997]
C84C8A [OrcaFlex.dpr][OrcaFlex][OrcaFlex.OrcaFlex][351]
75E633CA [BaseThreadInitThunk]
77519ED2 [Unknown function at RtlInitializeExceptionChain]
77519EA5 [Unknown function at RtlInitializeExceptionChain]
The block is currently used for an object of class: TStringList
It's truly wonderful. It tells me that the leaking memory was allocated in Main.pas line 997, and that's precisely where I put my intentional leak!
As you know, TApplication has an Icon property which you can f.i. assign in the application settings in project options. This property is reflected with an FIcon field of TApplication which is created in the constructor of the Application object. TIcon has a TIconImage field representing the actual image which gets created in its constructor. When the Application object loads and assigns the icon from the project resource file, this initial 'TIconImage' has to be freed in order to prevent a leak. All this happens even before Application.Initialize is called in the project source, because the Application object is constructed from the initialization section of 'controls.pas'.
Lots of things are happening when an application is launching or running. When launching, the streaming mechanism creates objects (resource streams, readers, class finders, component lists ..) and then frees them. Even a blank VCL form (with no controls on it) when running, creates a list each time it gets activated to find a control to put the focus on, and then frees this list. With complex GUI applications, a variety of graphics objects can be created and freed even if you hover the mouse on something. Or the alignment/arrangement code can create/free objects even if you press your mouse on to something.
To debug leaks you can take the course outlined by David's answer, or when using a 3rd party product concentrate on what it says leaked, not on every object which gets created/freed. :)
TObject.Free will be called whenever ANY instance of a class in Delphi is Free'd.
This includes a whole host of objects that are created and destroyed simply as part of the normal execution of a Delphi program, including in response to events processed automatically by a TForm object in response to the messages generated by the system simply to maintain the window object itself in existence in the Windows own Window Manager.
For example, consider this snipped fragment of code from the TCustomForm WndProc:
WM_MEASUREITEM:
begin
:
Canvas := TControlCanvas.Create;
with Canvas do
try
:
finally
Canvas.Free;
end;
:
end;
The key here being that in response to a WM_MEASUREITEM message, a custom form (and therefore a standard TForm derived class, since this ultimately derives from TCustomForm) creates a temporary TControlCanvas, which it then Free's when it is finished with it.
This may not necessarily be the source of the TObject.Free calls that you are seeing in your particular form's case, it is just an example, but shows how a TForm merely existing can result in other objects being brought into existence and destroyed in response to automatic, system generated messages.
I have some code that does some setup of internal objects in the Loaded() function. However, some external objects are not completely created yet, but are AFTER the Loaded() function is complete. What function does Delphi call after it calls Loaded()?
Better yet what is the creation sequence of a component?
Basically I have a TCP Server and Client. Most people will place those two components into two separate applications, some will place them in the same application for local access.
My Client tries to fetch data from the server in OnLoaded(), but the server may not be up yet! I want to know if another function is called after all the OnLoaded()'s are called.
Loaded is called immediately after the dfm is streamed in, and shouldn't be used to access the server. Your best bet is probably to post a custom message to yourself in the constructor, and have a message handler procedure that responds to that message. Posting the message puts it into the end of the message queue, and therefore it won't get processed until all the other messages ahead of it has been handled. This should delay things long enough for your components to be fully constructed for use.
I've used breakpoints in some components and firmly established that AFTERCONSTRUCTION is called BEFORE LOADED not AFTER.
I've also done the same thing on a FORM and firmly established that AFTERCONSTRUCTION is called AFTER LOADED not BEFORE.
Bear in mind that AfterConstruction is a method in TObject, but Loaded is not. It follows that Loaded is generated by code that may not necessarily put it in a specific order relative to AfterConstruction, since Loaded is not actually part of the construction sequence of a TObject and AfterConstruction is.
Indeed, if you study the RTL source, you will see that Loaded is not even invoked by any self.method of a TComponent, but is in fact invoked by a stream reader that is reading the DFM, and that will most probably be happening under the control of an "owner" component. I strongly suggest therefore that its relationship relative to the execution of AfterConstruction is not really guaranteed. The fact that it appears in a particular order for a form is because the form is most likely the component to initiate stream reading. In other words, it smacks of a convenient accident that Loaded is before AfterConstruction in a form.
Further research shows that NON-FORM components that include the following code may never call the event handler.
procedure Txxx.AfterConstruction; override;
begin
inherited AfterConstruction;
if Assigned(FOnCreate) then FOnCreate(Self);
end;
The reason is that AfterConstruction, if invoked before properties are loaded, will find FOnCreate has not been assigned yet!
In such cases, you really HAVE to use the following:
procedure Loaded; override;
begin
inherited Loaded;
if assigned(OnLoaded) then OnLoaded(self);
end;
Like I said, this will produce different outcomes for a component owned by a form than it would for the form itself! The TForm component is usually the invoker of the DFM stream reader and it is the stream reader that calls Loaded for each component it reads from the form. This process starts (fortunately) BEFORE the form's AfterConstruction, but each component that is loaded by that reader gets its AfterConstruction method called BEFORE its loaded method.
QED.
The great irony is that the Delphi 6 help file says "The AfterConstruction method implemented in TObject does nothing. Override this method when creating a class that takes some action after the object is created. For example, TCustomForm overrides AfterConstruction to generate an OnCreate event."
What it omits to say is that if you try this on anything other than a TCustomForm (which does it already), it doesn't work! Because only a form (which has it already) will load its OnCreate property before calling AfterConstruction. Any other component won't, because the DFM reader invoked by the form calls AfterConstruction before Loaded! A clear case of Borland et. al. not understanding their own code, or at best, writing a help file entry that implies something is possible when in fact it is not.
Note, if your component is not on a form and is created at runtime (even if this is as an "owned" component), its "Loaded" method will NOT be called, because there was no stream reader involved.
Another point of interest is something that "Dr" Bob Swart wrote some time ago about AfterConstruction, namely that it represents the point where virtual methods can be invoked. Evidently this is only partly true: if a form's Loaded method is invoked BEFORE AfterConstruction, then you would not be able to invoke any virtual methods from Loaded if that were true. This is not the case (obviously) because Loaded is itself a virtual method! Evidently, Loaded for a form is called between the constructor and AfterConstruction by the stream reader. It begs the question: by what method is the stream reader actually invoked? My guess is either that it runs under control of the application (not the form) and that it deliberately invokes AfterConstruction differently for a form than for other components, OR that it is the last thing the form's constructor does after having created the VMT and hence the last thing that occurs before AfterConstruction is called in the form. Accordingly all the AfterConstruction-Loaded couplets of components owned by the form are called before the form's AfterConstruction is called. Tracing the calls also shows that mostly AfterConstruction is called for ALL of those components before ALL of their loaded methods are called. I didn't however test the case where there are hierarchical "parents" (such as panels with components on them), so there may be variations on this.
Normally you would override TObject.AfterConstruction for that purpose.
The order of execution is:
each Component.AfterConstruction in creation order
(Form or DataModule).Loaded
each Component.Loaded in creation order
(Form or DataModule).AfterConstruction
Trace:
Debug Output: button AfterConstruction Process Project2.exe (4876)
Debug Output: Form Loaded Process Project2.exe (4876)
Debug Output: button Loaded Process Project2.exe (4876)
Debug Output: Form AfterConstruction Process Project2.exe (4876)
I'm not sure what you mean with
the server may not be up yet
Anyway, if the client and the server are both on the same application form or datamodule, I see alternatives:
You may "force" the system to create the server before the client and up the server in the server's OnLoad and it will be up at the client OnLoad, because documentation says:
When the streaming system loads a form or data module from its form file, it first constructs the form component by calling its constructor, then reads its property values from the form file. After reading all the property values for all the components, the streaming system calls the Loaded methods of each component in the order the components were created. This gives the components a chance to initialize any data that depends on the values of other components or other parts of itself.
Inform the "client" whenever the server is UP to let it initialize (pull data from the server). You can use a direct method call, post a message or whatever you feel comfortable with.
Let the client stand up the server inside it's own OnLoad method.
Why not use the onCreate event of the main Form ?
I'm currently working on porting an existing Delphi 5 application to Delphi 2010.
It's a multithreaded DLL (where the threads are spawned by Outlook) that loads into Outlook. When compiled through Delphi 2010, whenever I close a form I run into an "invalid pointer operation" inside TMonitor.Destroy... the one in system.pas, that is.
As this is an existing and kinda complex application, I have a lot of directions to look into, and the delphi help doesn't even document barely documents this particular TMonitor class to begin with (I traced it to some Allen Bauer posts with additional information) ... so I figured I'd first ask around if anyone had encountered this before or had any suggestions on what could cause this problem.
For the record: I am not using the TMonitor functionality explicitly in my code, we are talking a straight port of Delphi 5 code here.
Edit Callstack at the moment the problem occurs:
System.TMonitor.Destroy
System.TObject.Free
Forms.TCustomForm.CMRelease(???)
Controls.TControl.WndProc(???)
Controls.TWinControl.WndProc((45089, 0, 0, 0, 0, 0, 0, 0, 0, 0))
Forms.TCustomForm.WndProc(???)
Controls.TWinControl.MainWndProc(???)
Classes.StdWndProc(15992630,45089,0,0)
Forms.TApplication.ProcessMessage(???)
The pointer to the System.Monitor instance of each object is stored after all the data fields. If you write too much data to the last field of an object it could happen that you write a bogus value to the address of the monitor, which would most probably lead to a crash when the destructor of the object attempts to destroy the bogus monitor. You could check for this address being nil in the BeforeDestruction method of your forms, for a straight Delphi 5 port there shouldn't be any monitors assigned. Something like
procedure TForm1.BeforeDestruction;
var
MonitorPtr: PPMonitor;
begin
MonitorPtr := PPMonitor(Integer(Self) + InstanceSize - hfFieldSize + hfMonitorOffset);
Assert(MonitorPtr^ = nil);
inherited;
end;
If this is a problem in your original code you should be able to detect it in the Delphi 5 version of your DLL by using the FastMM4 memory manager with all checks activated. OTOH this could also be caused by the size increase of character data in Unicode builds, and in that case it would only manifest in DLL builds using Delphi 2009 or 2010. It would still be a good idea to use the latest FastMM4 with all checks.
Edit:
From your stack trace it looks like the monitor is indeed assigned. To find out why I would use a data breakpoint. I haven't been able to make them work with Delphi 2009, but you can do it easily with WinDbg.
In the OnCreate handler of your form put the following:
var
MonitorPtr: PPMonitor;
begin
MonitorPtr := PPMonitor(Integer(Self) + InstanceSize - hfFieldSize + hfMonitorOffset);
MessageDlg(Format('MonitorPtr: %p', [pointer(MonitorPtr)]), mtInformation,
[mbOK], 0);
DebugBreak;
// ...
Now load WinDbg and open and run the process that calls your DLL. When the form is created a message box will show you the address of the monitor instance. Write down the address, and click OK. The debugger will come up, and you set a breakpoint on write access to that pointer, like so:
ba w4 A32D00
replacing A32D00 with the correct address from the message box. Continue the execution, and the debugger should hit the breakpoint when the monitor gets assigned. Using the various debugger views (modules, threads, stack) you may get important information about the code that writes to that address.
An invalid pointer operation means your program attempted to free a pointer, but there was one of three things wrong with it:
It was allocated by some other memory manager.
It had already been freed once before.
It had never been allocated by anything.
It's unlikely that you'd have multiple memory managers allocating TMonitor records, so I think we can rule out the first possibility.
As for the second possibility, if there's a class in your program that either doesn't have a custom destructor or that doesn't free any memory in its destructor, then the first actual memory deallocation for that object could be in TObject, where it frees the object's monitor. If you have an instance of that class and you attempt to free it twice, that problem could appear in the form of an exception in TMonitor. Look for double-free errors in your program. The debugging options in FastMM can help you with that. Also, when you get that exception, use the call stack to find out how you got to TMonitor's destructor.
If the third possibility is the cause, then you have memory corruption. If you have code that makes assumptions about the size of an object, then that could be the cause. TObject is four bytes larger as of Delphi 2009. Always use the InstanceSize method to get an object's size; don't just add up the size of all its fields or use a magic number.
You say the threads are created by Outlook. Have you set the IsMultithread global variable? Your program normally sets it to True when it creates a thread, but if you're not the one creating threads, it will remain at its default False value, which affects whether the memory manager bothers to protects its global data structures during allocation and deallocation. Set it to True in your DPR file's main program block.
After a lot of digging it turns out I was doing a nice (read: horrifying, but it has been properly doing its job in our delphi 5 apps for ages)
PClass(TForm)^ := TMyOwnClass
somewhere deep down in the bowels of our application framework. Apparently Delphi 2010 has some class initialization to initialize the "monitor field" that now didn't happen, causing the RTL to try and "free the syncobject" upon form destruction because getFieldAddress returned a non-nil value. Ugh.
The reason why we were doing this hack in the first place was because I wanted to automatically change the createParams on all form instances, to achieve an iconless resizable form. I will open up a new question on how to do this without rtl-breaking hacks (and for now will simply add a nice shiny icon to the forms).
I will mark Mghie's suggestion as the answer, because it has provided me (and anyone reading this thread) with a very large amount of insight. Thanks everyone for contributing!
There are two TMonitor in Delphi:
System.TMonitor; which is a record, and is used for thread synchronization.
Forms.TMonitor; which is a class representing an attached monitor (display device).
System.TMonitor is added to Delphi since Delphi 2009; so if you are porting a code from Delphi 5, what your code was using was Forms.TMonitor, not System.TMonitor.
I think the class name is referenced without unit name in your code, and that is making the confusion.