I have a problem in a dll (including a COM object): when the dll is unloaded some finalization sections are executed and some are not.
in the debugger I could manage to locate the problem in System FinalizeUnits(). pseudo code of this function - for your convenience:
procedure FinalizeUnits;
var
Count: Integer;
Table: PUnitEntryTable;
P: Pointer;
begin
if InitContext.InitTable = nil then
exit;
Count := InitContext.InitCount;
Table := InitContext.InitTable^.UnitInfo;
try
while Count > 0 do
begin
Dec(Count);
InitContext.InitCount := Count;
P := Table^[Count].FInit;
if Assigned(P) and Assigned(Pointer(P^)) then
begin
// ISSUE: when this is called for item x the debugging just stops
// breakpoints in the except block or at the end of the function are not reached!
TProc(P)();
end;
end;
except
FinalizeUnits; { try to finalize the others }
raise;
end;
end;
there is one specific finalization call that causes the problem:
i.e. the InitContext.InitCount is about 400 and when item x (e.g. 363) is executed, the debugger just stops: it does not proceed to the except block and also not to the end of the FinalizeUnits() function (where I have set breakpoints).
BTW: how is that possible? I thought the except block (or the line after it) must be called in any case.
notes:
when I manually avoid this special call, all other functions are executed normally.
when I step into the problematic TProc I end up in TCriticalSection.Enter (then Acquire - FSection.Enter - EnterCriticalSection - WSACleanup)
some more info to the WSACleanup call: I use a 3rd party library UniDAC that opens a TCP/IP connection to a database - I think this library calls WSACleanup in one of it's finalization sections (but I don't have this source code).
The strange thing is, that when the debugger is on the WSACleanup line:
function WSACleanup; external winsocket name 'WSACleanup';
and I want to step over it (i.e. F8), the debugger just stops (as if the application had exited normally) - but it should continue the loop in FinalizeUnits: how is this possible? i.e. if it were a deadlock it would not stop, but hang forever, right?
The question is: how can I debug this issue? is it possible that a deadlock causes this problem (i.e. that the debugger just stops)?
Try switching to CPU view before stepping into the problematic TProc using F7. Sometimes this can give you a good hint.
You could also try looking up the address of "P" in the map file.
Related
I maintain an application that runs as a service in a server environment. It is multithreaded, where each thread does work according to a task queue. This task queue is just a list of strings with "job types" as their values. So while several threads may be running, each thread would be a different job, and each thread internally runs just one task at a time.
I'm experiencing an intermittent issue that happens when calling Open on a TADODataSet. Sometimes, not always, and with no discernible pattern, Data.Win.ADODB will throw an EArgumentOutOfRangeException, bypassing my own attempt to catch any exceptions. This exception hangs the entire thread and prevents future execution from being possible until I completely restart the service.
Being relatively new to the world of Delphi, I've been scratching my head at this issue for quite some time, and have struggled to find any answers. My question is: why is this happening, and how do I stop, catch, or fix it?
Here is a snippet of my offending code. This is the method from which the stack trace originates. It is called from another method in the same unit, where I open a different dataset, loop through its records, and on each record call this function to get some info based on the value passed in.
function TFQFoo.DoSomething(IncNo : Int64): string;
var
ItemList : string;
MySQL: string;
ComponentString: string;
begin
result:='';
if IncNo<=0 then
Exit;
ItemList := '';
MyQuery.Close;
MySQL := 'select ID from tbl ' +
' where val = ' + IntToStr(IncNo) +
' order by col1 DESC, col2, col3';
try
try
MyQuery.CommandText := (MySQL);
MyQuery.Open;
while not (MyQuery.EOF) do
begin
if (ItemList <> '') then
ItemList := ItemList + ',';
ItemList := ItemList +
MyQuery.FieldbyName('ID').asstring;
MyQuery.Next;
end;
except
// exception handling code omitted for brevity -- none of it
// is ever reached, anyway (see below)
end;
finally
MyQuery.Close;
end;
Result := ItemList;
end;
The call stack from the exception indicates that it's occurring at Open. No try..catch block will capture the exception and log it for me -- I have to use EurekaLog in order to see any details. The stack trace methods (too big to post here) look like:
TFQFoo.DoSomething
TDataSet.Open
... internal things
TADOConnection.ExecuteComplete
CheckForAsyncExecute
...
TCustomConnection.GetDataSet
TListHelper.GetItemRange
Thinking possibly my TADODataSet component was somehow getting corrupted / its properties altered at runtime, I added some logging to capture that data for me so I could see if something funky was going on there. I didn't see anything, but here it is in case it's pertinent.
object MyQuery: TADODataSet
AutoCalcFields = False
CacheSize = 15
Connection = FGlobals.RIMSDB
CursorType = ctStatic
LockType = ltReadOnly
CommandText =
'select ID from tbl where val = 202005070074 order by col1 ' +
'DESC, col2, col3'
ParamCheck = False
Parameters = <>
Left = 32
Top = 216
end
For the curious, this is the method actually throwing the exception, from Data.Win.ADODB. Note the except, which I guess hops over my own try..catch block and sends the exception straight to EurekaLog.
procedure CheckForAsyncExecute;
var
I: Integer;
begin
try
if not Assigned(pError) and Assigned(pRecordset) and
((pRecordset.State and adStateOpen) <> 0) then
for I := 0 to DataSetCount - 1 do
if (DataSets[I].Recordset = pRecordset) and (eoAsyncExecute in DataSets[I].ExecuteOptions) then
begin
DataSets[I].OpenCursorComplete;
Break;
end;
except
ApplicationHandleException(Self);
end;
end;
What I have tried:
Many, many iterations of tweaking component properties on the ADODataSet itself
Using the CommandText and Parameters from within the designer, and assigning the parameter prior to execution at runtime
Adding / removing a (NOLOCK) hint to the query itself
Taken the problem to senior members of my team for input
Googling (for hours)
Reading up on Delphi and ADO documentation (not fruitful for this)
Attempted reproduction - I've never been able to get this to occur on any test system I use. This leads me to think it may be environment-related, but I have absolutely no clue how
My question, restated:
How do I stop this from happening? What am I doing wrong? I don't want to just catch the EArgumentOutOfRangeException; I want to learn why it's happening in the first place and prevent it from happening in the future.
I know that sometimes, the query execution will not return results, but the typical CommandText does not return a result set message is never seen nor encountered due to the lower-level code bypassing my own catch statement. Beyond this, I don't know what else to look for.
I've only found one other occurrence of something similar so far, but it relates just to the exception not getting caught: http://www.delphigroups.info/2/d9/410191.html
The call to ApplicationHandleException(Self) in CheckForAsyncExecute() is swallowing exceptions, which is why your except block is not being triggered:
// in Data.Win.ADODB.pas:
procedure CheckForAsyncExecute;
var
I: Integer;
begin
try
...
except
ApplicationHandleException(Self); // <-- a caught exception is NOT re-raised here!
end;
end;
Inside of the Data.Win.ADODB unit, caught exceptions will call the unit's own ApplicationHandleException() function, which then calls System.Classes.ApplicationHandleException if assigned, otherwise it simply exits:
// in Data.Win.ADODB.pas:
procedure ApplicationHandleException(Sender: TObject);
begin
if Assigned(System.Classes.ApplicationHandleException) then
System.Classes.ApplicationHandleException(Sender);
end;
System.Classes.ApplicationHandleException is initialized to nil.
In both a VCL1 and FMX app, the TApplication constructor assigns the TApplication.HandleException() method to System.Classes.ApplicationHandleException, where HandleException() ignores EAbort exceptions, and calls the TApplication.OnException event handler (if assigned), the TApplication.ShowException() method, or the System.SyUtils.ShowException() function, depending on the type of exception being handled.
1: in a VCL TService app, TServiceApplication uses Vcl.Forms.TApplication internally.
TApplication.ShowException() displays the details of the exception to the user in a popup MessageBox and then exits, and System.SysUtils.ShowException() displays the details of the exception to the user in a Console or MessageBox and then exits.
So, at no point does ADO's CheckForAsyncExecute() re-raise a caught exception into user code. And needless to say, displaying a popup MessageBox in a service is not a good idea, as the user will likely not see it so they can dismiss it.
Of course, the best option would be to avoid the EArgumentOutOfRangeException exception from being raised in the first place. But there are other conditions that can also raise exceptions, too.
So, your only option to handle swallowed ADO exceptions yourself, and avoid popup MessageBoxes, is to assign a TApplication.OnException event handler (either directly, or via the TApplicationEvents component).
I had an issue where a file kept deleting on startup and I couldn't track down the code responsible. I wound up adding Vcl.Dialogs to all the units and creating an initialization section that looked like this:
initialization
begin
ShowMessage('Inside [Unit Name Here]');
end;
This was quite a pain. Is there an easy way to generate a list of forms/units in the order in which they fire off?
UPDATE: 2019-08-01 (Helpful MAP links)
Here are two links that may assist in understanding DELPHI map files
http://docwiki.embarcadero.com/RADStudio/Rio/en/API_%28%2A.map%29
Understanding Delphi MAP File
You really didn't need to go to all that trouble modifying your source units. I think you'll find that using the method below will find the misbehaving unit
much more quickly than somehow generating a list of units and then ploughing
your way through it.
If you look in System.Pas, you'll find a procedure InitUnits like this (from D7).
procedure InitUnits;
var
Count, I: Integer;
Table: PUnitEntryTable;
P: Pointer;
begin
if InitContext.InitTable = nil then
exit;
Count := InitContext.InitTable^.UnitCount;
I := 0;
Table := InitContext.InitTable^.UnitInfo;
[...]
try
while I < Count do
begin
P := Table^[I].Init;
Inc(I);
InitContext.InitCount := I;
if Assigned(P) then
begin
TProc(P)();
end;
end;
except
FinalizeUnits;
raise;
end;
end;
This is the code which causes the initialization code of each unit to be called. It works its way through the units and calls the initialization section (if any)
of each unit via the call
TProc(P)();
You can inspect the value of Count prior to the loop; don't be surprised if its upwards
of a couple of hundreds even for a relatively simple project.
Put a breakpoint on the TProc(P)(); line and right-click and set the PassCount to
half the value of Count. Run your app and when the breakpoint trips, check whether
the file has been deleted.
You can then do a binary search through the values of
Count (by continuing the current run if the file is still there, or resetting the app
and halving the Pass Count) to establish exactly which unit causes the file to be deleted.
Because you can use a binary search to do this, it will rapidly converge on the
unit which is deleting the file. Of course, you can trace into the unit's
initialization code (if it has been compiled with debug info) when the breakpoint
trips by pressing F7 on TProc(P)();
You can inspect the segments section of the map file. The entries with C=ICODE are those units with initialization parts in the order they are executed.
I got switched to other project at work and I noticed that Delphi XE2 debugger does not show the line that raised exception. When i got back at home i started to investigate. Then I found out that it can be disabled in Tools -> Options -> Debugger options and check Integrated debugging. Also I unchecked everything under Language exceptions in Exception types to ignore list. Notify on Language Exceptions left checked. Project -> Options -> Compiling, I have defaults there and Overflow and Range cheking enabled. I am running Debug build. I Cleaned it.
I have not noticed before, but now Delphi debugger doesn't give me the line when I call this code:
procedure TForm1.BitBtn1Click(Sender: TObject);
var
_List: TStringList;
begin
_List := TStringList.Create;
try
Caption := _List[0]; // 'List index out of bounds (0)' here
finally
FreeAndNil(_List);
end;
end;
but this works (provided only to show that debugger does show the line for some things):
{$R+} // Range check is ON
procedure TForm1.BitBtn2Click(Sender: TObject);
var
_myArray: array [1 .. 5] of string;
i: integer;
begin
for i := 0 to 5 do
begin
_myArray[i] := 'Element ' + IntToStr(i); // Range check error here
ShowMessage('myArray[' + IntToStr(i) + '] = ' + _myArray[i]);
end;
end;
What is happening here? How to make the debugger to show as much as possible?
Thank you.
Let me answer the question first.
How to make the compiler show as much as possible
The compiler shows you that the error is in the call to the btnclick.
The trick is to put a breakpoint on the first line of the btnclick proc with F5.
Then rebuild (!) the application and run again.
The execution will stop that the breakpoint.
Step through the code using F8 until the error shows up.
Put a breakpoint F5 on the line that generated the error.
Abort and rerun the application.
When you get to the second breakpoint instead of pressing F8, press F7 to step inside the routine that causes the error, keep on pressing F7/F8 until you see what exactly is causing the problem.
Why is this happening?
The compiler traces back the source of the exception by following the stack trace.
Because in your case the code that generated the exception does not have stack trace (because it is not debug code), the compiler cannot do this trick and instead follows the stack trace that is does have; it moves one level up in your code and flags the exception there.
A look at this in detail
You're comparing apples and oranges.
This code (Exhibit A):
Caption := _List[0]; // 'List index out of bounds (0)' here
Has absolutely nothing in common with this code (Exhibit B):
_myArray: array [1 .. 5] of string;
....
_myArray[0]:= 'Hallo';
Exhibit A uses the TStringList class's items property, which is defined more or less as follows (I've simplified it a bit, but the fundamentals are correct):
type
TStringList = class(TStrings)
strict private
FList: array of string;
....
private
procedure Put(index: integer; const value: string);
function Get(index: integer): string;
published
property Items[index: integer]: string read Get write Put; default;
// ------------------------------------------------------^^^^^^^
....
end;
Notice the default keyword at the end of the Items property.
What this all means is that when you call _List[0], you are really calling _List.Items[0], which gets translated into Caption:= _List.Getitems(0), because of the read modifier on the property.
The default keyword allows you to omit the .Items.
The Get code looks like this:
function TStringList.Get(Index: Integer): string;
begin
if Cardinal(Index) >= Cardinal(FCount) then
Error(#SListIndexError, Index); <<-Here is the line that generates the error*
Result := FList[Index].FString;
end;
*Actually the error is generated inside the Error routine
Unless you have the RTL/VCL source code and you're running with debug DCU's you will not get a break on the exact trigger of the exception (which is inside the system.classes) unit.
Note that this error does not depend on range checking, it will always fire.
Because Delphi does not have debug info for the exact line where the error is generated, it does the next best thing and tries to make a guess.
Short version
The stringlist is a complex abstraction pretending to be an array.
Lots of code gets called, making the pinpointing of the error difficult for the compiler.
In Exhibit B:
_myArray: array [1 .. 5] of string;
....
i:= 0;
_myArray[i]:= 'Hallo';
Either a range check error or an access violation is generated.
Both of these errors occur on the exact line, allowing the compiler to stop at the correct spot.
Short version
The plain array is a basic building block with no hidden calls to code elsewhere, making pinpointing errors very easy for the compiler.
Understanding properties
class and record properties (and now class operators) look like simple assignments/operations to/with variables, but are in fact calls to (possibly complex) subroutines.
i have a unit in Delphi that (has the option) to provide a single global object:
var
InternalThemeParkServices: TThemeParkServices;
function ThemeParkServices: TThemeParkServices;
begin
if InternalThemeParkServices= nil then
InternalThemeParkServices := TThemeParkServices.Create();
Result := InternalThemeParkServices ;
end;
...
initialization
finalization
FreeAndNil(InternalThemeServices);
end.
And i destroy myself during process shutdown.
Note: Another code variant is:
var
InternalThemeParkServices: IThemeParkServices;
function ThemeParkServices: TThemeParkServices;
begin
if InternalThemeParkServices= nil then
InternalThemeParkServices := TThemeParkServices.Create();
Result := InternalThemeParkServices ;
end;
Where the interface variable is implicitly destroyed when it's
reference count goes to zero during program shutdown
When my object is no longer used (i.e. during its destructor), i call call various WinAPI functions.
The problem is that if someone uses my class from a DLL (something which i cannot control), then anything being called during:
finalization
is the Delphi moral equivalent of DLL_PROCESS_DETACH. There are all kinds of things i should not be doing during DLL_PROCESS_DETACH when the process is terminating (e.g. CoCreateInstance).
i know Embarcadero uses:
initialization
if not IsLibrary then
begin
...
Which i perhaps i could adapt, changing my code from:
var
InternalThemeParkServices: IThemeParkServices;
(using implicit cleanup), to:
var
InternalThemeParkServices: IThemeParkServices;
...
finalization
if IsLibrary then
Pointer(InternalThemeParkServices) := nil; //defeat reference counting and let the object leak
end;
and let it leak.
But is this the best resolution? i assume it means that if the dll running my code is unloaded (but not during process shutdown) that i'll leak memory. If the dll is attached and detached, i'll leak each time.
What i really want is Delphi to run its finalization blocks before ExitProcess/DllMain(DLL_PROCESS_DETACH). Is this possible?
Bonus Chatter
#pa deciphered the Delphi application shutdown scheme:
The hierarchy of shutdown is as follows
Application.Terminate()
performs some unidentified housekeeping of application
calls Halt()
Halt()
calls ExitProc if set
alerts the user in case of runtime error
get rid of PackageLoad call contexts that might be pending
finalize all units
clear all exception handlers
call ExitprocessProc if set
and finally, call ExitProcess() from 'kernel32.dll'
ExitProcess()
unloads all DLLs
uses TerminateProcess() to kill the process
With DLLs being unloaded after a call to ExitProcess - because Windows is the one who does it.
To tell if you're being called during DLL_PROCESS_DETACH after ExitProcess has been called, you can write initialization code for your library so that your code is executed when FreeLibrary is called from the main program. The 'lpReserved' parameter will be '1' if ExitProcess have already been called, '0' otherwise:
..
var
SaveDllProcEx: TDllProcEx;
procedure DllMainEx(Reason: Integer; Reserved: Integer);
begin
if (Reason = DLL_PROCESS_DETACH) and (Reserved = 0) then
// Main app is still running, cleanup.
if Assigned(SaveDllProcEx) then
SaveDllProcEx(Reason, Reserved);
end;
initialization
if IsLibrary then begin
SaveDllProcEx := DllProcEx;
DllProcEx := #DllMainEx;
end;
From DllMain entry point:
The lpReserved parameter indicates whether the DLL is being unloaded
as a result of a FreeLibrary call, a failure to load, or process
termination.
What I really want is Delphi to run its finalization blocks before DllMain(DLL_PROCESS_DETACH). Is this possible?
No it is not possible.
If you need to perform shutdown actions that cannot be done during DllMain(DLL_PROCESS_DETACH) then you will need to add an exported function to your DLL that peforms the finalization. You should then require your DLL's clients to call this function before unloading the DLL. This is the same pattern as CoInitialize/CoUninitialize.
SOLVED
I am using delphi 2009. My program listens for usb drives being connected and remove. Ive used a very similar code in 10 apps over the past year. It has always worked perfectly. When i migrated i had to give up using thddinfo to get the drive model. This has been replaced by using WMI. The WMI query requires the physical disk number and i happen to already have a function in the app for doing just that.
As i test I put this in a button and ran it and it successfully determines the psp is physical drive 4 and returns the model (all checked in the debugger and in another example using show message):
function IsPSP(Drive: String):Boolean;
var
Model: String;
DriveNum: Byte;
begin
Result := False;
Delete(Drive, 2, MaxInt);
DriveNum := GetPhysicalDiskNumber(Drive[1]);
Model := (MagWmiGetDiskModel(DriveNum));
if Pos('PSP',Model) > 0 then Result := True;
end;
procedure TfrmMain.Button1Click(Sender: TObject);
var DriveNum: Byte;
begin
IsPSP('I');
end;
It works perfectly that is until i allow the WMDeviceChange that ive been using for a year to call up the getphysicaldisknumber and the wmi query statement. Ive tried them by themselves theyre both a problem. GetPhysicalDiskNumber freezes real bad when its doing a CloseHandle on the logical disk but does return the number eventually. The WMI query fails with no error just returns '' debugger points into the wbemscripting_tlb where the connection just never happened. Keep in mind the only thing thats changed in a year is what im calling to get the model i was using an api call and now im using something else.
Below is the rest of the code involved at this time sans the ispsp that is displayed above:
procedure TfrmMain.WMDeviceChange(var Msg: TMessage);
var Drive: String;
begin
case Msg.wParam of
DBT_DeviceArrival: if PDevBroadcastHdr(Msg.lParam)^.dbcd_devicetype = DBT_DevTyp_Volume then
begin
Drive := GetDrive(PDevBroadcastVolume(Msg.lParam)) + '\';
OnDeviceInsert(Drive);
end;
DBT_DeviceRemoveComplete: if PDevBroadcastHdr(Msg.lParam)^.dbcd_devicetype = DBT_DevTyp_Volume then
begin
Drive := GetDrive(PDevBroadcastVolume(Msg.lParam)) + '\';
OnDeviceRemove(Drive);
end;
end;
end;
Procedure TfrmMain.OnDeviceInsert(Drive: String);
var PreviousIndex: Integer;
begin
if (getdrivetype(Pchar(Drive))=DRIVE_REMOVABLE) then
begin
PreviousIndex := cbxDriveList.Items.IndexOf(cbxDriveList.Text);
cbxDriveList.Items.Append(Drive);
if PreviousIndex = -1 then //If there was no drive to begin with then set index to 0
begin
PreviousIndex := 0;
cbxDriveList.ItemIndex := 0;
end;
if isPSP(Drive) then
begin
if MessageDlg('A PSP was detect # ' + Drive + #10#13 + 'Would you like to select this drive?',mtWarning,[mbYes,mbNo], 0) = mrYes then
cbxDriveList.ItemIndex := cbxDriveList.Items.IndexOf(Drive)
else cbxDriveList.ItemIndex := PreviousIndex;
end
else if MessageDlg('USB Drive ' + Drive + ' Detected' + #10#13 + 'Is this your target drive?',mtWarning,[mbYes,mbNo], 0) = mrYes then
cbxDriveList.ItemIndex := cbxDriveList.Items.IndexOf(Drive)
else cbxDriveList.ItemIndex := PreviousIndex;
end;
end;
Procedure TfrmMain.OnDeviceRemove(Drive: String);
begin
if not (getdrivetype(Pchar(Drive)) = DRIVE_CDROM) then
begin
if cbxDriveList.Text = (Drive) then ShowMessage('The selected drive (' + Drive + ') has been removed');
cbxDriveList.Items.Delete(cbxDriveList.Items.IndexOf(Drive));
if cbxDriveList.Text = '' then cbxDriveList.ItemIndex := 0;
if Drive = PSPDrive then //Check Detect PSP and remove reference if its been removed
begin
PSPDrive := '';
end;
end;
end;
Rob has said something below about im not calling the inherited message handler, ive read the document i see a couple of things i can return... but im not really sure i understand but i will look into it. Im not a very good pascal programmer but ive been learning alot. The transition to 2009 has had some rough patches as well.
The USB drive detection and all that works perfectly. If i remove the two things from is psp the user is greeted right away with wis this your whatever and adds I:\ to the list. Its just the two new things that have changed in the app that fail when called by wmdevicechange and as said before they work on their own.
EDIT - SOLVED
Alright well im using a timer as suggested and the problem seems to be solved. One note is that when called by the timer very shortly after the wmdevicechange getting the physical disk number still seems to be slow. I attribute this to the device still being attached to the system.
On that note im using a P2 450 on the regular. I hooked the PSP and app to a 1.8Ghz Dual Core Laptop and the program detected the psp and notified the user very fast. So the app wont freeze unless there on a very very slow computer and on this slow onw its only for a matter of seconds and doesnt affect the operation of the program though isnt very cool. But i feel that all modern computers will run the detection fast especially because they can attach the device alot faster.
It's possible that the information you're querying becomes available only after the WMDeviceChange message handler runs. If the very same code works when called from a button, try this:
Refactor your WMDeviceChange handler code into one or more separate methods.
In the WMDeviceChange handler, activate a precreated timer and have it fire one second later, or something like that.
Call the former WMDeviceChange handler code from the timer handler code.
You haven't indicated what "statement 1" is in your code.
I have a few comments about parts of the code, which may or may not be related to the problem you're having.
First, you assign a value to DriveNum in IsPSP, but you don't use it. The compiler should have issued a hint about that; don't ignore hints and warnings. You also pass the magic number 4 into MagWmiGetDiskModel; was that supposed to be DriveNum instead?
You aren't calling the inherited message handler, and you aren't returning a result in your message handler. The documentation tells what values you're supposed to return. To return a value from a Delphi message handler, assign a value to the Msg.Result field. For the cases that your message handler doesn't handle, make sure you call inherited so that the next handler up the chain can take care of them. If there is no next handler, then Delphi will call DefWindowProc to get the operating system's default behavior.
The change you've illustrated is called refactoring, and it will do nothing to affect how your code runs. It makes the code easier to read, though, so please keep the second version. As for finding the problem, my best advice is to use the debugger to step through the code to identify the point where things stat to go wrong and the parts that run slower than you'd like. You can also try removing portions of the code to confirm that the other parts work correctly in isolation.