Indy, TidNotify and closing TidTCPServer - connection

I have a TidTCPServer which use database manipulating inside onExcecute event (by using TidNotify). Everything works very good instead of possibility closing application.
During closing application I do not know whether everything Notify instances finished their work or not and usually I get Runtime Error 216 (I think I close database before "notify" work end).
Is any way to check - are there waiting old Notify posts or not to be sure I can close application.
Other question is how to protect TidTCPServer from accepting new connection during closing server process.
I use code like below but I obtain the error still.
type
TShutdownThread = class(TThread)
protected
procedure Execute; override;
end;
procedure TShutdownThread.Execute;
begin
IdTCPServer.Active := false;
end;
//closing...
if IdTCPServer.Active then
begin
with TShutdownThread.Create(false) do
try
WaitFor; // internally processes sync requests...
finally
Free;
end;
end;

Is any way to check - are there
waiting old Notify posts or not to be sure I can close
application.
TIdNotify is asynchronous, it posts requests to the main thread message queue for later execution. It is possible that pending requests are still in the queue after TShutdownThread.WaitFor() has exited. You can call the RTL's CheckSynchronize() function to process any remaining requests, eg:
if IdTCPServer.Active then
begin
with TShutdownThread.Create(false) do
try
WaitFor;
finally
Free;
end;
CheckSynchronize;
end;
how to protect TidTCPServer from accepting new connection during closing server process.
While TIdTCPServer is being deactivated, it closes its listening port(s) for you. However, there is a very small window of opportunity when new clients could be accepted before the server closes the port(s). The server will close those connections as part of its shutdown, but if you do not want the OnExecute event to be called for those connections then you can set a flag somewhere in your code before deactivating the server, then check for that flag in the OnConnect event, and if it is set then disconnect the client immediately, eg:
var
ShuttingDown: boolean = False;
procedure TForm1.IdTCPServer1Connect(AContext: TIdContext);
begin
if ShuttingDown then
begin
AContext.Connection.Disconnect;
Exit;
end;
...
end;
...
if IdTCPServer.Active then
begin
ShuttingDown := True;
try
with TShutdownThread.Create(false) do
try
WaitFor;
finally
Free;
end;
CheckSynchronize;
finally
ShuttingDown := False;
end;
end;

Related

Indy / Libssl32 access violation in SSL_accept at TIdTCPServer stop

I use Delphi 10.1 Update 2 and Indy 10.6.2.5341.
We experience access violations in SSL_accept. This happens if a TIdTCPServer is setup using SSL and there is an open connection that has NOT yet negotiated TLS if the TIdTCPServer is stopped.
This looks like a problem in Libssl32 or Indy. This can be simply reproduced with the following code and Putty using a RAW connection. Does anyone knows a solution (or workaround) to prevent these crashes?
procedure TSslCrash.HandlerOnExecute(AContext: TIdContext);
begin
//
end;
procedure TSslCrash.HandlerOnConnect(AContext: TIdContext);
begin
TIdSSLIOHandlerSocketBase(AContext.Connection.IOHandler).PassThrough := False;
end;
procedure TSslCrash.ButtonStartClick(Sender: TObject);
begin
LServer := TIdTCPServer.Create;
LIOHandler := TIdServerIOHandlerSSLOpenSSL.Create;
LIOHandler.SSLOptions.Mode := sslmServer;
LIOHandler.SSLOptions.Method := sslvTLSv1_2;
LIOHandler.SSLOptions.VerifyMode := [];
LIOHandler.SSLOptions.VerifyDepth := 0;
LIOHandler.SSLOptions.CertFile := 'localhost.crt';
LIOHandler.SSLOptions.RootCertFile := 'localhost.crt';
LIOHandler.SSLOptions.KeyFile := 'localhost.key';
LServer.Bindings.Add.Port := 10000;
LServer.IOHandler := LIOHandler;
LServer.OnExecute := HandlerOnExecute;
LServer.OnConnect := HandlerOnConnect;
LServer.Active := True;
//Now open a RAW connection with Putty on port 10000 and keep it open
end;
procedure TSslCrash.ButtonStopClick(Sender: TObject);
begin
if Assigned(LServer) then begin
LServer.Active := False; //This causes an AV in TIdSSLSocket.Accept
FreeAndNil(LIOHandler);
FreeAndNil(LServer);
end;
end;
When Putty is connected in Raw mode, there is no SSL/TLS handshake performed, so SSL_accept() is stuck waiting for a handshake request that never arrives.
When TIdTCPServer is being deactivated, it disconnects active socket connections, failing any blocking socket operations in progress in other threads. In the case of SSL_accept(), that should unblock it so it can exit with an error code that TIdSSLSocket.Accept() can then detect and wrap into a raised exception (EIdOSSLUnderlyingCryptoError, EIdOSSLAcceptError, EIdSocketError, etc depending on the nature of the error code) in the context of the client thread that is waiting for the handshake to complete.
However, when TIdTCPServer is disconnecting a socket connection during deactivation, TIdTCPConnection.Disconnect() is called, which calls TIdIOHandler.Close(), which TIdSSLIOhandlerSocketOpenSSL has overridden to free its internal TIdSSLSocket object - the same object that is calling SSL_accept(). So it is quite likely that the underlying OpenSSL SSL object is being freed in TIdSSLSocket.Destroy() (which calls SSL_shutdown() and SSL_free()) in the context of the deactivating thread while still actively being used in TIdSSLObject.Accept() (which calls SSL_accept()) in the context of the client thread, thus causing an Access Violation.
There is not much that can be done about this without altering Indy's source code. For instance, maybe change TIdCustomTCPServer.DoTerminateContext() to call AContext.Binding.CloseSocket() instead of AContext.Connection.Disconnect(False) so the IOHandler itself is not closed, just the underlying socket (similar to what TIdCustomTCPServer.StopListening() does when terminating its listening accept() threads).
I have opened a ticket in Indy's issue tracker for you:
#218: Access Violation in SSL_accept() when deactivating TIdTCPServer

What is the proper way to free and terminate a thread which uses the TIdTCPClient component?

I'm using Delphi 10 Seattle to build a simple Client/Server application using the TIdTCPClientand TIdTCPServer components.
For read the data arrived from the server application (TIdTCPServer) I'm using a Thread in the Client Application.
This is the Execute method
procedure TClientReadThread.Execute;
begin
while not Terminated do
begin
try
if FClient.Connected() then //FClient is TIdTCPClient
begin
if not FClient.IOHandler.InputBufferIsEmpty then
begin
AResponse := FClient.IOHandler.ReadLn();
Synchronize(NotifyReadln);
end
else
FClient.IOHandler.CheckForDataOnSource(10);
end;
except
on E: Exception do
begin
// Send the exception message to the logger
FE:=E;
Synchronize(LogException);
end;
end;
end;
end;
Under normal circumstances all is working fine, but now I'm doing some tests to restore the connection on the client application in case which the server or the network is down. So I shutdown the server App to simulate a issue when the comm fails.
When that happens the client application detects which the server is gone using the TIdTCPClient.OnStatus event.
After that I try to terminate the reading thread using this code
if Assigned(FClientReadThr) then
begin
FClientReadThr.Terminate;
FClientReadThr.WaitFor; // This never returns.
FreeAndNil(FClientReadThr);
end;
But the WaitFor function never returns.
SO the question is , there is something wrong on my execute procedure which is preventing the finalization of the thread?
Exist a better way to terminate the thread?
First, you should not be using Connected() in this manner. Just call ReadLn() unconditionally and let it raise an exception if an error/disconnect occurs:
procedure TClientReadThread.Execute;
begin
while not Terminated do
begin
try
AResponse := FClient.IOHandler.ReadLn();
Synchronize(NotifyReadln);
except
// ...
end;
end;
end;
If you want to poll the socket for data manually, it should look more like this:
procedure TClientReadThread.Execute;
begin
while not Terminated do
begin
try
if FClient.IOHandler.InputBufferIsEmpty then
begin
FClient.IOHandler.CheckForDataOnSource(10);
FClient.IOHandler.CheckForDisconnect;
if FClient.IOHandler.InputBufferIsEmpty then Continue;
end;
AResponse := FClient.IOHandler.ReadLn();
Synchronize(NotifyReadln);
except
// ...
end;
end;
end;
DO NOT use the TIdTCPClient.OnStatus event to detect a disconnect in this situation. You are deadlocking your code if you are terminating the thread directly in the OnStatus event handler. That event will be called in the context of the thread, since the thread is the one reading the connection and detecting the disconnect. So your thread ends up waiting on itself, that is why WaitFor() does not exit.
I would suggest an alternative approach. DON'T terminate the thread at all. To recover the connection, add another level of looping to the thread and let it detect the disconnect and reconnect automatically:
procedure TClientReadThread.Execute;
var
I: Integer;
begin
while not Terminated do
begin
try
// don't call Connect() in the main thread anymore, do it here instead
FClient.Connect;
except
// Send the exception message to the logger
// you should wait a few seconds before attempting to reconnect,
// don't flood the network with connection requests...
for I := 1 to 5 do
begin
if Terminated then Exit;
Sleep(1000);
end;
Continue;
end;
try
try
while not Terminated do
begin
AResponse := FClient.IOHandler.ReadLn();
Synchronize(NotifyReadln);
end;
except
// Send the exception message to the logger
end;
finally
FClient.Disconnect;
end;
end;
end;
You can then Terminate() and WaitFor() the thread normally when you want to stop using your socket I/O.

the Difference of TIdTCPConnection.Disconnect and TIdIOHandler.Close

I use Indy 10.6.2.5298.
What is the difference of TIdTCPConnection.Disconnect and TIdIOHandler.Close? Both of them disconnect the line but sometimes the former makes an access violation error.
I am sorry that I can't understand it through the help documents and their source codes.
type
TForm1 = class(TForm)
IdTCPServer1: TIdTCPServer;
procedure FormClick(Sender: TObject);
procedure IdTCPServer1Execute(AContext: TIdContext);
private
TestContext: TIdContext;
end;
procedure TForm1.FormClick(Sender: TObject);
begin
TestContext.Connection.Disconnect; // access violation
TestContext.Connection.IOHandler.Close; // always works well
end;
procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);
begin
TestContext := AContext;
AContext.Connection.Disconnect; // works well
end;
TIdTCPConnection.Disconnect() calls IOHandler.Close() internally, if an IOHandler is assigned and has been opened (it also calls TIdTCPConnection.DisconnectNotifyPeer() and triggers the OnDisconnected and OnStatus events):
procedure TIdTCPConnection.Disconnect(ANotifyPeer: Boolean);
var
// under ARC, convert a weak reference to a strong reference before working with it
LIOHandler: TIdIOHandler;
begin
try
// Separately to avoid calling .Connected unless needed
if ANotifyPeer then begin
// TODO: do not call Connected() here if DisconnectNotifyPeer() is not
// overriden. Ideally, Connected() should be called by overridden
// DisconnectNotifyPeer() implementations if they really need it. But
// to avoid any breakages in third-party overrides, we could check here
// if DisconnectNotifyPeer() has been overridden and then call Connected()
// to maintain existing behavior...
//
try
if Connected then begin
DisconnectNotifyPeer;
end;
except
// TODO: maybe allow only EIdConnClosedGracefully and EIdSocketError?
end;
end;
finally
{
there are a few possible situations here:
1) we are still connected, then everything works as before,
status disconnecting, then disconnect, status disconnected
2) we are not connected, and this is just some "rogue" call to
disconnect(), then nothing happens
3) we are not connected, because ClosedGracefully, then
LConnected will be false, but the implicit call to
CheckForDisconnect (inside Connected) will call the events
}
// We dont check connected here - we realy dont care about actual socket state
// Here we just want to close the actual IOHandler. It is very possible for a
// socket to be disconnected but the IOHandler still open. In this case we only
// care of the IOHandler is still open.
//
// This is especially important if the socket has been disconnected with error, at this
// point we just want to ignore it and checking .Connected would trigger this. We
// just want to close. For some reason NS 7.1 (And only 7.1, not 7.0 or Mozilla) cause
// CONNABORTED. So its extra important we just disconnect without checking socket state.
LIOHandler := IOHandler;
if Assigned(LIOHandler) then begin
if LIOHandler.Opened then begin
DoStatus(hsDisconnecting);
LIOHandler.Close;
DoOnDisconnected;
DoStatus(hsDisconnected);
//LIOHandler.InputBuffer.Clear;
end;
end;
end;
end;
TIdIOHandler.Close() simply closes the socket, if one has been allocated:
procedure TIdIOHandlerSocket.Close;
begin
if FBinding <> nil then begin
FBinding.CloseSocket;
end;
inherited Close;
end;
procedure TIdIOHandler.Close;
//do not do FInputBuffer.Clear; here.
//it breaks reading when remote connection does a disconnect
var
// under ARC, convert a weak reference to a strong reference before working with it
LIntercept: TIdConnectionIntercept;
begin
try
LIntercept := Intercept;
if LIntercept <> nil then begin
LIntercept.Disconnect;
end;
finally
FOpened := False;
WriteBufferClear;
end;
end;
The reason for your access violation error is likely because your test code is not thread-safe to begin with. TIdTCPServer is a multi-threaded component. Its OnConnect, OnDisconnect, OnExecute, and OnException events are triggered in the context of a worker thread that manages the TIdContext object. Your OnClick handler is accessing the TIdContext object outside of that thread. As soon as the socket is closed, TIdTCPServer will detect that and stop the thread, destroying the TIdContext and its TIdTCPConnection and TIdIOHandler objects. Due to thread timing and context switching, your OnClick handler may very well be continuing to access those objects after they have been destroyed. You don't have that problem inside of the OnExecute handler because the objects are still valid while the thread is running.
To make your OnClick code play nice with TIdTCPServer, you need to lock the TIdTCPServer.Contexts list so the TIdContext object cannot be destroyed while OnClick is still trying to use it, eg:
procedure TForm1.FormClick(Sender: TObject);
var
List: TIdContextList;
begin
List := IdTCPServer1.Contexts.LockList;
try
//has the context already been removed?
if List.IndexOf(TestContext) <> -1 then
TestContext.Connection.Disconnect;
finally
IdTCPServer1.Contexts.UnlockList;
end;
end;

Cancel an ADO Connection's attempt to connect?

I have a TADOConnection inside a thread. In the event that it fails to connect to the database (timeout), when closing the app, the thread is held up and it takes time until the attempt is finished before my app is able to close. I don't want to reduce the connection timeout of the connection, this isn't the issue. Is there any way I can forcefully abort the attempt to connect?
The TADOConnection connects at the beginning of the thread execution and automatically reconnects repeatedly until success. Then, upon closing the app, if the database is failing to connect, the thread hangs until the connection attempt is finished (timed out).
EDIT
This is a sample of how the thread works:
procedure TMyThread.Init;
begin
CoInitialize(nil);
FDB:= TADOConnection.Create(nil);
FDB.LoginPrompt:= False;
FDB.ConnectionTimeout:= 5;
FDB.ConnectOptions:= coAsyncConnect;
end;
procedure TMyThread.Uninit;
begin
if FDB.Connected then
FDB.Connected:= False;
FDB.Free;
CoUninitialize;
end;
function TMyThread.Reconnect: Boolean;
begin
Result:= False;
if FDB.Connected then
FDB.Connected:= False;
FDB.ConnectionString:= FConnectionString;
try
FDB.Connected:= True; //How to abort?
Result:= True;
except
on e: exception do begin
//MessageDlg(e.Message, mtError, [mbOK], 0);
FDB.Connected:= False;
Result:= False;
end;
end;
end;
procedure TMyThread.Process;
begin
if Reconnect then begin //Once connected, keep alive in loop
while FActive do begin
if Terminated then Break;
if not Connected then Break;
//Do Some Database Work
end;
end else begin
//Log connection failure
end;
end;
procedure TMyThread.Execute;
begin
while not Terminated do begin
if FActive then begin
Init; //CoInitialize, create DB, etc.
try
while (FActive) and (not Terminated) do begin
try
Process; //Actual processing procedure
except
on e: exception do begin
//Record error to log
end;
end;
end;
finally
Uninit; //CoUninitialize, destroy DB, etc.
end;
end;
end;
end;
(Tried to include just relevant things to the question)
First thing that comes to mind is to reduce connection's timeout. Why do you not want that? And why do you want to establish a connection when closing the application? Especially when you prefer to abort it when it takes more time than expected, why connect at all? Sounds like we could know more background info.
In the special case that you really need it on the condition that it connects quickly, ánd when this issue only applies to application's destruction, then I suggest not to wait for the thread to finish. Just do not free it, terminate the application, and let Windows kill the process including all its threads.
In the case that the connection does succeed, then this approach could backfire, so signal your main thread when the thread dóes connect, and postpone its termination by yet waiting for the thread. You may need another timeout for that again.
Edit:
I suppose the OnWillConnect event will occur every time the attempt to connect is made. Try returning EventStatus := esCancel within its handler.

Delphi How to wait for socket answer inside procedure?

For some specific needs i need to create procedure that waits for socket request (or answer) in dll:
TForm1 = class(TForm)
ServerSocket1: TServerSocket;
......
procedure MyWaitProc; stdcall;
begin
Go := false;
while not Go do
begin
// Wating...
// Application.ProcessMessages; // Works with this line
end;
end;
procedure TForm1.ServerSocket1ClientRead(Sender: TObject;
Socket: TCustomWinSocket);
begin
MessageBoxA(0, PAnsiChar('Received: '+Socket.ReceiveText), '', MB_OK);
Go := true;
end;
exports
MyWaitProc;
When I call Application.ProcessMessages everything works fine: application waits for request and then continues. But in my case calling Application.ProcessMessages causes to unlocking main form on host application (not dll's one). When I don't call Application.ProcessMessages application just hangs couse it cannot handle message...
So, how to create such a procedure that's wating for socket answer ?
Maybe there a way to wait for socket answer without using Application.ProcessMessages ?
EDIT
I also tried to use TIdTCPServer, for some reasons, the result is the same.
TForm1 = class(TForm)
IdTCPServer1: TIdTCPServer;
.....
procedure MyWaitProc; stdcall;
begin
Go := false;
while not Go do
begin
// Waiting ...
// Application.ProcessMessages;
end;
end;
procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);
var
s: string;
begin
s := AContext.Connection.Socket.ReadString(1);
AllText := AllText + s;
Go := True;
end;
TServerSocket runs in non-blocking mode by default, which depends on processing window messages. To remove that dependancy, you have to switch it to blocking mode instead.
TIdTCPServer runs in blocking mode exclusively, so no window messages. If you are having a problem with it, then you are misusing it. For example, in your TServerSocket code, you do not set Go = True until after a response has been received, but in your TServerSocket code you are setting Go = True before reading a response instead.
As an alternative, have a look at Indy's TIdSimpleServer component. TIdSimpleServer is synchronous and only accepts 1 connection at a time, whereas TIdTCPServer is asynchronous and accepts many connections at a time. For example:
TForm1 = class(TForm)
ServerSocket: TIdSimpleServer;
procedure MyWaitProc; stdcall;
var
s: String;
begin
ServerSocket.Listen;
s := ServerSocket.IOHandler.ReadLn;
ServerSocket.Disconnect;
MessageBox(0, PChar('Received: '+s), '', MB_OK);
end;
exports
MyWaitProc;
Rather than creating a loop that occasionally calls Application.ProcessMessages you can create a descendant of TThread and move the socket request to the TThread.Execute method. Use TThread.OnTerminate to notify your form(or any other class) when the thread has completed its work.
There is sample code which gives more details about how to use TThread.
There are several other 3rd party threading libraries that either provide more flexibility or are easier to use than TThread and I would highly recommend any of them over TThread if you are new to multi-threading.
Note: There are some serious side-effects to using Application.ProcessMessages. You are seeing one of them in your code with the dll unlocking the application's mainform. It breaks the single-threaded UI model the VCL is build upon. ProcessMessages has its place but using threads is more appropriate for the situation you're describing.
var Slowpoke: TMyLongRunningProcessThread;
procedure MyWaitProc(Completed:TNotifyEvent)
begin
Slowpoke := TMyLongRunningProcessThread.Create(True);
Slowpoke.FreeOnTerminate := True;
Slowpoke.OnTerminate := Completed;
Slowpoke.Resume;
end;
MyWaitProc returns immediately after starting the thread so the GUI is free to respond to user actions. When the thread terminates it calls the event handler pointed to by Completed.
Obviously if you need to retrieve data from the thread you'll want to either have the thread write to an accessible memory location before it Frees itself or remove the FreeOnTerminate so the data can be retreived from the thread through a property.

Resources