TIdTCPServer not able to send data after re-connection of client - delphi

I am using TIdTCPServer in my application. Here hardware acting as tcp client. Client establishes the connection with SYN command (observed in wire shark which is a tool). My application has multiple clients so every client connects to my server. For the first time connection, data sending & receiving is fine. But when hardware power off & on happens, my server not able to send data, to hardware, until application restart. Following are the observations regarding this:
1.When first time client connects SYN with Seq No = 0 received, SYN ACK with Seq No = 1 send to the client from server
2.Data sending & receiving are working fine
3.Hardware power off happened
4.In command prompt by using “netstat” i observed there is connection established for the disconnected IP & port number.
5.I send some data (in wire shark it displayed 6 times retransmission)
6.After this in “command prompt” corresponded connection established data not appeared
7.I send data to client now "Connection closed" exception raised by the IdTCPServer (after this exception, in on except, i closed the connection by using connection.disconnect in code & deleted that particular client from Locklist of IdTCPServer.)
8.Hardware powered on & send SYN with Seq No = 0
9.In wire shark SYN ACK with Seq No like 45678452 sent to hardware
10.After that in command prompt connection establishment was observed
11.I tried to send data to client, but “Locklist” not updated with the client IP& port again so data not sent the client (my code is like if IP not present in “Locklist” then not sending data).
Is there any solutions?
Following is my code:
try
for Count := 0 to frmtcpserver.IdTCPServer1.Contexts.LockList.Count - 1
do
begin
if TIdContext(frmtcpserver.IdTCPServer1.Contexts.LockList.Items[Count]).Binding.PeerIP = Destination_IP then
begin
DestinationIPIdx := Count;
end;
end;
frmtcpserver.IdTCPServer1.Contexts.UnlockList;
if DestinationIPIdx > -1 then
begin
// sending data here
TIdContext(frmtcpserver.IdTCPServer1.Contexts.LockList.Items[DestinationIPIdx])
.Connection.IOHandler.Write(TempBuf, NoofBytesToSend,0);
end;
end;
on E: EidException do
begin
TIdContext(frmtcpserver.IdTCPServer1.Contexts.LockList.Items[DestinationIPIdx]).Connection.Disconnect;
frmtcpserver.IdTCPServer1.Contexts.LockList.Delete(DestinationIPIdx);
end;

You are calling Contexts.LockList() WAY too many times. The contexts list is protected by a critical section. The calls to LockList() and UnlockList() MUST be balanced or you will deadlock the server, preventing clients from connecting and disconnecting.
LockList() returns the actual list. So, you should lock it once, access its items as needed, and then unlock it once.
Try something more like this instead:
list := frmtcpserver.IdTCPServer1.Contexts.LockList;
try
for i := 0 to list.Count - 1 do
begin
ctx := TIdContext(list[i]);
if ctx.Binding.PeerIP = Destination_IP then
begin
// sending data here
try
ctx.Connection.IOHandler.Write(TempBuf, NoofBytesToSend, 0);
except
on E: EIdException do
begin
ctx.Connection.Disconnect;
end;
end;
break;
end;
end;
finally
frmtcpserver.IdTCPServer1.Contexts.UnlockList;
end;
That being said, if the server's OnExecute event is communicating back and forth with the client then it is generally not safe to directly send data to a client from outside of the client's OnExecute event, like you are doing. You risk corrupting the communications. It is safer to give each client context its own thread-safe queue of outgoing data, and then use the OnExecute event to send that data when it is safe to do so. For example:
type
TMyContext = class(TIdServerContext)
public
Queue: TThreadList;
...
constructor Create(AConnection: TIdTCPConnection; AYarn: TIdYarn; AList: TThreadList = nil); override;
destructor Destroy; override;
end;
PIdBytes := ^TIdBytes;
constructor TMyContext.Create(AConnection: TIdTCPConnection; AYarn: TIdYarn; AList: TThreadList = nil);
begin
inherited;
Queue := TThreadList.Create;
end;
destructor TMyContext.Destroy;
var
list: TList;
I: integer;
begin
list := Queue.LockList;
try
for i := 0 to list.Count-1 do
begin
PIdBytes(list[i])^ := nil;
Dispose(list[i]);
end;
finally
Queue.UnlockList;
end;
Queue.Free;
inherited;
end;
procedure TFrmTcpServer.FormCreate(Sender: TObject);
begin
IdTCPServer1.ContextClass := TMyContext;
end;
procedure TFrmTcpServer.IdTCPServer1Execute(AContext: TIdContext);
var
Queue: TList;
tmpList: TList;
i: integer;
begin
...
tmpList := nil;
try
Queue := TMyContext(AContext).Queue.LockList;
try
if Queue.Count > 0 then
begin
tmpList := TList.Create;
tmpList.Assign(Queue);
Queue.Clear;
end;
finally
TMyContext(AContext).Queue.UnlockList;
end;
if tmpList <> nil then
begin
for i := 0 to tmpList.Count-1 do
begin
AContext.Connection.IOHandler.Write(PIdBytes(tmpList[i])^);
end;
end;
finally
if tmpList <> nil then
begin
for i := 0 to tmpList.Count-1 do
begin
PIdBytes(tmpList[i])^ := nil;
Dispose(tmpList[i]);
end;
end;
tmpList.Free;
end;
...
end;
var
list: TList;
ctx: TIdContext;
I: integer;
data: PIdBytes;
begin
list := IdTCPServer1.Contexts.LockList;
try
for i := 0 to list.Count - 1 do
begin
ctx := TIdContext(list[i]);
if ctx.Binding.PeerIP = Destination_IP then
begin
New(data);
try
data^ := Copy(TempBuf, 0, NoofBytesToSend);
TMyContext(ctx).Queue.Add(data);
except
data^ := nil;
Dispose(data);
end;
break;
end;
end;
end;
finally
IdTCPServer1.Contexts.UnlockList;
end;

Related

Delphi service app crashes at a random time

I have a Delphi Service app. Indy TCP server and many clients (up to 50), ADO connection to Firebird and simply network exchange. App randomly crashes (may be workin 7 days, may be 1 hour) with next event (for example):
Имя сбойного приложения: rollcontrol.exe, версия: 1.1.20.2, метка времени: 0x60acd5f2
Имя сбойного модуля: ntdll.dll, версия: 6.3.9600.19678, метка времени: 0x5e82c0f7
Код исключения: 0xc0000005
Смещение ошибки: 0x00058def
Идентификатор сбойного процесса: 0x4178
or:
Имя сбойного приложения: rollcontrol.exe, версия: 1.1.1.9, метка времени: 0x607b239c
Имя сбойного модуля: msvcrt.dll, версия: 7.0.9600.16384, метка времени: 0x52158ff5
Код исключения: 0xc0000005
Смещение ошибки: 0x00009e80
All jobs in app makes in anonimius threads or in tcp/ip connections threads. All code in each thread executed in try except statments. There no memory leaks or growing threads count. The main code of service thread very simple:
procedure TRollControl_Svc.ServiceExecute(Sender: TService);
begin
while not Terminated do
try
ServiceThread.ProcessRequests(False);
ServiceThread.Sleep(100);
except
on e : exception do LogException('ServiceExecute', E);
end;
end;
How I can handled this exception and prevent app crash? How it possible to crash service thread with two simple lines of code?
Thanks
UPDATE: Example of connections to DB:
function TRollControl_Svc.GetNodeIdByIP(ip: string): integer;
Var
SQLConnection : TADOConnection;
SQLQuery : TADOQuery;
Thread : TThread;
fResult : integer;
begin
fResult := 0;
try
Thread := nil;
Thread := TThread.CreateAnonymousThread(
procedure
begin
try
SQLConnection := nil;
SQLQuery := nil;
CoInitialize(nil);
SQLConnection := TADOConnection.Create(nil);
SQLConnection.ConnectionString := 'Provider=MSDASQL.1;Password=' + Psw + ';Persist Security Info=True;User ID=' + Usr + ';Data Source=' + Srv ;
SQLConnection.LoginPrompt := false;
SQLQuery := TADOQuery.Create(nil);
SQLQuery.Connection := SQLConnection;
SQLQuery.LockType := ltReadOnly;
try SQLConnection.Open; except SQLConnection.Open; end;
SQLConnection.BeginTrans;
SQLQuery.Close;
SQLQuery.SQL.Text := 'select nodes.* from nodes where nodes.ip = :ip';
SQLQuery.Parameters.ParamByName('ip').Value := ip;
try SQLQuery.Open; except SQLQuery.Open; end;
if SQLQuery.IsEmpty then exit;
fResult := SQLQuery.FieldByName('ID').AsInteger;
if SQLConnection.InTransaction then
SQLConnection.CommitTrans;
finally
TryFree(SQLQuery);
TryFree(SQLConnection);
CoUninitialize;
end;
end
);
Thread.FreeOnTerminate := false;
Thread.Start;
Thread.WaitFor;
finally
TryFree(Thread);
end;
result := fResult;
end;
Error Handling
This isn't an answer as to what is causing your problem, but I thought it probably wouldn't be clear in a comment.
In languages that support structured exception handling the language gives the programmer an opportunity to fail gracefully when things don't work. That's not how you are using it. From your example anonymous thread you have:
try SQLConnection.Open; except SQLConnection.Open; end;
So you are told that the connection can't be made and instead of responding to that situation you go ahead and attempt to connect again. There are lots of reasons why a connection may not work, some of those are transient so the attempt may work a little later but if you simply try doing it again without any pause it seems reasonable to expect it to fail again.
It's obviously important to catch errors, but you have to have appropriate failure paths.
I have no way of knowing if this is related to what's actually going wrong.
I found the reason. The problem was in the ADO source codes (Data.Win.ADODB.pas):
procedure RefreshFromOleDB;
var
I: Integer;
ParamCount: ULONG_PTR;
ParamInfo: PDBParamInfoArray;
NamesBuffer: POleStr;
Name: WideString;
Parameter: _Parameter;
Direction: ParameterDirectionEnum;
OLEDBCommand: ICommand;
OLEDBParameters: ICommandWithParameters;
CommandPrepare: ICommandPrepare;
begin
OLEDBCommand := (Command.CommandObject as ADOCommandConstruction).OLEDBCommand as ICommand;
OLEDBCommand.QueryInterface(ICommandWithParameters, OLEDBParameters);
OLEDBParameters.SetParameterInfo(0, nil, nil); // ----- Error here
if Assigned(OLEDBParameters) then
begin
ParamInfo := nil;
NamesBuffer := nil;
try
OLEDBCommand.QueryInterface(ICommandPrepare, CommandPrepare);
if Assigned(CommandPrepare) then CommandPrepare.Prepare(0);
if OLEDBParameters.GetParameterInfo(ParamCount, PDBPARAMINFO(ParamInfo), #NamesBuffer) = S_OK then
for I := 0 to ParamCount - 1 do//
begin
{ When no default name, fabricate one like ADO does }
if ParamInfo[I].pwszName = nil then
Name := 'Param' + IntToStr(I+1) else { Do not localize }
Name := ParamInfo[I].pwszName;
{ ADO maps DBTYPE_BYTES to adVarBinary }
if ParamInfo[I].wType = DBTYPE_BYTES then ParamInfo[I].wType := adVarBinary;
{ ADO maps DBTYPE_STR to adVarChar }
if ParamInfo[I].wType = DBTYPE_STR then ParamInfo[I].wType := adVarChar;
{ ADO maps DBTYPE_WSTR to adVarWChar }
if ParamInfo[I].wType = DBTYPE_WSTR then ParamInfo[I].wType := adVarWChar;
Direction := ParamInfo[I].dwFlags and $F;
{ Verify that the Direction is initialized }
if Direction = adParamUnknown then Direction := adParamInput;
Parameter := Command.CommandObject.CreateParameter(Name, ParamInfo[I].wType, Direction, ParamInfo[I].ulParamSize, EmptyParam);
Parameter.Precision := ParamInfo[I].bPrecision;
Parameter.NumericScale := ParamInfo[I].bScale;
//if ParamInfo[I].dwFlags and $FFFFFFF0 <= adParamSigned + adParamNullable + adParamLong then
Parameter.Attributes := ParamInfo[I].dwFlags and $FFFFFFF0; { Mask out Input/Output flags }
AddParameter.FParameter := Parameter;
end;
finally
if Assigned(CommandPrepare) then CommandPrepare.Unprepare;
if (ParamInfo <> nil) then GlobalMalloc.Free(ParamInfo);
if (NamesBuffer <> nil) then GlobalMalloc.Free(NamesBuffer);
end;
end;
end;
Line
OLEDBParameters.SetParameterInfo(0, nil, nil)
executed before
if Assigned(OLEDBParameters)
I moved this line after checking on nil and all working fine
I managed to isolate the problem. Errors occur periodically when working with ADO. If I try to use TADOQuery objects again, the application more susceptible to crashes. What I've done:
System.NeverSleepOnMMThreadContention: = false;
Significantly reduces errors when working with ADO
All uses of TADOQuery are single use.
For example it was:
for ii := 0 to SettingsXML.Root.NamedItem['sql_clear_base'].NamedItem['XML'].Count - 1 do
begin
try
SQLQuery.Close;
SQLQuery.SQL.Text := SettingsXML.Root.NamedItem['sql_clear_base'].NamedItem['XML'][ii].AsString;
SQLQuery.ExecSQL;
except
on e : exception do LogException('ClearBase', '', E);
end;
end;
Became:
for ii := 0 to SettingsXML.Root.NamedItem['sql_clear_base'].NamedItem['XML'].Count - 1 do
begin
SQLQuery := nil;
try
SQLQuery := TADOQuery.Create(nil);
SQLQuery.Connection := SQLConnection;
try
SQLQuery.Close;
SQLQuery.SQL.Text := SettingsXML.Root.NamedItem['sql_clear_base'].NamedItem['XML'][ii].AsString;
SQLQuery.ExecSQL;
except
on e : exception do LogException('ClearBase', '', E);
end;
finally
TryFree(SQLQuery);
end;
end;
I make self-control:
main procces started as windows service (process A)
process A starts a copy of itself as B
one per minutes A check if B alive and restart if not
one per minutes B check if A alive and restart if not
check - simple TCP packet and answer
For example:
TThread.CreateAnonymousThread(
procedure
var tcpClient : TidTCPClient;
begin
tcpClient := nil;
LastKeepAlive := Date + Time;
while ServerMode do
begin
try
if not Assigned(tcpClient) then
begin
tcpClient := TIdTCPClient.Create(nil);
tcpClient.Host := '127.0.0.1';
tcpClient.Port := RollControl_Svc.TCPServer.Bindings[0].Port;
tcpClient.Connect;
tcpClient.IOHandler.ReadTimeout := 1000;
end;
tcpClient.IOHandler.Write(START_PACKET + #0 + END_PACKET);
tcpClient.IOHandler.ReadString(3);
LastKeepAlive := Date + Time;
except
TryFree(tcpClient);
end;
sleep(15 * 1000);
end;
end).Start;
TThread.CreateAnonymousThread(
procedure
Var Res: TRequestResult;
begin
while ServerMode do
begin
if Date + Time - LastKeepAlive > OneMinute then
begin
Res.Clear('', '');
Res.Nodes_ID := -1;
Res.Data_In := 'KeepAlive';
Res.Data_Out := 'Exception: ExitProcess(1)';
try
Log(Res, true);
finally
ExitProcess(1);
end;
end;
sleep(1000);
end;
end).Start;
P.S. Local tests never crashed applications. The program simply proceed a million requests (connect, request, disconnect), there are no memory leaks or failures. On several clients servers are crashed. In the future I want to port to Lazarus to use ODBC directly insteed ADO

delphi tcp server on multi port hanged on close

i used a multi port tcp server to receive some connections
like this
procedure TForm2.IdTCPServer1Execute(AContext: TIdContext);
var
aByte: Byte;
i,j , tmBodyFrameLength:integer;
myThread : tthread;
begin
if not Assigned( allOfflineStringList ) then
begin
allOfflineStringList := TStringlist.Create;
end;
allOfflineStringList.Clear;
case AContext.Binding.Port of
55000: begin {offline and image}
AContext.Connection.IOHandler.ReadBytes(data, 1099, False);
rowFrame :='';
for I := 0 to length(data)-1 do
begin
rowFrame := rowFrame + (data[i].ToHexString);
end;
newFrame := copy( rowFrame , 9 , maxInt );
allOfflineStringList.Append( newFrame );
TThread.Synchronize (TThread.CurrentThread,
procedure ()
begin
Label985.caption := 'Offline : ' + allOfflineStringList.Count.ToString ;
//Memo14.Lines.Add( datetimetostr(now) +':'+ newFrame );
form2.AbLED601.Tag := DateTimeToUnix(now);
form2.AbLED601.Checked := true;
end);
end;
55001: begin {tm online}
repeat
aByte := AContext.Connection.IOHandler.ReadByte;
if aByte=$C0 then
begin
SDRtmOnlineRowFrame2 := SDRtmOnlineRowFrame;
SDRtmOnlineRowFrame := '';
TThread.Synchronize (TThread.CurrentThread,
procedure ()
begin
form2.Memo14.Lines.Add('tm:'+ SDRtmOnlineRowFrame2 );
end);
end
else
begin
SDRtmOnlineRowFrame := SDRtmOnlineRowFrame + aByte.ToHexString;
end;
until true;
end;
55003: begin {beacon online}
repeat
aByte := AContext.Connection.IOHandler.ReadByte;
if aByte=$C0 then
begin
SDRtmOnlineBeaconRowFrame2 := SDRtmOnlineBeaconRowFrame;
SDRtmOnlineBeaconRowFrame := '';
TThread.Synchronize (TThread.CurrentThread,
procedure ()
begin
form2.Memo14.Lines.Add('beacon:'+ SDRtmOnlineBeaconRowFrame2 );
end);
end
else
begin
SDRtmOnlineBeaconRowFrame := SDRtmOnlineBeaconRowFrame + aByte.ToHexString;
end;
until true;
end;
end;
end;
every thing working good
but when data is receiving if i close the connection
app will hange and dont responding any more!
enable and disable is like this:
procedure TForm2.CheckBox6Click(Sender: TObject);
var
ic:integer;
allIpList : TStringList;
begin
AbLED412.Checked := CheckBox6.Checked;
if CheckBox6.Checked=true then
begin
IdTCPServer1.Active := False;
IdTCPServer1.Bindings.Clear;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55000;
end;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55001;
end;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55003;
end;
IdTCPServer1.Active := True;
IdTCPServer1.StartListening;
TIdStack.IncUsage;
try
allIpList := TStringList.Create;
GStack.AddLocalAddressesToList( allIpList );
memo14.lines.clear;
for ic := 0 to allIpList.Count-1 do
begin
memo14.lines.Add('Create tcp connection on ip : '+allIpList[ic]+' and port : 55000');
memo14.lines.Add('Create tcp connection on ip : '+allIpList[ic]+' and port : 55001');
memo14.lines.Add('Create tcp connection on ip : '+allIpList[ic]+' and port : 55003');
end;
finally
TIdStack.DecUsage;
end;
end
else
begin
IdTCPServer1.StopListening;
IdTCPServer1.Active := False;
IdTCPServer1.Bindings.Clear;
memo14.lines.clear;
end;
end;
also when data is receiving if i close the app it hanged again
but when sender disconnected closing the app dont make any problem
how can i fix this?
Your TIdTCPServer.OnExecute handler is using multiple variables in a thread-unsafe manner. You are not protecting them from multiple threads accessing them at the same time, thus causing race conditions on their data.
But, more importantly, your use of TThread.Synchronize() is a common cause of deadlock for TIdTCPServer because it is a multi-threaded component. Its OnConnect, OnDisconnect, OnExecute, and OnError events are called in the context of client worker threads, not in the main UI thread. TThread.Synchronize() blocks the calling thread until the main UI thread processes the request. Deactivating TIdTCPServer terminates all running client threads and waits for them to fully terminate. So, if you call TThread.Synchronize() in a client thread while the main UI thread is blocked deactivating the server, then the client thread is waiting on the main UI thread while the main UI thread is waiting on the client thread - deadlock!
You have a few options to solve this:
avoid calling TThread.Synchronize() while deactivating the server. Easier said than done though, as you might already be in a pending TThread.Synchronize() by the time you decide to deactivate TIdTCPServer. And it is a race condition when making the decision whether to call TThread.Synchronize() or not.
deactivate TIdTCPServer in a separate worker thread, leave the main UI thread free to process TThread.Synchronize() and TThread.Queue() requests. If you use a TThread for the deactivation, calling the TThread.WaitFor() method in the main UI thread will process Synchronize()/Queue() requests while it is waiting for the thread to terminate.
Use TThread.Queue() instead of TThread.Synchronize(), especially when performing actions that your client threads don't actually need to wait on, such as UI updates.
On a side note, in your CheckBox6Click():
you should not be calling TIdTCPServer.StartListening() or TIdTCPServer.StopListening() at all. The TIdTCPServer.Active property setter calls them internally for you.
you don't need to call TIdStack.IncUsage() or TIdStack.DecUsage() either, as TIdTCPServer's constructor and destructor call them for you.
you are leaking allIpList as you don't Free() it. And TIdStack.AddLocalAddressesToList() is deprecated anyway, you should be using TIdStack.GetLocalAddressList() instead.
Try this:
procedure TForm2.CheckBox6Click(Sender: TObject);
var
ic: integer;
allIpList : TIdStackLocalAddressList;
begin
AbLED412.Checked := CheckBox6.Checked;
if CheckBox6.Checked then
begin
IdTCPServer1.Active := False;
IdTCPServer1.Bindings.Clear;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55000;
end;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55001;
end;
with IdTCPServer1.Bindings.Add do
begin
//IP := '192.168.1.5';
Port := 55003;
end;
IdTCPServer1.Active := True;
allIpList := TIdStackLocalAddressList.Create;
try
GStack.GetLocalAddressesList( allIpList );
Memo14.Lines.Clear;
{
for ic := 0 to IdTCPServer1.Bindings.Count-1 do
begin
Memo14.Lines.Add('Create tcp connection on ip : ' + IdTCPServer1.Bindings[ic].IP + ' and port : ' + IntToStr(IdTCPServer1.Bindings[ic].Port));
end;
}
for ic := 0 to allIpList.Count-1 do
begin
if allIpList[ic].IPVersion = ID_DEFAULT_IP_VERSION then
begin
Memo14.Lines.Add('Create tcp connection on ip : ' + allIpList[ic].IPAddress + ' and port : 55000');
Memo14.Lines.Add('Create tcp connection on ip : ' + allIpList[ic].IPAddress + ' and port : 55001');
Memo14.Lines.Add('Create tcp connection on ip : ' + allIpList[ic].IPAddress + ' and port : 55003');
end;
end;
finally
allIpList.Free;
end;
end
else
begin
IdTCPServer1.Active := False;
IdTCPServer1.Bindings.Clear;
Memo14.Lines.Clear;
end;
end;

Delphi Indy client sends 64 KB package and the Server receives 2 packages totaling 64 KB

With the TIdTCPServer component of Indy, a package is received in two fractions but the client sent one with 64 KB.
How do I receive the complete package in the Server OnExecute event?
Now I put a prototype (Server and Client) code to recreate the situation.
Server Code
procedure TFrmServer.IdTCPServer1Execute(AContext: TIdContext);
Var
ReceivedBytesTCP : Integer;
IBuf : TIdBytes;
begin
if Not AContext.Connection.IOHandler.InputBufferIsEmpty then Begin
Try
ReceivedBytesTCP := AContext.Connection.IOHandler.InputBuffer.Size;
SetLength(IBuf,ReceivedBytesTCP);
AContext.Connection.IOHandler.ReadBytes(IBuf,ReceivedBytesTCP,False);
AContext.Connection.IOHandler.Write(IBuf,Length(IBuf),0);
Except
On E : Exception Do Begin
Memo1.Lines.Add('Except Server TCP: ' + E.Message);
End;
End;
End Else Begin
Sleep(1);
End;
end;
Client Code
procedure TFrm_TCP_Client.BtnSendClick(Sender: TObject);
Var
IBuf,RBuf : TIdBytes;
I : Integer;
LenPacket : Integer;
begin
LenPacket := StrToInt(EdtLength.Text);
if IdTCPClient1.Connected then Begin
SetLength(IBuf,LenPacket);
for I := 1 to LenPacket do
IBuf[I] := 1;
IdTCPClient1.IOHandler.Write(IBuf,Length(IBuf),0);
I := 0;
Repeat
IdTCPClient1.IOHandler.CheckForDataOnSource(50);
Inc(I);
Until Not IdTCPClient1.IOHandler.InputBufferIsEmpty or (I >= 10);
If Not IdTCPClient1.IOHandler.InputBufferIsEmpty Then Begin
SetLength(RBuf,IdTCPClient1.IOHandler.InputBuffer.Size);
IdTCPClient1.IOHandler.ReadBytes(RBuf,IdTCPClient1.IOHandler.InputBuffer.Size,False);
if Length(RBuf) = Length(IBuf) then
Memo1.Lines.Add('Response Received OK: '+IntToStr(Length(RBuf)))
Else
Memo1.Lines.Add('Response Received With Different Length: '+IntToStr(Length(RBuf)));
if Not IdTCPClient1.IOHandler.InputBufferIsEmpty then
Memo1.Lines.Add('Llego otro Mensaje');
End Else Begin
Memo1.Lines.Add('NO Response Received');
End;
End;
end;
How to know that a message is the first or the second fragment?
How to force the receive of second fragment?
There is no 1-to-1 relationship between sends and reads in TCP. It is free to fragment data however it wants to optimize network transmissions. TCP guarantees only that data is delivered, and in the same order it was sent, but nothing about HOW data is fragmented during transmission. TCP will reconstruct the fragments on the receiving end. This is simply how TCP works, it is not unique to Indy. Every TCP app has to deal with this issue regardless of which TCP framework is used.
If you are expecting 64KB of data, then simply read 64KB of data, and let the OS and Indy handle the fragments internally for you. This fragmentation of TCP is exactly why Indy's IOHandler uses an InputBuffer to collect the fragments when piecing data back together.
Update: stop focusing on fragments. That is an implementation detail at the TCP layer, which you are not operating at. You don't need to deal with fragments in your code. Let Indy handle it for you. Just focus on your application level protocol instead.
And FYI, you have essentially implemented an ECHO client/server solution. Indy has actual ECHO client/server components, TIdECHO and TIdECHOServer, you should have a look at them.
In any case, your server-side exception handling is very problematic. It is not syncing with the main UI thread (OnExecute is called in a worker thread). But, more importantly, it as preventing TIdTCPServer from processing any notifications issued by Indy itself when the client connection is lost/disconnected, so the client thread will keep running and not stop until you deactivate the server. DO NOT swallow Indy's own exceptions (which are derived from EIdException). If you need to catch them in your code, you should re-raise them when done, let TIdTCPServer process them. But, in your example, it would be easier to remove the try..except altogether and use the server's OnException event instead.
Also, your client-side reading loop is wrong for what you are attempting to do with it. You are not initializing IBuf correctly. But, more importantly, you are using a very short timeout (TCP connections may have latency), and you are breaking your reading loop as soon as any data arrives or 500ms max have elapsed, even if there is more data still coming. You should be reading until there is nothing left to read.
Try something more like this instead:
Server:
procedure TFrmServer.IdTCPServer1Execute(AContext: TIdContext);
var
IBuf : TIdBytes;
begin
AContext.Connection.IOHandler.ReadBytes(IBuf, -1);
AContext.Connection.IOHandler.Write(IBuf);
end;
procedure TFrmServer.IdTCPServer1Exception(AContext: TIdContext, AException: Exception);
var
Msg: string;
begin
if AException <> nil then
Msg := AException.Message
else
Msg := 'Unknown';
TThread.Queue(nil,
procedure
begin
Memo1.Lines.Add('Except Server TCP: ' + Msg);
end
);
end;
Client:
procedure TFrm_TCP_Client.BtnSendClick(Sender: TObject);
Var
IBuf,RBuf : TIdBytes;
LenPacket : Integer;
begin
if not IdTCPClient1.Connected then Exit;
LenPacket := StrToInt(EdtLength.Text);
if LenPacket < 1 then Exit;
SetLength(IBuf, LenPacket);
FillBytes(IBuf, LenPacket, $1);
try
IdTCPClient1.IOHandler.InputBuffer.Clear;
IdTCPClient1.IOHandler.Write(IBuf);
except
Memo1.Lines.Add('Request Send Error');
Exit;
end;
try
while IdTCPClient1.IOHandler.CheckForDataOnSource(500) do;
if not IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
IdTCPClient1.IOHandler.ReadBytes(RBuf, IdTCPClient1.IOHandler.InputBuffer.Size, True);
if Length(RBuf) = Length(IBuf) then
Memo1.Lines.Add('Response Received OK: ' + IntToStr(Length(RBuf)))
else
Memo1.Lines.Add('Response Received With Different Length. Expected: ' + IntToStr(Length(IBuf)) + ', Got: ' + IntToStr(Length(RBuf)));
end
else
Memo1.Lines.Add('NO Response Received');
except
Memo1.Lines.Add('Response Receive Error');
end;
end;
A better solution would be to not rely on such logic at all, be more explicit about the structure of your data protocol, for instance <length><data>, eg:
Server:
procedure TFrmServer.IdTCPServer1Execute(AContext: TIdContext);
var
IBuf : TIdBytes;
LenPacket : Int32;
begin
LenPacket := AContext.Connection.IOHandler.ReadInt32;
AContext.Connection.IOHandler.ReadBytes(IBuf, LenPacket, True);
AContext.Connection.IOHandler.Write(LenPacket);
AContext.Connection.IOHandler.Write(IBuf);
end;
procedure TFrmServer.IdTCPServer1Exception(AContext: TIdContext, AException: Exception);
var
Msg: string;
begin
if AException <> nil then
Msg := AException.Message
else
Msg := 'Unknown';
TThread.Queue(nil,
procedure
begin
Memo1.Lines.Add('Except Server TCP: ' + Msg);
end
);
end;
Client:
procedure TFrm_TCP_Client.BtnSendClick(Sender: TObject);
Var
IBuf,RBuf : TIdBytes;
LenPacket : Int32;
begin
if not IdTCPClient1.Connected then Exit;
LenPacket := StrToInt(EdtLength.Text);
if LenPacket < 1 then Exit;
SetLength(IBuf, LenPacket);
FillBytes(IBuf, LenPacket, $1);
try
IdTCPClient1.IOHandler.InputBuffer.Clear;
IdTCPClient1.IOHandler.Write(LenPacket);
IdTCPClient1.IOHandler.Write(IBuf);
except
Memo1.Lines.Add('Request Send Error');
Exit;
end;
try
IdTCPClient1.IOHandler.ReadTimeout := 5000;
LenPacket := IdTCPClient1.IOHandler.ReadInt32;
IdTCPClient1.IOHandler.ReadBytes(RBuf, LenPacket, True);
except
Memo1.Lines.Add('Response Receive Error');
Exit;
end;
Memo1.Lines.Add('Response Received OK');
end;

TIdHttp freezes when the internet gets slower

How to avoid freezing the idHTTP when the internet become slower or no connectivity. My application get freeze and I could not even close the form.
This is how I setup my code
procedure TDownloader.IdHTTPWork(ASender: TObject; AWorkMode: TWorkMode;
AWorkCount: Int64);
var
lwElapsedMS: LongWord;
iBytesTransferred: Int64;
iBytesPerSec: Int64;
iRemaining: Integer;
begin
if AWorkMode <> wmRead then Exit;
lwElapsedMS := GetTickDiff(FLastTicks, Ticks);
if lwElapsedMS = 0 then lwElapsedMS := 1; // avoid EDivByZero error
if FTotalBytes > 0 then
FPercentDone := Round(AWorkCount / FTotalBytes * 100.0)
else
FPercentDone := 0;
iBytesTransferred := AWorkCount - FLastWorkCount;
iBytesPerSec := Round(iBytesTransferred * 1000 / lwElapsedMS);
if Assigned(OnDownloadProgress) then
begin
if FContinueDownload <> 0 then //previous file downloaded
begin
iRemaining := 100 - FContinueDownload;
iRemaining := Round(FPercentDone * iRemaining / 100);
OnDownloadProgress(Self, FContinueDownload + iRemaining, AWorkCount, FTotalBytes, iBytesPerSec);
end else
OnDownloadProgress(Self, FPercentDone, AWorkCount, FTotalBytes, iBytesPerSec);
end;
FLastWorkCount := AWorkCount;
FLastTicks := Ticks;
if FCancel then
begin
Abort;
TidHttp(ASender).Disconnect;
end;
end;
procedure TDownloader.IdHTTPWorkBegin(ASender: TObject; AWorkMode: TWorkMode;
AWorkCountMax: Int64);
begin
if AWorkMode <> wmRead then Exit;
FPercentDone := 0;
FTotalBytes := AWorkCountMax;
FLastWorkCount := 0;
FLastTicks := Ticks;
end;
procedure TDownloader.IdHTTPWorkEnd(ASender: TObject; AWorkMode: TWorkMode);
begin
if AWorkMode <> wmRead then Exit;
if Assigned(OnDownloadComplete) and (FPercentDone >= 100) then
OnDownloadComplete(Self)
else if Assigned(OnDownloadCancel) then
OnDownloadCancel(Self);
end;
function TDownloader.EXDownload(AURL, ADestFile: String;
AAutoDisconnect: Boolean): Boolean;
var
fsBuffer: TFileStream;
idHttp: TIdHttp;
begin
if FileExists(ADestFile) then
fsBuffer := TFileStream.Create(ADestFile, fmOpenReadWrite)
else
fsBuffer := TFileStream.Create(ADestFile, fmCreate);
fsBuffer.Seek(0, soFromEnd);
try
idHttp := TIdHttp.Create(nil);
idHttp.OnWorkBegin := idHttpWorkBegin;
idHttp.OnWork := idHttpWork;
idHttp.OnWorkEnd := idHttpWorkEnd;
idHttp.Request.CacheControl := 'no-store';
try
...
idHttp.Get(AURL, fsBuffer);
...
finally
idHttp.Free;
end;
finally
fsBuffer.Free;
end;
end;
......
procedure TDownloader.Execute;
begin
Inherited;
while not Terminated do
begin
if FUrl <> '' then
begin
EXDownload(FUrl, FFilename, True);
end;
end;
end;
...
on the main form progress
procedure TfrmDownloadList.DownloadProgress(Sender: TObject; aPercent:Integer;
aProgress, aProgressMax, aBytesPerSec: Int64);
var
yts: PYoutubeSearchInfo;
begin
if Assigned(FCurrentDownload) then
begin
yts := vstList.GetNodeData(FCurrentDownload);
yts.Tag := aPercent;
ProgressBar.Position := aPercent;
vstList.InvalidateNode(FCurrentDownload);
StatusBar.Panels.Items[1].Text := 'Download: ' + FormatByteSize(aProgress) + '/' +
FormatByteSize(aProgressMax);
StatusBar.Panels.Items[2].Text := 'Speed: ' + FormatByteSize(aBytesPerSec) + 'ps';
Application.ProcessMessages;
end;
end;
I don't have problem when the internet is good only when it drops due to poor signal.
this is my app lookslike
If we assume that TDownloader.OnDownloadProgress is assigned to the TfrmDownloadList.DownloadProgress method, then your problem is that you are calling VCL code (your update of the progress bar) from a secondary thread (ie. not from the Main thread). This is not supported.
You'll need to wrap the call with a Synchronize statement from within your thread. Synchronize calls a parameterless method on the main thread. So you need to store the variables that are needed and then call Synchronize on a method in your TDownloader class that then calls on to TfrmDownloadList.DownloadProgress
You cannot call TfrmDownloadList.DownloadProgress directly or indirectly from within code that runs on another thread than the main thread, as it updates VCL objects, and the VCL is not thread-safe.
The same goes for your DownloadComplete event, if it updates any VCL objects...
How about you using TIdAntiFreeze ?
TIdAntiFreeze implements a GUI-integration class that ensures
processor time is allocated for the Application main thread.
Indy works on the blocking sockets model. Calls made to methods in the
Indy components do not return until they are complete. If calls are
made in the main thread, this will cause the Application User
Interface to "freeze" during Indy calls. TIdAntiFreeze counteracts
this effect.
TIdAntiFreeze allows Indy to process Application messages so that
Windows messages continue to be executed while Indy blocking socket
calls are in effect.
Only one TIdAntiFreeze can be active in an application.

How to track number of clients with Indy TIdTCPServer

I want to know the number of current client connections to an Indy 9 TIdTCPServer (on Delphi 2007)
I can't seem to find a property that gives this.
I've tried incrementing/decrementing a counter on the server OnConnect/OnDisconnect events, but the number never seems to decrement when a client disconnects.
Any suggestions?
The currently active clients are stored in the server's Threads property, which is a TThreadList. Simply lock the list, read its Count property, and then unlock the list:
procedure TForm1.Button1Click(Sender: TObject);
var
NumClients: Integer;
begin
with IdTCPServer1.Threads.LockList do try
NumClients := Count;
finally
IdTCPServer1.Threads.UnlockList;
end;
ShowMessage('There are currently ' + IntToStr(NumClients) + ' client(s) connected');
end;
In Indy 10, the Threads property was replaced with the Contexts property:
procedure TForm1.Button1Click(Sender: TObject);
var
NumClients: Integer;
begin
with IdTCPServer1.Contexts.LockList do try
NumClients := Count;
finally
IdTCPServer1.Contexts.UnlockList;
end;
ShowMessage('There are currently ' + IntToStr(NumClients) + ' client(s) connected');
end;
Not sure why using OnConnect and OnDisconnect wouldn't work for you, but what we have done is to create a descendant of TIdCustomTCPServer; to override its DoConnect and DoDisconnect methods and create and use our own descendant of TIdServerContext (a thread descendant that will "serve" a connection).
You make the TIdCustomTCPServer aware of your own TIdServerContext class by:
(Edit Added conditional defines to show how to make it work for Indy9)
type
// Conditional defines so that we can use the same ancestors as in Indy10 and we
// can use the same method signatures for DoConnect and DoDisconnect regardless
// of the Indy version. Add other conditional defines as needed.
// Note: for INDY9 to be defined, you need to include the appropriate includes
// from Indy, or define it in your own include file.
{$IFDEF INDY9}
TIdContext = TIdPeerThread;
TIdServerContext = TIdContext;
TIdCustomTCPServer = TIdTCPServer;
{$ENDIF}
TOurContext = class(TIdServerContext)
private
FConnectionId: cardinal;
public
property ConnectionId: cardinal ...;
end;
...
constructor TOurServer.Create(aOwner: TComponent);
begin
inherited Create(aOwner);
...
{$IFDEF INDY10_UP}
ContextClass := TOurContext;
{$ELSE}
ThreadClass := TOurContext;
{$ENDIF}
...
end;
In the DoConnect override of our TIdCustomTCPServer descendant we set the ConnectionID of our context class to a unique value:
procedure TOurServer.DoConnect(AContext: TIdContext);
var
OurContext: TOurContextabsolute AContext;
begin
Assert(AContext is TOurContext);
HandleGetNewConnectionID(OurContext, OurContext.FConnectionID);
inherited DoConnect(AContext);
...
end;
Our DoDisconnect override clears the ConnectionID:
procedure TOurServer.DoDisconnect(AContext: TIdContext);
var
OurContext: TOurContextabsolute AContext;
begin
Assert(AContext is TOurContext);
OurContext.FConnectionID := 0;
...
inherited DoDisconnect(AContext);
end;
Now it is possible to get a count of the current connections at any time:
function TOurServer.GetConnectionCount: Integer;
var
i: Integer;
CurrentContext: TOurContext;
ContextsList: TList;
begin
MyLock.BeginRead;
try
Result := 0;
if not Assigned(Contexts) then
Exit;
ContextsList := Contexts.LockList;
try
for i := 0 to ContextsList.Count - 1 do
begin
CurrentContext := ContextsList[i] as TOurContext;
if CurrentContext.ConnectionID > 0 then
Inc(Result);
end;
finally
Contexts.UnLockList;
end;
finally
MyLock.EndRead;
end;
end;
How about incrementing / decrementing a counter from OnExecute (or DoExecute if you override that)? That can't go wrong!
If you use InterlockedIncrement and InterlockedDecrement you don't even need a critical section to protect the counter.
This should work on Indy 9, but it is pretty outdated nowadays, and maybe something is broken in your version, try to update to the latest Indy 9 available.
I made a simple test using Indy 10, which works very well with a simple interlocked Increment/Decrement in the OnConnect/OnDisconnect event handlers. This is my code:
//closes and opens the server, which listens at port 1025, default values for all properties
procedure TForm2.Button1Click(Sender: TObject);
begin
IdTCPServer1.Active := not IdTCPServer1.Active;
UpdateUI;
end;
procedure TForm2.FormCreate(Sender: TObject);
begin
UpdateUI;
end;
//Just increment the count and update the UI
procedure TForm2.IdTCPServer1Connect(AContext: TIdContext);
begin
InterlockedIncrement(FClientCount);
TThread.Synchronize(nil, UpdateUI);
end;
//Just decrement the count and update the UI
procedure TForm2.IdTCPServer1Disconnect(AContext: TIdContext);
begin
InterlockedDecrement(FClientCount);
TThread.Synchronize(nil, UpdateUI);
end;
//Simple 'X' reply to any character, A is the "command" to exit
procedure TForm2.IdTCPServer1Execute(AContext: TIdContext);
begin
AContext.Connection.IOHandler.Writeln('Write anything, but A to exit');
while AContext.Connection.IOHandler.ReadByte <> 65 do
AContext.Connection.IOHandler.Write('X');
AContext.Connection.IOHandler.Writeln('');
AContext.Connection.IOHandler.Writeln('Good Bye');
AContext.Connection.Disconnect;
end;
//Label update with server status and count of connected clients
procedure TForm2.UpdateUI;
begin
Label1.Caption := Format('Server is %s, %d clients connected', [
IfThen(IdTCPServer1.Active, 'Open', 'Closed'), FClientCount]);
end;
then, opening a couple of clients with telnet:
then, closing one client
That's it.
INDY 10 is available for Delphi 2007, my main advise is to upgrade anyway.

Resources