I have an application that uses TIdTCPServer. I send null terminated messages to the server (Delphi XE2, Indy package which ships with it) sucessfully as null terminated strings.
The OnExecute procedure is as follows:
procedure TSimpleSslServerForm.TCPServerExecute(AContext: TIdContext);
var
RxBufStr: string;
begin
with AContext.Connection.IOHandler do
begin
if not InputBufferIsEmpty then
begin
RxBufStr := InputBufferAsString;
Display(RXBufStr);
lbl_EventsReceived.Caption := IntToStr(StrToInt(lbl_EventsReceived.Caption) + 1);
end;
end;
end;
My issue is that if i send alot of messages, if i do not put a 200 ms delay between messages then i loose data.
Without added delay:
I either loose data completely (13 messages received from 107 sent) and/ot the data is incorrect:
<38>Jul 10 09:37:39 cilad71 QJRN: ope=JOB WAS CHANGED date=10/07/15 time=07:59:26 sys=CILAD71 user=GCOX job=QZSOSIGN jobn= ipadr=192.168.5.121 pgm=QZSOSIGN pgmlib=QSYS date=07/10/15 time=07:59:26 user=GCOX action=PROFILE CHANGED jobname=QZSOSIGN jobnumber=189191 jobusername=QUSER jobd=QZBSJOBD ipaddress=192.168.5.121%
07/10/15 time=08:01:25 user=GCOX action=PROFILE CHANGED jobname=QZSOSIGN jobnumber=189191 jobusername=QUSER jobd=QZBSJOBD ipaddress=192.168.5.121%
<38>Jul 10 09:37:39 cilad71 QJRN: ope=JOB WAS CHANGED date=10/07/15 time=08:01:35 sys=CILAD71 user=GCOX job=QPADEV000D jobn= ipadr=192.168.5.121 pgm=QWTPIIPP pgmlib=QSYS date=07/10/15 time=08:01:35 user=GCOX action=START jobname=QPADEV000D jobnumber=189401 jobusername=GCOX jobd=QDFTJOBD ipaddress=192.168.5.121%
The event in bold is missing data.
I see the same behaviour with the TIdUDPServer component also. The only difference i see is that for UDP the required delay to receive all the data correctly is 100 ms, whereas for the TCP server a delay of less than 200 ms between messages always results in data loss.
For UDP when sending to a Windows Event service so the event appears in the corresponding Windows journal no delay is required and I see all the events correctly.
Thanks,
Geoff Cox
The TIdTCPServer code you have shown is NOT reading null-terminated messages from the socket. It is simply reading whatever raw data happens to be present on the socket at that particular moment. There might be no messages at that moment, or there might be a full message, or there might be partial pieces of multiple messages. This is just how TCP works.
If your messages are truly null-terminated, you need to read them as such, by waiting for the null terminator to arrive and then process whatever precedes it. For example, you can use the TIdIOHandler.ReadLn() or TIdIOHandler.WaitFor() method for that purpose:
procedure TSimpleSslServerForm.TCPServerExecute(AContext: TIdContext);
var
RxBufStr: string;
begin
RxBufStr := AContext.Connection.IOHandler.ReadLn(#0);
...
end;
procedure TSimpleSslServerForm.TCPServerExecute(AContext: TIdContext);
var
RxBufStr: string;
begin
RxBufStr := AContext.Connection.IOHandler.WaitFor(#0);
...
end;
Related
I have an app that receives a TCP socket connection from another app, processes the content that is streamed across the socket, builds a request based on the received data, sends the request using TIdHTTP to an external server, processes the response, and sends an ack back to the tcp client.
procedure TMyClass.TCPExecute(AContext: TIdContext);
var
s: AnsiString;
begin
s := TransferTCPBytesToString(AContext.Connection.IOHandler);
// test string for terminating characters
if Pos(#28#13, s) > 0 then
begin
//********PROBLEM: THIS WILL BE CALLED MULTIPLE TIMES
//********WITH THE SAME COMPLETE DATA STREAM CONTENT!
BuildAndSendExternalRequest(s);
AContext.Connection.IOHandler.InputBuffer.Clear;
// send the response back to tcp client
WriteTCPResponse(AContext.Connection.IOHandler);
end;
end;
function TMyClass.TransferTCPBytesToString(AnIOHandler: TIdIOHandler): AnsiString;
begin
if AnIOHandler.InputBufferIsEmpty then
Exit;
// read message from tcp client connection
AnIOHandler.ReadBytes(FTCPBytes, AnIOHandler.InputBuffer.Size, True);
SetLength(Result, Length(FTCPBytes));
// copy message bytes to string
Move(FTCPBytes[0], Result[1], Length(FTCPBytes));
end;
procedure TMyClass.WriteTCPResponse(AnIOHandler: TIdIOHandler);
var
bytes: TBytes;
begin
// FTCPBytes contains data from tcp client
// bytes will contain processed response from external server
BuildACK(FTCPBytes,bytes);
// FTCPBytes now contains ACK data
AnIOHandler.WriteDirect(FTCPBytes, Length(FTCPBytes));
end;
The request/response cycle completes successfully; the response sent back to the tcp client requestor is correct and the process executes and completes as expected.
However, because the OnExecute event fires multiple times with the same data, two requests are received, built, sent, acknowledged etc.
In looking at the logs on the tcp client app, the logs only show one request being sent. Not sure what is going on here. But I need to make sure that the request/response cycle is 1-1, not 1-many!
Thanks.
the OnExecute event fires multiple times ...
It is supposed to, as it is a looped event. The loop runs for as long as the connection is alive. The code inside your OnExecute event handler is responsible for deciding what happens on each loop iteration.
The ideal situation is where the event handler receives 1 message from the client, sends 1 response back to the client, and then exits, so that the next loop iteration is ready to handle the next request/response pair.
Which, based on your description, you are trying to do, but the code shown is not handling that very well. Most notably, the code is not accounting for the byte-stream nature of TCP. It does not differentiate one incoming message from another. It just dumps the entire IOHandler.InputBuffer content as-is into an AnsiString (why an AnsiString?), and if that AnsiString happens to contain the delimiter #28#13 then you are processing the entire AnsiString and then throwing it away, losing anything received after the delimiter (ie, part of the next message). And if the AnsiString doesn't contain the delimiter, then you are still throwing away the AnsiString, thus losing all data received so far for the current message before the delimiter arrives.
If all of your messages are delimited by #28#13, then I suggest you use the IOHandler.ReadLn() or IOHandler.WaitFor() method to read each message properly.
Also, you are re-using the same FTCPBytes member for both input and output, which in of itself is wrong, but you are also not providing any concurrent access protection for it if you have multiple client connections present.
With that said, try something more like this instead:
procedure TMyClass.TCPConnect(AContext: TIdContext);
begin
AContext.Connection.IOHandler.DefStringEncoding := IndyTextEncoding_8Bit;
end;
procedure TMyClass.TCPExecute(AContext: TIdContext);
var
s: String;
response: TIdBytes:
begin
s := TransferTCPBytesToString(AContext.Connection.IOHandler);
BuildAndSendExternalRequest(s, response);
// send the response back to tcp client
WriteTCPResponse(AContext.Connection.IOHandler, response);
end;
function TMyClass.TransferTCPBytesToString(AnIOHandler: TIdIOHandler): String;
begin
// read message from tcp client connection
Result := AnIOHandler.ReadLn(#28#13);
or:
Result := AnIOHandler.WaitFor(#28#13);
end;
procedure TMyClass.BuildAndSendExternalRequest(const Msg: String; var Response: TIdBytes);
begin
// send Msg to external server as needed...
// store response bytes in Response...
end;
procedure TMyClass.WriteTCPResponse(AnIOHandler: TIdIOHandler; const Response: TIdBytes);
var
bytes: TIdBytes;
begin
// Response contains processed response from external server
BuildACK(bytes, Response);
// bytes now contains ACK data
AnIOHandler.Write(bytes);
end;
... with the same data
That is not possible given the code you have shown. The call to IOHandler.ReadBytes() in TransferTCPBytesToString() is reading and discarding all bytes from the IOHandler.InputBuffer, so there is no way the same bytes can still be in that buffer when TransferTCPBytesToString() is called again at a later time. So, the only way you could be seeing the same data over and over is if the client is sending the same data over and over to begin with.
(Using Delphi 2009) I'm trying to write a simple network connection monitor using Indy TidIcmpClient. The idea is to ping an address, then inside an attached TidIcmpClient.OnReply handler test how much data was returned. If the reply contains > 0 bytes then I know the connection succeeded but if either the TidIcmpClient timed out or the reply contains 0 bytes I will assume the link is down
I'm having difficulty understanding the logic of TidIcmpClient as there is no 'OnTimeout' event.
Two sub questions...
Does TidIcmpClient.OnReply get called anyway, either (a) when data is received OR (b) when the timeout is reached, whichever comes first?
How can I distinguish a zero byte reply because of a timeout from a real reply within the timeout period that happens to contain zero bytes (or can't this happen)?
In other words is this sort of code OK or do I need to do something else to tell if it timed out or not
procedure TForm1.IdIcmpClient1Reply(ASender: TComponent; const AReplyStatus: TReplyStatus);
begin
if IdIcmpClient1.ReplyStatus.BytesReceived = 0 then
//we must have timed out, link is down
else
//got some data, connection is up
end;
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
end;
When Ping() exits, the ReplyStatus property contains the same information that is passed to the AReplyStatus parameter of the OnReply event (you are ignoring that parameter). Ping() simply calls the OnReply handler right before exiting, passing it the ReplyStatus property, so you don't actually need to use the OnReply event in your example. All that is doing is breaking up your code unnecessarily.
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
// process IdIcmpClient1.ReplyStatus here as needed...
end;
That being said, you are not processing the ReplyStatus data correctly. The BytesReceived field can be greater than 0 even if the ping fails. As its name implies, it simply reports how many bytes were actually received for the ICMP response. ICMP defines many different kinds of responses. The ReplyStatusType field will be set to the type of response actually received. There are 20 values defined:
type
TReplyStatusTypes = (rsEcho,
rsError, rsTimeOut, rsErrorUnreachable,
rsErrorTTLExceeded,rsErrorPacketTooBig,
rsErrorParameter,
rsErrorDatagramConversion,
rsErrorSecurityFailure,
rsSourceQuench,
rsRedirect,
rsTimeStamp,
rsInfoRequest,
rsAddressMaskRequest,
rsTraceRoute,
rsMobileHostReg,
rsMobileHostRedir,
rsIPv6WhereAreYou,
rsIPv6IAmHere,
rsSKIP);
If the ping is successful, the ReplyStatusType will be rsEcho, and the ReplyData field will contain the (optional) data that was passed to the ABuffer parameter of Ping(). You might also want to pay attention to the FromIpAddress and ToIpAddress fields as well, to make sure the response is actually coming from the expected target machine.
If a timeout occurs, the ReplyStatusType will be rsTimeOut instead.
Try this:
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
if IdIcmpClient1.ReplyStatus.ReplyStatusType = rsEcho then
begin
// got some data, connection is up
end
else if IdIcmpClient1.ReplyStatus.ReplyStatusType = rsTimeout then
begin
// have a timeout, link is down
end
else
begin
// some other response, do something else...
end;
end;
IdIcmpClient companent has OnReply event. This event has AReplyStatus param as type TReplyStatus. TReplyStatus has ReplyStatusType property. This property's type is TReplyStatusTypes. TReplyStatusTypes has rsTimeOut value. So add code to OnReply event and check timeout or other error.
procedure TForm1.IdIcmpClient1Reply(ASender: TComponent;
const AReplyStatus: TReplyStatus);
begin
if AReplyStatus.ReplyStatusType = rsTimeOut then
begin
//do someting on timeout.
end;
end;
I have an application (the "server") which updates a block of data in memory - around 100k bytes - every second.
There are 1 to 4 other instances of a "client" application running on other workstations on the same network, and these need to read the same 100k image every second.
This has been implemented up til now by writing the image to a file on the server and having the clients read from that file across the network. This has worked with no problems for many years, but lately (coincident with a move to windows 8-based hardware) it has developed a problem where the file becomes inaccessible to all nodes except one. Exiting the client application running on this node frees up the file and it then becomes accessible again to everyone.
I'm still perplexed as to the the cause of this lockout, but I'm wondering if it may be the mechanism discussed here, where a file isn't closed due to a network glitch. I'm thinking that having the clients request the data over TCP/IP would avoid this.
There doesn't need to be any handshaking other than the clients failing to connect or read data - the server just needs to go about it's business and respond to requests by grabbing the data and sending it. I'm pretty hazy however about the best architecture to achieve this. Are TidTCPClient and TidTCPServer going to cut it? I'm assuming the clients would request the data in a thread, but does this mean the server needs to run a thread continuously to respond to requests?
TIdTCPServer is a multi-threaded component. Its clients run in worker threads that it manages for you. All you have to do is implement the OnExecute event to send your data.
TIdTCPClient is not a multi-threaded component. It runs in whatever thread you use it in. So if you need to read data continuously, best to run your own worker thread to handle the reading. Indy has a TIdThreadComponent component that wraps a thread, or you can write your own TThread code manually.
100K is not a lot of data, so I would suggest simply forgetting the file altogether and allocate a buffer in memory instead. Your TIdTCPServer.OnExecute event handler can read from that buffer whenever needed. And I wouldn't even bother having the clients request data, just have the server continuously push the latest data to active clients.
Try something like this:
server:
var
Buffer: TIdBytes;
Lock: TMREWSync;
procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);
begin
Lock.BeginRead;
try
AContext.Connection.IOHandler.Write(Buffer);
finally
Lock.EndRead;
end;
Sleep(1000);
end;
procedure TForm1.UpdateBuffer;
begin
Lock.BeginWrite;
try
// update the Buffer content as needed...
finally
Lock.EndWrite;
end;
end;
initialization
Lock := TMREWSync.Create;
SetLength(Buffer, 1024 * 100);
finalization
SetLength(Buffer, 0);
Lock.Free;
client:
procedure TForm1.IdThreadComponent1Run(Sender: TIdThreadComponent);
var
Buffer: TIdBytes;
begin
IdTCPClient1.IOHandler.ReadBytes(Buffer, 1024 * 100);
// use Buffer as needed...
end;
procedure TForm1.Connect;
begin
IdTCPClient1.Connect;
try
IdThreadComponent1.Start;
except
IdTCPClient1.Disconnect;
raise;
end;
end;
procedure TForm1.Disconnect;
begin
IdTCPClient1.Disconnect;
IdThreadComponent1.Stop;
end;
I have three questions:
is it possible to destroy IdTCPServer by to many connection?
I tried to test my application and when I have several connections - it works very good (even several days) but when sometimes number of connection increases application gives acess violation. I wrote application similates 50 clients sending data constantly (with only sleep(200)). And in this situation IdTCPServer gives exceptions?
My application reseives information from clients by onExecute event and modyfies databases table using
TidNotify and TIdSync classes. I believe it protects crosses connections threads?
Sending information to clients is doing by TTimer (it is only now, I'll change it to other thread).
Have I use in this situation special protection or something like that is enough:
type
PClient = ^TClient;
TClient = record
Activity_time:TDateTime;
AContext: TIdContext;
end;
...
list := server.Contexts.LockList;
try
for i := 0 to list.Count - 1 do
with TIdContext(list[i]) do
begin
if SecondsBetween(now(), PClient(data)^.activity_time) > 6 then
begin
Connection.IOHandler.Close;
Continue;
end;
try
Connection.IOHandler.writeln('E:');
Except
Connection.IOHandler.Close;
end;
end;
finally
server.Contexts.UnlockList;
end;
2.Is a simple way to refuse connection when server is to busy (I think my database isn't complicated (100 rows, only one row is modyfied by one connection) but maybe here is a way to keep stability of server?
3.I know that this question was repeating many times but I didn't find satisfying answer: how to protect application to avoid message exception: "Connection closed gracefully" and "Connection reset by peer"?
Thank You for all advices
is it possible to destroy IdTCPServer by to many connection?
You are asking the wrong question, because you are not actually destroying TIdTCPServer itself, you are simply closing idle connections from an outside thread. That kind of logic can be (and should be) handled inside of the OnExecute event instead, where it is safest to access the connection, eg:
type
PClient = ^TClient;
TClient = record
Activity_time: TDateTime;
Heartbeat_time: TDateTime;
AContext: TIdContext;
end;
procedure TForm1.serverConnect(AContext: TIdContext);
var
Client: PClient;
begin
New(Client);
Client^.Activity_time := Now();
Client^.Heartbeat_time := Client^.Activity_time;
AContext.Data := TObject(Client);
end;
procedure TForm1.serverDisconnect(AContext: TIdContext);
var
Client: PClient;
begin
Client := PClient(AContext.Data);
AContext.Data := nil;
if Client <> nil then Dispose(Client);
end;
procedure TForm1.serverExecute(AContext: TIdContext);
var
Client: PClient;
dtNow: TDateTime;
begin
Client := PClient(AContext.Data);
dtNow := Now();
if SecondsBetween(dtNow, Client^.Activity_time) > 6 then
begin
AContext.Connection.Disconnect;
Exit;
end;
if SecondsBetween(dtNow, Client^.Heartbeat_time) > 2 then
begin
AContext.Connection.IOHandler.WriteLn('E:');
Client^.Heartbeat_time := dtNow;
end;
if AContext.Connection.IOHandler.InputBufferIsEmpty then
begin
if not AContext.Connection.IOHandler.CheckForDataOnSource(100) then
Exit;
end;
// process incoming data as needed ...
Client^.Activity_time := Now();
end;
Is a simple way to refuse connection when server is to busy (I think my database isn't complicated (100 rows, only one row is modyfied by one connection) but maybe here is a way to keep stability of server?
The current architecture does not allow for refusing connections from being accepted. You can let the server accept connections normally and then close accepted connections when needed. You can do that in the OnConnect event, or you can set the server's MaxConnection property to a low non-zero number to allow the server to auto-disconnect new connections for you without wasting resources creating new TIdContext objects and threads for them.
Another option is to call the server's StopListening() method when the server is busy, so that new connections cannot reach the server anymore, and then call the server's StartListening() method when you are ready to accept new clients again. Existing clients who are already connected should not be affected, though I have not actually tried that myself yet.
I know that this question was repeating many times but I didn't find satisfying answer: how to protect application to avoid message exception: "Connection closed gracefully" and "Connection reset by peer"?
You should not avoid them. Let them happen, they are normal errors. If they are happening inside of the server's events, just let the server handle them normally for you. That is how TIdTCServer is designed to be used. If they are happening outside of the server's events, such as in your timer, then just wrap the socket operation(s) in a try/except block and move on.
i am trying to send a picture from 'C:\picture.bmp' to 'c:\temp\picture.bmp' using server and client socket
clients onconnect event handler is as follow:
procedure TForm2.ClientSocket1Connect(Sender: TObject;
Socket: TCustomWinSocket);
var
fs : tfilestream;
begin
fs := TFileStream.create('C:\picture.bmp', fmOpenRead);//picture allready exists
socket.SendStream(fs);
fs.free;
end;
and servers onclientread as :
procedure TForm2.ServerSocket1ClientRead(Sender: TObject;
Socket: TCustomWinSocket);
var
fmm : tfilestream;
iLen: Integer;
Bfr: Pointer;
begin
iLen := Socket.ReceiveLength;
GetMem(Bfr, iLen);
fmm := TFileStream.Create('c:\temp\picture.bmp', fmCreate or
fmShareDenyWrite);
try
Socket.ReceiveBuf(Bfr^, iLen);
fmm.Write(Bfr^, iLen);
finally
FreeMem(Bfr);
fmm.Free;
end;
end;
picture is recieved/created but is either corrupt on was never recieved i.e created because of tfilestream.create method?
please help!what am i doing wrong?
Despite its name, SendStream() is NOT guaranteed to send the entire stream (especially if you are using a non-blocking socket). Its return value returns how many bytes are actually sent. If less than the full size of the stream are sent in one call, you have to call SendStream() again, potentially many times, to finish sending the entire stream (the same problems exists with SendText() as well).
On the other side, ReceiveLength() only reports how many bytes are available on the socket AT THAT MOMENT. That is likely to be less than the full stream being sent (likewise, ReceiveText() may not receive a full sent string either because it uses ReceiveLength() internally).
The best way to send a stream (or any arbitrary data in general) is to send the data's size first, then send the actual data afterwards. Keep calling SendBuf/Stream/Text() until that size is reached (if -1 is returned by a non-blocking socket without raising an exception, you have to wait for the socket's OnWrite event to trigger before the socket can accept more data again). On the receiving end, read the size first, then keep reading until the specified size is reached. You may have to read in multiple triggering of the OnRead event before you get all of the data.
Go to http://www.deja.com and http://forums.embarcadero.com to search the Borland/CodeGear/Embarcadero newsgroup/forum archives. I have posted example code many times before.
I don't know what's wrong, but I'd try troubleshooting a simpler problem. i.e. can you even transfer somethign simple? See if you can transfer c:\hello.txt containing just "Hello" and have it arrive in the right order. It should be easier to examine the stream and resulting file, to see if/where things are getting garbled. If you don't receive "Hello" on the server, then you know it's got nothing to do with the size or complexity of the data.