Reading unknown amount of bytes from IdTCPServer connection in delphi - delphi

I am setting up a client/server program to send and receive unknown amount of byte in my application. How to receive unknown buffer size in my program?
I pass my data to TIdBytes variable and sending it with this line:
IdTCPClient.IOHandler.Write(TIdBytes_Var);
The first byte of TIdBytes_Var determine packet size and varies due to other condition. I take care of the size of the package to not exceed 1024 byte.
On the receiver side (IdTCPServerExecute) I add this line to read all bytes:
var
byte_buffer: TIdBytes;
begin
AContext.Connection.IOHandler.ReadBytes(byte_buffer,-1,false);
end;
On debugging mode, after I send first packet from client to the server, an exception is raised:
raised exception class EConvertError with message ''' is not a valid floating point value
And on running mode only loss my connection.
Where is the problem, and how can I receive all bytes from Indy's input buffer?
Here is my minimal code:
In the client making packet and finally set event that is passed to other thread
//create packet. Buffer_to_send is a TIdBytes variable on unit of network thread (U_network_thread).
SetLength(U_network_thread.Buffer_to_send,(6 + (tmp_ints * 12)));//set buffer lenght dynamically, tmp_ints is my record counter(shortint)
tmp_int:= 2;//tmp_int is the integer variable
U_network_thread.Buffer_to_send[0]:= $1;//header sign
U_network_thread.Buffer_to_send[1]:= tmp_ints;//tmp_ints is my record counter as short integer
while tmp_ints > 0 do
begin
Read(My_File,tmp_rec);//read record from bin file
Move(tmp_rec,U_network_thread.Buffer_to_send[tmp_int],12);//12 is record lenght
tmp_int:= tmp_int + 12; //add pointer
dec(tmp_ints);
end;
tmp_int2:= zone_i;//integer info
Move(tmp_int2,U_network_thread.Buffer_to_send[tmp_int],4); //add info to the packet
Frq_Evt.SetEvent; //set event to triger sending on other thread
On the net_thread, after connecting to the server and receive ack sign by ReadLn command, I wait for event
procedure TNetThread.Execute;
.//create IdTCPClient1 and connect to server
.//receive ack from server by IdTCPClient1.IOHandler.ReadLn command
.//wait for event
while(true) do
if (FEvent.WaitFor(200) = wrSignaled) then//Buffer_to_send fill on main thread
begin
//send buff
if sending_buf(Buffer_to_send) then
begin
//sending ok
end
else
begin
//sending error
end;
end;
end;
end;
function TNetThread.sending_buf(T_Buffer: TIdBytes): boolean;
begin
try
IdTCPClient1.IOHandler.Write(T_Buffer)
result := true;
except
result := false;
end;
end;
On the server side, after connecting, sending ack sign by WriteLn command, and on the IdTCPServerExecute I try to do
procedure Tmain_form.IdTCPServerExecute(AContext: TIdContext);
var
byte_buffer: TIdBytes;
begin
AContext.Connection.IOHandler.ReadBytes(byte_buffer,-1,false);
//do something else
end;
All work fine and connection passed and server send ack sign to client. Client receive it and wait for event. After event occurs, client making up packet and pass it to net_thread. Packet is good and sending is too. But on the server side raise the problem.

the first byte of TIdBytes_Var determine packet size and varies due to other condition.
So, then simply read 1 byte first, then read how many additional bytes it says, eg :
AContext.Connection.IOHandler.ReadBytes(byte_buffer, AContext.Connection.IOHandler.ReadByte, false);
I take care of the size of the package to not exceed 1024 byte.
A single byte can not exceed 255, so your max packet size would be 256, including the size byte.
on debugging mode after i send first packet from client to the server rise a problem and receive:
"raised exception class EConvertError with message ''' is not a valid floating point value"
There is no way the code you have shown can raise that exception.
Update
Here is my minimal code
Seeing your code now, I can see that despite your earlier claim, the first byte of your packet is NOT a packet size, it is always $01. In fact, there is no packet size stored in the packet at all. The 2nd byte contains the number of 12-byte records stored in the packet, and the packet has 6 fixed bytes, so the max packet size is actually 3066 bytes.
Try something more like this:
passed to other thread
// create packet. Buffer_to_send is a TIdBytes variable on unit of network thread (U_network_thread).
SetLength(U_network_thread.Buffer_to_send, (6 + (tmp_ints * 12))); // set buffer length dynamically, tmp_ints is my record counter(shortint)
tmp_int := 2; // tmp_int is the integer variable
U_network_thread.Buffer_to_send[0] := $1; //header sign
U_network_thread.Buffer_to_send[1] := tmp_ints; // tmp_ints is my record counter as short integer
while tmp_ints > 0 do begin
Read(My_File, tmp_rec); // read record from bin file
Move(tmp_rec, U_network_thread.Buffer_to_send[tmp_int], 12); // 12 is record length
Inc(tmp_int, 12); // add pointer
Dec(tmp_ints);
end;
tmp_int2 := GStack.HostToNetwork(UInt32(zone_i)); // integer info, in network byte order
Move(tmp_int2, U_network_thread.Buffer_to_send[tmp_int], 4); // add info to the packet
Frq_Evt.SetEvent; // set event to trigger sending on other thread
procedure Tmain_form.IdTCPServerExecute(AContext: TIdContext);
var
byte_buffer: TIdBytes;
Header: Byte;
Count: Word;
Zone: Integer;
begin
Header := AContext.Connection.IOHandler.ReadByte; // should always be $01
Count := AContext.Connection.IOHandler.ReadByte; // record counter
AContext.Connection.IOHandler.ReadBytes(byte_buffer, Count * 12); // records
Zone := AContext.Connection.IOHandler.ReadInt32; // zone
// process as needed...
end;

Related

TIdTCPServer OnExecute fires multiple times

I have an app that receives a TCP socket connection from another app, processes the content that is streamed across the socket, builds a request based on the received data, sends the request using TIdHTTP to an external server, processes the response, and sends an ack back to the tcp client.
procedure TMyClass.TCPExecute(AContext: TIdContext);
var
s: AnsiString;
begin
s := TransferTCPBytesToString(AContext.Connection.IOHandler);
// test string for terminating characters
if Pos(#28#13, s) > 0 then
begin
//********PROBLEM: THIS WILL BE CALLED MULTIPLE TIMES
//********WITH THE SAME COMPLETE DATA STREAM CONTENT!
BuildAndSendExternalRequest(s);
AContext.Connection.IOHandler.InputBuffer.Clear;
// send the response back to tcp client
WriteTCPResponse(AContext.Connection.IOHandler);
end;
end;
function TMyClass.TransferTCPBytesToString(AnIOHandler: TIdIOHandler): AnsiString;
begin
if AnIOHandler.InputBufferIsEmpty then
Exit;
// read message from tcp client connection
AnIOHandler.ReadBytes(FTCPBytes, AnIOHandler.InputBuffer.Size, True);
SetLength(Result, Length(FTCPBytes));
// copy message bytes to string
Move(FTCPBytes[0], Result[1], Length(FTCPBytes));
end;
procedure TMyClass.WriteTCPResponse(AnIOHandler: TIdIOHandler);
var
bytes: TBytes;
begin
// FTCPBytes contains data from tcp client
// bytes will contain processed response from external server
BuildACK(FTCPBytes,bytes);
// FTCPBytes now contains ACK data
AnIOHandler.WriteDirect(FTCPBytes, Length(FTCPBytes));
end;
The request/response cycle completes successfully; the response sent back to the tcp client requestor is correct and the process executes and completes as expected.
However, because the OnExecute event fires multiple times with the same data, two requests are received, built, sent, acknowledged etc.
In looking at the logs on the tcp client app, the logs only show one request being sent. Not sure what is going on here. But I need to make sure that the request/response cycle is 1-1, not 1-many!
Thanks.
the OnExecute event fires multiple times ...
It is supposed to, as it is a looped event. The loop runs for as long as the connection is alive. The code inside your OnExecute event handler is responsible for deciding what happens on each loop iteration.
The ideal situation is where the event handler receives 1 message from the client, sends 1 response back to the client, and then exits, so that the next loop iteration is ready to handle the next request/response pair.
Which, based on your description, you are trying to do, but the code shown is not handling that very well. Most notably, the code is not accounting for the byte-stream nature of TCP. It does not differentiate one incoming message from another. It just dumps the entire IOHandler.InputBuffer content as-is into an AnsiString (why an AnsiString?), and if that AnsiString happens to contain the delimiter #28#13 then you are processing the entire AnsiString and then throwing it away, losing anything received after the delimiter (ie, part of the next message). And if the AnsiString doesn't contain the delimiter, then you are still throwing away the AnsiString, thus losing all data received so far for the current message before the delimiter arrives.
If all of your messages are delimited by #28#13, then I suggest you use the IOHandler.ReadLn() or IOHandler.WaitFor() method to read each message properly.
Also, you are re-using the same FTCPBytes member for both input and output, which in of itself is wrong, but you are also not providing any concurrent access protection for it if you have multiple client connections present.
With that said, try something more like this instead:
procedure TMyClass.TCPConnect(AContext: TIdContext);
begin
AContext.Connection.IOHandler.DefStringEncoding := IndyTextEncoding_8Bit;
end;
procedure TMyClass.TCPExecute(AContext: TIdContext);
var
s: String;
response: TIdBytes:
begin
s := TransferTCPBytesToString(AContext.Connection.IOHandler);
BuildAndSendExternalRequest(s, response);
// send the response back to tcp client
WriteTCPResponse(AContext.Connection.IOHandler, response);
end;
function TMyClass.TransferTCPBytesToString(AnIOHandler: TIdIOHandler): String;
begin
// read message from tcp client connection
Result := AnIOHandler.ReadLn(#28#13);
or:
Result := AnIOHandler.WaitFor(#28#13);
end;
procedure TMyClass.BuildAndSendExternalRequest(const Msg: String; var Response: TIdBytes);
begin
// send Msg to external server as needed...
// store response bytes in Response...
end;
procedure TMyClass.WriteTCPResponse(AnIOHandler: TIdIOHandler; const Response: TIdBytes);
var
bytes: TIdBytes;
begin
// Response contains processed response from external server
BuildACK(bytes, Response);
// bytes now contains ACK data
AnIOHandler.Write(bytes);
end;
... with the same data
That is not possible given the code you have shown. The call to IOHandler.ReadBytes() in TransferTCPBytesToString() is reading and discarding all bytes from the IOHandler.InputBuffer, so there is no way the same bytes can still be in that buffer when TransferTCPBytesToString() is called again at a later time. So, the only way you could be seeing the same data over and over is if the client is sending the same data over and over to begin with.

Indy TIdIcmpClient - how to detect a timeout?

(Using Delphi 2009) I'm trying to write a simple network connection monitor using Indy TidIcmpClient. The idea is to ping an address, then inside an attached TidIcmpClient.OnReply handler test how much data was returned. If the reply contains > 0 bytes then I know the connection succeeded but if either the TidIcmpClient timed out or the reply contains 0 bytes I will assume the link is down
I'm having difficulty understanding the logic of TidIcmpClient as there is no 'OnTimeout' event.
Two sub questions...
Does TidIcmpClient.OnReply get called anyway, either (a) when data is received OR (b) when the timeout is reached, whichever comes first?
How can I distinguish a zero byte reply because of a timeout from a real reply within the timeout period that happens to contain zero bytes (or can't this happen)?
In other words is this sort of code OK or do I need to do something else to tell if it timed out or not
procedure TForm1.IdIcmpClient1Reply(ASender: TComponent; const AReplyStatus: TReplyStatus);
begin
if IdIcmpClient1.ReplyStatus.BytesReceived = 0 then
//we must have timed out, link is down
else
//got some data, connection is up
end;
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
end;
When Ping() exits, the ReplyStatus property contains the same information that is passed to the AReplyStatus parameter of the OnReply event (you are ignoring that parameter). Ping() simply calls the OnReply handler right before exiting, passing it the ReplyStatus property, so you don't actually need to use the OnReply event in your example. All that is doing is breaking up your code unnecessarily.
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
// process IdIcmpClient1.ReplyStatus here as needed...
end;
That being said, you are not processing the ReplyStatus data correctly. The BytesReceived field can be greater than 0 even if the ping fails. As its name implies, it simply reports how many bytes were actually received for the ICMP response. ICMP defines many different kinds of responses. The ReplyStatusType field will be set to the type of response actually received. There are 20 values defined:
type
TReplyStatusTypes = (rsEcho,
rsError, rsTimeOut, rsErrorUnreachable,
rsErrorTTLExceeded,rsErrorPacketTooBig,
rsErrorParameter,
rsErrorDatagramConversion,
rsErrorSecurityFailure,
rsSourceQuench,
rsRedirect,
rsTimeStamp,
rsInfoRequest,
rsAddressMaskRequest,
rsTraceRoute,
rsMobileHostReg,
rsMobileHostRedir,
rsIPv6WhereAreYou,
rsIPv6IAmHere,
rsSKIP);
If the ping is successful, the ReplyStatusType will be rsEcho, and the ReplyData field will contain the (optional) data that was passed to the ABuffer parameter of Ping(). You might also want to pay attention to the FromIpAddress and ToIpAddress fields as well, to make sure the response is actually coming from the expected target machine.
If a timeout occurs, the ReplyStatusType will be rsTimeOut instead.
Try this:
procedure DoPing;
begin
IdIcmpClient1.ReceiveTimeout := 200;
IdIcmpClient1.Host := '8.8.8.8';
IdIcmpClient1.Ping;
if IdIcmpClient1.ReplyStatus.ReplyStatusType = rsEcho then
begin
// got some data, connection is up
end
else if IdIcmpClient1.ReplyStatus.ReplyStatusType = rsTimeout then
begin
// have a timeout, link is down
end
else
begin
// some other response, do something else...
end;
end;
IdIcmpClient companent has OnReply event. This event has AReplyStatus param as type TReplyStatus. TReplyStatus has ReplyStatusType property. This property's type is TReplyStatusTypes. TReplyStatusTypes has rsTimeOut value. So add code to OnReply event and check timeout or other error.
procedure TForm1.IdIcmpClient1Reply(ASender: TComponent;
const AReplyStatus: TReplyStatus);
begin
if AReplyStatus.ReplyStatusType = rsTimeOut then
begin
//do someting on timeout.
end;
end;

Data loss in Delphi using TIdTCPServer/TIdTCPClient components

I'm writing a client-server application using TIdTCPClient/TIdTcpServer indy components in Delphi.
Data transfering usually works fine, but often I read wrong data from server; I get previous requests' answers, not current ones.
During debug, both apps are working locally, so there are no way for data to be lost during transfer.
Timeouts are 1000-3000 msec, this is far enough to avoid sending second requests before receiving answer on the first one.
I use simple data format: first 4 bytes is data packet length, the rest is binary data of that length.
The server-side code is (simplified for sending strings only; I also use binary buffers the same way, but this code is simpler to understand and check):
Var
lng: LongInt;
ib: TIdBytes;
begin
// Prepare data to send:
lng:=length(s);// s is an AnsiString to be sent
SetLength(ib,lng+4);
Move(lng,ib[0],4);
Move(s[1],ib[4],length(s));
// Send:
AContext.Connection.IOHandler.WriteDirect(ib);
end;
The client-side code for sending request is the same (calling TIdTcpClient.IOHandler.WriteDirect() in last line).
The client-side code for reading server answer is:
Var
ib: TIdBytes;
size,done,lng: LongInt;
begin
Result:=false;
// answer length:
try
SetLength(ib,0);
tcp.IOHandler.ReadBytes(ib,4,false);
Move(ib[0],size,4);
if length(ib)<0 then Exit;// wrong data
except
on E: Exception do;// code skipped
end;
// read answer body:
done:=0;
b.Clear;// b is my buffer, TStream descendant
while done<size do
begin
lng:=Min(size-done,MaxBlockSize);
// read:
SetLength(ib,0);// to be sure
tcp.IOHandler.ReadBytes(ib,lng,false);
if length(ib)=0 then Exit;// error reading
// append my buffer:
b.Wr(ib[0],length(ib));
// progress:
Inc(done,length(ib));
end;
end;
The data exchange order is:
Client sends request to server,
Server reads the request and sends answer back to client,
Client reads the answer.
Wrong data appears on step 3.
Maybe I'm doing something generally wrong?
I've tried to ReadBytes() right before sending request to server to clear the incoming buffer, but that doesn't helps too, as many other things I've tried...
Now I'm just out of ideas :(
Your I/O logic is much more complicated than it needs to be, especially on the client side. You are manually doing things that Indy can do for you automatically.
On the client side, since you are saving the data into a TStream, you can have Indy read the data into the TStream directly:
begin
...
b.Clear;// b is my buffer, TStream descendant
// ReadStream() can read a '<length><bytes>' formatted
// message. When its ASize parameter is -1 and its
// AReadUntilDisconnect parameter is False, it reads
// the first 4 or 8 bytes (depending on the LargeStream
// property) and interprets them as the byte count,
// in network byte order...
tcp.IOHandler.RecvBufferSize := MaxBlockSize;
tcp.IOHandler.LargeStream := False; // read 4-byte length
// read answer:
try
tcp.IOHandler.ReadStream(b, -1, false);
except
on E: Exception do begin
// the socket is now in an indeterminate state.
// You do not know where the reading left off.
// The only sensible thing to do is disconnect
// and reconnect...
tcp.Disconnect;
...
end;
end;
...
end;
On the server side, there are two different ways you can send a message that would be compatible with the above code:
var
lng: LongInt;
ib: TIdBytes;
begin
// Prepare data to send:
// s is an AnsiString to be sent
lng := Length(s);
SetLength(ib, lng);
Move(PAnsiChar(s)^, PByte(ib)^, lng);
// Send:
AContext.Connection.IOHandler.Write(lng); // send 4-byte length, in network byte order
AContext.Connection.IOHandler.Write(ib); // send bytes
end;
Or:
var
strm: TIdMemoryBufferStream;
begin
// Prepare data to send:
// s is an AnsiString to be sent
strm := TIdMemoryBufferStream.Create(PAnsiChar(s), Length(s));
try
// Send:
// Write(TStream) can send a '<length><bytes>' formatted
// message. When its ASize parameter is 0, it sends the
// entire stream, and when its AWriteByteCount parameter
// is True, it first sends the byte count as 4 or 8 bytes
// (depending on the LargeStream property), in network
// byte order...
AContext.Connection.IOHandler.LargeStream := False; // send 4-byte lengtb
AContext.Connection.IOHandler.Write(strm, 0, True);
finally
strm.Free;
end;
end;
As you can see, this code is sending the same type of messages you were originally sending, what changed is the code that manages the messages. Also, it is forcing the message byte count to be sent in network byte order whereas you were sending it in host byte order instead. Multi-byte integers should always be sent in network byte order when possible, for consistency and multi-platform compatibility.

Serial Port Synchronization in Delphi

I am still having issues with the TComPort component but this time is not the component itself is the logic behind it. I have a device witch sends some ascii strings via serial port i need to prase those strings the problem is the computer reacts very fast so in the event char it captures only a part of the string the rest of the string comes back later... so parsing it when it is recived makes it impossible.
I was thinking in writing a timer witch verify if there was no serial activity 10 secons or more and then prase the string that i am saving into a buffer. But this method is unprofessional isn't there a idle event witch i can listen...Waiting for the best solution for my problem. Thanks.
After using a number of serial-port-components, I've got the best results until now, by using CreateFile('\\?\COM1',GENERIC_READ or GENERIC_WRITE,0,nil,OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,0), passing that handle to a THandleStream instance, and starting a dedicated thread to read from it. I know threads take a little more work than writing an event handler, but it still is the best way to handle any synchronization issues that arise from using serial ports.
Typical handler for OnRXChar event:
procedure XXX.RXChar(Sender: TObject; Count: Integer);
begin
ComPort.ReadStr(s, Count);
Accumulator := Accumulator + s;
if not AccumContainsPacketStart then
Accumulator := ''
else if AccumContainsPacketEndAfterStart then begin
ExtractFullStringFromAccum;
ParseIt;
end;
end;
Note.
Most com-port components do not have a clue when to report back to the owner. Normally the thread that is responsible to gather the bytes from the port is informed by the OS that one or more bytes are ready to be processed. This information is then simply popped up to your level. So when you expect the message to be transferred, you get what the OS is giving you.
You have to buffer all incoming characters in a global buffer. When you get the final character in your message string, handle the message.
Here is an example where the message start is identified with a special character and the end of the message is identified with another character.
If your message is constructed in another way, I'm sure you can figure out how to adapt the code.
var
finalBuf: AnsiString;
{- Checking message }
Function ParseAndCheckMessage(const parseS: AnsiString) : Integer;
begin
Result := 0; // Assume ok
{- Make tests to confirm a valid message }
...
end;
procedure TMainForm.ComPortRxChar(Sender: TObject; Count: Integer);
var
i,err: Integer;
strBuf: AnsiString;
begin
ComPort.ReadStr(strBuf, Count);
for i := 1 to Length(strBuf) do
case strBuf[i] of
'$' :
finalBuf := '$'; // Start of package
#10 :
begin
if (finalBuf <> '') and (finalBuf[1] = '$') then // Simple validate check
begin
SetLength( finalBuf, Length(finalBuf) - 1); // Strips CR
err := ParseAndCheckMessage(finalBuf);
if (err = 0) then
{- Handle validated string }
else
{- Handle error }
end;
finalBuf := '';
end;
else
finalBuf := finalBuf + strBuf[i];
end;
end;
If your protocol has begin/end markers, you can use TComDataPacket to provide you full packets, when they are available.
For certain amount of character we can use delay some miliseconds before ReadStr to make sure the data is completely sent. Example for 4 amount of character:
procedure TForm1.ComPort1RxChar(Sender: TObject; Count: Integer);
var
Str: String;
tegangan : real;
begin
sleep(100); //delay for 100ms
ComPort1.ReadStr(Str, 4);
...

Delphi — TClientSocket ReceiveText MaxLen?

I've been trying to set the length of the amount of characters you recover from the ReceiveText TClientSocket function and nothing seems to be working. E.g., Receiving the first leftmost character(s) from the recovered data or otherwise data stream. Is there a way to accomplish this in Delphi using this specific object?
Help would be much appreciated. Thanks in advance.
ReceiveText doesn't have any means to control the maximum length of the received text.
The easiest way in ClientType := ctBlocking mode is to use a TWinSocketStream as the documentation states:
http://docwiki.embarcadero.com/VCL/XE2/en/ScktComp.TClientSocket.ClientType
When ClientType is ctBlocking, use a TWinSocketStream object for reading and writing. TWinSocketStream prevents the application from hanging indefinitely if a problem occurs while reading or writing. It also can wait for the socket connection to indicate its readiness for reading.
Example code:
var
Stream : TWinSocketStream;
Buffer : TBytes;
S : string;
begin
SetLength(Buffer, 100); // 100 bytes buffer size
Stream := TWinSocketStream.Create(Socket, 5000); // 5 seconds or 5000 milliseconds
try
Stream.ReadBuffer(Buffer[0], Length(Buffer)); // raises an Exception if it couldn't read the number of bytes requested
S := TEncoding.Default.GetString(Buffer); // Works in Delphi 2009+
finally
Stream.Free;
end;
end;
here is a little tipp for sending and receiving text
first you must send the length of yout text too
Socket.SendText(IntToStr(Length(text)) + seperator + text);
then you can check at your server socket on receiving data streams, if your incoming text is complete
procedure TMyServer.OnClientRead(Sender: TObject; Socket: TCustomWinSocket);
begin
if (xRecLength = 0) then begin
if Length(Socket.ReceiveText) <= 0 then EXIT;
xRecLength:= StrToIntDef(GetFirstFromSplitted(Socket.ReceiveText, seperator), -1);
if xRecLength = -1 then EXIT;
end;
xActLength:= xActLength + Length(Socket.ReceiveText);
xRecPuffer:= xRecPuffer + Socket.ReceiveText;
isComplete:= xActLength = xRecLength;
if isComplete then begin
// complete text received
end;
end;
hope that helps you...
I'm not at home with Delphi, but a quick Google search turned up this page that indicates that ReceiveText does not accept any parameters, but instead returns a string of as much as it can read.
What you might need is might be ReceiveBuf instead.

Resources