I'm setting up a TCP client/server app, where I send a header record to the server (and possibly a file stream), and it will respond with a response record (and possibly a file stream).
But some times, I get stream errors (both sides), and I can't seem to clear them with IOHandler.InputBuffer.Clear().
Here's some example code:
CLIENT SIDE
var
FS: TFileStream;
FClient.IOHandler.LargeStream := True;
FS := TFileStream.Create(FileName, fmOpenRead);
try
FClient.IOHandler.Write(FS, FS.Size, True); //send the file w/size
finally
FreeAndNil(FS);
end;
SERVER SIDE
var
FS: TFileStream;
//in ServerExecute
FS := TFileStream.Create(FileName, fmCreate Or fmOpenWrite); //create the file
IdContext.Connection.IOHandler.LargeStream := True;
try
if IdContext.Connection.IOHandler.InputBufferIsEmpty then //if no data yet
if not IdContext.Connection.IOHandler.CheckForDataOnSource(30000) then
begin //read timeout
SErr := 'read time out, stream not received!';
Exit;
end;
IdContext.Connection.IOHandler.ReadStream(FS, -1, False); //read stream with size specified at start of stream
if FS.Size <> (expected file size sent in header packet) then
begin
SErr := 'not all stream bytes received!';
Exit;
end;
finally
FreeAndNil(FS); //close the file
end;
Regardless of whether it is the server or client side receiving, when I do IdContext.Connection.IOHandler.ReadStream(FS, -1, False) it will freeze on the stream intermittently (probably because the transfer count is less than the expected bytes). Once I detect a timeout condition, I will clear the input buffer, but from then on it's like the transfers are out of sync; i.e. I send a command packet, then the file packet, but I get an error like it received the file packet first, and it keeps doing that for the following failure retries.
Related
I have just started "playing" with Indy. Used this post how to send file from server to client using indy and altered it so that my Client is the one Sending Data to the Server.
Client has following code to send Data :
procedure TForm1.btnConnectClick(Sender: TObject);
begin
TCPClient.Host:='192.168.88.117';
TCPClient.Port:=32832;
TCPClient.Connect;
end;
procedure TForm1.btnSendClick(Sender: TObject);
var
AStream : TFileStream;
begin
if not TCPClient.Connected then Exit;
TCPClient.IOHandler.WriteLn('SEND_FILE '+GetComputerName+FormatDateTime('yyyy-mm-dd_hhnnsszzz',now)+'.txt');
try
AStream := TFileStream.Create(ExtractFilePath(Application.ExeName)+'\1.txt', fmOpenRead or fmShareDenyWrite);
TCPClient.IOHandler.LargeStream := true;
TCPClient.IOHandler.Write(AStream, 0, True);
finally
AStream.Free;
end;
TCPClient.Disconnect; // otherwise the file is locked on the server side
end;
Server has following code to receive Data :
procedure TForm1.TCPServerExecute(AContext: TIdContext);
var
AStream : TFileStream;
cmd, params, filename : string;
begin
params := AContext.Connection.IOHandler.ReadLn();
cmd := Fetch(params);
if cmd = 'SEND_FILE' then
begin
filename := ExtractFileName(params);
TThread.Queue(nil,
procedure
begin
Memo1.Lines.Add('Command '+cmd+' File Name '+filename);
end
);
AStream := TFileStream.Create(ExtractFilePath(Application.ExeName)+'\'+filename, fmCreate);
try
AContext.Connection.IOHandler.LargeStream:=true;
AContext.Connection.IOHandler.ReadStream(AStream, -1, false);
finally
AStream.Free;
end;
end;
end;
It seems to function correctly , I have made two clients for my test and ran them from separate computers , the server was receiving data from both of them and was creating the data correctly.
I have only two Problems :
1. for some reason the File Received on the Server has always the same amount of leading spaces followed by a # symbol.
Original File looks like this
HERE IS SOME STUFF
Received file on Server looks like this :
#HERE IS SOME STUFF
2. It looks like that after every file Send I need to disconnect , otherwise Indy TCPServer keeps the file locked , is this expected behavior ? How can I tell if the File is done ? I will need to process the received files in another Thread one-by-one .
Thank you.
UPDATE 1
So as recommended by Remy I've altered the Server like this :
procedure TForm1.TCPServerExecute(AContext: TIdContext);
var
AStream : TFileStream;
cmd, params, filename : string;
begin
params := AContext.Connection.IOHandler.ReadLn();
cmd := Fetch(params);
if cmd = 'SEND_FILE' then
begin
filename := ExtractFileName(params);
AStream := TFileStream.Create(ExtractFilePath(Application.ExeName)+'\'+filename, fmCreate);
try
AContext.Connection.IOHandler.WriteLn('OK');
AContext.Connection.IOHandler.LargeStream:=true;
AContext.Connection.IOHandler.ReadStream(AStream, -1, false);
finally
AStream.Free;
end;
end;
params := AContext.Connection.IOHandler.ReadLn();
cmd := Fetch(params);
if cmd = 'SENT' then
RenameFile(filename,ChangeFileExt(filename,'.dat'));
end;
On the Client I moved the entire Transfer into a Thread using ITask like this :
procedure TForm1.BackgroundTransfer;
var
TCPClient : TidTCPClient;
AStream : TFileStream;
Files : TStringDynArray;
i : integer;
isDone : boolean;
begin
Files := TDirectory.GetFiles(ExtractFilePath(Application.ExeName)+'\1\','*.out');
TThread.Queue(TThread.CurrentThread,
procedure
begin
Memo1.Lines.Add(IntToStr(Length(Files)));
end
);
if Length(Files) = 0 then Exit;
TCPClient := TidTCPClient.Create(nil);
try
TransferActive:=true;
TCPClient.Disconnect;
TCPClient.Host:='192.168.88.117';
TCPClient.Port:=32832;
TCPClient.Connect;
for i := Low(Files) to High(Files) do
begin
isDone:=false;
if FileExists(Files[i]) = true then
begin
TCPClient.IOHandler.WriteLn('SEND_FILE '+Files[i]);
try
// wait for server
repeat
sleep(100);
if TCPClient.IOHandler.ReadLn() = 'OK' then break;
until ( true );
AStream := TFileStream.Create(Files[i], fmOpenRead or fmShareDenyWrite);
TCPClient.IOHandler.LargeStream := true;
TCPClient.IOHandler.Write(AStream, 0, True);
isDone:=true;
TCPClient.IOHandler.WriteLn('SENT '+Files[i]);
finally
AStream.Free;
end;
if isDone = true then
System.SysUtils.DeleteFile(Files[i]);
end;
end;
finally
TCPClient.Free;
TransferActive:=false;
end;
end;
I Check with a Timer every 10 seconds if the Task is running if not I create a new one like this :
procedure TForm1.Timer2Timer(Sender: TObject);
if TransferActive = false then
begin
inc(ThreadTimer);
Panel1.Caption:=ThreadTimer.ToString;
Panel1.Color:=clRed;
end
else
begin
Panel1.Color:=clGreen;
end;
if ThreadTimer >= 10 then
begin
ThreadTimer:=0;
TransferTask := TTask.Create(BackGroundTransfer);
TransferTask.Start;
end;
end;
If I understood it correctly Indy Keeps Track of the Connections in the Background . Connection A-A , B-B , C-C , they cannot get mixed up . I am asking this because from the Server and Client side I just send OK or SENT and it works . ( hopefully not just by pure luck )
This is important because once it is fully working , I will have multiple client (android devices) sending data to this server . And there will be quiet a high chance that sometimes more then one client will be uploading data.
I also tested this like what happens if I start copying files into the Clients INPUT Directory and the Copy is not Done but the Task start . It worked , on first run it detected 350 files on the next run 500 .
Also tested if I simply stop the Server what happens that works too . If I use the TCPServer.Active:=false;
On the Client side the WriteLn and ReadLn properly causes an Exception ( Server Timeout I guess ) if the connection is lost .
On the Server Side I simply rename the Received file from OUT to DAT once it is done . I am not 100% sure if that guarantees that the file was really 100% correctly transferred . I was however unable to produce a damaged file during my testings.
Anyway the Entire idea is :
TCPClient is running on a Android phone, where the User Scanns Barcodes , I create a control file from this which is then in 10 sec intervals uploaded to the server . And from then on Processed by the Server and sent to another server .
Regards
UPDATE 2
Removed this line from server :
DelFilesFromDir(ExtractFilePath(Application.ExeName), '*.out', FALSE);
was a bad idea to put in inside the Execute method .
I will need to find another way to remove possible junk files .
Also TransferActive is a Global variable , and is being modified by the Transfer Background Thread. But since only one Thread is running at a Time, I thought it should be safe.
Hello im trying to send a file Via TCP with delphi using Indy Components with a Client/Server
i manage to send and recieve the files correctly, the problem is before sending the file i would like to send its size aswell to compare it after i get it the server.
Now im sending the files from the Client To Server.
Client:
Ms := TMemoryStream.Create;
Ms.LoadFromFile('cliente.exe');
Ms.Position := 0;
Result := True;
Client.IOHandler.LargeStream := True;
try
Client.IOHandler.Write(ms, 0, True);// (Ms, 0, true);
except
Result := False;
end;
Ms.Free;
Server:
AStream := TFileStream.Create('C:\temp\file.exe', fmCreate + fmShareDenyNone);
try
AContext.Connection.IOHandler.LargeStream := True;
AContext.Connection.IOHandler.ReadStream(AStream, -1,false);
finally
FreeAndNil(AStream);
Memo1.Lines.Add('File received');
end;
So the question would be how could i send the file size with the file?
Your code is already sending the file size. You are setting the AWriteByteCount parameter of Write(TStream) to True, which tells it to send the stream size before sending the stream data. And you are telling ReadStream() to read the stream size before reading the stream data. So Indy is already validating the size for you before ReadStream() exits. You don't need to do it manually at all.
I've this code snippet in Python:
s = socket.create_connection(('192.168.0.111', 123), timeout=2.0)
s.sendall('REQUEST,status,interface'); result = s.recv(1024)
How I can do the "s.recv(1024)" in Delphi using TIdTCPClient from Indy components? Server returns a string without any terminator, so ReadLn doesn't work.
In Python, recv(1024) simply reads whatever is available on the socket, up to 1024 bytes max. It is possible to do the same thing with TIdTCPClient (see below), but it is not the best way to handle socket programming in general. You really need to know how the server is actually terminating the data. Does it close its end of the connection after sending the data? If not, does it expect you to simply read whatever bytes are available regardless of the actual length? And if so, how does it expect you to handle TCP packet fragmentation (TCP/IP can separate transmitted data into multiple packets, and may receive data as multiple packets even if they were not sent as such)? Does it expect you to keep reading until some timeout occurs, indicating no more data is being sent?
It makes a big difference in how you write code to handle the reading properly. For example:
IdTCPClient1.Host := '192.168.0.111';
IdTCPClient1.Port := 123;
IdTCPClient1.ConnectTimeout := 2000;
IdTCPClient1.Connect;
try
IdTCPClient1.IOHandler.Write('REQUEST, status, interface');
// wait for a disconnect then return everything that was received
Result := IdTCPClient1.IOHandler.AllData;
finally
IdTCPClient1.Disconnect;
end;
Vs:
IdTCPClient1.Host := '192.168.0.111';
IdTCPClient1.Port := 123;
IdTCPClient1.ConnectTimeout := 2000;
IdTCPClient1.Connect;
try
IdTCPClient1.IOHandler.Write('REQUEST, status, interface');
// set the max number of bytes to read at one time
IdTCPClient1.IOHandler.RecvBufferSize := 1024;
// read whatever is currently in the socket's receive buffer, up to RecvBufferSize number of bytes
IdTCPClient1.IOHandler.CheckForDataOnSource;
// return whatever was actually read
Result := IdTCPClient1.IOHandler.InputBufferAsString;
finally
IdTCPClient1.Disconnect;
end;
Vs:
IdTCPClient1.Host := '192.168.0.111';
IdTCPClient1.Port := 123;
IdTCPClient1.ConnectTimeout := 2000;
IdTCPClient1.Connect;
try
IdTCPClient1.IOHandler.Write('REQUEST, status, interface');
// keep reading until 5s of idleness elapses
repeat until not IdTCPClient1.IOHandler.CheckForDataOnSource(5000);
// return whatever was actually read
Result := IdTCPClient1.IOHandler.InputBufferAsString;
finally
IdTCPClient1.Disconnect;
end;
I have a problem with Indy TCP connection. I use Turbo Delphi 2006 with Indy 10.
I want to send multiple TCP packages from idTCPClient to idTCPServer.
It works finely, when I want to send only one package, or I insert a sleep(100) command between two calls of the function. But if I call this function too frequently, it doesn't call the server's onExecute every time.
My code for sending:
procedure SendData(var data: TIdBytes) ;
begin
FormMain.IdTCPClient.Connect ;
FormMain.IdTCPClient.Socket.Write(data);
FormMain.IdTCPClient.Disconnect ;
end ;
I call this function several times (5-10 times in a second), and want to process all of these packages in my server application:
procedure TFormMain.IdTCPServerMainExecute(AContext: TIdContext);
var
data: TIdBytes ;
begin
AContext.Connection.IOHandler.ReadBytes(data, 4, false)
// processing data
end
Thank you for your answers in advance!
Every time you call Connect(), you are creating a new connection, and TIdTCPServer will start a new thread to handle that connection (unless you enable thread pooling, that is). Is that what you really want? It would be more efficient to have the client leave the connection open for a period of time and reuse the existing connection as much as possible. Disconnect the connection only when you really do not need it anymore, such as when it has been idle for awhile. Establishing a new connection is an expensive operation on both ends, so you should reduce that overhead as much as possible.
On the client side, when you call Write(data), it will send the entire TIdBytes, but you are not sending the length of that TIdBytes to the server so it knowns how many bytes to expect. TIdIOHandler.Write(TIdBytes) does not do that for you, you have to do it manually.
On the server side, you are telling ReadBytes() to read only 4 bytes at a time. After each block of 4 bytes, you are exiting the OnExecute event handler and waiting for it to be called again to read the next block of 4 bytes. Unless the length of the client's source TIdBytes is an even multiple of 4, ReadBytes() will raise an exception (causing the server to disconnect the connection) when it tries to read the client's last block that is less than 4 bytes, so your server code will not receive that block.
Try this instead:
procedure SendData(var data: TIdBytes) ;
begin
FormMain.IdTCPClient.Connect;
try
FormMain.IdTCPClient.IOHandler.Write(Longint(Length(data)));
FormMain.IdTCPClient.IOHandler.Write(data);
finally
FormMain.IdTCPClient.Disconnect;
end;
end;
procedure TFormMain.IdTCPServerMainExecute(AContext: TIdContext);
var
data: TIdBytes;
begin
with AContext.Connection.IOHandler do
ReadBytes(data, ReadLongint, false);
// process data
end;
With that said, if changing the client code to send the TIdBytes length is not an option for whatever reason, then use this server code instead:
procedure TFormMain.IdTCPServerMainExecute(AContext: TIdContext);
var
LBytes: Integer;
data: TIdBytes;
begin
// read until disconnected. returns -1 on timeout, 0 on disconnect
repeat until AContext.Connection.IOHandler.ReadFromSource(False, 250, False) = 0;
AContext.Connection.IOHandler.InputBuffer.ExtractToBytes(data);
// process data
end;
Or:
procedure TFormMain.IdTCPServerMainExecute(AContext: TIdContext);
var
strm: TMemoryStream;
data: TIdBytes;
begin
strm := TMemoryStream.Create;
try
// read until disconnected
AContext.Connection.IOHandler.ReadStream(strm, -1, True);
strm.Position := 0;
ReadTIdBytesFromStream(strm, data, strm.Size);
finally
strm.Free;
end;
// process data
end;
Or:
procedure TFormMain.IdTCPServerMainExecute(AContext: TIdContext);
var
strm: TMemoryStream;
begin
strm := TMemoryStream.Create;
try
// read until disconnected
AContext.Connection.IOHandler.ReadStream(strm, -1, True);
// process strm.Memory up to strm.Size bytes
finally
strm.Free;
end;
end;
I am trying to exchange data between a IdTCPServer and an IdTCPClient using a collection that I convert to a stream before I send it over the network. Unfortunately, no matter how I try, I can't seem to pass the stream between the client and the server. The code always hangs on the IdTCPClient1.IOHandler.ReadStream(myStream, -1, True) line.
The relevant portion of my code is shown below:
Client side
with ClientDataModule do
begin
try
try
intStreamSize := StrToInt(IdTCPClient1.IOHandler.ReadLn); // Read stream size
IdTCPClient1.IOHandler.ReadStream(myStream, -1, True); // Read record stream
finally
ReadCollectionFromStream(TCustomer, myStream);
end;
except
ShowMessage('Unable to read the record from stream');
end;
end;
Server side
try
try
SaveCollectionToStream(ACustomer, MStream);
finally
MStream.Seek(0, soFromBeginning);
IOHandler.WriteLn(IntToStr(MStream.Size)); // Send stream size
IOHandler.Write(MStream, 0); // Send record stream
end;
except
ShowMessage('Unable to save the record to stream');
end;
I would greatly appreciate your assistance with solving this problem.
Thanks,
JDaniel
You are setting the AReadUntilDisconnect parameter of ReadStream() to True, which tells it to keep reading until the connection is closed. You need to set the parameter to False instead. You also need to pass in the stream size in the AByteCount parameter, since you are sending the stream size separately so you have to tell ReadStream() how much to actually read.
Try this:
Client:
with ClientDataModule do
begin
try
intStreamSize := StrToInt(IdTCPClient1.IOHandler.ReadLn);
IdTCPClient1.IOHandler.ReadStream(myStream, intStreamSize, False);
myStream.Position := 0;
ReadCollectionFromStream(TCustomer, myStream);
except
ShowMessage('Unable to read the record from stream');
end;
end;
Server:
try
SaveCollectionToStream(ACustomer, MStream);
MStream.Position := 0;
IOHandler.WriteLn(IntToStr(MStream.Size));
IOHandler.Write(MStream);
except
ShowMessage('Unable to save the record to stream');
end;
If you can change your protocol, then you can let Write() and ReadStream() exchange the stream size internally for you, like this:
Client:
with ClientDataModule do
begin
try
// set to True to receive a 64bit stream size
// set to False to receive a 32bit stream stream
IdTCPClient1.IOHandler.LargeStream := ...;
IdTCPClient1.IOHandler.ReadStream(myStream, -1, True);
myStream.Position := 0;
ReadCollectionFromStream(TCustomer, myStream);
except
ShowMessage('Unable to read the record from stream');
end;
end;
Server:
try
SaveCollectionToStream(ACustomer, MStream);
MStream.Position := 0;
// set to True to send a 64bit stream size
// set to False to send a 32bit stream stream
IOHandler.LargeStream := ...;
IOHandler.Write(MStream, 0, True);
except
ShowMessage('Unable to save the record to stream');
end;