I'm new to delphi and this is my first project.
Here's a little bit of code:
procedure TForm1.Button2Click(Sender: TObject);
responseStringFromServer:TStringStream;
begin
try
if IdTCPClient1.Connected then
begin
dataSentToDevice:= 'http/1.0 content-length: 344 content-type: text/xml <?xml version="1.0" encoding="UTF-8" ?> ...'
IdTCPClient1.IOHandler.WriteLn(dataSentToDevice);
responseStringFromServer := TStringStream.Create;
IdTCPClient1.IOHandler.ReadStream(responseStringFromServer);
...
I have a device connected to local network. I manage to connect to it succesfully. Sending commands is working too, but when i do
IdTCPClient1.IOHandler.ReadStream(responseStringFromServer);
then it waits til device is done processing and an exception occurs: "Connection Closed Gracefully". So i'm not able to read data the device is supposed to send me. Device is not shutting down. I've read other posts and i understand that device itself drops connection.
I have a demo program that communicates with it and it works fine. I need to get response xml before it drops the connection.
Also the request is http and i am using IdTCPClient (i need to use xml request, i don't know how to do it with TidHTTP).
May it be that after device is sending response it drops the connection, so that my tcpclient gets connection dropped flag before receiving data.
Any help would be appreciated!
Also the request is http and i am
using IdTCPClient (i need to use xml
request, i don't know how to do it
with TidHTTP).
Working with IdHTTP is simple...
Drop an instance of it on your form,
select it.
In Object Inspector, go to
ProtocolVersion property, and set it
to pv1_0, then open its
Request property set, and set
Request.ContentType to text/xml, and
Request.ContentEncoding to UTF-8,
and set other properties if
required.
Add a button to your form and
double-click on it.
In your code, create an instance of
TStringStream, and load your XML
content into it.
Assign your stream to
IdHttp.Request.Source.
Call IdHttp.Get() method by giving
it a host address.
IdHttp.Get() returns a string which
is the response the server sent you.
The way you are calling ReadStream(), it will interpret the first 4 bytes (or 8 bytes if the TIdIOHandler.LargeStream property is True) as an Integer (or Int64) in network-byte order that specifies the length of the data, and then it will try to read that many bytes. Is the device actually sending such a length value? If not, then ReadStream() will attempt to read the wrong number of bytes. An EIdConnClosedGracefully exception means the device is closing the socket on its end. So either the device is closing the connection immediately after sending its data, or it is timing out waiting for you to send the next command, which you cannot do since you are blocked waiting for the wrong data from the previous command.
As said by #Remy Lebeau and the documentation (press F1 when your edit caret is over the ReadStream sentence):
AByteCount indicates the number of bytes from the IOHandler to be read into AStream. When AByteCount contains -1 and AReadUntilDisconnect contains False, the byte count is read as an Integer value from the IOHandler. The size of AStream is adjusted to match the size expected from the IOHandler.
This is very useful when you're connecting to servers written also in INDY WriteStream in one end, ReadStream in the other, or any other language if you send that byte cound as expected by INDY client.
If you're reading from a device, be sure the device is sending that info up-front the stream or change the way you read the data. If you know the size of the stream, just pass the second parameter: AByteCount. If the device will close the channel when the stream ends, pass -1 as the second parameter and True as the third parameter: AReadUntilDisconnect.
If the device send text of unknown length but using a known terminator (like CR/LF), you better use ReadLn method of IOHandler to get that string.
Take a look at if neither fits your needs, take a look on Read* methods of TIOHandlerClass and RTFM for each to find the correct one to get the data sent by the device.
Related
I have in TCP/IP a client that send a list of strings with writeln (all at the same time).
How can the TCPserver know that it has read all the data?
In Onexecute event, I have used:
If (not AThread.Terminated)and(athread.Connection.Connected) then
Memo1.lines.add(AThread.Connection.readln);
But the problem, is that some last lines are not read.
You should send each string with its own WriteLn(), then you can either:
Send the number of strings before sending the actual strings. The server can then read the number first, then call ReadLn() however many times the number says.
Send a unique terminating line after the strings. The server can then keep calling ReadLn() until the terminator is read. If you need to send a string that could be ambiquious with the terminator, escape the string before sending it, and then have the server unescape it after reading it.
Indy has reading/writing methods for handling both scenarios, such as WriteStrings(), ReadStrings(), WriteRFCStrings(), Capture(), etc.
On an unrelated note, you should not be checking the AThread.Terminated and AThread.Connection.Connected properties. Let Indy raise an exception if you try to read from or write to a disconnected socket, and let the server handle the exception. Also, TIdTCPServer is multi-threaded, and accessing UI controls in the OnExecute event (or OnConnect, OnDisconnect, or OnException) is not thread-safe. You MUST synchronize with the main UI thread to access them safely.
I have a device that sends data to my server via gprs . The problem is that it sends raw data and i don't know where i can stop the reading
Currently i am using something TIdHttpServer and something like this to read the strings :
var
s : string;
repeat
s:=s+acontext.Connection.Socket.ReadChar;
until acontext.Connection.Socket.InputBufferIsEmpty;
Is there a better solution to my problem ?
TCP is stream oriented. If the protocol is unknown, the server only can try to read into a byte array (if memory is not a problem) or a file stream. If the client disconnects normally, the data is 'complete'. Unfortunately, if the protocol is unknown, the server can not tell wether the client died or disconnected normally.
InputBufferIsEmpty does not help, as it only says if there is data in the (TCP) buffer - and depending on latency this can happen frequently, but it does not mean that there are no more in-flight bytes.
You could try to 'reverse engineer' the protocol, by sending known strings over the client devices. But if the sender is a black box, there can be many special cases - think of encoding or 'escape' characters etc.
You could make up you own protocol.
Some ideas are:
use a special character or characters combo to define the end of the
message.
append at the start of the message some fixed size field with the size of the message
I am using the Internet Direct TIdTCPClient component to communicate with a remote service to retrieve a message that is normally about 5k is size. During a typical operation I will send about 400 requests to the service, each one taking about 1 second to complete. Most of the time everything works perfectly. However, about one percent of the time the request takes 189 seconds and I receive no data at all. For ease of discussion, I'll call this a failure.
I am particularly interested in understanding exactly what is happening when the failure occurs so that I can take my evidence to the publisher of the service. First of all, the failure is not reproducible. If I re-send a failed request there is a very high probability (maybe 99 percent) that it will work.
I also capture the request that I send when a failure occurs, so I am able to confirm that the request is well formed.
I am assuming that during a failure that I am getting some data, just not all of it. Here is why. My IdTCPClient has a 30 second timeout (I've even set it to 5 seconds, but that didn't make a difference). When the failure occurs, it always fails after 189 seconds (plus about 500 milliseconds).
As a result, I am thinking that during a failure my component is receiving a trickle of data, which is why my client is not timing out. And, I am assuming that the disconnection is happening at the service since none of my timeout values are ever set to 189 seconds. On the other hand, reading IOHandler.AllData does not raise an exception (not even an EIdConnClosedGracefully exception). Am I interpreting this evidence correctly?
What I want to do is to confirm that I am getting some data, just not all of it, before the service terminates the connection. Furthermore, I want to know what that partial data looks like as I believe it can help identify the source of the failure.
Currently, my request is similar to the following:
//ExceptionName is a temporary global variable
//that I am using while trying to solve this issue
ExceptionName = 'no exception';
try
s := GetRequest(id);
IdTcpClient1.Host := Host;
IdTcpClient1.Port := StrToInt(Port);
IdTcpClient1.ReadTimeout := ReadTimeout;
try
IdTcpClient1.Connect;
except
on e: exception do
begin
ExceptionName := e.ClassName;
raise EConnectionFailure.Create('Connection refused: ' + e.Message)
end;
end;
IdTcpClient1.IOHandler.Writeln(s);
try
Result := IdTcpClient1.IOHandler.AllData;
except
on E: EIdConnClosedGracefully do
begin
ExceptionName := e.ClassName;
//eat this exception
end;
on e: Exception do
begin
ExceptionName := e.ClassName;
raise;
end;
end;
finally
if IdTcpClient1.Connected then
IdTcpClient1.Disconnect;
end;
Using IOHandler.AllData to read the data is very convenient, but I cannot retrieve any data following a failure (AllData returns an empty string). I've tested IOHandler.InputBufferIsEmpty after a failure, and it returns True.
I've also tried other methods to read the data, such as IOHandler.ReadStream (this produced the same result as reading AllData). I also used IOHandler.ReadBytes and IOHandler.ReadByte (in conjunction with IOHandler.CheckForDataOnSource). Nothing has worked.
Am I wrong about partial data transmission? If so, why would I see a consistent 189.nnnn seconds delay before the failure.
If partial data transmission is a possibility, what approach should I take to capture every byte of data received before the failure.
I am using Delphi 2009 for this project and Indy 10, but I don't think the version has anything to do with it. I don't think this is an Indy issue.
Edit: I have inspected the communication between my Indy client and the server using WireShark. When one of these failures occur, after sending my request the server sent two [ACK] packets followed by silence for just over 189 seconds. After that delay, the response included [FIN, PSH, ACK] but no application data.
When the communication worked normally, the two ACK packets returned by the server in response to my request was followed by an application data packet.
Edit: Have reported the issue to the publisher of the Web service, and am waiting for a reply.
Edit: Ok, the publisher of the Web service has responded. They acknowledged problems on their end and have addressed some of those. We are no longer getting timeouts. Most responses are received in about 2 seconds, with a few taking slightly longer. The publisher is working to fix the remaining issues.
Thank you everyone for your input.
You need to run Fiddler2 and watch the traffic. It inserts itself as a proxy and sniffs evertying that uses the WinInet stack.
Then you know whether you've gotten any data or not, and exactly what you're sending and receiving.
http://www.fiddler2.com/fiddler2/
I'm facing problem with TCpindy connection.readln method , I had no control in the other side sending data , when using Readln method in server side application hang (because receiving data don't contain carrige return ) , i'm trying readstring method but without success
Is there any suggestion to encouter this problem , me be looking for other component rather than indy ,
I need to get data from other client (tcp connection ) without any information about size of receiving data and without carriage return at the end of each frame.
You have to know how the data is being sent in order to read it properly. TCP is a byte stream, the sender needs to somehow indicate where one message ends and the next begins, either by:
prefixing each message with its
length
putting unique delimiters in between
each message
pausing in time between each message
Indy can handle all of these possibilities, but you need to identify which one is actually being used first.
Worse case scenerio, use the CurrentReadBuffer() method, which returns a String of whatever raw bytes are available at that moment.
I am sending data to the server twice. First, I send "Hello world" and then I send "Server".
But the server received the data at 1 read. But the server have to read the data in a two-read operation.
Also, I write the data. Then read data from server and then I write the data.
In this case, the server can read the first data. But server can not read the second data.
The server uses read, write, read.
So how to overcome this issue? How do I write data to socket in BlackBerry?
What you describe is how TCP is supposed to work by default. What you are seeing is the Nagle algorithm (RFC 896) at work, reducing the number of outbound packets being sent so they are processed as efficiently as possible. You may be sending 2 packets in your code, but they are being transmitted together as 1 packet. Since TCP is a byte stream, the receiver should not be making any assumptions about how many packets it gets. You have to delimit your packet data in a higher-level protocol, and the receiver has to process data according to that protocol. It has to handle cases where multiple packets arrive in a single read, a single pakcet arriving in multiple reads, and everything in between, only processing packet data when they have been received in full, caching whatever is left over for subsequent reads to process when needed.
Hard to say without a little more detail, but it sounds like you're using 1-directional communication in the first case - i.e. the client writes, then writes again. There are any number of reasons that the server would receive the 2 writes as 1 read. Buffering on the client, somewhere in the wireless stack (or in the BES), buffering on the server side. All of those are legal with TCP/IP.
Without knowing anything more about your solution, have you thought about defining a small protocol - i.e. the client writes a known byte or bytes (like a 0 byte?) before sending the second write? Then the server can read, then recognize the delimiting byte, and say 'aha, this is now a different write from the client'?
As previously said this is an expected TCP behavior to save bandwidth. Note that to deliver your package TCP adds lot of data (e.g. destination port,sequence number, checksums...).
Instead of flushing the data I´ll recommend you to put more work in your protocol. For example you can define a header that contains the number of bytes to read and then the payload (the actual data).
The following code is a protocol encoded in an string with the structure [length];[data]
StringBuffer headerStr = new StringBuffer();
StringBuffer data = new StringBuffer();
//read header
char headerByte = dataInputStream.readChar();
while (headerByte != ';') {
headerStr.append(headerByte);
headerByte = dataInputStream.readChar();
}
//header has the number of character to read
int header= Integer.parseInt(headerStr.toString());
int bytesReaded = 1;
char dataByte = dataInputStream.readChar();
//we should read the number of characters indicated in the header
while (bytesReaded < header) {
data.append(dataByte);
dataByte = dataInputStream.readChar();
bytesReaded++;
}
For the first query, I guess you are using TCP. If you use UDP, then the server will read the packets in the order you want.
Can you be more clear/elaborative on the second query ?
I would try explicitly telling Connector.open to open up the stream as read_write. Then I would ensure that I flush my connections after each time I talked to the server.
SocketConnection connection = (SocketConnection) Connector.open(url, Connector.READ_WRITE);
OutputStream out = connection.openOutputStream();
// ... write to server
out.flush()
I got a solution to overcome to extract both the string
On sender device
Create a header which contains details of that data eg the data
length, datatype etc.
Add this header to the actual data and send it
On recipient device
read the header
retrieve the actual data length from the header
read the next data upto the data length as specified by the header