I have a device that sends data to my server via gprs . The problem is that it sends raw data and i don't know where i can stop the reading
Currently i am using something TIdHttpServer and something like this to read the strings :
var
s : string;
repeat
s:=s+acontext.Connection.Socket.ReadChar;
until acontext.Connection.Socket.InputBufferIsEmpty;
Is there a better solution to my problem ?
TCP is stream oriented. If the protocol is unknown, the server only can try to read into a byte array (if memory is not a problem) or a file stream. If the client disconnects normally, the data is 'complete'. Unfortunately, if the protocol is unknown, the server can not tell wether the client died or disconnected normally.
InputBufferIsEmpty does not help, as it only says if there is data in the (TCP) buffer - and depending on latency this can happen frequently, but it does not mean that there are no more in-flight bytes.
You could try to 'reverse engineer' the protocol, by sending known strings over the client devices. But if the sender is a black box, there can be many special cases - think of encoding or 'escape' characters etc.
You could make up you own protocol.
Some ideas are:
use a special character or characters combo to define the end of the
message.
append at the start of the message some fixed size field with the size of the message
Related
I am making a software for GSM Modem. It works on serial communication using AT commands. We give AT commands to it and it respond via serial communication. I am giving it a command to check balance in a SIM AT+CUSD=1,"*141#". Its response is like this:
+CUSD: 0, "Your balance is ... xxxxxxx "
Now I want to display this on a messagebox. This is the small code I am using:
String data = serialPort1.ReadExisting(); //to receive serial data and store it in data strig
logsTextBox.AppendText(data); // display it in text box
logsTextBox.AppendText("\n");
if (data.Contains("+CUSD:"))
{
MessageBox.Show(data);
}
Now when I put breakpoint and debug the code, it works properly and show complete data in message box but when I run it normally it shows just few characters in message box. Like this:
Instead it should be like this:
The problem what I have found is when debug all the data content which is shown in 2nd image gets save in data variable so it is displayed completely in message box. But when in normal run, the complete data is not received in string data so thats why it shows less data as shown in first image. How to solve this issue. What could be the reason. Please help.
This is a typical behavior for a serial port. They are very slower. When the DataReceived event fires, you'd typically only get one or two characters. Notably is that it works well when you debug because single-stepping through the code gives the lots of time to serial port to receive additional characters. But it will go Kaboom as soon as you run without a debugger because the string isn't long enough.
You'll need to modify the code by appending the string you receive to a string variable at class scope. Only parse the string after you've received all the characters you expected. You'll need some way to know that you've received the full response. Most typically serial devices will terminate the string with a special character. Often a line-feed.
If that's the case then you can make it easy by setting the SerialPort.NewLine property to that terminator and calling ReadLine() instead of ReadExisting().
You should call ReadExisting until empty string is returned, concatenating the results to data on each call. Perhaps debug mode has a larger read buffer for the serial port than normal mode.
I have in TCP/IP a client that send a list of strings with writeln (all at the same time).
How can the TCPserver know that it has read all the data?
In Onexecute event, I have used:
If (not AThread.Terminated)and(athread.Connection.Connected) then
Memo1.lines.add(AThread.Connection.readln);
But the problem, is that some last lines are not read.
You should send each string with its own WriteLn(), then you can either:
Send the number of strings before sending the actual strings. The server can then read the number first, then call ReadLn() however many times the number says.
Send a unique terminating line after the strings. The server can then keep calling ReadLn() until the terminator is read. If you need to send a string that could be ambiquious with the terminator, escape the string before sending it, and then have the server unescape it after reading it.
Indy has reading/writing methods for handling both scenarios, such as WriteStrings(), ReadStrings(), WriteRFCStrings(), Capture(), etc.
On an unrelated note, you should not be checking the AThread.Terminated and AThread.Connection.Connected properties. Let Indy raise an exception if you try to read from or write to a disconnected socket, and let the server handle the exception. Also, TIdTCPServer is multi-threaded, and accessing UI controls in the OnExecute event (or OnConnect, OnDisconnect, or OnException) is not thread-safe. You MUST synchronize with the main UI thread to access them safely.
I'm facing problem with TCpindy connection.readln method , I had no control in the other side sending data , when using Readln method in server side application hang (because receiving data don't contain carrige return ) , i'm trying readstring method but without success
Is there any suggestion to encouter this problem , me be looking for other component rather than indy ,
I need to get data from other client (tcp connection ) without any information about size of receiving data and without carriage return at the end of each frame.
You have to know how the data is being sent in order to read it properly. TCP is a byte stream, the sender needs to somehow indicate where one message ends and the next begins, either by:
prefixing each message with its
length
putting unique delimiters in between
each message
pausing in time between each message
Indy can handle all of these possibilities, but you need to identify which one is actually being used first.
Worse case scenerio, use the CurrentReadBuffer() method, which returns a String of whatever raw bytes are available at that moment.
I'm new to delphi and this is my first project.
Here's a little bit of code:
procedure TForm1.Button2Click(Sender: TObject);
responseStringFromServer:TStringStream;
begin
try
if IdTCPClient1.Connected then
begin
dataSentToDevice:= 'http/1.0 content-length: 344 content-type: text/xml <?xml version="1.0" encoding="UTF-8" ?> ...'
IdTCPClient1.IOHandler.WriteLn(dataSentToDevice);
responseStringFromServer := TStringStream.Create;
IdTCPClient1.IOHandler.ReadStream(responseStringFromServer);
...
I have a device connected to local network. I manage to connect to it succesfully. Sending commands is working too, but when i do
IdTCPClient1.IOHandler.ReadStream(responseStringFromServer);
then it waits til device is done processing and an exception occurs: "Connection Closed Gracefully". So i'm not able to read data the device is supposed to send me. Device is not shutting down. I've read other posts and i understand that device itself drops connection.
I have a demo program that communicates with it and it works fine. I need to get response xml before it drops the connection.
Also the request is http and i am using IdTCPClient (i need to use xml request, i don't know how to do it with TidHTTP).
May it be that after device is sending response it drops the connection, so that my tcpclient gets connection dropped flag before receiving data.
Any help would be appreciated!
Also the request is http and i am
using IdTCPClient (i need to use xml
request, i don't know how to do it
with TidHTTP).
Working with IdHTTP is simple...
Drop an instance of it on your form,
select it.
In Object Inspector, go to
ProtocolVersion property, and set it
to pv1_0, then open its
Request property set, and set
Request.ContentType to text/xml, and
Request.ContentEncoding to UTF-8,
and set other properties if
required.
Add a button to your form and
double-click on it.
In your code, create an instance of
TStringStream, and load your XML
content into it.
Assign your stream to
IdHttp.Request.Source.
Call IdHttp.Get() method by giving
it a host address.
IdHttp.Get() returns a string which
is the response the server sent you.
The way you are calling ReadStream(), it will interpret the first 4 bytes (or 8 bytes if the TIdIOHandler.LargeStream property is True) as an Integer (or Int64) in network-byte order that specifies the length of the data, and then it will try to read that many bytes. Is the device actually sending such a length value? If not, then ReadStream() will attempt to read the wrong number of bytes. An EIdConnClosedGracefully exception means the device is closing the socket on its end. So either the device is closing the connection immediately after sending its data, or it is timing out waiting for you to send the next command, which you cannot do since you are blocked waiting for the wrong data from the previous command.
As said by #Remy Lebeau and the documentation (press F1 when your edit caret is over the ReadStream sentence):
AByteCount indicates the number of bytes from the IOHandler to be read into AStream. When AByteCount contains -1 and AReadUntilDisconnect contains False, the byte count is read as an Integer value from the IOHandler. The size of AStream is adjusted to match the size expected from the IOHandler.
This is very useful when you're connecting to servers written also in INDY WriteStream in one end, ReadStream in the other, or any other language if you send that byte cound as expected by INDY client.
If you're reading from a device, be sure the device is sending that info up-front the stream or change the way you read the data. If you know the size of the stream, just pass the second parameter: AByteCount. If the device will close the channel when the stream ends, pass -1 as the second parameter and True as the third parameter: AReadUntilDisconnect.
If the device send text of unknown length but using a known terminator (like CR/LF), you better use ReadLn method of IOHandler to get that string.
Take a look at if neither fits your needs, take a look on Read* methods of TIOHandlerClass and RTFM for each to find the correct one to get the data sent by the device.
I am sending data to the server twice. First, I send "Hello world" and then I send "Server".
But the server received the data at 1 read. But the server have to read the data in a two-read operation.
Also, I write the data. Then read data from server and then I write the data.
In this case, the server can read the first data. But server can not read the second data.
The server uses read, write, read.
So how to overcome this issue? How do I write data to socket in BlackBerry?
What you describe is how TCP is supposed to work by default. What you are seeing is the Nagle algorithm (RFC 896) at work, reducing the number of outbound packets being sent so they are processed as efficiently as possible. You may be sending 2 packets in your code, but they are being transmitted together as 1 packet. Since TCP is a byte stream, the receiver should not be making any assumptions about how many packets it gets. You have to delimit your packet data in a higher-level protocol, and the receiver has to process data according to that protocol. It has to handle cases where multiple packets arrive in a single read, a single pakcet arriving in multiple reads, and everything in between, only processing packet data when they have been received in full, caching whatever is left over for subsequent reads to process when needed.
Hard to say without a little more detail, but it sounds like you're using 1-directional communication in the first case - i.e. the client writes, then writes again. There are any number of reasons that the server would receive the 2 writes as 1 read. Buffering on the client, somewhere in the wireless stack (or in the BES), buffering on the server side. All of those are legal with TCP/IP.
Without knowing anything more about your solution, have you thought about defining a small protocol - i.e. the client writes a known byte or bytes (like a 0 byte?) before sending the second write? Then the server can read, then recognize the delimiting byte, and say 'aha, this is now a different write from the client'?
As previously said this is an expected TCP behavior to save bandwidth. Note that to deliver your package TCP adds lot of data (e.g. destination port,sequence number, checksums...).
Instead of flushing the data I´ll recommend you to put more work in your protocol. For example you can define a header that contains the number of bytes to read and then the payload (the actual data).
The following code is a protocol encoded in an string with the structure [length];[data]
StringBuffer headerStr = new StringBuffer();
StringBuffer data = new StringBuffer();
//read header
char headerByte = dataInputStream.readChar();
while (headerByte != ';') {
headerStr.append(headerByte);
headerByte = dataInputStream.readChar();
}
//header has the number of character to read
int header= Integer.parseInt(headerStr.toString());
int bytesReaded = 1;
char dataByte = dataInputStream.readChar();
//we should read the number of characters indicated in the header
while (bytesReaded < header) {
data.append(dataByte);
dataByte = dataInputStream.readChar();
bytesReaded++;
}
For the first query, I guess you are using TCP. If you use UDP, then the server will read the packets in the order you want.
Can you be more clear/elaborative on the second query ?
I would try explicitly telling Connector.open to open up the stream as read_write. Then I would ensure that I flush my connections after each time I talked to the server.
SocketConnection connection = (SocketConnection) Connector.open(url, Connector.READ_WRITE);
OutputStream out = connection.openOutputStream();
// ... write to server
out.flush()
I got a solution to overcome to extract both the string
On sender device
Create a header which contains details of that data eg the data
length, datatype etc.
Add this header to the actual data and send it
On recipient device
read the header
retrieve the actual data length from the header
read the next data upto the data length as specified by the header