Serial Port Synchronization in Delphi - delphi

I am still having issues with the TComPort component but this time is not the component itself is the logic behind it. I have a device witch sends some ascii strings via serial port i need to prase those strings the problem is the computer reacts very fast so in the event char it captures only a part of the string the rest of the string comes back later... so parsing it when it is recived makes it impossible.
I was thinking in writing a timer witch verify if there was no serial activity 10 secons or more and then prase the string that i am saving into a buffer. But this method is unprofessional isn't there a idle event witch i can listen...Waiting for the best solution for my problem. Thanks.

After using a number of serial-port-components, I've got the best results until now, by using CreateFile('\\?\COM1',GENERIC_READ or GENERIC_WRITE,0,nil,OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,0), passing that handle to a THandleStream instance, and starting a dedicated thread to read from it. I know threads take a little more work than writing an event handler, but it still is the best way to handle any synchronization issues that arise from using serial ports.

Typical handler for OnRXChar event:
procedure XXX.RXChar(Sender: TObject; Count: Integer);
begin
ComPort.ReadStr(s, Count);
Accumulator := Accumulator + s;
if not AccumContainsPacketStart then
Accumulator := ''
else if AccumContainsPacketEndAfterStart then begin
ExtractFullStringFromAccum;
ParseIt;
end;
end;

Note.
Most com-port components do not have a clue when to report back to the owner. Normally the thread that is responsible to gather the bytes from the port is informed by the OS that one or more bytes are ready to be processed. This information is then simply popped up to your level. So when you expect the message to be transferred, you get what the OS is giving you.
You have to buffer all incoming characters in a global buffer. When you get the final character in your message string, handle the message.
Here is an example where the message start is identified with a special character and the end of the message is identified with another character.
If your message is constructed in another way, I'm sure you can figure out how to adapt the code.
var
finalBuf: AnsiString;
{- Checking message }
Function ParseAndCheckMessage(const parseS: AnsiString) : Integer;
begin
Result := 0; // Assume ok
{- Make tests to confirm a valid message }
...
end;
procedure TMainForm.ComPortRxChar(Sender: TObject; Count: Integer);
var
i,err: Integer;
strBuf: AnsiString;
begin
ComPort.ReadStr(strBuf, Count);
for i := 1 to Length(strBuf) do
case strBuf[i] of
'$' :
finalBuf := '$'; // Start of package
#10 :
begin
if (finalBuf <> '') and (finalBuf[1] = '$') then // Simple validate check
begin
SetLength( finalBuf, Length(finalBuf) - 1); // Strips CR
err := ParseAndCheckMessage(finalBuf);
if (err = 0) then
{- Handle validated string }
else
{- Handle error }
end;
finalBuf := '';
end;
else
finalBuf := finalBuf + strBuf[i];
end;
end;

If your protocol has begin/end markers, you can use TComDataPacket to provide you full packets, when they are available.

For certain amount of character we can use delay some miliseconds before ReadStr to make sure the data is completely sent. Example for 4 amount of character:
procedure TForm1.ComPort1RxChar(Sender: TObject; Count: Integer);
var
Str: String;
tegangan : real;
begin
sleep(100); //delay for 100ms
ComPort1.ReadStr(Str, 4);
...

Related

How to pass multilined TStrings data from a TIdTCPServer to TIdTCPClient

I tried to pass a database record from my server-side application to my client-side application. On the client-side I need to store my data into a TStrings collection.
When I pass a multiline field, I receive two separate data items at the client-side, instead of one multiline data item! I've also tried to do that with Unicode UTF8 based commands, but unfortunately the result is same.
Server-side code:
procedure TForm1.IdCmdTCPServer1CommandHandlers0Command(ASender: TIdCommand);
var
myData: TStrings;
begin
myData := TStringList.Create;
myData.Add('12'); // ID
myData.Add('This is a multi line' + #13#10 + 'description.'); // Descriptions
myData.Add('Thom Smith'); // Name
try
ASender.Context.Connection.Socket.Write(myData, True{, TIdTextEncoding.UTF8});
finally
myData.Free;
end;
end;
myData debug-time values on server-side are:
myData[0] = '12'
myData[1] = 'This is a multi line'#$D#$A'description.'
myData[2] = 'Thom Smith'
Client-side code:
procedure TForm1.Button1Click(Sender: TObject);
var
myData: TStrings;
begin
with TIdTCPClient.Create(nil) do
begin
Port := 1717;
Host := 'localhost';
try
Connect;
//IOHandler.DefStringEncoding := TIdTextEncoding.UTF8;
myData := TStringList.Create;
try
SendCmd('greating');
Socket.ReadStrings(myData, -1{, TIdTextEncoding.UTF8});
eID.Text := myData[0]; // ID TEdit
mDes.Text := myData[1]; // Descriptions TMemo
eTelNo.Text := myData[2]; // Name TEdit
finally
myData.Free;
end;
finally
Disconnect;
Free;
end;
end;
end;
myData debug-time valuese on client-side:
myData[0] = '12'
myData1 = 'This is a multi line'
myData[2] = 'description.'
Telnet result:
Actually, myData[2] that should keep 'Thom Smith' was replaced with the second line of the Description field! and there are no items after myData[2]. myData[3] is not accessible any more.
I think this issue is related to Indy's Write or ReadStrings procedures, because it sends ItemCount as 3, but it sends two items (one correct, and next beaked to two items!).
How can I pass a Carriage Return character to the other side without having the Write procedure break myData[1] into two separate lines?
Thanks a lot.
If you want TStrings.Text be oblivious to special characters - you should escape them before sending by net, and un-escape after that. There are a lot of ways of escaping, so choose one that suits you.
function EscapeString:(String): String --- your choice
function DeEscapeString:(String): String --- your choice
procedure SendEscapedStrings(const socket: TIdSocket; const data: TStrings);
var s: string; temp: TStringList;
begin
temp := TStringList.Create;
try
temp.Capacity := data.Count;
for s in data do
temp.Add( EscapeString( s ) );
socket.Write(temp);
finally
temp.Destroy;
end;
end;
procedure ReadDeescapedStrings(const socket: TIdSocket; const data: TStrings);
var s: string; temp: TStringList;
begin
temp := TStringList.Create;
try
Socket.ReadStrings(temp, -1);
data.Clear;
data.Capacity := temp.Count;
for s in temp do
temp.Add( DeEscapeString( s ) );
finally
temp.Destroy;
end;
end;
Now the question is what would you choose for DeEscapeString and EscapeString ? The options are many.
You can choose convert string to base64 before sending and from base64 after reading
You can choose UUEEncode for escapgin and UUEDecode for de-escaping
You can choose yEnc: http://en.wikipedia.org/wiki/YEnc
Or you can choose very simplistic functions StrStringToEscaped and StrEscapedToString from JclString unit of from Jedi CodeLib ( http://jcl.sf.net ):
what kind of escaping
If you ask for suggestion i would suggest not using raw TCP Server. There is well-known and standard HTTP protocol, there are many libraries for Delphi implementing both HTTP server and HTTP client. And in the protocol (and libraries) there are already decided things like ciphering, compressing, languages support, etc. And if somethign goes wrong - you can take any HTTP sniffer and see who is in the wrong- clent or server - with your own eyes. Debugging is much simpler.
If you are just starting, i suggest you looking into HTTP+JSON Synopse mORMot library, maybe it would cover your needs. You can take sample server code from http://robertocschneiders.wordpress.com/2012/11/22/datasnap-analysis-based-on-speed-stability-tests/ for example, or from demos in the lib.
Then, if to arrange around raw TCP server, i'd send compressed data, so it would work better (networks are slower than CPU usually). See http://docwiki.embarcadero.com/CodeExamples/XE5/en/ZLibCompressDecompress_(Delphi).
Sending:
1: Send into network (int32) - TStringList.Count
2: for every string doing
2.1 creating TStringStream from the string[i]
2.2 passing it via TZCompressionStream
2.3 sending (int32) size of compressed data
2.4 sending the data itself
2.5 freeing the temporary streams
Receiving
1: Receive from net (int32) - count of packets
1.1 ResultStringList.Clear; ResultStringList.Capacity := read_count.
2: for every string doing
2.1 creating TBytesStream
2.2 read from net (int32) size of compressed data
2.3 read N bytes from the network into BytesStream
2.4 unpack it via TZDecompressionStream into TStringStream
2.5 ResultStringList.Add( StringStream -> string );
2.6 freeing the temporary streams
Now, if you really don't want ot change almost anything, then JCL escaping would hopefully be enough for you. At least it worked for me, but my task was very different and was not about networks at all. But you can just test them all and see how it works for you.
Don't use the TStrings overload as it seems to use line breaks as separator between strings which does not work if your strings contain line breaks themselves.
You can easily write your own wrapper method to send a list of strings over the wire (take that as pseudocode):
procedure WriteStrings(IOHandler : TIdIOHandler; Strings : TStrings);
var
Str : String;
begin
IOHandler.WriteBufferOpen;
try
IOHandler.Write(Strings.Count);
for Str in Strings do
IOHandler.Write(Str);
finally
IOHandler.WriteBufferClose;
end;
end;
procedure ReadStrings(IOHandler : TIdIOHandler; Strings : TStrings);
var
Count, I : Integer;
begin
Count := IOHandler.ReadInteger;
for I := 1 to Count do
Strings.Add(IOHandler.ReadString);
end;

Delphi — TClientSocket ReceiveText MaxLen?

I've been trying to set the length of the amount of characters you recover from the ReceiveText TClientSocket function and nothing seems to be working. E.g., Receiving the first leftmost character(s) from the recovered data or otherwise data stream. Is there a way to accomplish this in Delphi using this specific object?
Help would be much appreciated. Thanks in advance.
ReceiveText doesn't have any means to control the maximum length of the received text.
The easiest way in ClientType := ctBlocking mode is to use a TWinSocketStream as the documentation states:
http://docwiki.embarcadero.com/VCL/XE2/en/ScktComp.TClientSocket.ClientType
When ClientType is ctBlocking, use a TWinSocketStream object for reading and writing. TWinSocketStream prevents the application from hanging indefinitely if a problem occurs while reading or writing. It also can wait for the socket connection to indicate its readiness for reading.
Example code:
var
Stream : TWinSocketStream;
Buffer : TBytes;
S : string;
begin
SetLength(Buffer, 100); // 100 bytes buffer size
Stream := TWinSocketStream.Create(Socket, 5000); // 5 seconds or 5000 milliseconds
try
Stream.ReadBuffer(Buffer[0], Length(Buffer)); // raises an Exception if it couldn't read the number of bytes requested
S := TEncoding.Default.GetString(Buffer); // Works in Delphi 2009+
finally
Stream.Free;
end;
end;
here is a little tipp for sending and receiving text
first you must send the length of yout text too
Socket.SendText(IntToStr(Length(text)) + seperator + text);
then you can check at your server socket on receiving data streams, if your incoming text is complete
procedure TMyServer.OnClientRead(Sender: TObject; Socket: TCustomWinSocket);
begin
if (xRecLength = 0) then begin
if Length(Socket.ReceiveText) <= 0 then EXIT;
xRecLength:= StrToIntDef(GetFirstFromSplitted(Socket.ReceiveText, seperator), -1);
if xRecLength = -1 then EXIT;
end;
xActLength:= xActLength + Length(Socket.ReceiveText);
xRecPuffer:= xRecPuffer + Socket.ReceiveText;
isComplete:= xActLength = xRecLength;
if isComplete then begin
// complete text received
end;
end;
hope that helps you...
I'm not at home with Delphi, but a quick Google search turned up this page that indicates that ReceiveText does not accept any parameters, but instead returns a string of as much as it can read.
What you might need is might be ReceiveBuf instead.

Stream read error

I'm getting this error message under heavy load. Here is code abstract and message from my error log.
I tried everything I could think of. Any suggestion would be greatly appreciated.
Procedure tCacheInMemory.StreamValue(Name: String; IgnoreCase: Boolean; Var Stream: TStringStream);
Var
i: Integer;
Begin
i := 0;
Try
If Not active Then
exit;
arrayLock.BeginRead;
Try
i := Search(Name);
If i > -1 Then Begin
If fItems[i].value = Nil Then
exit;
fItems[i].value.Position := 0;
Stream.Position := 0;
Stream.CopyFrom(fItems[i].value, fItems[i].value.Size);
End;
Finally
arrayLock.EndRead;
End;
Except { ...execution jumps to here }
On E: Exception Do Begin
x.xLogError('LogErrorCacheInMemory.txt', 'StreamValue:' + E.Message + ' ItemsCount:' + IntToStr( High(fItems)) + 'Memory:' + IntToStr(x.GetMemoryInfoMemory) + endLn + 'StreamSize : ' + IntToStr(fItems[i].value.Size) + ' i=' + IntToStr(i) + 'Name: ' + Name);
Clear;
End
End;
End;
Log Entries:
3/10/2011 10:52:59 AM: StreamValue:Stream read error ItemsCount:7562 Memory:240816
StreamSize : 43 i=7506 Name: \\xxxxxxxx\WebRoot\\images\1x1.gif
3/10/2011 12:39:14 PM: StreamValue:Stream read error ItemsCount:10172 Memory:345808
StreamSize : 849 i=10108 Name: \\xxxxxxxx\WebRoot\\css\screen.add.css
3/10/2011 3:45:29 PM: StreamValue:Stream read error ItemsCount:11200 Memory:425464
StreamSize : 3743 i=11198 Name: \\xxxxxxxx\WebRoot\\JS\ArtWeb.js
P.S.
arrayLock: TMultiReadExclusiveWriteSynchronizer;
fItems: Array Of rCache;
Type
rCache = Record
Name: String;
value: TStringStream;
expired: TDateTime;
End;
And calling function:
Function tCacheInMemory.CacheCheck(cName: String; Out BlobStream: TStringStream): Boolean;
Begin
Result := False;
If Not IfUseCache Then
exit;
BlobStream.SetSize(0);
BlobStream.Size := 0;
StreamValue(trim(cName), True, BlobStream);
If BlobStream.Size > 0 Then
Result := True;
End;
`
You're not using correct locking. You're acquiring a read lock on the array of cache entries, but once you find the item you want, you modify it. First, you explicitly modify it by assigning its Position property, and then you implicitly modify it by reading from it, which modifies its Position property again. When other code attempts to read from that same cache item, you'll have interference. If the source stream's Position property changes between the time the destination stream calculates how many bytes are available and the time it actually requests to read those bytes, you'll get a stream-read error.
I have a couple pieces of advice related to this:
Don't use streams as a storage device in the first place. You're apparently holding the contents of files. You're not going to change those, so you don't need a data structure designed for making sequential changes. Instead, just store the data in simple arrays of bytes: TBytes. (Also, use of TStringStream in particular introduces confusion over whether those strings' encodings are important. A simple file cache shouldn't be concerned with string encodings at all. If you must use a stream, use a content-agnostic class like TMemoryStream.)
Don't quell an exception that you haven't actually handled. In this code, you're catching all exception types, logging some information, clearing the cache, and then proceeding as though everything is normal. But you haven't done anything to resolve the problem that triggered the exception, so everything is not normal. Since you're not really handling the exception, you need to make sure it propagates to the caller. Call raise after to call Clear. (And when you log the exception, make sure you log the exception's ClassName value as well as its message.)
It looks like something external is blocking your stream files.
You could try to use Process Monitor to see what blocks it.
Another thing you can try is to open the stream in read-deny-write mode (please show us how you open the stream).
Something like this:
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite) ;
Edit 1: Disregard the strike through part: you are using TStringStream.
I'll keep the answer just in case anyone ever gets this kind of error when using TFileStream.
Edit 2: Yuriy posted this interesting addendum, but I'm not sure it will work, as the BlobStream is not initialized, just like Robert Love suspected:
Function TCacheInMemory.CacheCheck(cName: String; Out BlobStream: TStringStream): Boolean;
Begin
Result := False;
Try
If Not IfUseCache Then
exit;
BlobStream.SetSize(0);
BlobStream.Size := 0;
StreamValue(trim(cName), True, BlobStream);
If BlobStream.Size > 0 Then
Result := True;
Except
On E: Exception Do
Begin
x.xLogError('LogErrorCacheInMemory.txt', 'CheckCacheOutStream:' + E.Message + ' ItemsCount:' + IntToStr( High(fItems)) + 'Memory:' + IntToStr(x.GetMemoryInfoMemory));
End;
End;
End;
--jeroen

Improve speed of own debug visualizer for Delphi 2010

I wrote Delphi debug visualizer for TDataSet to display values of current row, source + screenshot: http://delphi.netcode.cz/text/tdataset-debug-visualizer.aspx . Working good, but very slow. I did some optimalization (how to get fieldnames) but still for only 20 fields takes 10 seconds to show - very bad.
Main problem seems to be slow IOTAThread90.Evaluate used by main code shown below, this procedure cost most of time, line with ** about 80% time. FExpression is name of TDataset in code.
procedure TDataSetViewerFrame.mFillData;
var
iCount: Integer;
I: Integer;
// sw: TStopwatch;
s: string;
begin
// sw := TStopwatch.StartNew;
iCount := StrToIntDef(Evaluate(FExpression+'.Fields.Count'), 0);
for I := 0 to iCount - 1 do
begin
s:= s + Format('%s.Fields[%d].FieldName+'',''+', [FExpression, I]);
// FFields.Add(Evaluate(Format('%s.Fields[%d].FieldName', [FExpression, I])));
FValues.Add(Evaluate(Format('%s.Fields[%d].Value', [FExpression, I]))); //**
end;
if s<> '' then
Delete(s, length(s)-4, 5);
s := Evaluate(s);
s:= Copy(s, 2, Length(s) -2);
FFields.CommaText := s;
{ sw.Stop;
s := sw.Elapsed;
Application.MessageBox(Pchar(s), '');}
end;
Now I have no idea how to improve performance.
That Evaluate needs to do a surprising amount of work. The compiler needs to compile it, resolving symbols to memory addresses, while evaluating properties may cause functions to be called, which needs the debugger to copy the arguments across into the debugee, set up a stack frame, invoke the function to be called, collect the results - and this involves pausing and resuming the debugee.
I can only suggest trying to pack more work into the Evaluate call. I'm not 100% sure how the interaction between the debugger and the evaluator (which is part of the compiler) works for these visualizers, but batching up as much work as possible may help. Try building up a more complicated expression before calling Evaluate after the loop. You may need to use some escaping or delimiting convention to unpack the results. For example, imagine what an expression that built the list of field values and returned them as a comma separated string would look like - but you would need to escape commas in the values themselves.
Because Delphi is a different process than your debugged exe, you cannot direct use the memory pointers of your exe, so you need to use ".Evaluate" for everything.
You can use 2 different approaches:
Add special debug dump function into executable, which does all value retrieving in one call
Inject special dll into exe with does the same as 1 (more hacking etc)
I got option 1 working, 2 should also be possible but a little bit more complicated and "ugly" because of hacking tactics...
With code below (just add to dpr) you can use:
Result := 'Dump=' + Evaluate('TObjectDumper.SpecialDump(' + FExpression + ')');
Demo code of option 1, change it for your TDataset (maybe make CSV string of all values?):
unit Unit1;
interface
type
TObjectDumper = class
public
class function SpecialDump(aObj: TObject): string;
end;
implementation
class function TObjectDumper.SpecialDump(aObj: TObject): string;
begin
Result := '';
if aObj <> nil then
Result := 'Special dump: ' + aObj.Classname;
end;
initialization
//dummy call, just to ensure it is linked c.q. used by compiler
TObjectDumper.SpecialDump(nil);
end.
Edit: in case someone is interested: I got option 2 working too (bpl injection)
I have not had a chance to play with the debug visualizers yet, so I do not know if this work, but have you tried using Evaluate() to convert FExpression into its actual memory address? If you can do that, then type-cast that memory address to a TDataSet pointer and use its properties normally without going through additional Evaluate() calls. For example:
procedure TDataSetViewerFrame.mFillData;
var
DS: TDataSet;
I: Integer;
// sw: TStopwatch;
begin
// sw := TStopwatch.StartNew;
DS := TDataSet(StrToInt(Evaluate(FExpression)); // this line may need tweaking
for I := 0 to DS.Fields.Count - 1 do
begin
with DS.Fields[I] do begin
FFields.Add(FieldName);
FValues.Add(VarToStr(Value));
end;
end;
{
sw.Stop;
s := sw.Elapsed;
Application.MessageBox(Pchar(s), '');
}
end;

Why do I get an AccessViolation in Indy Sockets 9 IdTcpServer ServerExecute?

First Question:
Is the following routine a correct implementation of an Indy 9 IdTcpServer.OnExecute routine?
procedure TMyConnServer.ServerExecute(AContext: TIdPeerThread);
var
buffSize: integer;
str: string;
begin
AContext.Connection.ReadFromStack(True, 600, False);
buffSize := AContext.Connection.InputBuffer.Size;
if (buffSize > 0) then
{ Extract input buffer as string }
str := AContext.Connection.ReadString(buffSize);
{ Notify connection object of received data }
if (AContext.Data <> nil) then
begin
TConnectionHandler(AContext.Data).Read(str);
end;
end;
end;
Second (actually more important) Question:
Now there is occasionally an access violation (read from address 000000). Obviously in the line:
AContext.Connection.ReadFromStack(True, 600, False);
but checking if AContext / Connection / InputBuffer / IOHandler = nil BEFORE is false.
AFTER the call (and after the exception was raised) the IOHandler is nil.
We are using RAD Studio / Delphi 2007.
The only way the IOHandler can become nil like you describe is if another thread in your app called Disconnect() on the connection while your worker thread was still running.
Well, the simplest onExecute handler I have is like this. (excuse C++ instead of Delphi, but you'll get the idea.
void __fastcall MyPanel::DoTCPExecute(TIdPeerThread * AThread)
{
AnsiString text =AThread->Connection->ReadLn();
// now do something with text
}
The only obvious issue I can see is that you're trying to use the "timing" of data to determine when you have a complete string. This is a real no-no with TCP. You might just have the first byte of a string, or you might have several strings all sent at once. With TCP there's no guarantee that each "send" ends up as single "receive" at the other end.
You need to "delimit" your string in some other way. Readln uses the newline character as a terminator - another approach is to prefix each chunk of data with a length field. You read the length, then read the remaining data.
The code works like that, but I don't think it's a clean option:
if (AContext.Connection.Connected) then
begin
try
AContext.Connection.ReadFromStack(false, 1, false);
except on E: EAccessViolation do
// ignore
end;
end;
buffSize := AContext.Connection.InputBuffer.Size;

Resources