I know, I'm asking a lot of questions...but as a new delphi developer I keep falling over all these questions :)
This one deals with TCP communication using indy 10. To make communication efficient, I code a client operation request as a single byte (in most scenarios followed by other data bytes of course, but in this case only one single byte). Problem is that
var Bytes : TBytes;
...
SetLength (Bytes, 1);
Bytes [0] := OpCode;
FConnection.IOHandler.Write (Bytes, 1);
ErrorCode := Connection.IOHandler.ReadByte;
does not send that byte immediately (at least the servers execute handler is not invoked). If I change the '1' to a '9' for example everything works fine. I assumed that Indy buffers the outgoing bytes and tried to disable write buffering with
FConnection.IOHandler.WriteBufferClose;
but it did not help. How can I send a single byte and make sure that it is immediatly sent? And - I add another little question here - what is the best way to send an integer using indy? Unfortunately I can't find function like WriteInteger in the IOHandler of TIdTCPServer...and
WriteLn (IntToStr (SomeIntVal))
seems not very efficient to me. Does it make a difference whether I use multiple write commands in a row or pack things together in a byte array and send that once?
Thanks for any answers!
EDIT: I added a hint that I'm using Indy 10 since there seem to be major changes concerning the read and write procedures.
Write buffering is disabled by default. You can check write buffering to see if it's active in your code by testing the fConnection.IOHandler.WriteBufferingActive property.
As far as the best way to send an integer... 'it depends' on your protocol and overall goals. Specifically, use FConnection.IOHandler.Write() as there are overloaded methods to write just about any type of data, including an integer.
Taken from IdIOHandler:
// Optimal Extra Methods
//
// These methods are based on the core methods. While they can be
// overridden, they are so simple that it is rare a more optimal method can
// be implemented. Because of this they are not overrideable.
//
//
// Write Methods
//
// Only the ones that have a hope of being better optimized in descendants
// have been marked virtual
procedure Write(const AOut: string; const AEncoding: TIdEncoding = enDefault); overload; virtual;
procedure WriteLn(const AEncoding: TIdEncoding = enDefault); overload;
procedure WriteLn(const AOut: string; const AEncoding: TIdEncoding = enDefault); overload; virtual;
procedure WriteLnRFC(const AOut: string = ''; const AEncoding: TIdEncoding = enDefault); virtual;
procedure Write(AValue: TStrings; AWriteLinesCount: Boolean = False; const AEncoding: TIdEncoding = enDefault); overload; virtual;
procedure Write(AValue: Byte); overload;
procedure Write(AValue: Char; const AEncoding: TIdEncoding = enDefault); overload;
procedure Write(AValue: LongWord; AConvert: Boolean = True); overload;
procedure Write(AValue: LongInt; AConvert: Boolean = True); overload;
procedure Write(AValue: SmallInt; AConvert: Boolean = True); overload;
procedure Write(AValue: Int64; AConvert: Boolean = True); overload;
procedure Write(AStream: TStream; ASize: Int64 = 0; AWriteByteCount: Boolean = False); overload; virtual;
Another question you had was "Does it make a difference whether I use multiple write commands in a row or pack things together in a byte array and send that once?" For the majority of cases, yes it makes a difference. For highly stressed servers you are going to have to get more involved in how bytes are sent back and forth, but at this level you should abstract out your sends into a separate protocol type class that builds the data to be sent and sends it in a burst and have a receiving protocol that receives a bunch of data and processes it as a complete unit instead of breaking things down to sending/receiving an integer, character, byte array, etc..
As a very rough quick example:
TmyCommand = class(TmyProtocol)
private
fCommand:Integer;
fParameter:String;
fDestinationID:String;
fSourceID:String;
fWhatever:Integer;
public
property Command:Integer read fCommand write fCommand;
...
function Serialize;
procedure Deserialize(Packet:String);
end;
function TmyCommand.Serialize:String;
begin
//you'll need to delimit these to break them apart on the other side
result := AddItem(Command) +
AddItem(Parameter) +
AddItem(DestinationID) +
AddItem(SourceID) +
AddItem(Whatever);
end;
procedure TMyCommand.Deserialize(Packet:String);
begin
Command := StrToInt(StripOutItem(Packet));
Parameter := StripOutItem(Packet);
DesintationID := StripOutItem(Packet);
SourceID := StripOutItem(Packet);
Whatever := StrToInt(StripOutItem(Packet));
end;
Then send this via:
FConnection.IOHandler.Write(myCommand.Serialize());
On the other side you can receive the data via Indy and then
myCommand.Deserialize(ReceivedData);
I'm not familiar with Indy, but you might want to look around its API for a TCP_NODELAY option (you might want to grep the Indy source tree for something like that - case insensitive for "delay" should do it.)
Edit: Rob Kennedy pointed out that the property I was referring to is TIdIOHandlerSocket.UseNagle - thanks!
The problem is inherent in the nature of TCP. TCP does guarantee data delivery in the same order as it was emitted but does not guarantee message boundaries. In other words, the operating system of the source, of the target, and any routers along the way are free to coalesce packets from the connection or to fragment them at will. You must look at a TCP transmission as a stream, not as a series of individual packets. Thus you will have to implement a mechanism by which you either delimit the individual messages (by a magic byte, for example, which you must escape if it can also occur in your message data), or you could send the length of the following message first, then the actual message.
I've always used UDP coupled with a naive ACK/retransmission scheme when I needed to send messages where the message boundary was important, such as is your case. Might want to take that into account. UDP is much better suited for command messages.
Sounds like you have to flush your buffer. Try this:
TIdTCPConnection.FlushWriteBuffer;
If you don't want a write buffer, use this:
TIdTCPConnection.CancelWriteBuffer;
According to the help, this first calls ClearWriteBuffer, to clear the buffer and then calls CloseWriteBuffer.
The best way to send an integer (using Indy 10) is using TIdIOHandler.Write (according to Indy 10 help Write is overloaded to handle different kinds of data, including Integers)
Related
I try to write a ping thread searching for connected devices to a local network. I need to avoid indy because it works on an administration account in windows.
I translated c++ example of msdn. Should I use IcmpCloseHandle? If so why should I close the handle which is just a cardinal variable. If not so what's the method for?
function IcmpSendEcho(ICMPHandle: Cardinal; DestinationAddress: Integer; RequestData: Pointer; RequestSize: Word; RequestOptions: Pointer; ReplyBuffer: Pointer; ReplySize: Cardinal; TimeOut: Cardinal): Cardinal; stdcall; external 'icmp.dll';
function TForm1.Ping(IPAddress: string; TimeOut: Integer): Integer;
var
ICMPHandle: Cardinal;
DestinationAddress: Integer;
ReplyBuffer: pICMPEchoReply;
RequestData: array[0..31] of AnsiChar;
ReplySize: Cardinal;
begin
ICMPHandle := IcmpCreateFile;
DestinationAddress := inetaddr(pansichar(AnsiString(IPAddress)));
RequestData := 'data buffer';
ReplySize := SizeOf(ticmpechoreply) + SizeOf(RequestData);
ReplyBuffer := AllocMem(ReplySize);
IcmpSendEcho(ICMPHandle, DestinationAddress, #RequestData, SizeOf(requestdata), nil, ReplyBuffer, ReplySize, TimeOut);
Result := replybuffer.Status;
FreeMem(ReplyBuffer);
end;
The documentation for IcmpCreateFile is rather weak. It should make the point that an handle successfully returned by IcmpCreateFile should be closed with a call to IcmpCloseHandle when it is no longer needed. So yes, you do need a call to IcmpCloseHandle.
You are mistaken in thinking that IcmpCreateFile returns a Cardinal. A Cardinal is a 32 bit type, but IcmpCreateFile returns HANDLE which is a pointer sized type. Your code will fail when compiled on 64 bit. Some of the other types that you use in your IcmpSendEcho header translation look dubious. If I were you I would either bone up on how to translate Win32 header files, and find and use trustworthy header translations.
I would also like to stress that your code makes no attempt whatsoever to handle errors. You are playing with fire here. We see this so many times here on SO, and I am sure I have personally told you this very thing on a number of early occasions. So, read the documentation for each API function that you call, and handle errors appropriately.
Unless you are an expert in Win32 API header translations and use, then you are likely to produce code that has defects. It might be prudent to use a third party header translation from, for instance, JEDI.
I tried to pass a database record from my server-side application to my client-side application. On the client-side I need to store my data into a TStrings collection.
When I pass a multiline field, I receive two separate data items at the client-side, instead of one multiline data item! I've also tried to do that with Unicode UTF8 based commands, but unfortunately the result is same.
Server-side code:
procedure TForm1.IdCmdTCPServer1CommandHandlers0Command(ASender: TIdCommand);
var
myData: TStrings;
begin
myData := TStringList.Create;
myData.Add('12'); // ID
myData.Add('This is a multi line' + #13#10 + 'description.'); // Descriptions
myData.Add('Thom Smith'); // Name
try
ASender.Context.Connection.Socket.Write(myData, True{, TIdTextEncoding.UTF8});
finally
myData.Free;
end;
end;
myData debug-time values on server-side are:
myData[0] = '12'
myData[1] = 'This is a multi line'#$D#$A'description.'
myData[2] = 'Thom Smith'
Client-side code:
procedure TForm1.Button1Click(Sender: TObject);
var
myData: TStrings;
begin
with TIdTCPClient.Create(nil) do
begin
Port := 1717;
Host := 'localhost';
try
Connect;
//IOHandler.DefStringEncoding := TIdTextEncoding.UTF8;
myData := TStringList.Create;
try
SendCmd('greating');
Socket.ReadStrings(myData, -1{, TIdTextEncoding.UTF8});
eID.Text := myData[0]; // ID TEdit
mDes.Text := myData[1]; // Descriptions TMemo
eTelNo.Text := myData[2]; // Name TEdit
finally
myData.Free;
end;
finally
Disconnect;
Free;
end;
end;
end;
myData debug-time valuese on client-side:
myData[0] = '12'
myData1 = 'This is a multi line'
myData[2] = 'description.'
Telnet result:
Actually, myData[2] that should keep 'Thom Smith' was replaced with the second line of the Description field! and there are no items after myData[2]. myData[3] is not accessible any more.
I think this issue is related to Indy's Write or ReadStrings procedures, because it sends ItemCount as 3, but it sends two items (one correct, and next beaked to two items!).
How can I pass a Carriage Return character to the other side without having the Write procedure break myData[1] into two separate lines?
Thanks a lot.
If you want TStrings.Text be oblivious to special characters - you should escape them before sending by net, and un-escape after that. There are a lot of ways of escaping, so choose one that suits you.
function EscapeString:(String): String --- your choice
function DeEscapeString:(String): String --- your choice
procedure SendEscapedStrings(const socket: TIdSocket; const data: TStrings);
var s: string; temp: TStringList;
begin
temp := TStringList.Create;
try
temp.Capacity := data.Count;
for s in data do
temp.Add( EscapeString( s ) );
socket.Write(temp);
finally
temp.Destroy;
end;
end;
procedure ReadDeescapedStrings(const socket: TIdSocket; const data: TStrings);
var s: string; temp: TStringList;
begin
temp := TStringList.Create;
try
Socket.ReadStrings(temp, -1);
data.Clear;
data.Capacity := temp.Count;
for s in temp do
temp.Add( DeEscapeString( s ) );
finally
temp.Destroy;
end;
end;
Now the question is what would you choose for DeEscapeString and EscapeString ? The options are many.
You can choose convert string to base64 before sending and from base64 after reading
You can choose UUEEncode for escapgin and UUEDecode for de-escaping
You can choose yEnc: http://en.wikipedia.org/wiki/YEnc
Or you can choose very simplistic functions StrStringToEscaped and StrEscapedToString from JclString unit of from Jedi CodeLib ( http://jcl.sf.net ):
what kind of escaping
If you ask for suggestion i would suggest not using raw TCP Server. There is well-known and standard HTTP protocol, there are many libraries for Delphi implementing both HTTP server and HTTP client. And in the protocol (and libraries) there are already decided things like ciphering, compressing, languages support, etc. And if somethign goes wrong - you can take any HTTP sniffer and see who is in the wrong- clent or server - with your own eyes. Debugging is much simpler.
If you are just starting, i suggest you looking into HTTP+JSON Synopse mORMot library, maybe it would cover your needs. You can take sample server code from http://robertocschneiders.wordpress.com/2012/11/22/datasnap-analysis-based-on-speed-stability-tests/ for example, or from demos in the lib.
Then, if to arrange around raw TCP server, i'd send compressed data, so it would work better (networks are slower than CPU usually). See http://docwiki.embarcadero.com/CodeExamples/XE5/en/ZLibCompressDecompress_(Delphi).
Sending:
1: Send into network (int32) - TStringList.Count
2: for every string doing
2.1 creating TStringStream from the string[i]
2.2 passing it via TZCompressionStream
2.3 sending (int32) size of compressed data
2.4 sending the data itself
2.5 freeing the temporary streams
Receiving
1: Receive from net (int32) - count of packets
1.1 ResultStringList.Clear; ResultStringList.Capacity := read_count.
2: for every string doing
2.1 creating TBytesStream
2.2 read from net (int32) size of compressed data
2.3 read N bytes from the network into BytesStream
2.4 unpack it via TZDecompressionStream into TStringStream
2.5 ResultStringList.Add( StringStream -> string );
2.6 freeing the temporary streams
Now, if you really don't want ot change almost anything, then JCL escaping would hopefully be enough for you. At least it worked for me, but my task was very different and was not about networks at all. But you can just test them all and see how it works for you.
Don't use the TStrings overload as it seems to use line breaks as separator between strings which does not work if your strings contain line breaks themselves.
You can easily write your own wrapper method to send a list of strings over the wire (take that as pseudocode):
procedure WriteStrings(IOHandler : TIdIOHandler; Strings : TStrings);
var
Str : String;
begin
IOHandler.WriteBufferOpen;
try
IOHandler.Write(Strings.Count);
for Str in Strings do
IOHandler.Write(Str);
finally
IOHandler.WriteBufferClose;
end;
end;
procedure ReadStrings(IOHandler : TIdIOHandler; Strings : TStrings);
var
Count, I : Integer;
begin
Count := IOHandler.ReadInteger;
for I := 1 to Count do
Strings.Add(IOHandler.ReadString);
end;
I am still having issues with the TComPort component but this time is not the component itself is the logic behind it. I have a device witch sends some ascii strings via serial port i need to prase those strings the problem is the computer reacts very fast so in the event char it captures only a part of the string the rest of the string comes back later... so parsing it when it is recived makes it impossible.
I was thinking in writing a timer witch verify if there was no serial activity 10 secons or more and then prase the string that i am saving into a buffer. But this method is unprofessional isn't there a idle event witch i can listen...Waiting for the best solution for my problem. Thanks.
After using a number of serial-port-components, I've got the best results until now, by using CreateFile('\\?\COM1',GENERIC_READ or GENERIC_WRITE,0,nil,OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,0), passing that handle to a THandleStream instance, and starting a dedicated thread to read from it. I know threads take a little more work than writing an event handler, but it still is the best way to handle any synchronization issues that arise from using serial ports.
Typical handler for OnRXChar event:
procedure XXX.RXChar(Sender: TObject; Count: Integer);
begin
ComPort.ReadStr(s, Count);
Accumulator := Accumulator + s;
if not AccumContainsPacketStart then
Accumulator := ''
else if AccumContainsPacketEndAfterStart then begin
ExtractFullStringFromAccum;
ParseIt;
end;
end;
Note.
Most com-port components do not have a clue when to report back to the owner. Normally the thread that is responsible to gather the bytes from the port is informed by the OS that one or more bytes are ready to be processed. This information is then simply popped up to your level. So when you expect the message to be transferred, you get what the OS is giving you.
You have to buffer all incoming characters in a global buffer. When you get the final character in your message string, handle the message.
Here is an example where the message start is identified with a special character and the end of the message is identified with another character.
If your message is constructed in another way, I'm sure you can figure out how to adapt the code.
var
finalBuf: AnsiString;
{- Checking message }
Function ParseAndCheckMessage(const parseS: AnsiString) : Integer;
begin
Result := 0; // Assume ok
{- Make tests to confirm a valid message }
...
end;
procedure TMainForm.ComPortRxChar(Sender: TObject; Count: Integer);
var
i,err: Integer;
strBuf: AnsiString;
begin
ComPort.ReadStr(strBuf, Count);
for i := 1 to Length(strBuf) do
case strBuf[i] of
'$' :
finalBuf := '$'; // Start of package
#10 :
begin
if (finalBuf <> '') and (finalBuf[1] = '$') then // Simple validate check
begin
SetLength( finalBuf, Length(finalBuf) - 1); // Strips CR
err := ParseAndCheckMessage(finalBuf);
if (err = 0) then
{- Handle validated string }
else
{- Handle error }
end;
finalBuf := '';
end;
else
finalBuf := finalBuf + strBuf[i];
end;
end;
If your protocol has begin/end markers, you can use TComDataPacket to provide you full packets, when they are available.
For certain amount of character we can use delay some miliseconds before ReadStr to make sure the data is completely sent. Example for 4 amount of character:
procedure TForm1.ComPort1RxChar(Sender: TObject; Count: Integer);
var
Str: String;
tegangan : real;
begin
sleep(100); //delay for 100ms
ComPort1.ReadStr(Str, 4);
...
I am developing application which intend to be cross platform. I used to use Windows Messages but now I am dropping it out. I replaced messages with callbacks but regardless I can use different technologies I am not aware of different possibilites when not using windows messages.
Well I have main exe aplication and some dll plugins. I have some objects and threads in dll and I would like to notify main application about some changes that DLL made to data structure.
As I said I am currently working with some callbacks. To provide compatibility with different languages (C++, VB, C#) I have non-object type of callback. I am not sure if other languages supports callback of object.
So my questions are:
What are the alternatives (cross-platform) to windows messages? Can callbacks replace messages?
Do other languages support callback of object?
I guess other languages have different technologies as alternative to messages?
You can certainly use callback functions instead of messages. You can't use callback methods because only Delphi and C++ Builder understand how to invoke Delphi method pointers. However, you can use callback objects with any language that supports COM. Here's an example for a plug-in to notify the application that the data structure has changed:
Define an interface.
type
IDataStructureChanged = interface
['{GUID}']
procedure Call; stdcall;
end;
You could add some parameters to the method so the plug-in can tell how the data structure changed, or pass some value indicating which plug-in is making the notification.
Implement it in the application.
type
TDataStructureChangedListener = class(TInterfacedObject, IDataStructureChanged)
private
FForm: TForm;
procedure Call; stdcall;
public
constructor Create(Form: TForm);
end;
When you instantiate that class, you can pass it a reference to your program's main form, or whatever other information your program will need to be able to take action when a plug-in eventually calls the Call method. Implement Call to make your application do whatever it needs to do when a data structure changes.
Pass a reference to each of the plug-ins when you initialize them.
ChangeListener := TDataStructureChangedListener.Create(Self);
for i := 0 to Pred(PlugIns.Count) do
PlugIns[i].Init(ChangeListener);
The plug-in should store a reference to the listener object, and when the data structure changes, it can call the Call method to notify your application.
What I've described here is what's generally known as an event sink. You can have more than one in your program. If there are multiple events to handle, you could have a separate interface for each kind of event, or you could group them all into a single interface and have a different method for each event. You could have a different sink object for each plug-in, or you could give each plug-in a reference to the same sink object, and then pass a plug-in-ID parameter.
I would definately use callbacks. The main app could give a callback function to the DLL to call when needed, and then the callback function itself can send window messages to the app if it needs to.
I agree with Remy, (!). A straightforward callback allows the handler to implement any kind of further communication it chooses - it might post a message, it may push a parameter onto a queue, whatever it wants. If you want to be cross-platform, you are going to have to resort to passing in, and out, simple types. It's usual to pass in a 'user context' pointer when callbacks are set up. The callback passes this pointer into the handler. This allows callers to pass in a context object as a pointer/int and to recover it in the handler, (by casting the pointer/int back to an object). The handler can then call methods on the context, no matter whether it's Delphi, C++ etc.
So my questions are:
What are the alternatives (cross-platform) to windows messages? Can callbacks replace messages?
Yes you can replace messages with callbacks.
Do other languages support callback of object?
You shouldn't use object methods as callbacks. Common practice in portable code is use of handles (notify calling convention):
DLL source:
type
THandle = LongWord;
{$IF SizeOf(THandle) < SizeOf(Pointer))}
{$MESSAGE Error 'Invallid handle type'}
{$ENDIF}
TCallback = procedure(const aHandle: THandle); cdecl;
var
gCallback: record
Routine: TCallback;
Obj: TObject;
Info: string
end;
function Object2Handle(const aObj: TObject): THandle;
begin
Result:= THandle(Pointer(aObj))
end;
function Handle2Object(const aHandle: THandle; out aObj: TObject): Boolean;
begin
if gCallback.Obj <> nil then
if aHandle = Object2Handle(gCallback.Obj) then
begin
aObj:= gCallback.Obj;
Result:= true;
Exit // WARRNING: program flow disorder
end;
aObj:= nil;
Result:= false
end;
procedure DoCallback();
begin
if Assigned(gCallback.Routine) then
gCallback.Routine(Object2Handle(gCallback.Obj))
end;
procedure SetupCallback(const aCallback: TCallback); cdecl;
begin
gCallback.Routine:= aCallback;
end;
procedure DoSomething(const aHandle: THandle; out aInfo: string); cdecl;
var
O: TObject;
begin
if Handle2Object(aHandle, O) then
aInfo:= Format('%s class object %s', [O.ClassName(), gCallback.Info])
end;
procedure Test();
begin
gCallback.Obj:= TStream.Create();
try
gCallback.Info:= 'created';
DoCallback();
finally
FreeAndNil(gCallback.Obj)
end;
gCallback.Obj:= TMemoryStream.Create();
try
gCallback.Info:= 'will be freed';
DoCallback();
finally
FreeAndNil(gCallback.Obj)
end
end;
exports
SetupCallback,
DoSomething,
Test;
Executable source:
procedure Cb(const aHandle: THandle); cdecl;
const
STUPID: THandle = 1;
EQUALLY_STUPID = $DEAD;
var
S: string;
begin
DoSomething(STUPID, S);
DoSomething(aHandle, S);
DoSomething(EQUALLY_STUPID, S)
end;
begin
SetupCallback(#Cb);
Test()
end.
Edited: You can't shoot yourself in you leg now.
I guess other languages have different technologies as alternative to messages?
OS have a few message alternatives. However not many truly portable.
You can also use:
sockets,
(IMO too big in this case?) ready messaging system (my favorite 0MQ)
I'm again in a situation where I've spend an obscene amount of time on trying to customize datasnap callback samples to my needs.
I'm old school OOP programmer and have several very large Object hierakies in my "toolbox" PODO style :-) .. and having this great datasnap feature, I want to utilize the forces of the callback.
BUT - when I implement it ... it simply fails ... (FASTMM4 reports mem leaks).
Try and create a simple VCL datasnap server - TCP.
And add a button and this source ...
procedure TForm1.Button1Click(Sender: TObject);
var
// AObject : TObject;
aJSONVal : TJSONValue;
begin
// AObject := TObject.Create;
// ServerContainer1.DSServer1.BroadcastObject('SomeChannel','SomeCallbackID', AObject);
// AObject.Free;
aJSONVal := TJSONObject.Create;
ServerContainer1.DSServer1.BroadcastMessage('SomeChannel','SomeCallbackID',aJSONVal);
// aJSONVal.Free; // Mat pointed out that this is done by the broadcast.
end;
It will work - as long as you keep using TJSONValue ...
But try and switch the commented code - and you will see what I mean.
I could of course change all my existing code to JSON ... but that is simply not acceptable.
Does anyone have any idea on how to use the BroadcastOBJECT or NotifyOBJECT ?
Regards
Bjarne
The object which you give to a Notify or Broadcast call is then owned by that call. Therefore do not call "AObject.Free;" or "aJSONVal.Free;". Doing so will often result in an Access Violation or other memory management related problems.
Note also that Broadcasted messages get put in a queue on the server and are later sent, in a different thread. Meaning, when your call to Broadcast returns, it hasn't actually sent the message to all the clients yet.
I hope that helps,
Mat
Possible answer: Your question was vague but based on what you've said, I'd start here:
Delphi XE help: (ms-help://embarcadero.rs_xe/vcl/DSServer.TDSServer.BroadcastObject.html): function BroadcastObject(const ChannelName: String; const CallbackId: String; const Msg: TObject): boolean; overload;
The second overload sends an object to all client callbacks with a given registered callback identifier. For this purpose, an additional CallbackId parameter is required in the call."
You are using the second overload which takes 3 params - are your callback identifiers set up properly?