Sending the right record size over socket - delphi

I have this record:
type
TSocks5_Packet = record
Socks_ID: String[6];
Socks_Packet: array of byte;
end;
I put String[6] because I'm sure that this string will always have 6 characters (and to try facilitate my job, but wasn't enough).
I try to send this record here:
procedure TForm1.SocksServerClientRead(Sender: TObject;
Socket: TCustomWinSocket);
var
Socks5_Request: TSocks5_Packet;
begin
SetLength(Socks5_Request.Socks_Packet, Socket.ReceiveLength);
Socket.ReceiveBuf(Socks5_Request.Socks_Packet[0], Length(Socks5_Request.Socks_Packet));
Socks5_Request.Socks_ID:= PSocket_Identity(Socket.Data).Sock_ID;
TunnelClient.SendBuf(Socks5_Request, Length(Socks5_Request.Socks_Packet + Length(Socks5_Request.Socks_ID)));
end;
I'm pretty sure the problem is on SendBuf second parameter, where I specify the count of bytes to be sent. What is the correct approach, and how I should study to learn it?

There are several problems with your code:
you are expecting String[6] to be 6 bytes, but it is actually 7 bytes instead. You are declaring a ShortString with a max length of 6 AnsiChar characters, and a ShortString contains a leading byte for the length.
your record is not packed, so it is subject to alignment padding. So don't try to send it as-is. You are dealing with variable-length data, so you should be serializing the record's content instead.
array of byte is a dynamic array. A dynamic array is a pointer to data allocated elsewhere in memory. The byte size of the array variable itself is SizeOf(Pointer), which is 4 on 32bit and 8 on 64bit. You have to dereference the pointer in order to access the allocated memory block. You are doing that on your receive, but are not doing that on your send.
TCP is a streaming transport, it has no concept of messages at all. Socket.ReceiveLength reports the number of unread bytes that are currently in the socket's internal receive buffer at that particular moment. Those bytes are arbitrary, there may be as few as 1 byte in the buffer. And more bytes may be received after you query the length and before performing the actual read.
when sending data, it is not guaranteed to send as many bytes are you request, it may send fewer bytes. The return value of SendBuf() indicates the actual number of bytes accepted for sending, so you may need to call SendBuf() multiple times to send a particular piece of data. So loop the send until the data is exhausted.
Since you are trying to tunnel arbitrary data, you need to specify how many bytes you are actually tunneling.
With that said, try something more like this:
function TForm1.SendData(Socket: TCustomWinSocket; const Data; DataLen: Integer): Boolean;
var
PData: PByte;
NumSent: Integer;
begin
Result := False;
PData := PByte(#Data);
while DataLen > 0 do
begin
// SendBuf() returns -1 on error. If that error is WSAEWOULDBLOCK
// then retry the send again. Otherwise, TCustomWinSocket disconnects
// itself, and if its OnError event handler does not set the ErrorCode
// to 0 then it raises an ESocketError exception...
//
NumSent := Socket.SendBuf(PData^, DataLen);
if NumSent = -1 then
begin
if WSAGetLastError() <> WSAEWOULDBLOCK then
Exit;
end else
begin
Inc(PData, NumSent);
Dec(DataLen, NumSent);
end;
end;
Result := True;
end;
function TForm1.SendInteger(Socket: TCustomWinSocket; Value: Integer): Boolean;
begin
Value := htonl(Value);
Result := SendData(Socket, Value, SizeOf(Value));
end;
function TForm1.SendString(Socket: TCustomWinSocket; const Value: String): Boolean;
var
S: AnsiString; // or UTF8String
begin
S := AnsiString(Value);
// or: S := UTF8Encode(Value);
Result := SendInteger(Socket, Length(S));
if Result then
Result := SendData(Socket, PAnsiChar(S)^, Length(S));
end;
function TForm1.SendBytes(Socket: TCustomWinSocket; const Data: array of Byte): Boolean;
begin
Result := SendInteger(Socket, Length(Data));
if Result then
Result := SendData(Socket, PByte(Data)^, Length(Data));
end;
procedure TForm1.SocksServerClientRead(Sender: TObject; Socket: TCustomWinSocket);
var
Packet: array of byte;
begin
Len := Socket.ReceiveLength;
if Len <= 0 the Exit;
SetLength(Packet, Len);
Len := Socket.ReceiveBuf(PByte(Packet)^, Len);
if Len <= 0 the Exit;
if Len < Length(Packet) then
SetLength(Packet, Len);
if SendString(TunnelClient, PSocket_Identity(Socket.Data).Sock_ID) then
SendBytes(TunnelClient, Packet);
end;
And then adjust your tunnel receiver accordingly to de-serialize the values as needed (read an Integer byte count, convert it to host byte order using ntohl(), then read the specified number of bytes).

Related

Writing tList<string> to tFileStream

I use Berlin in Windows 10. I try to save tList<string> to a file.
I know how to handle tStringlist, tStreamWriter and tStreamReader but I need to use tFileStream because the other type of data should be added.
In the following code the loop of Button2Click which reads the data raises an eOutOfMemory exception. When I allocate simple string value to _String it works well but if I put tList value to the same _String it seems that wrong data were written on the file. I can't understand the difference between _String := _List.List[i] and _String := 'qwert'.
How can I write tList<string> to tFileSteam?
procedure TForm1.Button1Click(Sender: TObject);
var
_List: TList<string>;
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_List := TList<string>.Create;
_List.Add('abcde');
_List.Add('abcde12345');
_FileStream := TFileStream.Create('test', fmCreate);
for i := 0 to 1 do
begin
_String := _List.List[i]; // _String := 'qwert' works well
_FileStream.Write(_string, 4);
end;
_FileStream.Free;
_List.Free;
end;
procedure TForm1.Button2Click(Sender: TObject);
var
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_FileStream := TFileStream.Create('test', fmOpenRead);
for i := 0 to 1 do
begin
_FileStream.Read(_String, 4);
Memo1.Lines.Add(_String);
end;
_FileStream.Free;
end;
If you lookup in the docs what TFileStream.Write does, it tells you (inherited from THandleStream.Write):
function Write(const Buffer; Count: Longint): Longint; override;
function Write(const Buffer: TBytes; Offset, Count: Longint): Longint; override;
Writes Count bytes from the Buffer to the current position in the
resource.
Now, Buffer is untyped and as such is expected to be the memory address of the data to be written. You are passing a string variable which is a reference to the actual string data, the address of the variable holds a pointer to string data. You are therefore writing a pointer to the file.
To correct it pass the strings first character for the buffer, ....write(_string[1], ...
If you have compiler directive {$ZEROBASEDSTRINGS ON} you would use index 0.
Alternatively, typecast the string to PChar and dereference it: ....write(PChar(_String)^, ...
Then look at the second parameter, Count. As the docs say, it indicates the number of bytes to be written, specifically not characters. In Delphi 2009 and later strings are UnicodeString, so each character is 2 bytes. You need to pass the strings size in bytes.
This will write 4 characters (8 bytes) to the file stream:
_FileStream.Write(_String[1], 4 * SizeOf(Char));
or better
_FileStream.Write(PChar(_String)^, 4 * SizeOf(Char));
For reading you need to make corresponding changes, but most notable, you need to set the strings length before reading (length is counted in characters).
SetLength(_String, 4);
for i := 0 to 1 do
begin
_FileStream.Read(_String[1], 4 * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
To continue with this low-level approach you could generalize string writing and reading as follows:
Add a variable to hold the length of a string
var
_String: string;
_Length: integer;
then writing
begin
...
for ....
begin
_String := _List.List[i];
_Length := Length(_String);
_FileStream.Write(_Length, SizeOf(Integer));
_FileStream.Write(PChar(_List.List[i])^, _Length * SizeOf(Char));
end;
and reading
begin
...
for ....
begin
_FileStream.Read(_Length, SizeOf(Integer));
SetLength(_String, _Length);
_FileStream.Read(_String[1], _Length * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
IOW, you write the length first and then the string. On reading you read the length and then the string.

How to make call to DLL function from Delphi?

// Get a list of accounts in a domain separated by \x00 and ended by \x00\x00
Function GetUserList(AName: PAnsiChar; Var List; Size: Longint): Longint; StdCall;
I need to call the above from XE6.
Would someone be kind enough to post an example of how I can
get this buffer, and put it to a stream or a string.
The variable "List" is supposed to fill up some buffer, which I can read
off the list of users.
After trying for a couple of options, I have tried all options such as:
thanks!
var
Buffer: array of Byte;
iCount : Integer;
sName : AnsiString;
begin
...
SetLength(Buffer, 4096);
iCount := GetUserListTest(PAnsiChar(sName)#Buffer[0], Length(Buffer)); // cannot
// iCount := GetUserList(PAnsiChar(sName), Buffer, Length(Buffer));
That is not a Win32 API function, so it must be a third-party function. Ask the vendor for an example.
A var parameter expects you to pass a variable to it. The var receives the address of the variable. #Buffer[0] does not satisfy that requirement, as # returns a Pointer, and then the var ends up with the address of the pointer itself, not the address of the variable being pointed at. The function is expecting a pointer to a buffer. By using a var to receive that pointer, you need to drop the # and pass the first array element, so that the address of that element (effectively the address of the buffer) will be passed to the function, eg:
iCount := GetUserList(PAnsiChar(sName), Buffer[0], iCount);
Alternatively, you can use this syntax instead, which will pass the same address of the first element:
iCount := GetUserList(PAnsiChar(sName), PByte(Buffer)^, iCount);
Now, with that said, chances are that the function may allow you to query it for the necessary array size so you can allocate only what is actually needed (but check the documentation to be sure, I'm making an assumption here since you have not said otherwise)), eg:
procedure GetDomainUsers(const Domain: AnsiString; Users: TStrings);
var
Buffer: array of AnsiChar;
iCount : Integer;
User: PAnsiChar;
begin
// this call ASSUMES the function returns the needed
// bytecount when given a NULL/empty array - check
// the documentation!!!
iCount := GetUserList(PAnsiChar(Domain), PAnsiChar(nil)^, 0);
if iCount > 0 then
begin
SetLength(Buffer, iCount);
iCount := GetUserList(PAnsiChar(Domain), Buffer[0]{or: PAnsiChar(Buffer)^}, iCount);
end;
if iCount > 0 then
begin
Users.BeginUpdate;
try
User := PAnsiChar(Buffer);
while User^ <> #0 do
begin
Users.Add(User);
Inc(User, StrLen(User)+1);
end;
finally
Users.EndUpdate;
end;
end;
end;
If that does not work, then you will have to pre-allocate a large array:
procedure GetDomainUsers(const Domain: AnsiString; Users: TStrings);
var
Buffer: array of AnsiChar;
User: PAnsiChar;
begin
SetLength(Buffer, 1024);
if GetUserList(PAnsiChar(Domain), Buffer[0]{or: PAnsiChar(Buffer)^}, Length(Buffer)) > 0 then
begin
Users.BeginUpdate;
try
User := PAnsiChar(Buffer);
while User^ <> #0 do
begin
Users.Add(User);
Inc(User, StrLen(User)+1);
end;
finally
Users.EndUpdate;
end;
end;
end;

how to convert a record to byte array and extract values back in delphi?

i am using the move command its working for one byte to string conversion for record values,but when i add more values it displays garbage value. here is the code
interface
type
tcommand = (
cmd_login,
cmd_logout,
cmd_userinfo,
cmd_removeuser,
cmd_response
);
tprotocol = record
username: string;
receipent_username: string;
arr: tbytes;
case command_id: tcommand of
cmd_userinfo:
(username2: shortstring; ip: shortstring; port: word); // new info
cmd_response:
(status: boolean);
cmd_removeuser:
(username_remove: shortstring);
end;
pprotocol = ^tprotocol;
procedure encode_packet(obj: pprotocol);
procedure decode_packet(arr1: tbytes);
implementation
procedure encode_packet(obj: pprotocol);
begin
setlength(obj.arr, length(obj.username) * 2);
move(obj.username[1], obj.arr[0], length(obj.username) * 2);
setlength(obj.arr, length(obj.receipent_username) * 2);
// SetLength(Destination, SourceSize);
move(obj.receipent_username[1], obj.arr[1],
length(obj.receipent_username) * 2);
// problem starts from here
end;
procedure decode_packet(arr1: tbytes);
begin
setlength(username, length(arr1));
move(arr1[0], username[1], length(arr1));
setlength(s, length(arr1));
move(arr1[1], s[1], length(arr1));
end;
Usage:
showmessage(username);
// displays correct value if recepient_username is not encoded
showmessage(s);
procedure TForm1.encodeClick(Sender: TObject); // button click
var
obj2: pprotocol;
begin
new(obj);
new(obj2);
memo1.Lines.Add('encode click');
obj.username := 'ahmd';
obj.receipent_username := 'ali';
encode_packet(obj);
decode_packet(obj.arr);
end;
i think i have to make some indexing from where to start and stop decoding the byte array but i dont know how ? can anyone explain me how a string is stored in a byte array (when i debug i saw there are numbers and some nil values how could one get the string from indexing if they are so mixed up with nil values?)
You copy with move command length*2 bytes on one side, but length only bytes on other side.
If you use unicode strings, then you need to use length*2 bytes on the both sides.
Another problem here is that you copy two strings in one array, one by one. If you want to save both strings in one array, then you have to allocate enough of space inside array, also put information about length of strings and write content of both strings. Example how to put 2 strings into 1 array of bytes:
procedure test;
var
len: integer;
buf: array of byte;
a,b: string;
begin
a := 'try';
b := 'that';
// save
setlength(buf, SizeOf(Char)*(Length(a)+Length(b))+8);
len := length(a);
move(len, buf[0], 4);
len := length(b);
move(len, buf[4], 4);
move(a[1], buf[8], length(a)*SizeOf(char));
move(b[1], buf[8+length(a)*SizeOf(char)], length(a)*SizeOf(char));
// restore
a := '';
b := '';
move(buf[0], len, 4);
setlength(a, len);
move(buf[4], len, 4);
setlength(b, len);
move(buf[8], a[1], length(a)*SizeOf(char));
move(buf[8+length(a)*SizeOf(char)], b[1], length(a)*SizeOf(char));
end;
But i recommend you do not play with pointers and use any kind of serialization instead, for example memory streams.

How to read all bytes from server using Indy Client in Delphi?

I am using Indy client to read the message the server is sending to me (client). It sends 512 bytes of data to me in one go. This 512 bytes of data is composed of two datatypes (Word and String). For example, it sends Word of 2 bytes, then again Word of 2 bytes and then String of 50 bytes and so on. I am trying following code to cope with this problem:
var BufferArray : Array[0..512] of Byte;
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
begin
Edit1.Text := idtcpclient1.IOHandler.ReadBytes(BufferArray ,512, true);
end;
end;
I am getting error on line Edit1.Text := idtcpclient1.IOHandler.ReadBytes(BufferArray ,512, true); Error: Type of actual and formal var parameter must be identical.
Is it right approach I am using. I want to store whole 512 bytes on Edit1.Text and then will do whatever I want to do with that data. Please help me in getting all 512 bytes from the server.
Update: Alternating Approach
I am using this approach to read word and string values
WordArray : array[0..5] of word;
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
begin
i := 0;
while i < 6 do //Read all the words
begin
//Fill WORD data in array
WordArray[i] := (IdTCPClient1.Socket.ReadWord(True));
end;
end;
end;
Similar approach for string like
WordArray[i] := (IdTCPClient1.Socket.ReadString(50));
Thats working fine, but I have to remain the connection open while I read all the data in loop. If in between connection goes, I lose everything and have to request the whole package again from server.
It's hard to answer you, unless you precisely describe what's written in documentation you have. So far we know that your 512B packet consists from 6 words and 10x50B strings. So, take this just as starting point, until you tell us more:
procedure TForm1.Button1Click(Sender: TObject);
var
I: Integer;
Buffer: TBytes;
WordArray: array[0..5] of Word;
StringArray: array[0..9] of AnsiString;
begin
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
IdTCPClient1.IOHandler.ReadBytes(Buffer, 512, False);
for I := 0 to High(WordArray) do
begin
WordRec(WordArray[I]).Hi := Buffer[I * 2];
WordRec(WordArray[I]).Lo := Buffer[I * 2 + 1];
end;
for I := 0 to High(StringArray) do
SetString(StringArray[I], PAnsiChar(#Buffer[I * 50 + 12]), 50);
// here you have the arrays prepared to be processed
end;
end;
1: what is the charset of the string ? is it 1-byte windows-1251 ? or 2-bytes Unicode UCS-2 ? or variable-length UTF-8 or UTF-16 ?
2: what is the length of the string ? always 50 ?
reading the buffer:
reading the manuals
1.1 http://www.indyproject.org/docsite/html/TIdIOHandler_ReadBytes#TIdBytes#Integer#boolean.html
1.2 http://www.indyproject.org/docsite/html/TIdIOHandler_ReadSmallInt#Boolean.html
1.3 http://docwiki.embarcadero.com/Libraries/XE2/en/System.SetString
making code accurately following types and parameter descriptions.
2.1 Reading header: That should result in something like
var Word1, Word2: word;
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
reading single-byte string
3.1 reading buffer
3.2 converting buffer to string
var Word1, Word2: word; Buffer: TIdBytes;
var s: RawByteString;
// or AnsiString; or maybe UTF8String; but probably not UnicodeString aka string
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
// You should check that you really received 50 bytes,
// then do something like that:
IOHandler.ReadBytes(Buffer, 50, false);
Assert(Length(Buffer)=50);
SetString (s, pointer(#Buffer[0]), 50);
Continue reading the rest - you only read 50+2+2 = 54 bytes of 512 bytes packet - there should be more data.
512 = 54*9+26 - so it might look like a loop - and discarding the 26 bytes tail.
var Word1, Word2: word; Buffer: TIdBytes;
var s: RawByteString;
for i := 1 to 9 do begin
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
IOHandler.ReadBytes(Buffer, 50, false);
Assert(Length(Buffer)=50);
SetString (s, pointer(#Buffer[0]), 50);
SomeOutputCollection.AppendNewElement(Word1, Word2, s);
end;
IOHandler.ReadBytes(Buffer, 512 - 9*(50+2+2), false); // discard the tail

Upgrade Indy9 to Indy10

I want to upgrade my application from Indy 9 to 10 with Delphi 2007.
In this thread there is a call to Indy9 TIdUDPBase.SendBuffer but this won't compile in Indy10 as the method parameter don't exists. The third parameter aBuffer is a var parameter and I didn't find any such method signature in Indy10.
Any alternative method to call ?
procedure TSenderThread.Execute;
var
vTimeData: TTimeDataRecord;
I: Integer;
FElapsed: Int64;
FTimerElappsed,
vLastTimerElappsed: Int64;
begin
vTimeData.Size := SizeOf(TTimeDataRecord);
vTimeData.ClientCount := 1;
Priority := tpHighest;
FIdUDPClient := TIdUDPClient.Create(nil);
FIdUDPClient.BroadcastEnabled := True;
try
while not (Terminated or Application.Terminated) do
begin
Sleep(1000);
//Measure Time frame
vLastTimerElappsed := FTimerElappsed;
QueryPerformanceCounter(FTimerElappsed);
FElapsed := ((FTimerElappsed-vLastTimerElappsed)*1000000) div FFrequency;
vTimeData.TotalTimeFrame := FElapsed;
if FRunning then
begin
FElapsed := ((FTimerElappsed-FStart)*1000000) div FFrequency;
vTimeData.CurrentMessageTime := FElapsed;
end
else
vTimeData.CurrentMessageTime := 0;
//Copy Values
vTimeData.AccumulatedTime := InterlockedExchange(TimeData.AccumulatedTime,0);
vTimeData.MessageCount := InterlockedExchange(TimeData.MessageCount,0);
for I := 0 to TimeClassMax do
vTimeData.TimeClasses[I] := InterlockedExchange(TimeData.TimeClasses[I],0);
// Calls procedure TIdUDPBase.SendBuffer(AHost: string; const APort: Integer; var ABuffer; const AByteCount: integer);
// This is changed in Indy10, unable to compile
FIdUDPClient.SendBuffer('255.255.255.255', UIPerfPort, vTimeData, TimeData.Size);
end;
finally
FreeAndNil(FIdUDPClient);
end;
end;
EDIT:
vTimeData is basically an array of integers.
TTimeDataRecord = record
Size: Integer; //Size of record structure is transfered and compared for safty reasons.
ClientCount: Integer;
AccumulatedTime: Integer; //This is the accumulated time busy in microseconds
CurrentMessageTime: Integer; //This is the time the current message has been processed. If several computers report a high value at the same time it indicates a freeze!
TotalTimeFrame: Integer; //This is the total time measured in microseconds
MessageCount: Integer;
TimeClasses: array [0..TimeClassMax] of Integer;
end;
you have a method with same name
procedure TIdUDPClient.SendBuffer(const AHost: string; const APort: TIdPort;
const ABuffer: TIdBytes);
Instead of an untyped buffer it expects an array of bytes. What is your data like? You just need to write your data as an array of bytes. Something like:
var
Buffer: TIdBytes;
begin
SetLength(Buffer, YourSizeOfData);
Move(YourData, Buffer[0], YourSizeOfData);
FIdUDPClient.SendBuffer('255.255.255.255', UIPerfPort, Buffer);
end;
But as I said it depends on the type of the data. The approach is ok however.
EDIT:
Now that I can see that you have a record you have two options:
Just move the whole record to array of bytes.
Move(#aRecord, Buffer[0], (6 + TimeClassMax) * SizeOf(Integer));
Have a CopyToBytes method in your record that does the actual copy. More general I guess.
TTimeDataRecord = record
Size: Integer; //Size of record structure is transfered and compared for safty reasons.
ClientCount: Integer;
AccumulatedTime: Integer; //This is the accumulated time busy in microseconds
CurrentMessageTime: Integer; //This is the time the current message has been processed. If several computers report a high value at the same time it indicates a freeze!
TotalTimeFrame: Integer; //This is the total time measured in microseconds
MessageCount: Integer;
TimeClasses: array [0..TimeClassMax] of Integer;
procedure CopyToBytes(var Buffer: TIdBytes);
end
Implementation of the CopyToBytes
procedure TTimeDataRecord.CopyToBytes(var Buffer: TIdBytes);
begin
// copy the data however you see fit
end;

Resources