I am using Indy client to read the message the server is sending to me (client). It sends 512 bytes of data to me in one go. This 512 bytes of data is composed of two datatypes (Word and String). For example, it sends Word of 2 bytes, then again Word of 2 bytes and then String of 50 bytes and so on. I am trying following code to cope with this problem:
var BufferArray : Array[0..512] of Byte;
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
begin
Edit1.Text := idtcpclient1.IOHandler.ReadBytes(BufferArray ,512, true);
end;
end;
I am getting error on line Edit1.Text := idtcpclient1.IOHandler.ReadBytes(BufferArray ,512, true); Error: Type of actual and formal var parameter must be identical.
Is it right approach I am using. I want to store whole 512 bytes on Edit1.Text and then will do whatever I want to do with that data. Please help me in getting all 512 bytes from the server.
Update: Alternating Approach
I am using this approach to read word and string values
WordArray : array[0..5] of word;
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
begin
i := 0;
while i < 6 do //Read all the words
begin
//Fill WORD data in array
WordArray[i] := (IdTCPClient1.Socket.ReadWord(True));
end;
end;
end;
Similar approach for string like
WordArray[i] := (IdTCPClient1.Socket.ReadString(50));
Thats working fine, but I have to remain the connection open while I read all the data in loop. If in between connection goes, I lose everything and have to request the whole package again from server.
It's hard to answer you, unless you precisely describe what's written in documentation you have. So far we know that your 512B packet consists from 6 words and 10x50B strings. So, take this just as starting point, until you tell us more:
procedure TForm1.Button1Click(Sender: TObject);
var
I: Integer;
Buffer: TBytes;
WordArray: array[0..5] of Word;
StringArray: array[0..9] of AnsiString;
begin
if IdTCPClient1.IOHandler.InputBufferIsEmpty then
begin
if IdTCPClient1.IOHandler.CheckForDataOnSource(1000) then
IdTCPClient1.IOHandler.ReadBytes(Buffer, 512, False);
for I := 0 to High(WordArray) do
begin
WordRec(WordArray[I]).Hi := Buffer[I * 2];
WordRec(WordArray[I]).Lo := Buffer[I * 2 + 1];
end;
for I := 0 to High(StringArray) do
SetString(StringArray[I], PAnsiChar(#Buffer[I * 50 + 12]), 50);
// here you have the arrays prepared to be processed
end;
end;
1: what is the charset of the string ? is it 1-byte windows-1251 ? or 2-bytes Unicode UCS-2 ? or variable-length UTF-8 or UTF-16 ?
2: what is the length of the string ? always 50 ?
reading the buffer:
reading the manuals
1.1 http://www.indyproject.org/docsite/html/TIdIOHandler_ReadBytes#TIdBytes#Integer#boolean.html
1.2 http://www.indyproject.org/docsite/html/TIdIOHandler_ReadSmallInt#Boolean.html
1.3 http://docwiki.embarcadero.com/Libraries/XE2/en/System.SetString
making code accurately following types and parameter descriptions.
2.1 Reading header: That should result in something like
var Word1, Word2: word;
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
reading single-byte string
3.1 reading buffer
3.2 converting buffer to string
var Word1, Word2: word; Buffer: TIdBytes;
var s: RawByteString;
// or AnsiString; or maybe UTF8String; but probably not UnicodeString aka string
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
// You should check that you really received 50 bytes,
// then do something like that:
IOHandler.ReadBytes(Buffer, 50, false);
Assert(Length(Buffer)=50);
SetString (s, pointer(#Buffer[0]), 50);
Continue reading the rest - you only read 50+2+2 = 54 bytes of 512 bytes packet - there should be more data.
512 = 54*9+26 - so it might look like a loop - and discarding the 26 bytes tail.
var Word1, Word2: word; Buffer: TIdBytes;
var s: RawByteString;
for i := 1 to 9 do begin
Word1 := IOHandler.ReadSmallInt(false);
Word2 := IOHandler.ReadSmallInt(false);
IOHandler.ReadBytes(Buffer, 50, false);
Assert(Length(Buffer)=50);
SetString (s, pointer(#Buffer[0]), 50);
SomeOutputCollection.AppendNewElement(Word1, Word2, s);
end;
IOHandler.ReadBytes(Buffer, 512 - 9*(50+2+2), false); // discard the tail
Related
I have this record:
type
TSocks5_Packet = record
Socks_ID: String[6];
Socks_Packet: array of byte;
end;
I put String[6] because I'm sure that this string will always have 6 characters (and to try facilitate my job, but wasn't enough).
I try to send this record here:
procedure TForm1.SocksServerClientRead(Sender: TObject;
Socket: TCustomWinSocket);
var
Socks5_Request: TSocks5_Packet;
begin
SetLength(Socks5_Request.Socks_Packet, Socket.ReceiveLength);
Socket.ReceiveBuf(Socks5_Request.Socks_Packet[0], Length(Socks5_Request.Socks_Packet));
Socks5_Request.Socks_ID:= PSocket_Identity(Socket.Data).Sock_ID;
TunnelClient.SendBuf(Socks5_Request, Length(Socks5_Request.Socks_Packet + Length(Socks5_Request.Socks_ID)));
end;
I'm pretty sure the problem is on SendBuf second parameter, where I specify the count of bytes to be sent. What is the correct approach, and how I should study to learn it?
There are several problems with your code:
you are expecting String[6] to be 6 bytes, but it is actually 7 bytes instead. You are declaring a ShortString with a max length of 6 AnsiChar characters, and a ShortString contains a leading byte for the length.
your record is not packed, so it is subject to alignment padding. So don't try to send it as-is. You are dealing with variable-length data, so you should be serializing the record's content instead.
array of byte is a dynamic array. A dynamic array is a pointer to data allocated elsewhere in memory. The byte size of the array variable itself is SizeOf(Pointer), which is 4 on 32bit and 8 on 64bit. You have to dereference the pointer in order to access the allocated memory block. You are doing that on your receive, but are not doing that on your send.
TCP is a streaming transport, it has no concept of messages at all. Socket.ReceiveLength reports the number of unread bytes that are currently in the socket's internal receive buffer at that particular moment. Those bytes are arbitrary, there may be as few as 1 byte in the buffer. And more bytes may be received after you query the length and before performing the actual read.
when sending data, it is not guaranteed to send as many bytes are you request, it may send fewer bytes. The return value of SendBuf() indicates the actual number of bytes accepted for sending, so you may need to call SendBuf() multiple times to send a particular piece of data. So loop the send until the data is exhausted.
Since you are trying to tunnel arbitrary data, you need to specify how many bytes you are actually tunneling.
With that said, try something more like this:
function TForm1.SendData(Socket: TCustomWinSocket; const Data; DataLen: Integer): Boolean;
var
PData: PByte;
NumSent: Integer;
begin
Result := False;
PData := PByte(#Data);
while DataLen > 0 do
begin
// SendBuf() returns -1 on error. If that error is WSAEWOULDBLOCK
// then retry the send again. Otherwise, TCustomWinSocket disconnects
// itself, and if its OnError event handler does not set the ErrorCode
// to 0 then it raises an ESocketError exception...
//
NumSent := Socket.SendBuf(PData^, DataLen);
if NumSent = -1 then
begin
if WSAGetLastError() <> WSAEWOULDBLOCK then
Exit;
end else
begin
Inc(PData, NumSent);
Dec(DataLen, NumSent);
end;
end;
Result := True;
end;
function TForm1.SendInteger(Socket: TCustomWinSocket; Value: Integer): Boolean;
begin
Value := htonl(Value);
Result := SendData(Socket, Value, SizeOf(Value));
end;
function TForm1.SendString(Socket: TCustomWinSocket; const Value: String): Boolean;
var
S: AnsiString; // or UTF8String
begin
S := AnsiString(Value);
// or: S := UTF8Encode(Value);
Result := SendInteger(Socket, Length(S));
if Result then
Result := SendData(Socket, PAnsiChar(S)^, Length(S));
end;
function TForm1.SendBytes(Socket: TCustomWinSocket; const Data: array of Byte): Boolean;
begin
Result := SendInteger(Socket, Length(Data));
if Result then
Result := SendData(Socket, PByte(Data)^, Length(Data));
end;
procedure TForm1.SocksServerClientRead(Sender: TObject; Socket: TCustomWinSocket);
var
Packet: array of byte;
begin
Len := Socket.ReceiveLength;
if Len <= 0 the Exit;
SetLength(Packet, Len);
Len := Socket.ReceiveBuf(PByte(Packet)^, Len);
if Len <= 0 the Exit;
if Len < Length(Packet) then
SetLength(Packet, Len);
if SendString(TunnelClient, PSocket_Identity(Socket.Data).Sock_ID) then
SendBytes(TunnelClient, Packet);
end;
And then adjust your tunnel receiver accordingly to de-serialize the values as needed (read an Integer byte count, convert it to host byte order using ntohl(), then read the specified number of bytes).
I use Berlin in Windows 10. I try to save tList<string> to a file.
I know how to handle tStringlist, tStreamWriter and tStreamReader but I need to use tFileStream because the other type of data should be added.
In the following code the loop of Button2Click which reads the data raises an eOutOfMemory exception. When I allocate simple string value to _String it works well but if I put tList value to the same _String it seems that wrong data were written on the file. I can't understand the difference between _String := _List.List[i] and _String := 'qwert'.
How can I write tList<string> to tFileSteam?
procedure TForm1.Button1Click(Sender: TObject);
var
_List: TList<string>;
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_List := TList<string>.Create;
_List.Add('abcde');
_List.Add('abcde12345');
_FileStream := TFileStream.Create('test', fmCreate);
for i := 0 to 1 do
begin
_String := _List.List[i]; // _String := 'qwert' works well
_FileStream.Write(_string, 4);
end;
_FileStream.Free;
_List.Free;
end;
procedure TForm1.Button2Click(Sender: TObject);
var
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_FileStream := TFileStream.Create('test', fmOpenRead);
for i := 0 to 1 do
begin
_FileStream.Read(_String, 4);
Memo1.Lines.Add(_String);
end;
_FileStream.Free;
end;
If you lookup in the docs what TFileStream.Write does, it tells you (inherited from THandleStream.Write):
function Write(const Buffer; Count: Longint): Longint; override;
function Write(const Buffer: TBytes; Offset, Count: Longint): Longint; override;
Writes Count bytes from the Buffer to the current position in the
resource.
Now, Buffer is untyped and as such is expected to be the memory address of the data to be written. You are passing a string variable which is a reference to the actual string data, the address of the variable holds a pointer to string data. You are therefore writing a pointer to the file.
To correct it pass the strings first character for the buffer, ....write(_string[1], ...
If you have compiler directive {$ZEROBASEDSTRINGS ON} you would use index 0.
Alternatively, typecast the string to PChar and dereference it: ....write(PChar(_String)^, ...
Then look at the second parameter, Count. As the docs say, it indicates the number of bytes to be written, specifically not characters. In Delphi 2009 and later strings are UnicodeString, so each character is 2 bytes. You need to pass the strings size in bytes.
This will write 4 characters (8 bytes) to the file stream:
_FileStream.Write(_String[1], 4 * SizeOf(Char));
or better
_FileStream.Write(PChar(_String)^, 4 * SizeOf(Char));
For reading you need to make corresponding changes, but most notable, you need to set the strings length before reading (length is counted in characters).
SetLength(_String, 4);
for i := 0 to 1 do
begin
_FileStream.Read(_String[1], 4 * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
To continue with this low-level approach you could generalize string writing and reading as follows:
Add a variable to hold the length of a string
var
_String: string;
_Length: integer;
then writing
begin
...
for ....
begin
_String := _List.List[i];
_Length := Length(_String);
_FileStream.Write(_Length, SizeOf(Integer));
_FileStream.Write(PChar(_List.List[i])^, _Length * SizeOf(Char));
end;
and reading
begin
...
for ....
begin
_FileStream.Read(_Length, SizeOf(Integer));
SetLength(_String, _Length);
_FileStream.Read(_String[1], _Length * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
IOW, you write the length first and then the string. On reading you read the length and then the string.
When I use this code and function in Delphi 7 an error message will be displayed :
This code convert MemoryStream content to WideString
function ReadWideString(stream: TStream): WideString;
var
nChars: LongInt;
begin
stream.Position := 0;
stream.ReadBuffer(nChars, SizeOf(nChars));
SetLength(Result, nChars);
if nChars > 0 then
stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
end;
procedure TForm1.Button2Click(Sender: TObject);
var
mem: TMemoryStream;
begin
mem := TMemoryStream.Create;
mem.LoadFromFile('C:\Users\User1\Desktop\wide.txt');
Memo1.Lines.Add(ReadWideString(mem));
end;
Any help would be greatly appreciated.
Your code works fine as is. The problem is that the input that you pass to your function is not in the expected format.
The function that you are using expects a 4 byte integer containing the length, followed by the UTF-16 payload.
It looks like you actually have straight UTF-16 text, without the length prepended. Read that like this:
stream.Position := 0;
nChars := stream.Size div SizeOf(Result[1]);
SetLength(Result, nChars);
if nChars > 0 then
stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
Now, your input may contain a UTF-16 BOM. If so you'll need to decide how to handle that.
The bottom line here is that you need your code to match the input you provide.
If I compile for 64-bit-Windows my Byte-Arrays haven't got the correct Input-Values.
If I compile this procedure for x32-Windows the values are correct.
Can anyone help me?
procedure doAnything(AText: String); //for example "<xml><s name="hello"/></xml>"
var
myArray:array of Byte absolute AText;
begin
... (* myArray for x32: correct Length and Values (60, 0, 120, 0, 109, 0, ...) *)
... (* myArray for x64: Length: 2 (60, 0) *)
end
The memory layout for a string is not the same as a dynamic array.
Using the absolute keyword here is plain wrong.
In 32 bit it happens that the length is read correctly, but the value is in characters, not in bytes.
You can do something like this to access the string as bytes:
procedure doAnything(AText: String); //for example "<xml><s name="hello"/></xml>"
var
pB : PByte;
i,len : Integer;
begin
pB := Pointer(AText);
len := Length(AText)*SizeOf(Char);
for i := 1 to len do
begin
WriteLn(pB^);
Inc(pB);
end;
// Or
for i := 0 to len-1 do
begin
WriteLn(pB[i]);
end;
end;
If you want to access the character data of a String as raw bytes, you have to use a type-cast instead, DO NOT use absolute as the memory layout of a String and a dynamic array are not compatible, as others have pointed out to you:
procedure doAnything(AText: String);
var
myBytes: PByte;
myBytesLen: Integer;
begin
myBytes := PByte(PChar(AText));
myBytesLen := ByteLength(AText);
// or: myBytesLen := Length(AText) * SizeOf(Char);
// use myBytes up to myBytesLen as needed...
end;
If you really wanted to use absolute, you would have to use it more like this instead:
procedure doAnything(AText: String);
var
myChars: PChar;
myBytes: PByte absolute myChars;
myBytesLen: Integer;
begin
myChars := PChar(AText);
myBytesLen := ByteLength(AText);
// or: myBytesLen := Length(AText) * SizeOf(Char);
// use myBytes up to myBytesLen as needed...
end;
From what I understand, the problem is that you are mapping apples and pears in the 64 bit world. If you look at this:
http://docwiki.embarcadero.com/RADStudio/XE5/en/Internal_Data_Formats#Dynamic_Array_Types
And the string:
http://docwiki.embarcadero.com/RADStudio/XE5/en/Internal_Data_Formats#Long_String_Types
You will see that the lengths have a different number of bytes for these two. The offsets also don't match. Basically they are not compatible.
i am using the move command its working for one byte to string conversion for record values,but when i add more values it displays garbage value. here is the code
interface
type
tcommand = (
cmd_login,
cmd_logout,
cmd_userinfo,
cmd_removeuser,
cmd_response
);
tprotocol = record
username: string;
receipent_username: string;
arr: tbytes;
case command_id: tcommand of
cmd_userinfo:
(username2: shortstring; ip: shortstring; port: word); // new info
cmd_response:
(status: boolean);
cmd_removeuser:
(username_remove: shortstring);
end;
pprotocol = ^tprotocol;
procedure encode_packet(obj: pprotocol);
procedure decode_packet(arr1: tbytes);
implementation
procedure encode_packet(obj: pprotocol);
begin
setlength(obj.arr, length(obj.username) * 2);
move(obj.username[1], obj.arr[0], length(obj.username) * 2);
setlength(obj.arr, length(obj.receipent_username) * 2);
// SetLength(Destination, SourceSize);
move(obj.receipent_username[1], obj.arr[1],
length(obj.receipent_username) * 2);
// problem starts from here
end;
procedure decode_packet(arr1: tbytes);
begin
setlength(username, length(arr1));
move(arr1[0], username[1], length(arr1));
setlength(s, length(arr1));
move(arr1[1], s[1], length(arr1));
end;
Usage:
showmessage(username);
// displays correct value if recepient_username is not encoded
showmessage(s);
procedure TForm1.encodeClick(Sender: TObject); // button click
var
obj2: pprotocol;
begin
new(obj);
new(obj2);
memo1.Lines.Add('encode click');
obj.username := 'ahmd';
obj.receipent_username := 'ali';
encode_packet(obj);
decode_packet(obj.arr);
end;
i think i have to make some indexing from where to start and stop decoding the byte array but i dont know how ? can anyone explain me how a string is stored in a byte array (when i debug i saw there are numbers and some nil values how could one get the string from indexing if they are so mixed up with nil values?)
You copy with move command length*2 bytes on one side, but length only bytes on other side.
If you use unicode strings, then you need to use length*2 bytes on the both sides.
Another problem here is that you copy two strings in one array, one by one. If you want to save both strings in one array, then you have to allocate enough of space inside array, also put information about length of strings and write content of both strings. Example how to put 2 strings into 1 array of bytes:
procedure test;
var
len: integer;
buf: array of byte;
a,b: string;
begin
a := 'try';
b := 'that';
// save
setlength(buf, SizeOf(Char)*(Length(a)+Length(b))+8);
len := length(a);
move(len, buf[0], 4);
len := length(b);
move(len, buf[4], 4);
move(a[1], buf[8], length(a)*SizeOf(char));
move(b[1], buf[8+length(a)*SizeOf(char)], length(a)*SizeOf(char));
// restore
a := '';
b := '';
move(buf[0], len, 4);
setlength(a, len);
move(buf[4], len, 4);
setlength(b, len);
move(buf[8], a[1], length(a)*SizeOf(char));
move(buf[8+length(a)*SizeOf(char)], b[1], length(a)*SizeOf(char));
end;
But i recommend you do not play with pointers and use any kind of serialization instead, for example memory streams.