I use Berlin in Windows 10. I try to save tList<string> to a file.
I know how to handle tStringlist, tStreamWriter and tStreamReader but I need to use tFileStream because the other type of data should be added.
In the following code the loop of Button2Click which reads the data raises an eOutOfMemory exception. When I allocate simple string value to _String it works well but if I put tList value to the same _String it seems that wrong data were written on the file. I can't understand the difference between _String := _List.List[i] and _String := 'qwert'.
How can I write tList<string> to tFileSteam?
procedure TForm1.Button1Click(Sender: TObject);
var
_List: TList<string>;
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_List := TList<string>.Create;
_List.Add('abcde');
_List.Add('abcde12345');
_FileStream := TFileStream.Create('test', fmCreate);
for i := 0 to 1 do
begin
_String := _List.List[i]; // _String := 'qwert' works well
_FileStream.Write(_string, 4);
end;
_FileStream.Free;
_List.Free;
end;
procedure TForm1.Button2Click(Sender: TObject);
var
_FileStream: TFileStream;
_String: string;
i: Integer;
begin
_FileStream := TFileStream.Create('test', fmOpenRead);
for i := 0 to 1 do
begin
_FileStream.Read(_String, 4);
Memo1.Lines.Add(_String);
end;
_FileStream.Free;
end;
If you lookup in the docs what TFileStream.Write does, it tells you (inherited from THandleStream.Write):
function Write(const Buffer; Count: Longint): Longint; override;
function Write(const Buffer: TBytes; Offset, Count: Longint): Longint; override;
Writes Count bytes from the Buffer to the current position in the
resource.
Now, Buffer is untyped and as such is expected to be the memory address of the data to be written. You are passing a string variable which is a reference to the actual string data, the address of the variable holds a pointer to string data. You are therefore writing a pointer to the file.
To correct it pass the strings first character for the buffer, ....write(_string[1], ...
If you have compiler directive {$ZEROBASEDSTRINGS ON} you would use index 0.
Alternatively, typecast the string to PChar and dereference it: ....write(PChar(_String)^, ...
Then look at the second parameter, Count. As the docs say, it indicates the number of bytes to be written, specifically not characters. In Delphi 2009 and later strings are UnicodeString, so each character is 2 bytes. You need to pass the strings size in bytes.
This will write 4 characters (8 bytes) to the file stream:
_FileStream.Write(_String[1], 4 * SizeOf(Char));
or better
_FileStream.Write(PChar(_String)^, 4 * SizeOf(Char));
For reading you need to make corresponding changes, but most notable, you need to set the strings length before reading (length is counted in characters).
SetLength(_String, 4);
for i := 0 to 1 do
begin
_FileStream.Read(_String[1], 4 * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
To continue with this low-level approach you could generalize string writing and reading as follows:
Add a variable to hold the length of a string
var
_String: string;
_Length: integer;
then writing
begin
...
for ....
begin
_String := _List.List[i];
_Length := Length(_String);
_FileStream.Write(_Length, SizeOf(Integer));
_FileStream.Write(PChar(_List.List[i])^, _Length * SizeOf(Char));
end;
and reading
begin
...
for ....
begin
_FileStream.Read(_Length, SizeOf(Integer));
SetLength(_String, _Length);
_FileStream.Read(_String[1], _Length * SizeOf(Char));
Memo1.Lines.Add(_String);
end;
IOW, you write the length first and then the string. On reading you read the length and then the string.
Related
When I use this code and function in Delphi 7 an error message will be displayed :
This code convert MemoryStream content to WideString
function ReadWideString(stream: TStream): WideString;
var
nChars: LongInt;
begin
stream.Position := 0;
stream.ReadBuffer(nChars, SizeOf(nChars));
SetLength(Result, nChars);
if nChars > 0 then
stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
end;
procedure TForm1.Button2Click(Sender: TObject);
var
mem: TMemoryStream;
begin
mem := TMemoryStream.Create;
mem.LoadFromFile('C:\Users\User1\Desktop\wide.txt');
Memo1.Lines.Add(ReadWideString(mem));
end;
Any help would be greatly appreciated.
Your code works fine as is. The problem is that the input that you pass to your function is not in the expected format.
The function that you are using expects a 4 byte integer containing the length, followed by the UTF-16 payload.
It looks like you actually have straight UTF-16 text, without the length prepended. Read that like this:
stream.Position := 0;
nChars := stream.Size div SizeOf(Result[1]);
SetLength(Result, nChars);
if nChars > 0 then
stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
Now, your input may contain a UTF-16 BOM. If so you'll need to decide how to handle that.
The bottom line here is that you need your code to match the input you provide.
If I compile for 64-bit-Windows my Byte-Arrays haven't got the correct Input-Values.
If I compile this procedure for x32-Windows the values are correct.
Can anyone help me?
procedure doAnything(AText: String); //for example "<xml><s name="hello"/></xml>"
var
myArray:array of Byte absolute AText;
begin
... (* myArray for x32: correct Length and Values (60, 0, 120, 0, 109, 0, ...) *)
... (* myArray for x64: Length: 2 (60, 0) *)
end
The memory layout for a string is not the same as a dynamic array.
Using the absolute keyword here is plain wrong.
In 32 bit it happens that the length is read correctly, but the value is in characters, not in bytes.
You can do something like this to access the string as bytes:
procedure doAnything(AText: String); //for example "<xml><s name="hello"/></xml>"
var
pB : PByte;
i,len : Integer;
begin
pB := Pointer(AText);
len := Length(AText)*SizeOf(Char);
for i := 1 to len do
begin
WriteLn(pB^);
Inc(pB);
end;
// Or
for i := 0 to len-1 do
begin
WriteLn(pB[i]);
end;
end;
If you want to access the character data of a String as raw bytes, you have to use a type-cast instead, DO NOT use absolute as the memory layout of a String and a dynamic array are not compatible, as others have pointed out to you:
procedure doAnything(AText: String);
var
myBytes: PByte;
myBytesLen: Integer;
begin
myBytes := PByte(PChar(AText));
myBytesLen := ByteLength(AText);
// or: myBytesLen := Length(AText) * SizeOf(Char);
// use myBytes up to myBytesLen as needed...
end;
If you really wanted to use absolute, you would have to use it more like this instead:
procedure doAnything(AText: String);
var
myChars: PChar;
myBytes: PByte absolute myChars;
myBytesLen: Integer;
begin
myChars := PChar(AText);
myBytesLen := ByteLength(AText);
// or: myBytesLen := Length(AText) * SizeOf(Char);
// use myBytes up to myBytesLen as needed...
end;
From what I understand, the problem is that you are mapping apples and pears in the 64 bit world. If you look at this:
http://docwiki.embarcadero.com/RADStudio/XE5/en/Internal_Data_Formats#Dynamic_Array_Types
And the string:
http://docwiki.embarcadero.com/RADStudio/XE5/en/Internal_Data_Formats#Long_String_Types
You will see that the lengths have a different number of bytes for these two. The offsets also don't match. Basically they are not compatible.
i am using the move command its working for one byte to string conversion for record values,but when i add more values it displays garbage value. here is the code
interface
type
tcommand = (
cmd_login,
cmd_logout,
cmd_userinfo,
cmd_removeuser,
cmd_response
);
tprotocol = record
username: string;
receipent_username: string;
arr: tbytes;
case command_id: tcommand of
cmd_userinfo:
(username2: shortstring; ip: shortstring; port: word); // new info
cmd_response:
(status: boolean);
cmd_removeuser:
(username_remove: shortstring);
end;
pprotocol = ^tprotocol;
procedure encode_packet(obj: pprotocol);
procedure decode_packet(arr1: tbytes);
implementation
procedure encode_packet(obj: pprotocol);
begin
setlength(obj.arr, length(obj.username) * 2);
move(obj.username[1], obj.arr[0], length(obj.username) * 2);
setlength(obj.arr, length(obj.receipent_username) * 2);
// SetLength(Destination, SourceSize);
move(obj.receipent_username[1], obj.arr[1],
length(obj.receipent_username) * 2);
// problem starts from here
end;
procedure decode_packet(arr1: tbytes);
begin
setlength(username, length(arr1));
move(arr1[0], username[1], length(arr1));
setlength(s, length(arr1));
move(arr1[1], s[1], length(arr1));
end;
Usage:
showmessage(username);
// displays correct value if recepient_username is not encoded
showmessage(s);
procedure TForm1.encodeClick(Sender: TObject); // button click
var
obj2: pprotocol;
begin
new(obj);
new(obj2);
memo1.Lines.Add('encode click');
obj.username := 'ahmd';
obj.receipent_username := 'ali';
encode_packet(obj);
decode_packet(obj.arr);
end;
i think i have to make some indexing from where to start and stop decoding the byte array but i dont know how ? can anyone explain me how a string is stored in a byte array (when i debug i saw there are numbers and some nil values how could one get the string from indexing if they are so mixed up with nil values?)
You copy with move command length*2 bytes on one side, but length only bytes on other side.
If you use unicode strings, then you need to use length*2 bytes on the both sides.
Another problem here is that you copy two strings in one array, one by one. If you want to save both strings in one array, then you have to allocate enough of space inside array, also put information about length of strings and write content of both strings. Example how to put 2 strings into 1 array of bytes:
procedure test;
var
len: integer;
buf: array of byte;
a,b: string;
begin
a := 'try';
b := 'that';
// save
setlength(buf, SizeOf(Char)*(Length(a)+Length(b))+8);
len := length(a);
move(len, buf[0], 4);
len := length(b);
move(len, buf[4], 4);
move(a[1], buf[8], length(a)*SizeOf(char));
move(b[1], buf[8+length(a)*SizeOf(char)], length(a)*SizeOf(char));
// restore
a := '';
b := '';
move(buf[0], len, 4);
setlength(a, len);
move(buf[4], len, 4);
setlength(b, len);
move(buf[8], a[1], length(a)*SizeOf(char));
move(buf[8+length(a)*SizeOf(char)], b[1], length(a)*SizeOf(char));
end;
But i recommend you do not play with pointers and use any kind of serialization instead, for example memory streams.
I am using Delphi 2009.
I want to view the contents of a file (in hexadecimal) inside a memo.
I'm using this code :
var
Buffer:String;
begin
Buffer := '';
AssignFile(sF,Source); //Assign file
Reset(sF);
repeat
Readln(sF,Buffer); //Load every line to a string.
TempChar:=StrToHex(Buffer); //Convert to Hex using the function
...
until EOF(sF);
end;
function StrToHex(AStr: string): string;
var
I ,Len: Integer;
s: chr (0)..255;
//s:byte;
//s: char;
begin
len:=length(AStr);
Result:='';
for i:=1 to len do
begin
s:=AStr[i];
//The problem is here. Ord(s) is giving false values (251 instead of 255)
//And in general the output differs from a professional hex editor.
Result:=Result +' '+IntToHex(Ord(s),2)+'('+IntToStr(Ord(s))+')';
end;
Delete(Result,1,1);
end;
When I declare variable "s" as char (i know that char goes up to 255) I get results hex values up to 65535!
When i declare variable "s" as byte or chr (0)..255, it outputs different hex values, comparing to any Hexadecimal Editor!
Why is that? How can I see the correct values?
Check images for the differences.
1st image: Professional Hex Editor.
2nd image: Function output to Memo.
Thank you.
Your Delphi 2009 is unicode-enabled, so Char is actually WideChar and that's a 2 byte, 16 bit unsigned value, that can have values from 0 to 65535.
You could change all your Char declarations to AnsiChar and all your String declarations to AnsiString, but that's not the way to do it. You should drop Pascal I/O in favor of modern stream-based I/O, use a TFileStream, and don't treat binary data as Char.
Console demo:
program Project26;
{$APPTYPE CONSOLE}
uses SysUtils, Classes;
var F: TFileStream;
Buff: array[0..15] of Byte;
CountRead: Integer;
HexText: array[0..31] of Char;
begin
F := TFileStream.Create('C:\Temp\test', fmOpenRead or fmShareDenyWrite);
try
CountRead := F.Read(Buff, SizeOf(Buff));
while CountRead <> 0 do
begin
BinToHex(Buff, HexText, CountRead);
WriteLn(HexText); // You could add this to the Memo
CountRead := F.Read(Buff, SizeOf(Buff));
end;
finally F.Free;
end;
end.
In Delphi 2009, a Char is the same thing as a WideChar, that is, a Unicode character. A wide character occupies two bytes. You want to use AnsiChar. Prior to Delphi 2009 (that is, prior to Unicode Delphi), Char was the same thing as AnsiChar.
Also, you shouldn't use ReadLn. You are treating the file as a text file with text-file line endings! This is a general file! It might not have any text-file line endings at all!
For an easier to read output, and looking better too, you might want to use this simple hex dump formatter.
The HexDump procedure dumps an area of memory into a TStrings in lines of two chunks of 8 bytes in hex, and 16 ascii chars
example
406563686F206F66 660D0A6966206578 #echo off..if ex
69737420257E7331 5C6E756C20280D0A ist %~s1\nul (..
0D0A290D0A ..)..
Here is the code for the dump format function
function HexB (b: Byte): String;
const HexChar: Array[0..15] of Char = '0123456789ABCDEF';
begin
result:= HexChar[b shr 4]+HexChar[b and $0f];
end;
procedure HexDump(var data; size: Integer; s: TStrings);
const
sepHex=' ';
sepAsc=' ';
nonAsc='.';
var
i : Integer;
hexDat, ascDat : String;
buff : Array[0..1] of Byte Absolute data;
begin
hexDat:='';
ascDat:='';
for i:=0 to size-1 do
begin
hexDat:=hexDat+HexB(buff[i]);
if ((buff[i]>31) and (buff[i]<>255)) then
ascDat:=ascDat+Char(buff[i])
else
ascDat:=ascDat+nonAsc;
if (((i+1) mod 16)<>0) and (((i+1) mod 8)=0) then
hexDat:=hexDat+sepHex;
if ((i+1) mod 16)=0 then
begin
s.Add(hexdat+sepAsc+ascdat);
hexdat:='';
ascdat:='';
end;
end;
if (size mod 16)<>0 then
begin
if (size mod 16)<8 then
hexDat:=hexDat+StringOfChar(' ',(8-(size mod 8))*2)
+sepHex+StringOfChar(' ',16)
else
hexDat:=hexDat+StringOfChar(' ',(16-(size mod 16))*2);
s.Add(hexDat + sepAsc + ascDat);
end;
end;
And here is a complete code example for dumping the contents of a file into a Memo field.
procedure TForm1.Button1Click(Sender: TObject);
var
FStream: TFileStream;
buff: array[0..$fff] of Byte;
nRead: Integer;
begin
FStream := TFileStream.Create(edit1.text, fmOpenRead or fmShareDenyWrite);
try
repeat
nRead := FStream.Read(Buff, SizeOf(Buff));
if nRead<>0 then
hexdump(buff,nRead,memo1.lines);
until nRead=0;
finally
F.Free;
end;
end;
string is UnicodeString in Delphi 2009. If you want to use single-byte strings use AnsiString or RawByteString.
See String types.
I'm using Delphi7 (non-unicode VCL), I need to store lots of WideStrings inside a TFileStream. I can't use TStringStream as the (wide)strings are mixed with binary data, the format is projected to speed up loading and writing the data ... However I believe that current way I'm loading/writing the strings might be a bottleneck of my code ...
currently I'm writing length of a string, then writing it char by char ...
while loading, first I'm loading the length, then loading char by char ...
So, what is the fastest way to save and load WideString to TFileStream?
Thanks in advance
Rather than read and write one character at a time, read and write them all at once:
procedure WriteWideString(const ws: WideString; stream: TStream);
var
nChars: LongInt;
begin
nChars := Length(ws);
stream.WriteBuffer(nChars, SizeOf(nChars);
if nChars > 0 then
stream.WriteBuffer(ws[1], nChars * SizeOf(ws[1]));
end;
function ReadWideString(stream: TStream): WideString;
var
nChars: LongInt;
begin
stream.ReadBuffer(nChars, SizeOf(nChars));
SetLength(Result, nChars);
if nChars > 0 then
stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
end;
Now, technically, since WideString is a Windows BSTR, it can contain an odd number of bytes. The Length function reads the number of bytes and divides by two, so it's possible (although not likely) that the code above will cut off the last byte. You could use this code instead:
procedure WriteWideString(const ws: WideString; stream: TStream);
var
nBytes: LongInt;
begin
nBytes := SysStringByteLen(Pointer(ws));
stream.WriteBuffer(nBytes, SizeOf(nBytes));
if nBytes > 0 then
stream.WriteBuffer(Pointer(ws)^, nBytes);
end;
function ReadWideString(stream: TStream): WideString;
var
nBytes: LongInt;
buffer: PAnsiChar;
begin
stream.ReadBuffer(nBytes, SizeOf(nBytes));
if nBytes > 0 then begin
GetMem(buffer, nBytes);
try
stream.ReadBuffer(buffer^, nBytes);
Result := SysAllocStringByteLen(buffer, nBytes)
finally
FreeMem(buffer);
end;
end else
Result := '';
end;
Inspired by Mghie's answer, have replaced my Read and Write calls with ReadBuffer and WriteBuffer. The latter will raise exceptions if they are unable to read or write the requested number of bytes.
There is nothing special about wide strings, to read and write them as fast as possible you need to read and write as much as possible in one go:
procedure TForm1.Button1Click(Sender: TObject);
var
Str: TStream;
W, W2: WideString;
L: integer;
begin
W := 'foo bar baz';
Str := TFileStream.Create('test.bin', fmCreate);
try
// write WideString
L := Length(W);
Str.WriteBuffer(L, SizeOf(integer));
if L > 0 then
Str.WriteBuffer(W[1], L * SizeOf(WideChar));
Str.Seek(0, soFromBeginning);
// read back WideString
Str.ReadBuffer(L, SizeOf(integer));
if L > 0 then begin
SetLength(W2, L);
Str.ReadBuffer(W2[1], L * SizeOf(WideChar));
end else
W2 := '';
Assert(W = W2);
finally
Str.Free;
end;
end;
WideStrings contain a 'string' of WideChar's, which use 2 bytes each. If you want to store the UTF-16 (which WideStrings use internally) strings in a file, and be able to use this file in other programs like notepad, you need to write a byte order mark first: #$FEFF.
If you know this, writing can look like this:
Stream1.Write(WideString1[1],Length(WideString)*2); //2=SizeOf(WideChar)
reading can look like this:
Stream1.Read(WideChar1,2);//assert returned 2 and WideChar1=#$FEFF
SetLength(WideString1,(Stream1.Size div 2)-1);
Stream1.Read(WideString1[1],(Stream1.Size div 2)-1);
You can also use TFastFileStream for reading the data or strings, I pasted the unit at http://pastebin.com/m6ecdc8c2 and a sample below:
program Project36;
{$APPTYPE CONSOLE}
uses
SysUtils, Classes,
FastStream in 'FastStream.pas';
const
WideNull: WideChar = #0;
procedure WriteWideStringToStream(Stream: TFileStream; var Data: WideString);
var
len: Word;
begin
len := Length(Data);
// Write WideString length
Stream.Write(len, SizeOf(len));
if (len > 0) then
begin
// Write WideString
Stream.Write(Data[1], len * SizeOf(WideChar));
end;
// Write null termination
Stream.Write(WideNull, SizeOf(WideNull));
end;
procedure CreateTestFile;
var
Stream: TFileStream;
MyString: WideString;
begin
Stream := TFileStream.Create('test.bin', fmCreate);
try
MyString := 'Hello World!';
WriteWideStringToStream(Stream, MyString);
MyString := 'Speed is Delphi!';
WriteWideStringToStream(Stream, MyString);
finally
Stream.Free;
end;
end;
function ReadWideStringFromStream(Stream: TFastFileStream): WideString;
var
len: Word;
begin
// Read length of WideString
Stream.Read(len, SizeOf(len));
// Read WideString
Result := PWideChar(Cardinal(Stream.Memory) + Stream.Position);
// Update position and skip null termination
Stream.Position := Stream.Position + (len * SizeOf(WideChar)) + SizeOf(WideNull);
end;
procedure ReadTestFile;
var
Stream: TFastFileStream;
my_wide_string: WideString;
begin
Stream := TFastFileStream.Create('test.bin');
try
Stream.Position := 0;
// Read WideString
my_wide_string := ReadWideStringFromStream(Stream);
WriteLn(my_wide_string);
// Read another WideString
my_wide_string := ReadWideStringFromStream(Stream);
WriteLn(my_wide_string);
finally
Stream.Free;
end;
end;
begin
CreateTestFile;
ReadTestFile;
ReadLn;
end.