In Delphi 10.4.2, when I use the TWriter.WriteString two extra bytes are saved :
var
FileStream: TFileStream;
Writer: TWriter;
begin
FileStream := TFileStream.Create('stream.txt', fmCreate or fmOpenWrite or fmShareDenyNone);
Writer := TWriter.Create(FileStream, $FF);
try
Writer.WriteString('2');
finally
Writer.Free;
FileStream.free;
end
end;
What are these two bytes? How can I ignore them?
This is by design of TWriter.WriteString. Probably your use case is not the correct one.
The first byte ($06) is the value type (TValueType.vaString for your code). The second byte is the length of the string (1 byte for you).
You can find all that information in the source code provided by Embarcadero in file System.Classes.pas.
You cannot ignore them. Maybe you can use TStream.Write to write your string without extra payload?
Related
I have the following setup:
- Windows system language is English.
- I use Delphi 10.1 Berlin.
- In Windows Region & Language/Country set to Japan.
- Region/Administrative/Language for non-Unicode programs set to Japanese (Japan).
I have implemented communication client/server using strings.
Let's skip the question 'why not bytes' for now. I want to show the issue and find the reason why.
I write 2 things into TStringStream:
Header: which includes its size of Int64 (8 bytes), object size of Int64 (8 bytes) and class name: Header length - 2*SizeOf(Int64).
object (TComponent descendant)
procedure ComponentToStream(AComponent: TComponent; AStream: TStream; out HL,OL: Int64);
var
CN: TBytes;
MS1: TMemoryStream;
begin
MS1 := TMemoryStream.Create;
try
CN := TEncoding.Unicode.GetBytes(AComponent.ClassName);
SaveComponentToStream(MS1, AComponent);
OL := MS1.Size;
MS1.Position := 0;
HL := SizeOf(HL) + SizeOf(OL) + Length(CN);
AStream.Write(HL,SizeOf(HL));
AStream.Write(OL,SizeOf(OL));
AStream.Write(CN[0], Length(CN));
MS1.SaveToStream(AStream);
finally
FreeAndNil(MS1);
end;
end;
function PrepareDataBeforeSend(Component: TComponent): string;
var
HL, OL: Int64;
SS: TStringStream;
begin
SS := TStringStream.Create('', TEncoding.Unicode);
try
ComponentToStream(Component, SS, HL, OL);
Result := SS.DataString;
SS.SaveToFile('Orginal stream data.debug');
finally
FreeAndNil(SS);
end;
The result of this method saved in file here
click.
To verify the data I used code below right after calling of one above.
SS := TStringStream.Create({PrepareDataBeforeSend result}, TEncoding.Unicode);
SS.SaveToFile('New stream data.debug');
SS.Free;
Saved binary can be found here Click
And now 2 problems:
If I don't specify explicitly TEncoding.Unicode encoding in constructor of TStringStream, then TEncoding.Default will be used. But for Japanese code page it is ANSII and for English it is Unicode. As a result object size I read later
SS.Read(OL, SizeOf(OL));
is wrong.
Here's the binary to compare. See 8-15 bytes Click
OK, issue 1 was resolved, but still the binary I saved for verification does not match the original one: there is 1 byte missing at the end.
Can anyone tell where is a problem?
Important: there is no issues if I have English localization!!
I was trying to read data from a TBlobField using TADOBlobStream. I wrote the following function
function DecompressBlobFieldCustom(AField:TBlobField):String;
var
BLOBStream:TADOBlobStream;
Size:Integer;
begin
BLOBStream:= TADOBlobStream.Create(AField,bmRead);
Size:= BLOBStream.Size;
BLOBStream.Read(Result,Size);
end;
and i use the function as follows
Data := DecompressBlobFieldCustom(FldImage);
But when i try to do if Length(Data) > 0 then i am getting Access violation at address XXXX error. I couldn't figure out the problem. Please Help.
Result is a string type, but a blob stream operates on byte arrays. It is usually a mistake to try to treat a byte array as though it were a string. Furthermore, you did not allocate a buffer into which to read, which is the actual cause of the error.
Read into a byte array like this:
function ReadBlobField(Field: TBlobField): TBytes;
var
Stream: TStream;
begin
Stream := TADOBlobStream.Create(Field, bmRead);
try
SetLength(Result, Stream.Size);
if Stream.Size>0 then
Stream.ReadBuffer(Result[0], Stream.Size);
finally
Stream.Free;
end;
end;
It is preferable to use the CreateBlobStream method of TDataSet to create blob streams, as discussed in the documentation. So the code would be better like this:
function ReadBlobField(DataSet: TDataSet; Field: TField): TBytes;
var
Stream: TStream;
begin
Stream := DataSet.CreateBlobStream(Field, bmRead);
try
SetLength(Result, Stream.Size);
if Stream.Size>0 then
Stream.ReadBuffer(Result[0], Stream.Size);
finally
Stream.Free;
end;
end;
I've assumed that the data really is a byte array and is not holding text. The fact that it is held in a blob suggests that, as does the mention of decompression and images.
Some other comments:
You must destroy the stream when you are finished with it.
It is generally preferable to use ReadBuffer rather than Read because ReadBuffer raises an exception if the requested number of bytes are not read.
I need of a little help in delphi. I have search in very places but i can't find the answer for my question.
How i can read a file signature(4 bytes in my case) and put the value in HEX into a string? The signature my program will have to identify is $4E4553A1.
I need to test if the file, for example. C:\Happy.bin. Have this signature. To avoid people to put wrong format files in my software. Signature are the first 4 bytes in it.
Thank you so much, english isn't my first language, so sorry for the mistakes. Love you all
This is probably the easiest. You call it and pass in the filename.
function CheckSignature(aFilename: string): Boolean;
var
signature: UInt32;
myFile: TFileStream;
begin
myFile := TFileStream.Create(aFilename, fmOpenRead or fmShareDenyWrite);
try
if myFile.Read(signature, SizeOf(signature)) = SizeOf(signature) then
Result := (signature = $A153454E)
else
Result := False;
finally
myFile.Free;
end;
end;
The signature is reversed because of the way the integer stores it's data (little endian).
To use this function you would call it like this:
begin
if CheckSignature('C:\Happy.bin') then
ShowMessage('Matched')
else
ShowMessage('Didn''t match');
end;
okay, so I (VERY) recently started playing with lazaruz/free pascal, and I'm a little stuck with reading files with TMemoryStream and it's streaming kin.
I'm trying to write a simple base64 encoder, that can encode strings of text, or files (like images and WAVs) to then be used in html and javascript.
The following code compiles great but I get EReadError Illegal stream image when trying to load a file. I'll include the working string only procedure for reference:
procedure TForm1.TextStringChange(Sender: TObject);
begin
Memo1.Lines.Text := EncodeStringBase64(TextString.Text);
end;
procedure TForm1.FormCreate(Sender: TObject);
begin
Memo1.Lines.Text := '';
Form1.BorderIcons := [biSystemMenu,biMinimize];
end;
procedure TForm1.BitBtn1Click(Sender: TObject);
begin
if OpenDialog1.Execute then
begin
filename := OpenDialog1.Filename;
stream := TMemoryStream.Create;
try
StrStream := TStringStream.Create(s);
try
stream.LoadFromFile(filename);
stream.Seek(0, soFromBeginning);
ObjectBinaryToText(stream, StrStream);
StrStream.Seek(0, soFromBeginning);
Memo1.Lines.Text := EncodeStringBase64(StrStream.DataString);
finally
StrStream.Free;
end;
finally
stream.Free;
end;
end;
end;
Can anyone help me out?
You get the "illegal stream image" exception because the file you're loading probably isn't a binary DFM file. That's what ObjectBinaryToText is meant to process. It's not for arbitrary data. So get rid of that command.
You can skip the TMemoryStream, too. TStringStream already has a LoadFromFile method, so you can call it directly instead of involving another buffer.
StrStream.LoadFromFile(filename);
But a string isn't really the right data structure to store your file in prior to base64-encoding it. The input to base64 encoding is binary data; the output is text. Using a text data structure as an intermediate format means you may introduce errors into your data because of difficulties in encoding certain data as valid characters. The right interface for your encoding function is this:
function Base64Encode(Data: TStream): string;
You don't need to load the entire file into memory prior to encoding it. Just open the file with a TFileStream and pass it to your encoding function. Read a few bytes from it at a time with the stream's Read method, encode them as base64, and append them to the result string. (If you find that you need them, you can use an intermediate TStringBuilder for collecting the result, and you can add different buffering around the file reads. Don't worry about those right away, though; get your program working correctly first.)
Use it something like this:
procedure TForm1.BitBtn1Click(Sender: TObject);
var
filename: string;
stream: TStream;
begin
if OpenDialog1.Execute then begin
filename := OpenDialog1.Filename;
stream := TFileStream.Create(filename, fmOpenRead);
try
Memo1.Lines.Text := Base64Encode(stream);
finally
stream.Free;
end;
end;
end;
I never heard before about ObjectBinaryToText(), but looks like funky one. Also, what is EncodeStringBase64() function?
At first place, you shouldn't convert binary stream to text to encode it, instead you should directly B64 encode binary data. B64 algorithm is intended to work on array of bytes.
Since Delphi 6, there is EncdDecd.pas unit, which implements B64 encoding methods. I'm not sure if Lazarus/FPC have this, but if they do, your code to B64 encode file should look like this (add EncdDecd to uses list):
procedure TForm1.Button1Click(Sender: TObject);
var
instream : TFileStream;
outstream: TStringStream;
begin
if OpenDialog1.Execute then
begin
instream := TFileStream.Create(OpenDialog1.FileName, fmOpenRead or fmShareDenyNone);
try
outstream := TStringStream.Create;
try
EncodeStream(instream, outstream);
Memo1.Lines.Text := outstream.DataString;
finally
outstream.Free;
end;
finally
instream.Free;
end;
end;
end;
It's possible to convert the XML to UTF-8 encoding in Delphi 6?
Currently that's what I am doing:
Fill TXMLDocument with AnsiString
At the end convert the Data to UTF-8 by using WideStringVariable = AnsiToUtf8(Doc.XML.Text);
Save the value of WideStringVariable to file using TFileStream and Adding BOM for UTF8 at the file beggining.
CODE:
Procedure SaveAsUTF8( const Name:String; Data: TStrings );
const
cUTF8 = $BFBBEF;
var
W_TXT: WideString;
fs: TFileStream;
wBOM: Integer;
begin
if TRIM(Data.Text) <> '' then begin
W_TXT:= AnsiToUTF8(Data.Text);
fs:= Tfilestream.create( Name, fmCreate );
try
wBOM := cUTF8;
fs.WriteBUffer( wBOM, sizeof(wBOM)-1);
fs.WriteBuffer( W_TXT[1], Length(W_TXT)*Sizeof( W_TXT[1] ));
finally
fs.free
end;
end;
end;
If I open the file in Notepad++ or another editor that detects encoding, it shows me UTF-8 with BOM. However, it seems like the text it's not properly encoded.
What is wrong and how can I fix it?
UPDATE: XML Properties:
XMLDoc.Version := '1.0';
XMLDoc.Encoding := 'UTF-8';
XMLDoc.StandAlone := 'yes';
You can save the file using standard SaveToFile method over the TXMLDocument variable: http://docs.embarcadero.com/products/rad_studio/delphiAndcpp2009/HelpUpdate2/EN/html/delphivclwin32/XMLDoc_TXMLDocument_SaveToFile.html
Whether the file would be or not UTF8 you have to check using local tools like aforementioned Notepad++ or Hex Editor or anything else.
If you insist of using intermediate string and file stream, you should use the proper variable. AnsiToUTF8 returns UTF8String type and that is what to be used.
Compiling `WideStringVar := AnsiStringSource' would issue compiler warning and
It is a proper warning. Googling for "Delphi WideString" - or reading Delphi manuals on topic - shows that WideString aka Microsoft OLE BSTR keeps data in UTF-16 format. http://delphi.about.com/od/beginners/l/aa071800a.htm
Thus assignment UTF16 string <= 8-bit source would necessarily convert data and thus dumping WideString data can not be dumping UTF-8 text by the definition of WideString
Procedure SaveAsUTF8( const Name:String; Data: TStrings );
const
cUTF8: array [1..3] of byte = ($EF,$BB,$BF)
var
W_TXT: UTF8String;
fs: TFileStream;
Trimmed: AnsiString;
begin
Trimmed := TRIM(Data.Text);
if Trimmed <> '' then begin
W_TXT:= AnsiToUTF8(Trimmed);
fs:= TFileStream.Create( Name, fmCreate );
try
fs.WriteBuffer( cUTF8[1], sizeof(cUTF8) );
fs.WriteBuffer( W_TXT[1], Length(W_TXT)*Sizeof( W_TXT[1] ));
finally
fs.free
end;
end;
end;
BTW, this code of yours would not create even empty file if the source data was empty. It looks rather suspicious, though it is you to decide whether that is an error or not wrt the rest of your program.
The proper "uploading" of received file or stream to web is yet another issue (to be put as a separate question on Q&A site like SO), related to testing conformance with HTTP. As a foreword, you can readsome hints at WWW server reports error after POST Request by Internet Direct components in Delphi
In order to have the correct encoding inside the document, you should set it by using the Encoding property in your XML Document, like this:
myXMLDocument.Encoding := 'UTF-8';
I hope this helps.
You simply need to call the SaveToFile method of the document:
XMLDoc.SaveToFile(FileName);
Since you specified the encoding already, the component will use that encoding.
This won't include a BOM, but that's generally what you want for an XML file. The content of the file will specify the encoding.
As regards your SaveAsUTF8 method, it is not needed, but it is easy to fix. And that may be instructive to you.
The problem is that you are converting to UTF-16 when you assign to a WideString variable. You should instead put the UTF-8 text into an AnsiString variable. Changing the type of the variable that you named W_TXT to AnsiString is enough.
The function might look like this:
Procedure SaveAsUTF8(const Name: string; Data: TStrings);
const
UTF8BOM: array [0..2] of AnsiChar = #$EF#$BB#$BF;
var
utf8: AnsiString;
fs: TFileStream;
begin
utf8 := AnsiToUTF8(Data.Text);
fs:= Tfilestream.create(Name, fmCreate);
try
fs.WriteBuffer(UTF8BOM, SizeOf(UTF8BOM));
fs.WriteBuffer(Pointer(utf8)^, Length(utf8));
finally
fs.free;
end;
end;
Another solution:
procedure SaveAsUTF8(const Name: string; Data: TStrings);
var
fs: TFileStream;
vStreamWriter: TStreamWriter;
begin
fs := TFileStream.Create(Name, fmCreate);
try
vStreamWriter := TStreamWriter.Create(fs, TEncoding.UTF8);
try
vStreamWriter.Write(Data.Text);
finally
vStreamWriter.Free;
end;
finally
fs.free;
end;
end;