I have followed those questions and answers to try to clear a String from memory:
Delphi : how to completely remove String from the memory
Creating string from byte array
In my case I did something like this:
function generateEncryption(Textdata : String) : String;
var
SaltB, IVb, Padtext, keybytes, datas, SaltIVMessage : TBytes;
Cipher: TDCP_rijndael;
keysize, TextDatasize, saltivmsglen : Integer;
OutData, keygen : String;
begin
keygen := 'MyTargetedKey';
keybytes := TEncoding.UTF8.GetBytes(keygen);
Cipher := TDCP_rijndael.Create(nil);
try
cipher.CipherMode := cmCBC;
cipher.Algorithm := EncryptionType;
cipher.BlockSize := 16;
Cipher.Init(keybytes[0], keysize, #IVb[0]);
TextDatasize := high(PadText) + 1;
SetLength(datas, TextDatasize);
Cipher.EncryptCBC(PadText[0], datas[0], TextDatasize);
saltivmsglen := Length(SaltB) + Length(IVb) + Length(datas);
SetLength(SaltIVMessage, saltivmsglen);
SaltIVMessage := AddSaltIVMessage(SaltB, IVb, datas);
OutData := TNetEncoding.Base64.EncodeBytesToString(SaltIVMessage);
Result := OutData;
finally
cipher.Burn;
cipher.Free;
// No success
FillChar(keybytes, Sizeof(keybytes), 0);
// No success either
//ZeroMemory(#keybytes[0], Length(keybytes));
//FillChar(keybytes, Length(keybytes), 0);
//keybytes := nil;
// Still no success
//UniqueString(keygen);
//ZeroMemory(Pointer(keygen), Length(keygen)*SizeOf(Char));
//keygen:='';
end;
end;
I used the GitHub repository "strings2" to extract the String from the executable to read its Strings in the memory. I can see MyTargetedKey as following:
:TNetEncoding.:2
System.NetEncoding
0123456789ABCDEF
MyTargetedKey
In my code I tried different things without success (see comments). What is the proper way to totally clear a String from memory to avoid the capture of my app key, or how to make it harder to capture it?
Related
I started to interface yubicos fido2.dll in Delphi and was able to interface it
according to the provided examples. Now I want to go a step further and use the
dll on an e.g. apache server to handle credential creation and assertion.
So.. for this I basically use the javascripts found on THE testing site https://webauthn.io/
Basically I wanted to mimic some server functions for credential creation. On the website one can setup
some properties - in my environment they come from the server.
Currently I have made the communication from Client to issue a credential initialization - server responses with a challange. The Key is queried and the browser creates credentials and sends it back to the server. This though where I have a problem with the data coming from the server aka I have a
problem decoding the attestationObject part.
So here is the credential init json from my server:
{"publicKey":{"challenge":"LFJYIdXJfYpB1GZS+PzEOD8DNcYmdia4mZp2z0J4QcE=","pubKeyCredParams":[{"alg":-7,"type":"public-key"},{"alg":-257,"type":"public-key"},{"alg":-8,"type":"public-key"}],"authenticatorSelection":{"authenticatorAttachment":"cross-platform","userVerification":"required","requireResidentKey":false},"rp":{"id":"fidotest.com","name":"fidotest.com"},"user":{"id":"zVOUjBCxJNIbSSWNiGOv2\/kZP2UU8pPguVylFeiw4HE=","displayName":"test","name":"test"},"Timeout":60000,"attestation":"direct"}}
And the result from the server:
{"id":"MfcgyBxDxpq5S71fB45FFjecCGtvCepvb6IZexJpgaHyTPPsaz0srQyZc26HkE92eda7a2PmPIzvSpLbipktmw","rawId":"MfcgyBxDxpq5S71fB45FFjecCGtvCepvb6IZexJpgaHyTPPsaz0srQyZc26HkE92eda7a2PmPIzvSpLbipktmw","type":"public-key","response":{"attestationObject":"o2NmbXRmcGFja2VkZ2F0dFN0bXSjY2FsZyZjc2lnWEcwRQIhAO8fbE8iQcMFYE4KBwL6HK6OxSReRKriXZDWhcfGRMFxAiB7mIPZ7n-fWas7aWkEkWd-9CWvd8ncRVCh3BBFIzMuRmN4NWOBWQLAMIICvDCCAaSgAwIBAgIEBMX-_DANBgkqhkiG9w0BAQsFADAuMSwwKgYDVQQDEyNZdWJpY28gVTJGIFJvb3QgQ0EgU2VyaWFsIDQ1NzIwMDYzMTAgFw0xNDA4MDEwMDAwMDBaGA8yMDUwMDkwNDAwMDAwMFowbTELMAkGA1UEBhMCU0UxEjAQBgNVBAoMCVl1YmljbyBBQjEiMCAGA1UECwwZQXV0aGVudGljYXRvciBBdHRlc3RhdGlvbjEmMCQGA1UEAwwdWXViaWNvIFUyRiBFRSBTZXJpYWwgODAwODQ3MzIwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQc2Np2EaP17x-IXpULpl2A4zSFU5FYS9R_W3GcUyNcJCHk45m9tXNngkGQk1dmYUk8kUwuZyTfk5T8-n3qixgEo2wwajAiBgkrBgEEAYLECgIEFTEuMy42LjEuNC4xLjQxNDgyLjEuMTATBgsrBgEEAYLlHAIBAQQEAwIFIDAhBgsrBgEEAYLlHAEBBAQSBBD4oBHzjApNFYAGFxEfntx9MAwGA1UdEwEB_wQCMAAwDQYJKoZIhvcNAQELBQADggEBAHcYTO91LRoF8wpThdwthvj6wGNxcLAiYqUZXPX-0Db-AGVODSkVvEVSmj-JXmrBzNQel3FW4AupOgbgrJmmcWWEBZyXSpRQtYcl2LTNU0-Iz9WbyHNN1wQJ9ybFwj608xBuoNRC0rG8wgYbMC4usyRadt3dYOVdQi0cfaksVB2VNKnw-ttQUWKoZsPHtuzFx8NlazLQBep1W2T0FCONFEG7x_l-ZcfNhT13azAbaurJ2J0_ff6H0PXJP6h-Obne4xfz0-8ujftWDUSh9oaiVRYf-tgam_tzOKyEU38V2liV11zMyHKWrXiK0AfyDgb58ky2HSrn_AgE5MW_oXg_CXdoYXV0aERhdGFYxNNxx2kdwL5GE4VmZm0_PerRaSEQdriBtCqmPBobyJXTRQAAAA34oBHzjApNFYAGFxEfntx9AEAx9yDIHEPGmrlLvV8HjkUWN5wIa28J6m9vohl7EmmBofJM8-xrPSytDJlzboeQT3Z51rtrY-Y8jO9KktuKmS2bpQECAyYgASFYIPVUDt7LCfuPyhdowBAhHCaRp-4acTmevkowvQhYxQluIlggCOL0rfuXgGze8yGX38sBXzsSMqxQxiskjsXia6UQvtQ","clientDataJSON":"eyJjaGFsbGVuZ2UiOiJMRkpZSWRYSmZZcEIxR1pTLVB6RU9EOEROY1ltZGlhNG1acDJ6MEo0UWNFIiwiY2xpZW50RXh0ZW5zaW9ucyI6e30sImhhc2hBbGdvcml0aG0iOiJTSEEtMjU2Iiwib3JpZ2luIjoiaHR0cHM6Ly9maWRvdGVzdC5jb20iLCJ0eXBlIjoid2ViYXV0aG4uY3JlYXRlIn0"}}
Here is a console program that should verify the credentials:
(note you need to paste the above content to a textfile and load it in the console).
program VerifyCred;
{$APPTYPE CONSOLE}
uses
SysUtils,
Fido2 in '..\Fido2.pas',
Fido2dll in '..\Fido2dll.pas',
Fido2Json in '..\Fido2Json.pas',
windows,
classes,
cbor,
superobject;
function DoVerifyCred( credential : ISuperObject ) : string;
var clientData : ISuperObject;
s : string;
rawS : RawByteString;
credentialId : string;
rawId : TBytes;
credVerify : TFidoCredVerify;
cborItem : TCborMap;
sig : TBytes;
x5c : TBytes;
authData : TBytes;
fmt : string;
alg : integer;
i : integer;
aName : string;
res : boolean;
rawChallange : RawByteString;
credFMT : TFidoCredentialFmt;
challenge : TFidoChallenge;
authDataObj : TAuthData;
attStmt : TCborMap;
j : integer;
restBuf : TBytes;
begin
Result := '{"error":0,"msg":"Error parsing content"}';
s := credential.S['response.clientDataJSON'];
if s = '' then
exit;
ClientData := So( String(Base64URLDecode( s )) );
if clientData = nil then
exit;
rawChallange := Base64URLDecode(ClientData.S['challenge']);
if Length(rawChallange) <> sizeof(challenge) then
exit;
Move( rawChallange[1], challenge, sizeof(challenge));
clientData := SO( String( Base64URLDecode( credential.S['response.clientDataJSON'] ) ) );
s := credential.S['response.attestationObject'];
if s = '' then
exit;
// attestation object is a cbor encoded raw base64ulr encoded string
cborItem := TCborDecoding.DecodeBase64UrlEx(s, restBuf) as TCborMap;
if not Assigned(cborItem) then
exit;
try
alg := 0;
fmt := '';
sig := nil;
authData := nil;
x5c := nil;
for i := 0 to cborItem.Count - 1 do
begin
assert( cborItem.Names[i] is TCborUtf8String, 'CBOR type error');
aName := String((cborItem.Names[i] as TCborUtf8String).Value);
if SameText(aName, 'attStmt') then
begin
attStmt := cborItem.Values[i] as TCborMap;
for j := 0 to attStmt.Count - 1 do
begin
aName := String((attStmt.Names[j] as TCborUtf8String).Value);
if SameText(aName, 'alg')
then
alg := (attStmt.Values[j] as TCborNegIntItem).value
else if SameText(aName, 'sig')
then
sig := (attStmt.Values[j] as TCborByteString).ToBytes
else if SameText(aName, 'x5c')
then
x5c := ((attStmt.Values[j] as TCborArr)[0] as TCborByteString).ToBytes
end;
end
else if SameText(aName, 'authData')
then
authData := (cborItem.Values[i] as TCborByteString).ToBytes
else if SameText(aName, 'fmt')
then
fmt := String( (cborItem.Values[i] as TCborUtf8String).Value );
end;
finally
cborItem.Free;
end;
// check if anyhing is in place
if not (( alg = COSE_ES256 ) or (alg = COSE_EDDSA) or (alg= COSE_RS256)) then
raise Exception.Create('Unknown algorithm');
if Length(sig) = 0 then
raise Exception.Create('No sig field provided');
if Length(x5c) = 0 then
raise Exception.Create('No certificate');
if Length(authData) = 0 then
raise Exception.Create('Missing authdata');
credentialId := credential.S['id'];
s := credential.S['rawId'];
if s = '' then
raise Exception.Create('No Credential id found');
raws := Base64URLDecode( s );
SetLength( rawId, Length(rawS));
Move( rawS[1], rawId[0], Length(rawId));
authDataObj := nil;
if Length(restBuf) > 0 then
raise Exception.Create('Damend there is a rest buffer that should not be');
if Length(authDAta) > 0 then
authDataObj := TAuthData.Create( authData );
try
if fmt = 'packed'
then
credFmt := fmFido2
else if fmt = 'fido-u2f'
then
credFmt := fmU2F
else
credFmt := fmDef;
// now... verify the credentials
credVerify := TFidoCredVerify.Create( TFidoCredentialType(alg), credFmt,
authData, x5c, sig, FidoServer.RequireResidentKey,
authDataObj.UserVerified, 0) ;
try
// auth data seems bad!
res := credVerify.Verify(challenge);
if res then
begin
// -> save EVERYTHING to a database
end;
finally
credVerify.Free;
end;
finally
authDataObj.Free;
end;
// build result and generate a session
if res then
begin
// yeeeha we got a
Result := '{"success":true}';
end
else
Result := '{"success":false}';
end;
var credential: ISuperObject;
begin
try
with TStringList.Create do
try
LoadFromFile('D:\CredVerify_delphi.json');
credential := SO( Text );
finally
Free;
end;
Writeln( doVerifyCred(credential) );
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
end.
The cbor and fido2 projects can be found on github.
I actually have a problem with the CBOR encoded attestationObject returned. If the resident key property is set the
attestation object is only 63 bytes long - and there are bytes left that are actually not encoded. So...
The cbor decoding there either fails or I get data back that does not conform the webauthn attestation object which should at
these positions return the credential id and the public key (which is then also cbor encoded). If the resident key property is false
as this is the case in the above statement the fido dll returns bad auth data. So... anyone has a clue what I'm doing wrong?
It should basically look like in the diagram but it ends either after 63 bytes which is in the mids of the credential id or it fails in the dll.
Turned out I misused the indy base64 routines. The base64 decoder did not work properly for a Unicodstring (I assumed that the string was converted to an ansistring...)
So instead I use the following decoder:
function Base64Decode( s : string ) : RawByteString;
var aWrapStream : TWrapMemoryStream;
sconvStr : UTF8String;
lStream : TMemoryStream;
begin
sConvStr := UTF8String( s );
aWrapStream := TWrapMemoryStream.Create( #sConvStr[1], Length(sConvStr) );
lStream := TMemoryStream.Create;
try
with TIdDecoderMIME.Create(nil) do
try
DecodeBegin(lStream);
Decode( aWrapStream );
DecodeEnd;
SetLength(Result, lStream.Size );
if lStream.Size > 0 then
Move( PByte(lStream.Memory)^, Result[1], lStream.Size);
finally
Free;
end;
finally
lStream.Free;
end;
aWrapStream.Free;
end;
I need to read the last line in some very large textfiles (to get the timestamp from the data). TStringlist would be a simple approach but it returns an out of memory error. I'm trying to use seek and blockread, but the characters in the buffer are all nonsense. Is this something to do with unicode?
Function TForm1.ReadLastLine2(FileName: String): String;
var
FileHandle: File;
s,line: string;
ok: 0..1;
Buf: array[1..8] of Char;
k: longword;
i,ReadCount: integer;
begin
AssignFile (FileHandle,FileName);
Reset (FileHandle); // or for binary files: Reset (FileHandle,1);
ok := 0;
k := FileSize (FileHandle);
Seek (FileHandle, k-1);
s := '';
while ok<>1 do begin
BlockRead (FileHandle, buf, SizeOf(Buf)-1, ReadCount); //BlockRead ( var FileHandle : File; var Buffer; RecordCount : Integer {; var RecordsRead : Integer} ) ;
if ord (buf[1]) <>13 then //Arg to integer
s := s + buf[1]
else
ok := ok + 1;
k := k-1;
seek (FileHandle,k);
end;
CloseFile (FileHandle);
// Reverse the order in the line read
setlength (line,length(s));
for i:=1 to length(s) do
line[length(s) - i+1 ] := s[i];
Result := Line;
end;
Based on www.delphipages.com/forum/showthread.php?t=102965
The testfile is a simple CSV I created in excel ( this is not the 100MB I ultimately need to read).
a,b,c,d,e,f,g,h,i,j,blank
A,B,C,D,E,F,G,H,I,J,blank
1,2,3,4,5,6,7,8,9,0,blank
Mary,had,a,little,lamb,His,fleece,was,white,as,snow
And,everywhere,that,Mary,went,The,lamb,was,sure,to,go
You really have to read the file in LARGE chunks from the tail to the head.
Since it is so large it does not fit the memory - then reading it line by line from start to end would be very slow. With ReadLn - twice slow.
You also has to be ready that the last line might end with EOL or may not.
Personally I would also account for three possible EOL sequences:
CR/LF aka #13#10=^M^J - DOS/Windows style
CR without LF - just #13=^M - Classic MacOS file
LF without CR - just #10=^J - UNIX style, including MacOS version 10
If you are sure your CSV files would only ever be generated by native Windows programs it would be safe to assume full CR/LF be used. But if there can be other Java programs, non-Windows platforms, mobile programs - I would be less sure. Of course pure CR without LF would be the least probable case of them all.
uses System.IOUtils, System.Math, System.Classes;
type FileChar = AnsiChar; FileString = AnsiString; // for non-Unicode files
// type FileChar = WideChar; FileString = UnicodeString;// for UTF16 and UCS-2 files
const FileCharSize = SizeOf(FileChar);
// somewhere later in the code add: Assert(FileCharSize = SizeOf(FileString[1]);
function ReadLastLine(const FileName: String): FileString; overload; forward;
const PageSize = 4*1024;
// the minimal read atom of most modern HDD and the memory allocation atom of Win32
// since the chances your file would have lines longer than 4Kb are very small - I would not increase it to several atoms.
function ReadLastLine(const Lines: TStringDynArray): FileString; overload;
var i: integer;
begin
Result := '';
i := High(Lines);
if i < Low(Lines) then exit; // empty array - empty file
Result := Lines[i];
if Result > '' then exit; // we got the line
Dec(i); // skip the empty ghost line, in case last line was CRLF-terminated
if i < Low(Lines) then exit; // that ghost was the only line in the empty file
Result := Lines[i];
end;
// scan for EOLs in not-yet-scanned part
function FindLastLine(buffer: TArray<FileChar>; const OldRead : Integer;
const LastChunk: Boolean; out Line: FileString): boolean;
var i, tailCRLF: integer; c: FileChar;
begin
Result := False;
if Length(Buffer) = 0 then exit;
i := High(Buffer);
tailCRLF := 0; // test for trailing CR/LF
if Buffer[i] = ^J then begin // LF - single, or after CR
Dec(i);
Inc(tailCRLF);
end;
if (i >= Low(Buffer)) and (Buffer[i] = ^M) then begin // CR, alone or before LF
Inc(tailCRLF);
end;
i := High(Buffer) - Max(OldRead, tailCRLF);
if i - Low(Buffer) < 0 then exit; // no new data to read - results would be like before
if OldRead > 0 then Inc(i); // the CR/LF pair could be sliced between new and previous buffer - so need to start a bit earlier
for i := i downto Low(Buffer) do begin
c := Buffer[i];
if (c=^J) or (c=^M) then begin // found EOL
SetString( Line, #Buffer[i+1], High(Buffer) - tailCRLF - i);
exit(True);
end;
end;
// we did not find non-terminating EOL in the buffer (except maybe trailing),
// now we should ask for more file content, if there is still left any
// or take the entire file (without trailing EOL if any)
if LastChunk then begin
SetString( Line, #Buffer[ Low(Buffer) ], Length(Buffer) - tailCRLF);
Result := true;
end;
end;
function ReadLastLine(const FileName: String): FileString; overload;
var Buffer, tmp: TArray<FileChar>;
// dynamic arrays - eases memory management and protect from stack corruption
FS: TFileStream; FSize, NewPos: Int64;
OldRead, NewLen : Integer; EndOfFile: boolean;
begin
Result := '';
FS := TFile.OpenRead(FileName);
try
FSize := FS.Size;
if FSize <= PageSize then begin // small file, we can be lazy!
FreeAndNil(FS); // free the handle and avoid double-free in finally
Result := ReadLastLine( TFile.ReadAllLines( FileName, TEncoding.ANSI ));
// or TEncoding.UTF16
// warning - TFIle is not share-aware, if the file is being written to by another app
exit;
end;
SetLength( Buffer, PageSize div FileCharSize);
OldRead := 0;
repeat
NewPos := FSize - Length(Buffer)*FileCharSize;
EndOfFile := NewPos <= 0;
if NewPos < 0 then NewPos := 0;
FS.Position := NewPos;
FS.ReadBuffer( Buffer[Low(Buffer)], (Length(Buffer) - OldRead)*FileCharSize);
if FindLastLine(Buffer, OldRead, EndOfFile, Result) then
exit; // done !
tmp := Buffer; Buffer := nil; // flip-flop: preparing to broaden our mouth
OldRead := Length(tmp); // need not to re-scan the tail again and again when expanding our scanning range
NewLen := Min( 2*Length(tmp), FSize div FileCharSize );
SetLength(Buffer, NewLen); // this may trigger EOutOfMemory...
Move( tmp[Low(tmp)], Buffer[High(Buffer)-OldRead+1], OldRead*FileCharSize);
tmp := nil; // free old buffer
until EndOfFile;
finally
FS.Free;
end;
end;
PS. Note one extra special case - if you would use Unicode chars (two-bytes ones) and would give odd-length file (3 bytes, 5 bytes, etc) - you would never be ble to scan the starting single byte (half-widechar). Maybe you should add the extra guard there, like Assert( 0 = FS.Size mod FileCharSize)
PPS. As a rule of thumb you better keep those functions out of the form class, - because WHY mixing them? In general you should separate concerns into small blocks. Reading file has nothing with user interaction - so should better be offloaded to an extra UNIT. Then you would be able to use functions from that unit in one form or 10 forms, in main thread or in multi-threaded application. Like LEGO parts - they give you flexibility by being small and separate.
PPPS. Another approach here would be using memory-mapped files. Google for MMF implementations for Delphi and articles about benefits and problems with MMF approach. Personally I think rewriting the code above to use MMF would greatly simplify it, removing several "special cases" and the troublesome and memory copying flip-flop. OTOH it would demand you to be very strict with pointers arithmetic.
https://en.wikipedia.org/wiki/Memory-mapped_file
https://msdn.microsoft.com/en-us/library/ms810613.aspx
http://torry.net/quicksearchd.php?String=memory+map&Title=No
Your char type is two byte, so that buffer is 16 byte. Then with blockread you read sizeof(buffer)-1 byte into it, and check the first 2 byte char if it is equal to #13.
The sizeof(buffer)-1 is dodgy (where does that -1 come from?), and the rest is valid, but only if your input file is utf16.
Also your read 8 (or 16) characters each time, but compare only one and then do a seek again. That is not very logical either.
If your encoding is not utf16, I suggest you change the type of a buffer element to ansichar and remove the -1
In response to kopiks suggestion, I figured out how to do it with TFilestream, it works ok with the simple test file, though there may be some further tweeks when I use it on a variety of csv files. Also, I don't make any claims that this is the most efficient method.
procedure TForm1.Button6Click(Sender: TObject);
Var
StreamSize, ApproxNumRows : Integer;
TempStr : String;
begin
if OpenDialog1.Execute then begin
TempStr := ReadLastLineOfTextFile(OpenDialog1.FileName,StreamSize, ApproxNumRows);
// TempStr := ReadFileStream('c:\temp\CSVTestFile.csv');
ShowMessage ('approximately '+ IntToStr(ApproxNumRows)+' Rows');
ListBox1.Items.Add(TempStr);
end;
end;
Function TForm1.ReadLastLineOfTextFile(const FileName: String; var StreamSize, ApproxNumRows : Integer): String;
const
MAXLINELENGTH = 256;
var
Stream: TFileStream;
BlockSize,CharCount : integer;
Hash13Found : Boolean;
Buffer : array [0..MAXLINELENGTH] of AnsiChar;
begin
Hash13Found := False;
Result :='';
Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
StreamSize := Stream.size;
if StreamSize < MAXLINELENGTH then
BlockSize := StreamSize
Else
BlockSize := MAXLINELENGTH;
// for CharCount := 0 to Length(Buffer)-1 do begin
// Buffer[CharCount] := #0; // zeroing the buffer can aid diagnostics
// end;
CharCount := 0;
Repeat
Stream.Seek(-(CharCount+3), 2); //+3 misses out the #0,#10,#13 at the end of the file
Stream.Read( Buffer[CharCount], 1);
Result := String(Buffer[CharCount]) + result;
if Buffer[CharCount] =#13 then
Hash13Found := True;
Inc(CharCount);
Until Hash13Found OR (CharCount = BlockSize);
ShowMessage(Result);
ApproxNumRows := Round(StreamSize / CharCount);
end;
Just thought of a new solution.
Again, there could be better ones, but this one is the best i thought of.
function GetLastLine(textFilePath: string): string;
var
list: tstringlist;
begin
list := tstringlist.Create;
try
list.LoadFromFile(textFilePath);
result := list[list.Count-1];
finally
list.free;
end;
end;
I have been hounded this problem for a few days, I have two cliendatasets with data in them and I want to convert the olevariant data to string using two functions I found here in Stack Overflow.
The purpose of conversion to string is to be able to transfer the string to another location and convert it back again to olevariant and assign it to another clientdataset.
To simulate it, I created a sample app with the following partial code(see block below).
The code executes properly but my problem is when I convert the windows locale to japanese(which is the requirement), I encounter a datapacket mismatch in the data assignment on the second dataset. but if I do this in the japanese locale:
clientdataset2.data := clientdataset1.data
it works fine. English locale, the code works just fine.
Is there a problem in the string conversion? or is there anything I can do? I really would appreciate help with this.
//to simulate the conversion
TempData := ClientDataSet1.Data;
TempString := OleVariantToString(ClientDataset1.Data);
TempData2 := StringToOleVariant(TempString);
ClientDataSet2.Data := TempData2; //mismatch in data packet happens here in japanese locale
//conversion functions
function TForm1.OleVariantToString(const Value: OleVariant): string;
var
ss: TStringStream;
Size: integer;
Data: PByteArray;
begin
Result := '';
if Length(Value) = 0 then
Exit;
ss := TStringStream.Create;
try
Size := VarArrayHighBound(Value, 1) - VarArrayLowBound(Value, 1) + 1;
Data := VarArrayLock(Value);
try
ss.Position := 0;
ss.WriteBuffer(Data^, Size);
ss.Position := 0;
Result := ss.DataString;
finally
VarArrayUnlock(Value);
end;
finally
ss.Free;
end;
end;
function TForm1.StringToOleVariant(const Value: string): OleVariant;
var
ss: TStringStream;
MyBuffer: Pointer;
begin
Result := null;
if Value = '' then
Exit;
ss := TStringStream.Create(Value);
try
Result := VarArrayCreate([0, ss.Size - 1], varByte);
MyBuffer := VarArrayLock(Result);
try
ss.Position := 0;
ss.ReadBuffer(MyBuffer^, ss.Size);
finally
VarArrayUnlock(Result);
end;
finally
ss.Free;
end;
end;
Streaming to string is already implemented, you can use
Writing: TClientDataSet.SaveToFile or TClientDataSet.SaveToStream
Reading: TClientDataSet.LoadFromFile or TClientDataSet.LoadFromStream
procedure SaveToStream(Stream: TStream; Format: TDataPacketFormat = dfBinary);
procedure SaveToFile(const FileName: string = ''; Format: TDataPacketFormat = fBinary);
procedure LoadFromStream(Stream: TStream);
procedure LoadFromFile(const FileName: string = '');
the TDataPacketFormat options are:
dfBinary: Information is encoded in binary format.
dfXML:Information is encoded in XML, with extended characters encoded using an escape sequence.
dfXMLUTF8:Information is encoded in XML, with extended characters represented using UTF8.
Using dfXMLUTF8 you should have no problems with non/ansi characters sets.
for write something in a file i use for example this code:
procedure MyProc (... );
const
BufSize = 65535;
var
FileSrc, FileDst: TFileStream;
StreamRead: Cardinal;
InBuf, OutBuf: Array [0..bufsize] of byte;
begin
.....
FileSrc := TFileStream.Create (uFileSrc, fmOpenRead Or fmShareDenyWrite);
try
FileDst := TFileStream.Create (uFileTmp, fmCreate);
try
StreamRead := 0;
while ((iCounter < iFileSize) or (StreamRead = Cardinal(BufSize)))
begin
StreamRead := FileSrc.Read (InBuf, BufSize);
Inc (iCounter, StreamRead);
end;
finally
FileDst.Free;
end;
finally
FileSrc.Free;
end;
end;
And for I/O file i use a array of byte, and so is all ok, but when i use a string, for example declaring:
InBuf, OutBuf: string // in delphi xe2 = unicode string
then not work. In sense that file not write nothing. I have understood why, or just think to have understood it.
I think that problem maybe is why string contain just a pointer to memory and not static structure; correct?
In this case, there is some solution for solve it? In sense, is possible to do something for i can to write a file using string and not vector? Or i need necessary use a vector?
If possible, can i can to do ?
Thanks very much.
There are two issues with using strings. First of all you want to use RawByteString so that you ensure the use of byte sized character elements – a Unicode string has elements that are two bytes wide. And secondly you need to dereference the string which is really just a pointer.
But I wonder why you would prefer strings to the stack allocated byte array.
procedure MyProc (... );
const
BufSize = 65536;
var
FileSrc, FileDst: TFileStream;
StreamRead: Cardinal;
InBuf: RawByteString;
begin
.....
FileSrc := TFileStream.Create (uFileSrc, fmOpenRead Or fmShareDenyWrite);
try
FileDst := TFileStream.Create (uFileTmp, fmCreate);
try
SetLength(InBuf, BufSize);
StreamRead := 0;
while ((iCounter < iFileSize) or (StreamRead = Cardinal(BufSize)))
begin
StreamRead := FileSrc.Read (InBuf[1], BufSize);
Inc (iCounter, StreamRead);
end;
finally
FileDst.Free;
end;
finally
FileSrc.Free;
end;
end;
Note: Your previous code declared a buffer of 65536 bytes, but you only ever used 65535 of them. Probably not what you intended.
To use a string as a buffer (which I would not recommend), you'll have to use SetLength to allocate the internal buffer, and you'll have to pass InBuf[1] and OutBuf[1] as the data to read or write.
var
InBuf, OutBuf: AnsiString; // or TBytes
begin
SetLength(InBuf, BufSize);
SetLength(OutBuf, BufSize);
...
StreamRead := FileSrc.Read(InBuf[1], BufSize); // if TBytes, use InBuf[0]
// etc...
You can also use a TBytes, instead of an AnsiString. The usage remains the same.
But I actually see no advantage in dynamically allocating TBytes, AnsiStrings or RawByteStrings here. I'd rather do what you already do: use a stack based buffer. I would perhaps make it a little smaller in a multi-threaded environment.
Yes, you can save / load strings to / from stream, see the following example
var Len: Integer;
buf: string;
FData: TStream;
// save string to stream
// save the length of the string
Len := Length(buf);
FData.Write(Len, SizeOf(Len));
// save string itself
if(Len > 0)then FData.Write(buf[1], Len * sizeof(buf[1]));
// read string from stream
// read the length of the string
FData.Read(Len, SizeOf(Len));
if(Len > 0)then begin
// get memory for the string
SetLength(buf, Len);
// read string content
FData.Read(buf[1], Len * sizeof(buf[1]));
end else buf := '';
On a related note, to copy the contents from one TStream to another TStream, you could just use the TStream.CopyFrom() method instead:
procedure MyProc (... );
var
FileSrc, FileDst: TFileStream;
begin
...
FileSrc := TFileStream.Create (uFileSrc, fmOpenRead Or fmShareDenyWrite);
try
FileDst := TFileStream.Create (uFileTmp, fmCreate);
try
FileDst.CopyFrom(FileSrc, 0); // or FileDst.CopyFrom(FileSrc, iFileSize)
finally
FileDst.Free;
end;
finally
FileSrc.Free;
end;
...
end;
Which can be simplified by calling CopyFile() instead:
procedure MyProc (... );
begin
...
CopyFile(PChar(uFileSrc), PChar(uFileTmp), False);
...
end;
Either way, you don't have to worry about read/writing the file data manually at all!
I am trying to remotely read a binary (REG_BINARY) registry value, but I get nothing but junk back. Any ideas what is wrong with this code? I'm using Delphi 2010:
function GetBinaryRegistryData(ARootKey: HKEY; AKey, AValue, sMachine: string; var sResult: string): boolean;
var
MyReg: TRegistry;
RegDataType: TRegDataType;
DataSize, Len: integer;
sBinData: string;
bResult: Boolean;
begin
bResult := False;
MyReg := TRegistry.Create(KEY_QUERY_VALUE);
try
MyReg.RootKey := ARootKey;
if MyReg.RegistryConnect('\\' + sMachine) then
begin
if MyReg.KeyExists(AKey) then
begin
if MyReg.OpenKeyReadOnly(AKey) then
begin
try
RegDataType := MyReg.GetDataType(AValue);
if RegDataType = rdBinary then
begin
DataSize := MyReg.GetDataSize(AValue);
if DataSize > 0 then
begin
SetLength(sBinData, DataSize);
Len := MyReg.ReadBinaryData(AValue, PChar(sBinData)^, DataSize);
if Len <> DataSize then
raise Exception.Create(SysErrorMessage(ERROR_CANTREAD))
else
begin
sResult := sBinData;
bResult := True;
end;
end;
end;
except
MyReg.CloseKey;
end;
MyReg.CloseKey;
end;
end;
end;
finally
MyReg.Free;
end;
Result := bResult;
end;
And I call it like this:
GetBinaryRegistryData(
HKEY_LOCAL_MACHINE,
'\SOFTWARE\Microsoft\Windows NT\CurrentVersion',
'DigitalProductId', '192.168.100.105',
sProductId
);
WriteLn(sProductId);
The result I receive from the WriteLn on the console is:
ñ ♥ ???????????6Z ????1 ???????☺ ???♦ ??3 ? ??? ?
??
Assuming that you are already connected remotely, try using the GetDataAsString function
to read binary data from the registry.
sResult := MyReg.GetDataAsString(AValue);
You're using Delphi 2010, so all your characters are two bytes wide. When you set the length of your result string, you're allocating twice the amount of space you need. Then you call ReadBinaryData, and it fills half your buffer. There are two bytes of data in each character. Look at each byte separately, and you'll probably find that your data looks less garbage-like.
Don't use strings for storing arbitrary data. Use strings for storing text. To store arbitrary blobs of data, use TBytes, which is an array of bytes.