I need to hash a string, preferably as SHA512, although it could be SHA256, SHA1, MD5 or CRC32.
I have downloaded Lockbox 3, put a TCryptographicLibrary and a THash component on a form, set the Hash property to SHA-512 and used the following code to produce a test result:
procedure TForm1.Button1Click(Sender: TObject);
begin
Hash1.HashString('myhashtest');
Edit1.Text := Stream_To_AnsiString(Hash1.HashOutputValue);
end;
To best illustrate the problem, I have gone on to an online hash calculator and the MD5 hash of 'myhashtest' is ff91e22313f0a41b46719e7ee6f99451 but setting the hash property in my test program to MD5 results in ÿ‘â#ð¤Fqž~æù”Q which is clearly wrong. I have tried the same test using other Hash properties, including the SHA512 which i want, and they all return rubbish.
Where am I going wrong?
THash.HashOutputValue is a stream of the raw hashed bytes. It appears that Stream_To_AnsiString() merely copies those raw bytes as-is into an AnsiString, it does not encode the bytes in any way. What you are looking for is the hex encoded version of the raw bytes instead. I do know that LockBox has a Stream_To_Base64() function (as shown in this example), but I do not know if it has a Stream_To_Hex() type of function. If it does not, you can easily create your own, eg:
function Stream_To_Hex(Stream: TStream): AnsiString;
var
NumBytes, I: Integer;
B: Byte;
begin
NumBytes := Stream.Size - Stream.Position;
SetLength(Result, NumBytes * 2);
for I := 0 to NumBytes-1 do
begin
Stream.ReadBuffer(B, 1);
BinToHex(#B, #Result[(I*2)+1], 1);
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
Hash1.HashString('myhashtest');
Edit1.Text := Stream_To_Hex(Hash1.HashOutputValue);
end;
Many cryptographic functions 'silently' (i.e. without stating so in the docs) output and require Base64- or hex-encoded strings (and also often AnsiStrings). This is because encrypted text can contain any data, and as soon as you start treating that as 'strings', string handling functions can easily choke on that (e.g. null-terminated strings containing a null). By Base-64/hex encoding the cryptotext you make sure it will be plain old ASCII characters that evene old code can read/write.
If you dig around a little in the cryptocode or its method parameters you usually can determine that, and convert your strings accordingly.
I figured out where stream_to_hex, it is inside uTPLB_StreamUtils (pas or hpp) depending if you are using c builder or delphi.
Related
I am trying to pass an array of bytes from VBscript to my windows Delphi Application and can't seem to find the correct syntax to interpret the passed data.
The requirement is fairly simple as the VBscript snippet below demonstrates
Dim inst,arr(5)
Sub Main
set inst=instruments.Find("EP1")
arr(0) = 0
arr(1) = 1
arr(2) = 2
arr(3) = 3
arr(4) = 4
inst.writebytes arr,5
end Sub
I can get the server to accept the olevariant passed by the script but the data seems garbled, my example server code is shown below and is based on the Stackoverflow question here How to use variant arrays in Delphi
procedure TInstrument.WriteBytes(Data: OleVariant; Length: Integer);
var i,n:integer; Pdat:Pbyte; Adata:PvarArray;
begin
if VarIsArray(data) then
begin
n:=TVarData(Data).VArray^.Bounds[0].ElementCount;
Adata:= VarArrayLock(Data);
Getmem(Pdat,length);
try
for i:=0 to length-1 do
Pdat[i]:=integerArray(Adata.data^)[i];
Finstrument.WriteBytes(Pdat,Length);
finally
freemem(Pdat)
end;
end;
end;
So the idea is to accept the integers passed by the script, convert it to the local data representation (array of byte) then pass it on to my function to use the data.
I have tried several different data types and methods to try and get some ungarbled data out of the variant all to no avail.
What is the correct method of extracting the array data from the passed variant?
Also, TVarData(Data).VArray^.Bounds[0].ElementCount has a value of zero, why would that be?
Arrays created in VBScript are
zero based
untyped
declared with upper bound (not size as you assumed; size of array declared as Dim arr(5) is 6)
include dimension info in them (so you don't need to pass it along with the array)
When used in COM, they are passed as variant arrays of type varVariant (as the Ondrej Kelle points out in his comment). To process such an array in your method you have to assert that:
the value is a single dimensional array
each element can be converted to byte
You can write helper routine for that:
function ToBytes(const Data: Variant): TBytes;
var
Index, LowBound, HighBound: Integer;
ArrayData: Pointer;
begin
if not VarIsArray(Data) then
raise EArgumentException.Create('Variant array expected.');
if VarArrayDimCount(Data) <> 1 then
raise EArgumentException.Create('Single dimensional variant array expected.');
LowBound := VarArrayLowBound(Data, 1);
HighBound := VarArrayHighBound(Data, 1);
SetLength(Result, HighBound - LowBound + 1);
if TVarData(Data).VType = varArray or varByte then
begin
ArrayData := VarArrayLock(Data);
try
Move(ArrayData^, Result[0], Length(Result));
finally
VarArrayUnlock(Data);
end;
end
else
begin
for Index := LowBound to HighBound do
Result[Index - LowBound] := Data[Index];
end;
end;
for loop in the routine will be horribly slow when processing large arrays, so there's optimization for special case (variant array of bytes) that uses Move to copy bytes to result. But this will never happen with VBScript array. You might consider using VB.Net or PowerShell.
Using such a routine has downside of keeping 2 instances of the array in memory - as variant array and as byte array. Use it as a guide when applying it to your use case.
Consider the following code snippet (in Delphi XE2):
function PrepData(StrVal: string; Base64Val: AnsiString): OleVariant;
begin
Result := VarArrayCreate([0, 1], varVariant);
Result[0] := StrVal;
Result[1] := Base64Val;
end;
Base64Val is a binary value encoded as Base64 (so no null bytes). The (OleVariant) Result is automatically marshalled and sent between a client app and a DataSnap server.
When I capture the traffic with Wireshark, I see that both StrVal and Base64Val are transferred as Unicode strings. If I can, I would like to avoid the Unicode conversion for Base64Val. I've looked at all the Variant types and don't see anything other than varString that can transfer an array of characters.
I found this question that shows how to create a variant array of bytes. I'm thinking that I could use this technique instead of using an AnsiString. I'm curious though, is there another way to assign an array of non-Unicode character data to a Variant without a conversion to a Unicode string?
Delphi's implementation supports storing AnsiString and UnicodeString in a Variant, using custom variant type codes. These codes are varString and varUString.
But interop will typically use standard OLE variants and the OLE string, varOleStr, is 16 bit encoded. That would seem to be the reason for your observation.
You'll need to put the data in as an array of bytes if you do wish to avoid a conversion to 16 bit text. Doing so renders base64 encoding pointless. Stop base64 encoding the payload and send the binary in a byte array.
Keeping with the example in the question, this is how I made it work (using code and comments from David's answer to another question as referenced in my question):
function PrepData(StrVal: string; Data: TBytes): OleVariant;
var
SafeArray: PVarArray;
begin
Result := VarArrayCreate([0, 1], varVariant);
Result[0] := StrVal;
Result[1] := VarArrayCreate([1, Length(Data)], varByte);
SafeArray := VarArrayAsPSafeArray(Result[1]);
Move(Pointer(Data)^, SafeArray.Data^, Length(Data));
end;
Then on the DataSnap server, I can extract the binary data from the OleVariant like this, assuming Value is Result[1] from the Variant Array in the OleVariant:
procedure GetBinaryData(Value: Variant; Result: TMemoryStream);
var
SafeArray: PVarArray;
begin
SafeArray := VarArrayAsPSafeArray(Value);
Assert(SafeArray.ElementSize=1);
Result.Clear;
Result.WriteBuffer(SafeArray.Data^, SafeArray.Bounds[0].ElementCount);
end;
Question One
I have
var example : array[0..15] of char;
I want to assign the value from an input to that variable
example := inputbox('Enter Name', 'Name', '');
In the highscores unit I have record and array
type
points = record
var
_MemoryName : array[0..15] of char;
_MemoryScore : integer;
end;
var
rank : array[1..3] of points;
var s: string;
a: packed array[0..15] of char;
highscoresdata.position[1]._MemoryName := StrPLCopy(a, s, Length(a)) ;
returns -> (186): E2010 Incompatible types: 'array[0..15] of Char' and 'PWideChar'
var s: string;
a: packed array[0..15] of char;
s := InputBox('caption', 'Caption', 'Caption');
FillChar(a[0], length(a) * sizeof(char), #0);
Move(s[1], a[0], length(a) * sizeof(char));
scores.rank[1]._MemoryName := <<tried both s and a>> ;
returns (189): E2008 Incompatible types
Question One
There are many ways. One is:
procedure TForm1.FormCreate(Sender: TObject);
var
s: string;
a: packed array[0..15] of char;
begin
s := InputBox(Caption, Caption, Caption);
assert(length(s) <= 16);
FillChar(a[0], length(a) * sizeof(char), #0);
Move(s[1], a[0], length(s) * sizeof(char));
end;
But there might be a more elegant solution to your original problem, I suspect.
Question Two
Every time you wish a function/procedure didn't have a particular argument, you should realize that there might be a problem with the design of the project. Nevertheless, it isn't uncommon that Sender parameters are superfluous, because they are almost omnipresent because of the design of the VCL (in particular, the TNotifyEvent). If you know that the receiving procedure doesn't care about the Sender parameter, simply give it anything, like Self or nil.
Question Three
Consider this code:
procedure TForm4.FormCreate(Sender: TObject);
var
a: packed array[0..15] of char;
b: packed array[0..15] of char;
begin
a := b;
end;
This doesn't work. You cannot treat arrays like strings; in particular, you cannot assign static arrays like this (a := b).
Instead, you have to do something like...
Move(b[0], a[0], length(a) * sizeof(char));
...or simply loop and copy one value at a time. But the above simple assignment (a := b) does work if you declare a static array type:
type
TChrArr = packed array[0..15] of char;
procedure TForm4.FormCreate(Sender: TObject);
var
a: TChrArr;
b: TChrArr;
begin
b := a;
end;
Andreas has you covered for question 1.
Question 2
I would arrange that your event handler called another method:
procedure TForm5.Edit1KeyPress(Sender: TObject; var Key: Char);
begin
RespondToEditControlKeyPress;
end;
That way you can just call RespondToEditControlKeyPress directly.
I'd guess that you want to call it with no parameters because you want code to run when the edit control's text is modified. You could perhaps use the OnChange event instead. And it may be that OnChange is more appropriate because pressing a key is not the only way to get text into an edit control.
By the way, it's better to ask one question at a time here on Stack Overflow.
For a quick way to copy string-type values into array-of-character type values. I suggest a small helper function like this:
procedure StrToCharArray( inputStr:String; var output; maxlen:Integer);
type
ArrayChar = Array[0..1] of Char;
begin
StrLCopy( PChar(#ArrayChar(output)[0]),PChar(inputStr),maxlen);
end;
Each time you call it, pass in the maximum length to be copied. Remember that if the buffer length is 15, you should pass in 14 as the maxlen, so that you leave room for the terminating nul character, if you intend to always terminate your strings:
StrToCharArray( UserInputStr, MyRecord.MyField, 14 );
This function will ensure that the data you copy into the record is null terminated, assuming that's what you wanted. Remember that in a fixed length character array it's up to you to decide what the rules are. Null terminated? Fully padded with spaces or null characters.... Strings and arrays-of-characters are so different, that there exist multiple possible ways of converting between the two.
If you don't intend to terminate your strings with nul, then you should use the FillChar+Move combination shown in someone else's answer.
The obvious answer is of course.
Don't use a packed array of char.
Use a string instead.
If you use ansistring, 1 char will always take 1 byte.
If you use shortstring ditto.
Ansistring is compatible with Pchar which is a pointer to a packed array of char.
So you can write
function inputbox(a,b,c: ansistring): pchar;
begin
Result:= a+b+c;
end;
var s: ansistring;
begin
s:= inputbox('a','b','c');
end;
Some advice
It looks like your are translating code from c to Delphi.
a packed array of char is exactly the same as the old (1995) shortstring minus the length byte at the beginning of shortstring.
The only reason I can think of to use packed array of char is when you are reading data to and from disk, and you have legacy code that you don't want to change.
I would keep the legacy code to read and write from disk and then transfer the data into an ansistring and from there on only use ansistring.
It's soooooooo much easier, Delphi does everything for you.
And... ansistring is much faster, gets automatically created and destroyed, can have any length (up to 2GB), uses less memory --because identical strings only get stored once (which means stringa:= stringb where a string is 20 chars is at least 5x faster using ansistrings than array's of char).
And of course best of all, buffer overflow errors are impossible with ansistring.
What about unicodestring?
Unicodestring is fine to use, but sometimes translation of chars happens when converting between packed array of char and unicodestring, therefore I recommend using ansistring in this context.
What you try to do is impossible, indeed:
highscoresdata.position[1]._MemoryName := StrPLCopy(a, s, Length(a));
That tries to assign a pointer (the result of StrPLCopy, a PWideChar in the last few versions of Delphi) to an array, which is indeed impossible. You can't copy an array like that. I would do:
StrLCopy(highscoresdata.position[1]._MemoryName, PChar(s),
Length(highscoresdata.position[1]._MemoryName));
That should work, and is IMO the simplest solution to copy a string to an array of characters. There is no need to use a as some kind of intermediate, and using Move is, IMO, rather low level and therefore a little tricky (it is easy to forget to multiply by the size of a character, it is unchecked, it does not add a #0, etc.), especially if you don't know what exactly you are doing.
This solution should even work for versions of Delphi before Delphi 2009, as it does not rely on the size of the character.
FWIW, I would not use packed arrays. Packed doesn't have a meaning in current Delphi, but could confuse the compiler and make the types incompatible.
i have a function who's job is to convert an ADO Recordset into html:
class function RecordsetToHtml(const rs: _Recordset): WideString;
And the guts of the function involves a lot of wide string concatenation:
while not rs.EOF do
begin
Result := Result+CRLF+
'<TR>';
for i := 0 to rs.Fields.Count-1 do
Result := Result+'<TD>'+VarAsWideString(rs.Fields[i].Value)+'</TD>';
Result := Result+'</TR>';
rs.MoveNext;
end;
With a few thousand results, the function takes, what any user would feel, is too long to run. The Delphi Sampling Profiler shows that 99.3% of the time is spent in widestring concatenation (#WStrCatN and #WstrCat).
Can anyone think of a way to improve widestring concatenation? i don't think Delphi 5 has any kind of string builder. And Format doesn't support Unicode.
And to make sure nobody tries to weasel out: pretend you are implementing the interface:
IRecordsetToHtml = interface(IUnknown)
function RecordsetToHtml(const rs: _Recordset): WideString;
end;
Update One
I thought of using an IXMLDOMDocument, to build up the HTML as xml. But then i realized that the final HTML would be xhtml and not html - a subtle, but important, difference.
Update Two
Microsoft knowledge base article: How To Improve String Concatenation Performance
WideString are inherently slow because they were implemented for COM compatibility and go through COM calls. If you look at the code, it will keep on reallocating the string and call SysAllocStringLen() & C which are APIs from oleaut32.dll. It doesn't use the Delphi memory manager but AFAIK it uses the COM memory manager.
Because most HTML pages don't use UTF-16, you may get better result using the native Delphi string type and a string list, although you should be careful about conversion from UTF and the actual codepage, and the conversion will downgrade performance as well.
Also you're using a VarAsString() function that probably converts a variant to an AnsiString then converted to a WideString. Check if your version of Delphi has a VarAsWideString() or something alike function to avoid it, or rely on Delphi automatic conversion if you could be sure your variant will never be NULL.
Yup, your algorithm is clearly in O(n^2).
Instead of returning a string, try returning a TStringList, and replace your loop with
while not rs.EOF do
begin
Result.Add('<TR>');
for i := 0 to rs.Fields.Count-1 do
Result.Add( '<TD>'+VarAsString(rs.Fields[i].Value)+'</TD>' );
Result := Result.Add('</TR>');
rs.MoveNext;
end;
You can then save your Result using TStringList.SaveToFile
I'm unable to spend the time right now to give you the exact code.
But I think the fastest thing you can do is:
Loop through all the strings and total their length also adding for the extra table tags you'll need.
Use SetString to allocate one string of the proper length.
Loop through all the strings again and use the "Move" procedure to copy to the string to the proper place in the final string.
The key thing is that many concatenations to a string take longer and longer because of the constant allocating and freeing of memory. A single allocation will be your biggest timesaver.
i found the best solution. The open source HtmlParser for Delphi, has a helper TStringBuilder class. It is internally used to build what he calls DomStrings, which is actually an alias of WideString:
TDomString = WideString;
With a little bit of fiddling of his class:
TStringBuilder = class
public
constructor Create(ACapacity: Integer);
function EndWithWhiteSpace: Boolean;
function TailMatch(const Tail: WideString): Boolean;
function ToString: WideString;
procedure AppendText(const TextStr: WideString);
procedure Append(const value: WideString);
procedure AppendLine(const value: WideString);
property Length: Integer read FLength;
end;
The guts of the routine becomes:
while not rs.EOF do
begin
sb.Append('<TR>');
for i := 0 to rs.Fields.Count-1 do
sb.Append('<TD>'+VarAsWideString(rs.Fields[i].Value));
sb.AppendLine('</TR>');
rs.MoveNext;
end;
The code then feels to run infinitely afaster. Profiling shows much improvement; the WideString manipulation and length-counting became negligible. In its place was FastMM's own internal operations.
Notes
Nice catch on the mistaken forcing of all strings into current code-page (VarAsString rather than VarAsWideString)
Some HTML closing tags are optional; omitted ones that logically make no sense.
Widestring is not reference counted, any modification means a string manipulation. If your content is not unicode encoded, you can internally use the native string (reference counted) to concatenate string and then convert it to a Widestring. Example is as follows:
var
NativeString: string;
begin
// ...
NativeString := '';
while not rs.EOF do
begin
NativeString := NativeString + CRLF + '<TR>';
for i := 0 to rs.Fields.Count-1 do
NativeString := NativeString + '<TD>'+VarAsString(rs.Fields[i].Value) + '</TD>';
NativeString := NativeString + '</TR>';
rs.MoveNext;
end;
Result := WideString(NativeString);
I have also seen another approach: Encode Unicode to UTF8String (as reference counted), concatenate them and finally convert UTF8String to Widestring. But I am not sure, if two UTF8String can be concatenated directly. The time on encoding should also be considered.
Anyway, although Widestring concatenation is much slower than native string operations. But it is IMO still acceptable. Too much tuning on such kind of thing should be avoided. Seriously considering of performance, you should then upgrade your Delphi to at least 2009. The costs on buying a tool is for long-term cheaper than doing heavy hacks on an old Delphi.
In borland delphi 7 and even in delphi 2007 everything worked, but in delphi 2009 it just returns the wrong hash!
I use wcrypt2 script (http://pastebin.com/m2f015cfd)
Just have a look:
string : "123456"
hash:
Delphi 7 : "e10adc3949ba59abbe56e057f20f883e" - real hash.
Delphi 2007 : "e10adc3949ba59abbe56e057f20f883e" - real hash too.
And...
Delphi 2009 : "5fa285e1bebe0a6623e33afc04a1fbd5" - WTF??
I've tried a lot of md5 scripts, but delphi 2009 does the same with all of them. Any help? Thanks.
Your library is not Unicode aware. Just passing it an AnsiString won't be enough because it probably uses strings internally to store data.
You could try to update that library, wait for the author to update it, or just use the MessageDigest_5.pas that ships with Delphi 2009. It is in the source\Win32\soap\wsdlimporter folder, which you will need to either add to your path, or explicitly include it in your project.
Here is some sample code using it in Delphi 2009:
uses Types, MessageDigest_5;
procedure TForm16.Edit1Change(Sender: TObject);
var
MD5: IMD5;
begin
MD5 := GetMD5;
MD5.Init;
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(Edit1.Text));
Edit2.Text := LowerCase(MD5.AsString);
end;
And you are in business:
MD5(123456) = e10adc3949ba59abbe56e057f20f883e
You could wrap it in a simple function call if you wanted to. It is important you cast to a RawByteString before casting to a TByteDynArray since the RawByteString cast drops all the extra Unicode characters. Granted if the edit contains Unicode characters then you could end up with bad data.
Keep in mind that GetMD5 is returning an interface, so it is reference counted, etc.
Merry Christmas!
Before someone can comment on hashing algorithms, it helps if they have at least a fundamental understanding of the underlying concepts and principles. All of the responses so far which have focused on endless typecasting are completely overkill, but even worse, will result in unreliable results if a unicode string is being hashed.
The first thing you need to understand is that hashing and encryption algorithms operate at the byte-level. That means they don't care what you're hashing or encrypting. You can hash integers, chars, plain ASCII, full unicode, bytes, longwords, etc etc. The algorithm doesn't care.
When working with strings, the ONLY thing you have to ensure is that the internal function of your hashing library returns an AnsiString in the function which spits out your resulting hash. That's it. That's all that matters.
Your actual code for YOUR project can (and should) be based on normal string input, which maps to unicodestring in Delphi 2009. You shouldn't be typecasting anything to ansistring or rawbytestring. By doing so, you immediately create a broken hash if and when the user tries to hash anything outside the scope of the ANSI character set. And in the world of hashing, a broken hash is both unreliable AND insecure.
Have you checked that your library has been correctly updated for D2009 and unicodification?
I kinda doubt the same code would do D7/D2007 and D2009 for this sort of things.
It is obvious that your lib is not unicode enabled.
Convert your string to AnsiString or RawByteString or UTF8String by declaring temp AnsiString and assign your uniode string to it.
Note that if you are using unicode specific chars that can't be translated to single codepage, you should convert your string to UTF8.
Then call MD5(PAnsiChar(YourTempString)).
Check that your lib may have PWideChar or UNICODE declarations, to skip this.
If you have wcrypt2.pas, use this function.
function md5ansi(const Input: AnsiString): AnsiString;
var
hCryptProvider: HCRYPTPROV;
hHash: HCRYPTHASH;
bHash: array[0..$7f] of Byte;
dwHashBytes: Cardinal;
pbContent: PByte;
i: Integer;
begin
dwHashBytes := 16;
pbContent := Pointer(PAnsiChar(Input));
Result := '';
if CryptAcquireContext(#hCryptProvider, nil, nil, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT or CRYPT_MACHINE_KEYSET) then
begin
if CryptCreateHash(hCryptProvider, CALG_MD5, 0, 0, #hHash) then
begin
if CryptHashData(hHash, pbContent, Length(Input) * sizeof(AnsiChar), 0) then
begin
if CryptGetHashParam(hHash, HP_HASHVAL, #bHash[0], #dwHashBytes, 0) then
begin
for i := 0 to dwHashBytes - 1 do
begin
Result := Result + AnsiString(Format('%.2x', [bHash[i]]));
end;
end;
end;
CryptDestroyHash(hHash);
end;
CryptReleaseContext(hCryptProvider, 0);
end;
Result := AnsiString(AnsiLowerCase(String(Result)));
end;
Are you perchance casting a generic string (which in Delphi 2009 is a UnicodeString) to a PAnsiChar and passing that into the hash function? That will not work. You first must cast the string into an AnsiString and then cast that one to PAnsiChar, a la:
PAnsiChar(AnsiString('123456'))
Also, try using RawByteString instead of AnsiString like dmajkic suggested. Avoid UTF8String since that's not an AnsiString and any characters outside the ASCII range (0..127) might get reinterpreted into multibyte characters.
In Jim's answer:
if we change
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(Edit1.Text));
to
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(RawByteString(Edit1.Text)));
will support better while Chinese characters exists.