In borland delphi 7 and even in delphi 2007 everything worked, but in delphi 2009 it just returns the wrong hash!
I use wcrypt2 script (http://pastebin.com/m2f015cfd)
Just have a look:
string : "123456"
hash:
Delphi 7 : "e10adc3949ba59abbe56e057f20f883e" - real hash.
Delphi 2007 : "e10adc3949ba59abbe56e057f20f883e" - real hash too.
And...
Delphi 2009 : "5fa285e1bebe0a6623e33afc04a1fbd5" - WTF??
I've tried a lot of md5 scripts, but delphi 2009 does the same with all of them. Any help? Thanks.
Your library is not Unicode aware. Just passing it an AnsiString won't be enough because it probably uses strings internally to store data.
You could try to update that library, wait for the author to update it, or just use the MessageDigest_5.pas that ships with Delphi 2009. It is in the source\Win32\soap\wsdlimporter folder, which you will need to either add to your path, or explicitly include it in your project.
Here is some sample code using it in Delphi 2009:
uses Types, MessageDigest_5;
procedure TForm16.Edit1Change(Sender: TObject);
var
MD5: IMD5;
begin
MD5 := GetMD5;
MD5.Init;
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(Edit1.Text));
Edit2.Text := LowerCase(MD5.AsString);
end;
And you are in business:
MD5(123456) = e10adc3949ba59abbe56e057f20f883e
You could wrap it in a simple function call if you wanted to. It is important you cast to a RawByteString before casting to a TByteDynArray since the RawByteString cast drops all the extra Unicode characters. Granted if the edit contains Unicode characters then you could end up with bad data.
Keep in mind that GetMD5 is returning an interface, so it is reference counted, etc.
Merry Christmas!
Before someone can comment on hashing algorithms, it helps if they have at least a fundamental understanding of the underlying concepts and principles. All of the responses so far which have focused on endless typecasting are completely overkill, but even worse, will result in unreliable results if a unicode string is being hashed.
The first thing you need to understand is that hashing and encryption algorithms operate at the byte-level. That means they don't care what you're hashing or encrypting. You can hash integers, chars, plain ASCII, full unicode, bytes, longwords, etc etc. The algorithm doesn't care.
When working with strings, the ONLY thing you have to ensure is that the internal function of your hashing library returns an AnsiString in the function which spits out your resulting hash. That's it. That's all that matters.
Your actual code for YOUR project can (and should) be based on normal string input, which maps to unicodestring in Delphi 2009. You shouldn't be typecasting anything to ansistring or rawbytestring. By doing so, you immediately create a broken hash if and when the user tries to hash anything outside the scope of the ANSI character set. And in the world of hashing, a broken hash is both unreliable AND insecure.
Have you checked that your library has been correctly updated for D2009 and unicodification?
I kinda doubt the same code would do D7/D2007 and D2009 for this sort of things.
It is obvious that your lib is not unicode enabled.
Convert your string to AnsiString or RawByteString or UTF8String by declaring temp AnsiString and assign your uniode string to it.
Note that if you are using unicode specific chars that can't be translated to single codepage, you should convert your string to UTF8.
Then call MD5(PAnsiChar(YourTempString)).
Check that your lib may have PWideChar or UNICODE declarations, to skip this.
If you have wcrypt2.pas, use this function.
function md5ansi(const Input: AnsiString): AnsiString;
var
hCryptProvider: HCRYPTPROV;
hHash: HCRYPTHASH;
bHash: array[0..$7f] of Byte;
dwHashBytes: Cardinal;
pbContent: PByte;
i: Integer;
begin
dwHashBytes := 16;
pbContent := Pointer(PAnsiChar(Input));
Result := '';
if CryptAcquireContext(#hCryptProvider, nil, nil, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT or CRYPT_MACHINE_KEYSET) then
begin
if CryptCreateHash(hCryptProvider, CALG_MD5, 0, 0, #hHash) then
begin
if CryptHashData(hHash, pbContent, Length(Input) * sizeof(AnsiChar), 0) then
begin
if CryptGetHashParam(hHash, HP_HASHVAL, #bHash[0], #dwHashBytes, 0) then
begin
for i := 0 to dwHashBytes - 1 do
begin
Result := Result + AnsiString(Format('%.2x', [bHash[i]]));
end;
end;
end;
CryptDestroyHash(hHash);
end;
CryptReleaseContext(hCryptProvider, 0);
end;
Result := AnsiString(AnsiLowerCase(String(Result)));
end;
Are you perchance casting a generic string (which in Delphi 2009 is a UnicodeString) to a PAnsiChar and passing that into the hash function? That will not work. You first must cast the string into an AnsiString and then cast that one to PAnsiChar, a la:
PAnsiChar(AnsiString('123456'))
Also, try using RawByteString instead of AnsiString like dmajkic suggested. Avoid UTF8String since that's not an AnsiString and any characters outside the ASCII range (0..127) might get reinterpreted into multibyte characters.
In Jim's answer:
if we change
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(Edit1.Text));
to
MD5.Update(TByteDynArray(RawByteString(Edit1.Text)), Length(RawByteString(Edit1.Text)));
will support better while Chinese characters exists.
Related
I need to hash a string, preferably as SHA512, although it could be SHA256, SHA1, MD5 or CRC32.
I have downloaded Lockbox 3, put a TCryptographicLibrary and a THash component on a form, set the Hash property to SHA-512 and used the following code to produce a test result:
procedure TForm1.Button1Click(Sender: TObject);
begin
Hash1.HashString('myhashtest');
Edit1.Text := Stream_To_AnsiString(Hash1.HashOutputValue);
end;
To best illustrate the problem, I have gone on to an online hash calculator and the MD5 hash of 'myhashtest' is ff91e22313f0a41b46719e7ee6f99451 but setting the hash property in my test program to MD5 results in ÿ‘â#ð¤Fqž~æù”Q which is clearly wrong. I have tried the same test using other Hash properties, including the SHA512 which i want, and they all return rubbish.
Where am I going wrong?
THash.HashOutputValue is a stream of the raw hashed bytes. It appears that Stream_To_AnsiString() merely copies those raw bytes as-is into an AnsiString, it does not encode the bytes in any way. What you are looking for is the hex encoded version of the raw bytes instead. I do know that LockBox has a Stream_To_Base64() function (as shown in this example), but I do not know if it has a Stream_To_Hex() type of function. If it does not, you can easily create your own, eg:
function Stream_To_Hex(Stream: TStream): AnsiString;
var
NumBytes, I: Integer;
B: Byte;
begin
NumBytes := Stream.Size - Stream.Position;
SetLength(Result, NumBytes * 2);
for I := 0 to NumBytes-1 do
begin
Stream.ReadBuffer(B, 1);
BinToHex(#B, #Result[(I*2)+1], 1);
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
Hash1.HashString('myhashtest');
Edit1.Text := Stream_To_Hex(Hash1.HashOutputValue);
end;
Many cryptographic functions 'silently' (i.e. without stating so in the docs) output and require Base64- or hex-encoded strings (and also often AnsiStrings). This is because encrypted text can contain any data, and as soon as you start treating that as 'strings', string handling functions can easily choke on that (e.g. null-terminated strings containing a null). By Base-64/hex encoding the cryptotext you make sure it will be plain old ASCII characters that evene old code can read/write.
If you dig around a little in the cryptocode or its method parameters you usually can determine that, and convert your strings accordingly.
I figured out where stream_to_hex, it is inside uTPLB_StreamUtils (pas or hpp) depending if you are using c builder or delphi.
Please help me,
I know this may sound like very simple question, but i just can not figured it out how to make it work. I just started learning Unicode, so please give me some hint or example code.
I was converting my old encoding and decoding code from Delphi 5 to Delphi XE2. And when i call "Char" function it result in a different character, seem like it happen at the extended character of any encoding set.
At Delphi 5 :
Char(129) -> will result as empty char
At Delphi XE2 :
Char(129) -> will result #$81
I tried to used AnsiChar at delphi XE2, and the result is :
AnsiChar(129) -> will result as #129
What code should i used at delphi XE2, so it will return an empty char too. Not the #nn notation?
I need it to return the same result of Delphi 5, for the backward compatibility reason.
Is this have something to do with HIGHCHARUNICODE directive? I have read and tried it too, but still not luck.
Here the code that i tried at Delphi XE2, i make a simple one, but it did have a same logic with my encode / decode code. The code will get the char then put it into edit box.
procedure TForm1.Button1Click;
var
chars : Array[0..2] of AnsiChar;
ansi_string : AnsiString;
begin
chars[0] := AnsiChar(65);
chars[1] := AnsiChar(129);
chars[2] := AnsiChar(66);
ansi_string := chars;
// Here the ansi_string have a value of 'A'#$81'B'
EditBox1.Text := ansi_string;
// Here when i look the EditBox1.text in Evaluate/modify form,
// it shows 'A'#$0081'B'
// but at the form, it only show AB
end;
How can i make the ansi_string variable having a value of 'AB' instead of 'A'#$81'B'?
Thanks in Advance,
In Delphi 5, Char is AnsiChar, but in XE2 Char is WideChar instead. 127 is the highest signed value that an AnsiChar can hold, so a value of 129, which is hex $81, binary 10000001, would simply be interpreted as -127, which is also hex $81, binary 10000001. They are just different interpretations of the same bit value.
Depending on what your encoding/decoding code I actually doing, you will need to either use AnsiChar/AnsiString explicitly instead of Char/String generically, or switch to using Byte values, or else re-write the code to support Unicode properly and not make assumptions about the size of Char anymore. Hard to say since you did not show your actual code. But you should be OK with just using AnsiChar/AnsiString, since they do operate the same way they always have (the debugger may simply be displaying AnsiChar values differently, that's all).
I inherited a Delphi application and I know nothing about object pascal.
It's a BPL that I need to compile into the new version of C++ Builder XE.
When I run a make I get the error:
E2064 left side cannot be assigned to.
I've learned enough obj pascal to know I have a constant that is trying to be assigned a value.
But, apparently, you can over ride this behanvior; essentially turning constants into vars by going into Build options under the Delphi compiler and turning on "Assignable Typed constants".
I did that and I continue to get the same error.
I tried surrounding my code with {$J+} and {$J-} and still it will not compile.
procedure TChunkIDAT.CopyInterlacedRGB8(const Pass: Byte;
Src, Dest, Trans{$IFDEF Store16bits}, Extra{$ENDIF}: pChar );
var
Col: Integer;
begin
{Get first column and enter in loop}
Col := ColumnStart[Pass];
Dest := pChar(Longint(Dest) + Col * 3);
repeat
{Copy this row}
Byte(Dest^) := fOwner.GammaTable[pByte(Longint(Src) + 2)^]; inc(Dest);
Get the error on last line. If I change the const to a var, I then get the error that the declaration differs from the previous declaration but I have no idea where the previous declaration is....
You're type-casting a two-byte thing (Char) into a one-byte thing (Byte). Reading that value is easy to define, but making that value writable is tricky, probably for the same reason the types of formal and actual "var" parameters need to be identical.
Maybe you wanted to type-cast it to a two-byte thing, such as Word. Or maybe you want GammaTable to be an array of Char so you don't have to type-cast at all. Or maybe, if this code was originally written for a Delphi version earlier than 2009, you want those PChar declarations to be PAnsiChar — character types have gotten wider. Another option is to type-cast Dest to PByte, and then dereference the result. That's probably a bad idea, though, because you'll only be overwriting every other byte of the buffer.
Based on the name of the function, it sounds like PChar was never the right data type to use. That type is for character data, but I think this code is dealing with bytes. The correct thing to do is probably to change PChar to PByte, and then you don't need to type-cast Dest at all.
The $J directive is irrelevant; it controls whether the compiler will allow you to assign values to typed constants. You don't have any of those in this code.
The reason is that as of Delphi 2009, Char, PChar, and String are Unicode, and store more than one byte per character.
You should not cast those pointers to bytes, and the compiler prevents you from assigning them if you cast the left side of an assignment to a byte.
This compiles:
procedure CopyInterlacedRGB8(const Pass: Byte; Dest: pAnsiChar); overload;
begin
Byte(Dest^) := Pass;
end;
This doesn't:
procedure CopyInterlacedRGB8(const Pass: Byte; Dest: pChar); overload;
begin
Byte(Dest^) := Pass;
end;
Instead of pChar, you should use pByte, which makes the code simpler:
procedure CopyInterlacedRGB8(const Pass: Byte; Dest: PByte); overload;
begin
Dest^ := Pass;
end;
--jeroen
That looks like you're working with Gustavo Daud's TPngImage library. You don't need that code in an external BPL because it's been included in the RTL since D2009. Remove that unit from the BPL and you should be able to get at the updated version via the PngImage unit.
i have a function who's job is to convert an ADO Recordset into html:
class function RecordsetToHtml(const rs: _Recordset): WideString;
And the guts of the function involves a lot of wide string concatenation:
while not rs.EOF do
begin
Result := Result+CRLF+
'<TR>';
for i := 0 to rs.Fields.Count-1 do
Result := Result+'<TD>'+VarAsWideString(rs.Fields[i].Value)+'</TD>';
Result := Result+'</TR>';
rs.MoveNext;
end;
With a few thousand results, the function takes, what any user would feel, is too long to run. The Delphi Sampling Profiler shows that 99.3% of the time is spent in widestring concatenation (#WStrCatN and #WstrCat).
Can anyone think of a way to improve widestring concatenation? i don't think Delphi 5 has any kind of string builder. And Format doesn't support Unicode.
And to make sure nobody tries to weasel out: pretend you are implementing the interface:
IRecordsetToHtml = interface(IUnknown)
function RecordsetToHtml(const rs: _Recordset): WideString;
end;
Update One
I thought of using an IXMLDOMDocument, to build up the HTML as xml. But then i realized that the final HTML would be xhtml and not html - a subtle, but important, difference.
Update Two
Microsoft knowledge base article: How To Improve String Concatenation Performance
WideString are inherently slow because they were implemented for COM compatibility and go through COM calls. If you look at the code, it will keep on reallocating the string and call SysAllocStringLen() & C which are APIs from oleaut32.dll. It doesn't use the Delphi memory manager but AFAIK it uses the COM memory manager.
Because most HTML pages don't use UTF-16, you may get better result using the native Delphi string type and a string list, although you should be careful about conversion from UTF and the actual codepage, and the conversion will downgrade performance as well.
Also you're using a VarAsString() function that probably converts a variant to an AnsiString then converted to a WideString. Check if your version of Delphi has a VarAsWideString() or something alike function to avoid it, or rely on Delphi automatic conversion if you could be sure your variant will never be NULL.
Yup, your algorithm is clearly in O(n^2).
Instead of returning a string, try returning a TStringList, and replace your loop with
while not rs.EOF do
begin
Result.Add('<TR>');
for i := 0 to rs.Fields.Count-1 do
Result.Add( '<TD>'+VarAsString(rs.Fields[i].Value)+'</TD>' );
Result := Result.Add('</TR>');
rs.MoveNext;
end;
You can then save your Result using TStringList.SaveToFile
I'm unable to spend the time right now to give you the exact code.
But I think the fastest thing you can do is:
Loop through all the strings and total their length also adding for the extra table tags you'll need.
Use SetString to allocate one string of the proper length.
Loop through all the strings again and use the "Move" procedure to copy to the string to the proper place in the final string.
The key thing is that many concatenations to a string take longer and longer because of the constant allocating and freeing of memory. A single allocation will be your biggest timesaver.
i found the best solution. The open source HtmlParser for Delphi, has a helper TStringBuilder class. It is internally used to build what he calls DomStrings, which is actually an alias of WideString:
TDomString = WideString;
With a little bit of fiddling of his class:
TStringBuilder = class
public
constructor Create(ACapacity: Integer);
function EndWithWhiteSpace: Boolean;
function TailMatch(const Tail: WideString): Boolean;
function ToString: WideString;
procedure AppendText(const TextStr: WideString);
procedure Append(const value: WideString);
procedure AppendLine(const value: WideString);
property Length: Integer read FLength;
end;
The guts of the routine becomes:
while not rs.EOF do
begin
sb.Append('<TR>');
for i := 0 to rs.Fields.Count-1 do
sb.Append('<TD>'+VarAsWideString(rs.Fields[i].Value));
sb.AppendLine('</TR>');
rs.MoveNext;
end;
The code then feels to run infinitely afaster. Profiling shows much improvement; the WideString manipulation and length-counting became negligible. In its place was FastMM's own internal operations.
Notes
Nice catch on the mistaken forcing of all strings into current code-page (VarAsString rather than VarAsWideString)
Some HTML closing tags are optional; omitted ones that logically make no sense.
Widestring is not reference counted, any modification means a string manipulation. If your content is not unicode encoded, you can internally use the native string (reference counted) to concatenate string and then convert it to a Widestring. Example is as follows:
var
NativeString: string;
begin
// ...
NativeString := '';
while not rs.EOF do
begin
NativeString := NativeString + CRLF + '<TR>';
for i := 0 to rs.Fields.Count-1 do
NativeString := NativeString + '<TD>'+VarAsString(rs.Fields[i].Value) + '</TD>';
NativeString := NativeString + '</TR>';
rs.MoveNext;
end;
Result := WideString(NativeString);
I have also seen another approach: Encode Unicode to UTF8String (as reference counted), concatenate them and finally convert UTF8String to Widestring. But I am not sure, if two UTF8String can be concatenated directly. The time on encoding should also be considered.
Anyway, although Widestring concatenation is much slower than native string operations. But it is IMO still acceptable. Too much tuning on such kind of thing should be avoided. Seriously considering of performance, you should then upgrade your Delphi to at least 2009. The costs on buying a tool is for long-term cheaper than doing heavy hacks on an old Delphi.
In a sorting routine in Delphi 2007 I am using code like this:
(txt[n] in ['0'..'9'])
function ExtractNr(n: Integer; var txt: String): Int64;
begin
while (n <= Length(txt)) and (txt[n] in ['0'..'9']) do n:= n + 1;
Result:= StrToInt64Def(Copy(txt, 1, (n - 1)), 0);
Delete(txt, 1, (n - 1));
end;
where txt is a string. This works fine in D2007 but will give warnings in D2009 and D2010 I have no idea why but is there any way I can make it work without warnings in D2009 and D2010?
Roy M Klever
Are you getting the "WideChar reduced to byte Char in set expressions. Consider using 'CharInSet' function in 'SysUtils' unit" message?
Here's the issue. In D2009, the default string type was changed from AnsiString to UnicodeString. An AnsiString uses a single byte for each character, giving you 256 possible characters. A UnicodeString uses 2 bytes per character, giving up to 64K characters. But a Pascal set can only contain up to 256 elements. So it can't create a "set of WideChar" because there are too many possible elements.
The warning is a warning that you're attempting to compare txt[n], which is a WideChar from a Unicode string, against a set of chars. It can't make a set of WideChars, so it had to reduce them to AnsiChars to fit them into a Pascal set, and your txt[n] might be outside the Ansi boundaries entirely.
You can fix it by using CharInSet, or by making txt an AnsiString, if you're certain you won't need any Unicode characters for it. Or if that won't work well, you can disable the warning, but I'd consider that a last resort.
use CharInSet or better use Character.IsDigit