Delphi XE4 Indy compatibility issue between TBytes and TidBytes - delphi

Today I try to compile my XE3 project in XE4. First problem that I face is with Indy's FTCPClient.Socket.ReadBytes() method.
Before it was accepting TBytes type, now it insists on TidBytes.
Definitions:
TIdBytes = array of Byte;
TBytes, Im not sure I guess it is generics something like TArray which is array of Byte.
Question number 1:
Why does compiler complain by saying that'[dcc32 Error] HistoricalStockData.pas(298): E2033 Types of actual and formal var parameters must be identical'. As I see they are already identical.
Question number 2:
Should I modify my source code with the each new delphi version?
Thanks.

The reason TIdBytes was a simple alias for TBytes in earlier Indy 10 releases was primarily for compatibility with SysUtils.TEncoding, which uses TBytes. Indy's TIdTextEncoding type used to be a simple alias for SysUtils.TEncoding in D2009+, so TIdBytes needed to be a simple alias for TBytes to match.
However, TBytes caused quite a bit of trouble for Indy in XE3, mainly because of RTTI problems with Generics (TBytes is a simple alias for TArray<Byte> in recent Delphi releases). So, Indy 10.6 re-designed TIdTextEncoding to no longer rely on SysUtils.TEncoding at all (there were other reasons as well for doing so), which then allowed TIdBytes to change into its own array type in order to avoid the XE3 issues moving forward.
On the other hand, you were passing a TBytes where a TIdBytes was expected, so that is bad programming on your part for not following Indy's defined interface in the first place. All of Indy 10's byte-based operations, including ReadBytes(), have always operated on TIdBytes only. The fact that TIdBytes silently mapped to TBytes was an implementation detail that you should not have relied on in your code. Indy 10 expects TIdBytes, so use TIdBytes, then you would not have compiler errors about incompatible types.

The following two declarations are not the same, even though they appear to be. They're not assignment compatible, even though they're both based on array of string.
type
TStringArrayOne = array of string;
TStringArrayTwo = array of string;
var
AVar1, AVar2: TStringArrayOne;
AVar3, AVar4: TStringArrayTwo;
begin
AVar1 := TStringArrayOne.Create('a', 'b', 'c'); // Compiles
AVar2 := TStringArrayTwo.Create('a', 'b', 'c'); // Won't compile
AVar3 := TStringArrayTwo.Create('a', 'b', 'c'); // Compiles
AVar4 := TStringArrayOne.Create('a', 'b', 'c'); // Won't compile
end;
So TBytes and TIdBytes are not the same type, even if they're both defined as being array of Byte.
With regard to your question 2: It's a common problem with some third-party code. Indy in particular is known for making changes that breaks backward compatibility because they decide to reorganize or rewrite things between versions. Indy 10 was a major change from Indy 9, IIRC, and pretty much required a rewrite of most code that used it if you updated to the later version of Indy (even without updating Delphi at the same time). If you don't want to deal with those changes, you might want to look at using a more stable IP communications package. There are several available that are also free, open source packages.

In Indy 10.5.9 the type TIdBytes was defined differently depending on the presence of an existing TBytes type - see unit IdGlobal:
{$IFDEF HAS_TBytes}
TIdBytes = TBytes;
{$ELSE}
TIdBytes = array of Byte;
{$ENDIF}
In Indy 10.6 (included in XE4), the declaration changed to unconditionally
TIdBytes = array of Byte;
which means that starting with Indy 10.6, IdGlobal.TIdBytes is different from SysUtils.TBytes.
The second question is hard to answer, it is more a question of your priorities - other libraries are not immune against changes either, for example to improve performance or type-safety. Also changes in the Delphi language can always affect existing code.

Related

Delphi DLL (in XE) must handle TStringList (D2007, Ansi)

The DLL was originally written in D2007 and needed a quick, panic TStringList call (yes, it was one of those “I’m sure to regret”; though all the calls to the DLL, made by several modules, are all made by Delphi code and I wrongly presumed/hoped backwards compatibility when XE came out).
So now I’m moving the DLL to XE5 (& thus Unicode) and must maintain the call for compatibility. The worst case is I simply write a new DLL only for XE while keeping the old one for legacy, but feel there should be no reason why XE couldn’t deconstruct/overrride to an {ANSI} TStringList parameter. But my Delphi behind-the-scenes knowledge is not robust and a couple of attempts have not succeeded.
Here is the DLL call – it takes a list of file paths and in this stripped-down code, simply adds each string to an internal list (that is all the DLL does with the parameter, a single read-only reference):
function ViewFileList ( lstPaths: TStringList): Integer; Export; Stdcall;
begin
for iCount := 0 to lstPaths.Count - 1 do
lstInternal.Add(lstPaths.strings[iCount]);
end;
What I found is that when I compiled this in XE5, that lstPaths.Count is correct, so the basic structure aligns. But the strings were garbage. It seems the mismatch would be two-fold: (a) the string content naturally is being interpreted as two-bytes per character; (b) there is no Element size (at position -10) and code page (at position -12; so yes, garbage strings). I am also vaguely aware of behind-the-scenes memory management, though I only do read-only access. But the actual string pointers themselves should be correct (??) and thus is there a way to coerce my way through?
So, regardless of whether I have any of that right, is there any solution? Thanks in advance.
What you perhaps don't yet realise is that your code has always been wrong. In general, it is not supported to pass Delphi objects across module boundaries. You can make it work so long as you understand the implementation very well, so long as you don't call virtual methods, so long as you don't do memory allocation, so long as you use the same compiler on both sides, and probably many other reasons. Either use runtime packages (also requires same compiler on both sides), or use interop safe types (integers, floats, null terminated character arrays, pointers, records and arrays of interop safe types, etc.)
There's really no simple solution here. It should never have worked in the first place and if it did then you have been very unlucky. Unlucky because a much better outcome would have been a failure that would have led you to doing it properly.
Perhaps the best thing you can do is make an adapter DLL. The architecture goes like this, from bottom to top:
Original Delphi 2007 DLL at the bottom, with the bogus export that requires D2007 string list to be supplied.
New adapter Delphi 2007 DLL in the middle. It calls the bogus export, and is able to supply a D2007 string list. The adapter DLL exposes a proper interface that does not require Delphi objects to be passed across the module boundary.
New XE5 executable at the top. This talks to the adapter, but does so using valid interop types.
David and Jerry already told you what you should do - re-write the DLL to do the right thing when it comes to passing interop-safe data across module boundaries. However, to answer your actual question:
the actual string pointers themselves should be correct (??) and thus is there a way to coerce my way through?
So, regardless of whether I have any of that right, is there any solution?
You can try the following. It is dangerous, but it should work, if a re-write is not an option for you at this time:
// the ASSUMPTION here is that the caller has been compiled in D2007 or earlier,
// and thus is passing an AnsiString-based TStringList object. When this DLL is
// compiled in Delphi 2009 or later, TStringList is UnicodeString-based instead,
// so we have to re-interpret the data a little.
//
// The basic structure of TStringList itself should be the same, just the string
// content is different. For backwards compatibility, the refcnt and length
// fields of the StrRec record found in every AnsiString/UnicodeString payload
// are still at the same offsets. Delphi 2009 added some new fields, but we can
// ignore those here.
//
// Of course, XE is the version that removed the RTL support code for the {$STRINGCHECKS}
// compiler directive, which handled all of these details in Delphi 2009 and 2010
// when users were first migrating to Unicode. But in XE, we'll have to deal with
// it manually.
//
// These assumptions may change in future versions, but lets deal with that if/when
// the time comes...
function ViewFileList ( lstPaths: TStringList): Integer; Export; Stdcall;
{$IFDEF UNICODE}
var
tmp: AnsiString;
{$ENDIF}
begin
for iCount := 0 to lstPaths.Count - 1 do
begin
{$IFDEF UNICODE}
// the DLL is being compiled in Delphi 2009 or later...
//
// the Length(String) function simply returns the value of the string's
// StrRec.length field, which fortunately is in the same location in
// both pre-2009 AnsiString and 2009+ AnsiString/UnicodeString, and in
// this case will reflect the number of AnsiChar elements in the source
// AnsiString. We cannot simply typecast a "UnicodeString" directly to
// a PAnsiChar, nor can we typecast a PWideChar to a PAnsiChar, but we
// can typecast a string to a Pointer first and then cast that to a
// PAnsiChar. This code is assuming that it can safely get a pointer to
// the source AnsiString's underlying character data to make a local
// copy of it that can then be added to the internal list normally.
//
// Where this MIGHT fail is if the source AnsiString contains a reference
// to a string literal (StrRec.refcnt=-1) for its character data, in
// which case the RTL will try to copy the character data when assigning
// the source string to a variable, such as the one the compiler is
// likely to generate for itself to receive the TStringList.Strings[]
// property value before it can be casted to a Pointer. If that happens,
// this is likely to crash when the RTL tries to copy too many bytes from
// the source AnsiString! You can use the StringRefCount() function to
// detect that condition and do something else, if needed.
//
// But, if the source AnsiString is a normal allocated string (the usual
// case), then this should work OK. Even with the compiler-generated
// variable in play, the compiler should simply bump the reference count
// of the source AnsiString, without affecting the underlying character
// data, just long enough for this code to copy the data and release the
// reference count...
//
SetString(tmp, PAnsiChar(Pointer(lstPaths.strings[iCount])), Length(lstPaths.strings[iCount]) * SizeOf(AnsiChar));
lstInternal.Add(tmp);
{$ELSE}
// the DLL is being compiled in Delphi 2007 or earlier, so just add the
// source AnsiString as-is and let the RTL do its work normally...
//
lstInternal.Add(lstPaths.strings[iCount]);
{$ENDIF}
end;
end;

Problems with Delphi 2009 + HidD_GetSerialNumberString

I am having problems getting HidD_GetSerialNumberString to simply give me the serialnumber of some devices.
On some devices it works fine, but on others all I get is "ÿÿÿ" as the serialnumber.
I am suspecting it has something to do with Delphi 2009 and it's unicode, but I simply cannot get my head around it.
Is there a solution to this?
Edit (declaration) :
type
THidD_GetSerialNumberString = function(HidDeviceObject: THandle; Buffer: PWideChar; BufferLength: Integer): LongBool; stdcall;
var
HidD_GetSerialNumberString: THidD_GetSerialNumberString;
//in the LoadHid-function
#HidD_GetSerialNumberString := GetModuleSymbolEx(HidLib, 'HidD_GetSerialNumberString', Result);
Usage :
var
test : PWideChar;
...
GetMem(test, 512);
if HidD_GetSerialNumberString(HidFileHandle, Test, 512) then
FSerialNumber := test; //FSerialNumber is WideString
FreeMem(test);
Your code is fine in all versions of Delphi. Although clearly it involves a UTF-16 to ANSI conversion when compiled by a pre-Unicode Delphi. However, since you use Delphi 2009 that is not an issue.
There are no problems regarding Unicode conversions. The HidD_GetSerialNumberString function fills out the buffer that you supply with a null-terminated UTF-16 string. And since you declared test to be exactly that, Delphi will generate code necessary to copy to a Delphi string variable.
Is it possible that HidD_GetSerialNumberString is returning false and so FSerialNumber is not initialized properly? Other than that, the only conclusions that one can make are that:
HidD_GetSerialNumberString is lying to you when it returns True.
The serial number returned by calling HidD_GetSerialNumberString really is ÿÿÿ.

Delphi 7 calling DelphiXE2 dll getting corrupt widestrings

I have a Delphi 7 application that needs to call a SOAP API that is much too new for the available SOAP importers. I have satisfied myself that D7 can't call the SOAP API without too much effort to be worth while. But I also have Delphi XE2, and that can import the SOAP and call it quite happily. So I have written a simple dll wrapper in XE2 that exposes the necessary parts of the soap interface. I can call the dll from an XE program.
In Delphi7 I took the SOAP API import file from XE, stripped out the {$SCOPED_ENUMS ON} defines and the initialization section that calls unavailable SOAP wrappers, plus changed string to widestring throughout. That compiles. I'm using FastMM with ShareMM enabled to make string passing work and avoid making everything stdcall.
The reason I'm trying to do it this way is that if it works it will make the SOAP shim very easy to code and maintain, since 90% of the code is generated by the XE2 SOAP importer, and it will mean that when we move the D7 app to a modern Delphi the code will remain largely unchanged.
But when I run it, I get weird strings (and consequent access violations). I've got simple functions that don't use the SOAP code to make the problem more obvious.
Passing a widestring from Delphi7 exe into DelphiXE2 dll the string length is doubled (according to the Length() function), but there's no matching data conversion. So a widestring "123" in D7 becomes "1234...." in XE2, where the .... is whatever garbage happens to be on the stack. Viewed as byte arrays both have half zero bytes as expect.
Passing a widestring back from XE2 dll to D7 I get the mirror effect - the string length is halved and strings are simply truncated ("1234" becomes "12").
I'm pasting code in because I know you will ask for it.
In Delphi XE2 I'm exporting these functions:
// testing
function GetString(s:string):string; export;
function AddToString(s:string):string; export;
implementation
function GetString(s:string):string;
begin
Result := '0987654321';
end;
function AddToString(s:string):string;
begin
Result := s + '| ' + IntToStr(length(s)) + ' there is more';
end;
In Delphi 7:
function GetString(s:widestring):widestring; external 'SMSShim.dll';
function AddToString(s:widestring):widestring; external 'SMSShim.dll';
procedure TForm1.btnTestGetClick(Sender: TObject);
var
s: widestring;
begin
s := widestring('1234');
Memo1.Lines.Add(' GetString: ' + GetString(s));
end;
procedure TForm1.btnTestAddClick(Sender: TObject);
var
s: widestring;
begin
s := widestring('1234567890');
Memo1.Lines.Add(' AddToString: ' + AddToString('1234567890'));
end;
I can run from either side, using the D7 executable as the host app to debug the dll. Inspecting the parameters and return values in the debugger gives the results above.
Annoyingly, if I declare the imports in delphi7 as strings I get the correct length but invalid data. Declaring as shown I get valid data, wrong lengths, and access violations when I try to return.
Making it all stdcall doesn't change the behaviour.
The obvious solution is the just write simple wrapper functions that expose exactly the functionality I need right now. I can do that, but I'd prefer the above cunning way.
The DLL in question exports functions that expect to receive UnicodeString parameters. (As you know, the string type became an alias for UnicodeString in Delphi 2009.) A Delphi 7 application cannot consume that DLL; the run-time library doesn't not know how to operate on that type because it didn't exist back in 2002 when Delphi 7 was published.
Although the character size for UnicodeString is compatible with WideString, they are not the same types. UnicodeString is structured like the new AnsiString, so it has a length field, a reference count, a character size, and a code page. WideString has a length field, but any other metadata it carries is undocumented. WideString is simply Delphi's way of exposing the COM BSTR type.
A general rule to live by is to never export DLL functions that couldn't be consumed by C.1 In particular, this means using only C-compatible types for any function parameters and return types, so string is out, but WideString is safe because of its BSTR roots.
Change the DLL to use WideString for its parameters instead of string.
1 Maintaining C compatibility also means using calling conventions that C supports. Delphi's default register calling convention is not supported in Microsoft C, so use cdecl or stdcall instead, just like you've seen in every Windows DLL you've ever used.
There's not way to disable the UNICODE in Delphi XE2 (or any version greater than 2009) , however there are many resources that can help you to migrate your application.
White Paper: Delphi and Unicode (from Marco Cantù)
Delphi Conversion Unicode Issues
"Globalizing your Delphi applications" - Delphi Unicode Resources
Compilation of resources for migrate to Delphi 2009/2010 Unicode

Delphi - Problem using an existing dll

I have to use an existing dll, but got a problem with it.
Here's how I try to use it:
unit u_main;
...
implementation
procedure getUserData(var User, Pass: string); stdcall; external 'Common5.dll';
{$R *.dfm}
procedure TForm1.Button1Click(Sender: TObject);
var
u, p: string;
begin
getUserData(u,p);
end;
...
end.
When I press the button the get the userData, I get an EInvalidPointer exception.
The dll is registerd and in some other projects it's in use and work. any ideas?
EDIT:
The DLL was created in Delphi7 and is now used in a Delphi 2009 project.
Maybe there's a problem with unicode strings or something like that?
You need to rebuild the Delphi 7 DLL, make it follow the WinApi standard of getting PChar and BufferLen parameters. You've got multiple problems with the current implementation:
string is platform-specific, it's implementation may change between delphi versions (and did change). You're not supposed to use string outside the platform!
You're passing the parameters as "var", suggesting the DLL might change the value of user and/or pass. String is an special, managed type, changing it requires allocating memory for the new string. This in turns requires you to share the memory manager between the DLL and the EXE (using sharemem.pas and BorlandMM.dll - or variants). The trouble is, sharing the memory manager between different versions of Delphi is an unsupported configuration! (gotton from embarcadero forums)
The Delphi 7 is hoping to receive an simple AnsiString (1 byte chars), while the Delphi 2009 exe is sending Unicode strings (2 bytes per char).
Along with using PChar, be sure to pre-allocate the space before you call GetUserData. i.e. if you assign 'foo' into a pchar that's empty, you'll blow up. So either use static length PChar/PAnsiChar arrays, or use this technique:
var
s : AnsiString;
begin
setlength(s,256);
MyDLLProc(PAnsiChar(s));
end;

How can I pass a Delphi string to a Prism DLL?

We try to pass a string from a native Delphi program to a Delphi Prism DLL.
We have no problem passing integers, but strings are mismatched in the DLL.
We saw Robert Love's code snippet in response to another question, but there is no code for the native Delphi program.
How can we pass strings from Delphi to a Delphi Prism DLL?
The best way would be to use WideString.
For several reasons.
It is Unicode and works before D2009
It's memory is managed in ole32.dll, so no dependency on either Delphi's memory manager or the CLR GC.
You do not have to directly deal with pointers
In Oxygene, you could write it like so:
type
Sample = static class
private
[UnmanagedExport]
method StringTest([MarshalAs(UnmanagedType.BStr)]input : String;
[MarshalAs(UnmanagedType.BStr)]out output : String);
end;
implementation
method Sample.StringTest(input : String; out output : String);
begin
output := input + "ä ~ î 暗";
end;
"MarshalAs" tells the CLR how to marshal strings back and forth. Without it, strings are passed as Ansi (PAnsiChar), which is probably NOT what you would want to do.
This is how to use it from Delphi:
procedure StringTest(const input : WideString; out output : WideString);
stdcall; external 'OxygeneLib';
var
input, output : WideString;
begin
input := 'A b c';
StringTest(input, output);
Writeln(output);
end.
Also, never ever use types, that are not clearly defined, for external interfaces.
You must not use PChar for DLL imports or exports. Because if you do, you will run into exceptions when you compile it with D7 or D2009 (depending on what the original dev system was)
Strings in Delphi Win32 are managed differently from strings in .Net, so you can not pass a .Net string to Delphi Win32 or vice versa.
To exchange strings values you'd better use PChar type which is supported by both compilers. That is the same way you send string values to Windows API functions.
Regards
P.S. I am NOT Robert ;-)

Resources