How writing to MemoryStream has changed in Delphi 10.2? - delphi

I am porting my code from 10.1 to 10.2 and this gives me error:
procedure TForm4.FormCreate(Sender: TObject);
const
CFourBytes: array[0..3] of Byte = (1, 2, 3, 4);
var
LStream: TMemoryStream;
LBuffer: array of Byte;
begin
SetLength(LBuffer, 4);
LStream := TMemoryStream.Create;
LStream.Write(#CFourBytes[0], 4); // E2036 Variable required
LStream.Position := 0;
LStream.ReadData(#LBuffer[0], 4);
end;
I had to change offending line to LStream.Write(CFourBytes[0], 4);
What has changed? Have I been doing it wrong for the whole time?

The code in your question did compile in older versions, but it should not have done. The behaviour seen in 10.2 is correct.
What happens in old versions is very strange. The compiler selects this overload in TStream:
function Write(const Buffer: TBytes; Count: Longint): Longint; overload;
That is especially egregious because what has been passed to this method is the address of the static array CFourBytes. Which is categorically not a TBytes object.
Now it just so happens that a TBytes variable is the address of the first element of the array. And nothing in the TBytes override for TMemoryStream.Write refers to Length() of that bogus TBytes object. So your code happens to work as intended. This is very clearly a compiler error that has been fixed.
Your code has always been broken, you have just, up until now, been getting away with it by fortune. You should fix your code. Like this:
LStream := TMemoryStream.Create;
try
LStream.WriteBuffer(CFourBytes, SizeOf(CFourBytes));
SetLength(LBuffer, LStream.Size);
LStream.Position := 0;
LStream.ReadBuffer(LBuffer[0], LStream.Size);
finally
LStream.Free;
end;
Note that I am using WriteBuffer and ReadBuffer instead of Write and Read. These are the preferred methods to use with TStream. The reason being that they perform error checking and raise exceptions in case of errors, unlike Write and Read.

Perhaps nothing has been changed.
TStream.Write/Read methods always used untyped const/var parameter const Buffer (help) and using address of variable is wrong (because method (to be exact - compiler) finds address of variable itself).
Probably you accidentally confuse these methods with Read/WriteData ones that use typed parameter and one of overloaded versions gets Pointer type argument.
Here ReadData implementation dereferences this pointer and uses Read internally (Read in its turn calls Move and the last routine gets address of buffer again :) )

Related

E2250 There is no overloaded version of 'StrPas' that can be called with these arguments

My code :
function ThisModuleName: Char; //bulo String
var
p: array [0..512] of char;
fileNamePart: pchar;
begin
GetModuleFileName(HInstance, #p[0], 512);
GetFullPathName(#p[0], 512, #p[0], fileNamePart);
result := StrPas(WideString(#p[0])); //stalo WideString
end;
In Delphi 7 compiles.
In Delphi 10.2 it gives an error:
[dcc32 Error] verinfo.pas(98): E2250 There is no overloaded version of 'StrPas' that can be called with these arguments
This code is wrong on all Delphi versions. I doubt it compiles anywhere. I'm guessing that the code you presented is not the Delphi 7 code, but rather the code after you've hacked at it for a while.
The return type should be string and not char. Furthermore, the cast to WideString is bogus. Finally, a zero-based array of characters can be treated as PChar.
Your function should be translated like so:
function ThisModuleName: string;
var
p: array [0..511] of Char;
fileNamePart: PChar;
begin
GetModuleFileName(HInstance, p, Length(p));
GetFullPathName(p, Length(p), p, fileNamePart);
Result := p;
end;
Having said all of that, while this is a faithful translation of the code in the question, it does not return a module name. I really don't know what your code is trying to do, but the call to GetFullPathName appears to be wrong in your code.
My guess is that you are trying to convert potential short 8.3 file names to long names. I believe that you need an extra buffer to make that work. Here's what that code looks like, with some error checking added:
function ThisModuleName: string;
var
ModuleFileName, Buffer: array [0..511] of Char;
FilePart: PChar;
begin
Win32Check(GetModuleFileName(HInstance, ModuleFileName, Length(ModuleFileName))<>0);
Win32Check(GetFullPathName(ModuleFileName, Length(Buffer), Buffer, FilePart)<>0);
Result := Buffer;
end;
Instead of asking a question for every problem you encounter in your porting project it might pay dividends to learn a bit more about Unicode Delphi.
Instead of calling the API directly, you can call System.SysUtils.GetModuleName, which simply returns a string.
It wraps GetModuleFilename, and by doing so it also shows how to call that function. I hope I'm allowed to quote a couple of lines from the unit mentioned above. It also uses the MAX_PATH constant, which contains the maximum length of a path.
Note that GetModuleFilename already returns a fully qualified path, so calling GetFullPathName afterwards is redundant.
function GetModuleName(Module: HMODULE): string;
var
ModName: array[0..MAX_PATH] of Char;
begin
SetString(Result, ModName, GetModuleFileName(Module, ModName, Length(ModName)));
end;
This is mainly useful if you want the path of a dll, if you're interested in the main executable, you can simply use Application.ExeName.

Why does this one PAnsiChar get chopped when converted to AnsiString?

Please consider the following program:
program SO41175184;
{$APPTYPE CONSOLE}
uses
SysUtils;
function Int9999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString(IntToStr(9999)));
end;
function Int99999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString(IntToStr(99999)));
end;
function Int999999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString(IntToStr(999999)));
end;
function Str9999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString('9999'));
end;
function Str99999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString('99999'));
end;
function Str999999: PAnsiChar;
begin
Result := PAnsiChar(AnsiString('999999'));
end;
begin
WriteLn(Int9999); // '9999'
WriteLn(Int99999); // '99999'
WriteLn(Int999999); // '999999'
WriteLn(string(AnsiString(Str9999))); // '9999'
WriteLn(string(AnsiString(Str99999))); // '99999'
WriteLn(string(AnsiString(Str999999))); // '999999'
WriteLn(string(AnsiString(PAnsiChar(AnsiString(IntToStr(9999)))))); // '9999'
WriteLn(string(AnsiString(PAnsiChar(AnsiString(IntToStr(99999)))))); // '99999'
WriteLn(string(AnsiString(PAnsiChar(AnsiString(IntToStr(999999)))))); // '999999'
WriteLn(string(AnsiString(Int9999))); // '9999'
WriteLn(string(AnsiString(Int99999))); // '9999' <----- ?!
WriteLn(string(AnsiString(Int999999))); // '999999'
ReadLn;
end.
Only in one of these cases does the string lose a single character, in Delphi 2010 and Delphi XE3 both. With FPC the same program works correctly. Switching to PChar also makes the problem disappear.
I suppose it has something to do with memory management, but I don't have enough of a clue where to look to do a meaningful investigation. Could anyone clarify?
Dynamically created strings are reference counted and deallocated when no references remain.
Result := PAnsiChar(AnsiString(IntToStr(99999)));
causes a temporary AnsiString to be created, its address taken via the cast to PAnsiChar, and then the temporary string deallocated†. The resulting pointer points to now-unclaimed memory that may be overwritten for pretty much any reason, including during the allocations of more strings.
Neither Delphi nor FPC clears memory by default during deallocations, so if the memory hasn't been re-used yet, you may get lucky when reading what used to be there. Or, as you saw, you may not.
When returning PAnsiChar like this, you need an agreement between caller and callee on memory management. You need to make sure you do not free the memory early, and you need to make sure your callers know how to free the memory afterwards.
† Remy Lebeau points out that this deallocation happens when the procedure or function returns. If there is another statement after the assignment to Result, the string will still be available. This is normally correct, but there are also cases where it the temporary string gets deallocated before the return, for example when you create temporary strings in a loop. I would not recommend using temporary objects after the statement that creates them concludes, even in cases where it is valid, because it makes it too hard to verify whether the code is correct. For those cases, just use an explicit variable.

Can I create a VarArray OleVariant from a buffer (pByte) and size without copying?

I can copy the memory from the buffer into the safe array as follows
function GetVarArrayFromBuffer(ABuffer : pByte; ASizeInBytes: Cardinal) : OleVariant;
var
LVarArrayPtr: Pointer;
begin
Result := VarArrayCreate([0, ASizeInBytes - 1], varByte);
LVarArrayPtr := VarArrayLock(Result);
try
Move(ABuffer^, LVarArrayPtr^, ASizeInBytes);
finally
VarArrayUnLock(Result);
end;
end;
But, is there a way to directly pass my pointer and size into a varArray type OleVariant without copying memory?
[Edit]
I can see that the array inside the OleVariant is a SAFEARRAY (defined as PVarArray = ^TVarArray), so it seems like there should be a way to do this by populating the values in a TVarArray and setting the VType and VArray values in the OleVariant.
is there a way to directly pass my pointer and size into a varArray type OleVariant without copying memory?
Delphi's OleVariant type is a wrapper for OLE's VARIANT record. The only type of array that OLE supports is SAFEARRAY, and any SAFEARRAY created by a Win32 SafeArrayCreate...() function allocates and owns the data block that it points to. You have to copy your source data into that block.
To bypass that, you would have to skip VarArrayCreate() (which calls SafeArrayCreate()) and allocate the SAFEARRAY yourself using SafeArrayAllocDescriptor/Ex() so it does not allocate a data block. Then you can set the array's pvData field to point at your existing memory block, and enable the FADF_AUTO flag in its fFeatures field to tell SafeArrayDestroy() (which OleVariant calls when it does not need the SAFEARRAY anymore) to not free your memory block.
Try something like this:
uses
..., Ole2, ComObj;
// Delphi's Ole2 unit declares SafeArrayAllocDescriptor()
// but does not declare SafeArrayAllocDescriptorEx()...
function SafeArrayAllocDescriptorEx(vt: TVarType; cDims: Integer; var psaOut: PSafeArray): HResult; stdcall; external 'oleaut32.dll';
function GetVarArrayFromBuffer(ABuffer : pByte; ASizeInBytes: Cardinal) : OleVariant;
var
SA: PSafeArray;
begin
OleCheck(SafeArrayAllocDescriptorEx(VT_UI1, 1, SA));
SA.fFeatures := SA.fFeatures or FADF_AUTO or FADF_FIXEDSIZE;
SA.cbElements := SizeOf(Byte);
SA.pvData := ABuffer;
SA.rgsabound[0].lLbound := 0;
SA.rgsabound[0].cElements := ASizeInBytes;
TVarData(Result).VType := varByte or varArray;
TVarData(Result).VArray := PVarArray(SA);
end;
If you don't actually need to use OLE, such as if you are not passing your array to other people's applications via OLE, then you should use Delphi's Variant type instead. You can write a Custom Variant Type to hold whatever data you want, even a reference to your existing memory block, and then use Variant as needed and let your custom type implementation manage the data as needed.
You may be able to hack your way into having an OleVariant with your array data in it without copying it.
However, a problem you are going to have is when the OleVariant variable falls out of scope.
The RTL is going to call SafeArrayDestroy in oleaut32.dll to destroy the memory associated with the safe array, and that's going to fail because the memory did not come from where Windows expected.

How do I stop this Variant memory leak?

I'm using an old script engine that's no longer supported by its creators, and having some trouble with memory leaks. It uses a function written in ASM to call from scripts into Delphi functions, and returns the result as an integer then passes that integer as an untyped parameter to another procedure that translates it into the correct type.
This works fine for most things, but when the return type of the Delphi function was Variant, it leaks memory because the variant is never getting disposed of. Does anyone know how I can take an untyped parameter containing a variant and ensure that it will be disposed of properly? This will probably involve some inline assembly.
procedure ConvertVariant(var input; var output: variant);
begin
output := variant(input);
asm
//what do I put here? Input is still held in EAX at this point.
end;
end;
EDIT: Responding to Rob Kennedy's question in comments:
AnsiString conversion works like this:
procedure VarFromString2(var s : AnsiString; var v : Variant);
begin
v := s;
s := '';
end;
procedure StringToVar(var p; var v : Variant);
begin
asm
call VarFromString2
end;
end;
That works fine and doesn't produce memory leaks. When I try to do the same thing with a variant as the input parameter, and assign the original Null on the second procedure, the memory leaks still happen.
The variants mostly contain strings--the script in question is used to generate XML--and they got there by assigning a Delphi string to a variant in the Delphi function that this script is calling. (Changing the return type of the function wouldn't work in this case.)
Have you tried the same trick as with the string, except that with a Variant, you should put UnAssigned instead of Null to free it, like you did s := ''; for the string.
And by the way, one of the only reasons I can think of that requires to explicitly free the strings, Variants, etc... is when using some ThreadVar.

Delphi 2009 and copying memory

I'm testing DelphiModbus library on Delphi 2009 and don't get quite the results I want. I think the problem lies with the following line on IdModbusClient.pas:
Move(Buffer, ReceiveBuffer, iSize);
It looks like ReceiveBuffer is set to some garbage.
Buffer is defined as TIdBytes (from Indy components)
ReceiveBuffer is defined as TCommsBuffer:
TModBusFunction = Byte;
TModBusDataBuffer = array[0..256] of Byte;
TCommsBuffer = packed record
TransactionID: Word;
ProtocolID: Word;
RecLength: Word;
UnitID: Byte;
FunctionCode: TModBusFunction;
MBPData: TModBusDataBuffer;
Spare: Byte;
end; { TCommsBuffer }
And iSize is of course the size of the Buffer in bytes.
I wonder if this has anything to do with unicode conversion?
Indy's TIdBytes type is a dynamic array, defined in IdGlobal.pas:
type
TIdBytes = array of Byte;
You can't pass a variable of that type directly to Move and expect it to work because it will only copy the four-byte reference stored in that variable. (And if you told it to copy more than four bytes, then it will proceed to copy whatever else resides in memory after that variable — who knows what.) Given these declarations:
var
Buffer: TIdBytes;
ReceiveBuffer: TCommsBuffer;
The way to call Move on those variables is like this:
if Length(Buffer) > 0 then
Move(Buffer[0], ReceiveBuffer, iSize);
It works like that because Move's parameters are untyped, so you need to pass the value that you want to copy, not a pointer or reference to the value. The compiler handles the referencing by itself.
The code is a little weird because it looks like you're just copying one byte out of Buffer, but don't let it bother you too much. It's a Delphi idiom; it's just the way it works.
Also, this has nothing to do with Delphi 2009; it's worked this way ever since Delphi 4, when dynamic arrays were introduced. And Move has been this way forever.
It looks to me like you're missing a couple of pointer dereferences, and therefore corrupting memory addresses.
If I'm not mistaken, the Move() call should be:
Move(Buffer^, ReceiveBuffer^, iSize);
I've removed my totally worthless post content (leaving it for posterity and to give someone a good laugh).
I don't see anything that would be affected by Unicode at all. I'm going to edit the tags to include Delphi (without the 2009), as some of the CodeGear Delphi developers are currently posting there. Perhaps one of them can see what's happening.
I made up a contrived example (actually a pretty useless one):
uses
IdGlobal;
type
TModBusFunction = Byte;
TModBusDataBuffer = array[0..256] of Byte;
TCommsBuffer=packed record
TransactionID: Word;
ProtocolID: Word;
RecLength: Word;
UnitID: Byte;
FunctionCode: TModBusFunction;
MBPData: TModBusDataBuffer;
Spare: Byte;
end;
procedure TForm1.FormShow(Sender: TObject);
var
Buffer: TIdBytes;
ReceiveBuffer: TCommsBuffer;
//iSize: Word;
begin
FillChar(ReceiveBuffer, SizeOf(ReceiveBuffer), 0);
ReceiveBuffer.TransactionID := 1;
ReceiveBuffer.ProtocolID := 2;
ReceiveBuffer.RecLength := 3;
ReceiveBuffer.UnitID := 4;
ReceiveBuffer.FunctionCode := 5;
FillChar(ReceiveBuffer.MBPData[0], SizeOf(ReceiveBuffer.MBPData), 6);
ReceiveBuffer.Spare := 7;
SetLength(Buffer, SizeOf(ReceiveBuffer));
Move(ReceiveBuffer, Buffer, SizeOf(ReceiveBuffer));
Move(Buffer, ReceiveBuffer, SizeOf(ReceiveBuffer));
ReceiveBuffer.UnitID := 8;
end;
I then set a breakpoint on the last line before the end, and ran it. When the breakpoint was hit, I looked at the contents of ReceiveBuffer using ToolTip Evaluation, and everything looked perfectly fine. I could see all of the proper values, including the ReceiveBuffer.Spare being 7. I then single stepped, and looked at ReceiveBuffer.UnitID; it in fact had a value of 8.
However, pressing F9 to continue running (expecting to be able to just close the form and end the application), I ended up in the CPU window and got a message from Vista that the application wasn't responding. I was just outside ntdll.DebugBreakPoint, IIRC, and single stepping brought me into ntdll.RtlReportException. I'm not quite sure what's happening, but it ain't good. Still looking.
Edit2: I ran it again, with the same results. However, this time I noticed before I used Ctrl+F2 to terminate the app that Vista was giving me a popup tooltray window indicating that "Project1.exe has been closed" and mentioning DEP (which I have enabled in hardware on this machine).

Resources