Adding large resources with UpdateResource - delphi

Nowhere in the Windows documentation do I see a reference to a size limit to the resources one can add using UpdateResource, but it seems I have stumbled upon one - and it's tiny!
I was developing a Windows Ribbon app and wanted to programmatically build and attach the resource. Linking the resource using a $R directive worked just dandy, but I kept getting memory junk when attaching the very same thing from code.
I have managed to reduce it to a simple example using a string resource:
Handle := BeginUpdateResource(PChar(DestFileName), True);
try
AddResource(Handle, 'STRING', 'ManyXs', StrUtils.DupeString('X', 1000));
finally
EndUpdateResource(Handle, False);
end;
And AddResource is defined as:
procedure TForm2.AddResource(Handle: NativeUInt; ResType, ResName, Value: string);
begin
if not UpdateResource(Handle, PChar(ResType), PChar(ResName), 1033,
PChar(Value), Value.Length * SizeOf(Char)) then
RaiseLastOSError;
end;
Please ignore my hard-coded language for the moment.
When I inspect the resource subsequent to calling this, I see a thousand Xs. Fabulous.
I can change the resource to 1990 Xs and it's fine. The moment it goes to 1991, I get nonsense written to the DLL. The size of the resource is correctly indicated as 3982 (1991 * 2 because it's Unicode), but the contents is just a dump of stuff from memory.
I can view larger resources with my resource editor, and the IDE routinely inserts larger resources (Delphi forms, for example), so I'm definitely missing something.
I've tried the following, despite not thinking any of them would make a difference (they didn't):
Using just large memory buffers instead of strings
Using the Ansi version of the UpdateResource function
Many different resource types - what I really need to get working, is UIFILE
Looking for other functions in the API (I found none)
Combinations of 1, 2 and 3
Any ideas?
Update:
Inspired by the comments and Jolyon's answer, tried a few more things.
First, I tried in Delphi XE7 and XE5 as well (original was in XE6). I don't have XE2 installed anymore, so i cannot confirm what Sertak has said. I'll find out if someone else in my office still has it installed.
Second, here is the memory buffer version:
procedure TForm2.AddResource(Handle: NativeUInt; const ResType, ResName, Value: string);
var
Buffer: Pointer;
BuffLen: Integer;
begin
BuffLen := Value.Length * SizeOf(Char);
GetMem(Buffer, BuffLen);
try
StrPCopy(PChar(Buffer), Value);
if not UpdateResource(Handle, PChar(ResType), PChar(ResName), 1033,
Buffer, BuffLen) then
RaiseLastOSError;
finally
FreeMem(Buffer);
end;
end;
I actually had a previous version of this code where I dumped the contents of that pointer into a file before the call to UpdateResource and the file saved correctly but the resource still saved junk. Then I did this version, which doesn't involve strings at all:
procedure TForm2.AddResource(Handle: NativeUInt; const ResType, ResName: string;
C: AnsiChar; Len: Integer );
var
Buffer: Pointer;
BuffLen: Integer;
begin
BuffLen := Len;
GetMem(Buffer, BuffLen);
try
FillMemory(Buffer, Len, Byte(C));
if not UpdateResource(Handle, PChar(ResType), PChar(ResName), 1033,
Buffer, BuffLen) then
RaiseLastOSError;
finally
FreeMem(Buffer);
end;
end;
With this version I still have the same problem when I use 3882 Xs. Of course, I'm now using single-byte characters, that's why it's double. But I have the exact same issue.
I did notice a difference between the versions in the output of TDUMP though. For versions 1 (strings) and 2 (string copied to buffer), my resource size is suddenly indicated as FFFFFF90 when I use 1991 characters. With version 3 (no strings), the size is the actual hex value of whatever size I used.

The fact that you are getting "junk" data but data of the right size leads me to suspect the PChar() casting of the string value yielding an incorrect address. This normally should not be a problem, but I wonder if the issue is some strange behaviour as the result of passing the result of a function directly into a parameter of a method ? A behaviour which for some strange reason is only triggered when the string involved reaches a certain size, perhaps indicating some edge-case optimization behaviour.
This might also explain difficulties in reproducing the problem if it is some combination of optimization (and/or other compiler settings) in some specific version of Delphi.
I would suggest to try eliminating this possibility by creating your new resource string in an explicit variable and passing that to the AddResource() method. I would also suggest that you be explicit in your parameter semantics and since the string involved is not modified, nor intended to be modified, in the AddResource() method, declare it as a formally const parameter.
You do mention having tried an alternative approach using "memory buffers". If the above suggestions do not resolve the problem, perhaps it would be helpful to post a minimal example that reproduces the problem using those, to eliminate any possible influence on things by the rather more exotic "string" type.

Related

Can I modify a constant in the RTL class System.Classes.TStream and rebuild it at runtime in Delphi XE6?

I am trying to work around a known ugly performance limitation in System.Classes.pas, which has a 1980s era constant buffer limit ($F000) that looks like this:
function TStream.CopyFrom(const Source: TStream; Count: Int64): Int64;
const
MaxBufSize = $F000;
....
This is causing major performance penalties in our Delphi application. In delphi XE2 through XE5, we were able to modify this and use one of the following approaches:
I could modify the Delphi sources, and then, by invoking dcc32.exe from a batch file, rebuild the System.Classes.dcu file in the Delphi library folder. I realize this is ugly and I didn't like doing this, but I don't like this ugly performance issue in the RTL either, and our users can not live with the performance headaches it causes.
I could try to put a modified system.classes.pas file somewhere in my project search path.
Neither of the above approaches is working for me in Delphi XE6, now, thanks probably to some internal compiler changes. The error I get in a minimal command line application that includes System.Contnrs in its uses clause, is this:
[dcc32 Fatal Error] System.Classes.pas(19600): F2051 Unit System.Contnrs was compiled with a different version of System.Classes.TComponent
The sample program to reproduce this problem (assuming you have modified System.Classes.pas and changed the MaxBufSize constant), is shown here:
program consoletestproject;
{$APPTYPE CONSOLE}
{$R *.res}
uses
System.Contnrs,
System.SysUtils;
var
List:System.Contnrs.TObjectList;
begin
WriteLn('Hello world');
end.
Again, this problem reproduces easily in Delphi XE6, but is not a problem in XE5, or earlier.
What is the recommended practice when you absolutely MUST work around a fundamental RTL or VCL limitation using a modified copy of System.Classes.pas or System.SysUtils.pas or some other very low level unit? (Yes, I know you should NOT do this if you don't have to, don't bother with a lecture.)
Are there a magic set of command line parameters you can use via "dcc32.exe" on the command line, to produce a modified DCU that will link properly with the application example above?
As a secondary question, are there .dcu files for which no source exists that will break when one tries to do this, in which case the answer to all of the above is, "you can't fix this, and if there's a bug in the RTL, you're out of luck"?
One possible workaround is to include "$(BDS)\source\rtl\common" in your project search path (or library path), forcing each broken (needing recompile) DCU to rebuild EACH time, but this seems ugly and wrong.
You can overcome this limitation using a detour, try this sample which uses the Delphi Detours Library
First define the signature of the method to hook
var
Trampoline_TStreamCopyFrom : function (Self : TStream;const Source: TStream; Count: Int64): Int64 = nil;
then implement the detour
function Detour_TStreamCopyFrom(Self : TStream;const Source: TStream; Count: Int64): Int64;
const
MaxBufSize = 1024*1024; //use 1 mb now :)
var
BufSize, N: Integer;
Buffer: TBytes;
begin
if Count <= 0 then
begin
Source.Position := 0;
Count := Source.Size;
end;
Result := Count;
if Count > MaxBufSize then BufSize := MaxBufSize else BufSize := Count;
SetLength(Buffer, BufSize);
try
while Count <> 0 do
begin
if Count > BufSize then N := BufSize else N := Count;
Source.ReadBuffer(Buffer, N);
Self.WriteBuffer(Buffer, N);
Dec(Count, N);
end;
finally
SetLength(Buffer, 0);
end;
end;
Finally replace the original function by the trampoline (you can use this code in the initialization part of some unit)
Trampoline_TStreamCopyFrom := InterceptCreate(#TStream.CopyFrom, #Detour_TStreamCopyFrom);
And to release the hook you can use
if Assigned(Trampoline_TStreamCopyFrom) then
InterceptRemove(#Trampoline_TStreamCopyFrom);
Update 1: The suggestion below does not work for the Classes unit in XE6. The basic technique is sound and does solve similar problems. But for XE6, at least the Classes unit, it is not immediately obvious how to re-compile it.
This appears to be a fault introduced in XE6 because this technique is meant to work and is officially endorsed by Embarcadero: http://blog.marcocantu.com/blog/2014_august_buffer_overflow_bitmap.html
Update 2:
In XE7, this problem no longer exists. It would appear that whatever was broken in XE6 has been fixed.
You need the compiler options to match those used when the unit was compiled by Embarcadero. That's the reason why your implementation section only change fails when it seems like it ought to succeed.
Start a default project and use CTRL + O + O to generate these options. I get
{$A8,B-,C+,D+,E-,F-,G+,H+,I+,J-,K-,L+,M-,N-,O+,P+,Q-,R-,S-,T-,U-,V+,W-,X+,Y+,Z1}
when I do this in XE6.
Put that at the top of your copy of the unit and you should be good to go. You can probably get away with a cut-down subset of these, depending on your host project options. In my code I find that:
{$R-,T-,H+,X+}
suffices.

How Can I Efficiently Read The FIrst Few Lines of Many Files in Delphi

I have a "Find Files" function in my program that will find text files with the .ged suffix that my program reads. I display the found results in an explorer-like window that looks like this:
I use the standard FindFirst / FindNext methods, and this works very quickly. The 584 files shown above are found and displayed within a couple of seconds.
What I'd now like to do is add two columns to the display that shows the "Source" and "Version" that are contained in each of these files. This information is found usually within the first 10 lines of each file, on lines that look like:
1 SOUR FTM
2 VERS Family Tree Maker (20.0.0.368)
Now I have no problem parsing this very quickly myself, and that is not what I'm asking.
What I need help with is simply how to most quickly load the first 10 or so lines from these files so that I can parse them.
I have tried to do a StringList.LoadFromFile, but it takes too much time loading the large files, such at those above 1 MB.
Since I only need the first 10 lines or so, how would I best get them?
I'm using Delphi 2009, and my input files might or might not be Unicode, so this needs to work for any encoding.
Followup: Thanks Antonio,
I ended up doing this which works fine:
var
CurFileStream: TStream;
Buffer: TBytes;
Value: string;
Encoding: TEncoding;
try
CurFileStream := TFileStream.Create(folder + FileName, fmOpenRead);
SetLength(Buffer, 256);
CurFileStream.Read(Buffer[0], 256);
TEncoding.GetBufferEncoding(Buffer, Encoding);
Value := Encoding.GetString(Buffer);
...
(parse through Value to get what I want)
...
finally
CurFileStream.Free;
end;
Use TFileStream and with Read method read number of bytes needed. Here is the example of reading bitmap info that is also stored on begining of the file.
http://www.delphidabbler.com/tips/19
Just open the file yourself for block reading (not using TStringList builtin functionality), and read the first block of the file, and then you can for example load that block to a stringlist with strings.SetText() (if you are using block functions) or simply strings.LoadFromStream() if you are loading your blocks using streams.
I would personally just go with FileRead/FileWrite block functions, and load the block into a buffer. You could also use similair winapi functions, but that's just more code for no reason.
OS reads files in blocks, which are at least 512bytes big on almost any platform/filesystem, so you can read 512 bytes first (and hope that you got all 10 lines, which will be true if your lines are generally short enough). This will be (practically) as fast as reading 100 or 200 bytes.
Then if you notice that your strings objects has only less than 10 lines, just read next 512 byte block and try to parse again. (Or just go with 1024, 2048 and so on blocks, on many systems it will probably be as fast as 512 blocks, as filesystem cluster sizes are generally larger than 512 bytes).
PS. Also, using threads or asynchronous functionality in winapi file functions (CreateFile and such), you could load that data from files asynchronously, while the rest of your application works. Specifically, the interface will not freeze during reading of large directories.
This will make the loading of your information appear faster, (since the file list will load directly, and then some milliseconds later the rest of the information will come up), while not actually increasing the real reading speed.
Do this only if you have tried the other methods and you feel like you need the extra boost.
You can use a TStreamReader to read individual lines from any TStream object, such as a TFileStream. For even faster file I/O, you could use Memory-Mapped Views with TCustomMemoryStream.
Okay, I deleted my first answer. Using Remy's first suggestion above, I tried again with built-in stuff. What I don't like here is that you have to create and free two objects. I think I would make my own class to wrap this up:
var
fs:TFileStream;
tr:TTextReader;
filename:String;
begin
filename := 'c:\temp\textFileUtf8.txt';
fs := TFileStream.Create(filename, fmOpenRead);
tr := TStreamReader.Create(fs);
try
Memo1.Lines.Add( tr.ReadLine );
finally
tr.Free;
fs.Free;
end;
end;
If anybody is interested in what I had here before, it had the problem of not working with unicode files.
Sometimes oldschool pascal stylee is not that bad.
Even though non-oo file access doesn't seem to be very popular anymore, ReadLn(F,xxx) still works pretty ok in situations like yours.
The code below loads information (filename, source and version) into a TDictionary so that you can look it up easily, or you can use a listview in virtual mode, and look stuff up in this list when the ondata even fires.
Warning: code below does not work with unicode.
program Project101;
{$APPTYPE CONSOLE}
uses
IoUtils, Generics.Collections, SysUtils;
type
TFileInfo=record
FileName,
Source,
Version:String;
end;
function LoadFileInfo(var aFileInfo:TFileInfo):Boolean;
var
F:TextFile;
begin
Result := False;
AssignFile(F,aFileInfo.FileName);
{$I-}
Reset(F);
{$I+}
if IOResult = 0 then
begin
ReadLn(F,aFileInfo.Source);
ReadLn(F,aFileInfo.Version);
CloseFile(F);
Exit(True)
end
else
WriteLn('Could not open ', aFileInfo.FileName);
end;
var
FileInfo:TFileInfo;
Files:TDictionary<string,TFileInfo>;
S:String;
begin
Files := TDictionary<string,TFileInfo>.Create;
try
for S in TDirectory.GetFiles('h:\WINDOWS\system32','*.xml') do
begin
WriteLn(S);
FileInfo.FileName := S;
if LoadFileInfo(FileInfo) then
Files.Add(S,FileInfo);
end;
// showing file information...
for FileInfo in Files.Values do
WriteLn(FileInfo.Source, ' ',FileInfo.Version);
finally
Files.Free
end;
WriteLn;
WriteLn('Done. Press any key to quit . . .');
ReadLn;
end.

Getting size of a file in Delphi 2010 or later?

Delphi 2010 has a nice set of new file access functions in IOUtils.pas (I especially like the UTC versions of the date-related functions). What I miss so far is something like
TFile.GetSize (const Path : String)
What is the Delphi 2010-way to get the size of a file? Do I have to go back and use FindFirst to access TSearchRec.FindData?
Thanks.
I'm not sure if there's a "Delphi 2010" way, but there is a Windows way that doesn't involve FindFirst and all that jazz.
I threw together this Delphi conversion of that routine (and in the process modified it to handle > 4GB size files, should you need that).
uses
WinApi.Windows;
function FileSize(const aFilename: String): Int64;
var
info: TWin32FileAttributeData;
begin
result := -1;
if NOT GetFileAttributesEx(PChar(aFileName), GetFileExInfoStandard, #info) then
EXIT;
result := Int64(info.nFileSizeLow) or Int64(info.nFileSizeHigh shl 32);
end;
You could actually just use GetFileSize() but this requires a file HANDLE, not just a file name, and similar to the GetCompressedFileSize() suggestion, this requires two variables to call. Both GetFileSize() and GetCompressedFileSize() overload their return value, so testing for success and ensuring a valid result is just that little bit more awkward.
GetFileSizeEx() avoids the nitty gritty of handling > 4GB file sizes and detecting valid results, but also requires a file HANDLE, rather than a name, and (as of Delphi 2009 at least, I haven't checked 2010) isn't declared for you in the VCL anywhere, you would have to provide your own import declaration.
Using an Indy unit:
uses IdGlobalProtocols;
function FileSizeByName(const AFilename: TIdFileName): Int64;
You can also use DSiFileSize from DSiWin32. Works in "all" Delphis. Internally it calls CreateFile and GetFileSize.
function DSiFileSize(const fileName: string): int64;
var
fHandle: DWORD;
begin
fHandle := CreateFile(PChar(fileName), 0, 0, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
if fHandle = INVALID_HANDLE_VALUE then
Result := -1
else try
Int64Rec(Result).Lo := GetFileSize(fHandle, #Int64Rec(Result).Hi);
finally CloseHandle(fHandle); end;
end; { DSiFileSize }
I'd like to mention few Pure Delphi ways. Though i think Deltics made a most speed-effective answer for Windows platform, yet sometimes you want just rely on RTL and also make portable code that would work in Delphi for MacOS or in FreePascal/Virtual Pascal/whatever.
There is FileSize function left from Turbo Pascal days.
http://turbopascal.org/system-functions-filepos-and-filesize
http://docwiki.embarcadero.com/CodeExamples/XE2/en/SystemFileSize_(Delphi)
http://docwiki.embarcadero.com/Libraries/XE2/en/System.FileSize
The sample above lacks "read-only" mode setting. You would require that to open r/o file such as one on CD-ROM media or in folder with ACLs set to r/o. Before calling ReSet there should be zero assigned to FileMode global var.
http://docwiki.embarcadero.com/Libraries/XE2/en/System.FileMode
It would not work on files above 2GB size (maybe with negative to cardinal cast - up to 4GB) but is "out of the box" one.
There is one more approach, that you may be familiar if you ever did ASM programming for MS-DOS. You Seek file pointer to 1st byte, then to last byte, and check the difference.
I can't say exactly which Delphi version introduced those, but i think it was already in some ancient version like D5 or D7, though that is just common sense and i cannot check it.
That would take you an extra THandle variable and try-finally block to always close the handle after size was obtained.
Sample of getting length and such
http://docwiki.embarcadero.com/Libraries/XE2/en/System.SysUtils.FileOpen
http://docwiki.embarcadero.com/Libraries/XE2/en/System.SysUtils.FileSeek
Aside from 1st approach this is int64-capable.
It is also compatible with FreePascal, though with some limitations
http://www.freepascal.org/docs-html/rtl/sysutils/fileopen.html
You can also create and use TFileStream-typed object - which was the primary, officially blessed avenue for file operations since Delphi 1.0
http://www.freepascal.org/docs-html/rtl/classes/tfilestream.create.html
http://www.freepascal.org/docs-html/rtl/classes/tstream.size.html
http://docwiki.embarcadero.com/Libraries/XE2/en/System.Classes.TFileStream.Create
http://docwiki.embarcadero.com/Libraries/XE2/en/System.Classes.TStream.Size
As a side note, this avenue is of course integrated with aforementioned IOUtils unit.
http://docwiki.embarcadero.com/Libraries/XE3/en/System.IOUtils.TFile.OpenRead
This is a short solution using FileSize that does the job:
function GetFileSize(p_sFilePath : string) : Int64;
var
oFile : file of Byte;
begin
Result := -1;
AssignFile(oFile, p_sFilePath);
try
Reset(oFile);
Result := FileSize(oFile);
finally
CloseFile(oFile);
end;
end;
From what I know, FileSize is available only from XE2.
uses
System.Classes, System.IOUtils;
function GetFileSize(const FileName : string) : Int64;
var
Reader: TFileStream;
begin
Reader := TFile.OpenRead(FileName);
try
result := Reader.Size;
finally
Reader.Free;
end;
end;

Elegant way for handling this string issue. (Unicode-PAnsiString issue)

Consider the following scenario:
type
PStructureForSomeCDLL = ^TStructureForSomeCDLL;
TStructureForSomeCDLL = record
pName: PAnsiChar;
end
function FillStructureForDLL: PStructureForSomeDLL;
begin
New(Result);
// Result.pName := PAnsiChar(SomeObject.SomeString); // Old D7 code working all right
Result.pName := Utf8ToAnsi(UTF8Encode(SomeObject.SomeString)); // New problematic unicode version
end;
...code to pass FillStructureForDLL to DLL...
The problem in unicode version is that the string conversion involved now returns a new string on stack and that's reclaimed at the end of the FillStructureForDLL call, leaving the DLL with corrupted data. In old D7 code, there were no intermediate conversion funcs and thus no problem.
My current solution is a converter function like below, which is IMO too much of an hack. Is there a more elegant way of achieving the same result?
var gKeepStrings: array of AnsiString;
{ Convert the given Unicode value S to ANSI and increase the ref. count
of it so that returned pointer stays valid }
function ConvertToPAnsiChar(const S: string): PAnsiChar;
var temp: AnsiString;
begin
SetLength(gKeepStrings, Length(gKeepStrings) + 1);
temp := Utf8ToAnsi(UTF8Encode(S));
gKeepStrings[High(gKeepStrings)] := temp; // keeps the resulting pointer valid
// by incresing the ref. count of temp.
Result := PAnsiChar(temp);
end;
One way might be to tackle the problem before it becomes a problem, by which I mean adapt the class of SomeObject to maintain an ANSI Encoded version of SomeString (ANSISomeString?) for you alongside the original SomeString, keeping the two in step in a "setter" for the SomeString property (using the same UTF8 > ANSI conversion you are already doing).
In non-Unicode versions of the compiler make ANSISomeString be simply a "copy" of SomeString string, which will of course not be a copy, merely an additional ref count on SomeString. In the Unicode version it references a separate ANSI encoding with the same "lifetime" as the original SomeString.
procedure TSomeObjectClass.SetSomeString(const aValue: String);
begin
fSomeString := aValue;
{$ifdef UNICODE}
fANSISomeString := Utf8ToAnsi(UTF8Encode(aValue));
{$else}
fANSISomeString := fSomeString;
{$endif}
end;
In your FillStructure... function, simply change your code to refer to the ANSISomeString property - this then is entirely independent of whether compiling for Unicode or not.
function FillStructureForDLL: PStructureForSomeDLL;
begin
New(Result);
result.pName := PANSIChar(SomeObject.ANSISomeString);
end;
There are at least three ways to do this.
You could change SomeObject's class
definition to use an AnsiString
instead of a string.
You could
use a conversion system to hold
references, like in your example.
You could initialize result.pname
with GetMem and copy the result of the
conversion to result.pname^ with
Move. Just remember to FreeMem it
when you're done.
Unfortunately, none of them is a perfect solution. So take a look at the options and decide which one works best for you.
Hopefully you already have code in your application to properly dispose off of all the dynamically allocated records that you New() in FillStructureForDLL(). I consider this code highly dubious, but let's assume this is reduced code to demonstrate the problem only. Anyway, the DLL you pass the record instance to does not care how big the chunk of memory is, it will only get a pointer to it anyway. So you are free to increase the size of the record to make place for the Pascal string that is now a temporary instance on the stack in the Unicode version:
type
PStructureForSomeCDLL = ^TStructureForSomeCDLL;
TStructureForSomeCDLL = record
pName: PAnsiChar;
// ... other parts of the record
pNameBuffer: string;
end;
And the function:
function FillStructureForDLL: PStructureForSomeDLL;
begin
New(Result);
// there may be a bug here, can't test on the Mac... idea should be clear
Result.pNameBuffer := Utf8ToAnsi(UTF8Encode(SomeObject.SomeString));
Result.pName := Result.pNameBuffer;
end;
BTW: You wouldn't even have that problem if the record passed to the DLL was a stack variable in the procedure or function that calls the DLL function. In that case the temporary string buffers will only be necessary in the Unicode version if more than one PAnsiChar has to be passed (the conversion calls would otherwise reuse the temporary string). Consider changing the code accordingly.
Edit:
You write in a comment:
This would be best solution if modifying the DLL structures were an option.
Are you sure you can't use this solution? The point is that from the POV of the DLL the structure isn't modified at all. Maybe I didn't make myself clear, but the DLL will not care whether a structure passed to it is exactly what it is declared to be. It will be passed a pointer to the structure, and this pointer needs to point to a block of memory that is at least as large as the structure, and needs to have the same memory layout. However, it can be a block of memory that is larger than the original structure, and contain additional data.
This is actually used in quite a lot of places in the Windows API. Did you ever wonder why there are structures in the Windows API that contain as the first thing an ordinal value giving the size of the structure? It's the key to API evolution while preserving backwards compatibility. Whenever new information is needed for the API function to work it is simply appended to the existing structure, and a new version of the structure is declared. Note that the memory layout of older versions of the structure is preserved. Old clients of the DLL can still call the new function, which will use the size member of the structure to determine which API version is called.
In your case no different versions of the structure exist as far as the DLL is concerned. However, you are free to declare it larger for your application than it really is, provided the memory layout of the real structure is preserved, and additional data is only appended. The only case where this wouldn't work is when the last part of the structure were a record with varying size, kind of like the Windows BITMAP structure - a fixed header and dynamic data. However, your record looks like it has a fixed length.
Wouldn't PChar(AnsiString(SomeObject.SomeString)) work?

Delphi 2009 and copying memory

I'm testing DelphiModbus library on Delphi 2009 and don't get quite the results I want. I think the problem lies with the following line on IdModbusClient.pas:
Move(Buffer, ReceiveBuffer, iSize);
It looks like ReceiveBuffer is set to some garbage.
Buffer is defined as TIdBytes (from Indy components)
ReceiveBuffer is defined as TCommsBuffer:
TModBusFunction = Byte;
TModBusDataBuffer = array[0..256] of Byte;
TCommsBuffer = packed record
TransactionID: Word;
ProtocolID: Word;
RecLength: Word;
UnitID: Byte;
FunctionCode: TModBusFunction;
MBPData: TModBusDataBuffer;
Spare: Byte;
end; { TCommsBuffer }
And iSize is of course the size of the Buffer in bytes.
I wonder if this has anything to do with unicode conversion?
Indy's TIdBytes type is a dynamic array, defined in IdGlobal.pas:
type
TIdBytes = array of Byte;
You can't pass a variable of that type directly to Move and expect it to work because it will only copy the four-byte reference stored in that variable. (And if you told it to copy more than four bytes, then it will proceed to copy whatever else resides in memory after that variable — who knows what.) Given these declarations:
var
Buffer: TIdBytes;
ReceiveBuffer: TCommsBuffer;
The way to call Move on those variables is like this:
if Length(Buffer) > 0 then
Move(Buffer[0], ReceiveBuffer, iSize);
It works like that because Move's parameters are untyped, so you need to pass the value that you want to copy, not a pointer or reference to the value. The compiler handles the referencing by itself.
The code is a little weird because it looks like you're just copying one byte out of Buffer, but don't let it bother you too much. It's a Delphi idiom; it's just the way it works.
Also, this has nothing to do with Delphi 2009; it's worked this way ever since Delphi 4, when dynamic arrays were introduced. And Move has been this way forever.
It looks to me like you're missing a couple of pointer dereferences, and therefore corrupting memory addresses.
If I'm not mistaken, the Move() call should be:
Move(Buffer^, ReceiveBuffer^, iSize);
I've removed my totally worthless post content (leaving it for posterity and to give someone a good laugh).
I don't see anything that would be affected by Unicode at all. I'm going to edit the tags to include Delphi (without the 2009), as some of the CodeGear Delphi developers are currently posting there. Perhaps one of them can see what's happening.
I made up a contrived example (actually a pretty useless one):
uses
IdGlobal;
type
TModBusFunction = Byte;
TModBusDataBuffer = array[0..256] of Byte;
TCommsBuffer=packed record
TransactionID: Word;
ProtocolID: Word;
RecLength: Word;
UnitID: Byte;
FunctionCode: TModBusFunction;
MBPData: TModBusDataBuffer;
Spare: Byte;
end;
procedure TForm1.FormShow(Sender: TObject);
var
Buffer: TIdBytes;
ReceiveBuffer: TCommsBuffer;
//iSize: Word;
begin
FillChar(ReceiveBuffer, SizeOf(ReceiveBuffer), 0);
ReceiveBuffer.TransactionID := 1;
ReceiveBuffer.ProtocolID := 2;
ReceiveBuffer.RecLength := 3;
ReceiveBuffer.UnitID := 4;
ReceiveBuffer.FunctionCode := 5;
FillChar(ReceiveBuffer.MBPData[0], SizeOf(ReceiveBuffer.MBPData), 6);
ReceiveBuffer.Spare := 7;
SetLength(Buffer, SizeOf(ReceiveBuffer));
Move(ReceiveBuffer, Buffer, SizeOf(ReceiveBuffer));
Move(Buffer, ReceiveBuffer, SizeOf(ReceiveBuffer));
ReceiveBuffer.UnitID := 8;
end;
I then set a breakpoint on the last line before the end, and ran it. When the breakpoint was hit, I looked at the contents of ReceiveBuffer using ToolTip Evaluation, and everything looked perfectly fine. I could see all of the proper values, including the ReceiveBuffer.Spare being 7. I then single stepped, and looked at ReceiveBuffer.UnitID; it in fact had a value of 8.
However, pressing F9 to continue running (expecting to be able to just close the form and end the application), I ended up in the CPU window and got a message from Vista that the application wasn't responding. I was just outside ntdll.DebugBreakPoint, IIRC, and single stepping brought me into ntdll.RtlReportException. I'm not quite sure what's happening, but it ain't good. Still looking.
Edit2: I ran it again, with the same results. However, this time I noticed before I used Ctrl+F2 to terminate the app that Vista was giving me a popup tooltray window indicating that "Project1.exe has been closed" and mentioning DEP (which I have enabled in hardware on this machine).

Resources