My program have several worker threads that calling a function in a dynamically loaded DLL file. The performance is slower than calling function in EXE file. My program made using Delphi. I don't use ShareMM. The function in DLL has many routines to read file into memory. The used calling convention is stdcall. Actually, the speed is very poor!
I have no idea since I just learned about using DLL. So what should I do to optimize the performance/speed of my program/DLL?
Sorry if my question is non sense. I am sure there is nothing wrong with my exe, I just moved my functions into DLL and the performance be slower. Please ignore disk/memory cache factor as I have mentioned the routines of my DLL.
Edited:
This is how my program load the DLL
DLLHandle := LoadLibrary(pwchar(path));
if DLLHandle <> 0 then
#CheckFile := GetProcAddress(DLLHandle, 'CheckFile');
In my worker threads, I always check the function using if Assigned(CheckFile) then then call CheckFile function.
Here illustration of my function
type
TCheckFile = function(const FileName: string; var FileType: WideString)
: Boolean; stdcall;
var
CheckFile: TCheckFile ;
Now, the code in DLL
function CheckFile(const FileName: string; var FileType: WideString)
: boolean; stdcall;
var
testCheckFile: TBla;
begin
Result := false;
testCheckFile := TBla.Create;
try
if testCheckFile.DoSomeRoutine(FileName, FileType) then
Result := true;
finally
testCheckFile.Free;
end;
end;
exports CheckFile;
begin
IsMultiThread := true;
end.
What my DLL do? It plays with TFileStream like convert file to pointer.
I hope there is something wrong with my loading code and the calling code.
Code that resides in a DLL runs at just the same speed as code that resides in the host executable. It is exceedingly unlikely that moving code to a DLL will result in a discernible drop in performance.
However, you state in comments to the question that you have also ported from Delphi 2007 to Delphi XE2. That is almost certainly the change that resulted in the performance drop.
When measuring and comparing performance it is simply crucial to change one thing at a time so that you remove any possibility for confounding factors.
Maybe the problem is having to do with: " dynamically loaded DLL file". Dynamic is ok, but once you load it, keep it loaded right? If you keep loading/unloading for every function call, it's going to be slow (and a lot slower in the debugger!)
Related
I have created a DLL file with some functions and wish to reuse in a program multiple times in its different functions. But the Access-violation error comes after 2nd function of the program when calls the same DLL functions.
I'm currently using GetProcAddress. For example:
function xyz:boolean
var
dllHandle : cardinal;
EnBFStr : TEnBFStr;
StrToHex : TStrToHex;
Encodeddata , HexString : UnicodeString;
begin
dllHandle := LoadLibrary('Utilities.dll') ;
if dllHandle <> 0 then
begin
Encodeddata:='Sample';
#EnBFStr := GetProcAddress(dllHandle, 'EncodeBlowFishString') ;
#StrToHex := GetProcAddress(dllHandle, 'UniStrToUniHexStr') ;
if Assigned (EnBFStr) then
Encodeddata:=EnBFStr('Key','Text') ; //Sample would be replaced
if Assigned (StrToHex ) then
HexString :=StrToHex(Encodeddata) ; //call the function
FreeLibrary(dllHandle) ;
end;
There are other functions which is loading the library and calling these DLL functions multiple times. Also, within the same procedure/function, we are calling these DLL functions multiple times in (IF Else) conditions.
In earlier part of the program, I have tried to check for the DLL file is present. Also, I tried to directly load the functions as another alternative:
function EncodeBlowFishString (Const Key:UnicodeString; Const DecodedString:UnicodeString; ): UnicodeString; stdcall;
external 'Utilities.dll' name 'EncodeBlowFishString';
function UniStrToUniHexStr(Const aString:UnicodeString): UnicodeString; stdcall;
external 'Utilities.dll';
You are breaking the rules of memory allocation for DLLs. The return value is allocated by the callee but deallocated by the caller. Two solutions:
Use ShareMem as described in the comment at the top of a new library project.
Use standard interoperability techniques to ensure that allocation and deallocation always happens in the same module.
As an aside it is greatly wasteful to load and unload a DLL each time you want to use it. Load the DLL once only.
Furthermore I would like to point out that encryption operates on binary data and in my view you are storing up a world of pain by working instead with text.
I am trying to work around a known ugly performance limitation in System.Classes.pas, which has a 1980s era constant buffer limit ($F000) that looks like this:
function TStream.CopyFrom(const Source: TStream; Count: Int64): Int64;
const
MaxBufSize = $F000;
....
This is causing major performance penalties in our Delphi application. In delphi XE2 through XE5, we were able to modify this and use one of the following approaches:
I could modify the Delphi sources, and then, by invoking dcc32.exe from a batch file, rebuild the System.Classes.dcu file in the Delphi library folder. I realize this is ugly and I didn't like doing this, but I don't like this ugly performance issue in the RTL either, and our users can not live with the performance headaches it causes.
I could try to put a modified system.classes.pas file somewhere in my project search path.
Neither of the above approaches is working for me in Delphi XE6, now, thanks probably to some internal compiler changes. The error I get in a minimal command line application that includes System.Contnrs in its uses clause, is this:
[dcc32 Fatal Error] System.Classes.pas(19600): F2051 Unit System.Contnrs was compiled with a different version of System.Classes.TComponent
The sample program to reproduce this problem (assuming you have modified System.Classes.pas and changed the MaxBufSize constant), is shown here:
program consoletestproject;
{$APPTYPE CONSOLE}
{$R *.res}
uses
System.Contnrs,
System.SysUtils;
var
List:System.Contnrs.TObjectList;
begin
WriteLn('Hello world');
end.
Again, this problem reproduces easily in Delphi XE6, but is not a problem in XE5, or earlier.
What is the recommended practice when you absolutely MUST work around a fundamental RTL or VCL limitation using a modified copy of System.Classes.pas or System.SysUtils.pas or some other very low level unit? (Yes, I know you should NOT do this if you don't have to, don't bother with a lecture.)
Are there a magic set of command line parameters you can use via "dcc32.exe" on the command line, to produce a modified DCU that will link properly with the application example above?
As a secondary question, are there .dcu files for which no source exists that will break when one tries to do this, in which case the answer to all of the above is, "you can't fix this, and if there's a bug in the RTL, you're out of luck"?
One possible workaround is to include "$(BDS)\source\rtl\common" in your project search path (or library path), forcing each broken (needing recompile) DCU to rebuild EACH time, but this seems ugly and wrong.
You can overcome this limitation using a detour, try this sample which uses the Delphi Detours Library
First define the signature of the method to hook
var
Trampoline_TStreamCopyFrom : function (Self : TStream;const Source: TStream; Count: Int64): Int64 = nil;
then implement the detour
function Detour_TStreamCopyFrom(Self : TStream;const Source: TStream; Count: Int64): Int64;
const
MaxBufSize = 1024*1024; //use 1 mb now :)
var
BufSize, N: Integer;
Buffer: TBytes;
begin
if Count <= 0 then
begin
Source.Position := 0;
Count := Source.Size;
end;
Result := Count;
if Count > MaxBufSize then BufSize := MaxBufSize else BufSize := Count;
SetLength(Buffer, BufSize);
try
while Count <> 0 do
begin
if Count > BufSize then N := BufSize else N := Count;
Source.ReadBuffer(Buffer, N);
Self.WriteBuffer(Buffer, N);
Dec(Count, N);
end;
finally
SetLength(Buffer, 0);
end;
end;
Finally replace the original function by the trampoline (you can use this code in the initialization part of some unit)
Trampoline_TStreamCopyFrom := InterceptCreate(#TStream.CopyFrom, #Detour_TStreamCopyFrom);
And to release the hook you can use
if Assigned(Trampoline_TStreamCopyFrom) then
InterceptRemove(#Trampoline_TStreamCopyFrom);
Update 1: The suggestion below does not work for the Classes unit in XE6. The basic technique is sound and does solve similar problems. But for XE6, at least the Classes unit, it is not immediately obvious how to re-compile it.
This appears to be a fault introduced in XE6 because this technique is meant to work and is officially endorsed by Embarcadero: http://blog.marcocantu.com/blog/2014_august_buffer_overflow_bitmap.html
Update 2:
In XE7, this problem no longer exists. It would appear that whatever was broken in XE6 has been fixed.
You need the compiler options to match those used when the unit was compiled by Embarcadero. That's the reason why your implementation section only change fails when it seems like it ought to succeed.
Start a default project and use CTRL + O + O to generate these options. I get
{$A8,B-,C+,D+,E-,F-,G+,H+,I+,J-,K-,L+,M-,N-,O+,P+,Q-,R-,S-,T-,U-,V+,W-,X+,Y+,Z1}
when I do this in XE6.
Put that at the top of your copy of the unit and you should be good to go. You can probably get away with a cut-down subset of these, depending on your host project options. In my code I find that:
{$R-,T-,H+,X+}
suffices.
I need to use a 3rd party dll in our main app. When I staticly link to the provided DLL it works ok and I can the DLLs exported functions.
But we don't want our main app dependend on this dll on startup so I tried to dynamicly load the DLL when I need it :
DLLHandle := LoadLibrary('3rdparty.dll');
ret := GetLastError();
if DLLHandle = 0 then
begin
err := SysErrorMessage(ret);
Writeln(err);
end //...
but did doesnt work : The LoadLibrary function returns 0 and the LastErrorcode is 3221225616. Because I don't know what I'm doing wrong I tried the same (on the same pc) coded in c and it works : but what doesn't it work with delphi ? :
I call the same LoadLibrary function on the same dll!
When I monitor with ProcMon I see that the 3rdparty dll gets loaded and that also the dependand dlls of the 3rdparty dll gets loaded. : So windows certainly finds the DLL.
But somewhere it the loading process it fails :
When I try to load the DLL with LoadLibraryEX with DONT_RESOLVE_DLL_REFERENCES or LOAD_LIBRARY_AS_DATAFILE it also works (but I can't offcourse call the needed functions...)
I'm out of ideas : hope you guys can help me further...
thx in adv.
Kristof
Does this work?
var
SavedCW: word;
...
SavedCW := Get8087CW;
Set8087CW(SavedCW or $7);
DLLHandle := LoadLibrary('3rdparty.dll');
Set8087CW(SavedCW);
if DLLHandle = 0 then
begin
ret := GetLastError();
err := SysErrorMessage(ret);
Writeln(err);
end //...
Some discussion:
The error code, 3221225616, seems, when asking Google, to be the result of an invalid floating point operation. Now, this seems very technical; indeed, what does loading a library have to do with floating point computations? The floating point control word (CW) is a bitfield where the bits specify how the processor should handle floating-point errors; it is actually rather common that unexpected floating point errors can be dealt with by changing one of these bits to 1 (which by the way is the default state). For an other example, see this question of mine, in which I get a totally unexpected division by zero error, which is dealt with by setting the "div by zero" bit of the control word to 1.
3221225616 = STATUS_FLOAT_INVALID_OPERATION. My wild guess is that the FPU CW is different in your Delphi and C apps, and that your DLL's initialization is sensitive to this.
Possibly related: http://discuss.joelonsoftware.com/default.asp?joel.3.88583.15
Try using SafeLoadLibrary() in the Delphi RTL instead of the Win32 LoadLibrary. This function preserves the FP control word before calling LoadLibrary, and sets it back to what Delphi wants after the LoadLibrary returns.
I think that you should report to 3rdparty.dll's developers about a bug in their DLL.
I know this is an old thread but I just came across the same problem with a DLL written in VB.
This solution works for both x86 and x64
var ret:cardinal;
em:TArithmeticExceptionMask;
begin
result:= 1;
If Lib <> 0 Then exit; // already loaded
em:=GetExceptionmask;
SetExceptionmask(em+[exInvalidOp,exZeroDivide,exOverflow, exUnderflow]);
Lib := LoadLibrary(DLLname);
SetExceptionmask(em);
ret := GetLastError;
if ret<>0 then
raise exception.create(SysErrorMessage(ret));
I have a Com Object, setup/create/working from a DataModule.
creating/running/freeing the Datamodule from an Application works with out an issue.
but putting the datamodule into a DLL works fine the first time, runing the com object etc.. but after a few calls with out restarting the application, this error appears.
Error Message image http://darkaxi0m.name/so/errormessage.GIF
There is a fare bit of code in the App, so i cant post it all,
I have tried MadExcept in both the Application and Dll, with no luck. The IDE Breaks at a point that does not seem much help...
alt text http://darkaxi0m.name/so/cpubreak.gif
this is the code that handles the DataModule, the same function is used in the Application and the Dll in both tests
function GetAmount( Amount : integer; var Info: PChar): integer; stdcall;
var
tempInfo: string;
workerDM : TworkerDM;
begin
Result := 0;
workerDM := TworkerDM.Create(nil);
try
tempInfo:= Info;
Result := workerDM.GetAmount(Amount, tempInfo);
StrPCopy(Info, tempInfo);
finally
workerDM.Free;
end;
end;
i would like to blame the Ole Object, but it works fine out of the Dll
I'm at a loss to even think where to start looking.
In the finally, you are calling Free, but should call workerDM.Free.
I don't believe this question can be answered any more.
The project has be scraped, and the object that produce the error no longer used.
My Delete Requests have gone unanswered.
So this is now my answer.
Delphi 2010 has a nice set of new file access functions in IOUtils.pas (I especially like the UTC versions of the date-related functions). What I miss so far is something like
TFile.GetSize (const Path : String)
What is the Delphi 2010-way to get the size of a file? Do I have to go back and use FindFirst to access TSearchRec.FindData?
Thanks.
I'm not sure if there's a "Delphi 2010" way, but there is a Windows way that doesn't involve FindFirst and all that jazz.
I threw together this Delphi conversion of that routine (and in the process modified it to handle > 4GB size files, should you need that).
uses
WinApi.Windows;
function FileSize(const aFilename: String): Int64;
var
info: TWin32FileAttributeData;
begin
result := -1;
if NOT GetFileAttributesEx(PChar(aFileName), GetFileExInfoStandard, #info) then
EXIT;
result := Int64(info.nFileSizeLow) or Int64(info.nFileSizeHigh shl 32);
end;
You could actually just use GetFileSize() but this requires a file HANDLE, not just a file name, and similar to the GetCompressedFileSize() suggestion, this requires two variables to call. Both GetFileSize() and GetCompressedFileSize() overload their return value, so testing for success and ensuring a valid result is just that little bit more awkward.
GetFileSizeEx() avoids the nitty gritty of handling > 4GB file sizes and detecting valid results, but also requires a file HANDLE, rather than a name, and (as of Delphi 2009 at least, I haven't checked 2010) isn't declared for you in the VCL anywhere, you would have to provide your own import declaration.
Using an Indy unit:
uses IdGlobalProtocols;
function FileSizeByName(const AFilename: TIdFileName): Int64;
You can also use DSiFileSize from DSiWin32. Works in "all" Delphis. Internally it calls CreateFile and GetFileSize.
function DSiFileSize(const fileName: string): int64;
var
fHandle: DWORD;
begin
fHandle := CreateFile(PChar(fileName), 0, 0, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
if fHandle = INVALID_HANDLE_VALUE then
Result := -1
else try
Int64Rec(Result).Lo := GetFileSize(fHandle, #Int64Rec(Result).Hi);
finally CloseHandle(fHandle); end;
end; { DSiFileSize }
I'd like to mention few Pure Delphi ways. Though i think Deltics made a most speed-effective answer for Windows platform, yet sometimes you want just rely on RTL and also make portable code that would work in Delphi for MacOS or in FreePascal/Virtual Pascal/whatever.
There is FileSize function left from Turbo Pascal days.
http://turbopascal.org/system-functions-filepos-and-filesize
http://docwiki.embarcadero.com/CodeExamples/XE2/en/SystemFileSize_(Delphi)
http://docwiki.embarcadero.com/Libraries/XE2/en/System.FileSize
The sample above lacks "read-only" mode setting. You would require that to open r/o file such as one on CD-ROM media or in folder with ACLs set to r/o. Before calling ReSet there should be zero assigned to FileMode global var.
http://docwiki.embarcadero.com/Libraries/XE2/en/System.FileMode
It would not work on files above 2GB size (maybe with negative to cardinal cast - up to 4GB) but is "out of the box" one.
There is one more approach, that you may be familiar if you ever did ASM programming for MS-DOS. You Seek file pointer to 1st byte, then to last byte, and check the difference.
I can't say exactly which Delphi version introduced those, but i think it was already in some ancient version like D5 or D7, though that is just common sense and i cannot check it.
That would take you an extra THandle variable and try-finally block to always close the handle after size was obtained.
Sample of getting length and such
http://docwiki.embarcadero.com/Libraries/XE2/en/System.SysUtils.FileOpen
http://docwiki.embarcadero.com/Libraries/XE2/en/System.SysUtils.FileSeek
Aside from 1st approach this is int64-capable.
It is also compatible with FreePascal, though with some limitations
http://www.freepascal.org/docs-html/rtl/sysutils/fileopen.html
You can also create and use TFileStream-typed object - which was the primary, officially blessed avenue for file operations since Delphi 1.0
http://www.freepascal.org/docs-html/rtl/classes/tfilestream.create.html
http://www.freepascal.org/docs-html/rtl/classes/tstream.size.html
http://docwiki.embarcadero.com/Libraries/XE2/en/System.Classes.TFileStream.Create
http://docwiki.embarcadero.com/Libraries/XE2/en/System.Classes.TStream.Size
As a side note, this avenue is of course integrated with aforementioned IOUtils unit.
http://docwiki.embarcadero.com/Libraries/XE3/en/System.IOUtils.TFile.OpenRead
This is a short solution using FileSize that does the job:
function GetFileSize(p_sFilePath : string) : Int64;
var
oFile : file of Byte;
begin
Result := -1;
AssignFile(oFile, p_sFilePath);
try
Reset(oFile);
Result := FileSize(oFile);
finally
CloseFile(oFile);
end;
end;
From what I know, FileSize is available only from XE2.
uses
System.Classes, System.IOUtils;
function GetFileSize(const FileName : string) : Int64;
var
Reader: TFileStream;
begin
Reader := TFile.OpenRead(FileName);
try
result := Reader.Size;
finally
Reader.Free;
end;
end;