Has anyone come across a framework or library for Delphi to simplify the generation of x86 code? I am not looking for an assembler, but rather a framework that abstracts the code generation process above the low level bits and bytes. Ideally I would like to build on top of an existing library or framework rather than hardcode the logic on a case by case basis.
The initial usage will be to generate small code stubs at runtime similar to the way Delphi dispatches SOAP requests. If I cannot find something I will likely roll my own, but I would hate to reinvent the wheel. Something in "C" might me interesting provided the license will permit translation and use in commercial and open source projects.
Update:
Here is some more context: What I am working toward is runtime implementation of interfaces and/or classes as part of a persistence framework. Something sort of like Java annotation driven persistence (JPA/EJB3) except with a distinctly Delphi flavor. The invocation target is a modular/extensible framework which will implement a generalized persistence model. I need to dispatch and hook method calls based on RTTI and an annotation/attribute model (something similar to InstantObjects metadata) in a very dynamic and fluid manner.
Thanks,
David
The more I have thought about your question. I am not sure if all you trying to just do Dynamic Method Invocation. Even though your asking about generating x86 code.
There are several techiniques that do this.
If you know the signature of the method in question you can do it easily by using a
TMethod and setting the method address and data.
procedure TForm8.Button1Click(Sender: TObject);
begin
Showmessage('Hello1');
end;
procedure TForm8.Button2Click(Sender: TObject);
var
M : TMethod;
begin
M.Code := MethodAddress('Button1Click');
M.Data := Self;
TNotifyEvent(M)(self);
end;
If you don't know the method signature you can write the class with {$METHODINFO ON}
Then use the functionality in ObjAuto.pas to invoke the method.
I have an example in my RTTI Presentation code from DelphiLive on how to do that.
According to features of PaxCompiler, you can create stand alone executable files.
Very spectulative answer:
Something like LLVM? I am not sure if it can be used from delphi or not, but you should be able to create dll's wth it.
Logically you would simply generate delphi code, compile to a DLL/BPL by cmdline compiler and then dyn load that one?
Unfortunately Delphi Explorer doesn't come with the cmdline compiler though. And your main binary would also have to be in Delphi Explorer (or at least in D2006 if that is binary compatible enough)
Any mix of Delphi versions (or Free Pascal) will probably not work on the package or HLL level, only at basic procedural DLL level.
I just found an interesting framework that does much of what I was looking for when I originally posted the question. A little late for my purposes, but thought someone else might find this useful:
DAsmJit a Delphi port of the asmjit project
Related
I trying to implement some basic automated testing on a 10 million LOC project that don't follow good OO pratices (ex: isolating business logic into classes/units) with the DUnit that comes along with Delphi 2010. I can't do normal unit testing on this project since each part of business logic is spread across a dozens of interdependent units, these 'groups' of units are, however, centered around certain 'main business logic screens' (ex: all invoice logic related units are centered on the main invoice screen), and since those screens are classes I can do 'main business logic screen class testing' instead of unit testing, but those 'main screens' still need a lot of stuff that is created during the process startup.
So I need to both:
Be able to run the bad project's startup stuff
Be able to access its objects
The bad project already have some exported functions that return pointers that I can cast to access it's objects, but I'm unable to call them either way:
If I create the bad project as a child process of the test process, the startup code run fine, but I can't find a way to call the exported functions without complex IPC methods or substantial change on the bad project's structure.
If I load the bad project's .exe as an dll as with the LoadLibrary function, calling any function exported by bad project's result in access violation and/or segfault errors, even this simple procedure:
procedure Test; {safecall;} {stdcall;}
begin
showmessage('Yay!');
end;
How can I do both?
The approach you're talking about (using exported functions) is not going to fly. The simplest form of communication between two Win32 programs is to have them use SendMessage or PostMessage to talk to each other. Locating the window handle (usually by window class name) is step 1, sending a message is step 2.
Secondly, DUnit gets you nowhere near your goal, and TTestCase cannot be extended neatly to be a GUI Controller as that's not what it's for. It's for unit testing. Round peg, square hole. Write TTestCases for classes you can hive off and test, and use DUnit to provide test coverage for those parts of your system that you can provide test coverage for.
For UI testing, use a completely separate framework. There are two basic approaches done by Delphi programmers for automated integration tests of the sort you're proposing.
A custom hack job. Such is what you are describing. Inside Embarcadero, there exists a framework which is called Zombie, something Nick blogged about back in 2007. Its approach is based on several kinds of "primitive IPC", usually involving a Win32 SendMessage or PostMessage window message from outside the program to a window handle of a control inside the program. However, the internal code IS SIGNIFICANTLY MODIFIED to permit zombie testing. No you can't have Teh Codez, they're internal and proprietary to Embarcadero. But it does illustrate that the approach does work, and does not require rewriting the whole application or writing a huge number of mock-classes, like unit testing
would have done. If you want to go down the hack route, you will be writing your own User Interface Testing Framework, which should probably be completely separate from and use no DUnit code. You are welcome to try, but I'm telling you, it's a serious impedance mismatch. If I was starting my own custom framework in 2013, it would be DCOM based, because Delphi DCOM server code could be simply conditionally compiled into many programs, and DCOM would handle the function call "marshalling" details for you. I suspect I would get a year into the project, and I would give up, because in the end, I doubt I could make any system (DCOM or Win32 message based) pay off.
A complete external testing tool which you write test scripts in, like AutomatedQA/SmartBear TestComplete. Your tests would not be compiled into a delphi test program, but run inside TestComplete, and Pascal-like script syntax is just one of the available options for writing your test scripts.
We have had the same problem here. It looks like you really need a delphi library project (*.dll) for the export functions to work, (I suspect no initalization of the framework takes place when calling the function directly on an executable, no warranties).
NOTE: We are still using Delphi 5, so no dunit intergration here.
The solution we've used is to adding the dunit sources to our project (the .exe project) with a conditional DEFINE, and use this conditional define in the startup unit.
Sample startup code from our application:
if ComServer.StartMode <> smAutomation then
begin
OurApplication.Login ;
end;
{$IFDEF _AS_TESTRUNNER_}
GUITestRunner.RunRegisteredTests;
{$ELSE}
if OurApplication.HasStartCommands then
begin
Application.ShowMainForm := False ;
end
else begin
if ComServer.StartMode = smAutomation then
Application.ShowMainForm := False
end;
Application.Run;
{$ENDIF}
OurApplication.Finalize;
When I use the _AS_TESTRUNNER_ conditional define, I must login first so our app (and db connections) get initialised. Followed bij the GUITestrunner of DUnit.
Testcases can be registered in the initialization part exactly as in the examples.
Works like a charm.
I am using Delphi XE. I have come across a memory leak problem using Delphi Soap. It turns out to be due to a missing .Free call in TWSDLLookup.Destroy, as described in QC 91160
The problem that I have is the described work-around, which is simply to add FLookup.Free to the TWSDLLookup.Destroy method.
I don't want to change the Delphi source, so I tried copying the unit to my project folder, making the change and recompiling, as described here in Tom's answer. The problem with this technique is that it apparently only works if you also recompile all the dependent units. I have tried copying just WSDLLookup.pas to my project directory and I get a Stackoverflow error. I'm not familiar with Web Services / SOAP so I don't know what other units I should copy over if I do use this technique.
Rob Kennedy's answer on the same page describes a different technique involving code hooking - but it doesn't seem to apply to object methods. I have done as he suggests and downloaded the free code for the TNT Unicode controls and located the relevant procedures, but I have been unable to find info on how to hook an object's methods - if indeed this is possible. If I could do this, I would then hook TWSDLLookup.Destroy and add the FLookup.Free call.
Any ideas for how to fix this will be much appreciated. I'm a bit of a newbie programmer so I'm hoping that I've missed something obvious?
What you are trying to do does in fact work fine. I tested it out myself. Here's the project file I used:
program WSDLLookupTest;
{$APPTYPE CONSOLE}
uses
WSDLLookup in 'WSDLLookup.pas';
var
intf: IInterface;
begin
intf := GetWSDLLookup as IInterface;
end.
I made a copy of the WSDLLookup.pas file and placed it in the same directory as the .dpr file. Then, in the copy rather than the original, I modified TWSDLLookup.Destroy.
destructor TWSDLLookup.Destroy;
begin
Beep;
ClearWSDLLookup;
FLookup.Free;
inherited;
end;
I added the Beep to prove to myself that this code was indeed being executed.
In your position I would definitely use this solution instead of attempting code hooks. And of course the other simple solution is to upgrade to a later Delphi version.
One thing to be careful of is to remember to remove the modified unit when you do upgrade. The leak was fixed in XE2.
After this question, I need to know what principles should be followed in order to make an encapsulation of a class in a dll compatible to other version of Delphi.
I made a class using generics feature in RAD2010 and make a dll which has a function that return an instance of it. When I tried to use the dll using BDS2006 or Delphi 6, the DLL didn't work as expected. But if I use RAD2010 in other computer, there is no issue. Is it caused by using the feature that not available in previous Delphi version (the stack<> stuffs?)?
For string matters, I already follow the comment guidance in the library file, that I put ShareMem in both library first uses clause and my project. And I have copied borlndmm.dll from RAD2010 to the same folder where I tried the DLL using BDS2006. It didn't crash, but it didn't work es expected. A function return an empty string when in RAD2010 environment it worked very well.
Once again, I have a question : what principles should be followed in order to make an encapsulation of a class in a dll compatible to other version of Delphi? Thank you in advance. (For encapsulating functions in a dll when no OOP is used, I have no issued for other version of Delphi).
The definition of a string changed with D2009. If you want to make string communication safe, use a PAnsiChar or a WideString.
The basic rule of communication through DLLs is to not use anything specific to Delphi, so no Delphi strings and no TObject descendants. Interfaces, records and COM types work fine, though.
You ask:
Once again, I have a question : what principles should be followed in order to make an encapsulation of a class in a dll compatible to other version of Delphi?
and there is only one: Don't do it. You can't do it. Either you write a DLL, then use idioms and data types that can safely be used in DLLs, which precludes (among other things) classes.
Or you write a BPL, then you can safely export classes, use strings and such, but you are tied to the same Delphi version. This limitation is of technical nature, so writing a DLL will not work around it. There may be tricks to overcome this, and there may be different Delphi versions that use the same class layout so that it works, but you must not tie your public DLL interface to such implementation details.
Stick with only fundamental types. If you use interfaces, create them using the type library editor so your constrained by the compatible types by the start. A good rule of thumb is to look at the windows API and try to emulate its calling conventions.
You can use classes in your DLL, you just can't expose them as such. A good idiom that works well for DLL's is the handle concept. Your DLL creates an object and returns a handle to that object. When you need to work with that object again, you pass a function in the DLL a handle. Just remember that your DLL needs to be completely responsible for the memory and lifetime of the object. Its a trivial process to create DLL functions to expose the pieces of the class that you will need access too.
From the Delphi side, you can then write a proxy wrapper which hides the handle from the user. For events you can use a callback method. Basically you pass non object function pointers to the dll, which then invokes the function on the event. A quick overview of this process is available on Delphi 3000.
I have a class in a unit. Usually, when I changed the algorithm of its methods, I have to recompile it and deliver the patch as a whole.
I think to create the instance of the class using DLL. After searching in delphi.about.com, I found that instead of using DLL, I can use BPL. It is a DLL for Delphi. The problem is almost all examples I found is only telling how to export a function.
I want to dynamically load the BPL, and whenever I replace the BPL, I can get the latest algorithm of the class, not only the functions I export.
Article I have read:
- http://delphi.about.com/od/objectpascalide/a/bpl_vs_dll.htm
- Plugins system for Delphi application - bpl vs dll?
- http://delphi.about.com/library/weekly/aa012301a.htm
Any URL or SAMPLE how to create a BPL from scratch to encapsulate a component or a class is greatly appreciated.
Dear Guru,
Suppose I have code like this:
unit unitA;
interface
type
B = class(TObject)
public
procedure HelloB;
end;
A = class(TObject)
public
function GetB: B;
function HelloA: String;
procedure Help;
end;
implementation
uses
Dialogs;
{ B }
procedure B.HelloB;
begin
ShowMessage('B');
end;
{ A }
function A.GetB: B;
begin
Result := B.Create;
end;
function A.HelloA: String;
begin
Result := 'Hello, this is A';
end;
procedure A.Help;
begin
//do something
end;
end.
I want to export all public methods of A. How to make it a DLL?
How to use it from another unit where to import it?
let's say:
var a: A;
a := A.Create;
a.GetB;
showMessage(a.HelloA);
A is not declared in the unit (it is in the DLL).
Please advise.
Hurray. I got it last night. All I have to do is make the object implement an interface which is used in the caller unit to catch the instance of object returned by the DLL.
Thank you all.
Mason nailed it already, but let me elaborate on why BPLs aren't what you are looking for.
BPLs are a means for the Delphi IDE to load components that share the same memory manager and RTL. (Type identity works almost transparently using BPLs)
However, the dependencies you are getting tied up in are almost always unacceptable. Except for the IDE, which cannot handle different versions of RTL and VCL anyway.
When you pass only interface references between your application and its DLLs, then you don't have to share RTL, VCL or shared packages at all.
It also means that you could write some DLLs in another language (C++, C#, FPC, another Delphi version), and still use objects. Which can be tempting when you don't want to port your main app but still want to use existing libraries that are not available for Delphi, or your version of Delphi.
The problem with putting a class in an external file is that your main application needs to know some way to refer to it. It will either have to descend from a base class that exposes all the methods you need as virtual methods, or implement an interface that contains all the functionality you need from it.
If you already know what the interface of the object should look like, and all you're changing is implementation details such as internal algorithms, probably the easiest thing would be to make your class implement an interface and put it in a DLL that exports a function that returns an instance of this interface. That way you don't need to worry about breaking your app up into packages, which can be a real hassle.
I see nothing in your problem description suggesting you would need to explicitly export anything from the package or that you would need to load it dynamically at run time. Instead, it's enough that your functions reside in a run-time package that can be replaced separately from the main program.
Start a new package project and move your class's unit into that project along with any other units it depends on. Compile the project. If the compiler warns about "implicitly including" any other units, add those to the package, too.
Now, remove any of the package units from the EXE project. There should be no units that are members of both projects. Next, turn on the "build with run-time packages" checkbox in your EXE's project options. Add your package to the semicolon-separated list of package names. The RTL and VCL packages will probably also be on that list.
Compile both projects, and you're done.
If you make changes to your class implementation, you can recompile the package only and send a new version to customers. The program will automatically get the new changes when you replace the original file with the new one. The package is listed in the program's import table, so the OS will automatically load the BPL file when it loads the EXE. The EXE doesn't need to run any special code to load the package.
Delphi can create DLL to export functions or BPL to export component.
You can create component, compile it (use the same compiler settings as in your main app), and Delphi will create .bpl. Then import this component to Delphi and compile your app with this compomponent as a package.
My experience with components created with Delphi 4 proved that one, big application is more reliable than application with separate .bpls. It was multithreaded server and it worked fine if compiled standalone, while crashed after short time if compiled with packages. I hope newer versions of Delphi improved in this area.
Be aware of memory management (in app do not free memeory allocated in package and vice versa) and compiler settings.
If you like about.com then this link will be useful: Introduction to Packages; BPLs are special DLLs!
BPLs have their usage. For example if you have to make a very huge application like an Erp, you need to try to use BPLs seriously.
In the other hand, BPLs aren't responsible of crashing applications. Bad usage of BPLs does it.
you can try the MAF Components, they handle plugins and much more for you without extra code. Comes with tutorials and a demo application with source.
http://www.maf-components.com
I read this article and find the concept of Virtual Library Interfaces nice for runtime loading of DLLs. However it seems that they aren't available for Win32. Is this true? And if so: Why? I don't see what would tie the idea to .NET.
EDIT: I'm mostly rephrasing here what Rob already wrote. :-)
I'm not after plugins or similar - it's about plain old Win32 DLLs. What appeals to me is the idea to let the compiler deal with all the details of loading a DLL at runtime - no need to call GetProcAddress for every function in the DLL etc.
It seems to me like the three answers so far have completely missed the point of your question. That, or I have. You're asking why Win32 Delphi doesn't have something like the magical Supports function that Hallvard's article talks about, aren't you? Namely, a function that, given the name of a DLL and the type information of an interface, returns an object that implements that interface using the standalone functions exported from the DLL.
Hydra seems to be all about calling .Net code from a Win32 program, not about importing functions from a DLL. TJvPluginManager requires that the plug-in DLLs export a special self-registration function that the manager will call when it loads the DLL, and the function must return an instance of the TJvPlugin class, so the plug-in DLL must be written in Delphi or C++ Builder. The Supports function, on the other hand, works with any DLL written in any language. You could use it on kernel32, if you wanted.
I don't know why Win32 Delphi doesn't have such a thing. Maybe CodeGear didn't see much demand for it since Delphi and Turbo Pascal had already gone for so long without it.
It's certainly possible to write a function that works like that, and I don't expect it would be any harder to write than the .Net version must have been, unless Microsoft's .Net libraries already provide most of the pieces and Delphi just wraps them up into a convenient-to-call function that looks like the several other overloaded versions of Supports that Delphi has had for years.
There would be a few steps to implementing that function in Win32. (I'm providing only a sketch of what's necessary because I don't have a running copy of Delphi handy right now. Ask nicely, and maybe I'll find more details.) First, you'd need to make sure that type information for an interface held, at a minimum, the undecorated names of its methods. Then, Supports would need to generate a function stub for each method in the interface (besides _AddRef, _Release, and QueryInterface). The stub would go something like this, assuming the calling convention is stdcall:
asm
// Pop the return address,
// discard the "this" pointer,
// and restore the return address
pop eax
pop ecx
push eax
jmp AddressOfFunction
end;
As Supports generated each stub, it would fill in the actual function address, gotten from calling GetProcAddress with the name of the corresponding interface method. The stdcall calling convention is easy to wrap like that; cdecl is a little cumbersome; register is a pain in the neck.
Once it has all the stubs generated, it would need to generate an "object" that looks like it implements the given interface. It doesn't have to be an actual class. At compile time, Supports doesn't know the layout of the interface it's going to be asked to implement, so having a class wouldn't accomplish much.
The final step is to provide implementations of the _AddRef, _Release, and QueryInterface. _AddRef would be unremarkable; _Release is where you'd call FreeLibrary when the reference count reached zero; QueryInterface wouldn't do much at all, except claim that it supports IUnknown and the interface given to Supports.
Delphi used to come with a sample program that demonstrated implementing an interface without any classes at all. It was all done with records and function pointers (which is all an interface ultimately is, after all). Delphi also came with the corresponding code to do it with classes, in part to show how much easier Delphi can make things. I can't find the name of the demo program now, but I'm sure it's still around somewhere.
There are a number of Win32 options for this type of functionality. Project JEDI has an open source plugin system as part of the JVCL that loads either DLLs or packages, and can include forms and whatnot as additional functionality.
There are also a number of commercial products available, including the TMS Plugin Framework and RemObjects Hydra.
This is nothing new or special. The article's just talking about plugins. Native code's been able to do plugins for years. The only special thing about P/Invoke is that it allows native code and .NET to talk to each other in a plugin system, and the little trick where "the DLL can be seen as a singleton object that implements the interface [so that] you can use the Supports function from the Borland.Delphi.Win32 unit to check if the DLL and all the methods are available."
If you want to do what the article's talking about in Delphi for Win32, look at the LoadLibrary, GetProcAddress and FreeLibrary Windows API functions. If you absolutely must have an interface like the article describes, you have to write it yourself, either in the DLL (if you wrote the DLL yourself) by writing an exported function that returns an interface, or in the calling app, by writing a function that uses GetProcAddress to create an interface dynamically. (Caution: this requires mucking around with pointers, and is usually more trouble than it's worth.)
Your best bet is probably just to do what Tim Sullivan mentioned: use TJvPluginManager from the JEDI VCL if you only need native code, or Hydra if you have to talk to .NET assemblies.
I've used Hydra myself for a Delphi only solution (i.e., didn't interface to .NET) and it works great for that too. It's easier to use and adds some niceties, but I think that it's basically implemented the same way as the "roll-your-own" plugin framework that is well-described in this article:
http://www.saxon.co.uk/SinglePkg/
I would look for a plugin framework that's interface-based (like Hydra and the "roll-your-own" system in above paragraph), rather than one that simply sends messages between apps.
There is a Delphi plugin framework on sourceforge, don't know whether it's the same one as in JEDI project or not: http://sourceforge.net/projects/rd-dpf
There are also a couple of other commercial solutions, one of which is Dragonsoft's:
http://www.dragonsoft.us/products_dsps.php
What is wrong with doing this with simple com objects? Declare a simple interface that all of your plugins implement, and require that each com object include an exported function that returns its class guid. Then using the "plugins" is as simple as walking thru the plugins directory looking for DLL's which expose the special registration function, invoking it and then using the class guid to then invoke the com object.
I used something like this in a WIN32 commercial application with great success. The advantage was I could switch plugins in and out at will (provided of course the application wasn't running to remove existing ones), the magic was all in the interface that each one implemented.