I am looking to add the capability for users to write plugins to the program I have developed in Delphi. The program is a single executable with no DLLs used.
This would allow the user community to write extensions to my program to access the internal data and add capabilities that they may find useful.
I've seen the post at: Suggestions for Adding Plugin Capability? but its answers don't seem transferrable to a Delphi program.
I would like, if possible, to add this capability and keep my application as a single executable without any DLLs or additional modules required.
Do you know of any resources, components or articles that would suggest how to best do this in Delphi, or do you have your own recommendation?
The first question I would ask is, do you need the plugins to access the UI of your host application, or add any UI elements of their own? Or will the plugins be limited to querying and/or supplying data to your host app?
The latter is much easier and opens up two possibilities. Others have already mentioned DLLs, which is the first way to go. Certain caveats apply - in general you should be interfacing with a dll using only the data types that are used in Windows API. THat way you can be sure that the plugin DLLs will understand your data types, no matter what language/compiler they were created in. (So for example, use PChars, not strings. Do not as a rule pass Delphi classes such as TStream to a DLL. This will appear to work in some cases, but is unsafe in general, because even if the DLL was compiled in Delphi, it may have been a different version of the compiler, with a slightly different idea of what TStream is). Google for using DLLs in Delphi, and you'll find plenty more tips.
Another way that hasn't been mentioned yet is to enable scripting in your application itself. There are several very capable 3rd-party scripting engines, both commercial and free, and most of them allow you to exchange Delphi objects with the script. Some of these libraries support only Pascal as the scripting language, others will let you use Basic (perhaps better for beginner users) or other languages. See for example RemObjects Pascal Script (free) at http://www.remobjects.com/free.aspx .
My favorite scripting solution at the moment is Python for Delphi (P4D, also free) from http://mmm-experts.com/Products.aspx?ProductID=3 It can interface beautifully with your classes via RTTI, and allows you to execute Python code in your Delphi app, as well as use Delphi classes in Python scripts. Given the popularity of Python, this may be a viable solution if you want to attract developers to your project. However, every user will need to have a Python distribution installed.
It seems to me that the barrier to entry, from the point of view of potential plugin writers, is lower if you use scripting than if you choose DLLs.
Now, back to my initial question: things get much more complicated if you need the plugins to interact with your user interface, e.g. by placing controls on it. In general, DLLs cannot be used to do this. The Borland/CodeGear-sanctioned way is to use packages (BPLs). With BPLs, you can access and instantiate classes offered by a plugin as if they were declared in your host application. The catch is, all BPLs must be compiled with the same exact version and build of Delphi that your main application is. In my opinion this makes packages completely impractical, since it's hard to expect that all the potential plugin writers around the world will be using the same version of Delphi you are. A major pitfall.
To get around this, I have experimented with a different approach: keep using DLLs, and develop a syntax for the plugin to describe the UI it needs, then create the UI yourself in the host app. (XML is a convenient way to express the UI, since you get the concept of parenting / nesting for free.) The UI description syntax can include callbacks to the DLL triggered when the contents or state of the control changes. This method will restrict plugins to the set of VCL controls your application already uses or has registered, though. And it's not a one-nighter job, while BPLs certainly are.
Google for "Delphi plugin framework", too. There are some ready-made solutions, but as far as I know they usually use BPLs, with their limited usefulness.
Actually, the accepted answer to the question you cite is quite appropriate for Delphi as well. Your plug-ins will be DLLs, and you can dictate that they should export a function with a certain name and signature. Then, your program will load the DLL (with LoadLibrary) and get the address of the function (with GetProcAddress). If the DLL doesn't load, or the function isn't there, then the DLL is not a plug-in for your application.
After you have the address for the DLL, you can call it. You can pass the function an interface to something that represents the parts of your application that you wish to expose. You can also require that the function return an interface with methods that your application will call at various times.
When testing your plug-in system, it will be wise to write a plug-in yourself with a language other than Delphi. That way, you can be more confident that you haven't inadvertently required everyone to use Delphi.
At first I went for BPL and DLL base plugins. And found them hard to mantain.
If you use BPL system, then you need to match BPL version with EXE version. This includes Delphi updates which can break something. I found out (the hard way) that if I need to include all my plugins with every release, there is no point in having plugins at all.
Then I switched to plain DLL plugins. But that system just complicated code base. And that's not a good thing.
While crusing the net I found out Lua embedded script language, and delivered with it. Lua is 150K DLL, embedding bytecode compiler, interpreter and very simple and smart dynamic programing language.
My plugins are simple lua scripts. Easily mantaind and made. There are premade Delphi examples, so you can export any class or procedure as Lua table or function. GUI or not. For example I had TurboPower Abbrevia in my app for zipping. I exported my zipping class to lua, and all plugins now can call zip('.', 'dir.zip') and unzip(). Then I switched to 7zip and only implemented old class to use 7zip. All plugins work as they did, with support for new zip('.', 'dir.7z').
I made TLuaAction which calls Execute(), Update(), Hint() procedure from its script.
Lua allso have it's own plugin system that makes it easy to add funcionality to it. Eg luacom make is easy to use COM automation, luainterface allows calling .net from lua.
See luaforge for more. There is Lua IDE made in Delphi, with source.
I tried to make an overview of all such options some time ago. Together with my readers/commenters we built this list:
DLL/BPL loaded from the program.
DLL/BPL loaded from a sandbox (which could be another copy of the program or a specialized "server" application and which communicates with the main program via messages/sockets/msmq/named pipes/mailslots/memory mapped files).
COM (in any of its variants).
DDE (please not).
External program that communicates via stanard input/output.
External program that communicates via files (file name is a parameter to the program).
External program that works via drop folder (useful for batch processing).
External program that communicates with windows messages/windows sockets/msmq/named pipes/mailslots/memory mapped files/database publish-subscribe.
The most universal method of adding plug-in capability is to use COM. A good book to get you started on the road is Delphi Com Programming by Eric Harmon. While it was originally written for Delphi versions 3 through 5, the books contents are still valid with the latest versions of Delphi.
Personally, I have used this technique along with active scripting to allow end user customization.
I'm using plug-ins to implement most of the functionality in the game engine I'm building. The main EXE is made up of a script engine, a plug-in manager, some basic graphics routines and not much else. I'm using TJvPluginManager from the JEDI VCL library. It's a very good manager, and it can add just about anything you want to your program. Check out the demos included with the package to see how it works. The only downside is that it adds a lot of JCL/JVCL code to your program, but that's not really an issue if you're already using other JVCL components.
If plugins will be developed in Delphi or C++ builder, use packages + interfaces. Delphi OTA is a good example for that.
If plugins will be language independent, COM is a good way to go.
Addition: If you won't use COM, u may need to provide SDKs for each language. And datatype handling between different languages can be pain(for example delphi string type). Delphi COM support is excellent, u don't need to worry about kind of details. Those are mostly impilicit with Delphi COM support. Don't try to invent the wheel again. I'm surprised why people doesn't tend to mention about it.
I found an article by Tim Sullivan:
Implementing Plug-Ins for Your Delphi Applications
You could have a look at Hydra from Remobjects. Not only will this allow you to add plugins, but also to mix win32 and .net.
Related
What is the best way to implement an external module system for a DELPHI application?
What I need is very basic really:
The main APP detects if a module is present and loads it(Run time)
Modules can store form
Modules can store DataModules
Modules can Store code
I need the use the store forms inside other forms, and only as a standAlone
I use something like this
if Assigned(pNewClass) then begin
Application.CreateForm(pNewClass, _lFrm);
_lFrm.Hide;
_lFrm.BorderStyle := bsNone;
_lFrm.Parent := pBasePNL //(TPanel);
_lFrm.Align := alClient;
end;
So I create a TForm, but place it inside a TPanel.
As for DataModules I usally store ImageLists so the ideia is to change the app ICOs just bit changing the external module.
So what is the best way to achieve this?
Looked at runtime BPLs but don’t seem to understand how to do it.
Thanks.
UPDATE : .....................................
After reading some related question and answers i think I found my answer and solution.
http://edn.embarcadero.com/article/27178
The article is old stuff but amazingly simple.
Well the logic is there I just don’t seem to get it to Show the forms
I am just testing example 2
It loads the BPL, but doesn’t get the Form:
AClass := GetClass('TForm2');
Always retrievex ‘nil’
But the BPL as it registered:
RegisterClass(TForm2);
Can anyone help with this one.
Packages are an easy solution but they have one huge drawback. Using packages forces plugin authors to use not only Delphi, but the same version of the compiler as you do.
I personally would prefer to expose the functionality of the app through a number of interfaces. This allows accessibility from languages other than Delphi.
Typically the plugin would be implemented in a DLL and would export a function that the app would call to pass in the root interface representing the app. The plugin would then call methods of that interface thus establishing two-way interaction.
I made you a demo, it's so easy to get started! However... Started is not finished.
Every time I started using plugins, I later regretted it. However, as you say, you want a binary plugin system. So BPLs are the correct solution. David suggests using interfaces (with plain DLLs instead of the full runtime package BPL) and this would solve some of the commonly encountered BPL-instability problems due to not versioning your classes, and thus your Application-and-package-binary-compatibility dependencies, properly. If you do not need to share memory and do not need to use borlandmm.dll (shared memory management) then a straight DLL with interfaces will work fine.
If you can do everything you need to do with just scripting, and make your plugin system with just scripts, then do it that way. If you can get away with DLLs, interfaces, and no memory sharing, then use plain DLLs. If you must share Memory and class types, then yes, use BPLs.
Beware that using BPLs (Runtime Packages) comes with a lot of side effects you might not have been expecting. The largest BPL-based applications I have worked on have been more of a mess, and less stable, than any monolithic apps I have worked on. It seems that it is only possible to pine for packages until you try them, and then I find, I pine for monoliths again.
If you use BPL packages correctly, and you version your plugins properly, everything is fine. BPLs are great. But in the real world, it seems that sane versioning and ABI compatibility and interoperability, and stability are at a premium.
Update: I made you a demo it's here (plugin_r2.zip). It was done in Delphi XE, but if you are using an older version of delphi, you just delete the .dproj files and open the .dpr file for the main app, and the .dpk for the package.
I am working in a delphi IDE expert , now to avoid dependencies problems, i was thinking in rebuild this expert as a dll expert as was suggested in one of these answers, now my expert (compiled as bpl) access the Screen and Application global variables (instances of the Delphi IDE), so i was wondering if i compile my expert as a dll i can still accesing these variables and also i want to know which are the main differences between a bpl delphi expert and a dll delphi expert?
Should I compile my wizard as a DLL or a Package? Packages are easier to
load and unload without restarting the
IDE (and hence easier to debug), but
they can create unit naming conflicts
in the IDE. Conflicts happen when the
name a wizard's unit matches the name
of a unit in another loaded
design-time package. In this case,
both packages can not be loaded at the
same time. The recommended workaround
is to prefix all of your unit names
with a "unique" prefix. GExperts, for
example, uses "GX_" as the name prefix
for its units.
From this very good source about OTA: GExperts
When you access a global variable those would be global variables that are global to your DLL, not global to the main BDS.exe. I am not sure but I think your DLL would have its own Screen and Application global variable, if you linked in Forms, and the core of the VCL.
Those things which belong to the IDE itself are accessed through the Open Tools Api (OTA). I believe that you do not normally share any objects between the IDE your expert anyways, and if you were to try to do so, it would be problematic. Anything at all that you do that bypasses the OTA is going to be vulnerable to breaking in strange ways, especially in future versions of the IDE.
Dependency problems are of course a big reason to not use BPL based packages, but I think an even bigger reason is to maintain a complete separation between your tool's internals, and the internals of the IDE.
Remember that a DLL target, like an executable target, is statically linked. That is the core of the difference. If your expert provides functionality that uses the legal public documented OTA interfaces only, then moving to a DLL should be problem free. If you use some back door hacks that are possible with BPLs, then I can't advise you further.
Is it good idea to put Forms that have complete functionality in dll.
And main app will invoke dll function that returns form object.
The accepted way to do this in Delphi is to use packages rather than DLLs.
Packages are essentially DLLs but with Delphi specific capabilities that allow VCL objects to be used across package boundaries.
Trying to do this with DLLs will lead to a variety of problems that packages deal with. One downside of packages is that all modules must be compiled with the same version of Delphi. But if you are wanting to share objects across module boundaries then you would face the same restriction if you used DLLs.
The Delphi documentation has extensive coverage of packages.
Having said all that, I would add that if you can put all your code into a single module (.exe or .dll) then it does make life a lot simpler.
Adding to the answers about using packages:
Packages can only be used if both, the main app and all dlls (plugins) are written in Delphi and are written using the same version of Delphi
DLLs can be written in any programming language that can create them and can be used by any program regardless of the programming language
So, using dlls rather than packages does make sense.
Regarding the actual question: Yes, it is possible to put forms into dlls and they usually work fine. Just make sure that you do not pass them around because they are only valid objects within the context of the dll. You will experience the odd problem with forms losing focus or coming up behind other forms. This can usually be fixed by passing a window handle from the main executable to the dll which is then used as the parent for the form.
Also note: TObject of your dll is different from TObject of your application. The same applies to other commonly used classes and variables like (Forms.)Application.
I have done it and it was a pain in the lower back but it was not impossible. The main program was written in Visual Basic 6, some modules were written in Delphi 6, others were written in Delphi 7 and Delphi 2007.
Conclusion: If you are sure you will never use something different than Delphi for your app and for your dlls (plugins) and are willing to always recompile everything when you switch Delphi versions, you should use packages. Otherwise it might be better to use regular dlls. (And are you sure you will always be the only person writing these dlls? Maybe at some time there will be a 3rd party developer for one of the dlls who does not own the Delphi version he needs.)
IMO this is sometimes a very good idea and the only way to go - for the reasons others have mentioned, I'm not a fan of packages, and am very comfortable with DLL's. I am currently adding functionality to an app written in Delphi 5 using Delphi XE - it was either use DLL's or write in D5 - of course I opted for the former: D5 app calls DLL's written in XE that contain all the latest and greatest features. (The first projects I did in Delphi were done via the old Borland Paradox - Paradox app invoked DLL's written in Delphi 1!)
But, I don't send the form or module from the DLL back to the main app - I just send the DLL module a structure containing what it needs to know to do its work, and when it's done and the DLL's form closes, it cleans up and then and returns a numerical code or structure back to caller indicating success, failure etc ( old fashioned but very effective).
Passing the form instance from the DLL back to your main app across the DLL threshhold can be problematic - note #dummzeuch's excellent answer above with some good tips on how to negotiate some of those problems should you decide that is your only solution.
+1 for everything that David Heffernan says.
Strategically, you really only need to implement forms (or other functionality) in external files if you're implementing a plug in system.
If you're going to allow plugins to be authored in any language, then DLL's are the only way to go.
If your plugin system will be restricted to developers with the same version of Delphi (same team perhaps?) then go with BPL's. The additional drawback of Delphi packages, from my perspective, is the need to deploy the VCL BPL's with your app, which are always more Mb than a single compiled module.
If on the other hand you want to write a modular system, you can still do that by implementing loose coupling & "plugin" techniques within your code and still compile to a single module.
If you put a form in a normal dll the form won't be able to intercept the TAB or arrow keys. I have been told that this is due to the OnKeyDown not be passed through.
I want to design Delphi plugin framework. There are three options:
1. DLL
2. BPL
3. COM interface
Every option has some disadvantage.
DLL - Promblem with MDI apllication, forms from plugin cannot be embeded to the host exe - mdi application.
BPL - Every *.bpl plugin and *.exe host application must be compiled with the same version of Delphi.
COM - Interfaces {xxx-xx-xxx-xx} must be registered in system, (regsvr) So plugin framework cannot be portable!
Is everything true what I wrote above? If not, please correct me, or is there some other possibility?
thanks
Yes, what you wrote is correct. All of these have advantages and disadvantages, the question is what is important for you, you didn't say what you want to do, so we can't tell you how to do it.
In general, I would pick BPL by default, use DLL if you need to use it from non Delphi apps, and pick COM only if you really have to.
The alternative is to not build your own, since there are several Delphi plugin frameworks available.
Also, this topic has been discussed here before, check out:
how-best-to-add-plugin-capability-to-a-delphi-program
plugins-system-for-delphi-application-bpl-vs-dll
Another possibility is having a scripting interface, such as with Python or Pascalscript. There are many advantages to this approach, not least of which is that the plugin source is also the executable, making it easy to debug and share.
I would look at Hydra from Remobjects.
Another candidate would be a combination of PaxCompiler (scripting) and EControl Form Designer (form designer).
We use DLLs, even when using only with a Delphi application.
Dll forms are embedded in the main form and use Tabs instead of MDI.
For DLL to work properly we also use a shared memory manager and built with runtime packages: vcl, rtl and our own package.
Have you taken a look at http://www.tmssoftware.com/site/tpf.asp
I'm writing delphi app which should have capability of loading plugins. I'm using JvPluginManager as plugin system/manager ;) Now, in the new plugin wizard they say it's better to use .bpl type plugins instead of .dll plugins ... What are the pros of this solution versus dll type plugins?
So far I've found only cons of this solution:
I have to put all the common interface units in separate package so that while loading plugins it won't throw any error about the other package containing common unit
if, let's say, one of plugin developers decides to uses some well-known unit (like synapse), which doesn't have runtime package by default, and the second plugin developer does the same, than bump... it's crash here ...
So, what are actually the pros of using bpls instead of dlls compiled with runtime packages?
Thanks in advance
Another disadvantage to BPL's. When you switch Delphi versions you will have to re-distribute new plugins. After many attempts at attempting to find the perfect plugin system, I ended up with COM and I have never regretted that decision. In a commercial application which has had the plugin requirement for over 8 years, the application has continued to move forward and yet some of the plugins that were released with the first iteration still exist in their ORIGINAL form.
If you choose this method, do yourself a favor and start with a simple interface and then add new interfaces upon it. You don't want to ever change your base interface, so keep it simple and sweet.
As Alexander said, a BPL is basically a DLL. But there are some conditions (from a not-so-short summary I made: http://wiki.freepascal.org/packages ):
A unit can only exist once in BPL's +Exe. This avoids duplication of state (twice the heapmanager and other global vars of system etc, VMT tables etc)
.BPL's can only USE other .BPLs.
This means that dynamic types like ansistring and IS/AS work properly over BPL interfaces.
Initialization/finalization are separate procedure and their initialization order is strictly controlled. For static dynamic loading this is simpler, for dynamic loading (plugin-like) all dependancies on units are checked .
Everything is essentially one big program, which means that the BPL's must be compiled with the same compiler version and RTL and depends on the versions other dependancies. It might be harder to make .BPL's to plugin to an existing EXE, since the Delphi version must match.
This also means that you must deliver .dcp's for (non Delphi) .BPLs the plugin .BPLs depend on
In short: if the plugin architecture is open, make it a DLL. Otherwise people have to have the exact same Delphi version to write plugins.
Hybrid is also possible. An higher level .BPL interface for functionality you factor out into .BPL yourself and selected devels, and a rock bottom procedure DLL interface for the rest.
A third option is using DLLs, but ordain Sharemem. Strings will work, multiple Delphi versions will work. Objects can work but are unsafe (e.g. I guess e.g. D2009 with an earlier version wouldn't work). Even other language users might be able to allocate over COM, not entirely excluding non Delphi.
Your first con is also a pro. If you replicate shared code in each dll the dlls get bigger and bigger. Even when using dlls you can prevent this by moving shared code in a separate dll.
Pros:
Types are shared. No TFont is not a TFont problem
Memory manager is shared. Strings and classes can be used as parameter between plugins without problems.
Cons:
Plugins can be built using Delphi or BCB only.
Plugins should use the same Delphi or BCB version.
Have you considerd using COM? COM makes it possible to share types, strings and classes and the plugins can be written in many programming languages.
I'm not familiar of JvPluginManager, but it depends on how you're going to use BPLs.
Basically, BPL - is just a usual DLL, but its initialization/finalization work is stripped from DllMain to separate functions: 'Initialize'/'Finalize'.
So, if you're going to use BPL like usual DLL, there are no cons that I'm aware of, only pros: there will be no more troubles with DllMain. That's all. The only difference.
But BPL in Delphi also provide a convient way to share code. This means great advantages (common memory manager, no duplicated code, etc, etc). So usual BPL does a lot more than "being just a DLL".
But this also means, that now your plugin system is limited to Delphi only (well, may be C++ Builder too). I.e. both plugins and exe MUST be compiled in the very same compiler to run smoothly.
If this is acceptable for you (i.e. no MS Visual Studio, no, sir, never) - then go ahead, you can use all power of BPLs.
P.S. But upgrading such BPLs plugins can be also a nightmare, if you do not design interface side carefully. In certain worst cases, you may need to recompile everything.
P.P.S. Like I said: I have no idea, how it is applied to plugins, created by JvPluginManager.
Avoid blp approach as you will have to ship a big bag of bpl with you software and thus, distribution will become bulky.
why do we use Delphi to compile small stand alone programs that just RUN anywhere without any runtime dependency. Using bpls means defeating this very purpose.
I don't know as to how comfortable you are with DLLs, but I would suggest you to use DLLs.
This will give other developers (who
may get interested in your software)
a chance to use any development
language (as long as that language
can spit out dll) to write their own
plugins that can be used in your
developed software.
Another thing is that you will be
saved from Delphi's vcl version
dependency tyranny. A major weak
point of Delphi till date.