I was toying around with the idea of using dynamically loading BPL's and passing object instances from the main app to a method in a BPL. This poses a problem units between used by the application and by the BPL.
I wrote a small little prototype which did this and was curious how Delphi internally manages differences between classes defined in the app vs. the BPL.
For example, say a basic Widget class like:
TmyWidget = class
private
fId:Integer;
fDescription:String;
public
procedure DoSomething1();
end;
Now the app and the BPL are built using the unit containing TmyWidget class. Later, something changes in TMyWidget and the app is rebuilt, but the BPL is not (or vice-versa.) I added another method DoSomething2() and created an instance of TmyWidget in the app and passed it to the BPL for processing and in the basic example, it worked. But it's obviously fraught with potential problems.
If another dynamically loaded BPL also uses TmyWidget then things get even more interesting. It seems to work, but it definitely doesn't feel ideal.
The main question is - how does one typically pass objects to and from the main application and DLLs or BPLs? I've never attempted it before and likely for a good reason, but I've got this idea that lends itself to this approach...
I'd imagine that the best approach is to serialize the object and pass those bytes over and deserialize it in the DLL/BPL with this process being mindful of potential version differences between the host and the dynamically loaded module but I was hoping the new SimpleSharedMem option might bring this new functionality without the overhead of serialization, but it seems to be not very useful unless you are strict in keeping the app and dll rebuilt on any shared code changes...but in this prototype, the app would stay fairly constant and the dynamically loaded modules would be changing frequently with functionality being added to TmyWidget. (The server app serves as the factory for building TmyWidget's based on client requests and the app would pass instances to the various modules for processing.)
...was curious how Delphi internally manages differences between classes defined in the app vs. the BPL
Delphi manages this by not allowing it. You can't have a unit with the same name in multiple packages at the same time: if you do, you get an error message saying something similar to Package XYZ already contains ABC (haven't seen that in a while...). Since the type name includes the unit name, you can't have the same type in two different packages. Unless it's a Interface defined by it's GUID, but that's a different story.
... how does one typically pass objects to and from the main application and DLLs or BPLs?
You don't pass objects to DLL, that's not a good idea. When you need to pass objects to a BPL, make sure the base class for that BPL is defined into an 3rd BPL.
Example. Polymorphic behavior for your TmyWidget is probably defined using some virtual methods. Make sure you have a TmyWidgetBase class that defines all of those virtual methods, derive all your TmyWidget's from that base class and pass around objects with the type TmyWidgetBase. Make sure the TmyWidgetBase class is in it's own Package.
When I attempted doing this I ended up with an tiny "bootstrap" exe and lot's of BPL's. Essentially all the logic was in BPL's, to facilitate passing objects around.
One of the projects I've worked on has successfully used a large number of runtime packages for more than a decade now so I'll share a few of my experiences dealing with packages.
As Cosmin pointed out different packages cannot contain the same units. If you use implicit linking, by adding a package to the requires clause of another package or by adding a package to the the Runtime packages list in Project Options the compiler will do the work for you and report one of the following error messages:
E2199: Packages '%s' and '%s' both contain unit '%s' (if your compiling project that depends on two packages containing the same unit)
E2200: Package '%s' already contains unit '%s' (if your compiling a package that contains a unit this is already contained in one of the packages it depends on)
If you are using explicit linking, using LoadPackage, a check will normally be attempted a runtime (though it can be circumvented) and raise an:
EPackageError: Cannot load package
'%s.' It contains unit '%s', which is
also contained in package '%s'
Resolving these errors isn't really all that difficult.
If you have two packages that both need to use a unit just let one of them contain the unit and the other one require the first.
If you have two packages that need to use each other's contained unit you'll have to move those units to an new package that both can depend on.
Implicitly linked packages have the advantage that you can directly access class definitions as though they were statically linked. Just add a unit to the uses clause of the unit you need to use it in. The compiler and the runtime environment take care of resolving everything.
Explicitly linked packages will need to rely on class registration in the initialization section.
Related
I would like to write a component with the ability for a developer to either require a DLL or include a unit in their app which compiles the library with the project, without the DLL.
For example, let's say it's called "MyLibrary". If I include MyLibrary in the uses clause, it will by default require that a DLL be distributed with the application. On the other hand, if another unit MyLibraryImplementation is in the uses clause, it will compile everything inside the app without requiring the DLL.
This component is inside a package installed into the IDE.
I'm not even sure the terminology for this, or anything about how to go about this. I'm very familiar with writing DLL's, and not looking for someone to write any full code.
What are the fundamental things I need to know to make this optional DLL possible?
You probably won't find the result very satisfying. Delphi cannot directly load classes from a DLL. Instead, you'll need to define some class in your MyLibrary unit. You'll also have to implement most of the methods there. You're welcome to make those methods be little more than stubs that delegate all the work to functions in the DLL, though. In the stub constructor, call a DLL function that allocates a data structure and returns a handle. In all three other stub methods, pass that handle to the corresponding DLL functions along with the methods' other arguments.
You'll essentially have three parallel implementations of the class:
The one in MyLibrary.pas that's nothing but stubs.
The one in MyLibrary.dll that backs the stubs.
The one in MyLibraryImplementation.pas that does all the same things as the DLL, but internally.
You can probably avoid duplicating too much, but not completely.
You don't need to do any of this, though. For years, Delphi has already offered comparable functionality built in: packages.
If your customers want to have your class's implementation live in a separate module, they can choose the "build with runtime packages" option in their projects' linker settings. Disabling that setting will cause your code to be compiled in to their EXE files instead, and they won't need to distribute the BPL file anymore.
So, I have a myself written run-time package. If a package is statically linked, the project that uses has full access to exported data because the compiler has full knowledge of what is imported from it, am I right? But it's also possible to load a package dynamically via LoadPackage(). But, how to work with imported complex data structures like classes then? I couldn't find a feasible way other than constructing complex expressions like using FindClass('TSomeClass') and invoking RTTI to operate on an instance of the imported class.
The compiler has full knowledge of what's in the package because the DCU and DCP files tell it what's there.
The IDE knows what's in the package because it knows how to find the Register procedure in all the units, and that procedure tells the IDE about the available classes.
In most cases, a program knows what's in a package because the program used units from that package, and the compiler assured that mentioning names of things in those units would resolve to corresponding things in the BPL file at run time. This includes mentioning the BPL file in the program's import table, so the OS loads the BPL automatically.
If the list of BPLs you wish to load can only be determined at run time, then you cannot use any units from those packages. You have to load the package dynamically.
There's still the matter of how to use what's in those packages. You could try to discover the entire contents with RTTI. That's no picnic, though. Instead, define an intermediary package that all involved modules will use.
Define an interface or a common base class for all your packages' classes to have. Put the definition of that class in a unit that's in its own package, which we'll call Shared.bpl. Include that package in the "requires" list of all your other packages and your EXE. Now, everything can refer to the shared unit and the common base class.
This is exactly what Delphi itself does. The shared packages are called RTL and VCL. There are several common base classes already defined there, including TComponent. In your case, it sounds like you need some common definitions beyond what TComponent has.
Short answer:
What you need is the compiler/linker to set you up and use DCPs to do all the linking to types and such.
Then loading of BPLs should be delayed/done by you in custom code.
Unfortunately Delphi won't allow this, probably because of political reasons, you could try and hack it though, see below for more thorough answer.
Long answer:
Apperently DCP describe to the compiler/linker/Delphi yadayadayada what's in those packages/DLLs/BPLs type-wise.
These DCPs can be considered the interface to these DLLs/BPLs, these DCPs are probably somehow compiled into the executable.
Then the BPL probably contains the "implementation".
Now here is where the problem begins. These BPLs are "auto-loaded" as somebody else already mentioned by mentioning "import table".
What you could try is "nuking" / altering the import table so that these BPLs are not loaded automatically anymore.
Then you could try loading these BPLs manually.
I am not sure if it's just as simple as doing a "load package" operation.
Perhaps the loading involves more like acquiring pointers to routines/methods/classes, not sure how that works.
However perhaps that code can be re-used, so all you need to do is "hack" into the import table and hack into the "load bpl" and disable it.
Then you should be able to replace that with your own custom loading code and perhaps finally "re-patch" into other importing routines... like routines calling getprocaddress and such if so required, not sure about this last part.
Anyway this is very sloppy from Delphi developers that there is no functionality to do this little "import table"/"load step" yourself.
This is politics at play though, there is no technical reason why this could not be delayed, done custom by you and then call the rest if necessary.
For some reason they do not want you loading these BPLs manually. They probably want you to keep using the IDE this way.
If you were able to load these things manually, maybe the IDE would then no longer be necessary.
Currently the IDE is necessary to specify where to load this stuff from, without the IDE it will get difficult. Though there are probably also some compiler options somewhere to specify the same, though few would use that.
So to me it seems some very weird "tie-in" into this product, which I honestly must admit is pretty fucking retarded.
They did the same with winsock for example which has two different versions, very easy to load this yourself manually so you can choose which version to use instead of always v1 or v2. For easy for them to delay these getprocs and such, yet they do not...
If they did, Delphi would be much more backwards compatible with windows 95/98, currently Delphi exes no longer works for those older operating systems, though there is no real technical reason why it could not work, it basically has to do with which DLLs are loaded, just a few lines of code could very easily make it work.
This is probably again politics as play to make Delphi only support the latest Windows versions a deal struck between Delphi creators and Microsoft.
Same thing with Windows 10, it apperently detects older processors and then refuses to run on them, removing/hacking these few lines of code/instructions will make Windows 10 work on older processors. Surprise ! =D
I am writing a localization application in which i am reading the DFM information from the application resource through EnumResourceNames API call.
However, the function returns me a name of the form for which the DFM is associated. I tried getting the class from the FindClass, but since this whole operation is coded in a package, the FindClass fails. RegisterClass routine is called from the exe's intialization section.
FindClass works fine when called from within the code written in the exe project. So, i have developed my own registration framework wherein i add all the Form classes, but this is real pain as i need to add the unit of the form and then pass the form class to the RegisterClass routine.
I was hoping if anyone can provide a simple solution of getting all the classes that are in the executable from which the instance of the object can be created by searching the classname.
BTW I am using Delphi 6 Update 2.
Thanks
Rahul W
If the application is calling RegisterClass() and the package is calling FindClass() (or vice versa), that will only work if both the package and the application are compiled with Runtime Packages enabled so they share a single instance of the RTL (which means you have to deploy the RTL and VCL packages alongside your application and package). Otherwise, your application and package will have their own local copies of the RTL instead. In order to share classes in that situation, one project will have to export extra functions that the other project can call when needed to register its local classes in the other project's local class list.
As for detecting the available classes dynamically, that is not possible in D6. The RTTI system did not gain enough detailed information to perform that kind of enumeration until D2010.
Came by a curious case today, that got me thinking about how the object model in delphi really works.
The case:
We have imported a SOAP service that expose a couple of methods, taking objects as parameters. Delphi generates classes/interfaces that we use to communicate with the soap service, and the objects used as parameters all inherit from TRemotable.
For different reasons, we have put all the communication with the soap service into a dll.
We then tried to instantiate the objects that should be sent in the main executable, and pass it over to the library for serialization and sending.
Now, that did not work, but gave an exception that I did not expect.
It said that the object we where trying to send to the soap service, must inherit from TRemotable, but it does. By inspecting the object, we can see that the class is the imported class from the wsdl, and that the parent class is indeed TRemotable.
Building with packages solves this issue.
The question:
Is it so that a class defined in a source file, shared between two libraries, ends up as different classes at runtime? If so, why is that?
As far as I know, it should be ok to pass objects between libraries. How, then, is strong typing ensured, and to what extent will object instances be compatible to each other?
Yes, the same class in different DLLs are different. The classes in each DLL will be loaded at runtime and point to different memory, so A.ClassType = B.ClassType will fail, even for the same source files. You can still pass the objects around and they'll work correctly, except in cases like this where it's using "is" or "as" to compare the classes themselves. Strong typing is only ensured in that the compiler assumes the classes match when you compile the DLL and main application. There's no protection against loading a DLL with one version of an object and a newer application trying to use an modified object declaration. If you want that you'll need to use packages.
You can try passing the parameter as a TObject, and on either end, cast as your TRemotable. I haven't tried this particular case, but I know I've passed TObject into a DLL like that. I also have a similar DLL for SOAP, and in my case this wouldn't work as my SOAP DLL is compiled with D2007 SOAP libraries and for complicated/legacy reasons, the rest of the app is D2005.
I have a Delphi application that I have written a fairly simple wrapper .exe for.
Basically, there was a dll that had a bunch of functions, one of which I would call iteratively once my wrapper did what it needed to. I am not in control of this dll file, and will never be.
Well, now this DLL is a BPL, and I'm not sure how to call functions within that file. Thanks in advance.
The easy way to use functions from a package is to "use" the unit that contains the function, call it as usual, and put the package on the list of your project's runtime packages. For that to work, there are a few requirements:
Your project must use the same Delphi version as was used to compile the package.
You must have access to the DCU file for the unit, or at least the DCP file for the package.
The package must exist in the operating system's search path when your program starts.
If you can't satisfy the third requirement, or if you don't want to have the package loaded all the time, then you can call LoadPackage for it instead. The way to make that work is to have another package that is loaded all the time. It will be used by both your project and the package you wish to load. The intermediate package will expose an interface (such as some registration functions, a variable, or a class) that the main package can use to tell the application what its functions are. You won't be able to "use" the main package's unit in your application directly.
If you can't satisfy the first two requirements, then there is the much harder way, which is also what you'd need to do if your application isn't written in Delphi or C++ Builder. Treat the package like an ordinary DLL. Load it with LoadLibrary. Use GetProcAddress to load its Initialize function, and then call it. (Remember that the calling convention is register, not stdcall.) Then load the address of the function you wish to call, keeping in mind that the name of the function has been mangled to include some unit and type information. Call the Finalize function before you call FreeLibrary. Check the source for LoadPackage and UnloadPackage; whether you need to call CheckForDuplicateUnits probably depends on whether you can satisfy requirement number 1.
A BPL is just a DLL with a few specific additions to it. You should have no trouble calling functions from it just like you did with the DLL, with one specific caveat: The BPL has to be built in the same version of Delphi as you're using. This can be a major drawback if you don't have the source code. If this is a problem for you, you should probably talk with whoever created it and ask them to make it back into a DLL.
A BPL can eliminate a lot of DLL problems. If you can statically link it, the border becomes all but transparent. If you have to load it dynamically, you need one DLL-style access function (usually one that returns an object or an interface) and some common type (interface) definitions. All that should be supplied by the maker of the BPL.