Delphi and COM: TLB and maintenance issues - delphi

In the company that i work, we develop all the GUI in C#, but the application kernel is mainly developed in Delphi 5 (for historical reasons), with a lot of components made in COM+. Related to this very specific sort of application a I two questions:
Experienced guys in Delphi and/or COM, do you have any workrounds to work with the buggy TLB interface ?
Some of the bugs are: IDE crashing during edition of a large TLB, lost of methods IDs, TLB corruption, etc.
Here, we haven't found any good solution. Actually we tried do upgrade do the new 2007 version. But the new IDE TLB interface has the same bugs that we found before.
How do you control TLBs versions ? The TLB file is in a binary format and conflict resolutions are very hard to do. We tried to do it exporting the interfaces descriptions to IDL and commiting into CVS, but we didn't found any good way to generate TLBs from IDL using Delphi. Additionaly, the MIDL tool provided by Microsoft, didn't parse correctly the IDL files that we exported from delphi.

I think you should have a good look at Delphi 2009.
Delphi 2009 has changes to the COM support, including a text-based replacement for the binary TLB files.
You can read more on Chris Bensen's blog.

In the distant past (before I started working for CodeGear) I gave up on the odd Delphi-ized IDL language that the IDE presented, and wrote my own IDL and compiled it using MS midl. This largely worked; the only catch, IIRC, was making sure dispids (id attribute) were correct on automation interfaces (dispinterfaces) for property getters & setters - there was some invariant that tlibimp expected but midl didn't guarantee.
However, now that Delphi 2009 uses a safe subset of midl syntax, and includes a compiler for this midl in the box and integrated into the IDE, these problems should be a thing of the past.

We have also just installed Delphi 2009 and it does seem to have improved the support for Typelibraries. However I have worked with COM and type libraries for quite some time and here are my general gotchas that I have found over the years. I would agree its pretty buggy and is all the way up to Delphi 2006 (our version prior to using 2009).
Always have every file writeable before opening. This may sound obvious, but when working with source control sometimes we forget to do this and try to remove readonly flag after opening a file - Delphi cant deal with this. Ensure tlb is writable before opening.
If editing a standalone typelibrary you MUST have a project open. For some reason if you open a type library on its own it will not save. Create a blank project and then open your typelibrary. For some reason this allows the type library to be saved.
If your type library is used by an application or COM+ ensure that application is shut down or COM+ disabled before opening the type library. Any open apps will prevent the type library from being saved.
However I think your best solution is probably an upgrade. You get Unicode support too.

Using Delphi 2009 has greatly taken much of the pain out of huge TLB files, and conversion of our existing objects was painless, but our com objects don't use any third party libraries.
We will be migrating our gui applications over once the library vendors release supported versions.

Same experience with the TLB interface here: we simply stopped using it.
We work with several separate IDL files (hand-build) for different parts of our framework, making use of the #include construct to include them into the IDL of the actual application, then generate the single tlb using MIDL and tlibimp it. If the application has no IDL of it's own, pre-compiled version of the different framework TLB files are available.
Whenever the framework enters a new version, a script is run to re-generate the GUIDS on all necessary interfaces in the IDL files.
This has served us well for many years, and for us to move over the new Delphi 2009 IDL/TLB toolset will have to be not only integrated into the IDE, but also versatile when it comes to automated builds and whatnot. Can't wait to get my hands dirty with some experiments!

Related

Delphi forms in dlls

Is it good idea to put Forms that have complete functionality in dll.
And main app will invoke dll function that returns form object.
The accepted way to do this in Delphi is to use packages rather than DLLs.
Packages are essentially DLLs but with Delphi specific capabilities that allow VCL objects to be used across package boundaries.
Trying to do this with DLLs will lead to a variety of problems that packages deal with. One downside of packages is that all modules must be compiled with the same version of Delphi. But if you are wanting to share objects across module boundaries then you would face the same restriction if you used DLLs.
The Delphi documentation has extensive coverage of packages.
Having said all that, I would add that if you can put all your code into a single module (.exe or .dll) then it does make life a lot simpler.
Adding to the answers about using packages:
Packages can only be used if both, the main app and all dlls (plugins) are written in Delphi and are written using the same version of Delphi
DLLs can be written in any programming language that can create them and can be used by any program regardless of the programming language
So, using dlls rather than packages does make sense.
Regarding the actual question: Yes, it is possible to put forms into dlls and they usually work fine. Just make sure that you do not pass them around because they are only valid objects within the context of the dll. You will experience the odd problem with forms losing focus or coming up behind other forms. This can usually be fixed by passing a window handle from the main executable to the dll which is then used as the parent for the form.
Also note: TObject of your dll is different from TObject of your application. The same applies to other commonly used classes and variables like (Forms.)Application.
I have done it and it was a pain in the lower back but it was not impossible. The main program was written in Visual Basic 6, some modules were written in Delphi 6, others were written in Delphi 7 and Delphi 2007.
Conclusion: If you are sure you will never use something different than Delphi for your app and for your dlls (plugins) and are willing to always recompile everything when you switch Delphi versions, you should use packages. Otherwise it might be better to use regular dlls. (And are you sure you will always be the only person writing these dlls? Maybe at some time there will be a 3rd party developer for one of the dlls who does not own the Delphi version he needs.)
IMO this is sometimes a very good idea and the only way to go - for the reasons others have mentioned, I'm not a fan of packages, and am very comfortable with DLL's. I am currently adding functionality to an app written in Delphi 5 using Delphi XE - it was either use DLL's or write in D5 - of course I opted for the former: D5 app calls DLL's written in XE that contain all the latest and greatest features. (The first projects I did in Delphi were done via the old Borland Paradox - Paradox app invoked DLL's written in Delphi 1!)
But, I don't send the form or module from the DLL back to the main app - I just send the DLL module a structure containing what it needs to know to do its work, and when it's done and the DLL's form closes, it cleans up and then and returns a numerical code or structure back to caller indicating success, failure etc ( old fashioned but very effective).
Passing the form instance from the DLL back to your main app across the DLL threshhold can be problematic - note #dummzeuch's excellent answer above with some good tips on how to negotiate some of those problems should you decide that is your only solution.
+1 for everything that David Heffernan says.
Strategically, you really only need to implement forms (or other functionality) in external files if you're implementing a plug in system.
If you're going to allow plugins to be authored in any language, then DLL's are the only way to go.
If your plugin system will be restricted to developers with the same version of Delphi (same team perhaps?) then go with BPL's. The additional drawback of Delphi packages, from my perspective, is the need to deploy the VCL BPL's with your app, which are always more Mb than a single compiled module.
If on the other hand you want to write a modular system, you can still do that by implementing loose coupling & "plugin" techniques within your code and still compile to a single module.
If you put a form in a normal dll the form won't be able to intercept the TAB or arrow keys. I have been told that this is due to the OnKeyDown not be passed through.

Delphi RTTI Over Executable Project

in C# you can easily open an assembly (just another word from an EXE) and then get reflection information from that assembly. I've been looking for something similar for Delphi where I can write a Delphi program that can point to a Delphi EXE then get me reflection information. How can I archive this? Thanks
There are a handful of programs that can extract RTTI from Delphi programs and DLLs. It's not as rich as what's available for .NET, but it can be done to a limited degree. There's a lot more RTTI available in Delphi 2010 and later versions, but as far as I know there's no program out yet that can extract info from them.
I was at the conference where this system was formally announced, and a bunch of people asked how this would affect security. Barry Kelly, who created the new RTTI, said that there would not be enough metadata available to create a "Delphi Reflector".
Well, sad news is, you probably can't. You can easily view .NET assemblies because they a) are compiled to what is called intermediate language (thus you can even get the code back using the likes of Reflector etc.), and b) contain metadata. Delphi, on the other hand, compiles to native code that and produces regular PE files that do not include rich metadata like a .NET assembly.
Delphi just doesn't work that way.
There are tools that can assist in reversing (Delphi written) executables but it will never be easy and it requires good x86 assembly knowledge.
Some examples of such tools are:
IDA Pro and/or Hex Rays
DeDe
Interactive Delphi Reconstructor
PE Explorer
The resource sections of a Delphi Executable also reveals usefull info like the form and it's components.
See also Is there a program to decompile Delphi?
If you have Delphi2010+ you can load a bpl and then use the rich rtti over it lik. e you do with c# and a .net assembly. If you need to do this in a .exe AFAIK you can't.
TestComplete seems to identify most objects in a running Delphi executable. You can extract this information with a script. A demo can be downloaded from http://www.automatedqa.com/downloads/testcomplete
This is maybe not the type of application you want as it is big and made for GUI-testing, but yes it can identify the objects.
As mentioned Remko, IDR (Interactive Delphi Reconstructor) can extract all RTTI information (if program contains it!) for Delphi version from 2 to 2009. Version 2010 will be available later. Moreover IDR can use all information that it finds in program to create a lot of comments to disassemled program code (this is a first step of analyses). You can also look forms and easy go to event hadlers associated with controls.

How to keep forms and frames compilable in Delphi 6 - 2007?

We recently converted our long-running Delphi project to Open Source. Multiple people have contributed patches already, which is great, but unfortunately forms and frames when saved with Delphi 2006 (and later) contain new properties in the .dfm that are not handled by older versions. Forms are handled quite gracefully by the IDE ("ignore propery?"), but frames are loaded at runtime and result in errors. Not an option, as far as I'm concerned.
I now removed those properties from the .dmf files by hand, but I am really wondering if there is a more elegant solution. There is no way to save in a backwards compatible format from the new IDE's, is there? Are there existing tools to strip the nonsupported properties from the .dfm's ? Any other elegant solution I am missing?
Normally, with a commercial project, I'd just upgrade the project to the most recent version applicable... but as this is open source I really don't want to loose out on those developers still working on Delphi 7. That includes myself, actually.
The JEDI JVCL project uses a little utility dc.exe (delphi cleaner) and a list of properties in DFMs that are not present in older versions of delphi, mine contains the following text:
*.PopupMode
*.PopupParent
*.ExplicitLeft
*.ExplicitTop
*.ExplicitWidth
*.ExplicitHeight
*.BevelKind
*.BufferDocument
*.DoubleBuffered
*.ParentDoubleBuffered
*.DisableHyperlinks
*.AlwaysEnquoteFloats
*.PixelsPerInch
I run this dc.exe utility from a batch file which cleans my dfms before I commit
changes to subversion. The syntax in my batch file for this is:
dc.exe -s -fd10.txt *.dfm -i
You can grab my stuff at:
http://sites.google.com/site/warrenpostma/files
You can try Andy's DFMCheck. It can automatically open and close all forms and frames in a project, which makes the IDE remove unknown properties (as Marco wrote).
Well, for sake of completeness:
Open the dfms in the oldest Delphi supported, let it remove all unknown properties, change a property and save.
For your purposes, Warren's solution is better, but it can be a workaround. I did it for a while when we were dual D7-D2006ing.

Plugins system for Delphi application - bpl vs dll?

I'm writing delphi app which should have capability of loading plugins. I'm using JvPluginManager as plugin system/manager ;) Now, in the new plugin wizard they say it's better to use .bpl type plugins instead of .dll plugins ... What are the pros of this solution versus dll type plugins?
So far I've found only cons of this solution:
I have to put all the common interface units in separate package so that while loading plugins it won't throw any error about the other package containing common unit
if, let's say, one of plugin developers decides to uses some well-known unit (like synapse), which doesn't have runtime package by default, and the second plugin developer does the same, than bump... it's crash here ...
So, what are actually the pros of using bpls instead of dlls compiled with runtime packages?
Thanks in advance
Another disadvantage to BPL's. When you switch Delphi versions you will have to re-distribute new plugins. After many attempts at attempting to find the perfect plugin system, I ended up with COM and I have never regretted that decision. In a commercial application which has had the plugin requirement for over 8 years, the application has continued to move forward and yet some of the plugins that were released with the first iteration still exist in their ORIGINAL form.
If you choose this method, do yourself a favor and start with a simple interface and then add new interfaces upon it. You don't want to ever change your base interface, so keep it simple and sweet.
As Alexander said, a BPL is basically a DLL. But there are some conditions (from a not-so-short summary I made: http://wiki.freepascal.org/packages ):
A unit can only exist once in BPL's +Exe. This avoids duplication of state (twice the heapmanager and other global vars of system etc, VMT tables etc)
.BPL's can only USE other .BPLs.
This means that dynamic types like ansistring and IS/AS work properly over BPL interfaces.
Initialization/finalization are separate procedure and their initialization order is strictly controlled. For static dynamic loading this is simpler, for dynamic loading (plugin-like) all dependancies on units are checked .
Everything is essentially one big program, which means that the BPL's must be compiled with the same compiler version and RTL and depends on the versions other dependancies. It might be harder to make .BPL's to plugin to an existing EXE, since the Delphi version must match.
This also means that you must deliver .dcp's for (non Delphi) .BPLs the plugin .BPLs depend on
In short: if the plugin architecture is open, make it a DLL. Otherwise people have to have the exact same Delphi version to write plugins.
Hybrid is also possible. An higher level .BPL interface for functionality you factor out into .BPL yourself and selected devels, and a rock bottom procedure DLL interface for the rest.
A third option is using DLLs, but ordain Sharemem. Strings will work, multiple Delphi versions will work. Objects can work but are unsafe (e.g. I guess e.g. D2009 with an earlier version wouldn't work). Even other language users might be able to allocate over COM, not entirely excluding non Delphi.
Your first con is also a pro. If you replicate shared code in each dll the dlls get bigger and bigger. Even when using dlls you can prevent this by moving shared code in a separate dll.
Pros:
Types are shared. No TFont is not a TFont problem
Memory manager is shared. Strings and classes can be used as parameter between plugins without problems.
Cons:
Plugins can be built using Delphi or BCB only.
Plugins should use the same Delphi or BCB version.
Have you considerd using COM? COM makes it possible to share types, strings and classes and the plugins can be written in many programming languages.
I'm not familiar of JvPluginManager, but it depends on how you're going to use BPLs.
Basically, BPL - is just a usual DLL, but its initialization/finalization work is stripped from DllMain to separate functions: 'Initialize'/'Finalize'.
So, if you're going to use BPL like usual DLL, there are no cons that I'm aware of, only pros: there will be no more troubles with DllMain. That's all. The only difference.
But BPL in Delphi also provide a convient way to share code. This means great advantages (common memory manager, no duplicated code, etc, etc). So usual BPL does a lot more than "being just a DLL".
But this also means, that now your plugin system is limited to Delphi only (well, may be C++ Builder too). I.e. both plugins and exe MUST be compiled in the very same compiler to run smoothly.
If this is acceptable for you (i.e. no MS Visual Studio, no, sir, never) - then go ahead, you can use all power of BPLs.
P.S. But upgrading such BPLs plugins can be also a nightmare, if you do not design interface side carefully. In certain worst cases, you may need to recompile everything.
P.P.S. Like I said: I have no idea, how it is applied to plugins, created by JvPluginManager.
Avoid blp approach as you will have to ship a big bag of bpl with you software and thus, distribution will become bulky.
why do we use Delphi to compile small stand alone programs that just RUN anywhere without any runtime dependency. Using bpls means defeating this very purpose.
I don't know as to how comfortable you are with DLLs, but I would suggest you to use DLLs.
This will give other developers (who
may get interested in your software)
a chance to use any development
language (as long as that language
can spit out dll) to write their own
plugins that can be used in your
developed software.
Another thing is that you will be
saved from Delphi's vcl version
dependency tyranny. A major weak
point of Delphi till date.

Compile delphi 5 code in Delphi 2009

It is possible to work with a Delphi 5 project in the Delphi 2009 IDE by referencing the Delphi 5 version of dcc32?
If so are there any issues to watch out for concerning the way that project settings (search paths, conditional defines etc.) are implemented in 2009?
Edit: Just to clarify I am also upgrading the project to Unicode but will still need to debug and run releases in the old configuration
It depends on what you're trying to accomplish and what limitations you are willing to accept.
As far as I know, you can't use the Delphi 2009 IDE to maintain Delphi 5 projects directly. For example, even if you stick to functionality that's common between the two, some properties that are not supported in Delphi 5 are written to your DFMs, causing an error at run time.
I've maintained projects and library code that were written in Delphi 2005/2006/2007 that was also being used in Delphi 6/7. I usually edited and debugged these using the latest IDE. I had separate project files for each target version and made sure they all used the same memory manager. Finally, I had an automated build process and unit tests that would strip incompatible properties out of the DFMs (my own DFM Scrubber), make sure all of the targets always compile and run unit tests, which are also recompiled for each target.
All in all, it's more effort and I wouldn't recommend it unless you have a specific requirement to do so.
No. That said, it is still Delphi, and assuming you have source or D2009 versions of any custom components it can be modified to compile in Delphi 2009. The layout of the VCL has changed quite a bit since D5, so expect to have to modify your uses clauses and probably rewrite some small chunks here and there, but it is doable.
You either port your code to Delphi 2009/2010 level (Unicode), or you may as well not install the product.
I suggest you open the project and see where it fails, close the project (without saving anything), find the component versions you need and install them, and once the project opens up in design mode (all components are installed) you can start porting.
Read the Unicode Delphi migration (porting) information available at the website.
Ask your self each time you see PChar, and Char, if it needs to be PAnsiChar, or AnsiChar instead? If you are reading bytes from a disk, a com port, or a network connection, you will need to change from Char to AnsiChar, from PChar to PAnsiChar, otherwise, you might just leave the Char and PChar as they are and they will become Unicode. Always be aware that Char is not a Byte, anymore.
You also must replace explicit references to narrow Win32 API calls with versions without the A (ansi) suffix. Example: CreateFileA might need to become just CreateFile.
W

Resources