Solving Delphi BPL Package problems where BPLs won't load but you've already recompiled (Windows VirtualStore filesystem issue) - delphi

My general question is how do you troubleshoot "My BPL won't load due to a dependency that just won't go away, no matter how much I clean up and recompile". Update You may think you have a clean recompiled system, but thanks to the inverse-miracle that is Windows and its file system virtualization mis-features, you haven't.
When I try to load my designtime package (in this case named dclFsTee.bpl) into my Delphi IDE (it's the fast report 4 teechart wrapper component package), it complains:
The program can't start because tee7100.bpl is missing from your computer. Try reinstalling ...
That tee7100.bpl is not referenced on any DCP or DCU file on my system THAT I KNOW OF. But clearly, something is wrong, and I can't find the problem.
All Delphi users face a hundred "won't compile or won't load" problems with BPLs. The universal refrain when asked what to do is to clean up your computer.
However, I've now spent hours cleaning up my computer, and while everything compiles file, clearly there must be something out of date hiding somewhere, because the resulting BPL file that I'm trying to load still wants to load a version of a TeeChart BPL that I removed from this system days ago, along with every trace I could find.
The traces of TeeChart stuff in Delphi 2007 that I removed include everything in the $(BDS)\Lib and $(BDS)\Lib\debug folder, and all DCP and BPL folders on the system. Also every TeeChart-unit-named dcu file is gone.
Once you've gotten to the end of the road, what do you try next? (Format the hard drive, buy new computer.) Seriously. I think I'm a smart guy, but I have a 1 tb hard drive, a library path that runs to 80+ folders, and a source code repository that seems to be well organized, but clearly something is hiding where I can't find it.
I have TeeChart Standard 2012, with full source code, and as far as I know, my development machine no longer contains any old TeeChart BPLs or DCP files from the "tee chart tee7100.bpl" version that ships with delphi.
I have run the "recompile.exe" wizard that comes with teechart, which appears to just run MSBuild and build the packages, after writing a {$DEFINE x} declaration to the tee.inc files (there are two of them in the source distribution).
However, somehow, silently it seems like one of the implicit imports into one of the packages is drawing in some stale file which has not been rebuilt, and which therefore tries to load the tee7100.bpl. The new bpl name is tee911.bpl.
Rather than ask the pretty-specific-to-fastreport question, I'm only mentioning it as a specific instance of a general world of hurt that I have faced dozens of times while developing in Delphi.
I'm only giving the fast-report details so you can see that this is in fact a specific instance of a general problem that one faces sometimes inside Delphi IDE when dealing with a component source code or package, or set of packages, with dependencies. Cleaning up your computer so that your code even builds can be tricky.
So here is my Delphi package-to-package-dependency-resolution question:
What is the most effective way to find or trace implicit-load-of-some-no-longer-wanted BPL-problems so that my code (which builds and compiles just fine!) will actually load into the Delphi IDE. The BPL file that results from running Recompile seems to be linking properly to the right DCP files, and no old/stale DCP or DCU files are present. The new DCP file name is tee911.dcp, for instance.
Can you get somehow, any idea of what package is actually stale, and what is being read and linked and imported statically when the .bpl links? (I'm thinking maybe like a special MAP-like file for BPL files?)
Update After many hours of fighting with this, and using every trick I know, I realized I hadn't checked for some VirtualStore related issues caused by file virtualization in Windows 7. That means that Windows 7 lies to the programs that run on top of it. It gives you another version of the file, that isn't the one you want. This can be deadly in several ways; One; You recompile a BPL but that's not the one that loads. The BPL that was killing me was in the SysWow64 folder that was part of the VirtualStore. Note that the virtualstore basically makes phantom files appear that are only there if you're a certain "low privelege" program, which Delphi 2007 on Win7/64 bit, apparently is. To remove BPL files in your SysWow64 VIRTUALSTORE folder for your current user account:
del %HOMEPATH%\AppData\Local\VirtualStore\Windows\SysWow64\*.bpl
... Some days I just hate Windows architecture. Anyways, I'm not going to put the above as the answer, because I'd like to know if anyone has a better way or any tip or suggestion that might help next time.

Okay nobody else answered so I'll put this here to be helpful for future people:
-- Remember Windows VirtualStore when cleaning up broken systems which have old versions of DLLs on them including TeeChart, FastReport, Indy and so on which tend to be involved in messes because they can exist both as "out of box packages that ship with delphi" as well as frequently installed as upgraded versions if you purchased and installed them from the vendors directly, or third, you may have your own compiled copy in your company's mega-component-pack-directory.
-- When searching for duplicate or out of date BPLs, doing a file search in windows doesn't look in the virtualstores, you'll have to locate and zap the whole virtualstore area for your process or user, or program, manually.
The second level of this issue is this:
The dependency graph for FastReports is complex:
It depends on Indy and you might have your own version of Indy, and Delphi itself has one, and other things on your hard drive might have their own copy of Indy.
It supports various editions of TeeChart, including the binaries that come with Delphi, and perhaps the Standard version or other purchased version of TeeChart that you might have bought from Steema.
It uses a precompiled header include file to do the compilation and not just ONE but TWO different copies of an identically named include (.inc) file.
When you use their own compiler tool (recompile FastReport) it works pretty reliably but isn't the best for when you want to build everything in your project from a single build script, thus the source of my problem.
The key is to learn everything there is to know about the dependencies of all the components in your giant pile of packages, and to organize your system cleanly so that you don't have old stuff (like Indy and TeeChart bpls, dcp, or dcu files) lying around. Cleaning that up is quite a complex job if you don't know what you're doing.
A utility to really remove all traces of the version of Indy and TeeChart that ship with your system, and the "Embarcadero edition" of FastReports is key to getting this situation resolved. A general tip is that "if a version of X ships with Delphi and you are going to install a new version, prepare to suffer until your system is really cleaned up".
A really amazing technique to avoid all this crap is to just not install Indy, FastReport or TeeChart (uncheck them or skip them) during your initial Delphi IDE install, then install them yourself, one by one, from sources. Just because a version comes pre-installed in Delphi doesn't make that a good thing. (Update: You can no longer unselect Indy during install, it's part of the base Delphi product since at least Delphi XE8. A clean-up utility to remove the built-in Indy from Delphi's own lib dirs is necessary for anyone who builds their own.)
Another really amazing technique is to run the Installers for commercial components on a virtual machine, then just collect up the pascal source code and transfer that onto your clean development machine, and build it yourself. That way you can avoid the terrible things that happen when you've got BPLs and stuff scattered around your system, and even installed into C:\Windows\System32 (on 32 bit systems) and C:\Windows\SysWow64 (the equivalent path on 64 bit systems).

put that BPL (tee7100.bpl) under $(BDSCOMMONDIR)\Bpl
for XE: $(BDSCOMMONDIR)= "C:\Users\Public\Documents\RAD Studio\8.0"
for XE5: $(BDSCOMMONDIR)= "C:\Users\Public\Documents\RAD Studio\12.0"

The other issue that can cause this, is not having the folder where you've stored your .bpl files in your system path.
This happens because Delphi attempts to call the WinAPI function LoadLibrary with a file name, instead of an absolute path. So if Windows can't find the file, Delphi can't load it.
See this forum post for more information.
This seems to be an issue in Windows 7, though not in Windows 10.

Related

Delphi 5 BPLs Not Linking For One Program Only

I'm still using Delphi 5.
I have a program which is a rewrite of an earlier program and in fact uses the root unit from the earlier program. When I compile it, Delphi doesn't link the BPLs into the EXE for some reason. New programs and old programs compile fine, so the problem is obviously with this one program. I've compared the project options for Linking and Compiling with other programs which are linking just fine. The compiler options in the main unit are the standard ones.
While the program is on my development box, it runs as expected, obviously reading the BPLs in the development area. But of course it won't run on another machine unless I copy the BPLs over, which is obviously not very clever.
I've read everything I can find on the subject, but although I've found references to linking not working, I haven't found anything which helps to work out why the linking isn't working in just this one case.
Can anyone point me at some documentation about this or point out something I've obviously missed looking at? I know it's something I've done, but despite spending hours on it I can't work out why!
Since there are no errors generated, I can't show you where it went wrong. Any suggestions about where I might find the problem would be much appreciated.
If you go to the Packages tab of your Project Options, is the Build with runtime packages checked?
If it is, you need to deploy the other packages to other machines if they aren't already on those machines at locations where they can be found by the OS.
If it isn't, Delphi will compile and link your project as an .Exe file and the code from the DCUs in the packages will be statically linked into your .Exe so you won't need to deploy the .BPLs (and they would be ignored even if you did).
AFAIK, when Delphi compiles a project to an .Exe (that is, if Build with runtime packages isn't checked), the compiler gets the other units' code to link from the .DCU files, rather than from the .BPL files. If it can't find the .DCU files, or you have told the compiler to do a "Build" it will attempt to generate the .DCUs them from the corresponding .PAS files if it can find them either in the project directory or via the Seach Path under Directories/Conditionals.

Why are installed components not always available when Delphi starts

Delphi 6, and Indy 10.
On Delphi start I get
"Can't load package C:\Indy10_for_Delphi_6\D6\dclIndyProtocols60.bpl"
and
"Can't load package C:\Indy10_for_Delphi_6\D6\dclIndyCore60.bpl"
I know the bpl's are where they should be.
After reinstalling the components and reloading the project,
within Delphi, and NOT restarting Delphi, all is fine.
On the next Delphi start, the process repeats.
This has worked well before, and now suddenly it doesn't.
I get the feeling it may be timing issue,
as if the bpl's are unavailable a short time while Delphi starts.
I have seen related posts discussing this (ForceDemandLoadState(dlDisable))
but only for later Delphi versions.
I have also deleted the "Disabled packages" key in the registry.
What may be causing this?
That usually means dependant BPLs can't be found/loaded by the OS when the reported BPL is being loaded. The Current Working Directory when the IDE is first started may be different than when you go to install a BPL later on. Make sure any relevant folders for dependant BPLs are in your system's search path.

Same code, missing qtintf70.dll

Compiling some old code, my application will no longer run. Newly compiled exe-file won't start "because qtintf70.dll is missing from your computer".
Strange thing: an older exe file compiled from exact same code runs just fine. Both exe files tested on same system, but compiled on different Delphi installations (both Delphi 7 running on VirtualBox).
Googling, someone suggested "You have pulled in Clx somehow. Clx apps require the QT library. Look for units in your uses clauses that start with Q." but again: this is the exact same code. I've checked and can't find anything.
Suggestions?
In D7, the only source code units which reference qtintf70.dll are QForms.Pas and Qt.Pas.
So, what you need to do is to
Move all instances of these two files and their corresponding .DCU files to somewhere not on your project's search paths. Use a utility like SwiftSearch to make sure you find all of them.
Do a full build of the project.
The build should grind to a halt somewhere with a complaint that the compiler can't find one or other of these units. The source unit that is being compiled at the time is the one which contains a reference to one or other of them.

What IDE/tools for compiling Delphi Application

I am a C# developer by trade, but have been tasked with making some changes to a Delphi desktop application. I have never used Delphi and I'm trying to identify the version of Delphi the application has been compiled with, and find a (preferably free) development stack/environment.
Looking at the source, there are a bunch of .DCU, .DPP, .DFM, and .PAS files.
There's a .DPR file that would seem to reference all of the forms.
In the .CFG file I found reference to
c:\program files (x86)\borland\delphi7\Projects\Bpl
Based on the above info, what tools should I use to design and compile this application?
That entry in the .cfg file is a clear cut indication that the project was most recently built with Delphi 7.
However, it may be more tricky to build this project than just to install Delphi 7 and press build. You may very well need to install a bunch of third party packages and libraries. In an ideal world there would be documentation on how to build, or at the very least, how to create a build environment. Sadly, you may not be living in an ideal world!
I'm trying to .... find a (preferably free) development stack/environment.
Well, Delphi 7 is not free. There are free Delphi like development environments, most specifically Lazarus. However, you would need to port the existing code, which is likely to be a non-trivial task. Especially if you have no Delphi experience.
It appears to be Delphi 7.
But the compiler version is only part of your problem, as several commentators have mentioned here. Unless they used only Delphi components, you're going to need to find the component libraries that were used, and that could be nearly impossible in itself. If it used any DLLs or BPLs, you'll need to find them as well, and possibly re-build the interfaces.
I'd spend some time trying to track down a backup that was made of the system that this software was developed on, as that's your best bet. It won't cost you anything assuming everything is still intact.
But trying to load up the source into D7, or anything for that matter, is going to be a total crap shoot unless you know all of the libraries that were used, how things were configured, all that good stuff.
It would be far more efficient for your company to just hire a Delphi expert for a day or so to dig into this for you than for you to waste a week or two of your time on something you don't know anything about.

Moving to use of Delphi packages - best practice please?

I am trying to start making my own libraries avaialble as packages prior to compiling my Apps with these packages hence modularising my code. For years I've 'sort of' understood packages, breathing a sigh of relief when I load a component package and click on 'Install' and it does. I understand that the process of installing a component (or components) is via the creation of a BPL which is then registered with the IDE.
Where I begin to get lost is how to make files available so that I can compile with EITHER a package OR pre-compiled dcu's (like the third party vendors do) and without pointing my project at the source code all the time. I can create a package with the following settings:
where I've specified that all my output will go into 'c:\scratch\wow'. After a build I find TEST.BPL, TEST.DCP and lots of DUC's. Now, when I point another project at this folder to use the DCU's, I get a missing DFM error (one of the units is a form). Should I be manually copying needed DFM's into this output folder? The DPK knows about this form, so why do I not get the DFM copied for me? I presume that using TEST.BPL, that file contains everything, but I wish to work in the two modes. Of course I can get around this by including the source folder in my project search path to find the DFM but third party libraries seem to already have the DFM's in their output folder. Did they install them there using the installer?
Thanks
instead
As others say you could use post-build events to copy your DFM files into place. Other people use a one-time external batch file that copies the DFMs to the DCU folder.
Personally I see very little benefit to making packages for things which are not developed also as reusable components. I also see very little benefit in partitioning an existing application into packages, when you don't reasonably need to use the same subsection or package more than once, or at designtime.
Things I would put into packages:
Delphi visual and non-visual components.
Things which absolutely must be plugged in at runtime, or left out. For example, supposing I sell MetaWare Light and MetaWare Pro, and instead of using compiler IFDEFs to build a differnt binary, I preferred for some reason to simply not ship the ADVANCEDFEATURE.BPL with my systems.
Things to beware of with packages:
I have run into a lot of compiler bugs when combining packages with generics. I have also run into IDE crashes and lockups, in Delphi 2009, 2010, XE and XE2. (I believe XE3 is better)
You should learn a bit about BorlandMM.dll and shared memory management in the BPL world before you move to it. There are some subtleties.
Packages limits the ability of the linker to decide what to remove. In fact, it pretty much destroys it. Packages contain everything that is linked into them, and nothing publically accessible can be removed.
Once you've created a binary package and shipped it to even one customer, you have a pretty difficult to modify contract (this BPL contains a particular signature or application binary interface) you have to be careful in the future to never change them, or mix and match them. Beware of DLL hell, even among your own customers, and be prepared to use versioning on your packages. Just as delphi packages have a version suffix, I recommend you use version suffixes in your own packages right off the bat, and bump them whenever binary compatibility has changed.
Delphi handles build dependencies between packages about as well as could be hoped, which is less well than a single monolithic application. In applications that I have that make heavy use of packages, I find project groups that contain a bunch of packages that depend on each other are very difficult to manage and build quickly. In fact, I have experienced that both compiles and builds are slower and more frustrating than they would be in a singular 750Kline megaproject.
I really wonder if you're not that into the package area of Delphi (you breath a sigh of relief whenever a delphi component actually builds and installs without issue?) if you really want to move into the Packages World totally. By all means, you should experiment. But I wouldn't bet the farm on it yet. Learn some more first.
Yes, you should copy the .dfm to the directory with the compiled units (.dcus), if that is the only directory you want in your search path. The BPL will of course contain the .dfms, and you need a .dcp to be able to link a BPL with your app.
Third party tools must have put the .dfms together with the .dcus in the directory using their installer, indeed.
Instead of copying *.DFM manually you can use Post-Build Event (Project/Options/Build Event), ex:
copy “$(PROJECTDIR)\Unit1.DFM” “c:\Scratch\wow\Unit1.DFM”
I found a way to do this without moving .dfm files to the directory of .dcu files, so you can have a directory for .dcu files only one for .dcp files only and another for .bpl files only.
All you need to do is to create another directory on your good structure, as I do. The directory is called RES and in it should be placed all the resource files (.res files, not .dcr files) that are used by applications compiled using your packages (components). In the Delphi Library Path, you must include in addition to the DCU directory (you should already have) a directory named RES.
On your component (design time) do everything you want with the form (design it, put other components, etc). In the source code of the unit you replace {$R *.dfm} with {$R UnitName.dfm}. In doing so, save all and close the DPK. Now move the .dfm file (do not copy, move!) to the RES folder (the .dfm file is a resource file to the Delphi. The {$R} directive is proof!) and after that open the DPK again to understand what has changed.
First realize that you may not open the form (F12) from his unit, though no error was issued by Delphi about "DFM missing".
Now, do a Build on your package and then install it. Realized again? No errors displayed! This happened because you have indicated the location of .dfm file in the Delphi library search path (RES directory).
Done! You can use your component and dfm will be found when your component is included in an application.
Many of you can now say that this way I will not longer be able to visually edit a form in the component design time. Yes this is true, but if you think about it, why would I want to change so often a form into a component that, in practice, should only be used and slightly edited? Draw your own conclusions ;)

Resources