I want to apply a fix from QC to a Delphi 2009 unit (DBClient as it happens). I know I need to copy the unit to another directory and make the change to the copy. How do I then get Delphi to compile that unit and use it in favour of the DCU that already exists?
If you don't want to modify the original .Pas file, I do this by copy the .Pas file into my application folder, then choose built project, it will create new dcu file in my application folder, which will be used instead of the original one.
It's kind of a last resort (and not supported by CodeGear), but I do something similar to Mohammed when necessary. Except instead of putting any modified units into the application folder, I put them into their own folder with the rest of my library code and include this folder in my library path where it will be used by all of my projects. It also prevents me from having multiple (possibly slightly different) copies hanging around.
I also make a point of checking any updates to see what has changed so I can either remove the modified units or re-apply the changes to the newer (and presumably better) units from CodeGear.
I've never did this myself but there are projects in {RAD}\source\rtl along with batch build script. I believe this makes recompiling RTL functions easy. Other units should be recompiled easier.
If the changes you want to do are local and the units aren't widely used by other RTL/VCL units, the simplest way is to place copies of modified units separately from their standard place.
Another option is run-time patching aka detouring.
Related
I have this craving to do some experiments with modifying the underbelly of the Delphi run time library (RTL), system.pas and the likes... It is possible or not?
I'm very fond of challenges like "yes, but you'll have to provide custom .obj files for some assembler wizardry because they were never distributed with the official Delphi source". Fine with me, I just want to know.
I want to do this experiment with Delphi 7, but inside information on any other version is fine. It is one of the perks of being with a company that worked with Delphi since the Stone Age.
(I always figured this to be one of those RTFM questions, with the answer being a resounding "NO!", but for some reason google won't confirm it.)
You can recompile the RTL like any other unit.
For System.pas you must use the command line compiler.
For instance, here is a working batch file content (there is some not well documented command line switches):
del *.dcu /s
"c:\program files\borland\delphi7\bin\dcc32.exe" -O+ -Q -M -Y -Z -$D+ System.pas
This will recompile System.pas and SysInit.pas (both lowest level RTL files).
But in order to use your recreated dcu files, you'll have to put the folder containing the updated dcu files into the first position of your IDE: for instance, in Delphi 7 it's Option / Environment Options / Library, then put your folder FIRST in both "Libary path" and "Browsing path" field.
And it's perhaps worth deleting the original .dcu files in your Delphi installation directory.
But be sure you won't change the "interface" part of the unit, or you'll have troubles with compiling with other not modified units of the RTL (or third-party components). You can change the "implementation" part, apply fixes or rewrite some part for speed or such, but don't change the "interface" part to avoid any linking error.
Always make a backup of the original .pas and .dcu files which you are changing. And it's a good idea to make some automated compilation test, so that you could be sure that your modifications of the RTL won't add any regression.
We made such a RTL recompilation for our Enhanced Run Time Library for better speed of low-level RTL functions (mostly System.pas and SysUtils.pas). Designed for Delphi 7 and 2007. For more recent Delphi version, you still can use the same principle.
You can only recompile the RTL from the command-line. There should be a makefile in the RTL source directory of your installation. It is designed to be used with the make.exe command-line utility which should be in the "bin" folder of your installation. I would recommend you copy the relevant sources to a separate location for experimentation. I must caution you that the System unit is tightly coupled with the compiler which expects many functions to have a specific name and have particular parameter lists, if any are even declared. Many RTL "helper" functions don't have any formally declared parameters, yet expect parameters to be passed in a certain fashion.
Another bit of caution is changing the interface declarations of certain classes, functions or types. Doing so may cause serious incompatibilities with existing DCU files and components. For this reason you must be very careful when intermixing DCU files from the included RTL or third-party components with your custom modified versions. I would suggest you start by only making implementation section changes only before venturing into the mine-field of interface breaking changes.
For over a decade, I have been stuck (lovingly) on Delphi 6 and have developed hundred of thousands of lines of code in a logical (to me) unit structure, where a project tends to be a few hundred lines of code referencing high-level work routines in my "library". In trying to migrate to XE5, I just can't find a way to have all my library units compiled in one place on the search path, and then just used by the project (and recompiled if necessary), but that the dcus are off with the library source rather than in with each individual project.
I'm just starting to accept that "hello world" takes 2.5Mb in XE5, and I can't stand the idea that each library unit has to be separately compiled into dcus at the project level. In the "old" days, these unit dcus would sit next to the pas files and not be recompiled if nothing changed in the source file.
The obvious place to look is the Project options, but I can't find the right setting to make the project stop keeping copies of each dcu.
I am vaguely aware that multi-platform development will cause restructuring, but I can't help feeling that there is some compromise position.
There must be something big I am missing.
Starting in Delphi XE2, Delphi supports compilation for multiple platforms, as well as different build configurations. Because of this, Delphi needs to create DCU files for each combination. For example, Win32, Win64, and OS-X DCU files are saved in separate folders by default. Otherwise, if it weren't like this, the DCU files would overwrite each other, which you should avoid (if you use different configurations/platforms).
These settings can be changed in the Project Options in the very first section Delphi Compiler by modifying the Unit output directory. This is by default .\$(Platform)\$(Config) which creates a subfolder for the platform, then another subfolder for the config, for example \Win32\Debug\. Careful for the Target at the very top, which by default is set to your current platform/config. You would typically want to first change it to All Configurations. If you clear this field completely from the options, it will produce your default behavior from older versions.
It sounds like you should create a Package. This would allow you to group all your "library" units together in one place (BPL). This package can then be installed into your IDE, and if you have any components, those components can then be installed into your component pallet.
Or you can do without a package too. All the units from all these different projects should be moved to this central place though - a single folder containing all your "library" units. This way it's less maintenance, and you can just add that one folder to your global library path.
If you put your files in a central folder, and use those files from a project, the DCU files for both the project and this "library" will be saved for that project. Delphi doesn't know that these files are a "library", it just knows that you're using them, and since it can't find an already compiled version of those units, it creates one in your project. If you want the DCU files to be saved only once and in this central place, then you would need a package.
First, let me thank all the respondents to this question - all provided useful insight. I experimented with the various suggestions (including breaking XE5 so badly that I had to reinstall -- at least I learned some areas not to mess with.)
Important to me, but a known bad coding practice, is having individual projects edit shared library units (only my own units - I do not mess with code belonging to Delphi or 3-rd party). This is critical to having multiple applications working on the same data, but in bite-sized pieces. the shared code lets me make high-level pieces of an app available to other projects. There might be better ways (I would love to hear about them), but this has worked for me for a long time.
The multi-platform model really requires the dcu structure used by default, so I will adapt to it. Share the source code, but accept multiple compilations to individual projects. A good suggestion by JensG is just to clean up the dcus when the project is not actively being worked on. Should be straight-forward utility program.
The D6 -> XE5 migration (which will take months for some of the less used areas) requires me to know which units compile successfully, so I will maintain one project whose function is to include all units and recompile them all. This will make it practical to map library unit pas files to dcu files.
The AnsiString/AnsiChar <-> String/Char problem is the major migration problem area. Simply making edit-level changes may get the code past the compiler, but there is no guarantee that the code still works the same way. Especially troubling is at interface points to Windows calls and such. My answer will be to make the units compilable first, but then write test code for trouble areas. This is what will take the months - I need to get on with new stuff, as well as fixing the old. I REALLY don't know yet if I will be able to substitute the XE5 compatible code back into Delphi 6 without another layer of testing. I THINK it should work, but it will take careful checking.
A second major migration problem is 3rd party code such as VCLZip. XE5 has its own zip support, but I have a lot of places where I use VCLzip and the conversion will not be trivial. For this specific library, it may be possible to find XE5-level source and simply work it in. There are other pieces of code gotten from the internet that I used, but never needed to truly understand which will cause significant hassles.
Again - thank you to all. This has been an interesting 24 hours. Howard
I need (to make some quick and dirty tests) to modify the code of Variants and SysUtils.
What I need to do to "compile" the changes?
I can of course open those units in the IDE, but if I change them and I buoild a project again I don't see those units recompiled.
What is needed to be done?
The problem is you would need to compile ALL of the RTL/VCL against the 'new' units.
Instead modify a copy of the units in question and add them to your project when you want to use them. Delphi should use these over those in the RTL/VCL.
Unless you do not change the interface part of the unit (that is, you only modify the implementation side), you can make your own version of the RTL units (only exception is System.pas and SysInit.pas, but this is not in your scope - see our blog site for some enhancements of those units).
What you need is to put your own version of Variants.pas and SysUtils.pas in the search path of your project. They will be taken in account instead of the default RTL.
But be aware that you may easily break anything.
For testing purpose, this is OK, but if you want to use those modifications, you shall better use some automated regression tests, and know explicitly what you are doing.
Please note that you can use the "debug" version of the RTL units (from the project options), then step with the debugger within the official source code. It may help finding issues without touching the source.
If you change the interface part of the unit, you'll have to recompile all units which call the modified unit - for SysUtils and Variants, this is almost all RTL.
Delphi runtime DCUs are precompiled. It would be a waste of time to compile them at every build.
If the code you are trying to modify is a method of a built-in class, then a class helper may help:
http://docwiki.embarcadero.com/RADStudio/en/Class_and_Record_Helpers
So the question is which part of code do you want to modify in the runtime?
If you really wish to recompile the RTL you can do so (Make a backup first!). Versions of Delphi prior to Delphi 2010 had a makefile in the source folder that could be run from the command line to rebuild the rtl/vcl. I don't know for sure (I'm still using D2009) but from what I've heard this file is no longer present in newer versions. Hopefully there is an alternative. Otherwise you would wind up wasting a lot of time trying to guess that the compiler settings for each unit.
If you wish to "patch" a bug in the rtl for your project only you can copy the unit you want to modify into your project's folder and make your change. If the unit your modifying is used throughout the RTL/VCL you may find yourself copying quite a few dependent units into your project folder in order for it to compile.
If this significantly slows down the compile time for your project you can always do your initial compile then remove the "patched" units, leaving behind the compiled dcus.
I am trying to start making my own libraries avaialble as packages prior to compiling my Apps with these packages hence modularising my code. For years I've 'sort of' understood packages, breathing a sigh of relief when I load a component package and click on 'Install' and it does. I understand that the process of installing a component (or components) is via the creation of a BPL which is then registered with the IDE.
Where I begin to get lost is how to make files available so that I can compile with EITHER a package OR pre-compiled dcu's (like the third party vendors do) and without pointing my project at the source code all the time. I can create a package with the following settings:
where I've specified that all my output will go into 'c:\scratch\wow'. After a build I find TEST.BPL, TEST.DCP and lots of DUC's. Now, when I point another project at this folder to use the DCU's, I get a missing DFM error (one of the units is a form). Should I be manually copying needed DFM's into this output folder? The DPK knows about this form, so why do I not get the DFM copied for me? I presume that using TEST.BPL, that file contains everything, but I wish to work in the two modes. Of course I can get around this by including the source folder in my project search path to find the DFM but third party libraries seem to already have the DFM's in their output folder. Did they install them there using the installer?
Thanks
instead
As others say you could use post-build events to copy your DFM files into place. Other people use a one-time external batch file that copies the DFMs to the DCU folder.
Personally I see very little benefit to making packages for things which are not developed also as reusable components. I also see very little benefit in partitioning an existing application into packages, when you don't reasonably need to use the same subsection or package more than once, or at designtime.
Things I would put into packages:
Delphi visual and non-visual components.
Things which absolutely must be plugged in at runtime, or left out. For example, supposing I sell MetaWare Light and MetaWare Pro, and instead of using compiler IFDEFs to build a differnt binary, I preferred for some reason to simply not ship the ADVANCEDFEATURE.BPL with my systems.
Things to beware of with packages:
I have run into a lot of compiler bugs when combining packages with generics. I have also run into IDE crashes and lockups, in Delphi 2009, 2010, XE and XE2. (I believe XE3 is better)
You should learn a bit about BorlandMM.dll and shared memory management in the BPL world before you move to it. There are some subtleties.
Packages limits the ability of the linker to decide what to remove. In fact, it pretty much destroys it. Packages contain everything that is linked into them, and nothing publically accessible can be removed.
Once you've created a binary package and shipped it to even one customer, you have a pretty difficult to modify contract (this BPL contains a particular signature or application binary interface) you have to be careful in the future to never change them, or mix and match them. Beware of DLL hell, even among your own customers, and be prepared to use versioning on your packages. Just as delphi packages have a version suffix, I recommend you use version suffixes in your own packages right off the bat, and bump them whenever binary compatibility has changed.
Delphi handles build dependencies between packages about as well as could be hoped, which is less well than a single monolithic application. In applications that I have that make heavy use of packages, I find project groups that contain a bunch of packages that depend on each other are very difficult to manage and build quickly. In fact, I have experienced that both compiles and builds are slower and more frustrating than they would be in a singular 750Kline megaproject.
I really wonder if you're not that into the package area of Delphi (you breath a sigh of relief whenever a delphi component actually builds and installs without issue?) if you really want to move into the Packages World totally. By all means, you should experiment. But I wouldn't bet the farm on it yet. Learn some more first.
Yes, you should copy the .dfm to the directory with the compiled units (.dcus), if that is the only directory you want in your search path. The BPL will of course contain the .dfms, and you need a .dcp to be able to link a BPL with your app.
Third party tools must have put the .dfms together with the .dcus in the directory using their installer, indeed.
Instead of copying *.DFM manually you can use Post-Build Event (Project/Options/Build Event), ex:
copy “$(PROJECTDIR)\Unit1.DFM” “c:\Scratch\wow\Unit1.DFM”
I found a way to do this without moving .dfm files to the directory of .dcu files, so you can have a directory for .dcu files only one for .dcp files only and another for .bpl files only.
All you need to do is to create another directory on your good structure, as I do. The directory is called RES and in it should be placed all the resource files (.res files, not .dcr files) that are used by applications compiled using your packages (components). In the Delphi Library Path, you must include in addition to the DCU directory (you should already have) a directory named RES.
On your component (design time) do everything you want with the form (design it, put other components, etc). In the source code of the unit you replace {$R *.dfm} with {$R UnitName.dfm}. In doing so, save all and close the DPK. Now move the .dfm file (do not copy, move!) to the RES folder (the .dfm file is a resource file to the Delphi. The {$R} directive is proof!) and after that open the DPK again to understand what has changed.
First realize that you may not open the form (F12) from his unit, though no error was issued by Delphi about "DFM missing".
Now, do a Build on your package and then install it. Realized again? No errors displayed! This happened because you have indicated the location of .dfm file in the Delphi library search path (RES directory).
Done! You can use your component and dfm will be found when your component is included in an application.
Many of you can now say that this way I will not longer be able to visually edit a form in the component design time. Yes this is true, but if you think about it, why would I want to change so often a form into a component that, in practice, should only be used and slightly edited? Draw your own conclusions ;)
I need to know if is there any option to compile a delphi project only if the source or any used unit, package etc has been changed.
If this is not possible, second alternative : Is there any option to generate exactly the same binary compiling two times the same project.
Thanks.
Edit: The usage is for a hash based WebUpdate.
A Delphi "compile" will compile only changed units. But as said, unit have data so the compiler can check which needs updating and which not. And the executable can change because the build process can rearrange the exe. Your web updates should not use a file hash, it should use version information to decide what to update and what not. That's the way installers check which files should be replaced.
Delphi compiles only changed units on a compile, and compiles all units regardless of change-status on a build.
Exe's are never the same, on a binary level. Just built a project twice, renamed the exe's to have a txt extension and compared them with Beyond Compare: it shows differences.
One (hard) solution: Make your exes using a Makefile! A makefile allows you to say "this exe is made from those files, using those commands". Make will only run "those commands" if at least one of the files you list as making up your exe is newer then your exe.
The hard part in this is setting up the list of files that make up your exe: You can easily get the list of files listed in the DPR/DPROJ, but you'll also need to identify all the linked resources ($R), all the included files ($INCLUDE), all the files that are implictelly compiled by Delphi because they're used in the "uses" clauses and are found on the Library Path.
Generating a Makefile for the general case is very difficult, but for one particular project it might work. For example you might consider your file dependent on only the files listed in the DPR files and then make sure you add all the relevant files to the DPR.