I need to know if is there any option to compile a delphi project only if the source or any used unit, package etc has been changed.
If this is not possible, second alternative : Is there any option to generate exactly the same binary compiling two times the same project.
Thanks.
Edit: The usage is for a hash based WebUpdate.
A Delphi "compile" will compile only changed units. But as said, unit have data so the compiler can check which needs updating and which not. And the executable can change because the build process can rearrange the exe. Your web updates should not use a file hash, it should use version information to decide what to update and what not. That's the way installers check which files should be replaced.
Delphi compiles only changed units on a compile, and compiles all units regardless of change-status on a build.
Exe's are never the same, on a binary level. Just built a project twice, renamed the exe's to have a txt extension and compared them with Beyond Compare: it shows differences.
One (hard) solution: Make your exes using a Makefile! A makefile allows you to say "this exe is made from those files, using those commands". Make will only run "those commands" if at least one of the files you list as making up your exe is newer then your exe.
The hard part in this is setting up the list of files that make up your exe: You can easily get the list of files listed in the DPR/DPROJ, but you'll also need to identify all the linked resources ($R), all the included files ($INCLUDE), all the files that are implictelly compiled by Delphi because they're used in the "uses" clauses and are found on the Library Path.
Generating a Makefile for the general case is very difficult, but for one particular project it might work. For example you might consider your file dependent on only the files listed in the DPR files and then make sure you add all the relevant files to the DPR.
Related
I have this craving to do some experiments with modifying the underbelly of the Delphi run time library (RTL), system.pas and the likes... It is possible or not?
I'm very fond of challenges like "yes, but you'll have to provide custom .obj files for some assembler wizardry because they were never distributed with the official Delphi source". Fine with me, I just want to know.
I want to do this experiment with Delphi 7, but inside information on any other version is fine. It is one of the perks of being with a company that worked with Delphi since the Stone Age.
(I always figured this to be one of those RTFM questions, with the answer being a resounding "NO!", but for some reason google won't confirm it.)
You can recompile the RTL like any other unit.
For System.pas you must use the command line compiler.
For instance, here is a working batch file content (there is some not well documented command line switches):
del *.dcu /s
"c:\program files\borland\delphi7\bin\dcc32.exe" -O+ -Q -M -Y -Z -$D+ System.pas
This will recompile System.pas and SysInit.pas (both lowest level RTL files).
But in order to use your recreated dcu files, you'll have to put the folder containing the updated dcu files into the first position of your IDE: for instance, in Delphi 7 it's Option / Environment Options / Library, then put your folder FIRST in both "Libary path" and "Browsing path" field.
And it's perhaps worth deleting the original .dcu files in your Delphi installation directory.
But be sure you won't change the "interface" part of the unit, or you'll have troubles with compiling with other not modified units of the RTL (or third-party components). You can change the "implementation" part, apply fixes or rewrite some part for speed or such, but don't change the "interface" part to avoid any linking error.
Always make a backup of the original .pas and .dcu files which you are changing. And it's a good idea to make some automated compilation test, so that you could be sure that your modifications of the RTL won't add any regression.
We made such a RTL recompilation for our Enhanced Run Time Library for better speed of low-level RTL functions (mostly System.pas and SysUtils.pas). Designed for Delphi 7 and 2007. For more recent Delphi version, you still can use the same principle.
You can only recompile the RTL from the command-line. There should be a makefile in the RTL source directory of your installation. It is designed to be used with the make.exe command-line utility which should be in the "bin" folder of your installation. I would recommend you copy the relevant sources to a separate location for experimentation. I must caution you that the System unit is tightly coupled with the compiler which expects many functions to have a specific name and have particular parameter lists, if any are even declared. Many RTL "helper" functions don't have any formally declared parameters, yet expect parameters to be passed in a certain fashion.
Another bit of caution is changing the interface declarations of certain classes, functions or types. Doing so may cause serious incompatibilities with existing DCU files and components. For this reason you must be very careful when intermixing DCU files from the included RTL or third-party components with your custom modified versions. I would suggest you start by only making implementation section changes only before venturing into the mine-field of interface breaking changes.
I have all the files needed to open the unit and code but I can't make any changes or compile because the Project.dproj and Project.dpr files are missing. However, I have the .exe file. Is there any way by which the aforementioned files can be extracted from the .exe file?
If you have all of the source files, meaning the .pas and .dfm files, then the first thing I'd do is add them all into a new project and try to compile it. It'll either work, meaning you've got everything you need, or it will generate errors.
There would be three kinds of errors:
The first is the most likely -- the forms might use components that you don't have installed. This will be obivous if you try to open the forms in the IDE. If they open without fanfare, GREAT! If you get a warning box saying "Cannot locate component: remove references to it?" or something like that, then you're probably hosed.
The second is version-specific errors -- stuff that's old and crufty and no longer supported by the language.
Third, the program may use run-time libraries that you don't have.
The compiler adds RTTI metadata, including unit names, when it builds, and if you know about how the RTTI tables are laid out it's possible to extract this. The unit list is the principal component of the DPR file, but it takes some serious work to access it and I'm not aware of any tools out there that read the RTTI tables that have been kept up to date beyond the Delphi 7 era.
As for the .dproj file, you're out of luck. That doesn't actually contain any code that gets "compiled in" to the EXE; it's a set of build instructions for how to produce the EXE. But if the Delphi IDE doesn't have one, it can generate a default .dproj from the .dpr, if you can produce that.
I'm curious, though. How did you obtain the .pas files but not the .dpr?
I am trying to start making my own libraries avaialble as packages prior to compiling my Apps with these packages hence modularising my code. For years I've 'sort of' understood packages, breathing a sigh of relief when I load a component package and click on 'Install' and it does. I understand that the process of installing a component (or components) is via the creation of a BPL which is then registered with the IDE.
Where I begin to get lost is how to make files available so that I can compile with EITHER a package OR pre-compiled dcu's (like the third party vendors do) and without pointing my project at the source code all the time. I can create a package with the following settings:
where I've specified that all my output will go into 'c:\scratch\wow'. After a build I find TEST.BPL, TEST.DCP and lots of DUC's. Now, when I point another project at this folder to use the DCU's, I get a missing DFM error (one of the units is a form). Should I be manually copying needed DFM's into this output folder? The DPK knows about this form, so why do I not get the DFM copied for me? I presume that using TEST.BPL, that file contains everything, but I wish to work in the two modes. Of course I can get around this by including the source folder in my project search path to find the DFM but third party libraries seem to already have the DFM's in their output folder. Did they install them there using the installer?
Thanks
instead
As others say you could use post-build events to copy your DFM files into place. Other people use a one-time external batch file that copies the DFMs to the DCU folder.
Personally I see very little benefit to making packages for things which are not developed also as reusable components. I also see very little benefit in partitioning an existing application into packages, when you don't reasonably need to use the same subsection or package more than once, or at designtime.
Things I would put into packages:
Delphi visual and non-visual components.
Things which absolutely must be plugged in at runtime, or left out. For example, supposing I sell MetaWare Light and MetaWare Pro, and instead of using compiler IFDEFs to build a differnt binary, I preferred for some reason to simply not ship the ADVANCEDFEATURE.BPL with my systems.
Things to beware of with packages:
I have run into a lot of compiler bugs when combining packages with generics. I have also run into IDE crashes and lockups, in Delphi 2009, 2010, XE and XE2. (I believe XE3 is better)
You should learn a bit about BorlandMM.dll and shared memory management in the BPL world before you move to it. There are some subtleties.
Packages limits the ability of the linker to decide what to remove. In fact, it pretty much destroys it. Packages contain everything that is linked into them, and nothing publically accessible can be removed.
Once you've created a binary package and shipped it to even one customer, you have a pretty difficult to modify contract (this BPL contains a particular signature or application binary interface) you have to be careful in the future to never change them, or mix and match them. Beware of DLL hell, even among your own customers, and be prepared to use versioning on your packages. Just as delphi packages have a version suffix, I recommend you use version suffixes in your own packages right off the bat, and bump them whenever binary compatibility has changed.
Delphi handles build dependencies between packages about as well as could be hoped, which is less well than a single monolithic application. In applications that I have that make heavy use of packages, I find project groups that contain a bunch of packages that depend on each other are very difficult to manage and build quickly. In fact, I have experienced that both compiles and builds are slower and more frustrating than they would be in a singular 750Kline megaproject.
I really wonder if you're not that into the package area of Delphi (you breath a sigh of relief whenever a delphi component actually builds and installs without issue?) if you really want to move into the Packages World totally. By all means, you should experiment. But I wouldn't bet the farm on it yet. Learn some more first.
Yes, you should copy the .dfm to the directory with the compiled units (.dcus), if that is the only directory you want in your search path. The BPL will of course contain the .dfms, and you need a .dcp to be able to link a BPL with your app.
Third party tools must have put the .dfms together with the .dcus in the directory using their installer, indeed.
Instead of copying *.DFM manually you can use Post-Build Event (Project/Options/Build Event), ex:
copy “$(PROJECTDIR)\Unit1.DFM” “c:\Scratch\wow\Unit1.DFM”
I found a way to do this without moving .dfm files to the directory of .dcu files, so you can have a directory for .dcu files only one for .dcp files only and another for .bpl files only.
All you need to do is to create another directory on your good structure, as I do. The directory is called RES and in it should be placed all the resource files (.res files, not .dcr files) that are used by applications compiled using your packages (components). In the Delphi Library Path, you must include in addition to the DCU directory (you should already have) a directory named RES.
On your component (design time) do everything you want with the form (design it, put other components, etc). In the source code of the unit you replace {$R *.dfm} with {$R UnitName.dfm}. In doing so, save all and close the DPK. Now move the .dfm file (do not copy, move!) to the RES folder (the .dfm file is a resource file to the Delphi. The {$R} directive is proof!) and after that open the DPK again to understand what has changed.
First realize that you may not open the form (F12) from his unit, though no error was issued by Delphi about "DFM missing".
Now, do a Build on your package and then install it. Realized again? No errors displayed! This happened because you have indicated the location of .dfm file in the Delphi library search path (RES directory).
Done! You can use your component and dfm will be found when your component is included in an application.
Many of you can now say that this way I will not longer be able to visually edit a form in the component design time. Yes this is true, but if you think about it, why would I want to change so often a form into a component that, in practice, should only be used and slightly edited? Draw your own conclusions ;)
In a fresh VCL application Compile and Build operation produce the same binary and map file (with minor differences at the end of .exe file even if "include version information in project" option is switched off - already discussed). Map file is the same byte to byte. But wen I add any third-party component the binary and map(!) file produced by Build and Compile are significantly different!
Tested on two versions of Delphi:
- Version 7.0 (Build 8.1)
- CodeGear™ RAD Studio 2007 Version 11.0.2902.10471 (+December 2007 Update)
Step to reproduce:
Create New VCL Application. Possibly add any native Delphi component (I try all components from Standart, Additional, Win32 and System tab).
Turn on Detailed Map file on Linker tab of the Project Options.
Build project.
Rename output .exe and .map file (for example: project1.exe to project1b.exe and project1.map to project1b.map).
Compile project.
Rename output .exe and .map file (for example: project1.exe to project1c.exe and project1.map to project1c.map).
Compare files from step 4 and 6. (I use WinMerge
2.12.4.0).
We have little different .exe files and fully identical .map files. Then if we repeat all steps again but use in the project third-party component (I try ODAC, DOA, DevExpress and selfmade) we get more different .exe and different .map files.
Why? Any suggestions?
UPDATE
Some information about how I found this and why it's interests me:
Project is build from simple script with MSBuild. When in the project was added translation thru ITE (dll with resources) I found that when project was Build (from script or from IDE) - translated version work wrong - some text on button, labels etc. got from wrong place (literally from another button, labels). When project Compiled from IDE - everything is ok. So I start compare Build and Compile output...
What you're seeing is simply an artifact of the built-in make logic of the compiler. When you do a build, it tells the compiler to build all available sources. So Delphi processes each source file and for each unit in the uses lists for which it finds source, it will then build that file. It does this recursively. When you do a compile, only the existing .dcu files are loaded and if they're found to be up-to-date, nothing is done. This can actually lead to a different order in which the units are discovered since each .dcu will effectively "flatten" the uses list. Since the units are discovered and loaded in a different order, they are in-turn, linked in a different order. This is why your map files look so different. Given the same sources, the map file should be the same if you do two builds in a row or two compiles in a row.
Other causes for differences are more mundane and include things like the PE header time stamp, and other bits of padding and alignments.
There is two parts to this answer I believe.
Part of the problem your seeing, IIRC, is that the compiler doesn't zero out memory before doing a compile/build. Thus anything left in uninitialized memory becomes filler in the output for alignment purposes.
I also seem to recall that there is a date time stamp included in the pe header information for the application. that will cause a difference every time.
I'm not the best person to confirm this but this is what I seem to recall from past discussions.
People like Allen Bauer or Barry Kelly would probably be able to give better/more accurate information on this.
If you use compiler defines in your project and just changed those, if you do a compile, you won't see any changes to the dcu's and the resulting module (exe or dll). If you do a full rebuild the compiler defines are used in the newly created dcu's and modules.
I have seen this in a large project group where we use modules in different projects with different defines and all dcu's are stored in the same directory.
Ergo: The compiler does not enforce the dependancies on the defines in this case.
Perhaps you did see the same issue.
I want to apply a fix from QC to a Delphi 2009 unit (DBClient as it happens). I know I need to copy the unit to another directory and make the change to the copy. How do I then get Delphi to compile that unit and use it in favour of the DCU that already exists?
If you don't want to modify the original .Pas file, I do this by copy the .Pas file into my application folder, then choose built project, it will create new dcu file in my application folder, which will be used instead of the original one.
It's kind of a last resort (and not supported by CodeGear), but I do something similar to Mohammed when necessary. Except instead of putting any modified units into the application folder, I put them into their own folder with the rest of my library code and include this folder in my library path where it will be used by all of my projects. It also prevents me from having multiple (possibly slightly different) copies hanging around.
I also make a point of checking any updates to see what has changed so I can either remove the modified units or re-apply the changes to the newer (and presumably better) units from CodeGear.
I've never did this myself but there are projects in {RAD}\source\rtl along with batch build script. I believe this makes recompiling RTL functions easy. Other units should be recompiled easier.
If the changes you want to do are local and the units aren't widely used by other RTL/VCL units, the simplest way is to place copies of modified units separately from their standard place.
Another option is run-time patching aka detouring.