I'm interested in a symbol from .pdb file, which I want to programmatically identify its calling conventions - it might be either fastcall or stdcall in different versions of a certain dll.
I see that IDA Pro and similar disassemblers know how to specify it in the right way.
Is there any way that I can identify it from my code?
Related
We are using Delphi 10.4.2 Sydney. I would like to create a single unit (or possibly multiple units) that houses all the utility functions we use in our various software products. The idea is we would just include this unit in every project we start and then we can rely on it being there and use these functions freely in all our code. These would probably be mostly class routines and such.
I'm concerned though that some of our projects might only use a single routine in this large unit. The size of our apps is important and we want to keep that size to a minimum. I am not familiar with how Delphi's compiler works or how much space adding a superfluous unit might add or any other issues with this.
Is there perhaps a way to create a Unit where only the functions we actually use will contribute to the final build's size? Is this even something to worry about? How are common utilities library usually handled in Delphi?
The Delphi Linker is supposedly smart enough to detect unused functions and will not include the code for them in the executable it generates. This also applies to class methods. (*1)
So, go ahead and create these units.
You can easily check whether code was generated for a method by looking for the blue dots during debugging.
(*1: That feature can be a major pain in the lower back when you want to call such a function from the Evaluate/Modify dialog during debugging.)
Put the units into a separate package project. If you build your exe as a monolithic exe (so packages are linked in) then the compiler should only link in the units that are actually referenced, not the entire package.
So, I have a myself written run-time package. If a package is statically linked, the project that uses has full access to exported data because the compiler has full knowledge of what is imported from it, am I right? But it's also possible to load a package dynamically via LoadPackage(). But, how to work with imported complex data structures like classes then? I couldn't find a feasible way other than constructing complex expressions like using FindClass('TSomeClass') and invoking RTTI to operate on an instance of the imported class.
The compiler has full knowledge of what's in the package because the DCU and DCP files tell it what's there.
The IDE knows what's in the package because it knows how to find the Register procedure in all the units, and that procedure tells the IDE about the available classes.
In most cases, a program knows what's in a package because the program used units from that package, and the compiler assured that mentioning names of things in those units would resolve to corresponding things in the BPL file at run time. This includes mentioning the BPL file in the program's import table, so the OS loads the BPL automatically.
If the list of BPLs you wish to load can only be determined at run time, then you cannot use any units from those packages. You have to load the package dynamically.
There's still the matter of how to use what's in those packages. You could try to discover the entire contents with RTTI. That's no picnic, though. Instead, define an intermediary package that all involved modules will use.
Define an interface or a common base class for all your packages' classes to have. Put the definition of that class in a unit that's in its own package, which we'll call Shared.bpl. Include that package in the "requires" list of all your other packages and your EXE. Now, everything can refer to the shared unit and the common base class.
This is exactly what Delphi itself does. The shared packages are called RTL and VCL. There are several common base classes already defined there, including TComponent. In your case, it sounds like you need some common definitions beyond what TComponent has.
Short answer:
What you need is the compiler/linker to set you up and use DCPs to do all the linking to types and such.
Then loading of BPLs should be delayed/done by you in custom code.
Unfortunately Delphi won't allow this, probably because of political reasons, you could try and hack it though, see below for more thorough answer.
Long answer:
Apperently DCP describe to the compiler/linker/Delphi yadayadayada what's in those packages/DLLs/BPLs type-wise.
These DCPs can be considered the interface to these DLLs/BPLs, these DCPs are probably somehow compiled into the executable.
Then the BPL probably contains the "implementation".
Now here is where the problem begins. These BPLs are "auto-loaded" as somebody else already mentioned by mentioning "import table".
What you could try is "nuking" / altering the import table so that these BPLs are not loaded automatically anymore.
Then you could try loading these BPLs manually.
I am not sure if it's just as simple as doing a "load package" operation.
Perhaps the loading involves more like acquiring pointers to routines/methods/classes, not sure how that works.
However perhaps that code can be re-used, so all you need to do is "hack" into the import table and hack into the "load bpl" and disable it.
Then you should be able to replace that with your own custom loading code and perhaps finally "re-patch" into other importing routines... like routines calling getprocaddress and such if so required, not sure about this last part.
Anyway this is very sloppy from Delphi developers that there is no functionality to do this little "import table"/"load step" yourself.
This is politics at play though, there is no technical reason why this could not be delayed, done custom by you and then call the rest if necessary.
For some reason they do not want you loading these BPLs manually. They probably want you to keep using the IDE this way.
If you were able to load these things manually, maybe the IDE would then no longer be necessary.
Currently the IDE is necessary to specify where to load this stuff from, without the IDE it will get difficult. Though there are probably also some compiler options somewhere to specify the same, though few would use that.
So to me it seems some very weird "tie-in" into this product, which I honestly must admit is pretty fucking retarded.
They did the same with winsock for example which has two different versions, very easy to load this yourself manually so you can choose which version to use instead of always v1 or v2. For easy for them to delay these getprocs and such, yet they do not...
If they did, Delphi would be much more backwards compatible with windows 95/98, currently Delphi exes no longer works for those older operating systems, though there is no real technical reason why it could not work, it basically has to do with which DLLs are loaded, just a few lines of code could very easily make it work.
This is probably again politics as play to make Delphi only support the latest Windows versions a deal struck between Delphi creators and Microsoft.
Same thing with Windows 10, it apperently detects older processors and then refuses to run on them, removing/hacking these few lines of code/instructions will make Windows 10 work on older processors. Surprise ! =D
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Reduce exe file
What are some tools that, given an EXE file, remove all unused code and make a new EXE file with code really used by the application? I think that something like that should exist. Just for curiosity; but I think that it can be really good as tools for producing a smaller EXE file without unused code.
P.S.: Delphi produces standalone EXE files that contain code of all object used in an application, but not all elements of this object are really used. This makes big files.
I remember the first version of Pascal that include only code really used and did not insert unused code, and EXE files were smaller.
Current Delphi also excludes unused code. You can easily see this in Delphi. If you compile, you get 'blue dots' in the gutter of your code. Functions that are not used don't have blue dots, meaning they are excluded. If you check the Optimization checkbox in project options, code is rewritten to make more use of registers thus eliminating certain variables.
Nevertheless, Delphi executables grow, especiall when using certain units. I think the smallest .exe you can create in Delphi 7 is about 10Kb. In later versions this will probably be a bit larger.
Important causes of file size is
resources. Large or lots of images consume a lot of space. If you use icons on multiple forms, make sure to put them in a central image list (on a shared datasource). Use one of the available PngImageList implementations for smaller image size against better quality.
rtti. The runtime type information causes class definitions to consume extra space. Thise space is partly due to the meta information about the class, but mainly because all extra code. Any methods that may be called using RTTI could be bound in a kind of 'late binding' fashion. Therefor, the compiler cannot know whether the methods can be eliminated, so it needs to include them in the executable.
registered classes. Similar to 2. If a class is registered, it can be fetched and instantiated using its name as a string. Those classes must be included in the project, even if they are never used, just because the compiler cannot know if they are needed.
It's a fact that RTTI is expanded in recent Delphi versions. I think this also causes the RTTI meta information about classes to consume more space. There's only so much you can do about that.
In general, I think the Delphi compiler still does a lot of optimizing. You shouldn't have to worry about exe file size. If you do, you could try a packer like upx. I've got good esperiences with upx. It cuts down the executables to about a fifth or less of their original size, while retaining all functionality.
If you application is huge, check if you don't have a lot of debug info compiled in.
Try a release build and check if it's any smaller.
What i sometimes do is configure the project to compile all .dcu files into a single folder. That way you quickly see all of the useless units that get compiled into your .exe. It often happens that you include a single unit for a single function, but in turn you get a whole tree of dependent units. You'll just have to search your uses clauses and try to get rid of these dependencies somehow.
I think both gexperts and cnpack contain tools to show dependencies, or to scan for unused units in your project. They can be helpful with this.
After you've removed useless dependencies, you can always compress your compiled exe with upx. There are supposed disadvantages to that (barry kelly wrote about it some time ago), but i've got good experiences with it. It sometimes makes the file 4x as small which can be a big deal.
First of all, please forgive me for not knowing the proper terminology, I'm sure there's a very common technical name for this which I could simply Google for help - but I can't find help if I don't know the term to begin with.
I'm building a modular system in Delphi 7. There are a few applications and a bunch of DLL's. All the applications share these DLL's, as well as some of the DLL's use other DLL's. The DLL's are currently saved in the same location as the application. I would instead like to put all these DLL's in a sub-folder (elsewhere from the EXE's) but of course Delphi won't know how to find them.
Is there a way I can direct my Delphi Apps to look in a certain directory for DLL's? It can't be using Contstants, because there will be an option to specify where the DLL's are stored.
These DLL's are just a plain collection of StdCall functions in each, nothing special.
EDIT:
To explain the reason why I want to keep the DLL's in their own folder: This system I'm building considers these DLL's as add-ons. By default, the system might not even have any add-ons. On the other hand, it will also allow various vendors to build other DLL's and include them as add-ons. Then each application requiring these Add-ons will be directed to the folder where to find them.
The application its self will have its own DLL's which will be in the same directory as the applications. But the Vendors' DLL's I would like to keep separate.
As mentioned in the answers below, my best bet would be to implement the DLL Import method, because A) I can specify a path for each DLL it's importing, B) I can better control the use of each DLL (Does it need to be loaded or not?) and C) Each DLL can technically be in separate folders by themselves (Vendors might want to build their own folder structure). This system is still very pre-mature but I plan to make further flexibility with it.
If you are dynamically loading the DLLs in your code, you can store them whereever you want since you have to pass the full path to LoadLibrary/Ex() anyway. If you are statically linking to the DLLs instead, then you can use SetDllDirectory() to designate an additional path to include in the OS's DLL search path.
You can do this with PATH but I recommend you don't. It's a brutal and inflexible approach. And of course you need to change the system wide PATH for it to have any effect at executable load time.
You can load your DLLs explicitly with LoadLibrary and GetProcAddress. That's no fun if there are a lot of imports but it can be a good option otherwise. And remember that if you go down this route, every single DLL must switch to explicit linking.
There is something called DLL Redirection but MS don't recommend you use that. They recommend that you use side-by-side components. Having said that, the Visual Studio team moved away from side-by-side components with the MSVC runtime in VS2010 because of the pain that side-by-side had caused in previous release.
So, in spite of all the options, I really believe that the best solution is to put all the DLLs in the same directory as the executable. If you can get over the folder looking untidy then it will make life much simpler. It is a trivial no effort solution to the problem.
Update
The update to your question provides the extra information that these DLLs are optional add-ons. In this case you simply have no alternative but to use explicit linking with LoadLibrary and GetProcAddress.
I would highly recommend that you leave the DLL's in the same folder as the applications.
If you really want to go down the road of putting the DLL's in a separate folder then you need to know whether you can load the DLL's with the LoadLibrary API which allows the specification of the path too. However, if the DLL's are statically loaded then it is Windows that performs the search. The Windows search first looks in the application folder then searches the Windows PATH. Also, as Delphi 7 only creates 32 bit applications this can get messy under Windows 64 bit.
On Windows, there is a "DLL search order". One of those search paths is The directory from which the application loaded, which is why it works to have them in the same folder as the EXE.
If you are statically linked to the DLLs, they must be loaded when the EXE is loaded into memory. This is before your first line of code is executed. So, you're relying on the DLLs being in one of the search paths. In this case, you're stuck with setting the path, and you must set it prior to the program loading.
If you are dynamically linking to the DLLs, then you can use LoadLibrary/LoadLibraryEx to load the DLLs at run time, in your code. Using those functions, you must specify the path to the DLL, so the DLLs could be anywhere. In this case, I feel that it's valid to put the DLLs in a separate folder to keep things tidy. As long as you don't put the DLLs into a shared location like the Windows System32 folder, you'll avoid a lot of headaches.
A temporarly solution is:
You can set your DLL path in your application's Shortcut (in "Start in" box).
I am currently working with 'inherited' code that has (scattered randomly throughout) a whole bunch of conditional compiler directives based on the version of Delphi, going back to Delphi 2 . From now on, all development will be with Delphi 2009 or future. Is there a tool in Delphi 2009 , or a plugin, that will automatically remove compiler conditional code segments based on a specified 'minimum' version?
I highly recommend the Delphi Inspiration Pascal Preprocessor (DIPP)
This can do a number of things with a source file in addition to removing conditional defines, including the "inlining" of include files and removing comments (all of course highly configurable and controllable by options supplied to the processor).
The conditional defines functionality is especially useful as you can either have all such conditionals simply removed or provide a set of DEFINE's that you wish to apply. DIPP will then emit a source file that reflects how it would appear to the compiler with those symbols DEFINED, but without the conditional directives themselves.
So in your case you would simply defined the symbols appropriate to your "baseline" Delphi version.
You should give JEDI PreProcessor (Pascal PreProcessor) in the JCL a try.
In the trunk in our SVN the source can be found in the dir jcl/devtools/jpp and in our latest release (2.1) zip-file the jpp.exe can be found in the dir devtools.
I don't know of such a tool but it should be relatively straight-forward to write it.
Loop through all files in the directory using FindFirst and FindNext
Use TStringList.LoadFromFile to read all pas files.
Loop through the strings and look for {$IfDef} directives. If the version specified in the conditional section is older than D2009 remove all text until the {$EndIf}.
Use TStringList.SaveToFile to write the modified file to disk.
My advice would be to ONLY change code you completely manage. If however you are also going to modify existing 3rd party code, then I suggest you go through each IFDEF defined for validation. Some vendors do not use the standard IFDEF VERxxx calls, but create their own which might be called something like VER70UP or such. The most common place for this would be in include files, so look for a {$I ???.INC} file near the beginning of each file, then analyze this for what is being used and how.
The other reason to analyze each $DEFINE/$UNDEF is the fact that a version specific one might turn on a new define that your previously not checking...one that ultimately leads into dead code.
Use GExperts, you can use both GrepSearch or GrepRegularExpressions to search in your code and then use the Replace tool in the GrepResults to remove whatever you need.
You can do a search and replace
operation on all of the matches in the
list or only the selected file/match.
When you choose one of those options,
a dialog appears prompting for the
string to use in place of the matched
text. Note that forms that are
currently open can not have their text
replaced, due to limitations in the
IDE. Please close all forms before
trying to replace text in them.