The Delphi Linker strips out any functions that aren't actually used, thus reducing the executable size.
Is there any way to stop the Delphi Linker doing this? e.g. a compiler switch?
To those wondering "why?"...
I am trying to use the delphi-code-coverage tool, but it only reports on code that is actually compiled into the executable. Which makes it not very useful. If I could get Delphi to include all code, I'm hoping I could then get some useful code coverage statistics.
I should mention that I have DUnit tests in a separate project to my application. So even though the code is "unused" in the DUnit project, it is used in the actual application.
See here for more details.
Your code-coverage tool is measuring the wrong thing. It works off the map file instead of the source code, so it will only report on live code instead of on all code in a project. The linker already filters out the dead code, and in a blank unit-test project, all code is dead code. There is no way to tell Delphi to include dead code in an EXE.
Run the code-coverage tool on your application to get a list of functions that needs testing. Then, write code in your unit-test project that mentions all those functions. (It doesn't have to call everything yet, and it certainly doesn't have to test it all. We're just making sure it's linked to the unit-test project.) Now the coverage tool can get an accurate measurement of what's been executed and what hasn't.
Related
I have read an article from Xamarin, and came across a particular computer science word : Ahead of Time.
According to some google search result, this AOT does not allow for code generation during run time.
Does it mean, it does not support dynamic stuff?
I know this question may stupid and I have 0 knowledge in IOS, hopefully can get some answer from here. thanks
First, what is the definition of dynamic? For the general public, dynamic code mean the application can change functionality at run-time. For the iOS platform, the binaries are signed to prevent malware. And Apple don't like apps that can load functionality at run-time.
An ahead-of-time (AOT) compiler has nothing to do with dynamic code per se. It's has to do with intermediate language that are just-in-time compilation (JIT). The biggest example of intermediate language is Java bytecode; compile once, run anywhere. When a Java application is executing, the compiled code is JIT to native machine code. AOT compiler is just doing it ahead of time, to save time.
For the iOS platform, Xcode compiles Objective-C to a native binary for the device.
Another way of looking at this is with an example...
In .Net, you can use the Reflection.Emit namespace to generate and compile code at runtime.
Eg you could create an "IDE" with a textbox that accepts C#. When you click a button that C# could be compiled by the .Net framework into a custom library that's loaded dynamically or a fully-fledged executable which is launched as a new process.
This is insanely powerful when combined with the rest of the System.Reflection namespace. You can examine objects at runtime and compile custom code based on any criteria you like.
That said... The problems usually outweigh the benefits in most cases. Mainly, it's a massive security issue, especially when running on a consumer device.
It would be possible to create an app that wouldn't have anything close to malicious code, get it audited by apple, then have the app download code from your webserver, compile it, and execute it. This new code wouldn't be audited...
There really is no good reason to be doing this in a consumer app.
We're trying to untangle a hairball of 100's of units, removing some.
It would be helpful if there was tool that would show us what units were explicitly using unit X.
Penganza doesn't seem to have a report that does that. (Although it has lots of other useful reports.)
Can anyone suggest a tool or strategy for doing this, other than just hiding unit x and then hitting F9 ... repeatedly?
MMX (Model Maker Code Explorer) has a nice unit dependency analyzer (it is especially good at detecting cycles).
For more details, see this answer.
--jeroen
From a similar question here
You might want to take a look at at
CnPack.
CnPack includes a Uses cleaner
wizard wich hasn't failed me yet.
GExperts can show Project Dependencies.
Peganza Pascal Analyzer can do the work. I haven't worked with it much, but a former dev here wrote a system that uses PAL to do the analysis, then dumped the results into a database, and then there's a browser app that lets you enter a unit name and it returns the list of units affected, whether they would need to be rebuilt if the unit changed, or if the interface changed. We use lots of BPLs so you can sometimes change a unit and you don't have to re-build other binaries that use your unit, unless the interface changed. This saves us lots of work (hundreds of BPLs and EXEs).
Chris
Headway Software's Structure 101g (and Restructure 101g) can do that really well, with the Delphi plugin.
Disclaimer: I wrote the flavors to analyze Delphi. I use them professionally, helping clients.
We've just released a freeware utility that does exactly what you need plus quite a bit more. It's called the Delphi Unit Dependency Scanner (DUDs) and you can download it here: http://www.easy-ip.net/delphi-unit-dependency-scanner.html
Sorry it's a bit late!
I was going to mention Icarus, but when I googled them I got this stack overflow answer, which you might want to check out.
Then again, sometimes I just like to delete my whole Unit Output Directory, then count my new DCU's, and that works too.
The reason you may like Icarus and not GExperts is that it doesn't rely on you to have properly maintained the uses statements in your project file.
A newcomer in this field is the Delphi Plugin for Sonar. It does not list unit dependencies but can find unused files and "dead" code (and more).
Implemented features:
Counting lines of code, statements, number of files
Counting number of classes, number of packages, methods, accessors
Counting number of public API (methods, classes and fields)
Counting comments ratio, comment lines (including blank lines)
CPD (code duplication, how many lines, block and in how many files)
Code Complexity (per method, class, file; complexity distribution
over methods, classes and files)
LCOM4 and RFC
Code colorization
Unit tests reports
Assembler syntax in grammar
Include statement
Parsing preprocessor statements
Rules
Code coverage reports
Source code highlight for unit tests
“Dead” code recognition
Unused files recognition
I'm using TJvPluginManager in the JVCL to create and load BPL-based plugins for my program. Problem is, one of the plugins isn't loading properly, and I can't debug it. Every time I try to trace into the loading sequence, it gets as far as the LoadLibrary API call, and then the debugger seems to forget what it's there for. It completely loses the ability to associate program code with source lines, give meaningful data in a call stack, or display local variables. It will still stop at breakpoints, but it breaks to the CPU window, with all the inline source code stripped out.
This happens on Delphi 2007 and 2009, and it's driving me nuts. Does anyone know how to load a plugin without it breaking the debugger? Does anyone even know why it's breaking it in the first place?
NOTE: I'm not looking for alternative methods of debugging. I know all about tracing and logging and all the rest. What I want is to understand what's going wrong and how to fix it. Surely I'm not the only person who's ever used TJvPluginManager?
Not quite the answer to your question: Have you tried to debug the package project, by setting the host application and putting a breakpoint into the package's startup code?
I've found Ray Kanopka's (Raize) CodeSite to be invaluable for debugging in situations where the integrated debugger is acting up. Thinking about the things I want to monitor using CodeSite actually helps me focus on what's important - it enforces good habits.
Another alternative to Codesite is Overseer which is part of the nexus project, but stands alone so does not require you to use their framework. Codesite is by far the better option, but in a pinch Overseer would work just as well.
I found that using packages for plugins can be problematic and many years ago switched to a completely COM based implementation for plugins and never had any problems. The other advantage to COM based plugins, they don't require Delphi to write, do not need to be recompiled when the main app switches to a new version of the compiler (my plugins compiled with Delphi 5 still run fine against the main application compiled in Delphi 2009!) and they are easier to write test applications to assist in debugging.
The only side effect I notice, is that shared code ends up in both executables and the plugins require registration into the registry.
Hmmmm... This is a stupid question, but I have to ask: the initialization function have the EXACT declaration syntax like the other plugins that work ?(from your question, I deducted you made some others that work)
Check your dependencies. Make sure each unit is compiled into one package only. Whenever a package needs to reference a unit from another package, use the requires clause to do so. Watch for compiler warnings about implicitly linked units.
Our team had been using Delphi 6 for many years, then switched to Delphi 2006 years ago. With both versions we have the following problem: frequently the compiler complains about a unit which is supposedly used recursively. This unit is a 40k LOC unit which is at the core of a project with almost 1 million LOC (third party included).
The error message is incorrect: a full build on the project always works. Unfortunately, the error message does not tell us where the supposed circular reference is, just the name of that unit. Sometimes it even happens that valid error messages are listed 2-4 times until that circular reference problem is "found". Clearly the compiler is running in a circle here. Because of the size of that project it is hard to find the problem manually. Therefore I made a tool which proves that there really is no circular reference (the tool creates a directed dependency graph of the units and determines the coherence components in that graph - there are none except if I deliberately put some in).
This is not only affecting F9 compilation but also code completion / insight which is not working most of the time. Sometimes it works when I press ctrl-space a second time...
Any ideas how we can isolate or even fix the problem? Note that it will be very hard to split the 40k LOC unit into smaller ones because it contains about 15 large classes which depend on each other in the interface section (I know it's bad but should work anyway).
Update
We are constantly refactoring but this is one tough unit to refactor because everything depends on everything, almost. Have been trying to get around it via interfaces but we are talking about some classes with 100s of methods and properties. And it would be slower.
Upgrading to D2009 may be an option down the road but right now we're stuck with D2006 (the unicode stuff and the price tag are two of the stoppers here). Question is anyway if it would help since the problem is in there since D6 at least.
About trimming the uses clauses, we have been doing this frequently with Icarus. But that did not help so far. We are down to 90 custom units in the interface section now. However, with a true circular reference the problem could be in any unit. Also tried to add all units to the dpr.
The project shares a lot of code with other projects, and there are some IFDEFs. However, the defines are not set up in project options but via a common include file. Therefore all modules should see the same defines. Also, the problem reoccurs shortly after a full rebuild without switching to another project.
I will probably be downvoted for this. In D2005 I had a 10k loc unit (datamodule) that flat out stopped compiling. Had to separate out some datasets/code to another datamodule. That 10k unit was and is a mess. You really should consider refactoring out some code to other units. My module has since D2005 / separation grown even worse, but it still compiles in D2007. So my answer is a) refactor and b) upgrade to D2009.
It seems clear that this is due to a slight difference between the background compiler and the real thing. You could look around (QualityCentral) what's known on that topic.
Also, since you didn't explicitly state this, you should remove unnecessary units and move uses statements down to implementation if possible. Maybe your tool could help with this.
And just to be sure you should check the unit aliases and Path settings.
You write that a full build does always succeed, but shortly after the incremental build fails with this error. Assuming that you experience this in the IDE, have you tried to use the command line compiler dcc32 to do incremental builds?
If you don't feed it the "-Q" switch (which probably most Makefiles or scripts for command line builds do) it will output a lot of information what files it compiles in what order. You could either try to do an incremental build after the error appeared in the IDE, or you could keep a command line open next to the IDE and Alt+Tab to it for compilation, skipping compilation in the IDE completely.
I simply assume you have a way to build using dcc32, one way or another - with the size of your project I can't imagine otherwise.
We regularly fall in similar problems, and we never managed (or bothered long enough) to find the precise cause. There seems to be a problem in the order which Delphi chooses to compile the units when hitting Ctrl-F9, which is incompatible with the actual dependency order of the units.
Have you tried deleting "MyBigFatUnit.dcu" before hitting Ctrl-F9?
Have you tried to re-order the declaration of your units in your dpr/dpk files, so that units appear in a correct compilation order? (i.e.: if unit B depends on unit A, unit A should appear first in the dpr/dpk)
Do you have any other projects that use part of the same codebase? If you compile one of them under different compiler settings or IFDEFs, it might change certain things in some of the DCUs which would lead to a circular dependency. A full build rebuilds all DCUs and then the problem goes away.
Try Icarus (free) from Peganza. If that does not tell you what the problem is, try their Pascal Analyzer.
We have this problem as well, also with a fairly large codebase.
We are currently using D2009, but have had this problem with all previous versions of Delphi.
It most frequently happens immediately after doing an update from source control, so I suspect there is some timestamp issue within the Delphi build process.
In our case, if Ctrl-F9 fails and reports the circular reference, a second Ctrl-F9 will generally work
A way I have been told to deal with this is to open another arbitrary file in the project, change that file, save it, and then try running the incremental compile again. Surprisingly enough, this usually works.
We have a 4 MLOC project where this comes up from time to time and this "solution" works for me.
I've fought this before, in my experience the error is quasi-legitimate. It's been a quite a while (the error has nothing to do with the version) but my memory of the situation is that it involves a loop in which part of the loop is in the implementation.
Unit A uses B in the implementation. Unit B uses A in the interface. If you compile B first it calls for A but since the call for B is in the implementation it succeeds. If you compile A first it calls for B, B turns around and calls for A in the interface, boom. Such loops are only safe if both cross references are in the implementation.
The solution is to design things so there is a minimum of stuff used in the interface and to make certain there's nothing resembling a loop in those units. So long as you keep your type definitions separate from units with code this is pretty easy to do.
The error coming and going depending on what you are doing is a hallmark of this issue as it comes down to how you enter the loop. When you do a full build the order is consistent and you either get it 100% or 0%, it's not random.
Is there any way to measure code coverage with DUnit? Or are there any free tools accomplishing that? What do you use for that? What code coverage do you usually go for?
Jim McKeeth: Thanks for the detailed answer. I am talking about unit testing in the sense of a TDD approach, not only about unit tests after a failure occured. I'm interested in the code coverage I can achieve with some basic prewritten unit tests.
I have just created a new open source project on Google Code with a basic code coverage tool for Delphi 2010. https://sourceforge.net/projects/delphicodecoverage/
Right now it can measure line coverage but I'm planning to add class and method coverage too.
It generates html reports with a summary as well as marked up source showing you what lines are covered (green), which were not (red) and the rest of the lines that didn't have any code generated for them.
Update:
As of version 0.3 of Delphi Code Coverage you can generate XML reports compatible with the Hudson EMMA plugin to display code coverage trends within Hudson.
Update:
Version 0.5 brings bug fixes, increased configurability and cleaned up reports
Update:
Version 1.0 brings support for emma output, coverage of classes and methods and coverage of DLLs and BPLs
I don't know of any free tools. AQTime is almost the defacto standard for profiling Delphi. I haven't used it, but a quick search found Discover for Delphi, which is now open source, but just does code coverage.
Either of these tools should give you an idea of how much code coverage your unit tests are getting.
Are you referring to code coverage from unit tests or stale code? Generally I think only testable code that has a failure should be covered with a unit test (yes I realize that may be starting a holy war, but that is where I stand). So that would be a pretty low percentage.
Now stale code on the other hand is a different story. Stale code is code that doesn't get used. You most likely don't need a tool to tell you this for a lot of your code, just look for the little Blue Dots after you compile in Delphi. Anything without a blue dot is stale. Generally if code is not being used then it should be removed. So that would be 100% code coverage.
There are other scenarios for stale code, like if you have special code to handle if the date ever lands on the 31st of February. The compiler doesn't know it can't happen, so it compiles it in and gives it a blue dot. Now you can write a unit test for that, and test it and it might work, but then you just wasted your time a second time (first for writing the code, second for testing it).
There are tools to track what code paths get used when the program runs, but that is only simi-reliable since not all code paths will get used every time. Like that special code you have to handle leap year, it will only run every four years. So if you take it out then your program will be broken every four years.
I guess I didn't really answer your question about DUnit and Code Coverage, but I think I may have left you with more questions then you started with. What kind of code coverage are you looking for?
UPDATE: If you are taking a TDD approach then no code is written until you write a test for it, so by nature you have 100 test coverage. Of course just because each method is exercised by a test does not mean that its entire range of behaviors is exercised. SmartInspect provides a really easy method to measure which methods are called along with timing, etc. It is a little less then AQTime, but not free. With some more work on your part you can add instrumentation to measure every code path (branches of "if" statements, etc.) Of course you can also just add your own logging to your methods to achieve a coverage report, and that is free (well, expect for your time, which is probably worth more then the tools). If you use JEDI Debug then you can get a call stack too.
TDD really cannot easily be applied retroactively to existing code without a lot of refactoring. Although the newer Delphi IDEs have the ability to generate unit test stubs for each public method, which then gives you 100% coverage of your public methods. What you put in those stubs determines how effective that coverage is.
I use Discover for Delphi and it does the work, for unit testing with DUnit and Functional testing with TestComplete.
Discover can be configured to run from the command line for automation.
As in:
Discover.exe Project.dpr -s -c -m
Discover works great for me. It hardly slows down your application, unlike AQTime. This may not be a problem for you anyway, of course. I think the recent versions of AQTime perform better in this respect.
I've been using Discover" for years, worked excellently up to and including BDS2006 (which was the last pre-XE* version of Delphi i used and still use), but its current opensourced state, it's unclear how to make it work with XE* versions of Delphi. A shame really, because I loved this tool, fast and convenient in almost every way.
So now I'm moving to delphi-code-coverage...