Techniques and tools for debugging problems on remote machines? - delphi

Question

Users have been reporting problems/crashes/bugs that I can't reproduce on my machine. I'm finding these problems difficult to fix.
I've started using EurekaLog (fantastic!) and SmartInspect. Both these tools have helped greatly but I'm still finding it difficult to catch some problems.
I've just purchased Debugging by David Agans (and waiting for it to arrive).
Are there any other tools or techniques specific to Delphi that will help with catching these hard to find remote problems? The kinds of problems I'm finding difficult to track down are those that don't raise exceptions or have a clear cause. EurekaLog catches exceptions and SmartInspect is pretty good once I have a theory to check. But in some cases it is a seemingly random crash and there are several thousand lines of code that may may be at fault. How to narrow down to the root cause?

Answer 1

MadExcept is what I use, and it is fabulous. I have also used EurekaLog and find the functionality almost exactly identical, except that I have more experience and time using MadExcept. it's free for non-commercial use, and reasonably priced for commercial use.
Update: MadExcept 4 is now out and even supports 64 bit Delphi XE2 apps, and has memory-leak checking too.
When nothing blows up, I rely on heavy use of trace logging. I have a TraceMessage(integer,string) function which I call throughout all my apps, and when someone has problems I get them to click a menu item that turns up the debug trace level to the most verbose level; it gives me a complete history of everything my application did, and this has helped me even more than madExcept, to solve problems at customer sites. Customers get a crash, and that crash report sent by madexcept contains a log file (created by my app) that is attached automatically. I believe you can do this equally well with madExcept and EurekaLog. If you need a logging system you could download Log4D, or you could write your own, it's pretty simple.
For always-free, try JclDebug, which requires more work to set up, but which has worked fabulously for me, also.
For help with heap problems, learn more about fastMM (full version) debug options.
And you shouldn't forget that Delphi itself supports Remote debugging, if you can reproduce a crash on machines in your office that don't have delphi installed, use remote debug across the office network instead of installing a complete RAD Studio installation on that other machine at your work. You could also use remote debug to connect to a client PC computer across the internet, but I have not tried remote debug across the internet yet, so I can't say whether it works great over the internet or not. I do know that since remote debug doesn't support automatic deploy of the EXE file you built (you have to do that part yourself), remote debug over internet, to a client PC is more work.
You might also find lots of your problems by fixing all your hints and warnings, and then going through with CodeHealer or Pascal Analyzer (PAL) from Peganza. These static analysis tools can help you find real code problems.
If performance and memory usage are your problems, get the full version of AQTime, and use it to profile and watch your system operate. It will help you fix your memory leaks, and understand your app's runtime behaviour and memory usage, not just leaks but bottlenecks for memory and CPU usage. Some of those bottlenecks can also be the source of some odd problems. I have even used AQTime to help me find deadlocks, since it can generate traces of execution, that can help me figure out what code is running, and locate deadlocks. Update: AQTime is not installable on machines other than your main dev machine, without violating the newly modified license terms for AQTime. These terms were never this restrictive in the good old days.
If you gave more exact idea of what your problems are, I'm sure other people could give you some more ideas that are specific, but all of the above are general techniques that have served me well.

Answer 2

One of the best way is to use the Remote Debugger that comes with Delphi, so you can debug directly the application running on the remote machine. THe remote debugger is somewhat buggy in some Delphi releases, and requires to follow the instructions carefully to make it working, but when needed it's a tool to consider. Also check if there are updates available for your version, they could come in a separate installer for deployment on "remote" systems. Otherwise first install the remote debugger, than check if the files installed has newer versions in your local installation, and the copy tehm on the remote machine.

Answer 3

CodeSite has helped me a lot in these situations. Since XE it is bundled with Delphi.

Answer 4

Logging is the key, in this matter.
Take a look at our TSynLog class available in our Open Source SynCommons library.
It does have the JCL Debug / MadExcept features, with some additional (like customer-side profiling, and logging):
logging with a set of levels;
fast, low execution overhead;
can load .map file symbols to be used in logging;
compression of .map into binary .mab (900 KB -> 70 KB);
inclusion of the .map/.mab into the .exe;
reading of an external .map to add unit names and line numbers to a log file without .map available information at execution;
exception logging (Delphi or low-level exceptions) with unit names and line numbers;
optional stack trace with units and line numbers;
methods or procedure recursive tracing, with Enter and auto-Leave using interfaces;
high resolution time stamps, for customer-side profiling of the application execution;
set / enumerates / TList / TPersistent / TObjectList / TContainer / dynamic array JSON serialization;
per-thread or global logging;
multiple log files on the same process;
integrated log archival (in zip or any other format);
Open Source, works from Delphi 5 up to XE.