Is there a difference between ThreadSanitizer suppression files and blacklist files? -- when used by the llvm-specific compiler flag of -fsanitize-blacklist=
When should I use one over the other?
ASFAIK blacklisted code doesn't get tsan instrumentation at all, while suppressed code is a runtime thing so the supressed code still get monitored by tsan but simply not reported, so that a report won't get polluted by known races.
This can make a performance difference.
Related
if I accidentally forget to switch to release configuration before releasing my program, does it matter in terms of it being easier to decompile or reverse engineer my code?
For example if I accidentally use the following debug compiler options:
1. Compiling:
- StackFrames = True
2. Debugging
- Debug information = Debug information
- Local Symbols = True
- Symbol Reference info = Reference Info
3. Linking
- Map File = Detailed
I have read help and from what I can tell it doesn't make much difference unless the map file is also somehow released with the binary file, so I wonder how much a difference it makes if someone has the map file?
Let's clarify one by one the options you've mentioned:
Compiling > Stack frames: Stack frames are only needed for debugging (and maybe to generate stack traces for error reporting, as mentioned by #DavidHeffernan in the comments). Even if you enable it in release builds, that won't be very helpful for reverse engineering.
Debugging > Debug information: With this option set, the debug information is compiled inside the DCUs to help debugging inside the IDE. It's not linked into the exe, so it's obvious that it won't help reverse engineering.
Debugging > Local symbols: With this option set, the compiler includes basic symbol information in the debug info, but again, it only helps when debugging on the IDE and it's not linked into the final exe.
Debugging > Symbol reference info > Reference info: Additionally to the previous option, this one includes detailed information about unit-local symbols (variables, constants, classes and so forth) to aid in debugging. They're also not linked into the final exe.
Linking > Map file > Detailed: With this option set, the linker will create a detailed .map file containing all the information (type, name, address, size, etc.) about program's symbols, so, of course it would be helpful for reverse engineering IF you distribute this file along with your exe (as stated by #RemyLebeau in the comments).
There's also the option to generate remote debug symbols, as pointed by #dummzeuch:
Linker > Include remote debug symbols: This option tells the linker to generate a .rsm file, it's the Delphi equivalent of Microsoft's .pdb Program Database Files. If you distribute this file, you could be on real trouble, because one could easily debug your application, visualize symbols, functions and procedures, single-step your code and so on.
Also, I think it's important to say that .map files are not equivalent to .pdb files. For Delphi Win32, .rsm is the equivalent. I have not worked with Delphi for years, but as far as I can remember, no Delphi Win32 version can generate .pdb files. Only Delphi for .NET can.
That said, let's go back to your questions:
I wonder how much a difference it makes if someone has the map file?
Reverse engineering would be much easier having a .map file. I've seen some tools in the past that can even convert a .map file to a .dbg file for use with a debugger.
Is it easier to reverse engineer or break a Delphi program if the exe file was compiled with debug compiler settings?
Well, one important (and maybe the most noticeable) characteristic of Debug builds is the bigger exe size. That's mainly because in the Debug configuration the compiler disables a number of code optimizations in order to facilitate code debugging. There's also a lot of debug-conditional code (eg.: inside {$IFDEF DEBUG} directives) that gets linked into the exe.
As a side effect, the code generated by a Debug build is much easier to reverse engineer because it's simpler to understand.
I have a DPK project which sometimes shows errors ("out of memory") while compiling (into Delphi IDE), and sometime it compiles with success.
But, when I try compile it through the MSBUILD, it always throws this error message:
error F2084: Internal Error: AV004A784A-WFFFD3764-1
Someone have an idea to solve it?
An internal error like that is a compiler error. We cannot solve them, only the compiler developer can do so. You need to submit a bug report to Embarcadero. Given the antiquity of XE3 you are unlikely to ever receive a fix.
Modern versions of Delphi are less prone to out of memory failures so if upgrading is an option then doing so will likely side-step the problem.
If upgrading is not an option then you are probably going to have to keep muddling along and cursing every time you encounter one of these errors.
This is actual very common (at least i experienced these kind of errors very often). I did go with the approach to remove source code files step by step from the DPK until compiling works. This way you also see which file is causing these errors, then go ahead and outcomment source code parts in this file step by step until you find the offending line, then rewrite it (in what way you need to find out yourself).
As David Heffernan already said, there is no way you can really solve it, because the error is internal to the compiler or linker, and only the developers of these can remove that bug.
But often, there are some workarounds.
In my experience, such internal errors are often related to generics and anonymous methods, or to inlined code, especially if two or all of these are combined.
So find out what code change you made before the internal error occurred. If you use a versioning system, or Delphi's internal _history system, restore to a previous version until the error goes away. Then you can do a difference with your "offending" code and see which code caused the internal error.
Often, too complex expressions (especially for the types I mentioned above) cause internal errors. If that is the case, try to simplify the expressions by calculating subexpressions first. Also, instead of using ad-hoc generic declarations like
x := TList<SomeType<Integer>>.Create;
you should try to use pre-defined types:
type
SomeTypeInteger = SomeType<Integer>;
SometypeList = TList<SomeTypeInteger>;
...
x := SometypeList.Create;
In other words,
try to find the "offending" code by going back in your history until it goes away
once you have found it, try to simplify that piece of code. Disentangle expressions and predefine types
experiment with different ways to express the code until the error goes away
If you do that, you may be successful in avoiding the internal error. It has often taken me some time, but I have always found a way to achieve what I wanted. If you couldn't, ask about specific errors here (by posting the offending code, the exact error and if possible a Minimal, Complete, and Verifiable example).
Fortunately, these errors have become very rare in the newest compilers.
For performance reasons an MVC app should have compilation debug='false' set in its web.config.
What benefits does having it set to 'true' during development give? I know that it 'inserts debugging symbols into the compiled page' but what is that for?
Check out this quite interesting link (very old, but still pertaining), which tries to explain in more detail the workings of either compilation mode, or, at least contrast between them well.
In an attempt to directly answer your question regarding what debugging symbols are used for - these give extra information about the compiled code and allow for interrogation of the executing code-base when a debugger is attached; allowing you to step through the source, for instance, as the binary can now be mapped back to it's location in the source file, variable names etc.
This is used for the code that is compiled on the fly (such as aspx and ascx pages) and will instruct the compiler how to compile the files.
Rest of the files such as models, global.asax, etc will be compiled to bin folder.
I'm producing builds using MSBuild, and build configurations set up in the dproj on the command line. It's slightly disconcerting that the size of the executables thus produced are different (not by much, but still!) to what an IDE build produces. Any ideas why? I would have thought the same compiler is used?
The main power of building from the Delphi command-line compiler is standardization - you explicitly identify the options (on the command line, in the .cfg files, etc), and the compiler follows the options provided exclusively. In contrast, the IDE has many other behaviors that are not clear and explicit - for example, it may search library paths not specified in the Project Options. My guess is that something's happening in the IDE build of which you're not entirely aware - and this is why standardized builds are done from the command line.
To see what IDE is doind, check
Tools | Options | Environment Options | Compiling and Running | Show Command Line
And you can check the compiler messages.
The first answer on using the command line for build consistency is right on and it is probably something you needn't worry about if you are relying on a build system where production files are always sourced from the console builds.
On the other hand, if you really do want to figure out what is going on you should turn on map files (at the full detail level) and compare/diff them. If there are differences between the two they will show up there. Any other differences that may exist are likely a result of a commmand line option being different (such as a conditional flag that may be set in the IDE settings).
This behavior has existed in every version of Delphi I've used. (5 - 2006). I wouldn't worry to much about it. When I first discovered it I spent a lot of time trying to resolve the difference. Did I miss a compiler flag? Is there a discrepancy between the IDE and the command line compiler's supported options?
In the end I decided it wasn't that big of an issue. Both consistently produced functionally equivalent executables.
If you supply exactly the same params to the command line compiler the produced executables will virtually be identical.
In fact the IDE just calls the commandline compiler. Compile your project in the IDE and look at the messages window. you will see the full dcc32.exe call ...
Building the same project (without any changes) produces binary different exe-files: some small regions of them are different. Empty project, version information (and auto-increment on every build) is turned off.
Why it happens? And is it possible to make delphi produce binary equal files for the same projects?
The various structures in the PE executable file format used by Windows include timestamps that are set by the compiler and linker.
It is possible to post-process the file to reset these values to a defined constant (I wrote a tool to do exactly this for a secure product that needed exact hash values), but this should only be done on ready-to-ship executables, as some debuggers rely on the timestamps for source lookup, etc.
Try changing the problem into "How do I avoid compiling if there are no changes to the source", might be easier to deal with.
I suspect compiler insert to *.exe encoded time, special ordinal numbers (for versioning) and maybe other things :)
It's impossible to force Delphi to produce equal binary output.
it may be, that some actual time-stamps are compiled into the exe-file.