Consider the code:
procedure DoSmthSecret;
var
Seed: array[0..31] of Byte;
begin
// get random seed
..
// use the seed to do something secret
..
// erase the seed
FillChar(Seed, SizeOf(Seed), 0);
end;
The problem with the code is: FillChar is a compiler intrinsic, and potentially a compiler can "optimize it out". The problem is known for C/C++ compilers, see SecureZeroMemory. Can modern Pascal compiler (Delphi, FPC) do such optimization, and if they can, do they provide SecureZeroMemory equivalent?
FPC can't do such optimizations at the moment, and afaik even with C++ they belong into the "uncertain" class. (since the state of the program due to this optimization ignores what the programmer tells it to be)
Solving such problem is a matter of defining which constructs can be optimized out and which not. It doesn't need API/OS assistance per se, any externally linked object file with such function would do (since then global optimization wouldn't touch it)
Note that the article doesn't name the C++ compiler specifically, so I expect it is more a general utility function for when an user of a compiler gets into problems, without hitting the docs too hard, or when it must easily work on multiple (windows-only!) compilers without overly complicating the buildsystem.
Choosing a non inlinable API function might be non optimal in other cases, specially with small, constant sizes to zero, since it won't be inlined, so I would be careful with this function, and make sure there is a hard need
It might be important mainly when an external entity can change memory (DMA, memory mapping etc) of a program, or to erase passwords and other sensitive info from the memory image, even if the program according to the compiler will never read it
Even if FreePascal would optimize out writing to memory that is never read again (which I doubt it does atm, regardless of how long you guys discuss it), it does support the absolute type modifier which it guarantees (documented) to never optimize (somewhat similar to volatile in C/C++).
Related
In Delphi sane people use a class to define objects.
In Turbo Pascal for Windows we used object and today you can still use object to create an object.
The difference is that a object lives on the stack and a class lives on the heap.
And of course the object is depreciated.
Putting all that aside:
is there a benefit to be had, speed wise by using object instead of class?
I know that object is broken in Delphi 2009, but I've got a special use case1) where speed matters and I'm trying to find if using object will make my thing faster without making it buggy
This code base is in Delphi 7, but I may port it to Delphi 2007, haven't decided yet.
1) Conway's game of life
Long comment
Thanks all for pointing me in the right direction.
Let me explain a bit more. I'm trying to do a faster implementation of hashlife, see also here or here for simple sourcecode
The current record holder is golly, but golly uses a straight translation of Bill Gospher original lisp code (which is brilliant as an algorithm, but not optimized at the micro level at all). Hashlife enables you to calculate a generation in O(log(n)) time.
It does this by using a space/time trade off. And for this reason hashlife needs a lot of memory, gigabytes are not unheard of. In return you can calculate generation 2^128 (340282366920938463463374607431770000000) using generation 2^127 (170141183460469231731687303715880000000) in o(1) time.
Because hashlife needs to compute hashes for all sub-patterns that occur in a larger pattern, allocation of objects needs to be fast.
Here's the solution I've settled upon:
Allocation optimization
I allocate one big block of physical memory (user settable) lets say 512MB. Inside this blob I allocate what I call cheese stacks. This is a normal stack where I push and pop, but a pop can also be from the middle of the stack. If that happens I mark it on the free list (this is a normal stack). When pushing I check the free list first if nothing is free I push as normal. I'll be using records as advised it looks like the solution with the least amount of overhead.
Because of the way hashlife works, very little popping takes place and a lot of pushes. I keep separate stacks for structures of different sizes, making sure to keep memory access aligned on 4/8/16 byte boundaries.
Other optimizations
recursion removal
cache optimization
use of inline
precalculation of hashes (akin to rainbow tables)
detection of pathological cases and use of fall-back algorithm
use of GPU
For using normal OOP programming, you should always use the class kind. You'll have the most powerful object model in Delphi, including interface and generics (in later Delphi versions).
1. Records, pointers and objects
Records can be evil (slow hidden copy if you forgot to declare a parameter as const, record hidden slow cleanup code, a fillchar would make any string in record become a memory leak...), but they are sometimes very convenient to access a binary structure (e.g. some "smallish value"), via a pointer.
A dynamic array of tiny records (e.g. with one integer and one double field) will be much faster than a TList of small classes; with our TDynArray wrapper, you will have high-level access to the records, with serialization, sorting, hashing and such.
If using pointers, you must know what you are doing. It's definitively more preferable to stick with classes, and TPersistent if you want to use the magical "VCL component ownership model".
Inheritance is not allowed for records. You'll need either to use a "variant record" (using the case keyword in its type definition), either use nested records. When using C-like API, you'll sometimes have to use object-oriented structures. Using nested records or variant records is IMHO much less clear than the good old "object" inheritance model.
2. When to use object
But there are some places where objects are a good way of accessing already existing data.
Even the object model is better than the new record model, because it handles simple inheritance.
In a Blog entry last summer, I posted some possibilities to still use objects:
A memory mapped file, which I want to parse very quickly: a pointer to such an object is just great, and you still have methods at hand; I use this for TFileHeader or TFileInfo which map the .zip header, in SynZip.pas;
A Win32 structure, as defined by a API call, in which I put handy methods for easy access to the data (for that you may use record but if there is some object orientation in the struct - which is very common - you'll have to nest records, which is not the very handy);
A temporary structure defined on the stack, just used during a procedure: I use this for TZStream in SynZip.pas, or for our RTTI related classes, which map the Delphi generated RTTI in an Object-Oriented way not as the TypeInfo which is function/procedure oriented. By mapping the RTTI memory content directly, our code is faster than using the new RTTI classes created on the heap. We don't instanciate any memory, which, for an ORM framework like ours, is good for its speed. We need a lot of RTTI info, but we need it quick, we need it directly.
3. How object implementation is broken in modern Delphi
The fact that object is broken in modern Delphi is a shame, IMHO.
Normally, if you define a record on the stack, containing some reference-counted variables (like a string), it will be initialized by some compiler magic code, at the begin level of the method/function:
type TObj = object Int: integer; Str: string; end;
procedure Test;
var O: TObj
begin // here, an _InitializeRecord(#O,TypeInfo(TObj)) call is made
O.Str := 'test';
(...)
end; // here, a _FinalizeRecord(#O,TypeInfo(TObj)) call is made
Those _InitializeRecord and _FinalizeRecord will "prepare" then "release" the O.Str variable.
With Delphi 2010, I found out that sometimes, this _InitializeRecord() was not always made.
If the record has only some no public fields, the hidden calls are sometimes not generated by the compiler.
Just build the source again, and there will be...
The only solution I found out was using the record keyword instead of object.
So here is how the resulting code looks like:
/// used to store and retrieve Words in a sorted array
// - is defined either as an object either as a record, due to a bug
// in Delphi 2010 compiler (at least): this structure is not initialized
// if defined as a record on the stack, but will be as an object
TSortedWordArray = {$ifdef UNICODE}record{$else}object{$endif}
public
Values: TWordDynArray;
Count: integer;
/// add a value into the sorted array
// - return the index of the new inserted value into the Values[] array
// - return -(foundindex+1) if this value is already in the Values[] array
function Add(aValue: Word): PtrInt;
/// return the index if the supplied value in the Values[] array
// - return -1 if not found
function IndexOf(aValue: Word): PtrInt; {$ifdef HASINLINE}inline;{$endif}
end;
The {$ifdef UNICODE}record{$else}object{$endif} is awful... but the code generation error didn't occur since..
The resulting modifications in the source code are not huge, but a bit disappointing. I found out that older version of the IDE (e.g. Delphi 6/7) are not able to parse such declaration, so the class hierarchy will be broken in the editor... :(
Backward compatibility should include regression tests. A lot of Delphi users stay to this product because of the existing code. Breaking features are very problematic for the Delphi future, IMHO: if you have to rewrite a lot of code, which shouldn't you just switch the project to C# or Java?
Object was not the Delphi 1 method of setting up objects; it was the short-lived Turbo Pascal method of setting up objects, which was replaced by the Delphi TObject model in Delphi 1. It was kept around for backwards compatibility, but it should be avoided for a few reasons:
As you noted, it's broken in more recent versions. And AFAIK there are no plans to fix it.
It's a conceptualy wrong object model. The entire point of Object Oriented Programming, the one thing that really distinguishes it from procedural programming, is Liskov substitution (inheritance and polymorphism), and inheritance and value types don't mix.
You lose support for a lot of features that require TObject descendants.
If you really need value types that don't need to be dynamically allocated and initialized, you can use records instead. You can't inherit from them, but you can't do that very well with object either so you're not losing anything here.
As for the rest of the question, there aren't all that many speed benefits. The TObject model is plenty fast, especially if you're using the FastMM memory manager to speed up creation and destruction of objects, and if your objects contain lots of fields they can even be faster than records in a lot of cases, because they're passed by reference and don't have to be copied around for each function call.
When given a choice between "fast and possibly broken" and "fast and correct," always choose the latter.
Old-style objects offer no speed incentive over plain old records, so wherever you might be tempted to use old-style objects, you can use records instead without the risk of having uninitialized compiler-managed types or broken virtual methods. If your version of Delphi doesn't support records with methods, then just use standalone procedures instead.
Way back in older versions of Delphi which did not support records with methods then using object was the way to get your objects allocated on the stack. Very occasionally that would yield worthwhile performance benefits. Nowadays record is better. The only feature missing from record is the ability to inherit from another record.
You give up a lot when you change from class to record so only consider it if the performance benefits are overwhelming.
The goto statement is taboo at my work.
So the following question is born...
Is there a situation possible where a goto is the only valid solution?
Originally GOTO was added to Pascal for error handling, including inter procedural forms that Borland(/Embarcadero) never implemented (example: GOTOing from a inner procedure to the parent), just like Borland never implemented other inner function functionality like passing inner functions to procedure-typed parameters.(*)
In that way GOTO can be considered the precursor to exceptions.
There still some practical uses: The last time I checked, jumping out of a nested IF statement with goto was still faster in Delphi then letting the code exit from a nested if naturally.
Optimizations like these are sometimes used in e.g. compression code, and other complex tree processing code with deeply nested loops or conditional statements.
Such routines often still use goto for errorhandling, because it is faster. (exceptions are not only slow, but their border conditions inhibit some optimizations).
One could see this as part of the plain Pascal level of Object Pascal, just like C++ still allows plain C nearly completely.
(of course, since the optimized compression code in Delphi is only delivered in .o form, it is hard to find examples in the Delphi codebase. The JPEG code has some, but that is a C translation)
(*) Original pascal, and IIRC even Turbo Pascal doesn't allow prematurely exiting a procedure with EXIT. Same for CONTINUE and BREAK.
Is there a situation possible where a GOTO is the only valid solution?
I suppose it depends on what you mean by valid. I suppose you are asking if there exists a program that can only be written with the use of the goto statement. In which case the answer is that there is no such program. Delphi is Turing complete with or without the goto statement.
However, if we are prepared to widen the discussion to include other languages, there are situations where goto is a good solution, even the best solution. The scenario that most commonly comes to mind is implementing tidy-up and error handling in languages without structured exception handling. If you peruse the Linux source code you will find that goto is widely used. I expect that the same is true of the Windows source code.
Goto is very old. It predates sub-routines like functions and procedures! It is also very dangerous and can make your code less readable (to others, or to yourself a few months later).
In theory it's not possible to have a situation where goto is required. I won't repeat the theory about Turing tape machines here, but using selection and iteration, you can re-order the code so in all possible input values the same output comes about.
In practice though, it's sometimes 'handy' and 'better readable' to 'jump away' from the flow of code in certain conditions, and that's where Exceptions come in. raise breaks away from the current execution, and jump to the closest finally or except section. This is safer because they work cascaded, and provide a better way to handle the context in case of one of these border conditions. (And there's also breakand abort and exit)
GOTO is never necessary. Any computable algorithm can be expressed with assignment and the combination of IF...THEN, BEGIN...END, and your choice of WHILE...DO...END or REPEAT...UNTIL. You don't even need subroutines. :)
This is known as the structured program theorem.
For a proof, see the 1966 paper, Flow Diagrams, Turing Machines and Languages with Only Two Formation Rules (PDF) by Corrado Böhm and Giuseppe Jacopini.
Something like 15 years ago I used the goto statement in Delphi to convert one of Bob Jenkins's hash functions from C to Pascal. The C function has a switch() statement without breaks after each case, and you can't do that with Pascal's case statement. So I converted it into a bunch of Pascal labels and gotos. I guess you would still have to do it the same way with the newest Delphi versions.
Edit: I guess using gotos would still be a reasonable way to do this. Gets the job done, easy to understand, limited to a short block of code, not dangerous.
I have a colleague in my team which is extensively using closures in our projects developed in Delphi. Personal, I don't like this because is making code harder to read and I believe that closures should be used ONLY when you need them.
In the other hand I've read Can someone explain Anonymous methods to me? and other links related to this, and I'm taking into account that maybe I'm wrong, so I'm asking you to give me some examples when is better to use closures instead of a 'old-fashion' approach (not using closures).
I believe that this question calls for a very subjective judgement. I am an old-school delphi developer, and inclined to agree with you. Not only do closures add certain risks (as David H points out in comments) they also reduce readability for all classically trained Delphi developers. So why were they added to the language at all? In Delphi XE, the syntax-formatting function and closures weren't working well together, for example, and this increased my mistrust of closures; How much stuff gets added to the Delphi compiler, that the IDE hasn't been fully upgraded to support? You know you're a cranky old timer when you admit publically that you would have been happy if the Delphi language was frozen at the Delphi 7 level and never improved again. But Delphi is a living, powerful, evolving syntax. And that's a good thing. Repeat that to yourself when you find the old-crank taking over. Give it a try.
I can think of at least ten places where Anonymous methods really make sense and thus, reasons why you should use them, notwithstanding my earlier comment that I mistrust them. I will only point out the two that I have decided to personally use, and the limits that I place on myself when I use them:
Sort methods in container classes in the Generics.Collections accept an anonymous method so that you can easily provide a sorting bit of logic without having to write a regular (non-anonymous) function that matches the same signature that the sort method expects. The new generics syntax fits hand in hand with this style, and though it looks alien to you at first, it grows on you and becomes if not exactly really nice to use, at least more convenient than the alternatives.
TThread methods like Synchronize are overloaded, and in addition to supporting a single TThreadMethod as a parameter Thread.Synchronize(aClassMethodWithoutParameters), it has always been a source of pain to me, to get the parameters into that synchronize method. now you can use a closure (anonymous method), and pass the parameters in.
Limits that I recommend in writing anonymous methods:
A. I have a personal rule of thumb of only ONE closure per function, and whenever there is more than one, refactor out that bit of code to its own method. This keeps the cyclomatic complexity of your "methods" from going insane.
B. Also, inside each closure, I prefer to have only a single method invocation, and its parameters, and if I end up writing giant blocks of code, I rewrite those to be methods. Closures are for variable capture, not a carte-blanche for writing endlessly-twisted spaghetti code.
Sample sort:
var
aContainer:TList<TPair<String, Integer>>;
begin
aContainer.Sort(
TMyComparer.Construct(
function (const L, R: TPair<String, Integer>): integer
begin
result := SysUtils.CompareStr(L.Key,R.Key);
end ) {Construct end} ); {aContainer.Sort end}
end;
Update: one comment points to "language uglification", I believe that the uglification refers to the difference between having to write:
x.Sort(
TMyComparer.Construct(
function (const L, R: TPair<String, Integer>): integer
begin
result := SysUtils.CompareStr(L.Key,R.Key);
end ) );
Instead of, the following hypothetical duck-typed (or should I have said inferred types) syntax that I just invented here for comparison:
x.Sort( lambda( [L,R], [ SysUtils.CompareStr(L.Key,R.Key) ] ) )
Some other languages like Smalltalk, and Python can write lambdas more compactly because they are dynamically typed. The need for an IComparer, for example, as the type passed to a Sort() method in a container, is an example of complexity caused by the interface-flavor that strongly typed languages with generics have to follow in order to implement traits like ordering, required for sortability. I don't think there was a nice way to do this. Personally I hate seeing procedure, begin and end keywords inside a function invocation parenthesis, but I don't see what else could reasonably have been done.
I've ran Memory Validator on an application we're developing, and I've found that a Macro expressions we've defined is at the root of about 90% of the leaks. #define O_set.
Now, our macros are defined as follows:
#define O_SET_VALUE(ValueType, Value) boost::shared_ptr<ValueType>(new ValueType(Value))
.
.
#define O_set O_SET_VALUE
However, according to the Boost web site (at: http://www.boost.org/doc/libs/1_46_1/libs/smart_ptr/shared_ptr.htm):
A simple guideline that nearly
eliminates the possibility of memory
leaks is: always use a named smart
pointer variable to hold the result of
new. Every occurence of the new
keyword in the code should have the
form: shared_ptr p(new Y); It is,
of course, acceptable to use another
smart pointer in place of shared_ptr
above; having T and Y be the same
type, or passing arguments to Y's
constructor is also OK.
If you observe this guideline, it
naturally follows that you will have
no explicit deletes; try/catch
constructs will be rare.
This leads me to believe that this is indeed the major cause of our memory leaks. Or am I being naive or completely out of my depth here?
Question is, is there a way to work around the mentioned issue, with the above macro #defines?
Update:
I'm using them, for example, like this:
return O_set(int, 1);
_time_stamp(O_set(TO_DateTime, TO_DateTime())) (_time_stamp is a member of a certain class)
I'm working in Windows and used MemoryValidator for tracking the Memory Leaks - according to it there are leaks - as I state, the root of most of which (according to the stack traces) come down to that macro #define.
Smart pointers are tricky. The first thing I would do is to check your code for any 'new' statement which isn't inside either macro.
Then you have to think about how the pointers are being used; if you pass a smart pointer by reference, the reference counter isn't increased, for example.
Another thing to check is all instances of '.get()', which is a big problem if you are working with a legacy code base or other developers who don't understand the point of using smart pointers! (this is more to do with preventing random crashes than memory links persé, but worth checking)
Also, you might want to consider why you are using a macro for all smart pointer creation. Boost supply different smart pointers for different purposes. There isn't a one size fits all solution. Good old std::auto_ptr is fine for most uses, except storing in standard containers, but you knew that already.
The most obvious and overlooked aspect is, do you really need to 'new' something. C++ isn't Java, if you can avoid creating dynamic objects you are better off doing so.
If you are lucky enough to be working with a *NIX platform (you don't mention, sorry) then try the leak checking tool with Valgrind. It's very useful. There are similar tools available for windows, but often using you're software skilz is best.
Good luck.
can any one please say what is use of inline keyword in delphi
It is a hint to the compiler that the function/procedure should be (if possible) inlined, ie when you call it it should be expanded like a macro instead of being called.
This is an optimization for (very) small functions where the overhead of a call would be significant. You will find many examples in for example windows.pas
What actually happens depends on the complexity of the function, public/private access and your Delphi version.
It tells the compiler to generate code for the inline; routine on the place where it is called, instead of jumping to the routine and back.
For procedures that translate to very short assembler, this can be a benefit to performance, because the actual code is relatively short compared to the parameter preparation, the actual calling and the procedure prologue/epilogue.
If the procedure is too long, it can be a brake on performance though, and blow up your code gigantically. The "Auto" setting should make this decision for you, but in specific cases, you can locally set {$inline to on to force it. (e.g. for C macros translated to pascal functions, like the zlib functions to work with bitstreams )
Others have answered what inline does, but I just wanted to point out that there is a Compiler option to set inline on, off, or auto. Check out "Calling Procedures and Functions" in the D2009 docs for a very good explanation of the mechanics of inline. Here's the link to the online docs:
Delphi Online Docs for inline
It's borrowed from C in that it tells the compiler that this is a short routine that is frequently called and it recommends that the compiler treats the function as a macro and integrates the function code directly into the program at the point called rather than use a function call.
This gives faster code because there is no function call overhead, but at the expense of a larger program. Note too that like in C this is a recommendation to the compiler, it doesn't actually have to do so and the optimiser may override you.
Where to do this? Well like loop unwinding it's a technique that is very rarely of use nowadays. The classic place to use this is deep in a nested structure that is real-time critical, such as rendering code in a graphics application where a few machine calls shaved on each iteration can increase your screen refresh rate.
Press Ctrl + Alt + C (Entire CPU debug window) while debugging in Delphi, before call your inline function. And you will see that inline functions starts without "call" and wo jumping to another address. It's optimization feature.