I just wondered, why most Delphi examples use FillChar() to initialize records.
type
TFoo = record
i: Integer;
s: string; // not safe in record, better use PChar instead
end;
const
EmptyFoo: TFoo = (i: 0; s: '');
procedure Test;
var
Foo: TFoo;
s2: string;
begin
Foo := EmptyFoo; // initialize a record
// Danger code starts
FillChar(Foo, SizeOf(Foo), #0);
s2 := Copy("Leak Test", 1, MaxInt); // The refcount of the string buffer = 1
Foo.s = s2; // The refcount of s2 = 2
FillChar(Foo, SizeOf(Foo), #0); // The refcount is expected to be 1, but it is still 2
end;
// After exiting the procedure, the string buffer still has 1 reference. This string buffer is regarded as a memory leak.
Here (http://stanleyxu2005.blogspot.com/2008/01/potential-memory-leak-by-initializing.html) is my note on this topic. IMO, declare a constant with default value is a better way.
Historical reasons, mostly. FillChar() dates back to the Turbo Pascal days and was used for such purposes. The name is really a bit of a misnomer because while it says FillChar(), it is really FillByte(). The reason is that the last parameter can take a char or a byte. So FillChar(Foo, SizeOf(Foo), #0) and FillChar(Foo, SizeOf(Foo), 0) are equivalent. Another source of confusion is that as of Delphi 2009, FillChar still only fills bytes even though Char is equivalent to WideChar. While looking at the most common uses for FillChar in order to determine whether most folks use FillChar to actually fill memory with character data or just use it to initialize memory with some given byte value, we found that it was the latter case that dominated its use rather than the former. With that we decided to keep FillChar byte-centric.
It is true that clearing a record with FillChar that contains a field declared using one of the "managed" types (strings, Variant, Interface, dynamic arrays) can be unsafe if not used in the proper context. In the example you gave, however, it is actually safe to call FillChar on the locally declared record variable as long as it is the first thing you ever do to the record within that scope. The reason is that the compiler has generated code to initialize the string field in the record. This will have already set the string field to 0 (nil). Calling FillChar(Foo, SizeOf(Foo), 0) will just overwrite the whole record with 0 bytes, including the string field which is already 0. Using FillChar on the record variable after a value was assigned to the string field, is not recommended. Using your initialized constant technique is a very good solution this problem because the compiler can generate the proper code to ensure the existing record values are properly finalized during the assignment.
If you have Delphi 2009 and later, use the Default call to initialize a record.
Foo := Default(TFoo);
See David's answer to the question How to properly free records that contain various types in Delphi at once?.
Edit:
The advantage of using the Default(TSomeType) call, is that the record is finalized before it is cleared. No memory leaks and no explicit dangerous low level call to FillChar or ZeroMem. When the records are complex, perhaps containing nested records etc, the risk of making mistakes is eliminated.
Your method to initialize the records can be made even simpler:
const EmptyFoo : TFoo = ();
...
Foo := EmptyFoo; // Initialize Foo
Sometimes you want a parameter to have a non-default value, then do like this:
const PresetFoo : TFoo = (s : 'Non-Default'); // Only s has a non-default value
This will save some typing and the focus is set on the important stuff.
FillChar is fine to make sure you don't get any garbage in a new, uninitialized structure (record, buffer, arrray...).
It should not be used to "reset" the values without knowing what your are resetting.
No more than just writing MyObject := nil and expecting to avoid a memory leak.
In particulart all managed types are to be watched carefully.
See the Finalize function.
When you have the power to fiddle directly with the memory, there is always a way to shoot yourself in the foot.
FillChar is usually used to fill Arrays or records with only numeric types and array. You are correct that it shouldn't be used to when there are strings (or any ref-counted variables) in the record.
Although your suggestion of using a const to initialize it would work, an issue comes into play when I have a variable length array that I want to initialize.
The question may also be asking:
why FillChar
and not ZeroMemory?
There is no ZeroMemory function in Windows. In the header files (winbase.h) it is a macro that, in the C world, turns around and calls memset:
memset(Destination, 0, Length);
ZeroMemory is the language neutral term for "your platform's function that can be used to zero memory"
The Delphi equivalent of memset is FillChar.
Since Delphi doesn't have macros (and before the days of inlining), calling ZeroMemory meant you had to suffer the penalty of an extra function call before you actually got to FillChar.
So in many ways, calling FillChar is a performance micro-optimization - which no longer exists now that ZeroMemory is inlined:
procedure ZeroMemory(Destination: Pointer; Length: NativeUInt); inline;
Bonus Reading
Windows also contains the SecureZeroMemory function. It does the exact same thing as ZeroMemory. If it does the same thing as ZeroMemory, why does it exist?
Because some smart C/C++ compilers might recognize that setting memory to 0 before getting rid of the memory is a waste of time - and optimize away the call to ZeroMemory.
I don't think Delphi's compiler is as smart as many other compilers; so there's no need for a SecureFillChar.
Traditionally, a character is a single byte (no longer true for Delphi 2009), so using fillchar with a #0 would initalize the memory allocated so that it only contained nulls, or byte 0, or bin 00000000.
You should instead use the ZeroMemory function for compatibility, which has the same calling parameters as the old fillchar.
This question has a broader implication that has been in my mind for ages. I too, was brought up on using FillChar for records. This is nice because we often add new fields to the (data) record and of course FillChar( Rec, SizeOf( Rec), #0 ) takes care of such new fields. If we 'do it properly', we have to iterate through all fields of the record, some of which are enumerated types, some of which may be records themselves and the resulting code is less readable as well be possibly erroneous if we dont add new record fields to it diligently. String fields are common, thus FillChar is a no-no now. A few months ago, I went around and converted all my FillChars on records with string fields to iterated clearing, but I was not happy with the solution and wonder if there is a neat way of doing the 'Fill' on simply types (ordinal / float) and 'Finalize' on variants and strings?
Here is a better way to initialize stuff without using FillChar:
Record in record (Cannot initialize)
How to initialize a static array?
Related
I am using Delphi XE2 and I have to write a function which needs some constant arrays. If I declare these inside the function, when will they be created? Each time when the function runs or only once?
Example:
function Something;
const
Arr: array[0..100] of string = ('string1', 'string2'...);
begin
end;
The array will occupy a fixed, constant, region in your process's memory for the duration of the process's life.
In fact, you can easily see that yourself:
Each time the timer fires, you'll find that Pointer(#Arr) has the same value, and at that point in memory, you'll find your constant array.
And if you change it using the Memory panel (like string1 to orange1), the change will persist.
Just to elaborate a little more on Andreas Rejbrand's answer...
If you declare a local constant, it'll remain in compilation even if you declare the same values more than once.
These constants will be independent. In other words, if you write:
function Something;
const
Arr: array[0..100] of string = ('string1', 'string2'...);
begin
end;
function Otherhing;
const
Values: array[0..100] of string = ('string1', 'string2'...);
begin
end;
You'll get two sets of "string1" and "string2" in your compiled code.
If you change (in file or memory) one "string1" to "orange1" in one function, the other "string1" will remain the same.
There's no compiler optimization to "unify" those constants.
Aside from being a waste of code, if you have the same constant declared twice, it can become a mess if you change your code in one place and forget to check everywhere else.
In summary, if you have more than one place in your program where you'll use a constant, always consider declaring them global (or in the unit's scope) not locally.
I have a project that has intensive use of the case statement with many procedures coming off it. I know you can place case statements in a two tear arrangement divide in blocks of 10 and a second case statement to separate individual procedures. But I have a better idea if I can pull it off.
I want to call it assembly case
Prolist: array [1..500] of Pointer =
(#Procedure1, #Procedure2, #Procedure3, #Procedure4, #Procedure5);
Procedure ASMCase(Prolist: array of Pointer; No: Word; Var InRange: Boolean);
var Count : DWord;
PTR: Pointer;
Pro : Procedure;
begin
Count := No * 4;
InRange := boolean(Count <= SizeOf(Prolist));
If not InRange then Exit;
PTR := Pointer(DWord(#Prolist[1]) + Count);
If PTR <> nil then Pro := #PTR else Exit;
Pro; /run procedure
end;
The point is I'm creating a direct jump to the procedure.
In my case procedures can have an identical header and global data can be accessed for any odd information. Writing it in assembly would be faster I think but what I'm not sure on is running the procedures. Please do not ask why am I doing this as I have 500 procedures with many calls on the case statement and time is of essence with a fast processor.
It's expensive to pass that array by value. Pass it by const.
I can't see the point of the InRange flag and test. Don't pass out of range indices. And if you have to test, do it right. Don't use SizeOf which measures byte size. Use high or perhaps Length, if you have to. Which I doubt.
The pointer assignment test (PTR <> nil) is bogus. That condition always evaluates true. And the array indexing is very weird. What's wrong with []?
On top of that, your array is 1-based (usually a bad choice) but open arrays are always 0-based. Likely that's going to trip you up.
In short, I'd throw away all of that code. It's both wrong and needless. I'd just write it like this:
ProList[No]();
In order for this to compile your array would need to be defined as an array of procedural type rather than array of Pointer. Adding some type safety would be a good move.
It's pretty hard to see asm making much difference here. The compiler is going to emit optimal code.
If you are concerned with out of bound access, enable range checking in debug mode. Disable it for release if performance is paramount.
Bear in mind that global data structures don't tend to scale well as you add complexity. Most experienced programmers go to some length to avoid global state. Are you sure that global state is the right choice for you?
If you do need to improve performance, first identify opportunity for improvement. Reading from an array and calling a function are not likely candidates. Look at the procedures that you call. The bottlenecks are surely there.
One final point. Try to forget that you ever learn to use # with function pointers. Doing so yields an untyped pointer, of type Pointer that can be assigned to any pointer type. And thus you completely abandon type checking. Your procedure could have the wrong signature altogether and the compiler is not able to tell you. Declare your array of procedures with a type safe procedure type.
Let's go back to the basics. Frankly, I have never used New and Dispose functions before. However, after I read the New() documentation and the included examples on the Embarcadero Technologies's website and the Delphi Basics explanation of New(), it leaves questions in my head:
What are the advantages of using System.New() instead of a local variable, other than just spare a tiny amount of memory?
Common code examples for New() are more or less as follows:
var
pCustRec : ^TCustomer;
begin
New(pCustRec);
pCustRec^.Name := 'Her indoors';
pCustRec^.Age := 55;
Dispose(pCustRec);
end;
In what circumstances is the above code more appropriate than the code below?
var
CustRec : TCustomer;
begin
CustRec.Name := 'Her indoors';
CustRec.Age := 55;
end;
If you can use a local variable, do so. That's a rule with practically no exceptions. This results in the cleanest and most efficient code.
If you need to allocate on the heap, use dynamic arrays, GetMem or New. Use New when allocating a record.
Examples of being unable to use the stack include structures whose size are not known at compile time, or very large structures. But for records, which are the primary use case for New, these concerns seldom apply.
So, if you are faced with a choice of stack vs heap for a record, invariably the stack is the correct choice.
From a different perspective:
Both can suffer from buffer overflow and can be exploited.
If a local variable overflows, you get stack corruption.
If a heap variable overflows, you get heap corruption.
Some say that stack corruptions are easier to exploit than heap corruptions, but that is not true in general.
Note there are various mechanisms in operating systems, processor architectures, libraries and languages that try to help preventing these kinds of exploits.
For instance there is DEP (Data Execution Prevention), ASLR (Address Space Layout Randomization) and more are mentioned at Wikipedia.
A local static variable reserves space on the limited stack. Allocated memory is located on the heap, which is basically all memory available.
As mentioned, the stack space is limited, so you should avoid large local variables and also large parameters which are passed by value (absence of var/const in the parameter declaration).
A word on memory usage:
1. Simple types (integer, char, string, double etc.) are located directly on the stack. The amount of bytes used can be determined by the sizeof(variable) function.
2. The same applies to record variables and arrays.
3. Pointers and Objects require 4/8 bytes.
Every object (that is, class instances) is always allocated on the heap.
Value structures (simple numerical types, records containing only those types) can be allocated on the heap.
Dynamic arrays and strings content are always allocated on the heap. Only the reference pointer can be allocated on the stack. If you write:
function MyFunc;
var s: string;
...
Here, 4/8 bytes are allocated on the stack, but the string content (the text characters) will always be allocated on the heap.
So using New()/Dispose() is of poor benefit. If it contains no reference-counted types, you may use GetMem()/FreeMem() instead, since there is no internal pointer to set to zero.
The main drawback of New() or Dispose() is that if an exception occur, you need to use a try...finally block:
var
pCustRec : ^TCustomer;
begin
New(pCustRec);
try
pCustRec^.Name := 'Her indoors';
pCustRec^.Age := 55;
finally
Dispose(pCustRec);
end;
end;
Whereas allocating on the stack let the compiler do it for you, in an hidden manner:
var
CustRec : TCustomer;
begin // here a try... is generated
CustRec.Name := 'Her indoors';
CustRec.Age := 55;
end; // here a finally + CustRec cleaning is generated
That's why I almost never use New()/Dispose(), but allocate on stack, or even better within a class.
2
The usual case for heap allocation is when the object must outlive the function that created it:
It is being returned as a function result or via a var/out parameter, either directly or by returning some container.
It's being stored in some object, struct or collection that is passed in or otherwise accessible inside the procedure (this includes being signaled/queued off to another thread).
In cases of limited stack space you might prefer allocation from the heap.
Ref.
I've got an operation that's being called repeatedly in a loop. With a TRttiField:
if (field.name = '') or (field.Name[1] <> 'F') then
continue;
Profiling shows that I'm spending a lot of time in UStrAsg and UStrClr because of this. Field.Name has to make a virtual call to TRttiInstanceFieldEx.GetName, which has to perform a UTF8-to-string conversion on the underlying RTTI structure's name. This is happening twice per loop iteration.
I tried to cut all that out by bypassing the string conversions:
handle := PFieldExEntry(field.Handle);
if (handle.name = '') or (handle.Name[1] <> 'F') then
continue;
I expected to see about a 5% speed gain from this. Instead, the loop takes several seconds longer to execute, approximately 20-25% slower! I checked the generated ASM to make sure it wasn't doing anything screwy like making string copies from the RTTI structures to the local stack, but it isn't. I can't see any reason why this should have gotten slower. Anyone have any idea what might be going on here?
The field your new code reads is declared as a ShortString. As of Delphi 5, the compiler converts ShortStrings to long strings before generating the code for most string operations. (At least, that's the way it was with non-Unicode Delphi. Maybe Unicode Delphi restores some ShortString-related optimizations.)
Whereas the TRttiField wrapper might take advantage of the knowledge that it's populating a UTF-8 string with data that's already occupying one byte per character, I'd expect the ShortString-to-string code that your new loop employs might use a more general-purpose conversion routine, and you're paying the price for generality.
You might try foregoing string-conversion operations entirely. Instead, get a pointer to the first byte:
handle := PFieldExEntry(field.Handle);
NameP := PAnsiChar(#handle.name);
if (NameP[0] = #0) or (NameP[1] <> 'F') then
continue;
Note that although it's declared as a ShortString, it's not really one. It doesnn't really occupy 256 bytes. Instead, it occupies the minimum amount of memory required to hold its length byte and its characters.
I have been developing for some time now, and I have not used pointers in my development so far.
So what are the benefits of pointers? Does an application run faster or uses fewer resources?
Because I am sure that pointers are important, can you “point” me to some articles, basic but good to start using pointers in Delphi? Google gives me too many, too special results.
A pointer is a variable that points to a piece of memory. The advantages are:
you can give that piece of memory the size you want.
you only have to change a pointer to point to a different piece of memory which saves a lot of time copying.
Delphi uses a lot of hidden pointers. For example, if you are using:
var
myClass : TMyClass;
begin
myClass := TMyClass.Create;
myClass is a pointer to the object.
An other example is the dynamic array. This is also a pointer.
To understand more about pointers, you need to understand more about memory. Each piece of data can exist in different pieces of data.
For example global variables:
unit X;
interface
var
MyVar: Integer;
A global variable is defined in the datasegment. The datasegment is fixed. And during the lifetime of the program these variables are available. Which means the memory can not be used for other uses.
Local variables:
procedure Test;
var
MyVar: Integer;
A local variable exists on the stack. This is a piece of memory that is used for housekeeping. It contains the parameters for the function (ok some are put in a register but that is not important now). It contains the return adress so the cpu knows where to return if the program has ended. And it contains the local variables used in the functions.
Local variables only exists during the lifetime of a function. If the function is ended, you can't access the local variable in a reliable way.
Heap variables:
procedure Test2;
var
MyClass: TMyClass;
begin
MyClass := TMyClass.Create;
The variable MyClass is a pointer (which is a local variable that is defined on the stack). By constructing an object you allocate a piece of memory on the heap (the large piece of 'other' memory that is not used for programs and stacks). The variable MyClass contains the address of this piece of memory.
Heap variables exist until you release them. That means that if you exit the funcion Test2 without freeing the object, the object still exists on the heap. But you won't be able to access it because the address (variable MyClass) is gone.
Best practices
It is almost always preferably to allocate and deallocate a pointer variable at the same level.
For example:
var
myClass: TMyClass;
begin
myClass := TMyClass.Create;
try
DoSomething(myClass);
DoSomeOtherthing(myClass);
finally
myClass.Free;
end;
end;
If you can, try to avoid functions that return an instance of an object. It is never certain if the caller needs to dispose of the object. And this creates memory leaks or crashes.
You have been given a lot of good answers so far, but starting with the answer that you are already dealing with pointers when you use long strings, dynamic arrays and object references you should start to wonder why you would use pointers, instead of long strings, dynamic arrays and object references. Is there any reason to still use pointers, given that Delphi does a good job hiding them from you, in many cases?
Let me give you two examples of pointer use in Delphi. You will see that this is probably not at all relevant for you if you mostly write business apps. It can however become important if you ever need to use Windows or third party API functions that are not imported by any of the standard Delphi units, and for which no import units in (for example) the JEDI libraries can be found. And it may be the key to achieve that necessary last bit of speed in string-processing code.
Pointers can be used to deal with data types of varying sizes (unknown at compile time)
Consider the Windows bitmap data type. Each image can have different width and height, and there are different formats ranging from black and white (1 bit per pixel) over 2^4, 2^8, 2^16, 2^24 or even 2^32 gray values or colours. That means that it is unknown at compile time how much memory a bitmap will occupy.
In windows.pas there is the TBitmapInfo type:
type
PBitmapInfo = ^TBitmapInfo;
tagBITMAPINFO = packed record
bmiHeader: TBitmapInfoHeader;
bmiColors: array[0..0] of TRGBQuad;
end;
TBitmapInfo = tagBITMAPINFO;
The TRGBQuad element describes a single pixel, but the bitmap does of course contain more than one pixel. Therefore one would never use a local variable of type TBitmapInfo, but always a pointer to it:
var
BmpInfo: PBitmapInfo;
begin
// some other code determines width and height...
...
BmpInfo := AllocMem(SizeOf(TBitmapInfoHeader)
+ BmpWidth * BmpHeight * SizeOf(TRGBQuad));
...
end;
Now using the pointer you can access all pixels, even though TBitmapInfo does only have a single one. Note that for such code you have to disable range checking.
Stuff like that can of course also be handled with the TMemoryStream class, which is basically a friendly wrapper around a pointer to a block of memory.
And of course it is much easier to simply create a TBitmap and assign its width, height and pixel format. To state it again, the Delphi VCL does eliminate most cases where pointers would otherwise be necessary.
Pointers to characters can be used to speed up string operations
This is, like most micro optimizations, something to be used only in extreme cases, after you have profiled and found the code using strings to consume much time.
A nice property of strings is that they are reference-counted. Copying them does not copy the memory they occupy, it only increases the reference count instead. Only when the code tries to modify a string which has a reference count greater than 1 will the memory be copied, to create a string with a reference count of 1, which can then safely be modified.
A not-so-nice property of strings is that they are reference-counted. Every operation that could possibly modify the string has to make sure that the reference count is 1, because otherwise modifications to the string would be dangerous. Replacing a character in a string is such a modification. To make sure that the reference count is 1 a call to UniqueString() is added by the compiler whenever a character in a string is written to. Now writing n characters of a string in a loop will cause UniqueString() to be called n times, even though after the first time is is assured that the reference count is 1. This means basically n - 1 calls of UniqueString() are performed unnecessarily.
Using a pointer to the characters is a common way to speed up string operations that involve loops. Imagine you want (for display purposes) to replace all spaces in a string with a small dot. Use the CPU view of the debugger and compare the code executed for this code
procedure MakeSpacesVisible(const AValue: AnsiString): AnsiString;
var
i: integer;
begin
Result := AValue;
for i := 1 to Length(Result) do begin
if Result[i] = ' ' then
Result[i] := $B7;
end;
end;
with this code
procedure MakeSpacesVisible(const AValue: AnsiString): AnsiString;
var
P: PAnsiChar;
begin
Result := AValue;
P := PAnsiChar(Result);
while P[0] <> #0 do begin
if P[0] = ' ' then
P[0] := $B7;
Inc(P);
end;
end;
In the second function there will be only one call to UniqueString(), when the address of the first string character is assigned to the char pointer.
You probably have used pointers, but you just don't know it. A class variable is a pointer, a string is a pointer, a dynamic array is a pointer, Delphi just hides it for you. You will see them when you are performing API calls (casting strings to PChar), but even then Delphi can hide a lot.
See Gamecats answer for advantages of pointers.
In this About.com article you can find a basic explanation of pointers in Delphi.
Pointers are necessary for some data structures. The simplest example is a linked list. The advantage of such structures is that you can recombine elements without moving them in memory. For example you can have a linked list of large complex objects and swap any two of them very quickly because you really have to adjust two pointers instead of moving this objects.
This applies to many languages including Object Pascal (Delphi).