Here I have a few lines of delphi code running in delphi 7:
var
ptr:Pointer ;
begin
ptr:=AllocMem(40);
ptr:=Pchar('OneNationUnderGod');
if ptr<>nil then
FreeMem(ptr);
end;
Upon running this code snippet, FreeMem(ptr)will raise an error:'invalid pointer operation'. If I delete the sentence:
ptr:=Pchar('OneNationUnderGod');
then no error will occur. Now I have two questions,
1.Why is this happening? 2. If I have to use the Pchar sentence, how should I free the memory allocated earlier?
Much appreciation for your help!
The problem is that you are modifying the address held in the variable ptr.
You call AllocMem to allocate a buffer, which you refer to using ptr. That much is fine. But you must never change the value of ptr, the address of the buffer. And you do change it.
You wrote:
ptr:=AllocMem(40);
ptr:=Pchar('OneNationUnderGod');
The second line is the problem. You have modified ptr and now ptr refers to something else (a string literal held in read-only memory as it happens). You have now lost track of the buffer allocated by your call to AllocMem. You asked AllocMem for a new block of memory and then immediately discarded that block of memory.
What you presumably mean to do is to copy the string. Perhaps like this:
ptr := AllocMem(40);
StrCopy(ptr, 'OneNationUnderGod');
Now we are fine to call FreeMem, because ptr still contains the address that the call to AllocMem provided.
ptr := AllocMem(40);
try
StrCpy(ptr, 'OneNationUnderGod');
// do stuff with ptr
finally
FreeMem(ptr);
end;
Clearly in real code you would find a better and more robust way to specify buffer lengths than a hard-coded value.
In your code, assuming the above fix is applied, the test for ptr being nil is needless. AllocMem never returns nil. Failure of AllocMem results in an exception being raised.
Having said all that, it's not usual to operate on string buffers in this way. It is normal to use Delphi strings. If you need a PChar, for instance to use with interop, make one with PChar(str) where str is of type string.
You say that you must use dynamically allocated PChar buffers. Perhaps that is so, but I very much doubt it.
It crashes because you are freeing static memory that was not dynamically allocated. There is no need to free memory used by literals at all. Only free memory that is dynamically allocated.
I'd try to make what you've done more explicit. You seem to mistake the name of a variable with its value. However actually - considering values - what you did was
ptrA:=AllocMem(40);
ptrB:=Pchar('OneNationUnderGod');
if ptrB<>nil then
FreeMem(ptrB);
Every new assignment changes the value overwriting the previous one, thus you freeing another pointer that was allocated.
You may read documentation for functions like StrNew, StrDispose, StrCopy, StrLCopy and sample codes with those to see some patterns of working with PChar strings.
Related
Let's go back to the basics. Frankly, I have never used New and Dispose functions before. However, after I read the New() documentation and the included examples on the Embarcadero Technologies's website and the Delphi Basics explanation of New(), it leaves questions in my head:
What are the advantages of using System.New() instead of a local variable, other than just spare a tiny amount of memory?
Common code examples for New() are more or less as follows:
var
pCustRec : ^TCustomer;
begin
New(pCustRec);
pCustRec^.Name := 'Her indoors';
pCustRec^.Age := 55;
Dispose(pCustRec);
end;
In what circumstances is the above code more appropriate than the code below?
var
CustRec : TCustomer;
begin
CustRec.Name := 'Her indoors';
CustRec.Age := 55;
end;
If you can use a local variable, do so. That's a rule with practically no exceptions. This results in the cleanest and most efficient code.
If you need to allocate on the heap, use dynamic arrays, GetMem or New. Use New when allocating a record.
Examples of being unable to use the stack include structures whose size are not known at compile time, or very large structures. But for records, which are the primary use case for New, these concerns seldom apply.
So, if you are faced with a choice of stack vs heap for a record, invariably the stack is the correct choice.
From a different perspective:
Both can suffer from buffer overflow and can be exploited.
If a local variable overflows, you get stack corruption.
If a heap variable overflows, you get heap corruption.
Some say that stack corruptions are easier to exploit than heap corruptions, but that is not true in general.
Note there are various mechanisms in operating systems, processor architectures, libraries and languages that try to help preventing these kinds of exploits.
For instance there is DEP (Data Execution Prevention), ASLR (Address Space Layout Randomization) and more are mentioned at Wikipedia.
A local static variable reserves space on the limited stack. Allocated memory is located on the heap, which is basically all memory available.
As mentioned, the stack space is limited, so you should avoid large local variables and also large parameters which are passed by value (absence of var/const in the parameter declaration).
A word on memory usage:
1. Simple types (integer, char, string, double etc.) are located directly on the stack. The amount of bytes used can be determined by the sizeof(variable) function.
2. The same applies to record variables and arrays.
3. Pointers and Objects require 4/8 bytes.
Every object (that is, class instances) is always allocated on the heap.
Value structures (simple numerical types, records containing only those types) can be allocated on the heap.
Dynamic arrays and strings content are always allocated on the heap. Only the reference pointer can be allocated on the stack. If you write:
function MyFunc;
var s: string;
...
Here, 4/8 bytes are allocated on the stack, but the string content (the text characters) will always be allocated on the heap.
So using New()/Dispose() is of poor benefit. If it contains no reference-counted types, you may use GetMem()/FreeMem() instead, since there is no internal pointer to set to zero.
The main drawback of New() or Dispose() is that if an exception occur, you need to use a try...finally block:
var
pCustRec : ^TCustomer;
begin
New(pCustRec);
try
pCustRec^.Name := 'Her indoors';
pCustRec^.Age := 55;
finally
Dispose(pCustRec);
end;
end;
Whereas allocating on the stack let the compiler do it for you, in an hidden manner:
var
CustRec : TCustomer;
begin // here a try... is generated
CustRec.Name := 'Her indoors';
CustRec.Age := 55;
end; // here a finally + CustRec cleaning is generated
That's why I almost never use New()/Dispose(), but allocate on stack, or even better within a class.
2
The usual case for heap allocation is when the object must outlive the function that created it:
It is being returned as a function result or via a var/out parameter, either directly or by returning some container.
It's being stored in some object, struct or collection that is passed in or otherwise accessible inside the procedure (this includes being signaled/queued off to another thread).
In cases of limited stack space you might prefer allocation from the heap.
Ref.
I'm trying to modify the VirtualTreeView to see data in the tree nodes in the design mode.
The allocating node memory is in the private static method so I can't do anything about it. I'm trying to reallocate the memory to match the new size then.
For the test purposes I'm trying to reallocate the same amount of memory:
ReallocMemory(Node, sizeof(Node^))
But the IDE hangs up in the random iteration throwing a lot of AV. Since my knowledge of memory allocation is pretty lacking I think I'm forgetting something. Could you point me please?
ReallocMemory is a function. It returns the new pointer value; it does not modify its argument. You want to call ReallocMem instead, or else use the result of the function:
ReallocMem(Node, SizeOf(Node^));
or
Node := ReallocMemory(Node, SizeOf(Node^));
When either of those functions cannot resize the block of memory in-place, it allocates new memory, copies the old contents into the new buffer, and then frees the original buffer. If you ignore the ReallocMemory result, then you have discarded the new pointer and retained the old, stale pointer in the Node variable. Continued use of a stale pointer would explain access violations and other unpredictable behavior.
There are two versions of those functions for C++ compatibility. C++ doesn't have Delphi's "compiler magic," which is what allows the compiler to have a single ReallocMem function that accepts and modifies any pointer type.
The ReallocMemory function looks like the C++ realloc function, but they don't behave quite the same way, which is why it's safe to directly overwrite the input variable with the function's return value. When reallocation fails, the function throws an exception, just like ReallocMem, where as realloc just returns a null pointer.
I've got an operation that's being called repeatedly in a loop. With a TRttiField:
if (field.name = '') or (field.Name[1] <> 'F') then
continue;
Profiling shows that I'm spending a lot of time in UStrAsg and UStrClr because of this. Field.Name has to make a virtual call to TRttiInstanceFieldEx.GetName, which has to perform a UTF8-to-string conversion on the underlying RTTI structure's name. This is happening twice per loop iteration.
I tried to cut all that out by bypassing the string conversions:
handle := PFieldExEntry(field.Handle);
if (handle.name = '') or (handle.Name[1] <> 'F') then
continue;
I expected to see about a 5% speed gain from this. Instead, the loop takes several seconds longer to execute, approximately 20-25% slower! I checked the generated ASM to make sure it wasn't doing anything screwy like making string copies from the RTTI structures to the local stack, but it isn't. I can't see any reason why this should have gotten slower. Anyone have any idea what might be going on here?
The field your new code reads is declared as a ShortString. As of Delphi 5, the compiler converts ShortStrings to long strings before generating the code for most string operations. (At least, that's the way it was with non-Unicode Delphi. Maybe Unicode Delphi restores some ShortString-related optimizations.)
Whereas the TRttiField wrapper might take advantage of the knowledge that it's populating a UTF-8 string with data that's already occupying one byte per character, I'd expect the ShortString-to-string code that your new loop employs might use a more general-purpose conversion routine, and you're paying the price for generality.
You might try foregoing string-conversion operations entirely. Instead, get a pointer to the first byte:
handle := PFieldExEntry(field.Handle);
NameP := PAnsiChar(#handle.name);
if (NameP[0] = #0) or (NameP[1] <> 'F') then
continue;
Note that although it's declared as a ShortString, it's not really one. It doesnn't really occupy 256 bytes. Instead, it occupies the minimum amount of memory required to hold its length byte and its characters.
I want to get the size of any "record" type in following function. But seems it doesn't work:
function GetDataSize(P : Pointer) : Integer;
begin
Result := SizeOf(P^); // **How to write the code?**
end;
For example, the size of following record is 8 bytes
SampleRecord = record
Age1 : Integer;
Age2 : Integer;
end;
But GetDataSize(#a) always returns 1 (a is a variable of SampleRecord type of course). What should I do?
I noticed that Delphi has a procedure procedure New(var P: Pointer) which can allocate the memory block corresponds to the size of the type that P points to. How can it gets the size?
The reason New knows how much memory to allocate is that New is compiler magic. It's a language built-in, so when the compiler sees you call it, it rewrites it to something like this:
// New(foo);
foo := System._New(SizeOf(foo^), TypeInfo(TypeOf(foo^)));
TypeOf here is a made-up Delphi function for expository purposes. The compiler knows the declared type of foo because it knows where all your variable declarations are. You can look at the implementation of _New in System.pas. Similar rewriting occurs for Dispose so it knows what kind of finalization to do before freeing the memory.
The ideas of variables and declarations are compile-time concepts. At run time, they cease to exist. At run time, a pointer is just an address. The type of what it points to was determined at compile time. Types are what determine something's size.
If you need to write a function that accepts pointers to multiple things with different sizes, then you'll just have to provide a second parameter that describes what the first one points to.
Check out another question here, "How to know what type is a var." The asker wondered how to determine more information about a variable given only its address.
You cannot find the size of data structure using variable of type Pointer, because compiler cannot, make a guess and check it, since pointer can points to whatever data type you can think of. You can read some information here.
There's no safe way to determine the size of a record that a pointer points to. However, if you allocated the memory that the pointer points to, you can ask the size of that memory block. But then again, since you allocated that block, you should already know the size of that block!
The Delphi memory manager keeps track of every block of memory that gets allocated. With information from the memory manager it is possible to find this information, if your pointer points to the beginning of a memory block. However, if you allocated a large block of memory, loaded some data in it and your pointer points to some data inside this block, this method would be quite unreliable.
Also, if you use referenced types (dynamic arrays, strings, classes, etc.) in your record, the size it returns will still be unusable since you get the size of the reference (4 bytes) instead of the size of the data that is referenced to.
The NEW() command just uses the type information of the datatype that you pass to it to get it's size. To know how it does this exactly, you could just check the Delphi sourcecode. Open \source\Win32\rtl\sys\System.pas and search for "_New". (With the underscore in front of it. Using this sourcecode might help you to understand how Delphi handles memory allocations, although the sourcecode can be really complex.
Delphi has a built-in memory manager. I believe new has access to the heap object and uses HeapSize() (or similar routines) to get the size of a block, for some pointer.
I just wondered, why most Delphi examples use FillChar() to initialize records.
type
TFoo = record
i: Integer;
s: string; // not safe in record, better use PChar instead
end;
const
EmptyFoo: TFoo = (i: 0; s: '');
procedure Test;
var
Foo: TFoo;
s2: string;
begin
Foo := EmptyFoo; // initialize a record
// Danger code starts
FillChar(Foo, SizeOf(Foo), #0);
s2 := Copy("Leak Test", 1, MaxInt); // The refcount of the string buffer = 1
Foo.s = s2; // The refcount of s2 = 2
FillChar(Foo, SizeOf(Foo), #0); // The refcount is expected to be 1, but it is still 2
end;
// After exiting the procedure, the string buffer still has 1 reference. This string buffer is regarded as a memory leak.
Here (http://stanleyxu2005.blogspot.com/2008/01/potential-memory-leak-by-initializing.html) is my note on this topic. IMO, declare a constant with default value is a better way.
Historical reasons, mostly. FillChar() dates back to the Turbo Pascal days and was used for such purposes. The name is really a bit of a misnomer because while it says FillChar(), it is really FillByte(). The reason is that the last parameter can take a char or a byte. So FillChar(Foo, SizeOf(Foo), #0) and FillChar(Foo, SizeOf(Foo), 0) are equivalent. Another source of confusion is that as of Delphi 2009, FillChar still only fills bytes even though Char is equivalent to WideChar. While looking at the most common uses for FillChar in order to determine whether most folks use FillChar to actually fill memory with character data or just use it to initialize memory with some given byte value, we found that it was the latter case that dominated its use rather than the former. With that we decided to keep FillChar byte-centric.
It is true that clearing a record with FillChar that contains a field declared using one of the "managed" types (strings, Variant, Interface, dynamic arrays) can be unsafe if not used in the proper context. In the example you gave, however, it is actually safe to call FillChar on the locally declared record variable as long as it is the first thing you ever do to the record within that scope. The reason is that the compiler has generated code to initialize the string field in the record. This will have already set the string field to 0 (nil). Calling FillChar(Foo, SizeOf(Foo), 0) will just overwrite the whole record with 0 bytes, including the string field which is already 0. Using FillChar on the record variable after a value was assigned to the string field, is not recommended. Using your initialized constant technique is a very good solution this problem because the compiler can generate the proper code to ensure the existing record values are properly finalized during the assignment.
If you have Delphi 2009 and later, use the Default call to initialize a record.
Foo := Default(TFoo);
See David's answer to the question How to properly free records that contain various types in Delphi at once?.
Edit:
The advantage of using the Default(TSomeType) call, is that the record is finalized before it is cleared. No memory leaks and no explicit dangerous low level call to FillChar or ZeroMem. When the records are complex, perhaps containing nested records etc, the risk of making mistakes is eliminated.
Your method to initialize the records can be made even simpler:
const EmptyFoo : TFoo = ();
...
Foo := EmptyFoo; // Initialize Foo
Sometimes you want a parameter to have a non-default value, then do like this:
const PresetFoo : TFoo = (s : 'Non-Default'); // Only s has a non-default value
This will save some typing and the focus is set on the important stuff.
FillChar is fine to make sure you don't get any garbage in a new, uninitialized structure (record, buffer, arrray...).
It should not be used to "reset" the values without knowing what your are resetting.
No more than just writing MyObject := nil and expecting to avoid a memory leak.
In particulart all managed types are to be watched carefully.
See the Finalize function.
When you have the power to fiddle directly with the memory, there is always a way to shoot yourself in the foot.
FillChar is usually used to fill Arrays or records with only numeric types and array. You are correct that it shouldn't be used to when there are strings (or any ref-counted variables) in the record.
Although your suggestion of using a const to initialize it would work, an issue comes into play when I have a variable length array that I want to initialize.
The question may also be asking:
why FillChar
and not ZeroMemory?
There is no ZeroMemory function in Windows. In the header files (winbase.h) it is a macro that, in the C world, turns around and calls memset:
memset(Destination, 0, Length);
ZeroMemory is the language neutral term for "your platform's function that can be used to zero memory"
The Delphi equivalent of memset is FillChar.
Since Delphi doesn't have macros (and before the days of inlining), calling ZeroMemory meant you had to suffer the penalty of an extra function call before you actually got to FillChar.
So in many ways, calling FillChar is a performance micro-optimization - which no longer exists now that ZeroMemory is inlined:
procedure ZeroMemory(Destination: Pointer; Length: NativeUInt); inline;
Bonus Reading
Windows also contains the SecureZeroMemory function. It does the exact same thing as ZeroMemory. If it does the same thing as ZeroMemory, why does it exist?
Because some smart C/C++ compilers might recognize that setting memory to 0 before getting rid of the memory is a waste of time - and optimize away the call to ZeroMemory.
I don't think Delphi's compiler is as smart as many other compilers; so there's no need for a SecureFillChar.
Traditionally, a character is a single byte (no longer true for Delphi 2009), so using fillchar with a #0 would initalize the memory allocated so that it only contained nulls, or byte 0, or bin 00000000.
You should instead use the ZeroMemory function for compatibility, which has the same calling parameters as the old fillchar.
This question has a broader implication that has been in my mind for ages. I too, was brought up on using FillChar for records. This is nice because we often add new fields to the (data) record and of course FillChar( Rec, SizeOf( Rec), #0 ) takes care of such new fields. If we 'do it properly', we have to iterate through all fields of the record, some of which are enumerated types, some of which may be records themselves and the resulting code is less readable as well be possibly erroneous if we dont add new record fields to it diligently. String fields are common, thus FillChar is a no-no now. A few months ago, I went around and converted all my FillChars on records with string fields to iterated clearing, but I was not happy with the solution and wonder if there is a neat way of doing the 'Fill' on simply types (ordinal / float) and 'Finalize' on variants and strings?
Here is a better way to initialize stuff without using FillChar:
Record in record (Cannot initialize)
How to initialize a static array?