How to store dynamic arrays in a TList? - delphi

I need to store an unknown number of groups. Each group has an unknown number of elements/items.
This is my 'group':
TGroup= array of Integer; <------ dynamic array (as you can see) :)
I want to use a TList to hold my groups. The idea is that I may want to access the groups later and add more items to them.
I have this code, but I can't make it work:
TYPE
TGroup= array of Integer; // Each group has x items (x can be from 1 to 10000)
procedure TForm1.FormCreate(Sender: TObject);
VAR CurGroup: TGroup;
grp, item: Integer;
Groups: TList; // can contain up to 1 million groups
begin
Groups:= TList.Create;
{ Init }
for grp:= 1 to 4 DO // Put a dummy item in TList
begin
SetLength(CurGroup, 1); // Create new group
Groups.Add(#CurGroup); // Store it
end;
CurGroup:= NIL; // Prepare for next use
for grp:= 1 to 4 DO // We create 4 groups. Each group has 3 items
begin
CurGroup:= Groups[Groups.Count-1]; // We retrieve the current group from list in order to add more items to it
{ We add few items }
for item:= 0 to 2 DO
begin
SetLength(CurGroup, Length(CurGroup)+1); // reserve space for each new item added
CurGroup[item]:= Item;
end;
Groups[Groups.Count-1]:= #CurGroup; // We put the group back into the list
end;
{ Verify }
CurGroup:= NIL;
CurGroup:= Groups[0];
Assert(Length(CurGroup)> 0); // FAIL
if (CurGroup[0]= 0)
AND (CurGroup[1]= 1)
AND (CurGroup[2]= 2)
then Application.ProcessMessages;
FreeAndNil(Groups);
end;
Note: The code is complete. You can paste it in your Delphi (7) to try it.

Oh, this would be sooooo much nicer in newer versions of Delphi... You'd use the generic, TList<T>. var Groups: TList<TGroup>;
You're best bet is to use another dynamic array: Groups: array of TGroup;
The reason is that dynamic arrays are compiler managed and reference counted. TList only operates on Pointers. There is no straight forward way to keep the reference counts sane when trying to put the dynarrays into the TList.
Another issue you're having is that you're adding the stack address of the dynamic array variable to the TList, and not the actual array. The expression #CurGroup is the "address of the CurGroup variable" which being a local variable, is on the stack.

I've created a wrapper around dynamic array RTTI.
It's just a first draft, but it was inspired by your question, and the fact that the TList methods are missing.
type
TGroup: array of integer;
var
Group: TGroup;
GroupA: TDynArray;
i, v: integer;
begin
GroupA.Init(TypeInfo(TGroup),Group); // associate GroupA with Group
for i := 0 to 1000 do begin
v := i+1000; // need argument passed as a const variable
GroupA.Add(v);
end;
v := 1500;
if GroupA.IndexOf(v)<0 then // search by content
ShowMessage('Error: 1500 not found!');
for i := GroupA.Count-1 downto 0 do
if i and 3=0 then
GroupA.Delete(i); // delete integer at index i
end;
This TDynArray wrapper will work also with array of string or array of records... Records need only to be packed and have only not reference counted fields (byte, integer, double...) or string reference-counted fields (no Variant nor Interface within).
The IndexOf() method will search by content. That is e.g. for an array of record, all record fields (including strings) must match.
See TDynArray in the SynCommons.pas unit from our Source Code repository. Works from Delphi 6 up to XE, and handle Unicode strings.
And the TTestLowLevelCommon._TDynArray method is the automated unitary tests associated with this wrapper. You'll find out here samples of array of records and more advanced features.
I'm currently implementing SaveToStream and LoadToStream methods...
Perhaps a new way of using generic-like features in all Delphi versions.
Edit:
I've added some new methods to the TDynArray record/object:
now you can save and load a dynamic array content to or from a string (using LoadFromStream/SaveToStream or LoadFrom/SaveTo methods) - it will use a proprietary but very fast binary stream layout;
and you can sort the dynamic array content by two means: either in-place (i.e. the array elements content is exchanged) or via an external integer index look-up array (using the CreateOrderedIndex method - in this case, you can have several orders to the same data);
you can specify any custom comparison function, and there is a new Find method will can use fast binary search if available.
Here is how those new methods work:
var
Test: RawByteString;
...
Test := GroupA.SaveTo;
GroupA.Clear;
GroupA.LoadFrom(Test);
GroupA.Compare := SortDynArrayInteger;
GroupA.Sort;
for i := 1 to GroupA.Count-1 do
if Group[i]<Group[i-1] then
ShowMessage('Error: unsorted!');
v := 1500;
if GroupA.Find(v)<0 then // fast binary search
ShowMessage('Error: 1500 not found!');
Still closer to the generic paradigm, faster, and for Delphi 6 up to XE...

I don't have D7 on a machine here, but you might try something like this instead (console test app - it compiles in XE with no hints or warnings, but not sure how D7 will handle it):
program Project1;
{$APPTYPE CONSOLE}
uses
SysUtils, Classes;
type
TGroup = array of Integer;
THolder=class(TObject)
Group: TGroup;
end;
var
GroupList: TList;
i: Integer;
begin
GroupList := TList.Create;
for i := 0 to 2 do
begin
GroupList.Add(THolder.Create);
with THolder(GroupList[GroupList.Count - 1]) do
begin
SetLength(Group, 3);
Group[0] := i;
Group[1] := i + 10;
Group[2] := i + 20;
end;
end;
for i := 0 to GroupList.Count - 1 do
begin
Writeln(i, ':0 ', THolder(GroupList[i]).Group[0]);
Writeln(i, ':1 ', THolder(GroupList[i]).Group[1]);
Writeln(i, ':2 ', THolder(GroupList[i]).Group[2]);
end;
// Not cleaning up list content, because we're exiting the app.
// In a real app, of course, you'd want to free each THolder in
// the list, or use a TObjectList and let it own the objects.
// You'd also want a try..finally to make sure the list gets freed.
GroupList.Free;
Readln;
end.

Another thing you may want to try is using a TObjectList. TObjectList is able to store a list of Objects, so you can create a new class descendant of TObject and then manage the TObjectList using these objects.
Something like (untested):
type TGroupArray : array of Integer;
type TGroup = class(Tobject)
GroupArray : TGroupArray;
end;
GroupList : TobjectList;
procedure TForm1.FormCreate(Sender: TObject);
var CurGroup : TGroup;
begin
GroupList:= TObjectList.Create;
CurGroup := TGroup.Create;
SetLength(CurGroup.GroupArray,1);
CurGroup.GroupArray[0] := 10;
GroupList.Add(CurGroup);
RetreiveGroup := GroupList.Items[0];
FreeandNil(GroupList);
end;
etc...

When you code:
for grp:= 1 to 4 DO // Put a dummy item in TList
begin
SetLength(CurGroup, 1); // Create new group
Groups.Add(#CurGroup); // Store it
end;
in fact, SetLength(CurGroup) WON'T create a new group.
It will resize the only one existing CurGroup.
So #CurGroup won't change: it will always be some address on the stack, where CurGroup is added. You add the same address to the list several times.
So You'll have to create a dynamic array of TGroup instances, like that:
var GroupArray: array of TGroup;
SetLength(GroupArray,4);
for grp := 0 to high(GroupArray) do
begin
SetLength(GroupArray[grp],1);
Groups.Add(#GroupArray[grp]);
end;
But of course, the GroupArray must remain allocated during all the time Groups will need it. So you'll have perhaps to put this GroupArray as a property in the class, because if you create this GroupArray on the stack, all GroupArray[] items will be released when the method exits and its stack is freed.
But of course, the GroupArray[] will be a more direct access to the TGroup items... Groups[i] will be equal to GroupArray[]... With no reference counting problem... Because e.g. if you resize for instance a TGroup item from its pointer in Groups[], I'm not sure that you won't have any memory leak...

So, basically everybody suggests to create an array of array instead of using a TList. Well, I already have done that. I just wanted to "upgrade" from 'array of array' to a TList since it has Add(). It looks like I will return to my original code.
Thank you all and +1 for each answer.

Related

Writing a generic TList of records

I am trying to write a generic TList that contains records of a specific type. Starting from David's answer on this question, I have written this class:
Type
TMERecordList<T> = Class(TList<T>)
Public Type
P = ^T;
Private
Function GetItem(Index: Integer): P;
Public
Procedure Assign(Source: TMERecordList<T>); Virtual;
Function First: P; Inline;
Function Last: P; Inline;
Property Items[Index: Integer]: P Read GetItem;
End;
Procedure TMERecordList<T>.Assign(Source: TMERecordList<T>);
Var
SrcItem: T;
Begin
Clear;
For SrcItem In Source Do
Add(SrcItem);
End;
Function TMERecordList<T>.First: P;
Begin
Result := Items[0];
End;
Function TMERecordList<T>.GetItem(Index: Integer): P;
Begin
If (Index < 0) Or (Index >= Count) Then
Raise EArgumentOutOfRangeException.CreateRes(#SArgumentOutOfRange);
Result := #List[Index];
End;
Function TMERecordList<T>.Last: P;
Begin
Result := Items[Count - 1];
End;
Having methods that return a pointer to the record works well (not perfectly) as pointers to records can be used as if they were records in most use cases. Using a record with properties and setters, these test cases work as expected:
TMETestRecord = Record
Private
FID: Word;
FText: String;
FValues: TIntegers;
Procedure SetID(Const Value: Word);
Procedure SetText(Const Value: String);
Procedure SetValues(Const Value: TIntegers);
Public
Property ID: Word Read FID Write SetID;
Property Text: String Read FText Write SetText;
Property Values: TIntegers Read FValues Write SetValues;
End;
// TestSetItem1
rl2[0] := rl1[0];
// TestSetItem2
r.ID := 9;
r.Text := 'XXXX';
r.Values := [9, 99, 999, 9999];
rl1[0] := r;
// TestAssignEmpty (rl0 is empty... after assign so should rl2)
rl2.Assign(rl0);
// TestAssignDeepCopies (modifications after assign should not affect both records)
rl2.Assign(rl1);
r.ID := 9;
r.Text := 'XXXX';
r.Values := [9, 99, 999, 9999];
rl1[0] := r;
Problem 1 - modifying a contained record
... this test case compiles and runs but does not work as desired:
// TestSetItemFields
rl1[0].ID := 9;
rl1[0].Text := 'XXXX';
rl1[0].Values := [9, 99, 999, 9999];
Modifications are applied to temporary copy of the record and not to the one stored in the list. I know this is a known and expected behaviour, as documented in other questions.
But... is there a way around it? I was thinking that maybe if the TMERecordList<>.Items property had a setter the compiler could maybe do what is actually desired. Could it? I know David has got a solution, as hinted at in this question... but I can't seem to find it on my own.
This would really be nice to have, as it would allow me to have a way of using the list identical (or almost) to that of a TList of objects. Having the same interface means I could easily change from objects to records and viceversa, when the need arises.
Problem 2 - interface ambiguity
Having the TList<> return a record pointer does pose some interface ambiguity problems. Some TList<> methods accept T parameters, and we know that being records, these are going to be passed by value. So what should these methods do? Should I rethink them? I'm talking specifically about these sets of methods:
Remove and RemoveItem
Extract and ExtractItem
Contains IndexOf, IndexOfItem and LastIndexOf
There is some ambiguity as to how these should test contained items to see if they match the parameter record value. The list could very well contain identical records and this could become a source of bugs in user code.
I tried not deriving it from TList<>, so as not to have these methods, but it was a mess. I couldn't write a class similar to TList without also writing my own TListHelper. Unfortunately System.Generics.Collections's one has some needed fields that are private, like FCount, and cannot be used outside the unit.
Problem 1
Your Items property is not marked as being default. Hence your erroneous code is picking up the base class default property. Add the default keyword to your Items property:
property Items[Index: Integer]: P read GetItem; default;
Problem 2
This is really a consequence of deriving from TList<T>. I would not recommend doing that. Encapsulate an instance of TList<T> and therefore define the interface explicitly rather than inheriting it. Or implement the list functionality directly in your code. After all, it's not much more than a wrapper around a dynamic array.
For what it is worth my classes don't use TList<T> at all which is a decision that I was very happy with when Emba broke the class in a recent release.
In the recent versions TList<T> in System.Generics.Collections contains a List property that gives you direct access to the backing array of the list. You can use that to manipulate the records inside the list.

Storing interface pointer inside tree view nodes

I'm attempting to store interface pointers in a tree view under TTreeNode.Data properties. While I am able to store an interface pointer (Node.Data := Pointer(MyInterface);) it does not seem to work the other way around (MyInterface := ISomeInterface(Node.Data);). It always comes out nil.
I've also attempted to use manual reference counting, as I've seen required in another question. However, it is still coming out nil and now giving memory leaks.
//Clears tree view and adds drive letters
procedure TfrmMain.cmdRefreshBrowseClick(Sender: TObject);
var
Arr, O: ISuperObject;
X: Integer;
N, C: TTreeNode;
begin
//First clear all items and release their interface refs
for N in tvBrowse.Items do begin
O:= ISuperObject(N.Data);
O._Release;
end;
tvBrowse.Items.Clear;
Arr:= ListDirectory(''); //Returns ISuperObject array listing drives
for X := 0 to Arr.AsArray.Length-1 do begin
O:= Arr.AsArray.O[X];
N:= tvBrowse.Items.Add(nil, O.S['drive']+':\ ['+O.S['type']+']'); //Add root node
N.Data:= Pointer(O); // Assign interface pointer to node data
O._AddRef; //Manually increment interface reference count
C:= tvBrowse.Items.AddChild(N, ''); //Add a fake child node
end;
end;
procedure TfrmMain.tvBrowseExpanding(Sender: TObject; Node: TTreeNode;
var AllowExpansion: Boolean);
var
N, C: TTreeNode;
P, A, O: ISuperObject;
X: Integer;
begin
//Check first node if it's a fake node
N:= Node.getFirstChild;
if N.Text = '' then begin //if first node is a fake node...
P:= ISuperObject(Node.Data); // <-- P always comes out nil here???
N.Delete; //Delete first "fake" node
//Get child files/folders
if Node.Parent = nil then //If root (drive) node...
A:= ListDirectory(P.S['drive']+':\') //Returns ISuperObject array listing files/folders
else
A:= ListDirectory(P.S['name']); //Returns ISuperObject array listing files/folders
for X := 0 to A.AsArray.Length-1 do begin
O:= A.AsArray.O[X];
C:= tvBrowse.Items.AddChild(N, O.S['name']); //Add child node
C.Data:= Pointer(O); //Assign interface pointer to node data
O._AddRef; //Manually increment reference count
end;
end;
end;
What's the appropriate way to do this?
Essentially you are doing this correctly. Your casts are reasonable, and you understand the need to perform manual reference counting since you are holding the reference in a field of type Pointer which does not perform reference counting.
P := ISuperObject(Node.Data);
If P is assigned the value nil that means that Node.Data is equal to nil. There's nothing more to say. Presumably there is some rather mundane reason for Data being nil, but it's nothing to do with the way you are casting.
Looking at your code, I would criticise it for mixing all the different concerns up together. You will find this task much easier if you can maintain a degree of isolation between the various different aspects.
One way to make life much simpler is to avoid using the untyped pointer Data. Instead use a custom node type that can perform proper reference counting:
type
TMyTreeNode = class(TTreeNode)
private
FIntf: IInterface;
property
Intf: IInterface read FIntf write FIntf;
end;
You'll need to handle the OnCreateNodeClass event of the tree view to get the control to create your node class.
procedure TForm1.TreeView1CreateNodeClass(Sender: TCustomTreeView;
var NodeClass: TTreeNodeClass);
begin
NodeClass := TMyTreeNode;
end;
Now whenever the tree view control creates a node instance it creates one of type TMyTreeNode. Which happens to have a field to contain your interface. I've typed it as IInterface here, but you'd use the more specific interface that fits your needs. And of course you can add whatever capability you please to your custom node type.
The mild bind to this is that you need to cast node references from TTreeNode (as returned by the underlying tree view control) to TMyTreeNode in order to gain access to the interface property. However, this bind is well worth it in my view because you can rely on the compiler to managed lifetime correctly, and so forget all about that aspect of the code. This will allow you to concentrate on your program rather than tedious boilerplate. Continuing down the path you are currently on looks like a recipe for memory leaks and access violations. Get the compiler to manage things and you can be sure to avoid any such pitfalls.

How to properly free records that contain various types in Delphi at once?

type
TSomeRecord = Record
field1: integer;
field2: string;
field3: boolean;
End;
var
SomeRecord: TSomeRecord;
SomeRecAr: array of TSomeRecord;
This is the most basic example of what I have and since I want to reuse SomeRecord (with certain fields remaining empty, without freeing everything some fields would be carried over when I'm reusing SomeRecord, which is obviously undesired) I am looking for a way to free all of the fields at once. I've started out with string[255] and used ZeroMemory(), which was fine until it started leaking memory, that was because I switched to string. I still lack the knowledge to get why, but it appears to be related to it being dynamic. I am using dynamic arrays as well, so I assume that trying ZeroMemory() on anything dynamic would result in leaks. One day wasted figuring that out. I think I solved this by using Finalize() on SomeRecord or SomeRecAr before ZeroMemory(), but I'm not sure if this is the proper approach or just me being stupid.
So the question is: how to free everything at once? does some single procedure exist at all for this that I'm not aware of?
On a different note, alternatively I would be open to suggestions how to implement these records differently to begin with, so I don't need to make complicated attempts at freeing stuff. I've looked into creating records with New() and then getting rid of it Dispose(), but I have no idea what it means when a variable after a call to Dispose() is undefined, instead of nil. In addition, I don't know what's the difference between a variable of a certain type (SomeRecord: TSomeRecord) versus a variable pointing to a type (SomeRecord: ^TSomeRecord). I'm looking into the above issues at the moment, unless someone can explain it quickly, it might take some time.
Assuming you have a Delphi version that supports implementing methods on a record, you could clear a record like this:
type
TSomeRecord = record
field1: integer;
field2: string;
field3: boolean;
procedure Clear;
end;
procedure TSomeRecord.Clear;
begin
Self := Default(TSomeRecord);
end;
If your compiler doesn't support Default then you can do the same quite simply like this:
procedure TSomeRecord.Clear;
const
Default: TSomeRecord=();
begin
Self := Default;
end;
You might prefer to avoid mutating a value type in a method. In which case create a function that returns an empty record value, and use it with the assignment operator:
type
TSomeRecord = record
// fields go here
class function Empty: TSomeRecord; static;
end;
class function TSomeRecord.Empty: TSomeRecord;
begin
Result := Default(TSomeRecord);
end;
....
Value := TSomeRecord.Empty;
As an aside, I cannot find any documentation reference for Default(TypeIdentifier). Does anyone know where it can be found?
As for the second part of your question, I see no reason not to continue using records, and allocating them using dynamic arrays. Attempting to manage the lifetime yourself is much more error prone.
Don't make thinks overcomplicated!
Assigning a "default" record is just a loss of CPU power and memory.
When a record is declared within a TClass, it is filled with zero, so initialized. When it is allocated on stack, only reference counted variables are initialized: others kind of variable (like integer or double or booleans or enumerations) are in a random state (probably non zero). When it will be allocated on the heap, getmem will not initialize anything, allocmem will fill all content with zero, and new will initialize only reference-counted members (like on the stack initialization): in all cases, you should use either dispose, either finalize+freemem to release a heap-allocated record.
So about your exact question, your own assumption was right: to reset a record content after use, never use "fillchar" (or "zeromemory") without a previous "finalize". Here is the correct and fastest way:
Finalize(aRecord);
FillChar(aRecord,sizeof(aRecord),0);
Once again, it will be faster than assigning a default record. And in all case, if you use Finalize, even multiple times, it won't leak any memory - 100% money back warranty!
Edit: After looking at the code generated by aRecord := default(TRecordType), the code is well optimized: it is in fact a Finalize + bunch of stosd to emulate FillChar. So even if the syntax is a copy / assignement (:=), it is not implemented as a copy / assignment. My mistake here.
But I still do not like the fact that a := has to be used, where Embarcadero should have better used a record method like aRecord.Clear as syntax, just like DelphiWebScript's dynamic arrays. In fact, this := syntax is the same exact used by C#. Sounds like if Embacardero just mimics the C# syntax everywhere, without finding out that this is weird. What is the point if Delphi is just a follower, and not implement thinks "its way"? People will always prefer the original C# to its ancestor (Delphi has the same father).
The most simply solution I think of will be:
const
EmptySomeRecord: TSomeRecord = ();
begin
SomeRecord := EmptySomeRecord;
But to address all the remaining parts of your question, take these definitions:
type
PSomeRecord = ^TSomeRecord;
TSomeRecord = record
Field1: Integer;
Field2: String;
Field3: Boolean;
end;
TSomeRecords = array of TSomeRecord;
PSomeRecordList = ^TSomeRecordList;
TSomeRecordList = array[0..MaxListSize] of TSomeRecord;
const
EmptySomeRecord: TSomeRecord = ();
Count = 10;
var
SomeRecord: TSomeRecord;
SomeRecords: TSomeRecords;
I: Integer;
P: PSomeRecord;
List: PSomeRecordList;
procedure ClearSomeRecord(var ASomeRecord: TSomeRecord);
begin
ASomeRecord.Field1 := 0;
ASomeRecord.Field2 := '';
ASomeRecord.Field3 := False;
end;
function NewSomeRecord: PSomeRecord;
begin
New(Result);
Result^.Field1 := 0;
Result^.Field2 := '';
Result^.Field3 := False;
end;
And then here some multiple examples on how to operate on them:
begin
// Clearing a typed variable (1):
SomeRecord := EmptySomeRecord;
// Clearing a typed variable (2):
ClearSomeRecord(SomeRecord);
// Initializing and clearing a typed array variabele:
SetLength(SomeRecords, Count);
// Creating a pointer variable:
New(P);
// Clearing a pointer variable:
P^.Field1 := 0;
P^.Field2 := '';
P^.Field3 := False;
// Creating and clearing a pointer variable:
P := NewSomeRecord;
// Releasing a pointer variable:
Dispose(P);
// Creating a pointer array variable:
ReallocMem(List, Count * SizeOf(TSomeRecord));
// Clearing a pointer array variable:
for I := 0 to Count - 1 do
begin
Pointer(List^[I].Field2) := nil;
List^[I].Field1 := 0;
List^[I].Field2 := '';
List^[I].Field3 := False;
end;
// Releasing a pointer array variable:
Finalize(List^[0], Count);
Choose and/or combine as you like.
With SomeRecord: TSomeRecord, SomeRecord will be an instance/variable of type TSomeRecord. With SomeRecord: ^TSomeRecord, SomeRecord will be a pointer to a instance or variable of type TSomeRecord. In the last case, SomeRecord will be a typed pointer. If your application transfer a lot of data between routines or interact with external API, typed pointer are recommended.
new() and dispose() are only used with typed pointers. With typed pointers the compiler doesn't have control/knowlegde of the memory your application is using with this kind of vars. It's up to you to free the memory used by typed pointers.
In the other hand, when you use a normal variables, depending on the use and declaration, the compiler will free memory used by them when it considers they are not necessary anymore. For example:
function SomeStaff();
var
NativeVariable: TSomeRecord;
TypedPointer: ^TSomeRecord;
begin
NaviveVariable.Field1 := 'Hello World';
// With typed pointers, we need to manually
// create the variable before we can use it.
new(TypedPointer);
TypedPointer^.Field1 := 'Hello Word';
// Do your stuff here ...
// ... at end, we need to manually "free"
// the typed pointer variable. Field1 within
// TSomerecord is also released
Dispose(TypedPointer);
// You don't need to do the above for NativeVariable
// as the compiler will free it after this function
// ends. This apply also for native arrays of TSomeRecord.
end;
In the above example, the variable NativeVariable is only used within the SomeStaff function, so the compiler automatically free it when the function ends. This appy for almost most native variables, including arrays and records "fields". Objects are treated differently, but that's for another post.

Disposing pointers to complex records

I have list of pointers to some complex records. Sometimes when I try disposing them I get invalid pointer operation error. I'm not really sure if I'm creating and disposing them properly.
The record looks like this:
type
PFILEDATA = ^TFILEDATA;
TFILEDATA = record
Description80: TFileType80; // that's array[0..80] of WideChar
pFullPath: PVeryLongPath; // this is pointer to array of WideChar
pNext: PFILEDATA; // this is pointer to the next TFILEDATA record
end;
As I understand when I want a pointer to such record I need to initialize the pointer and the dynamic arrays like this:
function GimmeNewData(): PFILEDATA;
begin
New(Result);
New(Result^.pFullPath);
end;
Now to dispose of series of these records I wrote this:
procedure DisposeData(var pData: PFILEDATA);
var pNextData: PFILEDATA;
begin
while pData^.pNext <> nil do begin
pNextData := pData^.pNext; // Save pointer to the next record
Finalize(pData^.pFullPath); // Free dynamic array
Dispose(pData); // Free the record
pData := pNextData;
end;
Finalize(pData^.pFullPath);
Dispose(pData);
pData := nil;
end;
When I run my program in the debug mode (F9) in the Delphi 2010 IDE something weird happens. When I step trough DisposeData code with F8 it appears that program skips Finalize(pData^.pFullPath) line and jumps to Dispose(pData). Is this normal? Also when Dispose(pData) is executed the Local variables window that displays contents of the pointers does not change. Does this mean that dispose fails?
Edit:
PVeryLongPath is:
type
TVeryLongPath = array of WideChar;
PVeryLongPath = ^TVeryLongPath;
Edit2
So I create 2 TFILEDATA records then I dispose them. Then I create the same 2 records again. For some reason this time pNext in the second record is not nil. It points to the 1st record. Disposing this weird thing gets invalid pointer operation error.
Randomly I have inserted pData^.pNext := nil in the DisposeData procedure.
Now the code looks like this:
procedure DisposeData(var pData: PFILEDATA);
var pNextData: PFILEDATA;
begin
while pData <> nil do begin
pNextData := pData^.pNext;
pData^.pNext := nil; // <----
Dispose(pData^.pFullPath);
Dispose(pData);
pData := pNextData;
end;
end;
The error is gone.
I'll try to change PVeryLongPath into TVeryLongPath.
First, if you free something, the contents of pointers to it do not change. That is why you don't see a change in the local variables display.
EDIT: declare pFullPath as TVeryLongPath. This is a reference type already, and you should not use a pointer to such a type. New() doesn't do what you think it does, in such a case.
It would probably be better if you declared it as UnicodeString, or if your Delphi doesn't have that, WideString.
If pFullPath is declared as a dynamic "array of WideChar", then you should not use New() on it. For dynamic arrays, use SetLength() and nothing else. Dispose() will properly dispose of all items in your record, so just do:
New(Result);
SetLength(Result^.pFullPath, size_you_need);
and later:
Dispose(pData);
In normal code, you should never have to call Finalize(). This is all taken care of by Dispose, as long as you pass a pointer of the correct type to Dispose().
FWIW, I would recommend this and this article of mine.
The fact that you accepted Serg's answer indicates that there is something wrong with your node creation code. Your comment to that answer confirms that.
I'm adding this as a new answer because the edits to the question significantly change it.
Linked list code should look like this:
var
Head: PNode=nil;
//this may be a global variable, or better, a field in a class,
//in which case it would be initialized to nil on creation
function AddNode(var Head: PNode): PNode;
begin
New(Result);
Result.Next := Head;
Head := Result;
end;
Notice that we are adding the node to the head of the list. We don't need to initialize Next to nil anywhere because we always assign another node pointer to Next. That rule is important.
I've written this as a function which returns the new node. Since the new node is always added at the head this is somewhat redundant. Because you can ignore function return values it doesn't really do any harm.
Sometimes you may want to initialize the contents of the node when you add new nodes. For example:
function AddNode(var Head: PNode; const Caption: string): PNode;
begin
New(Result);
Result.Caption := Caption;
Result.Next := Head;
Head := Result;
end;
I much prefer this approach. Always make sure that your fields are initialized. If zero initialization is fine for you then you can use AllocMem to create your node.
Here's a more concrete example of using such a method:
type
PNode = ^TNode;
TNode = record
Caption: string;
Next: PNode;
end;
procedure PopulateList(Items: TStrings);
var
Item: string;
begin
for Item in Items do
AddNode(Head, Item);
end;
To destroy the list the code runs like this:
procedure DestroyList(var Head: PNode);
var
Next: PNode;
begin
while Assigned(Head) do begin
Next := Head.Next;
Dispose(Head);
Head := Next;
end;
end;
You can clearly see that this method can only return when Head is nil.
If you encapsulate your linked list in a class then you can make the head pointer a member of the class and avoid the need to pass it around.
The main point I would like to make is that manual memory allocation code is delicate. It is easy to make little mistakes in the details. In situations like that it pays to put the delicate code in helper functions or methods so you only need to write it once. Linked lists are a great example of a problem that loves to be solved with generics. You can write the memory management code once and re-use it for all sorts of different node types.
I recommend that you avoid using a dynamic array of WideChar which is not at all convenient to work with. Instead use string if you have Delphi 2009 or later, or WideString for earlier Delphi versions. Both of these are dynamic string types with WideChar elements. You can assign to them and Delphi deals with all the allocation.
So, assuming that you now have the following record:
TFILEDATA = record
Description80: TFileType80;
pFullPath: WideString;
pNext: PFILEDATA;
end;
you can simplify things considerably.
function GimmeNewData(): PFILEDATA;
begin
New(Result);
end;
procedure DisposeData(var pData: PFILEDATA);
var pNextData: PFILEDATA;
begin
while pData <> nil do begin
pNextData := pData^.pNext;
Dispose(pData);
pData := pNextData;
end;
end;
You should also initialize pNext field to nil - without it you will finally get access violation. Taking into account what was already said in the previous answers, you can change your code as
type
TFileType80 = array[0..80] of WideChar;
PFILEDATA = ^TFILEDATA;
TFILEDATA = record
Description80: TFileType80;
FullPath: WideString;
pNext: PFILEDATA;
end;
function GimmeNewData: PFILEDATA;
begin
New(Result);
Result^.pNext:= nil;
end;
I think most of your problems are caused by the assumption that New() gives you memory that is zeroed out. I'm pretty sure (and I'm also sure someone will correct me if I'm wrong), but Delphi does not guarantee that that is the case. This can be rectified by changing your code to this:
function GimmeNewData(): PFILEDATA;
begin
New(Result);
ZeroMemory(Result, SizeOf(TFILEDATA));
end;
You should always either zero the memory you get allocated for a record, or at least fill all the fields with something else relevant. This behavior is different to objects, which are guaranteed to be zeroed on allocation.
Hope this helps.

Tree-like Datastructure (for use with VirtualTreeview)

I have come to the point where I need to stop storing my data in a VCL component, and have an "underlying datastructure", as Mr. Rob Kennedy suggested.
First of all, this question is about "how do I make an underlying datastructure". :)
My hierachy consists of 2 levels of nodes.
Right now, I go thru my stuff by looping rootnodes, wherein I loop thru the rootnode's childnodes, to get what I need (Data). I would love to be able to store all my data in a so-called Underlying Datastructure, so that I can easily modify the entries using threads (I suppose I am able to do that?)
However, when looping through my entries (right now), the results are depending on the node's Checkstate - if I am using an underlying data structure, how do I know if my node is checked or not, when its my datastructure I loop thru, and not my nodes?
Let's say I wanted to use 2 levels.
This would be the Parent:
TRoot = Record
RootName : String;
RootId : Integer;
Kids : TList; //(of TKid)
End;
And the kid:
TKid = Record
KidName : String;
KidId : Integer;
End;
Thats basically what I do now. Comments state that this is not the best solution, so I am open to suggestions. :)
I hope you understand my question(s). :)
Thanks!
The data structure you're requesting is very simple, it's so simple I'd recommend using the windows-provided TTreeView: it allows storing the text and an ID straight into the tree's node with no additional work.
Despite my recommendation to use the simpler TTreeView I'm going to provide my take on the data structure problem. First of all I'm going to use classes, not records. In your very short code sample you're mixing records and classes in a very unfrotunate way: When you make a copy of the TRoot record (assigning records makes complete copies, because records are allways treated as "values"), you're not making a "deep copy" of the tree: The complete copy of TRoot will contain the same Kids:TList as the original, because classes, unlike records, are references: you're coping the value of the reference.
An other problem when you have a record with an object field is life cycle management: A record doesn't have an destructor so you'll need an other mechanism to free the owned object (Kids:TList). You could replace the TList with an array of Tkid but then you'll need to be very careful when passing the monster record around, because you might end making deep copies of huge records when you least expect it.
In my opinion the most prudent thing to do is to base the data structure on classes, not records: class instances (objects) are passed around as references, so you can move them around all you want with no problems. You also get built-in life cycle management (the destructor)
The base class would look like this. You'll notice it can be used as either the Root or the Kid, because both Root and Kid share data: The both have a name and an ID:
TNodeClass = class
public
Name: string;
ID: Integer;
end;
If this class is used as an Root, it needs a way to store the Kids. I assume you're on Delphi 2010+, so you have generics. This class, complete with a list, looks like this:
type
TNode = class
public
ID: integer;
Name: string;
VTNode: PVirtualNode;
Sub: TObjectList<TNode>;
constructor Create(aName: string = ''; anID: integer = 0);
destructor Destroy; override;
end;
constructor TNode.Create(aName:string; anID: Integer);
begin
Name := aName;
ID := anID;
Sub := TObjectList<TNode>.Create;
end;
destructor TNode.Destroy;
begin
Sub.Free;
end;
You might not immediately realize this, but this class alone is enough to implement a multi-level tree! Here's some code to fill up the tree with some data:
Root := TNode.Create;
// Create the Contacts leaf
Root.Sub.Add(TNode.Create('Contacts', -1));
// Add some contacts
Root.Sub[0].Sub.Add(TNode.Create('Abraham', 1));
Root.Sub[0].Sub.Add(TNode.Create('Lincoln', 2));
// Create the "Recent Calls" leaf
Root.Sub.Add(TNode.Create('Recent Calls', -1));
// Add some recent calls
Root.Sub[1].Sub.Add(TNode.Create('+00 (000) 00.00.00', 3));
Root.Sub[1].Sub.Add(TNode.Create('+00 (001) 12.34.56', 4));
You need a recursive procedure to fill the virtual tree view using this type:
procedure TForm1.AddNodestoTree(ParentNode: PVirtualNode; Node: TNode);
var SubNode: TNode;
ThisNode: PVirtualNode;
begin
ThisNode := VT.AddChild(ParentNode, Node); // This call adds a new TVirtualNode to the VT, and saves "Node" as the payload
Node.VTNode := ThisNode; // Save the PVirtualNode for future reference. This is only an example,
// the same TNode might be registered multiple times in the same VT,
// so it would be associated with multiple PVirtualNode's.
for SubNode in Node.Sub do
AddNodestoTree(ThisNode, SubNode);
end;
// And start processing like this:
VT.NodeDataSize := SizeOf(Pointer); // Make sure we specify the size of the node's payload.
// A variable holding an object reference in Delphi is actually
// a pointer, so the node needs enough space to hold 1 pointer.
AddNodesToTree(nil, Root);
When using objects, different nodes in your Virtual Tree may have different types of objects associated with them. In our example we're only adding nodes of TNode type, but in the real world you might have nodes of types TContact, TContactCategory, TRecentCall, all in one VT. You'll use the is operator to check the actual type of the object in the VT node like this:
procedure TForm1.VTGetText(Sender: TBaseVirtualTree; Node: PVirtualNode;
Column: TColumnIndex; TextType: TVSTTextType; var CellText: string);
var PayloadObject:TObject;
Node: TNode;
Contact : TContact;
ContactCategory : TContactCategory;
begin
PayloadObject := TObject(VT.GetNodeData(Node)^); // Extract the payload of the node as a TObject so
// we can check it's type before proceeding.
if not Assigned(PayloadObject) then
CellText := 'Bug: Node payload not assigned'
else if PayloadObject is TNode then
begin
Node := TNode(PayloadObject); // We know it's a TNode, assign it to the proper var so we can easily work with it
CellText := Node.Name;
end
else if PayloadObject is TContact then
begin
Contact := TContact(PayloadObject);
CellText := Contact.FirstName + ' ' + Contact.LastName + ' (' + Contact.PhoneNumber + ')';
end
else if PayloadObject is TContactCategory then
begin
ContactCategory := TContactCategory(PayloadObject);
CellText := ContactCategory.CategoryName + ' (' + IntToStr(ContactCategory.Contacts.Count) + ' contacts)';
end
else
CellText := 'Bug: don''t know how to extract CellText from ' + PayloadObject.ClassName;
end;
And here's an example why to store VirtualNode pointer to your node instances:
procedure TForm1.ButtonModifyClick(Sender: TObject);
begin
Root.Sub[0].Sub[0].Name := 'Someone else'; // I'll modify the node itself
VT.InvalidateNode(Root.Sub[0].Sub[0].VTNode); // and invalidate the tree; when displayed again, it will
// show the updated text.
end;
You know have an working example for a simple tree data structure. You'll need to "grow" this data structure to suite your needs: the possibilities are endless! To give you some ideas, directions to explore:
You can turn the Name:string into a virtual method GetText:string;virtual and then create specialized descendants of TNode that override GetText to provide specialized behavior.
Create a TNode.AddPath(Path:string; ID:Integer) that allows you to do Root.AddPath('Contacts\Abraham', 1); - that is, a method that automatically creates all intermediary nodes to the final node, to allow easy creation of the tree.
Include an PVirtualNode into TNode itself so you can check rather the Node is "checked" in the Virtual Tree. This would be a bridge of the data-GUI separation.
I believe you will be best served by finding an existing library containing a general tree implementation which you can then re-use to serve your needs.
To give you an idea why, here is some code I wrote to illustrate the most simple operation on the most simple tree structure imaginable.
type
TNode = class
Parent: TNode;
NextSibling: TNode;
FirstChild: TNode;
end;
TTree = class
Root: TNode;
function AddNode(Parent: TNode): TNode;
end;
function TTree.AddNode(Parent: TNode);
var
Node: TNode;
begin
Result := TNode.Create;
Result.Parent := Parent;
Result.NextSibling := nil;
Result.FirstChild := nil;
//this may be the first node in the tree
if not Assigned(Root) then begin
Assert(not Assigned(Parent));
Root := Result;
exit;
end;
//this may be the first child of this parent
if Assigned(Parent) and not Assigned(Parent.FirstChild) then begin
Parent.FirstChild := Result;
end;
//find the previous sibling and assign its next sibling to the new node
if Assigned(Parent) then begin
Node := Parent.FirstChild;
end else begin
Node := Root;
end;
if Assigned(Node) then begin
while Assigned(Node.NextSibling) do begin
Node := Node.NextSibling;
end;
Node.NextSibling := Result;
end;
end;
Note: I have not tested this code and so cannot vouch for its correctness. I expect it has defects.
All this does is add an new node to the tree. It gives you little control over where in the tree the node is added. If simply adds a new node as the last sibling of a specified parent node.
To take this sort of approach you would likely need to deal with:
Inserting after a specified sibling. Actually this is quite a simple variant of the above.
Removing a node. This is somewhat more complex.
Moving existing nodes within the tree.
Walking the tree.
Connecting the tree to your VST.
It's certainly feasible to do this, but you may be better advised to find a 3rd party library that already implements the functionality.
I asked similar question here. I didn't got any useful answers so I decide to make my own implementation, which you can find here.
EDIT:
I'll try to post example how you could use my data structure:
uses
svCollections.GenericTrees;
Declare your data type:
type
TMainData = record
Name: string;
ID: Integer;
end;
Declare your main data tree object somewhere in your code:
MyTree: TSVTree<TMainData>;
Create it (and do not forget to free later):
MyTree: TSVTree<TMainData>.Create(False);
Assign your VirtualStringTree to our data structure:
MyTree.VirtualTree := VST;
Then you can init your data tree with some values:
procedure TForm1.BuildStructure(Count: Integer);
var
i, j: Integer;
svNode, svNode2: TSVTreeNode<TMainData>;
Data: TMainData;
begin
MyTree.BeginUpdate;
try
for i := 0 to Count - 1 do
begin
Data.Name := Format('Root %D', [i]);
Data.ID := i;
svNode := MyTree.AddChild(nil, Data);
for j:= 0 to 10 - 1 do
begin
Data.Name := Format('Child %D', [j]);
Data.ID := j;
svNode2 := MyTree.AddChild(svNode, Data);
end;
end;
finally
MyTree.EndUpdate;
end;
end;
And set VST events to display your data:
procedure TForm1.vt1InitChildren(Sender: TBaseVirtualTree; Node: PVirtualNode;
var ChildCount: Cardinal);
var
svNode: TSVTreeNode<TMainData>;
begin
svNode := MyTree.GetNode(Sender.GenerateIndex(Node));
if Assigned(svNode) then
begin
ChildCount := svNode.FChildCount;
end;
end;
procedure TForm1.vt1InitNode(Sender: TBaseVirtualTree; ParentNode,
Node: PVirtualNode; var InitialStates: TVirtualNodeInitStates);
var
svNode: TSVTreeNode<TMainData>;
begin
svNode := MyTree.GetNode(Sender.GenerateIndex(Node));
if Assigned(svNode) then
begin
//if TSVTree<TTestas> is synced with Virtual Treeview and we are building tree by
// setting RootNodeCount, then we must set svNode.FVirtualNode := Node to
// have correct node references
svNode.FVirtualNode := Node; // Don't Forget!!!!
if svNode.HasChildren then
begin
Include(InitialStates, ivsHasChildren);
end;
end;
end;
//display info how you like, I simply get name and ID values
procedure TForm1.vt1GetText(Sender: TBaseVirtualTree; Node: PVirtualNode;
Column: TColumnIndex; TextType: TVSTTextType; var CellText: string);
var
svNode: TSVTreeNode<TMainData>;
begin
svNode := MyTree.GetNode(Sender.GenerateIndex(Node));
if Assigned(svNode) then
begin
CellText := Format('%S ID:%D',[svNode.FValue.Name, svNode.FValue.ID]);
end;
end;
At this point you work only with your MyTree data structure and all the changes made to it will be reflected in your assigned VST. You then can always save (and load) underlying structure to stream or file. Hope this helps.
If I understand correctly, you need a datastructure for your tree. Each individual node requires a record to hold its data. But the underlying heirarchy can be managed in a few different ways. Im guessing this is all to be managed in some sort of database - This has already been talked about on this site, so i will point you to:
Implementing a hierarchical data structure in a database
and here:
What is the most efficient/elegant way to parse a flat table into a tree?
and here:
SQL - How to store and navigate hierarchies?
Nested Set Model:
http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/
If you are using recent versions of Delphi that supports Generics, check GenericTree
Delphi has generics nowadays. I just invented a very nice tree data structure. Not gonna give code away just yet, not really an open source person, maybe in the near future though, also other reasons see below.
But I will give some hints on how to re-create it:
Assuming all your nodes can contain the same data structure (which seems to be the case from above, a string, an id, and then links.
The ingredients you need to re-create this is the following:
Generics
A generic type T
This type T needs to be constrained to class and constructor as follows:
<T : class, constructor> (In your case replace class with record, untested, but may also work)
two fields: node array of self (hint hint), data : T;
A property
Not just any propery, a default property ;)
A getter.
A recursive constructor with depth and child.
Some if statement to stop the construction.
And ofcourse SetLength to create the links/nodes and calling some creates in a for loop and then some subtraction of something ;)
Given all of you enough hints, would be fun and interesting to see if anybody can re-create it, otherwise I might just as well patent it, just kidding, not gonna throw money against it, might expand the class though with other facilities.
The class allocates all nodes during construction like a true data structure... noting with add and remove and such, at least not for now.
Now comes the most interesting and funny aspect of this (secret) design, something I kinda wanted and is now a reality. I can now write code as follows:
TGroup is just an example can be anything as long as it's a class in my case.
In this case it's just a class with mString
var
mGroupTree : TTree<TGroup>;
procedure Main;
var
Depth : integer;
Childs : integer;
begin
Depth := 2;
Childs := 3;
mGroupTree := TTree<TGroup>.Create( Depth, Childs );
mGroupTree.Data.mString := 'Basket'; // notice how nice this root is ! ;)
mGroupTree[0].Data.mString := 'Apples';
mGroupTree[1].Data.mString := 'Oranges';
mGroupTree[2].Data.mString := 'Bananas';
mGroupTree[0][0].Data.mString := 'Bad apple';
mGroupTree[0][1].Data.mString := 'Average apple';
mGroupTree[0][2].Data.mString := 'Good apple';
mGroupTree[1][0].Data.mString := 'Tiny orange';
mGroupTree[1][1].Data.mString := 'Medium orange';
mGroupTree[1][2].Data.mString := 'Big orange';
mGroupTree[2][0].Data.mString := 'Straight banana';
mGroupTree[2][1].Data.mString := 'Curved banana';
mGroupTree[2][2].Data.mString := 'Crooked banana';
Now what you may notice from this actual test code is that it allows "array expansion" like I have rarely seen thanks to this property, which self-references sort of...
So [] [] is depth 2.
[][][] would be depth 3.
I am still evaluating the use of this.
One potential problem is Delphi has no real technique to auto-expand these arrays, though none I have yet found and are statisfied with.
I would like a technique where I can write some code which can go to any depth level:
[0][0][0][0][0]
Not yet sure how to do that... simpelst option is "recursion".
real example:
procedure DisplayString( Depth : string; ParaTree : TTree<TGroup>);
var
vIndex : integer;
begin
if ParaTree <> nil then
begin
// if ParaTree.Data.mString <> '' then
begin
writeln( ParaTree.Data.mString );
Depth := Depth + ' ';
for vIndex := 0 to ParaTree.Childs-1 do
begin
DisplayString( Depth, ParaTree[vIndex] );
end;
end;
end;
end;
Kinda interesting isn't it.
Still exploring it's usefullness for "real applications" and if I want to go with recursion or not ;)
Maybe some day I will open source all of my code. I am close to 40 years old, when I go beyond 40, from 39 to 40, I was kinda planning on going open source. Still 4 months away from 40 =D
(I must say this is the first time I am impressed by Generics, tested it long ago, it was super buggy back then and maybe design-wise unusable, but now with the bugs fixed and constrained generics, it's very impressive in latest Delphi Toyko 10.2.3 version august 2018 ! ;) :))
I am just scratching the surface of what is impossible with latest Delphi tech, maybe with anonymous methods writing recursive routines to process this data structure might become a bit easier, also maybe parallel processing might come into consideration, Delphi help mentions this for anonymous methods.
Bye,
Skybuck.

Resources