[This is an updated version of a question posted earlier, the previous title was Selecting node by index in Delphi’s Virtual Treeview.]
After the better part of a day, I believe I've got the Virtual Treeview component (powerful but complex) working in a simple two table data aware fashion.
Now, I'm trying to simply select the 1,512th (for instance) of the top-level nodes. I can't see any way to do this other than getting the first top-level node and then calling GetNextSibling 1,511 in a loop.
This seems needlessly involved. Is there a simpler way?
UPDATE
Because initializing the nodes in my tree requires database access, initializing all the nodes at startup is not feasible. When the user starts with form with no record already selected, that's fine. As the user scrolls around the tree, enough nodes are populated to display the current window into the tree and the performance is fine.
When the user starts the form in dialog mode with a database record already selected, I must advance the tree to that node before the user sees the form. This is a problem because, if the record is towards the end of the tree, it can take ten seconds as I walk the tree from the first node. Each time I can GetNextSibling(), a node is initialized, even though the vast majority of those nodes are not displayed to the user. I would prefer to defer the initialization of those nodes to the point at which they become visible to the user.
I know that there must be a better way, because if I open the tree without a record selected and use the vertical scroll bar to move, in a single operation, to the middle of the tree then the correct nodes are displayed without having to initialize the nodes I skipped over.
This is the effect I'd like to achieve when opening the tree with a record selected. I know the index of the node I want to go to, but if I can't get there by index I could do a binary search on the tree assuming I can jump some number of nodes backwards and forwards (similar to scrolling directly to the middle of the tree).
Alternatively, perhaps there is some State setting I can make to the tree view that will leave the intermediate nodes uninitialized as I traverse the grid. I've tried Begin/End Update and that doesn't seem to do the trick.
To get a node's sibling without initializing it, just use the NextSibling pointer (see declaration of TVirtualNode).
The tree control is structured just like classical trees you would learn about in a computer-science class. The only way to get from the root of the tree to the 1512th child is to walk the links one by one. Whether you do it yourself or you use a method of the tree control, it still has to be done that way. I don't see anything provided in the control itself, so you could use this function:
function GetNthNextSibling(Node: PVirtualNode; N: Cardinal;
Tree: TBaseVirtualTree = nil): PVirtualNode;
begin
if not Assigned(Tree) then
Tree := TreeFromNode(Node);
Result := Node;
while Assigned(Result) and (N > 0) do begin
Dec(N);
Result := Tree.GetNextSibling(Result);
end;
end;
If you find yourself doing that often, you might want to make yourself an index. It could be as simple as making an array of PVirtualNode pointers and storing all the top-level values in it, so you can just read the 1512th value out of it. The tree control has no need for such a data structure itself, so it doesn't maintain one.
You might also reconsider whether you need a data structure like that. Do you really need to access the nodes by index like that? Or could instead maintain a PVirtualNode pointer, so its position relative to the rest of the nodes in the tree no longer matters (meaning you can, for example, sort them without losing reference to the node you wanted)?
You write in your update:
I know that there must be a better way, because if I open the tree without a record selected and use the vertical scroll bar to move, in a single operation, to the middle of the tree then the correct nodes are displayed without having to initialize the nodes I skipped over.
There is a difference here, because vertical scrolling changes the logical Y coordinate that is displayed at client position 0. The control calculates the offset from scrollbar position and scroll range, and then calculates which node is visible at the top of the control. Nodes are only initialized again when the area that has been scrolled into view needs to be painted.
If you have the Y coordinate of a node you can get the node pointer by calling
function TBaseVirtualTree.GetNodeAt(X, Y: Integer; Relative: Boolean;
var NodeTop: Integer): PVirtualNode;
The Y coordinate of a node is the sum of heights of all previous visible nodes. Assuming you don't have collapsed nodes (so either it is a flat list of records, or all nodes with child nodes are expanded) and they all have the default height this is easy. This code should be a good starting point:
procedure TForm1.SelectTreeNode(AIndex: integer; ACenterNodeInTree: boolean);
var
Y, Dummy: integer;
Node: PVirtualNode;
begin
Y := Round((AIndex + 0.5) * VirtualStringTree1.DefaultNodeHeight);
Node := VirtualStringTree1.GetNodeAt(0, Y, False, Dummy);
if Node <> nil then begin
Assert(Node.Index = AIndex);
VirtualStringTree1.ScrollIntoView(Node, ACenterNodeInTree);
VirtualStringTree1.Selected[Node] := True;
VirtualStringTree1.FocusedNode := Node;
end;
end;
Related
I am trying to optimize a piece of code that holds a bunch of relations betweens objects (pointers).
The situation is the following:
From a series of objects a series of drawings are generated:
TElement (A) ---> generates zero, one or more TDrawing (B)
(Classes are not relevant, so i'll call them A and B)
The current implementation is made with a dictionary
TDictionary< B, A>
so every object from class B comes from a unique A
But in order to identify all the objects of class B that belongs to a given object of class A this is very inefficient, as the dictionary need to be looped to find all the keys with a certain value.
The immediate solution is changing the data structure to the following:
TDictonary < TElements, TList < TDrawing > >
which is not completely efficient as a lot of small memory allocations needs to be done when filling the dictionary and list.
I wonder if someone could give me a hint about a good implementation, I would be grateful!
(sorry for my English)
A quite ugly code but I have come out with this solution:
PLinks:TDictionary<TElement, TObject>;
where the values are either a single object or a list. When adding a value, if the key was not previously set, it is set to the object. Only when trying to add a second value for a key, the value is converted to a list.
procedure AddLink(const source:TElement; object:TDrawing);
var
v:TObject;
list:TList<TDrawing>;
begin
if not PLinks.ContainsKey(source) then
PLinks.Add(source, object)
else begin
v:=PLinks[source];
if v is TList<TDrawing> then
(valor as TList<TDrawing>).add(n)
else begin // v is TDrawing
list:=TList<TDrawing>.Create;
list.Add(v as TDrawing); // adds previous unique item
list.Add(object);
PLinks[source]:=list;
end;
end;
end;
And obviously when looping through the values, it must be checked if the value is a single object or a list.
It's not very elegant, mostly because compile time type checking is lost, but in terms of speed and memory savings it's ok.
Need to add many items (more than 10k) in TComboBox (I know that TComboBox is not supposed to hold many items but its not up to me to change this) without adding duplicates.
So I need to search the full list before adding. I want to avoid TComboBox.items.indexof as I need a binary search but the binary find is not available in TStrings.
So I created a temporary Tstringlist, set sorted to true and used find. But now assigning the temporary Tstringlist back to TComboBox.Items
(myCB.Items.AddStrings(myList))
is really slow as it copies the whole list. Is there any way to move the list instead of copying it? Or any other way to efficient populate my TComboBox?
There is no way to "move" the list into the combo box because the combo box's storage belongs to the internal Windows control implementation. It doesn't know any way to directly consume your Delphi TStringList object. All it offers is a command to add one item to the list, which TComboBox then uses to copy each item from the string list into the system control, one by one. The only way to avoid copying the many thousands of items into the combo box is to avoid the issue entirely, such as by using a different kind of control or by reducing the number of items you need to add.
A list view has a "virtual" mode where you only tell it how many items it should have, and then it calls back to your program when it needs to know details about what's visible on the screen. Items that aren't visible don't occupy any space in the list view's implementation, so you avoid the copying. However, system combo boxes don't have a "virtual" mode. You might be able to find some third-party control that offers that ability.
Reducing the number of items you need to put in the combo box is your next best option, but only you and your colleagues have the domain knowledge necessary to figure out the best way to do that.
As Rudy Velthuis already mentioned in the comments and assuming you are using VCL, the CB_INITSTORAGE message could be an option:
SendMessage(myCB, CB_INITSTORAGE, myList.Count, 20 * myList.Count*sizeof(Integer));
where 20 is your average string length.
Results (on a i5-7200U and 20K items with random length betwen 1 and 50 chars):
without CB_INITSTORAGE: ~ 265ms
with CB_INITSTORAGE: ~215ms
So while you can speed up things a little by preallocating the memory, the bigger issue seems to be the bad user experience. How can a user find the right element in a combobox with such many items?
Notwithstanding that 10k items is crazy to keep in a TComboBox, an efficient strategy here would be to keep a cache in a separate object. For example, declare :
{ use a TDictionary just for storing a hashmap }
FComboStringsDict : TDictionary<string, integer>;
where
procedure TForm1.FormCreate(Sender: TObject);
var
i : integer;
spw : TStopwatch;
begin
FComboStringsDict := TDictionary<string, integer>.Create;
spw := TStopwatch.StartNew;
{ add 10k random items }
for i := 0 to 10000 do begin
AddComboStringIfNotDuplicate(IntToStr(Floor(20000*Random)));
end;
spw.Stop;
ListBox1.Items.Add(IntToStr(spw.ElapsedMilliseconds));
end;
function TForm1.AddComboStringIfNotDuplicate(AEntry: string) : boolean;
begin
result := false;
if not FComboStringsDict.ContainsKey(AEntry) then begin
FComboStringsDict.Add(AEntry, 0);
ComboBox1.Items.Add(AEntry);
result := true;
end;
end;
Adding 10k items initially takes about 0.5s this way.
{ test adding new items }
procedure TForm1.Button1Click(Sender: TObject);
var
spw : TStopwatch;
begin
spw := TStopwatch.StartNew;
if not AddComboString(IntToStr(Floor(20000*Random))) then
ListBox1.Items.Add('Did not add duplicate');
spw.Stop;
ListBox1.Items.Add(IntToStr(spw.ElapsedMilliseconds));
end;
But adding each subsequent item is very fast <1ms. This is a clumsy implementation, but you could easily wrap this behaviour into a custom class. The idea is to keep your data model as separate from the visual component as possible - keep them in sync when adding or removing items but do your heavy searches on the dictionary where the lookup is fast. Removing items would still rely on .IndexOf.
Let's set the context/limitations:
A linked-list consists of Node objects.
Nodes only have a reference to their next node.
A reference to the list is only a reference to the head Node object.
No preprocessing or indexing has been done on the linked-list other than construction (there are no other references to internal nodes or statistics collected, i.e. length).
The last node in the list has a null reference for its next node.
Below is some code for my proposed solution.
Node cursor = head;
Node middle = head;
while (cursor != null) {
cursor = cursor.next;
if (cursor != null) {
cursor = cursor.next;
middle = middle.next;
}
}
return middle;
Without changing the linked-list architecture (not switching to a doubly-linked list or storing a length variable), is there a more efficient way to find the middle element of singly-linked list?
Note: When this method finds the middle of an even number of nodes, it always finds the left middle. This is ideal as it gives you access to both, but if a more efficient method will always find the right middle, that's fine, too.
No, there is no more efficient way, given the information you have available to you.
Think about it in terms of transitions from one node to the next. You have to perform N transitions to work out the list length. Then you have to perform N/2 transitions to find the middle.
Whether you do this as a full scan followed by a half scan based on the discovered length, or whether you run the cursor (at twice speed) and middle (at normal speed) pointers in parallel is not relevant here, the total number of transitions remains the same.
The only way to make this faster would be to introduce extra information to the data structure which you've discounted but, for the sake of completeness, I'll include it here. Examples would be:
making it a doubly-linked list with head and tail pointers, so you could find it in N transitions by "squeezing" in from both ends to the middle. That doubles the storage requirements for pointers however so may not be suitable.
having a skip list with each node pointing to both it's "child" and its "grandchild". This would speed up the cursor transitions resulting in only about N in total (that's N/2 for each of cursor and middle). Like the previous point, there's an extra pointer per node required for this.
maintaining the length of the list separately so you could find the middle in N/2 transitions.
same as the previous point but caching the middle node for added speed under certain circumstances.
That last point bears some extra examination. Like many optimisations, you can trade space for time and the caching shows one way to do it.
First, maintain the length of the list and a pointer to the middle node. The length is initially zero and the middle pointer is initially set to null.
If you're ever asked for the middle node when the length is zero, just return null. That makes sense because the list is empty.
Otherwise, if you're asked for the middle node and the pointer is null, it must be because you haven't cached the value yet.
In that case, calculate it using the length (N/2 transitions) and then store that pointer for later, before returning it.
As an aside, there's a special case here when adding to the end of the list, something that's common enough to warrant special code.
When adding to the end when the length is going from an even number to an odd number, just set middle to middle->next rather than setting it back to null.
This will save a recalculation and works because you (a) have the next pointers and (b) you can work out how the middle "index" (one-based and selecting the left of a pair as per your original question) changes given the length:
Length Middle(one-based)
------ -----------------
0 none
1 1
2 1
3 2
4 2
5 3
: :
This caching means, provided the list doesn't change (or only changes at the end), the next time you need the middle element, it will be near instantaneous.
If you ever delete a node from the list (or insert somewhere other than the end), set the middle pointer back to null. It will then be recalculated (and re-cached) the next time it's needed.
So, for a minimal extra storage requirement, you can gain quite a bit of speed, especially in situations where the middle element is needed more often than the list is changed.
Well, I'm using VirtualStringTree to create kind of a process manager...
I run into trouble because of updating the tree with a timer set to 1000ms (cpu usage is too high for my application retrieving a lot of data (filling about 20 columns).
So I wonder how would one build kind of a cache system so I can update the tree only when something changed which I guess seems to be the key decrementing the cpu usage for my application a lot?
Snip:
type
TProcessNodeType = (ntParent, ntDummy);
PProcessData = ^TProcessData;
TProcessData = record
pProcessName : String;
pProcessID,
pPrivMemory,
pWorkingSet,
pPeakWorkingSet,
pVirtualSize,
pPeakVirtualSize,
pPageFileUsage,
pPeakPageFileUsage,
pPageFaults : Cardinal;
pCpuUsageStr: string;
pIOTotal: Cardinal;
...
end;
If my application starts I fill the tree with all running processes.
Remember this is called only once, later when the application runs I got notified of new processes or processes which are terminated via wmi so I dont need to call the following procedure in the timer later to update the tree...
procedure FillTree;
begin
var
NodeData: PProcessData;
Node: PVirtualNode;
ParentNode: PVirtualNode;
ChildNode: PVirtualNode;
Process: TProcessItem;
I : Integer;
begin
ProcessTree.BeginUpdate;
for I := 0 to FRunningProcesses.Count - 1 do
begin
Process := FRunningProcesses[i];
NodeData^.pProcessID := ProcessItem.ProcessID;
NodeData^.pProcessName := ProcessItem.ProcessName;
...
I have a Class which will retrieve all the data I want and store it into the tree like:
var
FRunningProcesses: TProcessRunningProcesses;
So if I want to enumerate all running processes I just give it a call like:
// clears all data inside the class and refills everything with the new data...
FRunningProcesses.UpdateProcesses;
The problem starts here while I enumerate everything and not only data which had changed which is quite cpu intensive:
procedure TMainForm.UpdateTimerTimer(Sender: TObject);
var
NodeData: PProcessData;
Node : PVirtualNode;
Process: TProcessItem;
I: Integer;
begin
for I := 0 to FRunningProcesses.Count - 1 do
begin
Application.ProcessMessages;
Process := FRunningProcesses[I];
// returns PVirtualNode if the node is found inside the tree
Node := FindNodeByPID(Process.ProcessID);
if not(assigned(Node)) then
exit;
NodeData := ProcessVst.GetNodeData(Node);
if not(assigned(NodeData)) then
exit;
// now starting updating the tree
// NodeData^.pWorkingsSet := Process.WorkingsSet;
....
Basically the timer is only needed for cpu usage and all memory informations I can retrieve from a process like:
Priv.Memory
Working Set
Peak Working Set
Virtual Size
PageFile Usage
Peak PageFile Usage
Page Faults
Cpu Usage
Thread Count
Handle Count
GDI Handle Count
User Handle Count
Total Cpu Time
User Cpu Time
Kernel Cpu Time
So I think the above data must be cached and compared somehow if its changed or not just wonder how and what will be most efficient?
You need only update the data in nodes which are currently are visible.
you can use vst.getfirstvisible vst.getnextvisible to iterate thru these nodes.
the second way is also easy.
use objects instead of the record. sample code of object usage
use getters for the different values.
those getters query the processes for the values.
maybe you need here a limit. refresh data only every second.
now you only need to set the vst into an invalidated status every second.
vst.invalidate
this forced the vst to repaint the visible area.
but all this works only if your data is not sorted by any changing values.
if this necessary you need to update all record and this is your bottle neck - i think.
remember COM and WMI are much slower than pure API.
avoid (slow) loops and use a profiler to find the slow parts.
I'd recommend you to have your VT's node data point directly to TProcessItem.
Pro's:
Get rid of FindNodeByPID. Just update all the items from
FRunningProcesses and then call VT.Refresh. When the process is
terminated, delete corresponding item from FRunningProcesses.
Currently you have quite expensive search in FindNodeByPID where
you loop through all VT nodes, retrieve their data and check for
PID.
Get rid of Process := FRunningProcesses[I] where you have
unnecessary data copy of the whole TProcessData record (btw, that
should be done anyway, use pointers instead).
Get rid of the whole // now starting updating the tree block.
In general, by this change you decrease excess entities what is very good for application updating and debugging.
Con's:
You'll have to keep VT and FRunningProcesses in sync. But that's quite trivial.
is that possible to hide specific nodes in VirtualStringTree?
I'm implementing "filtering" feature (the VST acts as a list with columns), and I'd like to avoid reloading content each time the filter is changed - instead, much faster would be to tell VST not to render specific items ... any solutions?
VirtualTree.IsVisible[Node] := False;
There are problem using .IsVisible[] or .IsFiltered[] and it is that is very slow, i've probe filter in a tree with amoung of 25,000 nodes and is too slow.
I've found one aproach that is faster and solves the problem with scrollbar size when using Include(Node.states,vsFiltered) or (Node.States,vsVisible) is used, it consists on change manually the Node.TotalHeight value acording with the number of visible nodes (not Filtered).
For example i'm filtering 25,000 nodes and the code i was using is the follow :
procedure TFC_Articulo.Filtrar(Filtro:String);
var
Node:PVirtualNode;
Data:PArticulo;
begin
Node := TV.GetFirst;
TV.RootNode.TotalHeight:=TV.DefaultNodeHeight; // The Trick
while Assigned(Node) do
begin
Data:=TV.GetNodeData(Node);
Exclude(Node.States,vsFiltered); // By default all nodes wil be Visible
if ComparationHereForDetermineFiltering then
Include(Node.States,vsFiltered) // This node wil be filtered
else
Inc(TV.RootNode.TotalHeight,Node.NodeHeight); // Determine the Height of scrollbar
Node:=TV.GetNext(Node);
end;
TV.RootNode.TotalHeight:=TV.RootNode.TotalHeight+TV.BottomSpace;
TV.UpdateScrollBars(True);
end;
Hope this helps
Sorry bad english...