is that possible to hide specific nodes in VirtualStringTree?
I'm implementing "filtering" feature (the VST acts as a list with columns), and I'd like to avoid reloading content each time the filter is changed - instead, much faster would be to tell VST not to render specific items ... any solutions?
VirtualTree.IsVisible[Node] := False;
There are problem using .IsVisible[] or .IsFiltered[] and it is that is very slow, i've probe filter in a tree with amoung of 25,000 nodes and is too slow.
I've found one aproach that is faster and solves the problem with scrollbar size when using Include(Node.states,vsFiltered) or (Node.States,vsVisible) is used, it consists on change manually the Node.TotalHeight value acording with the number of visible nodes (not Filtered).
For example i'm filtering 25,000 nodes and the code i was using is the follow :
procedure TFC_Articulo.Filtrar(Filtro:String);
var
Node:PVirtualNode;
Data:PArticulo;
begin
Node := TV.GetFirst;
TV.RootNode.TotalHeight:=TV.DefaultNodeHeight; // The Trick
while Assigned(Node) do
begin
Data:=TV.GetNodeData(Node);
Exclude(Node.States,vsFiltered); // By default all nodes wil be Visible
if ComparationHereForDetermineFiltering then
Include(Node.States,vsFiltered) // This node wil be filtered
else
Inc(TV.RootNode.TotalHeight,Node.NodeHeight); // Determine the Height of scrollbar
Node:=TV.GetNext(Node);
end;
TV.RootNode.TotalHeight:=TV.RootNode.TotalHeight+TV.BottomSpace;
TV.UpdateScrollBars(True);
end;
Hope this helps
Sorry bad english...
Related
Need to add many items (more than 10k) in TComboBox (I know that TComboBox is not supposed to hold many items but its not up to me to change this) without adding duplicates.
So I need to search the full list before adding. I want to avoid TComboBox.items.indexof as I need a binary search but the binary find is not available in TStrings.
So I created a temporary Tstringlist, set sorted to true and used find. But now assigning the temporary Tstringlist back to TComboBox.Items
(myCB.Items.AddStrings(myList))
is really slow as it copies the whole list. Is there any way to move the list instead of copying it? Or any other way to efficient populate my TComboBox?
There is no way to "move" the list into the combo box because the combo box's storage belongs to the internal Windows control implementation. It doesn't know any way to directly consume your Delphi TStringList object. All it offers is a command to add one item to the list, which TComboBox then uses to copy each item from the string list into the system control, one by one. The only way to avoid copying the many thousands of items into the combo box is to avoid the issue entirely, such as by using a different kind of control or by reducing the number of items you need to add.
A list view has a "virtual" mode where you only tell it how many items it should have, and then it calls back to your program when it needs to know details about what's visible on the screen. Items that aren't visible don't occupy any space in the list view's implementation, so you avoid the copying. However, system combo boxes don't have a "virtual" mode. You might be able to find some third-party control that offers that ability.
Reducing the number of items you need to put in the combo box is your next best option, but only you and your colleagues have the domain knowledge necessary to figure out the best way to do that.
As Rudy Velthuis already mentioned in the comments and assuming you are using VCL, the CB_INITSTORAGE message could be an option:
SendMessage(myCB, CB_INITSTORAGE, myList.Count, 20 * myList.Count*sizeof(Integer));
where 20 is your average string length.
Results (on a i5-7200U and 20K items with random length betwen 1 and 50 chars):
without CB_INITSTORAGE: ~ 265ms
with CB_INITSTORAGE: ~215ms
So while you can speed up things a little by preallocating the memory, the bigger issue seems to be the bad user experience. How can a user find the right element in a combobox with such many items?
Notwithstanding that 10k items is crazy to keep in a TComboBox, an efficient strategy here would be to keep a cache in a separate object. For example, declare :
{ use a TDictionary just for storing a hashmap }
FComboStringsDict : TDictionary<string, integer>;
where
procedure TForm1.FormCreate(Sender: TObject);
var
i : integer;
spw : TStopwatch;
begin
FComboStringsDict := TDictionary<string, integer>.Create;
spw := TStopwatch.StartNew;
{ add 10k random items }
for i := 0 to 10000 do begin
AddComboStringIfNotDuplicate(IntToStr(Floor(20000*Random)));
end;
spw.Stop;
ListBox1.Items.Add(IntToStr(spw.ElapsedMilliseconds));
end;
function TForm1.AddComboStringIfNotDuplicate(AEntry: string) : boolean;
begin
result := false;
if not FComboStringsDict.ContainsKey(AEntry) then begin
FComboStringsDict.Add(AEntry, 0);
ComboBox1.Items.Add(AEntry);
result := true;
end;
end;
Adding 10k items initially takes about 0.5s this way.
{ test adding new items }
procedure TForm1.Button1Click(Sender: TObject);
var
spw : TStopwatch;
begin
spw := TStopwatch.StartNew;
if not AddComboString(IntToStr(Floor(20000*Random))) then
ListBox1.Items.Add('Did not add duplicate');
spw.Stop;
ListBox1.Items.Add(IntToStr(spw.ElapsedMilliseconds));
end;
But adding each subsequent item is very fast <1ms. This is a clumsy implementation, but you could easily wrap this behaviour into a custom class. The idea is to keep your data model as separate from the visual component as possible - keep them in sync when adding or removing items but do your heavy searches on the dictionary where the lookup is fast. Removing items would still rely on .IndexOf.
I'm working on modernizing and fixing bugs in the codebase of a Delphi 4-era program written by someone else. A lot of the code is kinda scary by modern standards, and I can't help but wonder if some of the things I'm seeing are there because the original author didn't know about certain standard library features, or if they weren't available.
One of the more obnoxious "patterns" I see all over the app looks like this:
table := TClientDataset.Create;
with table do
begin
CloneCursor(dmDatabase.OriginalTable, false, true);
filtered := true;
active := true;
first;
while not EOF do
begin
if fieldByName('whatever').AsString = 'some criteria' then break;
next;
end;
if EOF then exit;
//do something based on the current row of the dataset
table.free;
end;
Almost every one of these groups could be replaced by a one-line call to either Lookup or Locate on the original dataset, with no need for an intermediary CDS at all. That makes me wonder, were these methods available back in the D4 days? When were Lookup and Locate first added?
Lookup and Locate were introduced in Delphi 2. It looks like the original author simply didn't take advantage of them.
Seems the Original programmer wanted to make sure that the row pointer is not changed
at all. Doing Locate (or Lookup) would change the row pointer, provoking all kinds
of data events (Datasource.OnDataChange, Dataset.AfterScroll and so on).
Doing the search with TClientDataset.CloneCursor, none of the these events ger triggered on the dmDatabase.OriginalTable and there's no need to reload the data from database.
Seems to me that is the intention. TClientDataset was presented on D3. And cloned cursors are a kind
of advanced feature - and need the dmDatabase.OriginalTable to be a CDS too.
Well, I'm using VirtualStringTree to create kind of a process manager...
I run into trouble because of updating the tree with a timer set to 1000ms (cpu usage is too high for my application retrieving a lot of data (filling about 20 columns).
So I wonder how would one build kind of a cache system so I can update the tree only when something changed which I guess seems to be the key decrementing the cpu usage for my application a lot?
Snip:
type
TProcessNodeType = (ntParent, ntDummy);
PProcessData = ^TProcessData;
TProcessData = record
pProcessName : String;
pProcessID,
pPrivMemory,
pWorkingSet,
pPeakWorkingSet,
pVirtualSize,
pPeakVirtualSize,
pPageFileUsage,
pPeakPageFileUsage,
pPageFaults : Cardinal;
pCpuUsageStr: string;
pIOTotal: Cardinal;
...
end;
If my application starts I fill the tree with all running processes.
Remember this is called only once, later when the application runs I got notified of new processes or processes which are terminated via wmi so I dont need to call the following procedure in the timer later to update the tree...
procedure FillTree;
begin
var
NodeData: PProcessData;
Node: PVirtualNode;
ParentNode: PVirtualNode;
ChildNode: PVirtualNode;
Process: TProcessItem;
I : Integer;
begin
ProcessTree.BeginUpdate;
for I := 0 to FRunningProcesses.Count - 1 do
begin
Process := FRunningProcesses[i];
NodeData^.pProcessID := ProcessItem.ProcessID;
NodeData^.pProcessName := ProcessItem.ProcessName;
...
I have a Class which will retrieve all the data I want and store it into the tree like:
var
FRunningProcesses: TProcessRunningProcesses;
So if I want to enumerate all running processes I just give it a call like:
// clears all data inside the class and refills everything with the new data...
FRunningProcesses.UpdateProcesses;
The problem starts here while I enumerate everything and not only data which had changed which is quite cpu intensive:
procedure TMainForm.UpdateTimerTimer(Sender: TObject);
var
NodeData: PProcessData;
Node : PVirtualNode;
Process: TProcessItem;
I: Integer;
begin
for I := 0 to FRunningProcesses.Count - 1 do
begin
Application.ProcessMessages;
Process := FRunningProcesses[I];
// returns PVirtualNode if the node is found inside the tree
Node := FindNodeByPID(Process.ProcessID);
if not(assigned(Node)) then
exit;
NodeData := ProcessVst.GetNodeData(Node);
if not(assigned(NodeData)) then
exit;
// now starting updating the tree
// NodeData^.pWorkingsSet := Process.WorkingsSet;
....
Basically the timer is only needed for cpu usage and all memory informations I can retrieve from a process like:
Priv.Memory
Working Set
Peak Working Set
Virtual Size
PageFile Usage
Peak PageFile Usage
Page Faults
Cpu Usage
Thread Count
Handle Count
GDI Handle Count
User Handle Count
Total Cpu Time
User Cpu Time
Kernel Cpu Time
So I think the above data must be cached and compared somehow if its changed or not just wonder how and what will be most efficient?
You need only update the data in nodes which are currently are visible.
you can use vst.getfirstvisible vst.getnextvisible to iterate thru these nodes.
the second way is also easy.
use objects instead of the record. sample code of object usage
use getters for the different values.
those getters query the processes for the values.
maybe you need here a limit. refresh data only every second.
now you only need to set the vst into an invalidated status every second.
vst.invalidate
this forced the vst to repaint the visible area.
but all this works only if your data is not sorted by any changing values.
if this necessary you need to update all record and this is your bottle neck - i think.
remember COM and WMI are much slower than pure API.
avoid (slow) loops and use a profiler to find the slow parts.
I'd recommend you to have your VT's node data point directly to TProcessItem.
Pro's:
Get rid of FindNodeByPID. Just update all the items from
FRunningProcesses and then call VT.Refresh. When the process is
terminated, delete corresponding item from FRunningProcesses.
Currently you have quite expensive search in FindNodeByPID where
you loop through all VT nodes, retrieve their data and check for
PID.
Get rid of Process := FRunningProcesses[I] where you have
unnecessary data copy of the whole TProcessData record (btw, that
should be done anyway, use pointers instead).
Get rid of the whole // now starting updating the tree block.
In general, by this change you decrease excess entities what is very good for application updating and debugging.
Con's:
You'll have to keep VT and FRunningProcesses in sync. But that's quite trivial.
[This is an updated version of a question posted earlier, the previous title was Selecting node by index in Delphi’s Virtual Treeview.]
After the better part of a day, I believe I've got the Virtual Treeview component (powerful but complex) working in a simple two table data aware fashion.
Now, I'm trying to simply select the 1,512th (for instance) of the top-level nodes. I can't see any way to do this other than getting the first top-level node and then calling GetNextSibling 1,511 in a loop.
This seems needlessly involved. Is there a simpler way?
UPDATE
Because initializing the nodes in my tree requires database access, initializing all the nodes at startup is not feasible. When the user starts with form with no record already selected, that's fine. As the user scrolls around the tree, enough nodes are populated to display the current window into the tree and the performance is fine.
When the user starts the form in dialog mode with a database record already selected, I must advance the tree to that node before the user sees the form. This is a problem because, if the record is towards the end of the tree, it can take ten seconds as I walk the tree from the first node. Each time I can GetNextSibling(), a node is initialized, even though the vast majority of those nodes are not displayed to the user. I would prefer to defer the initialization of those nodes to the point at which they become visible to the user.
I know that there must be a better way, because if I open the tree without a record selected and use the vertical scroll bar to move, in a single operation, to the middle of the tree then the correct nodes are displayed without having to initialize the nodes I skipped over.
This is the effect I'd like to achieve when opening the tree with a record selected. I know the index of the node I want to go to, but if I can't get there by index I could do a binary search on the tree assuming I can jump some number of nodes backwards and forwards (similar to scrolling directly to the middle of the tree).
Alternatively, perhaps there is some State setting I can make to the tree view that will leave the intermediate nodes uninitialized as I traverse the grid. I've tried Begin/End Update and that doesn't seem to do the trick.
To get a node's sibling without initializing it, just use the NextSibling pointer (see declaration of TVirtualNode).
The tree control is structured just like classical trees you would learn about in a computer-science class. The only way to get from the root of the tree to the 1512th child is to walk the links one by one. Whether you do it yourself or you use a method of the tree control, it still has to be done that way. I don't see anything provided in the control itself, so you could use this function:
function GetNthNextSibling(Node: PVirtualNode; N: Cardinal;
Tree: TBaseVirtualTree = nil): PVirtualNode;
begin
if not Assigned(Tree) then
Tree := TreeFromNode(Node);
Result := Node;
while Assigned(Result) and (N > 0) do begin
Dec(N);
Result := Tree.GetNextSibling(Result);
end;
end;
If you find yourself doing that often, you might want to make yourself an index. It could be as simple as making an array of PVirtualNode pointers and storing all the top-level values in it, so you can just read the 1512th value out of it. The tree control has no need for such a data structure itself, so it doesn't maintain one.
You might also reconsider whether you need a data structure like that. Do you really need to access the nodes by index like that? Or could instead maintain a PVirtualNode pointer, so its position relative to the rest of the nodes in the tree no longer matters (meaning you can, for example, sort them without losing reference to the node you wanted)?
You write in your update:
I know that there must be a better way, because if I open the tree without a record selected and use the vertical scroll bar to move, in a single operation, to the middle of the tree then the correct nodes are displayed without having to initialize the nodes I skipped over.
There is a difference here, because vertical scrolling changes the logical Y coordinate that is displayed at client position 0. The control calculates the offset from scrollbar position and scroll range, and then calculates which node is visible at the top of the control. Nodes are only initialized again when the area that has been scrolled into view needs to be painted.
If you have the Y coordinate of a node you can get the node pointer by calling
function TBaseVirtualTree.GetNodeAt(X, Y: Integer; Relative: Boolean;
var NodeTop: Integer): PVirtualNode;
The Y coordinate of a node is the sum of heights of all previous visible nodes. Assuming you don't have collapsed nodes (so either it is a flat list of records, or all nodes with child nodes are expanded) and they all have the default height this is easy. This code should be a good starting point:
procedure TForm1.SelectTreeNode(AIndex: integer; ACenterNodeInTree: boolean);
var
Y, Dummy: integer;
Node: PVirtualNode;
begin
Y := Round((AIndex + 0.5) * VirtualStringTree1.DefaultNodeHeight);
Node := VirtualStringTree1.GetNodeAt(0, Y, False, Dummy);
if Node <> nil then begin
Assert(Node.Index = AIndex);
VirtualStringTree1.ScrollIntoView(Node, ACenterNodeInTree);
VirtualStringTree1.Selected[Node] := True;
VirtualStringTree1.FocusedNode := Node;
end;
end;
I have a fairly complex and large application that hands loads and loads of data. Is there a speedy way to add items to ComboBox that doesn't take so long? On my P3 3.2ghz, the following snippet takes just under a second to add around 32,000 items. (MasterCIList is a StringList with strings typically 20 - 30 bytes long).
with LookupComboBox do
begin
Items.BeginUpdate;
Items.Clear;
for i := 0 to MasterCIList.Count - 1 do
Items.Add(MasterCIList[i]);
Items.EndUpdate;
end;
Drilling down into the VCL, it appears that in TComboBoxStrings.Add, there is a call to
Result := SendMessage(ComboBox.Handle, CB_ADDSTRING, 0, Longint(PChar(S)));
I'm guessing this is really taking up time (okay, I know it is). Is there another way to populate the Items that is speedier? Any high-speed combo boxes available? I have the TMS components but they seem to be extensions of TComboBox.
For instance, I have the PlusMemo which seems to be a total rewrite of a TMemo. I can easily add a million line in a second to a PlusMemo. A TMemo, I don't think so.
Thank you so much for your time!
Sorry if I'm a nuisance, but I doubt a TComboBox with 32,000 items is even remotely ''usable'' --- I'd say there's a reason why it's slow: it was never meant to do this :)
Would there be a possibility to filter the data, and only load a subset? To be more concrete, in one particular database application I've been working on, the user can search for a person. We let the user type at least 3 or 4 characters of the name, and only then begin to return results in a listbox. This has greatly increased usability of the search form, also greatly speeding up the whole process.
Would you be able to follow a similar approach?
Or, on a completely different take, perhaps you could take a look at the VirtualTreeView component --- either for direct use, or for inspiration.
I agree that 32K items is a ridiculous amount to have in a combobox... That being said, you might try adding the items to a TStringList first and then use a combination of Begin/EndUpdate and AddStrings:
SL := TStringList.Create;
try
// Add your items to the stringlist
ComboBox.Items.BeginUpdate;
try
ComboBox.Items.AddStrings(YourStringList);
finally
ComboBox.Items.EndUpdate;
end;
finally
SL.Free;
end;
The code compiles, but I didn't test it further than that; I've never felt the need to add more than a few dozen items to a combobox or listbox. If any more items might be needed than that, I find a way to filter before populating the list so there are fewer items.
Just out of curiosity, how do you expect a user to sort through that many items to make a decision?
var
Buffer: TStringList;
begin
Buffer := TStringList.Create;
try
// --> Add items to Buffer here <--
ComboBox.Items := Buffer;
finally
FreeAndNil(Buffer);
end;
end;
This is the fastest way we've found to update a visual control.
The VCL does BeginUpdate, Clear, and EndUpdate internally.
If you don't believe me, profile it.
perhaps cmb.Items.Assign(myStringList) will help.
here's a wild idea: i haven't tried it but you might check to see if there's a way to virtually load the combobox by setting the number of items and then owner drawing. please pardon this crazy idea but i think i've heard of this being available somehow. irrelevant: this is how it's done in Palm OS...where the faster way to load the combobox is to not load it all... ;-)
Not an answer, but why on earth would you want 32,000 items in a combo box? That is a terrible way to store that much data.
i agree; it's a bad practice...
It's me again. I'm adding 32,000 items cause I need to. That's one of many controls in my application that has "lots" of items. I need to have that many items. It works just fine looking things up. Perfectly in fact. I'm just trying to optimize things. The users find things just fine since they are in a certain logical order.
Everything I've seem so far with Assign and AddStrings is that they eventually end up in Add with the SendMessage call. So I'll keep looking.
Thanks for the feedback.
use backgroundworker for adding MasterCIList items.after complete adding items use only AddStrings.
procedure TForm2.BackgroundWorker1Work(Worker: TBackgroundWorker);
var
I: Integer;
begin
MasterCIList.BeginUpdate;
try
MasterCIList.Capacity := 32 * 1024; // if derminate count of items
for I := 1 to 32 * 1024 do
begin
MasterCIList.Add(IntToStr(I));
{ if need progess }
if I mod 300 = 0 then
Worker.ReportProgress((I * 100) div MasterCIList.Capacity);
{ if need cancelable }
if (I mod 100 = 0) and Worker.CancellationPending then
Break;
end;
finally
MasterCIList.EndUpdate;
end;
end;
procedure TForm2.BackgroundWorker1WorkComplete(Worker: TBackgroundWorker;
Cancelled: Boolean);
begin
LookupComboBox.Items.AddStrings(MasterCIList );
// AddStrings use beginupdate..endupdate in itself
end;
Maybe you can try this?
"Speeding up adding items to a combobox or listbox"
http://blogs.msdn.com/b/oldnewthing/archive/2004/06/10/152612.aspx
Perhaps you can use a database engine in the back-end and use a data aware component. Then the things will be much more quicker and usable. Perhaps if you'll try to describe what do you try to accomplish we'll help you further. In any case, your UI design is, let's say, odd. And for this perhaps the Embarcadero forums will help you better.
I implement this in a different manner. First i removed the combobox control and take textbox control and assign it autocomplete to custom source where the custom source string collection is 32k items.I get the selected value from a new query on controls validation.
So it can replace combobox functionality. Mostly about 32k items people dont scroll but they keep entering key strokes and is catched by our custom auto complete source..