I want the selected List Data to populate the tree
Here was my attempt towards it
if ListView1.Items[Count].Selected then
begin
Root := ListView1.Items[Count].Caption;
for Itr := TreeView1.Items.Count-1 downto 0 do Begin
if TreeView1.items[itr].Parent.Text = Root then begin
TreeNode := TreeView1.Items[itr].getFirstChild;
Treeview1.Items.AddChild(Treenode,ListView1.Items[Count].SubItems[0]);
break;
end;
end;
end;
But its creating new node and there is an index error.
Kindly help
You haven't provided any details about what your ListView data looks like, or what the TreeView is supposed to look like in relation to that data. But I suspect your code should probably look more like this instead:
if ListView1.Items[Count].Selected then
begin
Root := ListView1.Items[Count].Caption;
for Itr := TreeView1.Items.Count-1 downto 0 do begin
TreeNode := TreeView1.Items[itr];
if TreeNode.Text = Root then begin
TreeView1.Items.AddChild(TreeNode, ListView1.Items[Count].SubItems[0]);
break;
end;
end;
end;
Note that iterating through a TreeView backwards is VERY inefficient. Trees are not inherently indexable, so every time you access a node by index via the TreeView1.Items[] property, it has to start with the first node in the tree and iterate forwards counting nodes until it reaches the specified index. You are repeating that same forward scan for every node you access while going backwards. That is a lot of wasted overhead.
Related
I want to read the entire table from an MS Access file and I'm trying to do it as fast as possible. When testing a big sample I found that the loop counter increases faster when it's reading the top records comparing to last records of the table. Here's a sample code that demonstrates this:
procedure TForm1.Button1Click(Sender: TObject);
const
MaxRecords = 40000;
Step = 5000;
var
I, J: Integer;
Table: TADOTable;
T: Cardinal;
Ts: TCardinalDynArray;
begin
Table := TADOTable.Create(nil);
Table.ConnectionString :=
'Provider=Microsoft.ACE.OLEDB.12.0;'+
'Data Source=BigMDB.accdb;'+
'Mode=Read|Share Deny Read|Share Deny Write;'+
'Persist Security Info=False';
Table.TableName := 'Table1';
Table.Open;
J := 0;
SetLength(Ts, MaxRecords div Step);
T := GetTickCount;
for I := 1 to MaxRecords do
begin
Table.Next;
if ((I mod Step) = 0) then
begin
T := GetTickCount - T;
Ts[J] := T;
Inc(J);
T := GetTickCount;
end;
end;
Table.Free;
// Chart1.SeriesList[0].Clear;
// for I := 0 to Length(Ts) - 1 do
// begin
// Chart1.SeriesList[0].Add(Ts[I]/1000, Format(
// 'Records: %s %d-%d %s Duration:%f s',
// [#13, I * Step, (I + 1)*Step, #13, Ts[I]/1000]));
// end;
end;
And the result on my PC:
The table has two string fields, one double and one integer. It has no primary key nor index field. Why does it happen and how can I prevent it?
I can reproduce your results using an AdoQuery with an MS Sql Server dataset of similar size to yours.
However, after doing a bit of line-profiling, I think I've found the answer to this, and it's slightly counter-intuitive. I'm sure everyone who does
DB programming in Delphi is used to the idea that looping through a dataset tends to be much quicker if you surround the loop by calls to Disable/EnableControls. But who would bother to do that if there are no db-aware controls attached to the dataset?
Well, it turns out that in your situation, even though there are no DB-aware controls, the speed increases hugely if you use Disable/EnableControls regardless.
The reason is that TCustomADODataSet.InternalGetRecord in AdoDB.Pas contains this:
if ControlsDisabled then
RecordNumber := -2 else
RecordNumber := Recordset.AbsolutePosition;
and according to my line profiler, the while not AdoQuery1.Eof do AdoQuery1.Next loop spends 98.8% of its time executing the assignment
RecordNumber := Recordset.AbsolutePosition;
! The calculation of Recordset.AbsolutePosition is hidden, of course, on the "wrong side" of the Recordset interface, but the fact that the time to call it apparently increases the further you go into the recordset makes it reasonable imo to speculate that it's calculated by counting from the start of the recordset's data.
Of course, ControlsDisabled returns true if DisableControls has been called and not undone by a call to EnableControls. So, retest with the loop surrounded by Disable/EnableControls and hopefully you'll get a similar result to mine. It looks like you were right that the slowdown isn't related to memory allocations.
Using the following code:
procedure TForm1.btnLoopClick(Sender: TObject);
var
I: Integer;
T: Integer;
Step : Integer;
begin
Memo1.Lines.BeginUpdate;
I := 0;
Step := 4000;
if cbDisableControls.Checked then
AdoQuery1.DisableControls;
T := GetTickCount;
{.$define UseRecordSet}
{$ifdef UseRecordSet}
while not AdoQuery1.Recordset.Eof do begin
AdoQuery1.Recordset.MoveNext;
Inc(I);
if I mod Step = 0 then begin
T := GetTickCount - T;
Memo1.Lines.Add(IntToStr(I) + ':' + IntToStr(T));
T := GetTickCount;
end;
end;
{$else}
while not AdoQuery1.Eof do begin
AdoQuery1.Next;
Inc(I);
if I mod Step = 0 then begin
T := GetTickCount - T;
Memo1.Lines.Add(IntToStr(I) + ':' + IntToStr(T));
T := GetTickCount;
end;
end;
{$endif}
if cbDisableControls.Checked then
AdoQuery1.EnableControls;
Memo1.Lines.EndUpdate;
end;
I get the following results (with DisableControls not called except where noted):
Using CursorLocation = clUseClient
AdoQuery.Next AdoQuery.RecordSet AdoQuery.Next
.MoveNext + DisableControls
4000:157 4000:16 4000:15
8000:453 8000:16 8000:15
12000:687 12000:0 12000:32
16000:969 16000:15 16000:31
20000:1250 20000:16 20000:31
24000:1500 24000:0 24000:16
28000:1703 28000:15 28000:31
32000:1891 32000:16 32000:31
36000:2187 36000:16 36000:16
40000:2438 40000:0 40000:15
44000:2703 44000:15 44000:31
48000:3203 48000:16 48000:32
=======================================
Using CursorLocation = clUseServer
AdoQuery.Next AdoQuery.RecordSet AdoQuery.Next
.MoveNext + DisableControls
4000:1031 4000:454 4000:563
8000:1016 8000:468 8000:562
12000:1047 12000:469 12000:500
16000:1234 16000:484 16000:532
20000:1047 20000:454 20000:546
24000:1063 24000:484 24000:547
28000:984 28000:531 28000:563
32000:906 32000:485 32000:500
36000:1016 36000:531 36000:578
40000:1000 40000:547 40000:500
44000:968 44000:406 44000:562
48000:1016 48000:375 48000:547
Calling AdoQuery1.Recordset.MoveNext calls directly into the MDac/ADO layer, of
course, whereas AdoQuery1.Next involves all the overhead of the standard TDataSet
model. As Serge Kraikov said, changing the CursorLocation certainly makes a difference and doesn't exhibit the slowdown we noticed, though obviously it's significantly slower than using clUseClient and calling DisableControls. I suppose it depends on exactly what you're trying to do whether you can take advantage of the extra speed of using clUseClient with RecordSet.MoveNext.
When you open a table, ADO dataset internally creates special data structures to navigate dataset forward/backward - "dataset CURSOR". During navigation, ADO stores the list of already visited records to provide bidirectional navigation.
Seems ADO cursor code uses quadratic-time O(n2) algorithm to store this list.
But there are workaround - use server-side cursor:
Table.CursorLocation := clUseServer;
I tested your code using this fix and get linear fetch time - fetching every next chunk of records takes the same time as previous.
PS Some other data access libraries provides special "unidirectional" datasets - this datasets can traverse only forward and don't even store already traversed records - you get constant memory consumption and linear fetch time.
DAO is native to Access and (IMHO) is typically faster.
Whether or not you switch, use the GetRows method. Both DAO and ADO support it.
There is no looping. You can dump the entire recordset into an array with a couple of lines of code. Air code:
yourrecordset.MoveLast
yourrecordset.MoveFirst
yourarray = yourrecordset.GetRows(yourrecordset.RecordCount)
:)
First thing, my code
procedure TForm1.Button3Click(Sender: TObject);
var tempId,i:integer;
begin
tempId:=strtoint(edit5.Text);
plik:=TStringList.Create;
plik.LoadFromFile('.\klienci\'+linia_klient[id+1]+'.txt');
if (plik.Count=1) then
begin
label6.Caption:='then';
if (tempId=StrToInt(plik[0])) then
begin
Label6.Caption:='Zwrócono';
plik.Delete(0);
end
end
else
for i:=0 to plik.Count-2 do
begin
if (tempId=StrToInt(plik[i])) then
begin
Label6.Caption:='Zwrócono';
plik.Delete(i);
end;
end;
plik.SaveToFile('.\klienci\'+linia_klient[id+1]+'.txt');
plik.Free;
end;
When for i:=0 to plik.Count-2 do I can delete any element but not
last.
When for i:=0 to plik.Count-1 do I can delete any element without
but from end to start. Because otherwise List index out of bounds.
What's going one? How can I safety search and remove elements from TStringList?
When deleting intems from list you want to use downto loop, ie
for i := plik.Count-1 downto 0 do
begin
if (tempId=StrToInt(plik[i])) then
begin
Label6.Caption:='Zwrócono';
plik.Delete(i);
end;
end;
This ensures that if you delete item, the loop index stays valid as you move from the end of the list dowards beginning of the list.
This is a classic problem. A for loop evaluates the loop bounds once at the beginning of the loop, so you run off the end which explains your index out of bounds errors.
But even if for loops evaluated loop bounds every time like a while does that would not really help. When you delete an element, you reduce the Count by 1 and move the remaining elements down one in the list. So you change the index of all those still to be processed elements.
The standard trick is to loop down the list:
for i := List.Count-1 downto 0 do
if DeleteThisItem(i) then
List.Delete(i);
When you write it this way, the call to Delete affects the indices of elements that have already been processed.
For I := stringlist.count-1 downto 0 do
Now you can delete all items without any error
in an ascending loop like for i:=1 to count you just can't delete items of the list you are iterating over.
there are several solutions depending on the overall logic of what you want to achieve.
you may change the for loop into a while loop that reevaluates count and don't increment index on the delete iteration
you may reverse the loop, kinda for i:=count downto 1
instead of delete, you may create a temporary list and copy there only the items you want to keep, and recopy it back.
As others have said, using a downto loop is usually the best choice. Of course, it does change the semantics of the loop so it runs backwards instead of forwards. If you want to continue looping forwards, you have to use a while loop instead, eg:
I := 0;
while I < plik.Count do
begin
if (tempId = StrToInt(plik[I])) then
begin
...
plik.Delete(I);
end else
Inc(I);
end;
Or:
var
CurIdx, Cnt: Integer;
CurIdx := 0;
Cnt := plik.Count;
for I := 0 to Cnt-1 do
begin
if (tempId = StrToInt(plik[CurIdx])) then
begin
...
plik.Delete(CurIdx);
end else
Inc(CurIdx);
end;
I have two VirtualStringTrees, first VST has been populated with data and I want to check the second VST and add nodes that are not already in the first VST. Or I want to add those nodes from second VST that are not duplicates of first VST.
procedure Tcreatevtform.copy2tosimvt(vt: Tvirtualstringtree);
var
data: PMyRec;
simvtdata: PMyRectF;
rootnode, simvtnode: PVirtualNode;
ty: string;
begin
rootnode := vt.GetFirst; //vt is second virtualstringtree
while Assigned(rootnode) do
begin
data := vt.GetNodeData(rootnode);
ty := data^.caption;
if checksimduplicate(ty)=false then
begin
simvtnode := similarvt.AddChild(nil); //similarvt is the first virtualstringtree
simvtdata := similarvt.GetNodeData(simvtnode);
simvtdata^.caption := data^.caption;
end;
rootnode := vt.GetNext(rootnode,false);
end;
end;
function Tcreatevtform.checksimduplicate(t: string): boolean;
var
data: PMyRectf;
rootnode: PVirtualNode;
typew: string;
begin
Result := False;
rootnode := similarvt.GetFirst;
while Assigned(rootnode) do
begin
data := similarvt.GetNodeData(rootnode);
typew := data^.caption; // problem here, typew is always a constant or it is always the first
if t=typew then
begin
// node's caption of vt (letter 'a' is the first node's caption in my
// app. So this function is always false.
Result := True;
Break;
end;
similarvt.GetNext(rootnode, False);
end;
end;
I am using D7.
Finding all items that are in one list but not a second can be done easily with a list comparison algorithm. Here's the basic idea:
Sort both lists
Take 2 index variables, one for each list, and start at the start of each list
Compare the two indexed items from each list according to the sort order.
If they're equal, then the same item is in both lists. Increment both indexes.
If the first item is less than the second one, then it's not in the second list. Increment the first index.
If the second item is less than the first one, then it's not in the first list. Increment the second index.
Repeat until you reach the end of one of the lists. All remaining items in the other list are not in the first list.
You can add actions to the equals case or either the first-unique or second-unique case to determine what to do in these cases. In your specific case, you'll want to make the second-unique case add an item to the VST. It might get more complicated if you have to preserve tree structure, but that's the basic idea.
When I need to find the first node in a TTreeView, I call TTreeNodes.GetFirstNode. However, I sometimes need to locate the last node in a tree and there is no corresponding TTreeNodes.GetLastNode function.
I don't want to use Items[Count-1] since that results in the entire tree being walked with Result := Result.GetNext. Naturally this only matters if the tree views have a lot of nodes. I fully appreciate the virtues of virtual container controls but I am not going to switch to Virtual TreeView just yet.
So far I have come up with the following:
function TTreeNodes.GetLastNode: TTreeNode;
var
Node: TTreeNode;
begin
Result := GetFirstNode;
if not Assigned(Result) then begin
exit;
end;
while True do begin
Node := Result.GetNextSibling;
if not Assigned(Node) then begin
Node := Result.GetFirstChild;
if not Assigned(Node) then begin
exit;
end;
end;
Result := Node;
end;
end;
Can anyone:
Find a flaw in my logic?
Suggest improvements?
Edit 1
I'm reluctant to keep my own cache of the nodes. I have been doing just that until recently but have discovered some hard to track very intermittent AVs which I believe must be due to my cache getting out of synch. Clearly one solution would be to get my cache synchronisation code to work correctly but I have an aversion to caches because of the hard to track bugs that arise when you get it wrong.
Although I am not a non-Exit purist, I think that when it is doable without Exit while keeping readability intact, one might prefer that option.
So here is exactly the same code, for I don't think you can get any other way (faster) to the end node, but without Exit and slightly more compact:
function TTreeNodes.GetLastNode: TTreeNode;
var
Node: TTreeNode;
begin
Node := GetFirstNode;
Result := Node;
if Result <> nil then
repeat
Result := Node;
if Node <> nil then
Node := Result.GetNextSibling;
if Node = nil then
Node := Result.GetFirstChild;
until Node = nil;
end;
The method I've used before is the maintain a TList using List.Add on the OnAddition event and List.Remove on the OnDeletion event (OnRemove?). You've access to List.Count-1 (or whatever you need) pretty much instantly then.
Post edit - I have to say that although this worked fine, I then grew up and moved to Virtual Tree View :-)
If I was to implement it, this would probably be my first draft.
function TTreeNodes.GetLastNode: TTreeNode;
var
Node: TTreeNode;
function GetLastSibling(aNode : TTreeNode) : TTreeNode;
begin
if not Assigned(aNode) then
EXIT(nil);
repeat
Result := aNode;
aNode := Result.GetNextSibling;
until not Assigned(aNode) ;
end;
begin
Node := GetFirstNode;
if not Assigned(Node) then begin
exit;
end;
repeat
Result := GetLastSibling(Node);
Node := Result.GetFirstChild;
until not Assigned(Node);
end;
I find this slightly more readable. It might be slightly slower though.
I'm unsure about whether or not this approach would be faster than items[Count-1], in some cases, it could be slower, as the TTreeNodes actually caches the last node accessed through the items property.
Top-posted (sorry) answer, for those who don't have time to get into it but may have similar problems.
Rule #1, as always, move as much as you can out of loops.
2, moving TField var := ADODataSet.FieldByname() out of the loop
3, ADODataSet.DisableControls(); and ADODataSet.EnableControls(); around the loop
4, stringGrid.Rows[r].BeginUpdate() and EndUpdate() on each row (cannot do on teh whle control)
each of these shaved off a few seconds, but I got it down to "faster than the eye can see" by changing
loop
stringGrid.RowCount := stringGrid.RowCount + 1;
end loop
to putting stringGrid.RowCount := ADODataSet.RecordCount; before the loop
+1 and heartfelt thanks to all who helped.
(now I will go and see what I can do to optimize drawing a TChart, which is also slow ;-)
with about 3,600 rows in the table this takes 45 seconds to populate the string grid. What am I doing wrong?
ADODataSet := TADODataSet.Create(Nil);
ADODataSet.Connection := AdoConnection;
ADODataSet.CommandText := 'SELECT * FROM measurements';
ADODataSet.CommandType := cmdText;
ADODataSet.Open();
while not ADODataSet.eof do
begin
TestRunDataStringGrid.RowCount := TestRunDataStringGrid.RowCount + 1;
measurementDateTime := UnixToDateTime(ADODataSet.FieldByname('time_stamp').AsInteger);
DoSQlCommandWithResultSet('SELECT * FROM start_time_stamp', AdoConnection, resultSet);
startDateTime := UnixToDateTime(StrToInt64(resultSet.Strings[0]));
elapsedTime := measurementDateTime - startDateTime;
TestRunDataStringGrid.Cells[0, Pred(TestRunDataStringGrid.RowCount)] := FormatDateTime('hh:mm:ss', elapsedTime);
TestRunDataStringGrid.Cells[1, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('inputTemperature').AsFloat);
TestRunDataStringGrid.Cells[2, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('outputTemperature').AsFloat);
TestRunDataStringGrid.Cells[3, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('flowRate').AsFloat);
TestRunDataStringGrid.Cells[4, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('waterPressure').AsFloat * convert);
TestRunDataStringGrid.Cells[5, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('waterLevel').AsFloat);
TestRunDataStringGrid.Cells[6, Pred(TestRunDataStringGrid.RowCount)] := FloatToStrWithPrecision(ADODataSet.FieldByname('cod').AsFloat);
ADODataSet.Next;
end;
ADODataSet.Close();
ADODataSet.Free();
update:
Function DoSQlCommandWithResultSet(const command : String; AdoConnection : TADOConnection; resultSet : TStringList): Boolean;
var
i : Integer;
AdoQuery : TADOQuery;
begin
Result := True;
resultSet.Clear();
AdoQuery := TADOQuery.Create(nil);
try
AdoQuery.Connection := AdoConnection;
AdoQuery.SQL.Add(command);
AdoQuery.Open();
i := 0;
while not AdoQuery.eof do
begin
resultSet.Add(ADOQuery.Fields[i].Value);
i := i + 1;
AdoQuery.Next;
end;
finally
AdoQuery.Close();
AdoQuery.Free();
end;
end;
You are executing the command SELECT * FROM start_time_stamp 3,600 times, but it does not appear to me that it is correlated with your outer loop in any way. Why not execute it once before the loop?
That SELECT command appears to return only a single column of a single record, yet you use "*" to load all columns, and no WHERE clause to limit the results to a single row (if there's more than one row in the table).
You use only a limited number of columns from Measurements, but you retrieve all columns with "*".
You don't show the contents of DoSQlCommandWithResultSet, so it's not clear if there's a problem in that routine.
It's not clear whether the problem is in your database access or the string grid. Comment out all the lines pertaining to the string grid and run the program. How long does the database access alone take?
Additionally to Larry Lustig points:
In general, FieldByName is comparably slow method. You are calling it in loop for the same fields. Move the getting of field references out of the loop and store references in the variables. Like: InputTempField := ADODataSet.FieldByname('inputTemperature');
You are resizing the grid in the loop TestRunDataStringGrid.RowCount := TestRunDataStringGrid.RowCount + 1. That is the case, when you should use ADODataSet.RecordCount before the loop: TestRunDataStringGrid.RowCount := ADODataSet.RecordCount.
That is a good practice to call ADODataSet.DisableControls before loop and ADODataSet.EnableControls after loop. Even more actual that is for ADO dataset, which has not optimal implementation and those calls help.
Depending on a DBMS you are using, you can improve the fetching performance by setting a larger "rowset size". Not sure, how it control in ADO, probably setting ADODataSet.CacheSize to a greater value will help. Also, there are cursor settings :)
instead of calling ADODataSet.FieldByname('Fieldname') inside the loop you should declare local variables of type TField for each field, assign ADODataset.FindField('Fieldname') to the variables and use the variables inside the loop. FindFieldByName searches a list with every call.
Update:
procedure TForm1.Button1Click(Sender: TObject);
var
InputTemp, OutputTemp: TField;
begin
ADODataSet := TADODataSet.Create(Nil);
try
ADODataSet.Connection := ADOConnection;
ADODataSet.CommandText := 'SELECT * FROM measurements';
ADODataSet.Open;
InputTemp := ADODataSet.FindField('inputTemperature');
OutputTemp := ADODataSet.FindField('outputTemperature');
// assign more fields here
while not ADODataSet.Eof do begin
// do something with the fields, for example:
// GridCell := Format ('%3.2f', [InputTemp.AsFloat]);
// GridCell := InputTemp.AsString;
ADODataSet.Next;
end;
finally
ADODataSet.Free;
end;
end;
Another option would be to drop the TADODataset Componont on the form (or use a TDataModule) and define the fields at designtime.
Additional to the Larry Lustig answer, consider using data-aware controls instead, like the TDbGrid component.
If you aren't using data-aware controls you should use TestRunDataStringGrid.BeginUpdate before and TestRunDataStringGrid.EndUpdate after loop. Without this is your grid constantly redrawing after each modification (adding new row, cell update).
Another tip is set AdoQuery.LockType := ltReadOnly before opening query.
You could also try an instrumenting profiler instead of a sampling profiler to get better results (sampling profilers miss lot of detail info, and most time they have less then 1000 samples per second, and 1000 is already low: only good to get a quick overview).
Instrumenting profilers:
AQTime (commercial)
AsmProfiler (open source)
http://code.google.com/p/asmprofiler/wiki/AsmProfilerInstrumentingMode