I tried to use a TListView component to display rather large data lists (like 4000 rows large), and creating the list is incredibly slow - it takes something like 2-3 secs, which makes the UI all laggy and close to unusable.
I fill the TListView.Items inside a BeginUpdate/EndUpdate block, with only preallocated strings - I mean : I build a list of all strings to store (which takes no humanly noticeable time), then I put them in the TListView.
I wish to display the TListView's content in vsReport mode with several columns.
The code looks like this :
MyList.Items.BeginUpdate;
for i := 0 to MyCount - 1 do
begin
ListItem := MyList.Items.Add;
ListItem.Caption := StrCaptions[i];
ListItem.SubItems.Add(StrSubItems1[i]);
ListItem.SubItems.Add(StrSubItems2[i]);
end;
MyList.Items.EndUpdate;
Is there some other hack I missed in the TListView component's logic ? or should I just forget about using this component for performances ?
You can use listview in virtual mode. Have a look at the virtuallistview.dpr demo.
You can try Virtual Treeview component. It says "Virtual Treeview is extremely fast. Adding one million nodes takes only 700 milliseconds"
Use separate structure for holding your data. Set OwnerData of TListView to True.
#4000 rows I get only ~700 ms (D2009) times. For more responsiveness you could separate to other thread or add dirty Application.ProcessMessages() into loop.
rows generated with this code in 16 ms:
MyCount := 4000;
dw := GetTickCount();
for i := 0 to MyCount - 1 do begin
StrCaptions.Add('caption'+IntToStr(i));
StrSubItems1.Add('sub1'+IntToStr(i));
StrSubItems2.Add('sub2'+IntToStr(i));
end;
ShowMessageFmt('%u ms', [GetTickCount() - dw]);
Printed with:
MyList.Clear;
dw := GetTickCount();
MyList.Items.BeginUpdate;
for i := 0 to MyCount - 1 do
begin
ListItem := MyList.Items.Add;
ListItem.Caption := StrCaptions[i];
ListItem.SubItems.Add(StrSubItems1[i]);
ListItem.SubItems.Add(StrSubItems2[i]);
end;
MyList.Items.EndUpdate;
ShowMessageFmt('%u ms', [GetTickCount() - dw]);
EDIT:
I inserted Application.ProcessMessages() into print, but somewhy performance stays same
Related
In Delphi, is there a fast way of emptying a TStringgrid (containing in excess of 5000 rows) that will also free the memory?
Setting the rowcount to 1, empties the grid but does not free the memory.
Thanks in advance,
Paul
This should uninitialize the allocated strings (from the string list where the row texts are stored). Cleaning is done by columns since you have a lot of rows.
procedure TForm1.Button1Click(Sender: TObject);
var
I: Integer;
begin
for I := 0 to StringGrid1.ColCount - 1 do
StringGrid1.Cols[I].Clear;
StringGrid1.RowCount := 1;
end;
By "does not free the memory", do you mean that if you set RowCount := 1, and then set the RowCount := 10' you can still see the old content of theCells`?
If so, this is an old issue and has nothing to do with the memory not being freed; it's simply because you just happen to see the previous content of the memory when it's allocated again, because memory isn't zero'd out.
I have a pretty standard routine in a utility unit that deals with this visual glitch, and unless the grid is huge works fast enough. Just pass the TStringGrid before you change the RowCount or ColCount to a lower value.
procedure ClearStringGrid(const Grid: TStringGrid);
var
c, r: Integer;
begin
for c := 0 to Pred(Grid.ColCount) do
for r := 0 to Pred(Grid.RowCount) do
Grid.Cells[c, r] := '';
end;
Use it like this:
ClearStringGrid(StringGrid1);
StringGrid1.RowCount := 1;
I would suggest storing your string values in your own memory that you have full control over, and then use a TDrawGrid, or better a virtual TListView, to display the contents of that memory as needed.
The fastest way to use a TStringGrid is using OnGetValue/OnSetValue.
This way only the text of visible cells are requested dynamically.
Adding and removing rows is then lighting fast, otherwise TStringgrid is
very slooow when you have more than 5000 records.
This way I can fill and clear a grid with 700.000 records within a second!
When memory usage is the critical argument, consider using another grid. For example, NLDStringGrid that is (re)written by myself, and which has an additional property called MemoryOptions. It controls whether data can be stored beyond ColCount * RowCount, whether the storage is proportional (less memory usage for partially filled rows and columns), whether to store the Cols and Rows property results and whether the data is stored in sparse manner.
To clear such grid that has moBeyondGrid excluded from the memory options, setting RowCount to FixedRows suffices.
It's open source and downloadable from here.
I have a TStringGrid with 10 columns. Adding 500 rows to it takes around 2 seconds. Is this normal performance?
It seems a bit slow to me.
I am getting the data from a database query. If I loop through the query but don't write the results to the StringGrid, the process takes around 100ms, so it's not the database that's slowing things down.
Once the rows are added, the StringGrid performance is fine.
Here is the code I am using
Grid.RowCount := Query.RecordCount;
J := 0;
while not Query.EOF do
begin
Grid.Cells[0,J]:=Query.FieldByName('Value1').AsString;
Grid.Cells[1,J]:=Query.FieldByName('Value2').AsString;
Grid.Cells[2,J]:=Query.FieldByName('Value3').AsString;
// etc for other columns.
Inc(J);
Query.Next();
end;
The real code is actually a bit more complex (the table columns do not correspond exactly to the query columns) but that's the basic idea
One other thing I have found to be very important when going through a lot of records is to use proper TField variables for each field. FieldByName iterates through the Fields collection every time so is not the most performant option.
Before the loop define each field as in:
var
f1, f2: TStringField;
f3: TIntegerField;
begin
// MyStringGrid.BeginUpdate; // Can't do this
// Could try something like this instead:
// MyStringGrid.Perform(WM_SETREDRAW, 0, 0);
try
while ... do
begin
rowvalues[0] := f1.AsString;
rowvalues[1] := f2.AsString;
rowvalues[2] := Format('%4.2d', f3.AsInteger);
// etc
end;
finally
// MyStringGrid.EndUpdate; // Can't - see above
// MyStringGrid.Perform(WM_SETREDRAW, 1, 0);
// MyStringGrid.Invalidate;
end;
end;
That along with BeginUpdate/Endupdate and calling Query.DisableControls if appropriate.
The solution was to add all values in a row at once, using the "Rows" property.
My code now looks like this:
Grid.RowCount := Query.RecordCount;
rowValues:=TStringList.Create;
J := 0;
while not Query.EOF do
begin
rowValues[0]:=Query.FieldByName('Value1').AsString;
rowValues[1]:=Query.FieldByName('Value2').AsString;
rowValues[2]:=Query.FieldByName('Value3').AsString;
// etc for other columns.
Grid.Rows[J]:=rowValues;
Inc(J);
Query.Next();
end;
rowValues.Free; // for the OCD among us
This brought the time down from 2 seconds to about 50ms.
FieldByName used in a loop is very slow since it is calculated each time. You should do it out of the loop and then just use results inside of a loop.
TStringGrid works OK for a small number of records, but don't try it for more than 10.000 records.
We had severe performance problems with TAdvStringGrid from TMS (which is based on Delphi TStringGrid) when loading/sorting/grouping large grid sets, but also when inserting one row at the top of the grid (expanding a grid group node). Also memory usage was high.
And yes, I used the beginupdate/endupdate already. Also other tricks. But after diving into the structure of TStringGrid I concluded it could never be fast for many records.
As a general tip (for large grids): use the OnGetText (and OnSetText) event. This event is used for filling the grid on demand (only the cells that are displayed). Store the data in your own data objects. This made our grid very fast (1.000.000 record is no problem anymore, loads within seconds!)
First optimization is to replace very slow Query.FieldByName('Value1') calls by a local TQuery.
var
F1, F2, F3: TField;
Grid.RowCount := Query.RecordCount;
J := 0;
F1 := Query.FieldByName('Value1');
F2 := Query.FieldByName('Value2');
F3 := Query.FieldByName('Value3');
while not Query.EOF do
begin
Grid.Cells[0,J]:=F1.AsString;
Grid.Cells[1,J]:=F2.AsString;
Grid.Cells[2,J]:=F3.AsString;
// etc for other columns.
Inc(J);
Query.Next();
end;
If this is not enough, use the grid in virtual mode, i.e. retrieve all content in a TStringList or any in-memory structure, then use the OnGetText or OnDrawCell methods.
I believe it's slow because it has to repaint itself everytime you add a row. Since you are taking the values from a query i think it would be better for you to use a TDBGrid instead.
Best regards.
If you know how many rows you're about to add, store the current rowcount in a temporary variable, set the grid's rowcount to accommodate the current rowcount plus the rows you're about to add, then assign the new values to the rows (using the former rowcount you stored) rather than adding them. This will reduce a lot of background processing.
Try testing with AQTime or similar tool (profilers).
Without any code is difficult, but I thinks thar the poor performance is due to FieldByName, not StringGrid.
FieldByName make a liear search:
for I := 0 to FList.Count - 1 do
begin
Result := FList.Items[I];
...
If your Dataset have many columns (fields) the performance will still be lower.
Regards.
I was going to say "why not just use beginupdate/endupdate?" but now I see that the regular string grid doesn't support it.
While googling that, I found a way to simulate beginupdate/endupdate:
http://www.experts-exchange.com/Programming/Languages/Pascal/Delphi/Q_21832072.html
See the answer by ZhaawZ, where he uses a pair of WM_SETREDRAW messages to disable/enable the repainting. If this works, use in conjunction with the "eliminate use of FieldbyName" trick, and it should take no time to draw.
Set Grid.RowCount = 2 before the loop then when the loop is finished set the rowcount to the correct value.
That avoids lots of calls to the OnPaint event.
In my case it turned out that the Debug build was slow and the Release build was fast - a Heisenbug.
More specifically, FastMM4 FullDebugMode triggered the slowness.
I find that I might have to use a String Grid of less than 10 columns, but about 50k rows.
Experiments have showed it to be a very unresponsive CPU hog.
Any pointers?
Code or components?
Preferably Delphi (7) build in or TMS (for which I have a license) or FOSS (for commercial use).
Update: please don't just tell me to use Virtual Tree View, etc. Please tell me why, so that I can learn something. Thanks.
I don't think the problem came from adding this number to the TStringGrid.
Adding 100k rows took less than 1 second (700ms) (Not high end PC, just Dual Core).
procedure TForm1.btn1Click(Sender: TObject);
Const
arr : array[1..5] of string = ('One','Two','Three','Four','Five');
Rows = 100000;
var
I: Integer;
F,E : Integer;
begin
StringGrid1.RowCount := Rows;
F := GetTickCount;
for I := 0 to Rows do
begin
StringGrid1.Cells[1,I] := Arr[1] + IntToStr(I);
StringGrid1.Cells[2,I] := Arr[2]+ IntToStr(I);
StringGrid1.Cells[3,I] := Arr[3]+ IntToStr(I);
StringGrid1.Cells[4,I] := Arr[4]+ IntToStr(I);
StringGrid1.Cells[5,I] := Arr[5]+ IntToStr(I);
end;
E := GetTickCount;
ShowMessage(Inttostr(E-F));
end;
I think the slowness in your code, do you bring the data from database? if so this is will be the bottleneck of your code, also adding 50k to WHATEVER GRID to show for users called "Bad practice".
And it's hard to tell you why that's slow without showing any code.
The TListView component in virtual mode is recommended frequently (I have not tried it myself but it sounds rather easy to implement)
If you are interested in virtual treeview you should checkout the whole virtual treeview trunk from http://code.google.com/p/virtual-treeview/source/checkout. You will find Demos\Advanced subdirectory where and Demo application showing the virtual treeview functionality, for example how to use virtual treeview as a grid.
You need SVN to checkout the code from googlecode. If you have never used SVN download and install TortoiseSVN
You may want to look at Virtual Treeview, which is built for high volume:
http://www.delphi-gems.com/index.php?option=com_content&task=view&id=12&Itemid=38
I'm having a problem with an interface that consists of a number of frames (normally 25) within a TScrollBox.
There are 2 problems, and I am hoping that one is a consequence of the other...
Background:
When the application starts up, I create 25 frames, each containing approx. 20 controls, which are then populated with the default information. The user can then click on a control to limit the search to a subset of information at which point I free and recreate my frames (as the search may return < 25 records)
The problem:
If I quit the application after the initial search then it takes approx. 5 seconds to return to Delphi. After the 2nd search (and dispose / recreate of frames) it takes approx. 20 seconds)
Whilst I could rewrite the application to only create the frames once, I would like to understand what is going on.
Here is my create routine:
procedure TMF.CreateFrame(i: Integer; var FrameBottom: Integer);
var
NewFrame: TSF;
begin
NewFrame := TSF.Create(Self);
NewFrame.Name := 'SF' + IntToStr(i);
if i = 0 then
NewSF.Top := 8
else
NewSF.Top := FrameBottom + 8;
FrameBottom := NewFrame.Top + NewFrame.Height;
NewFrame.Parent := ScrollBox1;
FrameList.Add(NewFrame);
end;
And here is my delete routine:
procedure TMF.ClearFrames;
var
i: Integer;
SF: TSF;
begin
for i := 0 to MF.FrameList.Count -1 do
begin
SF := FrameList[i];
SF.Free;
end;
FrameList.Clear;
end;
What am I missing?
As you are taking control over the memory allocation of the Frames you are creating by Free'ing them, so there's no need to provide Self as the owner parameter in the create constructor. Pass nil instead to prevent the owner trying to free the frame.
Also, don't like the look of your ClearFrames routine. Try this instead:
while FrameList.count > 0 do
begin
TSF(Framelist[0]).free;
Framelist.delete(0);
end;
Framelist.clear;
If you want to know why your app is taking so long to do something, try profiling it. Try running Sampling Profiler against your program. The helpfile explains how to limit the profiling to only a specific section of your app, which you could use to only get sampling results on the clearing or creating parts. This should show you where you're actually spending most of your time and take a lot of the guesswork out of it.
i have a simple program that sorts a text file according to length of words per line
this program works without problems in my xp based old machine
now i run this program on my new win7/intel core i5 machine, it freezes whole system and back normal after it finishes it's work.
i'v invastigated the code and found the line causing the freeze
it was this specific line...
caption := IntToStr(i) + '..' + IntTostr(ii);
i'v changed it to
caption := IntTostr(ii); //slow rate change
and there is no freeze
and then i'v changed it to
caption := IntTostr(i); //fast rate change
and it freeze again
my procedure code is
var tword : widestring;
i,ii,li : integer;
begin
tntlistbox1.items.LoadFromFile('d:\new folder\ch.txt');
tntlistbox2.items.LoadFromFile('d:\new folder\uy.txt');
For ii := 15 Downto 1 Do //slow change
Begin
For I := 0 To TntListBox1.items.Count - 1 Do //very fast change
Begin
caption := IntToStr(i) + '..' + IntTostr(ii); //problemetic line
tword := TntListBox1.items[i];
LI := Length(tword);
If lI = ii Then
Begin
tntlistbox3.items.Add(Trim(tntlistbox1.Items[i]));
tntlistbox4.items.Add(Trim(tntlistbox2.Items[i]));
End;
End;
End;
end;
any idea why ? and how to fix it?
i use delphi 2007/win32
Is this happening inside an event handler on a form? I'm going to guess taht it is. In that case, "Caption" is in the scope of the form. The form's caption text isn't managed by the VCL, but by Windows, and if you're sending a new WM_SETTEXT message on every iteration of the loop.
A thorough explanation of why this is doing what it's doing would require knowledge of Windows internals that I don't have, but if I were to take a guess, I'd say it's something like this:
Every time you send that WM_SETTEXT message with a new caption, Windows checks to make sure it's not identical to the existing caption. If it is, it can exit immediately. That's why the infrequent change (the one that only uses ii) doesn't slow your system down. But if it does change on every iteration, then Windows has to perform some sort of task switch in order to change it.
As for why that would bog down the entire system under a Vista kernel (including Win7) but not XP, that's completely outside my area of expertise. But if you're trying to do this as some sort of progress indicator, there are better ways, especially if this loop is as tight as it looks.
The best way to handle progress updates in a tight loop is to count iterations and only fire once every X times. (100 or 1000 can be good values for X, depending on how many times it's running and how fast the whole thing takes.) This is basically what the ii only option does. You could also try putting a Progress Bar on the form to measure progress instead of doing it through the form's caption.
Changing a Form's caption releases a whole bunch of actions - especially under Vista and Win7 with Aero active.
A quick try would be using a TLabel instead for displaying progress. Something like
Label1.caption := IntToStr(i) + '..' + IntTostr(ii); //problemetic line
Label1.Refresh; // or Repaint
should do the trick unless your label is transparent or on a glass area.
It would probably be best to follow Mason Wheeler's advice and use a progressbar. As the overall number of iterations is 15*TntListBox1.items.Count you can calculate the progress value quite easily.
First: you forget tntlistbox3.items.BeginUpdate/tntlistbox3.items.EndUpdate calls (same for tntlistbox4).
Second: Why does my program run faster if I click and hold the caption bar?
Solution (example):
const
UpdateInterval = 500; // half a second
var
...
LastUpdate: Cardinal;
begin
...
LastUpdate := GetTickCount + 100000; // forces first update
For ii := 15 Downto 1 Do //slow change
Begin
For I := 0 To TntListBox1.items.Count - 1 Do //very fast change
Begin
if (GetTickCount > (LastUpdate + UpdateInterval)) or
(GetTickCount < LastUpdate) then
caption := IntToStr(i) + '..' + IntTostr(ii); //problemetic line
...
end;
end;