Is this economical? - delphi

Question

Just wanting to see if there is a better way to do the following(there is always a better way for everything) because it does delay the application when loading due the amount of data.
I want to fill an array of records with data I have stored in csv file, I currently have it fixed length for the array but will later make it dynamic so I can add to the csv file.
type
TStarCoords = Packed record
szSystem: String[40];
fCoordX: Single;
fCoordY: Single;
fCoordZ: Single;
end;
SystemCoords: Array [0 .. 22379] of TStarCoords;
Const
SYSTEMS = 'Data\Systems.csv';
I then fill the array on the oncreate event
procedure TForm1.FormCreate(Sender: TObject);
var
szFile, sRecord: string;
Row, Index, i: Integer;
slList: TStringList;
begin
szFile := ExtractFilePath(ParamStr(0)) + SYSTEMS;
if FileExists(szFile) then
try
slList := TStringList.Create;
slList.LoadFromFile(szFile);
for Row := 0 to slList.Count - 1 do
begin
sRecord := slList[Row];
index := Pos(',', sRecord);
if index > 0 then
begin
SystemCoords[Row].szSystem := Copy(sRecord, 1, index - 1);
Delete(sRecord, 1, index);
end;
index := Pos(',', sRecord);
if index > 0 then
begin
SystemCoords[Row].fCoordX := StrToFloat(Copy(sRecord, 1, index - 1));
Delete(sRecord, 1, index);
end;
index := Pos(',', sRecord);
if index > 0 then
begin
SystemCoords[Row].fCoordY := StrToFloat(Copy(sRecord, 1, index - 1));
Delete(sRecord, 1, index);
end;
SystemCoords[Row].fCoordZ := StrToFloat(sRecord);
end;
finally
slList.Free;
end;
for i := Low(SystemCoords) to High(SystemCoords) do
begin
cbSystem.Items.Add(SystemCoords[i].szSystem);
end;
end;
As you can see I am using "Pos" function to parse the csv file and also loop the array at the end to add the Star name to a combobox, Is there a more economical way of doing this?
Any suggestions are welcomed

Answer 1

It doesn't look very efficient.
Allocating a fixed length global array looks poor. Use a dynamic array of length determined at runtime.
Short strings are not recommended. Don't use them in modern programming. They are legacy and don't handle Unicode.
Don't pack records. That results in misaligned data.
There seems to be far more heap allocations that are needed. Avoid Delete if you can.
Loading into a string list won't be efficient. Use a line reader based approach for speed. Delphi's built in class though is rubbish. If you want speed and effective use of memory, roll your own.
Probably the bulk of the time is spent populating the combo! Adding 22380 items to a combo will take a very long time. Don't do that. If the data set is smaller, only add as many items as there are in the data. Otherwise, use the virtual paradigm in your UI control.
Your next step though is to work out where the bottleneck is. We can only guess because we are missing so much information. We don't know if the data is static, how big it is, and so on.

Answer 2

Like others said, probably the majority of the time is spent populating the combo.
In my opinion, when dealing with big updates of a TStrings the BeginUpdate / EndUpdate technique proposed by the Jens Borrisholt's answer constitutes a valid approach.
As a minor issue, if your application is the only which writes and reads the data and neither machines nor humans care about the CSV format, you might consider to store the records adopting a different file format, using the BlockRead and BlockWrite functions.
type
TStarCoords = record
szSystem: string[40];
fCoordX,
fCoordY,
fCoordZ: Single;
end;
. . .
const
CFILENAME = '<your path to some file .dat>';
Reading the data:
procedure TForm1.FormCreate(Sender: TObject);
var
lstStarCoords: TList<TStarCoords>;
f: File;
starCoords: TStarCoords;
begin
lstStarCoords := TList<TStarCoords>.Create;
try
AssignFile(f, CFILENAME);
Reset(f, SizeOf(TStarCoords));
try
while not Eof(f) do begin
BlockRead(f, starCoords, 1);
lstStarCoords.Add(starCoords);
end;
finally
CloseFile(f);
end;
cbSystem.Items.BeginUpdate;
for starCoords in lstStarCoords do
cbSystem.Items.Add(starCoords.szSystem);
cbSystem.Items.EndUpdate;
finally
lstStarCoords.Free;
end;
end;
Writing the data:
procedure TForm1.WriteStarCoords;
var
lstStarCoords: TList<TStarCoords>;
f: File;
starCoords: TStarCoords;
i: Integer;
begin
lstStarCoords := TList<TStarCoords>.Create;
try
//let's insert 5k new items
for i:=1 to 5000 do begin
with starCoords do begin
szSystem := 'HYEL YE';
fCoordX := 122;
fCoordY := 12.375;
fCoordZ := 45.75;
end;
lstStarCoords.Add(starCoords);
end;
AssignFile(f, CFILENAME);
Rewrite(f, SizeOf(TStarCoords));
try
for starCoords in lstStarCoords do
BlockWrite(f, starCoords, 1);
finally
CloseFile(f);
end;
finally
lstStarCoords.Free;
end;
end;
EDIT: example using pointers to store the record information directly in the cbSystem component.
This approach is a little more "dangerous" since it allocates memory which has to be manually freed but allows to avoid the usage of a TDictionary to pair the TStarCoords.szSystem with the corresponding record.
Declare a new type which points to the TStarCoords record:
type
PStarCoords = ^TStarCoords;
Reading the data:
procedure TForm1.FormCreate(Sender: TObject);
var
lstStarCoords: TStringList;
f: File;
starCoords: PStarCoords;
begin
ClearCbSystem;
lstStarCoords := TStringList.Create(False);
{another minor enhancement:
since lstStarCoords does not own any TObject which needs to be freed
the OwnsObjects property of the TStringList can be set to False
in order to avoid some code to be execute in some method like Clear and Delete}
try
lstStarCoords.BeginUpdate;
AssignFile(f, CFILENAME);
Reset(f, SizeOf(TStarCoords));
try
while not Eof(f) do begin
New(starCoords);
BlockRead(f, starCoords^, 1);
lstStarCoords.AddObject(starCoords^.szSystem, TObject(starCoords));
end;
finally
CloseFile(f);
end;
lstStarCoords.EndUpdate;
cbSystem.Items.Assign(lstStarCoords);
finally
lstStarCoords.Free;
end;
end;
Clearing the list with cbSystem.Clear does not automatically dispose the underlying pointers which have to be manually freed. Use the ClearCbSystem procedure everytime the cbSystem list has to be cleared:
procedure TForm1.ClearCbSystem;
var
i: Integer;
begin
cbSystem.Items.BeginUpdate;
for i := cbSystem.Items.Count-1 downto 0 do
Dispose(PStarCoords(cbSystem.Items.Objects[i]));
cbSystem.Clear;
cbSystem.Items.EndUpdate;
end;
When the form is destroyed, a call to the ClearCbSystem procedure ensures the pointers are disposed before the cbSystem component is freed by the application itself:
procedure TForm1.FormDestroy(Sender: TObject);
begin
ClearCbSystem;
end;

Answer 3

You can use TStringlist for the parsing of the line. In the following I assume that you have you elements seperated by a comma.
Since you are putting the string representation of you records into a combobox I assunme you later on in your program needs to go the other way: Find a TStarCoords from string. Given that I woyls recoment you putting your elements in a TDictionary instread og a Array.
unit Unit1;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, Generics.Collections, StdCtrls;
type
TStarCoords = packed record
szSystem: string[40];
fCoordX: Single;
fCoordY: Single;
fCoordZ: Single;
end;
const
SYSTEMS = 'Data\Systems.csv';
type
TForm1 = class(TForm)
ComboBox1: TComboBox;
procedure FormCreate(Sender: TObject);
procedure ComboBox1Change(Sender: TObject);
private
SystemCoords: TDictionary<string, TStarCoords>;
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.dfm}
procedure TForm1.ComboBox1Change(Sender: TObject);
var
StarCoord: TStarCoords;
begin
if not SystemCoords.TryGetValue(ComboBox1.Text, StarCoord) then
exit; //todo : Make some error handling
Caption := FloatToStr(StarCoord.fCoordX);
end;
procedure TForm1.FormCreate(Sender: TObject);
var
Lines, Elements: TStringlist;
Line: string;
SystemCoord: TPair<string, TStarCoords>;
begin
if not FileExists(ExtractFilePath(ParamStr(0)) + SYSTEMS) then
exit; //todo: Some error handling
SystemCoords := TDictionary<string, TStarCoords > .Create;
Lines := TStringlist.Create;
Elements := TStringlist.Create;
Elements.LineBreak := ',';
try
for Line in Lines do
begin
Elements.Text := Line;
SystemCoord.Key := Elements[0];
with SystemCoord.Value do
begin
szSystem := string(Elements[0]);
fCoordX := StrToFloat(Elements[1]);
fCoordY := StrToFloat(Elements[2]);
fCoordZ := StrToFloat(Elements[3]);
end;
SystemCoords.Add(SystemCoord.Key, SystemCoord.Value);
end;
finally
Lines.Free;
Elements.Free;
end;
try
ComboBox1.Items.BeginUpdate;
for SystemCoord in SystemCoords do
ComboBox1.Items.Add(SystemCoord.Key);
finally
ComboBox1.Items.EndUpdate;
end;
end;
end.