How to save / load this data? - delphi

I have a few items I need to save, Its all value for a Map. So each time a user can either load an old map or create and save a new one. But I am unsure how or what process to use to save all the data.
I have
TCube array (1..10000)
Row, Column : integers
NumberOfBlocks : integers
So when a user clicks save it should save the array data ( I am not sure it matters but not all 10,000 cubes are created) , along with the 3 integers.
Full code of the creation of the TCUBE array and integers.
procedure TForm2.createMap(r:integer;c:integer);
var
i : integer;
rows,columns,columnssave : integer;
x,y,z : single;
player : tmodel3d;
begin
columns := r;
rows := c;
i:=1;
x := 0;
y := 0;
z := 0;
NumberOfRows := rows;
NumberOfColumns := columns;
camera1.Position.X := rows/2;
camera1.Position.Y := columns/2;
dummy1.Position.X := rows/2;
dummy1.Position.Y := columns/2;
while rows>0 do
begin
columnssave := columns ;
while columns >0 do
begin
CreateCube[i]:=tcube.Create(self);
CreateCube[i].Visible := true;
CreateCube[i].Position.x := x;
CreateCube[i].Position.Y := y;
CreateCube[i].Position.Z := z;
CreateCube[i].Material.Texture.CreateFromFile(gamedir+'\pics\Grass.bmp');
CreateCube[i].Material.Modulation := TTextureMode.tmReplace;
CreateCube[i].Parent := viewport3d1;
CreateCube[i].OnClick := cubeClick;
CreateCube[i].OnMouseDown := mousedown;
y:= y+1;
i:=i+1;
//z:=z-1;
columns := columns -1;
end;
y:=0;
x:= x+1;
z:=0;
columns := columnssave;
rows:= rows-1;
end;
totalblocks := i;
setblocks := false;
label2.Text := inttostr(totalblocks);
end;
Thanks
Glen

There are lots of ways to save data out to a file. Normally I use NativeXML to read and write XML documents. This probably isn't the best way but it works for me and has been suitable in my projects so far.
It's human readable so you can see what's going on. Great for debugging when something isn't working right.
It's structured. That helps when you need to save data for objects contained by other objects. For example, in your game world, there is a village, the village contains 10 buildings, building 1 contains 7 lamps, building 2 contains 3 swords, etc.
It's a standard and commonly used format. Your game data files can be read and manipulated in other programs.
The XML file can be encrypted as a way to prevent users from modifying the data.
Example of what the save code could look like:
procedure TMyGame.SaveToFile(const FileName: string);
var
XML : TNativeXML;
RootNode : TXmlNode;
CubeNode : TXmlNode;
begin
XML := TNativeXML.CreateName('root');
try
RootNode := XML.Root;
for c1 := 0 to NumberOfCubes-1 do
begin
CubeNode := RootNode.NodeNew('Cube');
CubeNode.NodeNew('XPos').ValueAsInteger := 10;
CubeNode.NodeNew('YPos').ValueAsInteger := 2;
end;
XML.SaveToFile(FileName);
finally
XML.Free;
end;
end;

I strongly advocate for having human-readable data, such as with an XML or .ini file. If you need to block the user from being able to edit/modify the data, you can do that by "signing" the data and adding a validation key. The key can be generated by your program by taking your sensitive data, appending something that only you know, and then generating an MD5 hash or encrypting it. The MD5 result, or the first 100 bytes of the encrypted string, becomes your key which you would store in the .ini or whatever.
ex:
[MAPSTUFF]
SecretCoordinates=1,2,3,PIRATE TREASURE IS HERE
SignedKey=00042055215e2c6c0d751cb5b086e653
The SignedKey is actually an MD5 of the SecretCoordinates string + :XYZZY:
i.e. I ran this through MD5:
1,2,3,PIRATE TREASURE IS HERE:XYZZY:
and got:
00042055215e2c6c0d751cb5b086e653
So to check in the program, you read SecretCoordinates from the .ini file, add :ZYZZY: to it (in the program, so the user can't see (but if they have a debugger they still can)). And see if you get the same MD5. If it's a match, you're good. If it doesn't match, then the SecretCoordinates string has been tampered with, and you reject it. Very simple.

One of the best ways to save this type of data is by createing a record and saving the record. Then its not something human readable either. Sof first create a new type
TCubeSave = record
number : integer;
x : single;
y : single;
z : single;
texture : string[20];
end;
in this case we want to save the number in the array, along with the data x,y,z and texture?
If doing any string make sure its an array or you will get an error.
Next assign your variables and file and make it write ready.
var
myfile : File of TCubeSave;
cube : TcubeSave;
i : integer;
begin
assignfile(myfile, gamedir+'\maps\test.map');
ReWrite(myfile);
i:=0;
Now write each component of the array,
while (i < totalblocks) do
begin
Check if the component exsist, incase you have removed it or it was not created.
if FindComponent('cubename'+inttostr(i)) <> nil then
begin
Write the data to the file cube
cube.number := i;
cube.x := createcube[i].Position.X;
cube.y := createcube[i].Position.Y;
cube.z := createcube[i].Position.Z;
cube.texture := createcube[i].Material.Texture.ToString;
Write(myfile, cube);
end;
i := i+1;
end;
close the file when done.
CloseFile(myfile);
showmessage('Map Saved');
end;
when ready to read in the file..
// Reopen the file in read only mode
FileMode := fmOpenRead;
Reset(myFile);
// Display the file contents
while not Eof(myFile) do
begin
Read(myFile, cube);
i:= cube.number;
CreateCube[i]:=tcube.Create(self);
createCube[i].Position.x := cube.x;
createcube[i].position.y := cube.y;
ect.....
end;
// Close the file
CloseFile(myFile);

Related

Drag and drop from VirtualTreeView to shell (Ole drag and drop)

I am trying to drag and drop from VirtualTreeView to create a file in shell (drag and drop from VirtualTreeView to a folder in File Explorer or desktop folder).
I only found example of doing the opposite (shell to VirtualTreeView), but I cannot find any example for doing that. Help?
Doing any drag-drop operations in Windows involves creating an IDataObject, and giving that object to Windows.
The Virtual Treeview handles a lot of that grunt-work for you, creating an object that implements IDataObject for you. The tree then raises events when you need to help populate it.
When passing "file-like" things through a copy-paste or a drag-drop, you are require to add two clipboard formats to the IDataObject:
CF_FILEDESCRIPTOR, and
CF_FILECONTENTS
In addition to support for formats that the virtualtree itself will add, you can choose to indicate support for more clipboard format.
OnGetUserClipboardFormats Event
This is the event where you are given a chance to add additional clipboard formats to the IDataObject that the tree will be creating:
procedure TForm1.lvAttachmentsGetUserClipboardFormats(Sender: TBaseVirtualTree;
var Formats: TFormatEtcArray);
var
i: Integer;
begin
//Add formats for CF_FILEDESCRIPTOR and CF_FILECONTENTS
i := Length(Formats);
SetLength(Formats, i + 1);
Formats[i].cfFormat := CF_FILEDESCRIPTOR;
Formats[i].ptd := nil;
Formats[i].dwAspect := DVASPECT_CONTENT;
Formats[i].lindex := -1;
Formats[i].tymed := TYMED_HGLOBAL;
i := Length(Formats);
SetLength(Formats, i + 1);
Formats[i].cfFormat := CF_FILECONTENTS;
Formats[i].ptd := nil;
Formats[i].dwAspect := DVASPECT_CONTENT;
Formats[i].lindex := 0;
Formats[i].tymed := TYMED_ISTREAM;
end;
The tree will then given the IDataObject to the shell as part of the drag-drop operation.
Later, an application that the user dropped items onto will enumerate all formats in the IDataObject, e.g.:
CF_HTML ("HTML Format")
CFSTR_FILEDESCRIPTOR ("FileGroupDescriptorW")
CFSTR_FILECONTENTS ("FileContents")
CF_ENHMETAFILE
And it will see that the IDataObject contains FileDescriptor and FileContents.
The receiving application will then ask the IDataObject to actually cough up data. (This "delayed-rendering" is a good thing, it means your source application doesn't actually have to read any content unless it actually gets requested).
OnRenderOleData Event
This is the event where the virtual tree realizes its IDataObject has been asked to render something, and it needs you to finally render that actual content.
The general idea with these two clipboard formats is:
CF_FILEDESCRIPTOR lets you return a record that describes the file-like thing (e.g. filename, file size, created date, last modified date, last accessed date)
CF_FILECONTENTS lets you return an IStream that contains the actual file contents
procedure TForm1.lvAttachmentsRenderOLEData(Sender: TBaseVirtualTree; const FormatEtcIn: tagFORMATETC;
out Medium: tagSTGMEDIUM; ForClipboard: Boolean; var Result: HRESULT);
var
global: HGLOBAL;
stm: IStream;
begin
if FormatEtcIn.cfFormat = CF_FILEDESCRIPTOR then
begin
global := GetAttachmentFileDescriptorsFromListView(lvAttachments, ForClipboard);
if global = 0 then
Exit;
ZeroMemory(#Medium, SizeOf(Medium));
Medium.tymed := TYMED_HGLOBAL;
Medium.hGlobal := global;
Result := S_OK;
end
else if FormatEtcIn.cfFormat = CF_FILECONTENTS then
begin
ZeroMemory(#Medium, SizeOf(Medium));
Medium.tymed := TYMED_ISTREAM;
Result := GetAttachmentStreamFromListView(lvAttachments, ForClipboard, FormatEtcIn.lindex, stm);
if Failed(Result) then
Exit;
Medium.stm := Pointer(stm);
IUnknown(Medium.stm)._AddRef;
Result := S_OK;
end;
end;
The first helper function creates an array of FILE_DESCRIPTOR objects, and copies them to a HGLOBAL allocated memory:
function GetAttachmentFileDescriptorsFromListView(Source: TVirtualStringTree; ForClipboard: Boolean): HGLOBAL;
var
i: Integer;
nCount: Integer;
nodes: TNodeArray;
descriptors: TFileDescriptorDynArray;
data: TAttachment;
begin
Result := 0;
if ForClipboard then
nodes := Source.GetSortedCutCopySet(False)
else
nodes := Source.GetSortedSelection(False);
if Length(nodes) = 0 then
Exit;
nCount := 0;
for i := 0 to Length(nodes) - 1 do
begin
//Get the file thing from this node
data := GetNodeDataFromNode(nodes[i]);
if not Assigned(data) then
Continue;
//Increase the size of our descriptors array by one
Inc(nCount);
SetLength(Descriptors, nCount);
//Fill in the next descriptor
descriptors[nCount-1] := data.ToWindowsFileDescriptor;
end;
Result := FileDescriptorsToHGLOBAL(descriptors);
end;
The second helper copies your file-like thing's binary contents to an IStream:
function GetAttachmentStreamFromListView(Source: TVirtualStringTree; ForClipboard: Boolean; lindex: Integer; var stm: IStream): HResult;
var
nodes: TNodeArray;
data: TAttachment;
begin
Result := E_FAIL;
if ForClipboard then
nodes := Source.GetSortedCutCopySet(False)
else
nodes := Source.GetSortedSelection(False);
if Length(nodes) = 0 then
Exit;
if (lIndex < Low(Nodes)) or (lIndex > High(Nodes)) then
begin
Result := DV_E_LINDEX;
Exit;
end;
//Get the file thing from this node
data := GetNodeDataFromNode(nodes[i]);
if not Assigned(data) then
Continue;
//Fetch the content into a IStream wrapped memory stream
stm := data.GetStream(nil);
Result := S_OK;
end;
Your attachment object, whatever it is has to know:
how to represent itself as a TFileDescriptor
how to return the contents as an IStream

How to delete a specific line from a text file in Delphi

I have a text file with user information stored in it line by line. Each line is in the format: UserID#UserEmail#UserPassword with '#' being the delimiter.
I have tried to use this coding to perform the task:
var sl:TStringList;
begin
sl:=TStringList.Create;
sl.LoadFromFile('filename');
sl.Delete(Index);
sl.SaveToFile('filename');
sl.free;
end;
But I'm not sure what to put in the "index" space.
Is there any way I can receive the User ID as input and then delete the line of text from the text file that has this user ID in? Any help would be appreciated.
You can set the NameValueSeparator to # then use IndexOfName to find the user, as long as the username is the first value in the file.
sl.NameValueSeparator := '#';
Index := sl.IndexOfName('455115')
So in your example, like so
var sl:TStringList;
begin
sl:=TStringList.Create;
sl.LoadFromFile('filename');
sl.NameValueSeparator := '#';
Index := sl.IndexOfName('455115')
if (Index <> -1) then
begin
sl.Delete(Index);
sl.SaveToFile('filename');
end;
sl.free;
end;
This may be slow on large files as IndexOfName loops though each line in the TStringList and checks each string in turn until it finds a match.
Disclaimer: Tested/ works with Delphi 2007, Delphi 7 may be diffrent.
I don't see why so many people make this so hard. It is quite simple:
function ShouldDeleteLine(const UserID, Line: string): Boolean;
begin
// Remember: Pos(Needle, Haystack)
Result := Pos(UserID + '#', Line) = 1; // always 1-based!
end;
procedure DeleteLinesWithUserID(const FileName, UserID: string);
var
SL: TStringList;
I: Integer;
begin
if not FileExists(FileName) then
Exit;
SL := TStringList.Create;
try
SL.LoadFromFile(FileName); // Add exception handling for the
// case the file does not load properly.
// Always work backward when deleting items, otherwise your index
// may be off if you really delete.
for I := SL.Count - 1 downto 0 do
if ShouldDeleteLine(SL[I], UserID) then
begin
SL.Delete(I);
// if UserID is unique, you can uncomment the following line.
// Break;
end;
SL.SaveToFile(FileName);
finally
SL.Free;
end;
end;
As Arioch'The says, if you save to the same file name, you risk losing your data when the save fails, so you can do something like
SL.SaveToFile(FileName + '.dup');
if FileExists(FileName + '.old') then
DeleteFile(FileName + '.old');
RenameFile(FileName, FileName + '.old');
RenameFile(FileName + '.dup', FileName);
That keeps a backup of the original file as FileName + '.old'.
Explanations
Working backward
Why work backward? Because if you have the following items
A B C D E F G
^
And you delete the item at ^, then the following items will shift downward:
A B C E F G
^
If you iterate forward, you will now point to
A B C E F G
^
and E is never examined. If you go backward, then you will point to:
A B C E F G
^
Note that E, F and G were examined already, so now you will indeed examine the next item, C, and you won't miss any. Also, if you go upward using 0 to Count - 1, and delete, Count will become one less and at the end, you will try to access past the boundary of the list. This can't happen if you work backwards using Count - 1 downto 0.
Using + '#'
If you append '#' and test for Pos() = 1, you will be sure to catch the entire UserID up to the delimiter, and not a line with a user ID that only contains the UserID you are looking for. IOW, if UserID is 'velthuis', you don't want to delete lines like 'rudyvelthuis#rvelthuis01#password' or 'velthuisresidence#vr#password2', but you do want to delete 'velthuis#bla#pw3'.
E.g. when looking for a user name, you look for '#' + UserName + '#' for the same reason.
There is the only way to actually "delete a line from the text file" - that is to create a new file with changed content, to REWRITE it.
So you better just do it explicitly.
And don't you forget about protecting from errors. Your current code might just destroy the file and leak memory, if any error occurs...
var sl: TStringList;
s, prefix: string;
i: integer; okay: Boolean;
fs: TStream;
begin
prefix := 'UserName' + '#';
okay := false;
fs := nil;
sl:=TStringList.Create;
Try /// !!!!
sl.LoadFromFile('filename');
fs := TFileStream.Create( 'filename~new', fmCreate or fmShareExclusive );
for i := 0 to Prev(sl.Count) do begin
s := sl[ i ];
if AnsiStartsStr( prefix, Trim(s) ) then
continue; // skip the line - it was our haunted user
s := s + ^M^J; // add end-of-line marker for saving to file
fs.WriteBuffer( s[1], length(s)*SizeOf(s[1]) );
end;
finally
fs.Free;
sl.Free;
end;
// here - and only here - we are sure we successfully rewritten
// the fixed file and only no are able to safely delete old file
if RenameFile( 'filename' , 'filename~old') then
if RenameFile( 'filename~new' , 'filename') then begin
okay := true;
DeleteFile( 'filename~old' );
end;
if not okay then ShowMessage(' ERROR!!! ');
end;
Note 1: See if check for username should be case-sensitive or case-ignoring:
http://www.freepascal.org/docs-html/rtl/strutils/ansistartsstr.html
http://www.freepascal.org/docs-html/rtl/strutils/ansistartstext.html
Note 2: in Delphi 7 SizeOf( s[1] ) is always equal to one because string is an alias to AnsiString. But in newer Delphi version it is not. It might seems tedious and redundant - but it might save a LOT of headache in future. Even better would be to have a temporary AnsiString type variable like a := AnsiString( s + ^m^J ); fs.WriteBuffer(a[1],Length(a));
So far everyone has been suggesting the use for a For..Then Loop but can I suggest a Repeat..While.
The traditional For..Loop is a good option but could be inefficient if you have a long list of Usernames (they are usually unique). Once found and deleted the For Loop continues until the end of the list. That's ok if you have a small list but if you have 500,000 Usernames and the one you want is at position 10,000 there is no reason to continue beyond that point.
Therefore, try this.
Function DeleteUser(Const TheFile: String; Const TheUserName: String): Boolean;
Var
CurrentLine: Integer;
MyLines: TStringlist;
Found: Boolean;
Eof: Integer;
Begin
MyLines := TStringlist.Create;
MyLines.LoadFromFile(TheFile);
CurrentLine := 0;
Eof := Mylines.count - 1;
Found := false;
Repeat
If Pos(UpperCase(TheUserName), UpperCase(MyLines.Strings[CurrentLine])) = 1 Then
Begin
MyLines.Delete(CurrentLine);
Found := True;
End;
Inc(CurrentLine);
Until (Found) Or (CurrentLine = Eof); // Jump out when found or End of File
MyLines.SaveToFile(TheFile);
MyLines.Free;
result := Found;
End;
Once called the function returns True or False indicating the Username was deleted or not.
If Not DeleteUsername(TheFile,TheUsername) then
ShowMessage('User was not found, what were you thinking!');
Just for fun, here's a compact solution, which I like for its readability.
const fn = 'myfile.txt';
procedure DeleteUser(id: integer);
var s:string; a:TStringDynArray;
begin
for s in TFile.ReadAllLines(fn) do
if not s.StartsWith(id.ToString + '#') then
a := a + [s];
TFile.WriteAllLines(fn, a);
end;
Obviously it's not the most efficient solution. This could run faster by not appending single items to the array, or by caching the search string.
And to search for other fields, you could use s.split(['#'])[0] to find the username, s.split(['#'])[1] for email, etc.
For those who like one-liners. This works too:
const fn = 'users.txt';
procedure DeleteUserRegExp(id: string);
begin
TFile.WriteAllText(fn,TRegEx.Replace(TFile.ReadAllText(fn),''+id+'\#.*\r\n',''))
end;
Explanation
It loads the content of a file into a string.
The string is sent to TRegEx.Replace
The regular expression searches for the username followed by the hash sign, then any character, and then a CRLF. It replaces it with an empty string.
The resulting string is then written to the original file
This is just for fun though, because I saw long code where I thought that this would be possible with a single line of code.

How do I save the contents of this 2d array to a file

I Need some help trying to save the contents of the 2d array into a file.
First of all im not sure what type the file should be etc .txt or dat.
I have edited the post so that the code is in text format not an image.
This is what ive got so far.
program CaptureTheSarum;
{$APPTYPE CONSOLE}
uses
SysUtils;
Const BoardDimension = 8;
Type
TBoard = Array[1..BoardDimension, 1..BoardDimension] Of String;
Var
Board : TBoard;
GameOver : Boolean;
StartSquare : Integer;
FinishSquare : Integer;
StartRank : Integer;
StartFile : Integer;
FinishRank : Integer;
FinishFile : Integer;
MoveIsLegal : Boolean;
PlayAgain : Char;
SampleGame : Char;
WhoseTurn : Char;
savedFile : text;
procedure InitialiseSave;
var
fileName : string;
begin
fileName := 'SavedGame.dat';
assignfile(savedfile,fileName);
if not fileexists(fileName)
then
begin
rewrite(savedfile);
closefile(savedfile)
end
{endif};
end;
procedure saveGame;
var
save : string;
RankNo,FileNo : integer;
begin
writeln('Would you like to save the Game?');
readln(save);
if (save = 'y') or (save = 'Y')
then
begin
reset(SavedFile);
write(SavedFile,board[fileno,Rankno]);
closeFile(SavedFile);
end
{endif};
To answer your main question, you can save a two-dimensional string array as follows:
procedure TForm9.FileSaveClick(Sender: TObject);
var
i, j: integer;
fn: string;
fs: TFileStream;
fw: TWriter;
begin
fn := 'c:\tmp\mychessfile.dat';
fs := nil;
fw := nil;
try
fs := TFileStream.Create(fn, fmCreate or fmShareDenyWrite);
fw := TWriter.Create(fs, 1024);
for i := 1 to BoardDimension do
for j := 1 to BoardDimension do
fw.WriteString(Board[i, j]);
finally
fw.Free;
fs.Free;
end;
end;
Subsequently you can read the file back to the array with:
procedure TForm9.FileReadClick(Sender: TObject);
var
i, j: integer;
fn: string;
fs: TFileStream;
fr: TReader;
begin
fn := 'c:\tmp\mychessfile.dat';
fs := nil;
fr := nil;
try
fs := TFileStream.Create(fn, fmOpenRead or fmShareDenyWrite);
fr := TReader.Create(fs, 1024);
for i := 1 to BoardDimension do
for j := 1 to BoardDimension do
Board[i, j] := fr.ReadString;
finally
fr.Free;
fs.Free;
end;
end;
As you see I chose the general purpose .dat extension, because the file will contain also binary data, like length of each text, data type etc. Those details are dealt with by the TWriter/TReader classes.
You should also consider the comments you received regarding choise of file structure.
For example, Googling for 'chess file format' (assuming you are working on a chess game), brings up Portable_Game_Notation and another reference from that page: Forsyth-Edwards Notation.
It seems you are trying to make some sort of board game (probably chess).
The main problem you are facing is that you haven't defined your board type as fixed size. You see in Delphi strings are of dynamic size. And while in older versions of Delphi they were limited to 255 characters in newer versions their size is only limited by available memory.
So you should change your board definition (array) to be of fixed type. For most board games you could use 2D array of Char.
TBoard = Array [0..7, 0..7] of Char;
On older non-Unicode versions of Delphi Char will be an AnsiChar which allows you to store 256 different characters or 256 different figures.
On newer Delphi versions that support Unicode you have even more possibilities.
Anyway the best thing about using static array of fixed type is that you can save the whole static array into a file with a single command
procedure SaveGame;
//When having fixed size types you can use File of Type to quickly get
//ability to store whole type at once.
//Note this only works for fixed sized records who don't contain any
//dynamic sized members (strings, dynamic arrays) and static arrays of
//fixed sized type (no strings or other dynamic sized arrays)
//
//With arrays it doesn't even matter whether they are one dimensional
//or multidimensional. but they need to be static
var Savefile: File of TBoard;
FileName: String;
begin
Filename := 'D:\Proba.txt';
//Assign file
Assignfile(Savefile,FileName);
//Check if the file exists if it does open it for editing (reser)
//else open it in rewrite mode which also automatically creates new
//file if the file doesn't exists
if not Fileexists(Filename) then
Rewrite(Savefile)
else
Reset(SaveFile);
//Becouse we have a file of fixed sized type we can write the whole
//type with just one Write command
//your program already know how many bytes it has to write
//
//Note if you want to store multiple savegames in a single file you
//need to use seek to move your current position
//And because we have file of type the seek moves the current position
//by N times of the type size
//So if the size of your type is 64 bytes calling Seek(YourFile,2)
//will move current position to the 128th byte
Write(SaveFile, Board);
//Close file
CloseFile(SaveFile);
end;
Reading the data from your file is done in similar way.
Read(Savefile, Board);
EDIT: If you are on older version of Delphi and the char does not allow you enough possibilities to store the state of your board cell you can always use array of integers like most other grid based games do.
INI file like this:
[d1]
1=element1
2=element2
...
[d2]
1=element1
...
But,recommend XML, like this:
<array>
<d1>
<element1>value1</element1>
<element2>value1</element2>
...
</d1>
<d2>
<element1>value1</element1>
...
</d2>
</array>

Sorting Racers in timing application

I am creating an application which uses the AMB MyLaps decoder P3 Protocols.
I can't get my head around a way to sort the racers out based on laps and lap times. For example, the person in 1st has done 3 laps, the person in 2nd has done 2 laps. But then how do I order a situation where 2 people are on the same lap?
This is the record I'm using to hold the information:
type
TTimingRecord = record
position: integer;
transId: integer;
racerName: string;
kartNumber: integer;
lastPassingN: integer;
laps: integer;
lastRTCTime: TDateTime;
bestTimeMs: Extended;
lastTimeMs: Extended;
gapTimeMs: Extended;
splitTimeMs: Extended;
timestamp: TDateTime;
end;
A new record is created for each racer.
The code I'm currently using is:
procedure sortRacers();
var
Pos, Pos2: Integer;
Temp: TTimingRecord;
GapTime: Extended;
begin
for Pos := 0 to length(DriversRecord)-1 do
begin
for Pos2 := 0 to Length(DriversRecord)-2 do
begin
if(DriversRecord[Pos2].laps < DriversRecord[Pos2+1].laps)then
begin
Temp := DriversRecord[Pos2];
DriversRecord[Pos2] := DriversRecord[Pos2+1];
DriversRecord[Pos2+1] := Temp;
end
else if DriversRecord[Pos2].laps = DriversRecord[Pos2+1].laps then
begin
if DriversRecord[Pos2].lastRTCTime > DriversRecord[Pos2+1].lastRTCTime then
begin
Temp := DriversRecord[Pos2];
DriversRecord[Pos2] := DriversRecord[Pos2+1];
DriversRecord[Pos2+1] := Temp;
end;
end;
end;
end;
for pos := 1 to length(DriversRecord) -1 do //Gap Time
begin
if DriversRecord[Pos].laps = DriversRecord[0].laps then
begin
DriversRecord[Pos].gapTimeMs := DriversRecord[Pos].lastRTCTime - DriversRecord[0].lastRTCTime;
DriversRecord[Pos].splitTimeMs := DriversRecord[Pos].lastRTCTime - DriversRecord[Pos-1].lastRTCTime;
end;
end;
end;
But doesn't work too well :)
I'm assuming from your comment to the question, that you have decomposed the problem into sorting and comparing, and that you have got the sorting part covered. Which leaves order comparison.
You need a function that will perform a lexicographic order comparison based first on the number of laps completed, and secondly on the time since the start of this lap. Basically it will look like this:
function CompareRacers(const Left, Right: TTimingRecord): Integer;
begin
Result := CompareValue(Left.laps, Right.laps);
if Result=0 then
Result := CompareDateTime(Left.lastRTCTime, Right.lastRTCTime);
end;
You'll find CompareValue in Math and CompareDateTime in DateUtils.
What I'm not sure about is what the sense of the lastRTCTime values is. You may need to negate the result of the call to CompareDateTime to get the result you desire.
Result := -CompareDateTime(Left.lastRTCTime, Right.lastRTCTime);
Also, what happens if there is overtaking during the lap? Presumably you won't be able to detect that until the racers complete the current lap.
Instead of doing the sort algorithm yourself, try this technique (if you have a Delphi version compatible) : Best way to sort an array
And your function could look like this :
uses Types;
function CustomSort(const Left, Right: TTimingRecord): Integer
begin
If (left.laps > right.laps) then
result := GreaterThanValue
else
if (left.laps < right.laps) then
result := LessThanValue
else
begin
// Same laps count... Test on LastRTCTime
if (left.lastRTCTime < right.lastRTCTime) then
result := GreaterThanValue1
else
if (left.lastRTCTime > right.lastRTCTime) then
result := LessThanValue
else
result := EqualsValue;
end;
end));
It might be easier to look at this as 2 separate sorts.
Obviously you know the bubble-sort method, so I will not go into that.
Make 2 passes on your sorting.
1st, you sort the laps.
2nd, you run through the entire list of sorted laps. find begin point and end point in array for identical lap-values. Do the sorting again from begin and end points, but this time compare only the secondary value. iterate through all identical secondary values if the count of identical values are larger than 1.
This code is about sorting data using an Index. Way faster than bubble-sort.
It is dynamic and provides for ability to sort from a start-point to an end-point in an array.
The code itself is bigger than Bubble-Sort, but not many algorithms can compare on speed.
The code (when understanding how it works) can easily be modified to suit most kinds of sorting. On an array of 65536 strings, it only need to do 17 compares (or there about)
Some more CPU Cycles per compare cycle compared with Bubble Sort, but still among the fastest methods.
To search is equally as fast as BTREE. The actual sorting is perhaps slower, but the data is easier manageable afterwards with no real need for balancing the tree.
Enjoy.
Note: The routine is not the full solution to the actual problem, but it provides the beginning of an extreemely fast approach.
TYPE
DynamicIntegerArray = ARRAY OF INTEGER;
DynamicStringArray = ARRAY OF STRING;
VAR
BinSortLo, BinSortMid, BinSortHi : INTEGER;
FUNCTION FindMid:INTEGER;
BEGIN
FindMid:=BinSortLo+((BinSortHi-BinSortLo) DIV 2);
END;
PROCEDURE ShiftIndexUpAndStorePointer(VAR ArrParamIndex: DynamicIntegerArray; HighBound:INTEGER);
VAR
x : INTEGER;
BEGIN
FOR x:=HighBound-1 DOWNTO BinSortMid DO ArrParamIndex[x+1] := ArrParamIndex[x];// Shift the index.
ArrParamIndex[BinSortMid]:=HighBound;// Store the pointer to index at its sorted place
END;
PROCEDURE BinarySortUp(CONST ArrParam:DynamicStringArray; VAR ArrParamIndex: DynamicIntegerArray; CONST LoBound,HighBound:INTEGER); OVERLOAD;
VAR
TempVar : STRING;
BEGIN
BinSortLo:=LoBound; BinSortHi:=HighBound; BinSortMid:=FindMid;
TempVar := ArrParam[HighBound];
REPEAT
IF TempVar>ArrParam[ArrParamIndex[BinSortMid]] THEN BinSortLo:=BinSortMid ELSE BinSortHi:=BinSortMid;
BinSortMid:=FindMid;
UNTIL (BinSortMid=BinSortLo); {OR (BinSortMid=BinSortHi);}
IF TempVar>ArrParam[ArrParamIndex[BinSortMid]] THEN INC(BinSortMid);// We always need a last check just in case.
ShiftIndexUpAndStorePointer(ArrParamIndex,HighBound);
END;
PROCEDURE DynamicCreateIndex(CONST ArrParam:DynamicStringArray; VAR ArrParamIndex: DynamicIntegerArray; CONST LoBound,HighBound:INTEGER);
VAR
x : INTEGER;
BEGIN
FOR x:=LoBound TO HighBound DO
BinarySortUp(ArrParam,ArrParamIndex,LoBound,x);
END;
BEGIN
{
1. Create your STRING ARRAY as a DynamicStringArray.
2. Create your INDEX ARRAY as a DynamicIntegerArray.
3. Set the size of these arrays to any INTEGER size and fill the strings with data.
4. Run a call to DynamicCreateIndex(YourStringArray,YourIndexArray,0,SizeOfArray
Now you have a sorted Index of all the strings.
}
END.

delphi TFileStream "out of memory"

I am having trouble with some Delphi code that uses TFileStream to read chunks of data from a file to a dynamic array. The original objective in writing the code is to compare the contents of two files that have the same size but potentially different date and time stamps to see if the contents are the same. This is done by reading the data from each file of the pair into separate dynamic arrays and comparing each byte of one array with the corresponding byte of the other.
The code makes multiple calls to TFileStream.Read. After about 75 calls, the program crashes with an 'Out of Memory' Error message.
It does not seem to matter how large the blocks of data that are read, it seems to be the number of calls that results in the error message.
The code is a function that I have written that is called elsewhere whenever the program encounters two files that it needs to compare (which, for reasons that I won't go into, could be forty or fifty different file pairs). The 'Out of Memory' error occurs whether it is a single file that is being read in small blocks, or multiple files that are being read in their entirety. It seems to be the number of calls that is the determinant of the error.
While I realize that there might be more elegant ways of achieving the comparison of the files than what I have shown below, what I would really like to know is what is wrong with the use of the TFileStream and/or SetLength calls that are causing the memory problems. I have tried freeing the memory after every call (as shown in the code) and it seems to make no difference.
I would be grateful if someone could explain what is going wrong.
function Compare_file_contents(SPN,TPN : String; SourceFileSize : int64) : boolean;
var
SF : TFileStream; //First file of pair for comparison
TF : TFileStream; //Second file of pair
SourceArray : TBytes; // Buffer array to receive first file data
TargetArray : TBytes; //Buffer array to receive second file data
ArrayLength : int64; //Length of dynamic array
Position : int64; //Position within files to start each block of data read
TestPosition : int64; //Position within dynamic arrays to compare each byte
MaxArrayLength : integer; //Maximum size for the buffer arrays
LastRun : Boolean; //End first repeat loop
begin
{ The comparison has an arbitrary upper boundary of 100 MB to avoid slowing the
the overall program. The main files bigger than this will be *.pst files that
will most likely have new dates every time the program is run, so it will take
about the same time to copy the files as it does to read and compare them, and
it will have to be done every time.
The function terminates when it is confirmed that the files are not the same.
If the source file is bigger than 100 MB, it is simply assumed that they are
not identical, thus Result = False. Also, LongInt integers (=integers) have
a range of -2147483648..2147483647, so files bigger than 2 GB will have
overflowed to a negative number. Hence the check to see if the file size is
less than zero.
The outer repeat ... until loop terminates on LastRun, but LastRun should only
be set if SecondLastRun is True, because it will skip the final comparisons in
the inner repeat ... until loop otherwise. }
Result := True;
LastRun := False;
MaxArrayLength := 1024*1024;
if (SourceFileSize > 100*1024*1024) or (SourceFileSize < 0) then Result := False
else
begin
{ The comparison is done by using TFileStream to open and read the data from
the source and target files as bytes to dynamic arrays (TBytes). Then a repeat
loop is used to compare individual bytes until a difference is found or all
of the information has been compared. If a difference is found, Result is
set to False. }
if SourceFileSize > MaxArrayLength then ArrayLength := MaxArrayLength
else ArrayLength := SourceFileSize;
SF := TFileStream.Create(SPN,fmOpenRead);
TF := TFileStream.Create(TPN,fmOpenRead);
Position := 0;
SetLength(SourceArray,ArrayLength);
SetLength(TargetArray,ArrayLength);
try
SF.Read(SourceArray,ArrayLength);
TF.Read(TargetArray,ArrayLength);
Position := SF.Position;
finally
SF.Free;
TF.Free;
end;
repeat
TestPosition := 0;
repeat
if SourceArray[TestPosition] <> TargetArray[TestPosition] then
Result := False;
Inc(TestPosition);
until (Result = False) or (TestPosition = ArrayLength);
if SourceFileSize > Position then
begin
if SourceFileSize - Position - MaxArrayLength > 0 then
ArrayLength := MaxArrayLength
else ArrayLength := SourceFileSize - Position;
SF := TFileStream.Create(SPN,fmOpenRead);
TF := TFileStream.Create(TPN,fmOpenRead);
SF.Position := Position;
TF.Position := Position;
try
SF.Read(SourceArray,ArrayLength);
TF.Read(TargetArray,ArrayLength);
Position := SF.Position;
finally
SF.Free;
TF.Free;
end;
end else LastRun := True;
until (Result = False) or LastRun;
Finalize(SourceArray);
Finalize(TargetArray);
end;
end; { Compare_file_contents }
This routine seems to be far more complicated than it needs to be. Rather than trying to debug it, I offer you my routine that compares streams.
function StreamsEqual(Stream1, Stream2: TStream): Boolean;
const
OneKB = 1024;
var
Buffer1, Buffer2: array [0..4*OneKB-1] of Byte;
SavePos1, SavePos2: Int64;
Count: Int64;
N: Integer;
begin
if Stream1.Size<>Stream2.Size then begin
Result := False;
exit;
end;
SavePos1 := Stream1.Position;
SavePos2 := Stream2.Position;
Try
Stream1.Position := 0;
Stream2.Position := 0;
Count := Stream1.Size;
while Count <> 0 do begin
N := Min(SizeOf(Buffer1), Count);
Stream1.ReadBuffer(Buffer1, N);
Stream2.ReadBuffer(Buffer2, N);
if not CompareMem(#Buffer1, #Buffer2, N) then begin
Result := False;
exit;
end;
dec(Count, N);
end;
Result := True;
Finally
Stream1.Position := SavePos1;
Stream2.Position := SavePos2;
End;
end;
If you wish to add your 100MB size check to this function, it's obvious where and how to do it.
The routine above uses a stack allocated buffer. In contrast your version allocates on the heap. Perhaps your version leads to heap fragmentation.
I realise that this does not answer the direct question that you asked. However, it does solve your problem. I hope this proves useful.

Resources