Good morning all.
I'm currently trying to figure out something that i'm confident is simple enough but is proving to be a task and a half to actually work out.
I'm working on a project that's designed to minimize drive usage by relocating various files elsewhere. I've got an array (0..12) of int64 values that contains the file sizes of the files i might potentially want to move. The array is ordered in a way that's predicted largest file size down to predicted smallest file size. I've also got the names of these files stored in a different array (known as WoWData, also [0..12]). I've then got an "installation size", and a "desired size".
My task is to calculate which files i need to move in order to bring the "installation size" down to the "desired size" by going through the array of file sizes, and taking the value away from the Installation size until i reach <= desired size.
Here's some sample code (Delphi/Firemonkey) i've been trying to work with. It's confusing me trying to figure out how to go about such a task and so there'll no doubt be a lot of issues with it;
Global Vars;
_WoWDataFileSize : Array [0..12] of Int64;
// "TBWoWDir" is a TTrackBar (Firemonkey)
var
TotalSize, ReqSize, DiffSize, CurDiff : Int64;
i : Integer;
begin
// Set up initial values to work with
ReqSize := Round(TBWoWDir.Value); // Requested Size
TotalSize := Round(TBWoWDir.Max); // Actual installation size
CurDiff := 0; // Assume as "Current Difference in size"
// Calculate difference between install and requested size
DiffSize := TotalSize - ReqSize; // This calculates correctly
// The below is what i'm struggling with
repeat
for i := Low(_WoWDataFileSize) to High(_WoWDataFileSize) do
begin
CurDiff := ReqSize - _WoWDataFileSize[i];
end;
until CurDiff <= ReqSize;
end;
I did try using just a repeat .. until loop without the for loop, but again, i'm getting far too confused while trying to figure it out.
Let me provide an example. Let's assume that _WoWDataFileSize[0] is 200, and _WoWDataFileSize[1] through to _WoWDataFileSize[12] are the same value as their array index (e.g. _WoWDataFileSize[6] = 6, _WoWDataFileSize[8] = 8, etc).
If i wanted to calculate the value of 150 (which would be 200 - 12 - 11 - 10 - 9 - 8, or Array[0] - Array[12] - Array[11] - Array[10] - Array[9] - Array[8] according to the array), and get a list of files i need to move to meet this requirement from the WoWData array, how would i write the routine?
150 could be replaced by any number as i'm working towards a dynamic user-requested size specified by TBWoWDir.Value.
I'm thinking i might need to do a While loop and use i := i+1 setup. Realistically, i could go through and hardcode it so it takes away one value in the array at a time and check each time to see if i'm <= desired value-- it'd be 2-3 lines for each item (so a total of 24-36 lines), but this is both messy to maintain and not optimal. I'm interested to see how it would be done in a loop. I typically don't have trouble with loops, but this is hardly a standard one for me.
curdiff:= 0;
i:= Low(_WoWDataFileSize) - 1;
while (curdiff <= reqsize) and (i < High(_WoWDataFileSize)) do
begin
inc (i);
curdiff:= curdiff + _WoWDataFileSize[i];
end;
At the end of the loop, either you've attained the required reduction in size or you've iterated through the entire array.
It is IMHO just two line missing in your code :o)
CurDiff := ReqSize;
// repeat
for i := Low(_WoWDataFileSize) to High(_WoWDataFileSize) do
begin
CurDiff := CurrDiff - _WoWDataFileSize[i];
if CurDiff <= ReqSize then break; // breaks the for..to loop
end;
// until CurDiff <= ReqSize;
EDIT No need for the repeat...until loop
But IMHO it is not very useful only to count the sizes without storing which files match.
So using a CustomObject and Lists (thanx to Generics) it will be very simple:
type
TFileObject = class
private
FName : string;
FSize : Int64;
public
constructor Create( AName : string; ASize : Int64 );
published
property Name : string read FName;
property Size : Int64 read FSize;
end;
procedure MoveFileObject(AMaxSize : Int64; ASrcList, ATarList : TList<TFileObject> );
var
LItem : TFileObject;
LSize : Int64;
begin
LSize := 0;
for LItem in ASrcList do
begin
if LSize + LItem.Size <= AMaxSize then
begin
LSize := LSize + LItem.Size;
ATarList.Add( LItem );
end;
end;
end;
Thanks to everyone for their answers, i figured out where i was going wrong. When i was calculating in my initial question, i'd forgotten to account for my division on the values (for the sake of showing MB instead of Bytes as TBWoWDir.Value was livebound to a TLabel.text, but the actual size was being divided before assigning TBWoWDir.Max).
Thanks to a few tweaks from an answer by No'am Newman, i managed to figure this out for myself. Here's how i got the result i was after (or much closer to it);
Global Vars;
_WoWDataFileSize : Array [0..12] of Int64;
Global Const;
_WoWData : Array [0..12] of String;
// "TBWoWDir" is a TTrackBar (Firemonkey)
[...]
var
ReqSize : int64;
DiffSize, CurDiff : Int64;
i, ii : Integer;
FilesTot : Integer;
FILESMSG : String;
begin
// Set up initial values to work with
ReqSize := Round(TBWoWDir.Value) * 1024 * 1024; // Requested Size - Multiplied from formatting
TotalSize := Round(TBWoWDir.Max) * 1024 * 1024; // Actual installation size - Multiplied from formatting
DiffSize := TotalSize - ReqSize; // Calculate Difference
CurDiff := 0; // Reset Current Difference
i := -1; // Reset i
repeat
inc (i); // Increment i
CurDiff := CurDiff + _WoWDataFileSize[i]; // Add current array item file size to CurDiff
until (CurDiff >= (DiffSize)) or (i >= 12); // Repeat until we reach ideal size or the end of the array
// Calculate which array item we stopped at
for ii := 0 to i do // use i from previous loop as the max
begin
FILESMSG := FILESMSG + 'File: ' + WoWData[ii] +
' | Size: ' + IntToStr(_WoWDataFileSize[ii])+' '#13#10;
FilesTot := FilesTot + _WoWDataFileSize[ii];
end;
// Show Message providing details
ShowMessage('CurDiff:' + IntToStr(CurDiff div 1024 div 1024) +
' | DiffSize: ' + IntToStr(DiffSize div 1024 div 1024) +
' | Array i: ' +
IntToStr(i) +#13#10+
'Difference between CurDiff and DiffSize: '+ IntToStr(((DiffSize div 1024 div 1024) - (CurDiff div 1024 div 1024)))+#13#10#13#10+
'File Details' +#13#10#13#10+
FilesMsg +#13#10#13#10+
'Total Size: ' + IntToStr(FilesTot));
end;
The code is there to tell me which files i need to copy (so modifying it to copy the files now isn't too difficult), and the whole ShowMessage is there for self-proof (as i use ShowMessage during development when i need to verify a value is returning correctly, as i'm sure many others do as well).
Related
Please help:
I have two edit boxes on my form. The first one I use to type in an amount. The second one I use to divide the amount with. The problem is I try a number with a decimal like 5.5 and I keep on getting the error: "'5.5' is not a valid integer value".
Here is the code that I use:
var igroei,ipen, iper : integer;
rgroei, rper : real;
begin
ipen := strtoint(edtpen.Text); //the amount enter like 35060
iper := strtoint(edtper.Text); // The number use for the percentage like 5.5
iper := iper div 100;
rgroei := ipen + iper;
pnlpm.Caption := floattostrF(rgroei,ffcurrency,8,2);
end;
Thank you
5.5 is indeed not a valid integer. It is a floating point value. Use StrToFloat() instead of StrToInt(), and use Extended instead of Integer for the variable type.
var
ipen, iper, rgroei : Extended;
begin
ipen := StrToFloat(edtpen.Text); //the amount enter like 35060
iper := StrToFloat(edtper.Text); // The number use for the percentage like 5.5
iper := iper / 100.0;
rgroei := ipen + iper;
pnlpm.Caption := FloatToStrF(rgroei, ffcurrency, 8, 2);
end;
You should read the following to get started:
Integer and floating point numbers: The different number types in Delphi
I have 2 tables like this
As you can see, if you look at Total you can see the score of each player in 3 rounds. I have to do a list (from the 1st to the 12th) indicating the highest score.
Here the player with 28 points, must have the number 1 (instead of that 8 which is generated by default), the player with 22 must have the number 2 instead of 11... So I have to sort the TOTAL columns and return the position in the correct label.
When I click the button I underlined, the procedure is called:
var vettore:array[1..12] of integer;
indici:array[1..12] of integer;
i:smallint;
begin
for i := 1 to 6 do
begin
vettore[i]:= StrToInt(StringGrid1.Cells[5,i]); //col,row
indici[i] := i;
end;
for i := 6 to 12 do
begin
vettore[i]:= StrToInt(StringGrid2.Cells[5,i]); //col,row
indici[i] := i;
end;
In this way I load inside vettore all the TOTAL numbers in the rows of both tables, and in indici you can find the number of the label on the right of the table (they indicates the position). Now I thought I could use any sorting method since I have only 12 elements (like the Quick Sort).
My problem is this: how can I change the labels texts (the ones on right of the tables) according with the sorted array? It's like the picture above shows.
Every label is called (starting from 1) mvp1, mvp2, mvp3, mvp4... I think this can be helpful because if (maybe) I will have to do a for loop for change the text of each label, I can use a TFindComponent.
If it could be helpful, here there is the function I wrote with javascript on my website (it works):
var totals = [], //array with the scores
indices = []; //array with the indices
for (var i=0; i<6; i++) {
totals[i] = parseInt(document.getElementById('p'+i).value, 10);
indices[i] = i;
}
for (var i=6; i<12; i++) {
totals[i] = parseInt(document.getElementById('p'+i).value, 10);
indices[i] = i;
}
indices.sort(function(a, b) {
return totals[b]- totals[a];
});
for (var i=0; i<indices.length; i++) {
document.getElementById('mvp'+(indices[i]+1)).value = (i+1);
}
AS. Since only delphi is listed in tags, that means that any Delphi version is okay. I'd refer to delphi-xe2.
1st we would use Advanced Records to hold the data for a single participant. Some links are below, google for more.
http://docwiki.embarcadero.com/RADStudio/XE5/en/Structured_Types#Records_.28advanced.29
http://delphi.about.com/od/adptips2006/qt/newdelphirecord.htm
http://sergworks.wordpress.com/2012/03/13/record-constructors-in-delphi/
.
type
TClanResults = record
public
type All_GPs = 1..3;
var GP: array [All_GPs] of Cardinal;
var Players: string;
var Clan_ID: integer;
private
function CalcTotal: Cardinal;
function CalcAverage: single; inline;
public
property Total: Cardinal read CalcTotal;
property AVG: single read CalcAverage;
end;
{ TClanResults }
function TClanResults.CalcAverage: single;
begin
Result := Self.Total * ( 1.0 / Length(GP) );
end;
function TClanResults.CalcTotal: Cardinal;
var score: cardinal;
begin
Result := 0;
for score in GP do
Inc(Result, score);
end;
The expression Self.Total * ( 1.0 / Length(GP) ); can be also written as Self.Total / Length(GP). However i'd like to highlight some Delphi quirks here.
in Pascal there are two division operators: float and integer; 3 div 2 = 1 and 3 / 2 = 1.5. Choosing wrong one causes compilation errors at best and data precision losses at worst.
I'd prefer explicit typecast from integer Length to float, but Delphi does not support it. So i multiply by 1.0 to cast. Or i may add 0.0.
Division takes a lot longer than multiplication - just do it with pen and paper to see. When you have a data-crunching loop, where all elements are divided by the same number, it is good idea to cache 1 / value into a temp variable, and then mutiply each element by it instead. Since GP is of fixed size, it is compiler that calculates (1.0 / Length(GP)) and substitutes this constant. If you would allow different clans to have different amount of games - and turn GP into being dynamic arrays of different sizes - you would be to explicitly add a variable inside the function and to calc coeff := 1.0 / Length(GP); before loop started.
Now we should make a container to hold results and sort them. There can be several approaches, but we'd use generics-based TList<T>.
http://docwiki.embarcadero.com/Libraries/XE5/en/System.Generics.Collections.TList
http://docwiki.embarcadero.com/Libraries/XE5/en/System.Generics.Collections.TList.Sort
http://docwiki.embarcadero.com/Libraries/XE5/en/System.Generics.Defaults.TComparer.Construct
http://docwiki.embarcadero.com/Libraries/XE5/en/System.Generics.Defaults.TComparison
The TList is an object, so you would have to CREATE it and to FREE it. I think you can make it a PUBLIC property of your MainForm, then create the list in TMainForm.OnCreate event and free it in TMainForm.OnDestroy event.
Another, lazier approach, would be using a regular dynamic array and its extensions.
http://docwiki.embarcadero.com/RADStudio/XE5/en/Structured_Types#Dynamic_Arrays
http://docwiki.embarcadero.com/Libraries/XE5/en/System.TArray
http://docwiki.embarcadero.com/Libraries/XE5/en/System.SetLength
http://docwiki.embarcadero.com/Libraries/XE5/en/System.Generics.Collections.TArray.Sort
http://docwiki.embarcadero.com/CodeExamples/XE5/en/Generics_Collections_TArray_(Delphi)
However, i'll use TList below. Again, i assume that other routines in you program already and correctly create and destroy the given var ClanData: TList<TClanResults>; object instance.
type
TClansTable = TList<TClanResults>;
procedure TMainForm.Input;
var row: TClanResults
begin
Self.ClanData.Clear;
row.Clan_ID := 1;
row.Players := JclStringList.Add(['John', 'James', 'Jenny']).Join(' and ');
row.GP[1] := 2;
row.GP[1] := 5;
row.GP[1] := 7;
Self.ClanData.Add(row);
row.Clan_ID := 2;
row.Players := JclStringList.Add(['Mary', 'Mark', 'Marge']).Join(' and ');
row.GP[1] := 3;
row.GP[1] := 6;
row.GP[1] := 2;
Self.ClanData.Add(row);
...
end;
procedure SortOnTotal(const Table: TClansTable);
begin
Table.Sort(
TComparer<TClanResults>.Construct(
function(const Left, Right: TClanResults): Integer
begin Result := - (Left.Total - Right.Total) end
// negating since we need reversed order: large to little
)
);
end;
Now finally we need to know how to show that table on the screen. I would use typical TStringGrid as the most simplistic widget. I suggest you to look some advanced string grid from JediVCL or something from Torry.net so you would be able to specify columns styles. It is obvious that integers should be right-aligned on the screen and averages should be comma-aligned. However stock TStringGrid does not have kind of GetCellStyle event, so you would need some advanced grid derivative to add it. It is left as your home-task.
http://docwiki.embarcadero.com/RADStudio/XE5/en/String_Grids
http://docwiki.embarcadero.com/Libraries/XE5/en/Vcl.Grids.TStringGrid_Properties
Delphi TStringGrid Flicker - remains as your homework too.
.
procedure TMainForm.DumpTableToGrid(const Data: TClansTable; const grid: TStringGrid);
const TableFields = 8;
var row: integer;
ss: array of string;
res: TClanResults;
procedure DumpTheRow; var col: integer;
begin
for col := 0 to TableFields - 1 do begin
grid.Cells[ col, row ] := ss[ col ];
end;
begin
grid.Options := [ goFixedVertLine, goVertLine, goHorzLine, goColSizing, goColMoving, goThumbTracking ];
grid.ColCount := TableFields;
SetLength( ss, TableFields );
grid.RowCount := 1 + Data.Count;
grid.FixedRows := 1;
grid.FixedColumns := 1;
row := 0; // headers
ss[0] := ''; // number in the row, self-evident
ss[1] := 'Players';
ss[2] := 'GP 1';
....
ss[7] := 'Clan ID';
DumpTheRow;
for res in Data do begin // we assume Data already sorted before calling this
Inc(row);
ss[0] := IntToStr( row );
ss[1] := res.Players;
ss[2] := IntToStr( res.GP[1] );
...
ss[6] := FloatToStrF( res.AVG, ffFixed, 4, 2);
ss[7] := IntToStr( res.Clan_ID );
DumpTheRow;
end;
end;
Now, it is unclear what you mean by those labels. I can guess, that you want to show there ranks according to both your two clans combined positions. The externals labels are a bad idea for few reasons.
FindComponent is not too fast. Okay, you may find them once, cache in array of TLabel and be done. But why bother with extra workarounds?
user may resize the window, making it taller or shorter. Now there are 3 labels visible, in a minute there would be 30 labels visible, in a minute there will be 10 labels... How would you re-generate them in runtime ? So there would be enough of those always and in proper positions ? Actually just put them into the grid itself.
VCL sucks at form scaling. Now that Winodws 8.1 is out the fonts resolution might be different on different displays. There would be usually 96DPI on you main display, but as you would drag the window onto your secondary display there would be 120DPI, and on your mate's laptop (examples: Lenovo ThinkPad Yoga Pro and Lenovo IdeaPad Yoga 2) there might be like 200DPI or Retina-grade 300DPI. Still you would have to control your labels so their text would be shown exactly to the right of grid rows text, no matter what value would be rows of each height and each font.
So, i think they should be INSIDE the row. If you want to highlight them - use bold font, or coloured, or large, or whatever inside the grid.
TRanks = record min, max: word; end;
TClanResults = record
...
RanksCombined: TRanks;
...
end;
You correctly shown that some clans might have the same results and share the rank.
Before continuing you, as a JS user, have to notice a basis difference between record and class datatypes. record is operated by value while class is operated by reference. That means for class instances and variables you have to manually allocate memory for new elements and to dispose it for no longer used ones. Since class variable is a reference to some anonymous class instance(data). Hence the different containers of class-type elements can point to the single real element(data, instance), providing for easy data changing and cheaper sorting. Then for record instances (and record variable IS record data) you don't care about memory allocation and life times, yet would have copying data between different record instances, and if you change the one instance, to apply it to other containers you would have to copy it back. This difference is very visible in for element in container loops, whether we can change element.field or not.
http://docwiki.embarcadero.com/Libraries/XE2/en/System.Generics.Collections.TObjectList.Create
So let us have few more data structures for sorting and calculating. For example
TAvgAndRanks = class
avg: single; rank: TRanks;
table: TClansTable; idx: integer;
end;
We'll have then modification for the data dumper:
procedure TMainForm.DumpTableToGrid(const Data: TClansTable; const grid: TStringGrid);
const TableFields = 9;
...
row := 0; // headers
....
ss[7] := 'Clan ID';
ss[8] := 'Rank';
DumpTheRow;
...
ss[7] := IntToStr( res.Clan_ID );
with res.RanksCombined do
if min = max
then ss[9] := IntToStr(min)
else ss[9] := IntToStr(min) + ' - ' + IntToStr(max);
DumpTheRow;
Another approach would be to keep ranks externally using something like
TClanPtr = record table: TClansTable; idx: integer; end;
TClanSortData = record avg: single; rank: TRanks; end;
TClanRanksCombined = TDictionary<TClanPtr, TClanSortData>;
This approach is more extensible (allows in different window "attach" different extended data to the clans), but would require much more boilerplate. If you liek it more, your homework would be to implement it.
procedure MakeRanks(const clans: array of TClansTable);
var tab: TClansTable; idx: integer;
total: TObjectList<TAvgAndRanks>;
ar : TAvgAndRanks;
res: TClanResults;
// for spanning ranks with same avg
r_curr, r_min: word;
r_span, r_idx: integer;
r_avg: single;
r_chg: boolean;
begin
total := TObjectList<TAvgAndRanks>.Create( True ); // auto-free by container
try
for tab in clans do
for idx := 0 to tab.Count - 1 do begin
res := tab[ idx ];
ar := TAvgAndRanks.Create; // but creation is still manual
ar.table := tab;
ar.idx := idx;
ar.avg := res.AVG;
total.Add(ar);
end;
if total.Count <= 0 then Abort;
if total.Count = 1 then begin
ar := total[0];
res := ar.table[ ar.idx ];
res.RanksCombined.min := 1;
res.RanksCombined.max := 1;
ar.table[ ar.idx ] := res; // copying back updated data
Exit; // from procedure - nothing to do
end;
total.Sort(
TComparer<TAvgAndRanks>.Construct(
function(const Left, Right: TAvgAndRanks): Integer
begin Result := - (Left.avg - Right.avg) end
// negating since we need reversed order: large to little
)
);
(***** calculating ranks with spans ****)
r_curr := 1;
r_min := 1;
r_span := 0;
r_idx := 0;
r_avg := total[0].avg;
for idx := 1 to total.Count - 1 do begin
ar := total[ idx ];
inc(r_curr);
if r_avg = ar.avg then inc(r_span);
if (r_avg <> ar.avg) or (idx = total.Count - 1) then begin
for r_idx := r_idx to r_idx + r_span do begin
with total[ r_idx ] do begin // class == reference, can update directly
rank.min := r_min;
rank.max := r_min + r_span;
end;
end;
Assert( (r_curr = r_min + r_span + 1) or ( r_avg = ar.avg ) );
r_min := r_curr;
r_span := 0;
r_idx := idx;
r_avg := ar.avg;
end;
end;
(*** saving calculated ranks ***)
for ar in total do begin
res := ar.table[ ar.idx ];
res.RanksCombined := ar.ranks;
ar.table[ ar.idx ] := res; // copying back updated data
end;
finally
Total.Destroy;
end;
end;
How can I insert a string at a specified index in TMemoryStream? If you added a string
"a" to an existing string "b" at Index 0 it would move it forward "ab" etc. For example this is what TStringBuilder.Insert does.
Expand the stream so that there is room for the text to be inserted. So, if the text to be inserted has length N, then you need to make the stream N bytes larger.
Copy all the existing content, starting from the insertion point, to the right to make room for the insertion. A call to Move will get this done. You'll be moving this text N bytes to the right.
Write the inserted string at the insertion point.
I'm assuming an 8 bit encoding. If you use a 16 bit encoding, then the stream needs to be grown by 2N bytes, and so on.
You will find that this is a potentially expensive operation. If you care about performance you will do whatever you can to avoid ever having to do this.
P.S. I'm sorry if I have offended any right-to-left language readers with my Anglo-centric assumption that strings run from left to right!
You asked for some code. Here it is:
procedure TMyStringBuilder.Insert(Index: Integer; const Str: string);
var
N: Integer;
P: Char;
begin
N := Length(Str);
if N=0 then
exit;
FStream.Size := FStream.Size + N*SizeOf(Char);
P := PChar(FStream.Memory);
Move((P + Index)^, (P + Index + N)^, (FStream.Size - N - Index)*SizeOf(Char));
Move(Pointer(Str)^, (P + Index)^, N*SizeOf(Char));
end;
Note that I wrote this code, and then looked at the code in TStringBuilder. It's pretty much identical to that!
The fact that the code you end up writing for this operation is identical to that in TStringBuilder should cause you to stop and contemplate. It's very likely that this new string builder replacement class that you are building will end up with the same implementation as the original. It's highly likely that your replacement will perform no better than the original, and quite plausible that the performance of the replacement will be worse.
It looks a little to me as though you are optimising prematurely. According to your comments below you have not yet timed your code to prove that time spent in TStringBuilder methods is your bottleneck. That's really the first thing that you need to do.
Assuming that you do this timing, and prove that TStringBuilder methods are your bottleneck, you then need to identify why that code is performing below par. And then you need to work out how the code could be improved. Simply repeating the implementation of the original class is not going to yield any benefits.
To move the existing data to make some room for the new string data, you can use pointer operation and Move procedure for faster operation. But it only has to be done if the insertion index is lower then the size of the original stream. If the index is larger than stream size then you could: (1) expand the stream size to accomodate the index number and fill the extra room with zero values or spaces, or (2) reduce the index value to the stream size, so the string will be inserted or appended in the end of the stream.
Depend on your needs, you could: (1) make a class derived from TMemoryStream, or (2) make a function to process an instance of TMemoryStream. Here's the first case:
type
TExtMemoryStream = class(TMemoryStream)
public
procedure InsertString(Index: Integer; const S: string);
end;
procedure TExtMemoryStream.InsertString(Index: Integer; const S: string);
var
SLength, OldSize: Integer;
Src, Dst, PointerToS: ^Char;
begin
if Index > Size then Index := Size;
SLength := Length(S);
OldSize := Size;
SetSize(Size + SLength);
Src := Memory; Inc(Src, Index);
Dst := Src; Inc(Dst, SLength);
Move(Src^, Dst^, OldSize - Index);
PointerToS := #S[1];
Move(PointerToS^, Src^, SLength);
end;
or the second case:
procedure InsertStringToMemoryStream(MS: TMemoryStream;
Index: Integer; const S: string);
var
SLength, OldSize: Integer;
Src, Dst, PointerToS: ^Char;
begin
if Index > MS.Size then Index := MS.Size;
SLength := Length(S);
OldSize := MS.Size;
MS.SetSize(MS.Size + SLength);
Src := MS.Memory; Inc(Src, Index);
Dst := Src; Inc(Dst, SLength);
Move(Src^, Dst^, OldSize - Index);
PointerToS := #S[1];
Move(PointerToS^, Src^, SLength);
end;
There, hope it helps :)
I am having trouble with some Delphi code that uses TFileStream to read chunks of data from a file to a dynamic array. The original objective in writing the code is to compare the contents of two files that have the same size but potentially different date and time stamps to see if the contents are the same. This is done by reading the data from each file of the pair into separate dynamic arrays and comparing each byte of one array with the corresponding byte of the other.
The code makes multiple calls to TFileStream.Read. After about 75 calls, the program crashes with an 'Out of Memory' Error message.
It does not seem to matter how large the blocks of data that are read, it seems to be the number of calls that results in the error message.
The code is a function that I have written that is called elsewhere whenever the program encounters two files that it needs to compare (which, for reasons that I won't go into, could be forty or fifty different file pairs). The 'Out of Memory' error occurs whether it is a single file that is being read in small blocks, or multiple files that are being read in their entirety. It seems to be the number of calls that is the determinant of the error.
While I realize that there might be more elegant ways of achieving the comparison of the files than what I have shown below, what I would really like to know is what is wrong with the use of the TFileStream and/or SetLength calls that are causing the memory problems. I have tried freeing the memory after every call (as shown in the code) and it seems to make no difference.
I would be grateful if someone could explain what is going wrong.
function Compare_file_contents(SPN,TPN : String; SourceFileSize : int64) : boolean;
var
SF : TFileStream; //First file of pair for comparison
TF : TFileStream; //Second file of pair
SourceArray : TBytes; // Buffer array to receive first file data
TargetArray : TBytes; //Buffer array to receive second file data
ArrayLength : int64; //Length of dynamic array
Position : int64; //Position within files to start each block of data read
TestPosition : int64; //Position within dynamic arrays to compare each byte
MaxArrayLength : integer; //Maximum size for the buffer arrays
LastRun : Boolean; //End first repeat loop
begin
{ The comparison has an arbitrary upper boundary of 100 MB to avoid slowing the
the overall program. The main files bigger than this will be *.pst files that
will most likely have new dates every time the program is run, so it will take
about the same time to copy the files as it does to read and compare them, and
it will have to be done every time.
The function terminates when it is confirmed that the files are not the same.
If the source file is bigger than 100 MB, it is simply assumed that they are
not identical, thus Result = False. Also, LongInt integers (=integers) have
a range of -2147483648..2147483647, so files bigger than 2 GB will have
overflowed to a negative number. Hence the check to see if the file size is
less than zero.
The outer repeat ... until loop terminates on LastRun, but LastRun should only
be set if SecondLastRun is True, because it will skip the final comparisons in
the inner repeat ... until loop otherwise. }
Result := True;
LastRun := False;
MaxArrayLength := 1024*1024;
if (SourceFileSize > 100*1024*1024) or (SourceFileSize < 0) then Result := False
else
begin
{ The comparison is done by using TFileStream to open and read the data from
the source and target files as bytes to dynamic arrays (TBytes). Then a repeat
loop is used to compare individual bytes until a difference is found or all
of the information has been compared. If a difference is found, Result is
set to False. }
if SourceFileSize > MaxArrayLength then ArrayLength := MaxArrayLength
else ArrayLength := SourceFileSize;
SF := TFileStream.Create(SPN,fmOpenRead);
TF := TFileStream.Create(TPN,fmOpenRead);
Position := 0;
SetLength(SourceArray,ArrayLength);
SetLength(TargetArray,ArrayLength);
try
SF.Read(SourceArray,ArrayLength);
TF.Read(TargetArray,ArrayLength);
Position := SF.Position;
finally
SF.Free;
TF.Free;
end;
repeat
TestPosition := 0;
repeat
if SourceArray[TestPosition] <> TargetArray[TestPosition] then
Result := False;
Inc(TestPosition);
until (Result = False) or (TestPosition = ArrayLength);
if SourceFileSize > Position then
begin
if SourceFileSize - Position - MaxArrayLength > 0 then
ArrayLength := MaxArrayLength
else ArrayLength := SourceFileSize - Position;
SF := TFileStream.Create(SPN,fmOpenRead);
TF := TFileStream.Create(TPN,fmOpenRead);
SF.Position := Position;
TF.Position := Position;
try
SF.Read(SourceArray,ArrayLength);
TF.Read(TargetArray,ArrayLength);
Position := SF.Position;
finally
SF.Free;
TF.Free;
end;
end else LastRun := True;
until (Result = False) or LastRun;
Finalize(SourceArray);
Finalize(TargetArray);
end;
end; { Compare_file_contents }
This routine seems to be far more complicated than it needs to be. Rather than trying to debug it, I offer you my routine that compares streams.
function StreamsEqual(Stream1, Stream2: TStream): Boolean;
const
OneKB = 1024;
var
Buffer1, Buffer2: array [0..4*OneKB-1] of Byte;
SavePos1, SavePos2: Int64;
Count: Int64;
N: Integer;
begin
if Stream1.Size<>Stream2.Size then begin
Result := False;
exit;
end;
SavePos1 := Stream1.Position;
SavePos2 := Stream2.Position;
Try
Stream1.Position := 0;
Stream2.Position := 0;
Count := Stream1.Size;
while Count <> 0 do begin
N := Min(SizeOf(Buffer1), Count);
Stream1.ReadBuffer(Buffer1, N);
Stream2.ReadBuffer(Buffer2, N);
if not CompareMem(#Buffer1, #Buffer2, N) then begin
Result := False;
exit;
end;
dec(Count, N);
end;
Result := True;
Finally
Stream1.Position := SavePos1;
Stream2.Position := SavePos2;
End;
end;
If you wish to add your 100MB size check to this function, it's obvious where and how to do it.
The routine above uses a stack allocated buffer. In contrast your version allocates on the heap. Perhaps your version leads to heap fragmentation.
I realise that this does not answer the direct question that you asked. However, it does solve your problem. I hope this proves useful.
This should be easy, but I cannot seem to get it right as I seem to be confusing myself and converting to and from strings, integers and floats and whatnot.
Basically, I am populating a TListView with FileNames in one column, and in another column returning the File Size to the corresponding FileName. I am using a rather neat function found from here, which looks like this:
function FileSizeStr ( filename: string ): string;
const
// K = Int64(1000); // Comment out this line OR
K = Int64(1024); // Comment out this line
M = K * K;
G = K * M;
T = K * G;
var
size: Int64;
handle: integer;
begin
handle := FileOpen(filename, fmOpenRead);
if handle = -1 then
result := 'Unable to open file ' + filename
else try
size := FileSeek ( handle, Int64(0), 2 );
if size < K then result := Format ( '%d bytes', [size] )
else if size < M then result := Format ( '%f KB', [size / K] )
else if size < G then result := Format ( '%f MB', [size / M] )
else if size < T then result := Format ( '%f GB', [size / G] )
else result := Format ( '%f TB', [size / T] );
finally
FileClose ( handle );
end;
end;
This returns values such as: 235.40 KB
So with the above, my TListView may be populated like so:
Now in the Label Data Size, I would like to return the Total Size of the Files in the Listview, so in this example, the values from the Size column would need adding up to return the Total Size, something like:
1.28 MB + 313.90 KB + 541.62 KB + 270.96 KB
Obviously it cannot be added on just like that, because the values contain decimal points, some values may be in Kb, and other in Mb etc. This is my problem, I cannot think of an easy solution to add (get) the Total Size of the Files, and then return it in the same formatted string as shown.
I would really appreciate some insight or tips how to work with this kind of data, I am just endlessly confusing myself with different conversions etc and not really sure which way to do this.
Many Thanks in advance :)
UPDATE 1
Following the advice from Marc B, I changed the function to the following which seems to work:
var
iFileSize: Int64;
implementation
function GetSizeOfFile(FileName: string): Int64;
var
Handle: Integer;
begin
Handle := FileOpen(FileName, fmOpenRead);
if Handle = -1 then
MessageDlg('Unable to open file ' + FileName, mtError, [mbOk], 0)
else try
iFileSize := iFileSize + FileSeek(Handle, Int64(0), 2);
finally
FileClose(Handle);
end;
Result := iFileSize;
end;
function FormatFileSize(AValue: Int64): string;
const
K = Int64(1024);
M = K * K;
G = K * M;
T = K * G;
begin
if AValue < K then Result := Format ( '%d bytes', [AValue] )
else if AValue < M then Result := Format ( '%f KB', [AValue / K] )
else if AValue < G then Result := Format ( '%f MB', [AValue / M] )
else if AValue < T then Result := Format ( '%f GB', [AValue / G] )
else Result := Format ( '%f TB', [AValue / T] );
end;
It may be useful for anyone else should they need it :)
UPDATE 2
Additionally, see the answer Ken White posted which provides more valuable information, and a cleaner update of the GetSizeOfFile function, which works great:
Separate the "get file information" from the "format the size string" into two separate functions. The file information function fetches the file size and adds it to a running total, THEN calls the formatting function to convert the simple integer into the "nice" string.
The easiest way would be to change your function to return the file size, and use a separate function to format the results.
I know you've already accepted an answer, but the updated code you posted has a couple of problems (one was in the original version, too).
First, your method of getting the file size is extremely slow, especially if you're going to be using this to list a lot of files. You're actually opening the file, moving the file pointer to the end of the file to get the size, and then closing the file. Also, this may fail if the file is open by another application exclusively.
Second, your new version of GetSizeOfFile has a logic error. You're adding to the global cumulative value every time (which is what you want), but you're also returning that new global value, which you don't want according to the sample image you posted.
Here's a replacement for GetSizeOfFile that should work for you, along with sample use:
function GetSizeOfFile( const FileName: String ): Int64;
var
Rec : TSearchRec;
begin
Result := 0;
if (FindFirst(FileName, faAnyFile, Rec) = 0) then
begin
Result := Rec.Size;
FindClose(Rec);
end;
end;
Sample use:
var
FileSize: Int64;
FileSizeString: string;
begin
{ Whatever code }
FileSize := GetSizeOfFile(SomeFileName);
iFileSize := iFileSize + NewSize;
FileSizeString := FormatFileSize(NewSize);
{ Add your file to your ListView.}
end;