I'm redesigning some of my For-loops to run faster. Especially the ones that call functions to compare values with. I'm wondering if my basic Delphi knowledge is missing something really useful.
I have a lot of For-loops similar to this:
for i :=1 to 10000 do
begin
//--- Check if Condition for each Object (i) Property = Yes, Y or a Number ---
if (fGetPropValue(i,'Condition')='Yes') Or (fGetPropValue(i,'Condition')='Y') Or (StrToIntDef(fGetPropValue(i,'Condition'),0)>0)
then
// code...
end;
In this case function gets called 3 times, if result is different than Yes or Y. If it's Yes, it's only called once and skips the rest of calls.
I can use single call and store value to variable, like this:
for i :=1 to 10000 do
begin
//--- Get Condition for each Object (i) Property ---
vCondition:=fGetPropValue(i,'Condition');
//--- Check Condition = Yes, Y or a number ---
if (vCondition='Yes') Or (vCondition='Y') Or (StrtoIntDef(vCondition,0)>0)
then
// code...
end;
This is good, but what in case where I need to compare more complex, like:
for i :=1 to 10000 do
begin
if (fGetPropValue(i,'Condition')='Yes') Or (fGetPropValue(i,'Condition')='Y') Or (StrtoIntDef(fGetPropValue(i,'Condition'),0)>0) And
(fGetPropValue(i,'Enabled')='Yes') Or (fGetPropValue(i,'Enabled')='Y') Or (StrtoIntDef(fGetPropValue(i,'Enabled'),0)>0) And
(fGetPropValue(i,'Visible')='Yes') Or (fGetPropValue(i,'Visible')='Y') Or (StrtoIntDef(fGetPropValue(i,'Visible'),0)>0) And
(fGetPropValue(i,'AllowAccess')='Yes') Or (fGetPropValue(i,'AllowAccess')='Y') Or (StrtoIntDef(fGetPropValue(i,'AllowAccess'),0)>0)
then
// code...
end;
In this case if I introduce 4 variable and get all 4 values, I eliminate the speedy option if first logical comparison is true - without 4 variable it will not execute anymore calls.
Any way to redesign complex comparison?
You need to extract this into a function that you can re-use:
function PropValueIsTrue(const PropValue: string): Boolean;
begin
Result := (PropValue='Yes') or (PropValue='Y') or (StrtoIntDef(PropValue,0)>0);
end;
Then your code becomes:
if PropValueIsTrue(fGetPropValue(i,'Condition')) and
PropValueIsTrue(fGetPropValue(i,'Enabled')) and
PropValueIsTrue(fGetPropValue(i,'Visible')) and
PropValueIsTrue(fGetPropValue(i,'AllowAccess')) then
Related
In Delphi, I declared a 3x3 matrix table as an array of array of Single, like this:
m_Table: array [0..2] of array [0..2] of Single;
Now I want to memory compare the content with another table, or memory copy the table content from another table. I know that I can create a nested loop to do that, but I want to do the job without any loop, if possible.
My question is, it is correct to copy or compare the memory like this:
CompareMem(m_Table, other.m_Table, 9 * SizeOf(Single));
CopyMemory(m_Table, other.m_Table, 9 * SizeOf(Single));
If not, what is the correct way to do that?
And as a subsidiary question, is there a better way to get the length to copy instead of 9 * SizeOf(Single), like e.g. SizeOf(m_Table^)?
Regards
The code in the question works fine. Personally I would say that Move is the idiomatic way to copy memory. Further I would use SizeOf(m_Table) to obtain the size of the type.
I would point out that your comparison differs from the floating point equality operator. Perhaps that's what you want, but you should be aware of this. For instance zero and minus zero compare equal using floating point comparison but not with memory compare. And NaNs always compare not equal, even with identical bit patterns.
Let me also comment that it would make your code more extendible if you declared a type for these matrices. Without that you won't be able to write functions that accept such objects.
The correct and easiest way may be to define a type:
type
TMatrix3x3 = array [0..2,0..2] of Single;
Then you can directly write:
var
v1, v2: TMatrix3x3;
begin
fillchar(v1,sizeof(v1),0);
move(v1,v2,sizeof(v1));
if comparemem(#v1,#v2,sizeof(v1)) then
writeln('equals');
end;
Using sizeof() make your code safe and readable.
You may define a wrapper type with methods:
{ TMatrix3x3 }
type
TMatrix3x3 = record
v: array [0..2,0..2] of Single;
procedure Zero;
procedure Copy(var dest: TMatrix3x3);
procedure Fill(const source: TMatrix3x3);
function Equals(const other: TMatrix3x3): boolean;
end;
procedure TMatrix3x3.Copy(var dest: TMatrix3x3);
begin
move(v,dest,sizeof(v));
end;
function TMatrix3x3.Equals(const other: TMatrix3x3): boolean;
begin
result := CompareMem(#v,#other.v,sizeof(v));
end;
procedure TMatrix3x3.Fill(const source: TMatrix3x3);
begin
move(source,v,sizeof(v));
end;
procedure TMatrix3x3.Zero;
begin
fillchar(v,sizeof(v),0);
end;
Including then advanced features like implicit assignment, and operators, if needed.
But don't reinvent the wheel, if you really to work with matrix arithmetic. Use an already existing and fully tested library, which will save you a lot of trouble and debugging time.
You should use standard TMatrix type from System.Math.Vectors unit, then you can just compare it directly as if Matrix1 = Matrix2 then and assign as Matrix1 := Matrix2
When I call Functions to get a value, I usually initialize varible, in case function fails or doesn't return anything and I want to avoid dealing with uninitialized variable. I do the same for string, integer or any other type.
Example for integer variables:
vPropValue := 0;
vPropValue := GetPropValue(vObject,'Height');
IF vPropValue > 0 Then
...
this it the most common how I use it.
I know I could use:
If GetPropValue(vObject,'Height') > 0 Then
...
but with first example I avoid multiple calls to function, if I need result again later in the code.
Same for string (even though i know local strings are initialized to empty string, while integers are not an can hold any value)
vName := '';
vName := GetObjectName(vObject,'ObjectName');
IF Trim(vPropStrValue) <> '' Then
...
I know I could take steps to avoid duplicate value assignment,like making sure Function returns 0 if everything fails. But I have 100s of functions and I can't rely I never made a mistake how functions handle everything and I'm sure some don't return 0, if everything fails.
I'm trying to understand why this is not desirable practice and how to best avoid it.
EDIT
Here is example where function doesn't return proper value or 0:
function GetValue(vType:integer):integer;
begin
if vType=1 then
Result:=100
else if (vType>2) and (vType<=9) then
Result:=200;
end;
procedure TForm1.Button1Click(Sender: TObject);
var vValue:integer;
begin
vValue:=GetValue(11);
Button1.Caption:=IntToStr(vValue);
end;
In this case the value returned from function is some random number.
In this case the initialization appears to be valid approach. OR NOT?
EDIT 2:
As David pointed out in his answer, correct, there was a warning
[dcc32 Warning] Unit1.pas(33): W1035 Return value of function 'GetValue' might be undefined
but, I ignored it, for no reason, just didn't look there. As it let me compile it, I thought it was OK. So, I did look for the warning and I 'fixed' quite a few functions that had similar issue, because of all IFs Result might not have been defined.
EDIT 3 & CONCLUSION:
I hope it adds to the scope of question and explanation:
Maybe an example of another twist that I use in most of my Functions, would also explain why I thought my initialization of variable was needed, is that I wasn't sure my Functions would behave correctly all the times, especially in case of nested function. Mots of them still are set like this:
function GetProperty(vType:integer):integer;
begin
Try
if vType = 99 then
Result:=GetDifferentProperty(vType)// <-- Call to another Function, that could return whatever...
else
begin
if vType=1 then
Result:=100
else if (vType>2) and (vType<=9) then
Result:=200;
end;
except
end;
end;
Now I'm addressing these Try Except End; but some functions are 10 years old and to expect them to work 100%, based on my experience back then, is not something to rely on.
As the only developer on this project, I assume I should trust my functions (and he rest of my code), but I can't imagine in multiple developers environment that all function are set up properly.
So my conclusion: since I didn't take care of the basics - properly designed Functions, I need to have all these checks (variable initialization, Try Except lines..) and probably some other unneccessary stuff.
Assuming that vPropValue is a local variable then this code
vPropValue := 0;
vPropValue := GetPropValue(vObject,'Height');
is indistinguishable from
vPropValue := GetPropValue(vObject,'Height');
A simpler example might be like so:
i := 0;
i := 1;
What is the point of assigning 0 to i and then immediately assigning 1 to i? So, you would surely never write that. You would write:
i := 1;
In your code, at the top of this answer, you assign twice to the same variable. The value assigned in the first assignment is immediately replaced by the value assigned in the second assignment. Therefore, the first assignment is pointless and should be removed.
The second example is a little more complex. Assuming that your functions are written correctly, and always assign to their return value, and that vName is a local variable, then
vName := '';
vName := GetObjectName(vObject,'ObjectName');
is indistinguishable from
vName := GetObjectName(vObject,'ObjectName');
The reason why I have added an extra proviso relates to a quirk of the implementation of function return values, discussed below. The difference between this case and the case above is the return value type. Here it is a managed type, string, whereas in the first example the type is a simple Integer.
Again, given the proviso about the function always assigning to the return value, the first assignment is pointless because the value is immediately replaced. Remove that first assignment.
Regarding the function in your edit, the compiler will warn you of its erroneous implementation if you enable hints and warnings. The compiler will tell you that not all code paths return a value.
function GetValue(vType:integer):integer;
begin
if vType=1 then
Result:=100
else if (vType>2) and (vType<=9) then
Result:=200;
end;
If neither condition is met, then no value is assigned to the result variable. This function should be:
function GetValue(vType:integer):integer;
begin
if vType=1 then
Result:=100
else if (vType>2) and (vType<=9) then
Result:=200
else
Result:=0;
end;
I cannot stress how important it is that you always return a value from a function. In fact it is a terrible weakness that Delphi even allows your function to be compiled.
The reason that your double assignment sometimes appears useful to you is due to a quirk of of the implementation of function return values in Delphi. Unlike almost all other languages a Delphi function return value for certain more complex types is actually a var parameter. So this function
function foo: string;
is actually, semantically, the same as this:
procedure foo(var result: string);
This is a really odd decision made by the Delphi designers. In most other languages, like C, C++, C#, Java etc., a function return value is like a by-value parameter passed from callee to caller.
This means that you can, if you wish to be perverse, pass values into a function via its return value. For instance, consider this code:
// Note: this code is an example of very bad practice, do not write code like this
function foo: string;
begin
Writeln(Result);
end;
procedure main;
var
s: string;
begin
s := 'bar';
s := foo;
end;
When you call main, it will output bar. This is a rather strange implementation detail. You should not rely on it. Let me repeat myself. You should not rely on it. Do not get in the habit of initializing return values at the call site. That leads to unmaintainable code.
Instead follow the simple rule of ensuring that the function return value is always assigned by the function, and never read before it has been assigned.
More detail on the implementation of function return values is provided by the documentation, with my emphasis:
The following conventions are used for returning function result
values.
Ordinal results are returned, when possible, in a CPU register. Bytes are returned in AL, words are returned in AX, and double-words
are returned in EAX.
Real results are returned in the floating-point coprocessor's top-of-stack register (ST(0)). For function results of type Currency,
the value in ST(0) is scaled by 10000. For example, the Currency value
1.234 is returned in ST(0) as 12340.
For a string, dynamic array, method pointer, or variant result, the effects are the same as if the function result were declared as an
additional var parameter following the declared parameters. In other
words, the caller passes an additional 32-bit pointer that points to a
variable in which to return the function result.
Int64 is returned in EDX:EAX.
Pointer, class, class-reference, and procedure-pointer results are returned in EAX.
For static-array, record, and set results, if the value occupies one byte it is returned in AL; if the value occupies two bytes it is
returned in AX; and if the value occupies four bytes it is returned in
EAX. Otherwise, the result is returned in an additional var parameter
that is passed to the function after the declared parameters.
The following code (A)
vPropValue := 0;
vPropValue := GetPropValue(vObject,'Height');
is indistinguishable from (B)
vPropValue := GetPropValue(vObject,'Height');
The question of whether GetPropValue is "written correctly" is totally irrelevant.
Let's consider what happens even if you did write GetPropValue incorrectly E.g.
function GetPropValue(AObject: TObject; AStr: String): Integer;
begin
if AStr = 'Hello' then Result := 5;
end;
As you know when the input AStr is anything other than "Hello" the result of the function will be pretty much random. (For the sake of the discussion, lets assume it will return -42.)
Code block (A) will do the following:
Set vPropValue to 0
Then set vPropValue to - 42
Code block (B) will simply set vPropValue to - 42 immediately.
TIP: There is no point in writing a wasteful line of code just because you're worried you might have made a mistake in a function you call.
First, As David points out, you can avoid many mistakes simply by paying attention to your compiler hints and warnings.
Second, that sort of "paranoid" coding simply leads to more wasteful code because now you have to start considering invalid values as possible results.
This becomes worse when one day your "safe-value" is actually a valid value. E.g. how would you tell the difference between "default 0" and "correctly returned 0"?
Don't make programming artificially difficult by bloating code with unnecessary redundancies.
Side Note
There are a couple of special situations where the code can behave differently. But you should in any case avoid the designs that lead to those situations because they make it much more difficult to maintain the code.
I mention them purely for the sake of completeness, the advice above still stands.
1) if vPropValue is implemented as a property, the setter could have side-effects that cause different behaviour. While there's nothing wrong with properties, when they do unexpected things you have a serious problem on your hands.
2) if vPropValue is a field on the class (or worse a global variable), then (A) and (B) could behave differently but only if GetPropValue raises an exception. This is because the exception would prevent the result from being assigned. Note this is something to avoid except in special cases because it does make it more difficult to reason about what your code is doing.
And in fact, this is something that makes it so much more important to avoid the redundant initialisations. You want your special case code to look stand out from the rest.
Scavenging my advices from top-level comments in case Pastebin fails
function GetValueUpdate3(vType:integer):integer;
begin
// with SOME types like Integer you can use Pascal case-block instead of error-prone many-ifs-ladder
case vType of
99: Result:=GetDifferentProperty(vType);// <-- Call to another Function, that could return whatever...
1: Result:=100;
3..9: Result:=200;
else Result := 12345; // initialization with safe default value for illegal input like vType=2
end; // case-block
end;
function GetValueUpdate3e(vType:integer):integer;
begin
case vType of
99: Result:=GetDifferentProperty(vType);// <-- Call to another Function, that could return whatever...
1: Result:=100;
3..9: Result:=200;
else raise EInvalidArgument.Create('prohibited value - GetValue( ' + IntToStr(vType) +' )' );
// runtime eror when vType = 2 or any other illegal input
end;
end;
function GetValueUpdate1(vType:integer):integer;
begin
Result := 12345; // initialization with safe default value;
if vType=1 then Exit(100); // special value for special case #1
if (vType>2) and (vType<=9) then Exit(200); // special value for special case #2
// exit with default value
end;
procedure TForm1.Button1Click(Sender: TObject);
var vValue:integer;
begin
vValue:=GetValue(11);
Button1.Caption:=IntToStr(vValue);
end;
// http://stackoverflow.com/questions/33927750
You can use Assertions to validate if your input is correct.
Assertions help a lot in finding logical errors in your code.
Using assertions forces you to think about valid and invalid input data and helps writing better code.
Enabling and disabling assertions at compile time is done via the \$C or \$ASSERTIONS compiler (global switch)
In your example the function GetValue could use an assertion as follows:
function GetValue(vType:integer):integer;
begin
Assert((vType >= 1) and (vType <=9), 'Invalid input value in function GetValue');
if vType=1 then
Result:=100
else if (vType>2) and (vType<=9) then
Result:=200
else Result := 0; // always catch the last else!
end;
In addition every if-statement should catch the final else! (In my opinion ALWAYS!)
I have to check if I have duplicate paths in a FileListBox (FileListBox has the role of some kind of job list or play list).
Using Delphi's SameText, CompareStr, CompareText, takes 6 seconds. So I came with my own compare function which is (just) a bit faster but not fast enough. Any ideas how to improve it?
function SameFile(CONST Path1, Path2: string): Boolean;
VAR i: Integer;
begin
Result:= Length(Path1)= Length(Path2); { if they have different lenghts then obviously are not the same file }
if Result then
for i:= Length(Path1) downto 1 DO { start from the end because it is more likely to find the difference there }
if Path1[i]<> Path2[i] then
begin
Result:= FALSE;
Break;
end;
end;
I use it like this:
for x:= JList.Count-1 downto 1 DO
begin
sMaster:= JList.Items[x];
for y:= x-1 downto 0 DO
if SameFile(sMaster, JList.Items[y]) then
begin
JList.Items.Delete (x); { REMOVE DUPLICATES }
Break;
end;
end;
Note: The chance of having duplicates is small so Delete is not called often. Also the list cannot be sorted because the items are added by user and sometimes the order may be important.
Update:
The thing is that I lose the asvantage of my code because it is Pascal.
It would be nice if the comparison loop ( Path1[i]<> Path2[i] ) would be optimized to use Borland's ASM code.
Delphi 7, Win XP 32 bit, Tests were done with 577 items in the list. Deleting the items from list IS NOT A PROBLEM because it happens rarely.
CONCLUSION
As Svein Bringsli pointed, my code is slow not because of the comparing algorithm but because of TListBox. The BEST solution was provided by Marcelo Cantos. Thanks a lot Marcelo.
I accepted Svein's answer because it answers directly my question "how to make my comparison function faster" with "there is no point to make it faster".
For the moment I implemented the dirty and quick to implement solution: when I have under 200 files, I use my slow code to check the duplicates. If there are more than 200 files I use dwrbudr's solution (which is damn fast) considering that if the user has so many files, the order is irrelevant anyway (human brain cannot track so many items).
I want to thank you all for ideas and especially Svein for revealing the truth: (Borland's) visual controls are damn slow!
Don't waste time optimising the assembler. You can go from O(n2) to O(n log(n)) — bringing the time down to milliseconds — by sorting the list and then doing a linear scan for duplicates.
While you're at it, forget the SameFile function. The algorithmic improvement will dwarf anything you can achieve there.
Edit: Based on feedback in the comments...
You can perform an order-preserving O(n log(n)) de-duplication as follows:
Sort a copy of the list.
Identify and copy duplicated entries to a third list along with their duplication count minus one.
Walk the original list backwards as per your original version.
In the inner (for y := ...) loop, traverse the duplication list instead. If an outer item matches, delete it, decrement the duplication count, and delete the duplication entry if the count reaches zero.
This is obviously more complicated but it will still be orders of magnitude faster, even if you do horrible dirty things like storing duplication counts as strings, C:\path1\file1=2, and using code like:
y := dupes.IndexOfName(sMaster);
if y <> -1 then
begin
JList.Items.Delete(x);
c := StrToInt(dupes.ValueFromIndex(y));
if c > 1 then
dupes.Values[sMaster] = IntToStr(c - 1);
else
dupes.Delete(y);
end;
Side note: A binary chop would be more efficient than the for y := ... loop, but given that duplicates are rare, the difference ought to be negligible.
Using your code as a starting point, I modified it to take a copy of the list before searching for duplicates. The time went from 5,5 seconds to about 0,5 seconds.
vSL := TStringList.Create;
try
vSL.Assign(jList.Items);
vSL.Sorted := true;
for x:= vSL.Count-1 downto 1 DO
begin
sMaster:= vSL[x];
for y:= x-1 downto 0 DO
if SameFile(sMaster, vSL[y]) then
begin
vSL.Delete (x); { REMOVE DUPLICATES }
jList.Items.Delete (x);
Break;
end;
end;
finally
vSL.Free;
end;
Obviously, this is not a good way to do it, but it demonstrates that TFileListBox is in itself quite slow. I don't believe you can gain much by optimizing your compare-function.
To demonstrate this, I replaced your SameFile function with the following, but kept the rest of your code:
function SameFile(CONST Path1, Path2: string): Boolean;
VAR i: Integer;
begin
Result := false; //Pretty darn fast code!!!
end;
The time went from 5,6 seconds to 5,5 seconds. I don't think there's much more to gain there :-)
Create another sorted list with sortedList.Duplicates := dupIgnore and add your strings to that list, then back.
vSL := TStringList.Create;
try
vSL.Sorted := true;
vSL.Duplicates := dupIgnore;
for x:= 0 to jList.Count - 1 do
vSL.Add(jList[x]);
jList.Clear;
for x:= 0 to vSL.Count - 1 do
jList.Add(vSL[x]);
finally
vSL.Free;
end;
The absolute fastest way, bar none (as alluded to before) is to use a routine that generates a unique 64/128/256 bit hash code for a string (I use the SHA256Managed class in C#). Run down the list of strings, generate the hash code for the strings, check for it in the sorted hash code list, and if found then the string is a duplicate. Otherwise add the hash code to the sorted hash code list.
This will work for strings, file names, images (you can get the unique hash code for an image), etc, and I guarantee that this will be as fast or faster than any other impementation.
PS You can use a string list for the hash codes by representing the hash codes as strings. I've used a hex representation in the past (256 bits -> 64 characters) but in theory you can do it any way you like.
4 seconds for how many calls? Great performance if you call it a billion times...
Anyway, does Length(Path1) get evaluated every time through the loop? If so, store that in an Integer variable prior to looping.
Pointers may yield some speed over the strings.
Try in-lining the function with:
function SameFile(blah blah): Boolean; Inline;
That will save some time, if this is being called thousands of times per second. I would start with that and see if it saves anything.
EDIT: I didn't realize that your list wasn't sorted. Obviously, you should do that first! Then you don't have to compare against every other item in the list - just the prior or next one.
I use a modified Ternary Search Tree (TST) to dedupe lists. You simply load the items into the tree, using the whole string as the key, and on each item you can get back an indication if the key is already there (and delete your visible entry). Then you throw away the tree. Our TST load function can typically load 100000 80-byte items in well under a second. And it could not take any more than this to repaint your list, with proper use of begin- and end-update. The TST is memory-hungry, but not so that you would notice it at all if you only have of the order of 500 items. And much simpler than sorting, comparisons and assembler (if you have a suitable TST implementation, of course).
No need to use a hash table, a single sorted list gives me a result of 10 milliseconds, that's 0.01 seconds, which is about 500 times faster! Here is my test code using a TListBox:
procedure TForm1.Button1Click(Sender: TObject);
var
lIndex1: Integer;
lString: string;
lIndex2: Integer;
lStrings: TStringList;
lCount: Integer;
lItems: TStrings;
begin
ListBox1.Clear;
for lIndex1 := 1 to 577 do begin
lString := '';
for lIndex2 := 1 to 100 do
if (lIndex2 mod 6) = 0 then
lString := lString + Chr(Ord('a') + Random(2))
else
lString := lString + 'a';
ListBox1.Items.Add(lString);
end;
CsiGlobals.AddLogMsg('Start', 'Test', llBrief);
lStrings := TStringList.Create;
try
lStrings.Sorted := True;
lCount := 0;
lItems := ListBox1.Items;
with lItems do begin
BeginUpdate;
try
for lIndex1 := Count - 1 downto 0 do begin
lStrings.Add(Strings[lIndex1]);
if lStrings.Count = lCount then
Delete(lIndex1)
else
Inc(lCount);
end;
finally
EndUpdate;
end;
end;
finally
lStrings.Free;
end;
CsiGlobals.AddLogMsg('Stop', 'Test', llBrief);
end;
I'd also like to point out that your solution would take an extreme amount of time if applied to a huge list (like containing 100,000,000 items or more). Even constructing a hashtable or sorted list would take too much time.
In cases like that you could try another approach : Hash each member, but instead of populating a full-blown hashtable, create a bitset (large enough to contain a close factor to as many slots as there are input items) and just set each bit at the offset indicated by the hashfunction. If the bit was 0, change it to 1. If it was already 1, take note of the offending string-index in a separate list and continue. This results in a list of string-indexes that had a collision in the hash, so you'll have to run it a second time to find the first cause of those collisions. After that, you should sort & de-dupe the string-indexes in this list (as all indexes apart from the first one will be present twice). Once that's done you should sort the list again, but this time sort it on the string-contents in order to easily spot duplicates in a following single scan.
Granted it could be a bit extreme to go this all this length, but at least it's a workable solution for very large volumes! (Oh, and this still won't work if the number of duplicates is very high, when the hash-function has a bad spread or when the number of slots in the 'hashtable' bitset is chosen too small - which would give many collisions which aren't really duplicates.)
Whilst I'd love to solve this problem in python, I'm stuck in Delphi for this one. I have nested lists (actually objects with nested lists as properties, but nevermind), and I want to iterate over them in a generator fashion. That is, I want to write a Next function, which gives me the next item from the leaves of the tree described by the nested lists.
For example, lets say I have
[[1,2,3],[4,5],[],[6],[7,8]]
I want 8 consecutive calls to Next() to return 1..8.
How can I do this in a language without yield and generators?
Note that the depth of the nesting is fixed (2 in this example, 4 in real life), but answers which solve the more general case where depth is variable are welcome.
EDIT: Sorry, I should have mentioned, this is Delphi 2007.
If you're using any Delphi list that has a built-in enumerator, it can be done easily enough with a bit of recursion. Your base list is a list of numbers, like a TList<integer>. Then you have nested lists implemented as TList<TList<integer>>. (I'm assuming you have Delphi 2009 or 2010. If not, it gets a bit trickier.)
What you need is to make your own specialized list class descended from TList<T> and add a virtual Next() function to it, and a field for an enumerator for your list. The compiler uses enumerators internally when you set up a for..in loop, but you can run them manually. The Next() function creates the enumerator if it's not already assigned and puts it in the field, then calls MoveNext() on it. If this succeeds, call GetCurrent and get your number. Otherwise, you're done. FreeAndNil the enumerator, and signal to the calling function that you've got nothing to return. Probably the simplest way to do this is to put a var boolean parameter in Next() that returns the result of its call to MoveNext, and have the calling function check its value.
For the higher lists, it's a little more complicated, but not much. Descend from the generic class you just set up, and override Next(). This one will get an enumerator that enumerates over the lists that it holds, and return the value of FEnumerator.GetCurrent.Next() until that sub-list is exhausted, then call MoveNext on its enumerator.
This should work for any depth of nested lists. Just don't try to make a list that contains both numbers and lists.
Method 1 : a "layered list" is a tree, use (or write) a tree/treenode class, and do a simple depth-first search of its values.
Method 2 : explicitly write the depth-first search for your layered list type :
TEnumerator = RECORD
private
FStack : Stack of TNestedList; //a stack, holding which level of the tree you are exploring
FIndexStack : Stack of Integer; //a twin stack, holding the index of the child you are exploring at each layer
public
Init( AList : TNestedList );
Next( var AVal : Integer ) : boolean;
end;
procedure TEnumerator.Init( AList );
begin
SetLength(FStack, 1);
FStack[0] := AList;
SetLength( FIndexStack, 1 );
FIndexStack[0] := 0;
_GoToNextValue();
end;
procedure TEnumerator._GoToNextValue();
begin
while FStack.notEmpty()
and FIndexStack.last > FStack.last.length do
begin
//we have finished exploring FStack.last, we need to explore its next sibling :
//pop FStack.last :
FIndexStack.pop;
FStack.pop;
//go to next sibling :
FIndexStack.last := FIndexStack.last + 1;
end;
//nothing left to explore :
if FStack.empty then Exit;
//else :
// dig through the layers of list until you reach a value
while FStack.last.isAList() do
begin
FStack.push( FStack.last[ FIndexStack.last ] );
FIndexStack.push( 0 );
end;
end;
function Next( var AVal : Integer ) : boolean;
begin
_GoToNextValue();
Result := FStack.notEmpty();
if Result then
begin
AVal := FStack.last[ FIndexStack.last ];
FIndexSatck.last := FIndexSatck.last + 1;
end;
end;
You basically do explicitly what a "yield" would do (ie : save a "freeze copy" of the call stack, return the current value)
Usage :
LEnum : TEnumerator;
LEnum.Init( myList );
while LEnum.Next( LVal ) do
begin
<do something>
end;
To solve this problem, I ended up making a flat list of indexes and remembering my position in that: (in pythonesque, because its shorter)
class Nested:
nestedList = [[1,2,3],[4,5],[],[6],[7,8]]
positions = []
curr = 0
def Setup:
for a in nestedList:
for b in a:
positions.add((a,b))
def Next:
p = positions[curr]
curr += 1
return nestedList[p[0]][p[1]]
Obviously my list doesn't change during iteration or this probably wouldn't work...
I want to know how to increase the value in a FOR-loop statement.
This is my code.
function Check(var MemoryData:Array of byte;MemorySignature:Array of byte;Position:integer):boolean;
var i:byte;
begin
for i := 0 to Length(MemorySignature) - 1 do
begin
while(MemorySignature[i] = $FF) do inc(i); //<< ERROR <<
if(memorydata[i + position] <> MemorySignature[i]) then Result:=false;
end;
Result := True;
end;
The error is: E2081 Assignment to FOR-Loop variable 'i'.
I'm trying to translate an old code from C# to Delphi,but I can't increase 'i'.
Increasing 'i' is not the only way to go,but I want to know where the problem is.
Of course the others are (generally) correct. What wasn't said, is that 'i' in your loop doesn't exist. Delphi uses a CPU register for it. That's why you cannot change it and that's why you should use a 'for' loop (not a 'while') because the 'for' is way faster. Here is your code modified (not tested but I think that you got the idea) - also imho you had some bugs - fixed them also:
function Check(var MemoryData:Array of byte;MemorySignature:Array of byte;Position:integer):boolean;
var i:byte;
begin
Result := True; //moved at top. Your function always returned 'True'. This is what you wanted?
for i := 0 to Length(MemorySignature) - 1 do //are you sure??? Perhaps you want High(MemorySignature) here...
begin
if MemorySignature[i] <> $FF then //speedup - '<>' evaluates faster than '='
begin
Result:=memorydata[i + position] <> MemorySignature[i]; //speedup.
if not Result then
Break; //added this! - speedup. We already know the result. So, no need to scan till end.
end;
end;
end;
...also MemorySignature should have a 'const' or 'var'. Otherwise as it is now the array gets copied. Which means slowdown at each call of 'Check'. Having a 'var' the things are much faster with code unchanged because AFAIS the MemorySignature isn't changed.
HTH
in this case, you can just do a 'continue' instead of inc(i)
In addition to what Lasse wrote, assigning to a loop variable is generally considered a code smell. It makes code harder to read (if you want to leave the loop premataturely, you can express that a lot clearer using break/continue), and is often done by accident, causing all kind of nasty side-effects. So instead of jumping through hoops to make the compiler not do its optimizing fu on any loop where the loop variable is touched, Borland (now CodeGear) bit the bullet and made assigning to the loop variable illegal.
If you really want to mess about manually with loop indices, consider using a while-loop.
If you need to alter a loop counter inside a loop, try using a while loop instead.
BTW, you need your
Result := True
line to be the first line of the function for it to work properly. As it is, it will always return True.
The problem is that the compiler has taken the original FOR-loop code and assumed it knows what is happening, and thus it can optimize the code by outputting specific CPU instructions that runs the fastest, with those assumptions.
If it allowed you to mess with the variable value, those assumptions might go out the window, and thus the code might not work, and that's why it doesn't allow you to change it.
What you should do instead is just have a separate variable that you're actually using, and only use the FOR-loop indexing variable to keep track of how many iterations you've currently executed.
As an example, a typical optimization might be to write CPU-instructions that will stop iterating when the index register drops to zero, rewriting the loop in such a way that it internally counts down, instead of up, and if you start messing with the variable, it could not rewrite the code like that.
As per Mike Sutton, what you need is a while loop, not a for loop.
function Check(var MemoryData: Array of byte;
MemorySignature: Array of byte; Position: Integer):Boolean;
var
i:byte;
begin
Result := True;
i := 0;
while i < Length(MemorySignature) do
begin
while(MemorySignature[i] = $FF) do
Inc(i);
if(MemoryData[i + position] <> MemorySignature[i]) then
Result := False;
Inc(i);
end;
end;
The Delphi implementation of "for" is optimised, but as a result it is less flexible than the C-style