First off, I want to thank Rustan and the whole community for the work you put on Dafny. It is an amazing language!
I am working on my Master's thesis which is about formal verification of virtual machines using Dafny.
This is how I define (a stripped version of) a virtual machine:
class VM
{
var v: array<bv8>;
var I: bv16;
var memory: array<bv8>;
predicate Valid()
reads this
{
v.Length == 16
&& memory.Length == 0x0FFF
}
constructor Init()
ensures Valid()
{
v := new bv8[16];
I := 0;
memory := new bv8[0x0FFF];
}
}
So far so good. I have a few methods that mutate the state of this machine. In particular, here's one:
method parse_opcode_registers(vm: VM, opcode: bv16)
requires vm.Valid()
modifies vm`I, vm.v
{
var i: int := 0;
var bound := ((opcode & 0x0F00) >> 8) as int;
if opcode & 0xF0FF == 0xF065
{
while i < bound && vm.I as int < vm.memory.Length
decreases bound - i
{
vm.v[i] := vm.memory[vm.I];
i := i + 1;
vm.I := vm.I + 1;
}
}
}
This passes Dafny's verification. However, the issue occurs when there exists a caller for this method. Namely, the following code will produce an error call may violate context's modifies clause:
method Main() {
var vm := new VM.Init();
parse_opcode_registers(vm, 0xF018);
}
Any hints would be appreciated.
You need to add ensures fresh(v) to the Init constructor of VM.
Basically, the problem is that Dafny is worried because parse_opcode_register claims to modify vm.v, but Dafny isn't sure where vm.v came from. Remember that Dafny analyzes the program one method at a time, so it does not look inside the constructor Init while analyzing Main. Instead, Dafny only looks at the pre/postcondition. That's why adding fresh(v) to the postcondition fixes it.
The meaning of fresh(blah) is that blah was freshly allocated during the execution of the method.
For more, see the FAQ question about modifies clauses.
Related
This is my code:
int foo(int x) {
return x + 1; // I have more complex code here
}
int main() {
int s = 0;
for (int i = 0; i < 1000000; ++i) {
s += foo(42);
}
}
Without -O3 this code works for a few minutes. With -O3 it returns the same result in no time. Clang++, I believe, caches the value of foo(42) (it's a pure function) and doesn't call it a million times. How can I instruct it NOT to apply this particular optimization for this particular function call?
Out of curiosity, can you share why you would want to disable that optimization?
Anyway, about your question:
In your example code, s is never read after the loop, so the compiler would throw the whole loop away. So let's assume that s is used after the loop.
I'm not aware of any pragmas or compiler options to disable a particular optimization in a particular section of code.
Is changing the code an option?
To prevent that optimization in a portable manner, you can look for a creative way to compute the function call argument in a way such that the compiler is no longer able to treat the argument as constant. Of course the challenge here is to actually use a trick that does not rely on undefined behavior and that cannot be "outsmarted" by a newer compiler version.
See the commented example below.
pro: you use a trick that uses only the language that you can apply selectively
con: you get an additional memory access in every loop iteration; however, the access will be satisfied by your CPU cache most of the time
I verified the generated assembly for your particular example with clang++ -O3 -S. The compiler now generates your loop and no longer caches the result. However, the function gets inlined. If you want to prevent that as well, you can declare foo with __attribute__((noinline)), for example.
int foo(int x) {
return x + 1; // I have more complex code here
}
volatile int dummy = 0; // initialized to 0 and never changed
int main() {
int s = 0;
for (int i = 0; i < 1000000; ++i) {
// Because of the volatile variable, the compiler is forced to assume
// that the function call argument is different for each loop
// iteration and it is no longer able to use a cached result.
s += foo(42 + dummy);
}
}
The following code verifies:
function stepMany(i : int, steps : int) : int
requires steps >= 0;
decreases steps;
{
if steps > 0 then
stepMany(i + 1, steps - 1)
else
i
}
lemma ex2()
{
ghost var ex1 := 10;
ghost var ex1Done := stepMany(ex1, 33);
assert ex1Done == 43; // assertion verifies successfully
}
However, the following code does not verify:
function stepMany(i : int, steps : int) : int
requires steps >= 0;
decreases steps;
{
if steps > 0 then
stepMany(i + 1, steps - 1)
else
i
}
lemma ex2()
{
ghost var ex1 := 10;
ghost var ex1Done := stepMany(ex1, 34); <-- only differences here
assert ex1Done == 44; <-- and here
// assertion DOES NOT verify
}
The only difference is the second parameter of stepMany. The assertion works for all arguments up to 33 and fails for all arguments exceeding 34.
Is there a maximum number of recursive calls Dafny can handle? I have tried searching the documentation and found nothing. There appears to be a command-line argument "/recursionBound", but it does not influence the results.
The limit isn't so much on levels of recursion, but on levels of unfolding. When Dafny has a recursive definition it will unfold it a bit. But not without limit.
What you want is for Dafny to realize that stepMany is just an add function. One way to do that is to write a lemma. Then you need to remind Dafny of what you told it by invoking the lemma. Like this:
// Hey Dafny, the stepMany functions adds its arguments and returns the result.
lemma stepManyAddsItsArguments(i : int, steps : int)
requires steps >= 0
ensures stepMany(i, steps) == i+steps
decreases steps
{} // Hey Dafny, would you mind proving that?
lemma ex2()
{
ghost var ex1 := 10;
ghost var ex1Done := stepMany(ex1, 34);
stepManyAddsItsArguments( ex1, 34) ; // Heh Dafny, remember what I told you about stepMany!
assert ex1Done == 44;
}
I have a C DLL with a number of functions I'm calling from Delphi. One of the functions (say Func1) returns a pointer to a struct - this all works fine. The structs created by calling Func1 are stored in a global pool within the DLL. Using a second function (Func2) I get a pointer to a block of memory containing an array of pointers, and I can access the array elements using an offset.
I need to be able copy the address in the returned pointer for a struct (from Func1) to any of the memory locations in the array (from Func2). The idea is that I can build arrays of pointers to pre-defined structs and access the elements directly from Delphi using pointer offsets.
I tried using:
CopyMemory(Pointer(NativeUInt(DataPointer) + offset), PStruct, DataSize);
where DataPointer is the start of my array and PStruct is returned from Func1, but that doesn't copy the address I need.
In .NET it works using Marshal.WriteIntPtr and looking at the underlying code for this using Reflector I think I need something trickier than CopyMemory. Anyone got any ideas for doing this in Delphi?
Edit: This is part of a wrapper around vector structures returned from the R language DLL. I have a base vector class from which I derive specific vector types. I've got the wrapper for the numeric vector working, so my base class looks fine and this is where I get DataPointer:
function TRVector<T>.GetDataPointer: PSEXPREC;
var
offset: integer;
h: PSEXPREC;
begin
// TVECTOR_SEXPREC is the vector header, with the actual data behind it.
offset := SizeOf(TVECTOR_SEXPREC);
h := Handle;
result := PSEXPREC(NativeUInt(h) + offset);
end;
Setting a value in a numeric vector is easy (ignoring error handling):
procedure TNumericVector.SetValue(ix: integer; value: double);
var
PData: PDouble;
offset: integer;
begin
offset := GetOffset(ix); // -- Offset from DataPointer
PData := PDouble(NativeUInt(DataPointer) + offset);
PData^ := value;
end;
For a string vector I need to (i) create a base vector of pointers with a pre-specified length as for the numeric vector (ii) convert each string in my input array to an R internal character string (CHARSXP) using the R mkChar function (iii) assign the address of the character string struct to the appropriate element in the base vector. The string array gets passed into the constructor of my vector class (TCharacterVector) and I then call SetValue (see below) for each string in the array.
I should have thought of PPointer as suggested by Remy but neither that or the array approach seem to work either. Below is the code using the array approach from Remy and with some pointer vars for checking addresses. I'm just using old-fashioned pointer arithmetic and have shown addresses displayed for a run when debugging:
procedure TCharacterVector.SetValue(ix: integer; value: string);
var
PData: PSEXPREC;
offset: integer;
offset2: integer;
PTest: PSEXPREC;
PPtr: Pointer;
PPtr2: Pointer;
begin
offset := GetOffset(ix);
PPtr := PPointer(NativeUInt(DataPointer) + offset); // $89483D8
PData := mkChar(value); // $8850258
// -- Use the following code to check that mkChar is working.
offset2 := SizeOf(TVECTOR_SEXPREC);
PTest := PSEXPREC(NativeUInt(PData) + offset);
FTestString := FTestString + AnsiString(PAnsiChar(PTest));
//PPointerList(DataPointer)^[ix] := PData;
//PPtr2 := PPointer(NativeUInt(DataPointer) + offset); // Wrong!
PPointerArray(DataPointer)^[ix] := PData;
PPtr2 := PPointerArray(DataPointer)^[ix]; // $8850258 - correct
end;
I'd have thought the address in PData ($8850258) would now be in PPtr2 but I've been staring at this so long I'm sure I'm missing something obvious.
Edit2: The code for SetValue used in R.NET is as follows (ignoring test for null string):
private void SetValue(int index, string value)
{
int offset = GetOffset(index);
IntPtr stringPointer = mkChar(value);
Marshal.WriteIntPtr(DataPointer, offset, stringPointer);
}
From reflector, Marshal.WriteIntPtr uses the following C:
public static unsafe void WriteInt32(IntPtr ptr, int ofs, int val)
{
try
{
byte* numPtr = (byte*) (((void*) ptr) + ofs);
if ((((int) numPtr) & 3) == 0)
{
*((int*) numPtr) = val;
}
else
{
byte* numPtr2 = (byte*) &val;
numPtr[0] = numPtr2[0];
numPtr[1] = numPtr2[1];
numPtr[2] = numPtr2[2];
numPtr[3] = numPtr2[3];
}
}
catch (NullReferenceException)
{
throw new AccessViolationException();
}
}
You say you want to copy the struct pointer itself into the array, but the code you have shown is trying to copy the struct data that the pointer is pointing at. If you really want to copy just the pointer itself, don't use CopyMemory() at all. Just assign the pointer as-is:
const
MaxPointerList = 255; // whatever max array count that Func2() allocates
type
TPointerList = array[0..MaxPointerList-1] of Pointer;
PPointerList = ^TPointerList;
PPointerList(DataPointer)^[index] := PStruct;
Your use of NativeUInt reveals that you are using a version of Delphi that likely supports the {$POINTERMATH} directive, so you can take advantage of that instead, eg:
{$POINTERMATH ON}
PPointer(DataPointer)[index] := PStruct;
Or, use the pre-existing PPointerArray type in the System unit:
{$POINTERMATH ON}
PPointerArray(DataPointer)[index] := PStruct;
I'm trying to write what I would think of as an extremely simple piece of code in Rascal: Testing if list A contains list B.
Starting out with some very basic code to create a list of strings
public list[str] makeStringList(int Start, int End)
{
return [ "some string with number <i>" | i <- [Start..End]];
}
public list[str] toTest = makeStringList(0, 200000);
My first try was 'inspired' by the sorting example in the tutor:
public void findClone(list[str] In, str S1, str S2, str S3, str S4, str S5, str S6)
{
switch(In)
{
case [*str head, str i1, str i2, str i3, str i4, str i5, str i6, *str tail]:
{
if(S1 == i1 && S2 == i2 && S3 == i3 && S4 == i4 && S5 == i5 && S6 == i6)
{
println("found duplicate\n\t<i1>\n\t<i2>\n\t<i3>\n\t<i4>\n\t<i5>\n\t<i6>");
}
fail;
}
default:
return;
}
}
Not very pretty, but I expected it to work. Unfortunately, the code runs for about 30 seconds before crashing with an "out of memory" error.
I then tried a better looking alternative:
public void findClone2(list[str] In, list[str] whatWeSearchFor)
{
for ([*str head, *str mid, *str end] := In)
if (mid == whatWeSearchFor)
println("gotcha");
}
with approximately the same result (seems to run a little longer before running out of memory)
Finally, I tried a 'good old' C-style approach with a for-loop
public void findClone3(list[str] In, list[str] whatWeSearchFor)
{
cloneLength = size(whatWeSearchFor);
inputLength = size(In);
if(inputLength < cloneLength) return [];
loopLength = inputLength - cloneLength + 1;
for(int i <- [0..loopLength])
{
isAClone = true;
for(int j <- [0..cloneLength])
{
if(In[i+j] != whatWeSearchFor[j])
isAClone = false;
}
if(isAClone) println("Found clone <whatWeSearchFor> on lines <i> through <i+cloneLength-1>");
}
}
To my surprise, this one works like a charm. No out of memory, and results in seconds.
I get that my first two attempts probably create a lot of temporary string objects that all have to be garbage collected, but I can't believe that the only solution that worked really is the best solution.
Any pointers would be greatly appreciated.
My relevant eclipse.ini settings are
-XX:MaxPermSize=512m
-Xms512m
-Xss64m
-Xmx1G
We'll need to look to see why this is happening. Note that, if you want to use pattern matching, this is maybe a better way to write it:
public void findClone(list[str] In, str S1, str S2, str S3, str S4, str S5, str S6) {
switch(In) {
case [*str head, S1, S2, S3, S4, S5, S6, *str tail]: {
println("found duplicate\n\t<S1>\n\t<S2>\n\t<S3>\n\t<S4>\n\t<S5>\n\t<S6>");
}
default:
return;
}
}
If you do this, you are taking advantage of Rascal's matcher to actually find the matching strings directly, versus your first example in which any string would match but then you needed to use a number of separate comparisons to see if the match represented the combination you were looking for. If I run this on 110145 through 110150 it takes a while but works and it doesn't seem to grow beyond the heap space you allocated to it.
Also, is there a reason you are using fail? Is this to continue searching?
It's an algorithmic issue like Mark Hills said. In Rascal some short code can still entail a lot of nested loops, almost implicitly. Basically every * splice operator on a fresh variable that you use on the pattern side in a list generates one level of loop nesting, except for the last one which is just the rest of the list.
In your code of findClone2 you are first generating all combinations of sublists and then filtering them using the if construct. So that's a correct algorithm, but probably slow. This is your code:
void findClone2(list[str] In, list[str] whatWeSearchFor)
{
for ([*str head, *str mid, *str end] := In)
if (mid == whatWeSearchFor)
println("gotcha");
}
You see how it has a nested loop over In, because it has two effective * operators in the pattern. The code runs therefore in O(n^2), where n is the length of In. I.e. it has quadratic runtime behaviour for the size of the In list. In is a big list so this matters.
In the following new code, we filter first while generating answers, using fewer lines of code:
public void findCloneLinear(list[str] In, list[str] whatWeSearchFor)
{
for ([*str head, *whatWeSearchFor, *str end] := In)
println("gotcha");
}
The second * operator does not generate a new loop because it is not fresh. It just "pastes" the given list values into the pattern. So now there is actually only one effective * which generates a loop which is the first on head. This one makes the algorithm loop over the list. The second * tests if the elements of whatWeSearchFor are all right there in the list after head (this is linear in the size of whatWeSearchFor and then the last *_ just completes the list allowing for more stuff to follow.
It's also nice to know where the clone is sometimes:
public void findCloneLinear(list[str] In, list[str] whatWeSearchFor)
{
for ([*head, *whatWeSearchFor, *_] := In)
println("gotcha at <size(head)>");
}
Rascal does not have an optimising compiler (yet) which might possibly internally transform your algorithms to equivalent optimised ones. So as a Rascal programmer you are still asked to know the effect of loops on your algorithms complexity and know that * is a very short notation for a loop.
I want to enable Lua-Scripting (Lua 5.1) in my Delphi application. For this purpose I use the header Files of Thomas Lavergne.
Now I try to register a userdata type following this example: http://www.lua.org/pil/28.2.html
At the "new array function" it uses the command *luaL_getmetatable*.
static int newarray (lua_State *L) {
int n = luaL_checkint(L, 1);
size_t nbytes = sizeof(NumArray) + (n - 1)*sizeof(double);
NumArray *a = (NumArray *)lua_newuserdata(L, nbytes);
luaL_getmetatable(L, "LuaBook.array");
lua_setmetatable(L, -2);
a->size = n;
return 1; /* new userdatum is already on the stack */
}
Unfortunately the *luaL_getmetatable* Function is marked al old at my header File and commented out. I tried to activate it again but as expected I will get an error because the dll entrancepoint couldn't be found.
This is the Delphi-translation of that example (using another non array datatype)
Type
tMyType = tWhatever;
pMyType = ^tMyType;
{...}
Function newusertype(aState : pLua_State) : LongInt; cdecl;
Var
NewData : pMyType;
Begin
Result := 0;
NewData := lua_newuserdata(aState, SizeOf(tMyType ));
NewData^ := GetInitValue;
luaL_getMetaTable(aState, 'myexcample.mytype'); // Error/unknown function
lua_setmetatable(aState, -2);
Result := 1;
End;
Now I'm looking for an replacement of luaL_getMetaTable. I haven't found any information about one. In fact I haven't found any information that luaL_getMetaTable is outdated but it seems to be :(.
use lua_newmetatable(aState, 'myexample.mytype'). The thing is (if you only want to continue if the metatable already exists) you'll need to evaluate whether it returns a 0! If it returns 0, then it's wanting to create the metatable... in which case you can lua_pop(aState, 1).
Just remember that lua_newmetatable is a function returning an Integer (which in reality should be a Boolean).
Otherwise you can wait a few weeks for me to release Lua4Delphi version 2, which makes all of this super easy (and the Professional version actually automates the registration of Delphi Types and Instances with Lua)