I have never felt so woefully inadequate as I am when trying to prove to Dafny that my program is correct, so I need your help: The given program looks as follows:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M //given precondition
ensures 2*s == M*(M+1)-N*(N+1) //given postcondition
{
var a: int := N;
var r: int := 0;
while a < M
invariant FIND ME!
{
a := a+1;
r := r+a;
}
return r;
}
As first step I wanted to figure out what the loop does, so I made a table:
With that I worked out a loop invariant for r:
invariant r == (a-N)*N + (a-N)*((a-N)+1)/2
which holds before (0==0), and after each iteration of the loop (following the formula). Clearly it does not satisfy the Termination criteria
When the loop terminates, the loop invariant along with the reason that the loop terminated gives us a useful property.
Since the loop guard is simple enough I figured the complete invariant should be
invariant a<=M && r == (a-N)*N + (a-N)*((a-N)+1)/2
And thus my invariant satisfies Initialization, Maintenance and Termination. However Dafny complains that
Error: This loop invariant might not be maintained by the loop.
How do I make Dafny happy?
I managed to stay clear of any non-linear arithmetic hick-ups. Here's how I think of the problem:
You're trying to establish a postcondition that, for the sake of clarity, I will write as P(s, N, M), that is, some function of s, N, and M. One technique for coming up with a loop that does this is "replace a constant by a variable". What this means is that you pick one of the constants of the desired postcondition (here, you can choose either N or M, since s is not a constant) and replace it by a variable that is going to change in each loop iteration. Let's pick M as the constant and let's introduce (as you had already done in your program) a as the variable. Since we picked M as the constant, we'll want the final value of a to be M, so we'll start a at N. We then have:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures P(s, N, M)
{
var a := N;
while a < M
invariant N <= a <= M
invariant P(s, N, a) // postcondition, but with variable a instead of constant M
}
If you type in this program (but expand out the P(s, N, a) to the actual condition), then you will find that Dafny proves the postcondition. In other words, the verifier is giving you the information that if you can establish and maintain this loop invariant, then the program will correctly establish the postcondition.
You can see this yourself, too. The negation of the loop guard gives you M <= a, which combined with the loop invariant a <= M gives you a == M. When you combine a == M and the loop invariant P(s, N, a), you get the postcondition P(s, N, M).
Great. But the verifier issues a complaint that the loop invariant does not hold on entry. This is because we didn't provide any initial value for s. Since a has the initial value N, we need to find a value for s that satisfies P(s, N, N). That value is 0, so we update the program to
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures P(s, N, M)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant P(s, N, a)
}
Next, let's write the loop body. (Notice how I have started with the loop invariant, rather than starting with loop body and then trying to figure out an invariant. For these sorts of programs, I find that's the easiest way.) We already decided that we want to vary a from the initial value N up to the final value M, so we add the assignment a := a + 1;:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant P(s, N, a)
{
a := a + 1;
}
}
This addresses termination. The final thing we need to do is update s inside the loop so that the invariant is maintained. This is mostly easily done backward, in a goal-directed fashion. Here's how: At the end of the loop body, we want to make sure P(s, N, a) holds. That means we want the condition P(s, N, a + 1) to hold before the assignment to a. You obtain this condition (again, remember we're working backward) by replacing a in the desired condition with (the right-hand side of the assignment) a + 1.
Okay, so before the assignment to a, we want to have P(s, N, a + 1), and what we've got just inside the loop body is the invariant P(s, N, a). Now, it's time for me to expand P(...) to your actual condition. Alright, we have
2*s == a*(a+1) - N*(N+1) (*)
and we want
2*s == (a+1)*(a+2) - N*(N+1) (**)
Let's rewrite (a+1)*(a+2) in (**) as 2*(a+1) + a*(a+1). So, (**) can equivalently be written as
2*s == 2*(a+1) + a*(a+1) - N*(N+1) (***)
If you compare (***) (which is what we want) with (*) (which is what we've got), then you notice that the right-hand side of (***) is 2*(a+1) more than the right-hand side of (*). So, we must arrange to increase the left-hand side with the same amount.
If you increase s by a+1, then the left-hand side 2*s goes up by 2*(a+1), which is what we want. So, our final program is
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant 2*s == a*(a+1) - N*(N+1)
{
s := s + a + 1;
a := a + 1;
}
}
If you want, you can swap the order of the assignments to s and a. This will give you
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant 2*s == a*(a+1) - N*(N+1)
{
a := a + 1;
s := s + a;
}
}
In summary, we have built the loop body from the loop invariant, and we designed this loop invariant by "replacing a constant with a variable" in the postcondition.
Rustan
You are running afoul of the curse of nonlinear arithmetic. Any time you rely on nontrivial properties of multiplication, Dafny will have a hard time with your program.
Here is one way to fix your specific proof. Sorry that it is so messy. I'm sure it can be cleaned up, but I just hacked something together to show you the idea.
function {:opaque} mul(a: int, b: int): int
{
a * b
}
lemma MulZero1(a: int)
ensures mul(0, a) == 0
{
reveal mul();
}
lemma MulNeg1(a: int, b: int)
ensures mul(-a, b) == -mul(a, b)
{
reveal mul();
}
lemma MulNeg2(a: int, b: int)
ensures mul(a, -b) == -mul(a, b)
{
reveal mul();
}
lemma MulInd(a: nat, b: int)
ensures mul(a, b) == if a == 0 then 0 else mul(a-1, b) + b
{
reveal mul();
}
lemma MulEven(a: int, b: int)
requires b % 2 == 0
decreases if a < 0 then -a + 1 else a
ensures mul(a, b) % 2 == 0
{
if a < 0 {
MulNeg1(a, b);
MulEven(-a, b);
} else if a == 0 {
MulZero1(b);
} else {
calc {
mul(a, b) % 2;
{ MulInd(a, b); }
(mul(a-1, b) + b) % 2;
mul(a-1, b) % 2;
{ MulEven(a-1, b); }
0;
}
}
}
lemma MulComm(a: int, b: int)
ensures mul(a, b) == mul(b, a)
{
reveal mul();
}
lemma MulAdjEven(a: int)
ensures mul(a, a + 1) % 2 == 0
{
var m := a % 2;
if m == 0 {
MulComm(a, a+1);
MulEven(a+1, a);
} else {
assert m == 1;
assert (a + 1) % 2 == 0;
MulEven(a, a+1);
}
}
method doingMath(N: int, M: int) returns (s: int)
requires N <= M //given precondition
ensures 2*s == mul(M,M+1) - mul(N,N+1) //given postcondition
{
var a: int := N;
var r: int := 0;
assert mul(a-N, N) + mul(a-N, (a-N)+1)/2 == 0 by {
reveal mul();
}
while a < M
invariant a <= M
invariant r == mul(a-N, N) + mul(a-N, (a-N)+1)/2
{
a := a+1;
r := r+a;
assert r == mul(a-N, N) + mul(a-N, (a-N)+1)/2 by {
reveal mul();
}
}
calc {
2*r;
2* (mul(M-N, N) + mul(M-N, (M-N)+1)/2);
{ MulAdjEven(M-N); }
2*mul(M-N, N) + mul(M-N, (M-N)+1);
{ reveal mul(); }
mul(M,M+1) - mul(N,N+1);
}
return r;
}
Multiplication is hard for Dafny, so we manually wrap it in an opaque function. This gives us fine-grained control of when Dafny is allowed to "know" that the function is really multiplication.
Then we can replace all the occurrences of multiplication in your method by calls to mul. This makes Dafny fail quickly. (That's a big improvement over timing out!) Then we can selectively reveal the definition of mul where we need it, or we can prove lemmas about mul.
The hardest lemma is MulEven. Try replacing its body/proof by reveal mul(); and you will see that Dafny times out. Instead, I had to prove it by induction. This proof itself necessitated several other lemmas about multiplication. Fortunately, all of them were easy.
You may also want to take a look at the math library developed as part of the IronFleet project here. (Start by reading the files whose names contain the word "nonlinear"; those are the lowest-level proofs that are closest to the axioms.) They use a similar approach to build up a large body of facts about multiplication (and division and modulo), so that those functions can remain opaque everywhere else in the codebase, improving Dafny's performance.
I'm trying to run a dafny verified version of BFS (from here)
My input graph is perfectly fine, but for some reason it fails the pre-condition check.
Here is the permalink
And for self completeness here is the graph definition + validity conditions
class Graph
{
var adjList : seq<seq<int>>;
constructor (adjListInput : seq<seq<int>>)
{
adjList := adjListInput;
}
}
function ValidGraph(G : Graph) : bool
reads G
{
(forall u :: 0 <= u < |G.adjList| ==> forall v :: 0 <= v < |G.adjList[u]| ==> 0 <= G.adjList[u][v] < |G.adjList|) &&
(forall u :: 0 <= u < |G.adjList| ==> forall v,w :: 0 <= v < w < |G.adjList[u]| ==> G.adjList[u][v] != G.adjList[u][w])
}
method main()
{
var G : Graph := new Graph([[1,2],[0,2],[0,1]]);
assert (ValidGraph(G));
}
dafny's response is Error: assertion violation
You just need to add ensures adjList == adjListInput to the constructor. Because Dafny treats a constructor basically just like a method, and because Dafny analyzes each method in isolation, when Dafny analyzes main, it only uses the specification of the constructor, not the body of the constructor. So the reason the assert was failing was because from the perspective of main, the constructor was setting the field adjList to an arbitrary value that did not necessarily correspond to its argument.
I am wondering if there is a reason why dafny is unable to verify my program?
https://rise4fun.com/Dafny/Ip1s
Am I missing some additional invariant?
The problem is that your definition of s and your construction of o go in "different directions". The recursive case of s defines s(i) in terms of
i[0] and what is "previously" defined by s(i[1..]). In contrast, the loop iteration defines the new o in terms of i[n] and the previous value of o. It would take an inductively proven lemma to establish the proof obligations in your current program, and Dafny does not invent such lemmas by itself.
For the record in this answer, here is what you started with:
function s(i: seq<int>): seq<int> {
if |i| == 0 then [] else
if i[0] == 42 then [i[0]] + s(i[1..])
else s(i[1..])
}
method q (i: seq<int>) returns (o: seq<int>)
ensures o == s(i)
{
var n := 0;
o := [];
while n < |i|
invariant n <= |i| && o == s(i[..n])
{
if i[n] == 42 {
o := o + [i[n]];
}
n := n + 1;
}
}
There are four ways out.
One way out is to define a different version of s, call it s', that recurses from the other end of the given sequence. Then, replace s by s' in your method specification and loop invariant. This is a fine solution, unless for some reason you really prefer s, not s', in your method specification.
A second way out is to define such an s' and to prove a lemma that s(i) and s'(i) return the same value. This will let you keep s in your method specification, at the cost of having two function definitions and having to write (and prove and use) a lemma.
A third way out is to change the loop to iterate "downward" instead of "upward". That is, start n at |i| and decrement n in the loop body. (As usual, an increment of n is typically best done at the end of the loop body (post-increment), whereas a decrement of n is typically best done at the beginning of the loop body (pre-decrement).)
A fourth way out is to change the way you write the loop invariant about o. Currently, the invariant speaks about what you already have computed, that is, o == s(i[..n]). You can instead write the invariant in terms of what is yet to be computed, as in o + s(i[n..]) == s(i), which you can read as "once I have added s(i[n..]) to o, I will have s(i)". Here is that version of q:
method q(i: seq<int>) returns (o: seq<int>)
ensures o == s(i)
{
var n := 0;
o := [];
while n < |i|
invariant n <= |i| && o + s(i[n..]) == s(i)
{
if i[n] == 42 {
o := o + [i[n]];
}
n := n + 1;
}
}
You may also be interested in watching this episode of Verification Corner on this subject.
Rustan
Suppose I have the following type system:
Type :=
A ; type of A
B ; type of B
C ; type of C
Int ; type of int
Real ; type of real
v :=
abc ; value of abc
real ; value of real
int ; value of int
t :=
v ; values can be mapped to terms
<abc_0, abc_1,... abc_n> = break abc' into int ; this will break abc' into real parts (think of this as a generic partitioning or division operation.
abc = op abc' for int ;
+ ; math plus operator, assume this "works" for types A|B|C, and works as expected for Int|Real
Suppose that the body of op and break is as such:
; op return type could, maximally, be the union of x, y, z, or minimally be the union types of x, y or x, z
op(x, y): ; recall, x:{A|B|C} and y:{A|B|C}
A z;
while([some condition]):
if ([another condition]):
x += y
else:
x += z
return x
break(x, y):
; For simplicity, this can just return an array of y-size, whose type is that of x
; in implementation this is much more complicated than this, but for simplicity this suffices to articulate the concept.
return array[y];
Now I'm given this program, with dynamic typing:
a = ... ; this is of {A|B|C}
x = 5 ; typeof Int
b = op a for x
c = break b into x
I need to prove the satisfiability of this program, using Z3.
I'm really not sure how to represent this program in z3. The union typing, I think can be handled by the custom datatypes below. But I am unclear how to handle loops/conditionals. Just as in sentential logic, do they translate to forall or there exists at least one quantifiers?
(declare-datatypes () ((Type A B C)))
(define-sort Set (T) (Array T Type))
(declare-fun break_func (Type Real) (Set Type))
; not sure if this is correct or not...
(declare-fun op_func (Type Real) (Set Type))
(declare-const a Type)
(declare-const x Int)
; not quite sure what to put my for asserts....
; should I assert that the size of the break_func matches the input? If so, how
; I don't understand how to type check the output for op_func
(check-sat)
(get-model)
My main questions, aside from "is this correct" is that:
The Z3 implementation of this isn't a full representation of the program, but I don't understand what asserts I should be using for my Z3 implementation for this use-case
I don't understand how to account for union typing in Z3?
I have simple Z3 python code like below. I expect the "print" line will return me "y" which was stored in the line above it. Instead, I got back "A[x]" as result.
I = IntSort()
A = Array('A', I, I)
x = Int('x')
y = Int('y')
Store(A, x, y)
print Select(A,x)
Why does not Select() return the value stored by Store()?
Thanks.
There are two things to note:
First:
When you write
Store(A, x, y)
You create a term with three arguments , A, x, and y.
There is no side-effect to A.
You can create a name for this term by writing
B = Store(A,x,y)
Second:
Z3 does not simplify terms unless you want it to.
The python API exposes a simplification function called simplify.
You can obtain the reduced term by calling the simplifier.
The example is:
I = IntSort()
A = Array('A', I, I)
x = Int('x')
y = Int('y')
B = Store(A, x, y)
print Select(B,x)
print simplify (Select(B,x))