I have never felt so woefully inadequate as I am when trying to prove to Dafny that my program is correct, so I need your help: The given program looks as follows:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M //given precondition
ensures 2*s == M*(M+1)-N*(N+1) //given postcondition
{
var a: int := N;
var r: int := 0;
while a < M
invariant FIND ME!
{
a := a+1;
r := r+a;
}
return r;
}
As first step I wanted to figure out what the loop does, so I made a table:
With that I worked out a loop invariant for r:
invariant r == (a-N)*N + (a-N)*((a-N)+1)/2
which holds before (0==0), and after each iteration of the loop (following the formula). Clearly it does not satisfy the Termination criteria
When the loop terminates, the loop invariant along with the reason that the loop terminated gives us a useful property.
Since the loop guard is simple enough I figured the complete invariant should be
invariant a<=M && r == (a-N)*N + (a-N)*((a-N)+1)/2
And thus my invariant satisfies Initialization, Maintenance and Termination. However Dafny complains that
Error: This loop invariant might not be maintained by the loop.
How do I make Dafny happy?
I managed to stay clear of any non-linear arithmetic hick-ups. Here's how I think of the problem:
You're trying to establish a postcondition that, for the sake of clarity, I will write as P(s, N, M), that is, some function of s, N, and M. One technique for coming up with a loop that does this is "replace a constant by a variable". What this means is that you pick one of the constants of the desired postcondition (here, you can choose either N or M, since s is not a constant) and replace it by a variable that is going to change in each loop iteration. Let's pick M as the constant and let's introduce (as you had already done in your program) a as the variable. Since we picked M as the constant, we'll want the final value of a to be M, so we'll start a at N. We then have:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures P(s, N, M)
{
var a := N;
while a < M
invariant N <= a <= M
invariant P(s, N, a) // postcondition, but with variable a instead of constant M
}
If you type in this program (but expand out the P(s, N, a) to the actual condition), then you will find that Dafny proves the postcondition. In other words, the verifier is giving you the information that if you can establish and maintain this loop invariant, then the program will correctly establish the postcondition.
You can see this yourself, too. The negation of the loop guard gives you M <= a, which combined with the loop invariant a <= M gives you a == M. When you combine a == M and the loop invariant P(s, N, a), you get the postcondition P(s, N, M).
Great. But the verifier issues a complaint that the loop invariant does not hold on entry. This is because we didn't provide any initial value for s. Since a has the initial value N, we need to find a value for s that satisfies P(s, N, N). That value is 0, so we update the program to
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures P(s, N, M)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant P(s, N, a)
}
Next, let's write the loop body. (Notice how I have started with the loop invariant, rather than starting with loop body and then trying to figure out an invariant. For these sorts of programs, I find that's the easiest way.) We already decided that we want to vary a from the initial value N up to the final value M, so we add the assignment a := a + 1;:
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant P(s, N, a)
{
a := a + 1;
}
}
This addresses termination. The final thing we need to do is update s inside the loop so that the invariant is maintained. This is mostly easily done backward, in a goal-directed fashion. Here's how: At the end of the loop body, we want to make sure P(s, N, a) holds. That means we want the condition P(s, N, a + 1) to hold before the assignment to a. You obtain this condition (again, remember we're working backward) by replacing a in the desired condition with (the right-hand side of the assignment) a + 1.
Okay, so before the assignment to a, we want to have P(s, N, a + 1), and what we've got just inside the loop body is the invariant P(s, N, a). Now, it's time for me to expand P(...) to your actual condition. Alright, we have
2*s == a*(a+1) - N*(N+1) (*)
and we want
2*s == (a+1)*(a+2) - N*(N+1) (**)
Let's rewrite (a+1)*(a+2) in (**) as 2*(a+1) + a*(a+1). So, (**) can equivalently be written as
2*s == 2*(a+1) + a*(a+1) - N*(N+1) (***)
If you compare (***) (which is what we want) with (*) (which is what we've got), then you notice that the right-hand side of (***) is 2*(a+1) more than the right-hand side of (*). So, we must arrange to increase the left-hand side with the same amount.
If you increase s by a+1, then the left-hand side 2*s goes up by 2*(a+1), which is what we want. So, our final program is
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant 2*s == a*(a+1) - N*(N+1)
{
s := s + a + 1;
a := a + 1;
}
}
If you want, you can swap the order of the assignments to s and a. This will give you
method doingMath(N: int, M: int) returns (s: int)
requires N <= M
ensures 2*s == M*(M+1) - N*(N+1)
{
var a := N;
s := 0;
while a < M
invariant N <= a <= M
invariant 2*s == a*(a+1) - N*(N+1)
{
a := a + 1;
s := s + a;
}
}
In summary, we have built the loop body from the loop invariant, and we designed this loop invariant by "replacing a constant with a variable" in the postcondition.
Rustan
You are running afoul of the curse of nonlinear arithmetic. Any time you rely on nontrivial properties of multiplication, Dafny will have a hard time with your program.
Here is one way to fix your specific proof. Sorry that it is so messy. I'm sure it can be cleaned up, but I just hacked something together to show you the idea.
function {:opaque} mul(a: int, b: int): int
{
a * b
}
lemma MulZero1(a: int)
ensures mul(0, a) == 0
{
reveal mul();
}
lemma MulNeg1(a: int, b: int)
ensures mul(-a, b) == -mul(a, b)
{
reveal mul();
}
lemma MulNeg2(a: int, b: int)
ensures mul(a, -b) == -mul(a, b)
{
reveal mul();
}
lemma MulInd(a: nat, b: int)
ensures mul(a, b) == if a == 0 then 0 else mul(a-1, b) + b
{
reveal mul();
}
lemma MulEven(a: int, b: int)
requires b % 2 == 0
decreases if a < 0 then -a + 1 else a
ensures mul(a, b) % 2 == 0
{
if a < 0 {
MulNeg1(a, b);
MulEven(-a, b);
} else if a == 0 {
MulZero1(b);
} else {
calc {
mul(a, b) % 2;
{ MulInd(a, b); }
(mul(a-1, b) + b) % 2;
mul(a-1, b) % 2;
{ MulEven(a-1, b); }
0;
}
}
}
lemma MulComm(a: int, b: int)
ensures mul(a, b) == mul(b, a)
{
reveal mul();
}
lemma MulAdjEven(a: int)
ensures mul(a, a + 1) % 2 == 0
{
var m := a % 2;
if m == 0 {
MulComm(a, a+1);
MulEven(a+1, a);
} else {
assert m == 1;
assert (a + 1) % 2 == 0;
MulEven(a, a+1);
}
}
method doingMath(N: int, M: int) returns (s: int)
requires N <= M //given precondition
ensures 2*s == mul(M,M+1) - mul(N,N+1) //given postcondition
{
var a: int := N;
var r: int := 0;
assert mul(a-N, N) + mul(a-N, (a-N)+1)/2 == 0 by {
reveal mul();
}
while a < M
invariant a <= M
invariant r == mul(a-N, N) + mul(a-N, (a-N)+1)/2
{
a := a+1;
r := r+a;
assert r == mul(a-N, N) + mul(a-N, (a-N)+1)/2 by {
reveal mul();
}
}
calc {
2*r;
2* (mul(M-N, N) + mul(M-N, (M-N)+1)/2);
{ MulAdjEven(M-N); }
2*mul(M-N, N) + mul(M-N, (M-N)+1);
{ reveal mul(); }
mul(M,M+1) - mul(N,N+1);
}
return r;
}
Multiplication is hard for Dafny, so we manually wrap it in an opaque function. This gives us fine-grained control of when Dafny is allowed to "know" that the function is really multiplication.
Then we can replace all the occurrences of multiplication in your method by calls to mul. This makes Dafny fail quickly. (That's a big improvement over timing out!) Then we can selectively reveal the definition of mul where we need it, or we can prove lemmas about mul.
The hardest lemma is MulEven. Try replacing its body/proof by reveal mul(); and you will see that Dafny times out. Instead, I had to prove it by induction. This proof itself necessitated several other lemmas about multiplication. Fortunately, all of them were easy.
You may also want to take a look at the math library developed as part of the IronFleet project here. (Start by reading the files whose names contain the word "nonlinear"; those are the lowest-level proofs that are closest to the axioms.) They use a similar approach to build up a large body of facts about multiplication (and division and modulo), so that those functions can remain opaque everywhere else in the codebase, improving Dafny's performance.
Is it possible to treat int constants as uninterpreted in z3? For example, treat tuple(project(t, 0), project(t, 1)) = t as tuple(project(t, left), project(t, right)) = t. Context: my equations are essentially in QF_UF, but because they contain int constants I'm forced to use a logic with integer arithmetic which leads to nontermination sometimes.
You can declare Int to be an uninterpreted sort, so long as you set the logic to be something that doesn't define it already:
(set-logic QF_UF)
(declare-sort Int 0)
(declare-fun f ((Int)) Int)
(declare-fun i () Int)
(assert (distinct (f i) (f i)))
(check-sat)
z3 says:
unsat
If you add:
(assert (= (f 2) 2))
then you get:
(error "line 8 column 15: Sort mismatch at argument #1 for function (declare-fun f (Int) Int) supplied sort is Int")
which avoids confusions. (Though the error message is rather confusing to read!)
If you set your logic to be:
(set-logic QF_LIA)
then z3 says:
(error "line 3 column 18: sort already defined Int")
so, that works out as well. See Section 4.2.3 of the SMTLib specification for details.
Hope that helps!
I'm trying to prove the simple gcd algorithm in Dafny, so I wrote the following, but it seems I can not use the method divides inside the loop invariants:
method divides(d: int, x: int) returns (result: bool)
requires d > 0
requires x > 0
ensures (result == true ) ==> (exists q : int :: (d * q == x))
ensures (result == false) ==> (forall q : int :: (d * q != x))
{
// code omitted
}
method gcd(a: int, b: int) returns (result: int)
requires a > 0
requires b > 0
ensures (forall d : int :: ((exists q1 : int :: q1 * d == a) && (exists q2 :: (q2 * d == b))) ==>
(exists q3 : int :: (q3 * d == result)))
{
var x := a;
var y := b;
var fuel := a+b;
while ((x != y) && (fuel > 0))
decreases fuel
invariant x > 0
invariant y > 0
invariant (forall d : int :: (divides(d,x) && divides(d,y)) ==> (divides(d,a) && divides(d,b)))
{
// code omitted
}
return x;
}
Is there anyway to use a divides method/function/macro inside invariants?
Unlike methods, functions can appear in expressions. You can create a function:
function div(d: int, x: int): bool
{
if (d != 0 && x % d == 0) then true else false
}
Then in your method divides, you can have
ensures result == div(d,x)
and in your method gcd you can use the function div in your invariant.
Note, from the Dafny guide: One caveat of functions is that not only can they appear in annotations, they can only appear in annotations. Functions are never part of the final compiled program, they are just tools to help us verify our code.
Suppose I have the following type system:
Type :=
A ; type of A
B ; type of B
C ; type of C
Int ; type of int
Real ; type of real
v :=
abc ; value of abc
real ; value of real
int ; value of int
t :=
v ; values can be mapped to terms
<abc_0, abc_1,... abc_n> = break abc' into int ; this will break abc' into real parts (think of this as a generic partitioning or division operation.
abc = op abc' for int ;
+ ; math plus operator, assume this "works" for types A|B|C, and works as expected for Int|Real
Suppose that the body of op and break is as such:
; op return type could, maximally, be the union of x, y, z, or minimally be the union types of x, y or x, z
op(x, y): ; recall, x:{A|B|C} and y:{A|B|C}
A z;
while([some condition]):
if ([another condition]):
x += y
else:
x += z
return x
break(x, y):
; For simplicity, this can just return an array of y-size, whose type is that of x
; in implementation this is much more complicated than this, but for simplicity this suffices to articulate the concept.
return array[y];
Now I'm given this program, with dynamic typing:
a = ... ; this is of {A|B|C}
x = 5 ; typeof Int
b = op a for x
c = break b into x
I need to prove the satisfiability of this program, using Z3.
I'm really not sure how to represent this program in z3. The union typing, I think can be handled by the custom datatypes below. But I am unclear how to handle loops/conditionals. Just as in sentential logic, do they translate to forall or there exists at least one quantifiers?
(declare-datatypes () ((Type A B C)))
(define-sort Set (T) (Array T Type))
(declare-fun break_func (Type Real) (Set Type))
; not sure if this is correct or not...
(declare-fun op_func (Type Real) (Set Type))
(declare-const a Type)
(declare-const x Int)
; not quite sure what to put my for asserts....
; should I assert that the size of the break_func matches the input? If so, how
; I don't understand how to type check the output for op_func
(check-sat)
(get-model)
My main questions, aside from "is this correct" is that:
The Z3 implementation of this isn't a full representation of the program, but I don't understand what asserts I should be using for my Z3 implementation for this use-case
I don't understand how to account for union typing in Z3?
I am using z3 with the C++ API for the example:
context c;
sort I = c.int_sort();
sort B = c.bool_sort();
expr x = c.int_const("x");
expr x1 = c.int_const("x1");
func_decl p1 = function("p1", I, B);
func_decl p2 = function("p2", I, B);
solver s(c);
s.add(forall(x, (implies(p1(x), ((p2(x)))))));
s.add(p1(x1));
The generated model is:
sat
(define-fun x1 () Int
0)
(define-fun p1 ((x!1 Int)) Bool
true)
(define-fun p2 ((x!1 Int)) Bool
true)
The imagined to have the model p1(x1) and p2(x1). I tried also with the options:
p.set("mbqi", true);
p.set("smt.mbqi.max_iterations", "10000000");
p.set("auto-config", true);
But, I have the same result. Am I missing something?
Thank you.
Z3 produces a correct model, so I'm not exactly sure what the problem is. I can only imagine that there is some confusion about the argument names of the functions.
Per declaration, p1 is a function that takes an int and returns a Boolean. When Z3 builds an interpretation for this function, it names the first argument x!1 which has nothing to do with the constant function x1 (no bang). The model returned states
(define-fun x1 () Int
0)
(define-fun p1 ((x!1 Int)) Bool
true)
which means x1 is a constant function that always returns zero, i.e., x1() := 0. Additionally, p1 is a function with one argument (called x!1) and which returns true regardless of the input, i.e., for all x!1, p1(x!1) := true.