Can I allow preconditions on the argument to a higher-order function in Dafny? - dafny

Is there a way to say that a higher-order function permits preconditions on the function it takes as an argument?
Here's the concrete situation I'm trying to solve. I wrote this function for filtering items in a seq based on a predicate:
function method filterSeq<T>(s: seq<T>, fn: T -> bool): seq<T>
ensures forall i | 0 <= i < |filterSeq(s, fn)| :: fn(filterSeq(s, fn)[i]) == true
ensures forall i | 0 <= i < |filterSeq(s, fn)| :: filterSeq(s, fn)[i] in s
ensures forall i | 0 <= i < |s| :: fn(s[i]) ==> s[i] in filterSeq(s, fn)
{
if |s| == 0 then [] else (
if fn(s[0]) then [s[0]] + filterSeq(s[1..], fn) else filterSeq(s[1..], fn)
)
}
This covers the postconditions I care about, but doesn't say anything about the argument to fn. I want to say that for any property which holds for all T in s, this property is permitted as a precondition on fn.
The problem that lead me to this was an attempt to filter a sequence containing other sequences:
var sequences := [[1, 2, 3], [4, 5, 6], [7, 8, 9]];
assert forall i | 0 <= i < |sequences| :: |sequences[i]| == 3;
var result := filterSeq(sequences, sequence => sequence[0] == 4);
When I attempt this, I get an error on sequence[0], which says index out of range. Fair enough, I try to add a precondition to the lambda:
var sequences := [[1, 2, 3], [4, 5, 6], [7, 8, 9]];
assert forall i | 0 <= i < |sequences| :: |sequences[i]| == 3;
var result := filterSeq(sequences,
sequence requires |sequence| == 3 => sequence[0] == 4);
Now I get the error value does not satisfy the subset constraints of 'seq<int> -> bool' (possible cause: it may be partial or have read effects) on the lambda argument. This also makes sense, I'm passing a partial function where I wrote the type signature for a non-partial one.
My question is: How do I change filterSeq to allow this? Can I somehow write it so that it will work with arbitrary preconditions, or will I have to write a separate filterSeqOfSequences method to cover this specific use-case?

Great question. The answer is yes. You need to use Dafny's notion of partial functions, which are written with a doubly-dashed arrow like T --> bool. If f has this type, then f.requires is its precondition. (In fact, you can think of the total function type T -> bool as a special case of T --> bool where f.requires is true for all values of type T.)
Here is one way to rewrite your higher-order function to accept a partial function as an argument:
function method filterSeq<T>(s: seq<T>, fn: T --> bool): seq<T>
requires forall x | x in s :: fn.requires(x) // ***
ensures forall x | x in filterSeq(s, fn) :: x in s // ***
ensures forall i | 0 <= i < |filterSeq(s, fn)| :: fn(filterSeq(s, fn)[i]) == true
ensures forall i | 0 <= i < |filterSeq(s, fn)| :: filterSeq(s, fn)[i] in s
ensures forall i | 0 <= i < |s| :: fn(s[i]) ==> s[i] in filterSeq(s, fn)
{
if |s| == 0
then []
else if fn(s[0])
then [s[0]] + filterSeq(s[1..], fn)
else filterSeq(s[1..], fn)
}
method Test()
{
var sequences := [[1, 2, 3], [4, 5, 6], [7, 8, 9]];
assert forall i | 0 <= i < |sequences| :: |sequences[i]| == 3;
var result := filterSeq(sequences,
sequence requires |sequence| == 3 => sequence[0] == 4);
}
I just made two changes to your code, marked by ***.
First is the new precondition on filterSeq that you already mentioned in your question that requires that fn.requires holds on all elements of s. Second, we also need one new technical postcondition that guarantees the output of filterSeq is a subset of its input. This is needed to ensure the well formedness of the other postconditions, which try to call fn on elements of the output.
The Test method does not change at all. It just works with the new version of filterSeq.

Related

dafny pre-condition failure

I'm trying to run a dafny verified version of BFS (from here)
My input graph is perfectly fine, but for some reason it fails the pre-condition check.
Here is the permalink
And for self completeness here is the graph definition + validity conditions
class Graph
{
var adjList : seq<seq<int>>;
constructor (adjListInput : seq<seq<int>>)
{
adjList := adjListInput;
}
}
function ValidGraph(G : Graph) : bool
reads G
{
(forall u :: 0 <= u < |G.adjList| ==> forall v :: 0 <= v < |G.adjList[u]| ==> 0 <= G.adjList[u][v] < |G.adjList|) &&
(forall u :: 0 <= u < |G.adjList| ==> forall v,w :: 0 <= v < w < |G.adjList[u]| ==> G.adjList[u][v] != G.adjList[u][w])
}
method main()
{
var G : Graph := new Graph([[1,2],[0,2],[0,1]]);
assert (ValidGraph(G));
}
dafny's response is Error: assertion violation
You just need to add ensures adjList == adjListInput to the constructor. Because Dafny treats a constructor basically just like a method, and because Dafny analyzes each method in isolation, when Dafny analyzes main, it only uses the specification of the constructor, not the body of the constructor. So the reason the assert was failing was because from the perspective of main, the constructor was setting the field adjList to an arbitrary value that did not necessarily correspond to its argument.

missing invariant in dafny code involving sequences

I am wondering if there is a reason why dafny is unable to verify my program?
https://rise4fun.com/Dafny/Ip1s
Am I missing some additional invariant?
The problem is that your definition of s and your construction of o go in "different directions". The recursive case of s defines s(i) in terms of
i[0] and what is "previously" defined by s(i[1..]). In contrast, the loop iteration defines the new o in terms of i[n] and the previous value of o. It would take an inductively proven lemma to establish the proof obligations in your current program, and Dafny does not invent such lemmas by itself.
For the record in this answer, here is what you started with:
function s(i: seq<int>): seq<int> {
if |i| == 0 then [] else
if i[0] == 42 then [i[0]] + s(i[1..])
else s(i[1..])
}
method q (i: seq<int>) returns (o: seq<int>)
ensures o == s(i)
{
var n := 0;
o := [];
while n < |i|
invariant n <= |i| && o == s(i[..n])
{
if i[n] == 42 {
o := o + [i[n]];
}
n := n + 1;
}
}
There are four ways out.
One way out is to define a different version of s, call it s', that recurses from the other end of the given sequence. Then, replace s by s' in your method specification and loop invariant. This is a fine solution, unless for some reason you really prefer s, not s', in your method specification.
A second way out is to define such an s' and to prove a lemma that s(i) and s'(i) return the same value. This will let you keep s in your method specification, at the cost of having two function definitions and having to write (and prove and use) a lemma.
A third way out is to change the loop to iterate "downward" instead of "upward". That is, start n at |i| and decrement n in the loop body. (As usual, an increment of n is typically best done at the end of the loop body (post-increment), whereas a decrement of n is typically best done at the beginning of the loop body (pre-decrement).)
A fourth way out is to change the way you write the loop invariant about o. Currently, the invariant speaks about what you already have computed, that is, o == s(i[..n]). You can instead write the invariant in terms of what is yet to be computed, as in o + s(i[n..]) == s(i), which you can read as "once I have added s(i[n..]) to o, I will have s(i)". Here is that version of q:
method q(i: seq<int>) returns (o: seq<int>)
ensures o == s(i)
{
var n := 0;
o := [];
while n < |i|
invariant n <= |i| && o + s(i[n..]) == s(i)
{
if i[n] == 42 {
o := o + [i[n]];
}
n := n + 1;
}
}
You may also be interested in watching this episode of Verification Corner on this subject.
Rustan

Invariant set may vary

A method that copies the negative elements of an array of integers into another array has the property that the set of elements in the result is a subset of the elements in the original array, which stays the same during the copy.
The problem in the code below is that, as soon as we write something in the result array, Dafny somehow forgets that the original set is unchanged.
How to fix this?
method copy_neg (a: array<int>, b: array<int>)
requires a != null && b != null && a != b
requires a.Length == b.Length
modifies b
{
var i := 0;
var r := 0;
ghost var sa := set j | 0 <= j < a.Length :: a[j];
while i < a.Length
invariant 0 <= r <= i <= a.Length
invariant sa == set j | 0 <= j < a.Length :: a[j]
{
if a[i] < 0 {
assert sa == set j | 0 <= j < a.Length :: a[j]; // OK
b[r] := a[i];
assert sa == set j | 0 <= j < a.Length :: a[j]; // KO!
r := r + 1;
}
i := i + 1;
}
}
Edit
Following James Wilcox's answer, replacing inclusions of sets with predicates on sequences is what works the best.
Here is the complete specification (for an array with distinct elements). The post-condition has to be detailed a bit in the loop invariant and a dumb assert remains in the middle of the loop, but all ghost variables are gone, which is great.
method copy_neg (a: array<int>, b: array<int>)
returns (r: nat)
requires a != null && b != null && a != b
requires a.Length <= b.Length
modifies b
ensures 0 <= r <= a.Length
ensures forall x | x in a[..] :: x < 0 <==> x in b[..r]
{
r := 0;
var i := 0;
while i < a.Length
invariant 0 <= r <= i <= a.Length
invariant forall x | x in b[..r] :: x < 0
invariant forall x | x in a[..i] && x < 0 :: x in b[..r]
{
if a[i] < 0 {
b[r] := a[i];
assert forall x | x in b[..r] :: x < 0;
r := r + 1;
}
i := i + 1;
}
}
This is indeed confusing. I will explain why Dafny has trouble proving this below, but first let me give a few ways to make it go through.
First workaround
One way to make the proof go through is to insert the following forall statement after the line b[r] := a[i];.
forall x | x in sa
ensures x in set j | 0 <= j < a.Length :: a[j]
{
var j :| 0 <= j < a.Length && x == old(a[j]);
assert x == a[j];
}
The forall statement is a proof that sa <= set j | 0 <= j < a.Length :: a[j]. I will come back to why this works below.
Second workaround
In general, when reasoning about arrays in Dafny, it is best to use the a[..] syntax to convert the array to a mathematical sequence, and then work with that sequence. If you really need to work with the set of elements, you can use set x | x in a[..], and you will have a better time than if you use set j | 0 <= j < a.Length :: a[j].
Systematically replacing set j | 0 <= j < a.Length :: a[j] with set x | x in a[..] causes your program to verify.
Third solution
Popping up a level to specifying your method, it seems like you don't actually need to mention the set of all elements. Instead, you can get away with saying something like "every element of b is an element of a". Or, more formally forall x | x in b[..] :: x in a[..]. This is not quite a valid postcondition for your method, because your method may not fill out all of b. Since I'm not sure what your other constraints are, I'll leave that to you.
Explanations
Dafny's sets with elements of type A are translated to Boogie maps [A]Bool, where an element maps to true iff it is in the set. Comprehensions such as set j | 0 <= j < a.Length :: a[j] are translated to Boogie maps whose definition involves an existential quantifier. This particular comprehension translates to a map that maps x to
exists j | 0 <= j < a.Length :: x == read($Heap, a, IndexField(j))
where the read expression is the Boogie translation of a[j], which, in particular, makes the heap explicit.
So, to prove that an element is in the set defined by the comprehension, Z3 needs to prove an existential quantifier, which is hard. Z3 uses triggers to prove such quantifiers, and Dafny tells Z3 to use the trigger read($Heap, a, IndexField(j)) when trying to prove this quantifier. This turns out to not be a great trigger choice, because it mentions the current value of the heap. Thus, when the heap changes (ie, after updating b[r]), the trigger may not fire, and you get a failing proof.
Dafny lets you customize the trigger it uses for set comprehensions using a {:trigger} attribute. Unfortunately, there is no great choice of trigger at the Dafny level. However, a reasonable trigger for this program at the Boogie/Z3 level would be just IndexField(j) (though this is likely a bad trigger for such expressions in general, since it is overly general). Z3 itself will infer this trigger if Dafny doesn't tell it otherwise. You can Dafny to get out of the way by saying {:autotriggers false}, like this
invariant sa == set j {:autotriggers false} | 0 <= j < a.Length :: a[j]
This solution is unsatisfying and requires detailed knowledge of Dafny's internals. But now that we've understood it, we can also understand the other workarounds I proposed.
For the first workaround, the proof goes through because the forall statement mentions a[j], which is the trigger. This causes Z3 to successfully prove the existential.
For the second workaround, we have simplified the set comprehension expression so that it no longer introduces an existential quantifier. Instead the comprehension set x | x in a[..], is translated to a map that maps x to
x in a[..]
(ignoring details of how a[..] is translated). This means that Z3 never has to prove an existential, so the otherwise very similar proof goes through.
The third solution works for similar reasons, since it uses no comprehensions and thus no problematic existential quantifiers/

Pure pattern matching

I am building a function that counts of many times a character appears in a string after the nth position.
countCh ("aaabbbccc", 3, 'b')
val it: int = 2
In C, I would use an accumulator with a while loop. But I am trying to learn the F# functional face, where this approach is discouraged.
So I used guards to test few conditions and build the function:
let rec countCh (s:string, n:int, ch:char) =
match s, n, ch with
| (s, n, ch) when n > s.Length -> 0 //p1
| (s, n, ch) when n < 0 -> 0 //p2
| (s, n, ch) when s.[n] <> ch -> countCh(s, n + 1, ch) //p3
| (s, n, ch) when s.[n] = ch -> 1 + countCh(s, n + 1, ch) //p4
The coexistence of patterns 3 and 4 is problematic (impossible, I am afraid). Even if it compiles, I have not been able to make it work. How can this task functionally be handled?
First, the coexistence of these branches is not problematic. They don't conflict with each other. Why do you think that it's problematic? Is it because you get an "Incomplete pattern match" compiler warning? That warning does not tell you that the branches conflict, it tells you that the compiler can't prove that the four branches cover all possibilities. Or do you think that for some other reason? If you want your questions to be answered accurately, you'll have to ask them more clearly.
Second, you're abusing the pattern matching. Look: there are no patterns! The patterns in every branch are exactly the same, and trivial. Only guards are different. This looks very counterintuitively within a match, but would be plainly expressed with if..elif:
let rec countCh (s:string) n ch =
if n >= s.Length || n < 0 then 0
elif s.[n] = ch then 1 + countCh s (n + 1) ch
else countCh s (n + 1) ch
NOTE 1: see how I made the parameters curried? Always use curried form, unless there is a very strong reason to use tupled. Curried parameters are much more convenient to use on the caller side.
NOTE 2: your condition n > s.Length was incorrect: string indices go from 0 to s.Length-1, so the bail condition should be n >= s.Length. It is corrected in my code.
Finally, since this is an exercise, I must point out that the recursion is not tail recursion. Look at the second branch (in my code): it calls the function recursively and then adds one to the result. Since you have to do something with the result of the recursive call, the recursion can't be "tail". This means you risk stack overflow on very long inputs.
To make this into tail recursion, you need to turn the function "inside out", so to say. Instead of returning the result from every call, you need to pass it into every call (aka "accumulator"), and only return from the terminal case:
let rec countCh (s:string) n ch countSoFar =
if n >= s.Length || n < 0 then countSoFar
elif s.[n] = ch then countCh s (n+1) ch (countSoFar+1)
else countCh s (n+1) ch countSoFar
// Usage:
countCh "aaaabbbccc" 5 'b' 0
This way, every recursive call is the "last" call (i.e. the function doesn't do anything with the result, but passes it straight out to its own caller). This is called "tail recursion" and can be compiled to work in constant stack space (as opposed to linear).
I agree with the other answers, but I'd like to help you with your original question. You need to indent the function, and you have an off by one bug:
let rec countCh (s:string, n:int, ch:char) =
match s, n, ch with
| s, n, _ when n >= s.Length-1 -> 0 //p1
| s, _, _ when n < 0 -> 0 //p2
| s, n, ch when s.[n+1] <> ch -> countCh(s, n+2, ch) //p3
| s, n, ch when s.[n+1] = ch -> 1 + countCh(s, n+2, ch) //p4
I'd suggest to not write it yourself, but ask the library functions for help:
let countCh (s: string, n, c) =
s.Substring(n+1).ToCharArray()
|> Seq.filter ((=) c)
|> Seq.length
Or use Seq.skip, along with the fact that you can drop the conversion to character array:
let countCh (s: string, n, c) =
s
|> Seq.skip (n + 1)
|> Seq.filter ((=) c)
|> Seq.length

Dafny predicate neither true nor false

How can a Dafny predicate be neither true nor false?
This:
predicate sorted(s: seq<int>)
{
forall j, k :: 0 <= j < k < |s| ==> s[j] <= s[k]
}
lemma SortedTest()
{
assert sorted([1, 3, 2]);
assert !sorted([1, 3, 2]);
}
Produces double assertion violations:
Dafny program verifier version 1.9.7.30401, Copyright (c) 2003-2016, Microsoft.
Sort.dfy(8,10): Error: assertion violation
Sort.dfy(3,2): Related location
Sort.dfy(3,43): Related location
Execution trace:
(0,0): anon0
Sort.dfy(9,9): Error: assertion violation
Execution trace:
(0,0): anon0
Dafny program verifier finished with 2 verified, 2 errors
Dafny is not saying that the assertions are false, it is saying that it can't prove that they hold. If you give it a bit more help then it will prove the one that is true:
predicate sorted(s: seq<int>)
{
forall j, k :: 0 <= j < k < |s| ==> s[j] <= s[k]
}
lemma SortedTest()
{
var a := [1, 3, 2];
assert a[0] == 1 && a[1] == 3 && a[2] == 2;
assert sorted(a);
assert !sorted(a);
}

Resources