I'd like to write the following function in Dafny, which updates a map m1 with all mappings from m2, such that m2 overrides m1:
function update_map<K, V>(m1: map<K, V>, m2: map<K, V>): map<K, V>
ensures
(forall k :: k in m2 ==> update_map(m1, m2)[k] == m2[k]) &&
(forall k :: !(k in m2) && k in m1 ==> update_map(m1, m2)[k] == m1[k]) &&
(forall k :: !(k in m2) && !(k in m1) ==> !(k in update_map(m1, m2)))
{
map k | (k in m1 || k in m2) :: if k in m2 then m2[k] else m1[k]
}
I got the following errors:
Dafny 2.2.0.10923
stdin.dfy(7,2): Error: a map comprehension involved in a function definition is not allowed to depend on the set of allocated references; Dafny's heuristics can't figure out a bound for the values of 'k' (perhaps declare its type, 'K', as 'K(!new)')
stdin.dfy(7,2): Error: a map comprehension must produce a finite set, but Dafny's heuristics can't figure out how to produce a bounded set of values for 'k'
2 resolution/type errors detected in stdin.dfy
I don't understand the first error, and for the second, if m1 and m2 both have finite domains, then their union is certainly finite as well, but how can I explain that to Dafny?
UPDATE:
After applying James' fixes, it works:
function update_map<K(!new), V>(m1: map<K, V>, m2: map<K, V>): map<K, V>
ensures
(forall k :: k in m1 || k in m2 ==> k in update_map(m1, m2)) &&
(forall k :: k in m2 ==> update_map(m1, m2)[k] == m2[k]) &&
(forall k :: !(k in m2) && k in m1 ==> update_map(m1, m2)[k] == m1[k]) &&
(forall k :: !(k in m2) && !(k in m1) ==> !(k in update_map(m1, m2)))
{
map k | k in (m1.Keys + m2.Keys) :: if k in m2 then m2[k] else m1[k]
}
Good questions! You are running across some known sharp edges in Dafny that are under-documented.
In the first error, Dafny is basically saying that the type variable K needs to be constrained to not be a reference type. You can do that by changing the function signature to start with
function update_map<K(!new), V>...
Here, (!new) is Dafny syntax meaning exactly that K may only be instantiated with value types, not reference types. (Unfortunately, !new is not yet documented, but there is an open issue about this.)
In the second error, you are running afoul of Dafny's limited syntactic heuristics to prove finiteness, as described in this question and answer. The fix is to use Dafny's built-in set union operator instead of boolean disjunction, like this:
map k | k in m1.Keys + m2.Keys :: ...
(Here, I use .Keys to convert each map to the set of keys in its domain so that I can apply +, which works on sets but not maps.)
With those two type-checking-time errors fixed, you now get two new verification-time errors. Yay!
stdin.dfy(3,45): Error: element may not be in domain
stdin.dfy(4,59): Error: element may not be in domain
These are telling you that the statement of the postcondition itself is ill-formed, because you are indexing into maps using keys without properly hypothesizing that those keys are in the domain of the map. You can fix this by adding another postcondition (before the others), like this:
(forall k :: k in m1 || k in m2 ==> k in update_map(m1, m2)) && ...
After that, the whole function verifies.
Related
I am trying to find strategies to prove universally quantified assertions in Dafny. I see Dafny proves universal elimination
quite easily:
predicate P<X>(k:X)
lemma unElim<X>(x:X)
ensures (forall a:X :: P(a)) ==> P(x)
{ }
lemma elimHyp<H> ()
ensures forall k:H :: P(k)
lemma elimGoal<X> (x:X)
ensures P(x)
{ elimHyp<X>(); }
but I can not find how to prove the introduction rule:
//lemma unInto<X>(x:X)
// ensures P(x) ==> (forall a:X :: P(a))
// this definition is wrong
lemma introHyp<X> (x:X)
ensures P(x)
lemma introGoal<H> ()
ensures forall k:H :: P(k)
{ }
all ideas appreciated
Universal introduction is done using Dafny's forall statement.
lemma introHyp<X>(x: X)
ensures P(x)
lemma introGoal<H>()
ensures forall k: H :: P(k)
{
forall k: H
ensures P(k)
{
introHyp<H>(k);
}
}
In general, it looks like this:
forall x: X | R(x)
ensures P(x)
{
// for x of type X and satisfying R(x), prove P(x) here
// ...
}
So, inside the curly braces, you prove P(x) for one x. After the forall statement, you get to assume the universal quantifier
forall x: X :: R(x) ==> P(x)
If, like in my introGoal above, the body of the forall statement is exactly one lemma call and the postcondition of that lemma is what you what in the ensures clause of the forall statement, then you can omit the ensures clause of the forall statement and Dafny will infer it for you. Lemma introGoal then looks like this:
lemma introGoal<H>()
ensures forall k: H :: P(k)
{
forall k: H {
introHyp(k);
}
}
There's a Dafny Power User note on Automatic induction that may be helpful, or at least gives some additional examples.
PS. A natural next question would be how to do existential elimination. You do it using Dafny's "assign such that" statement. Here is an example:
type X
predicate P(x: X)
lemma ExistentialElimination() returns (y: X)
requires exists x :: P(x)
ensures P(y)
{
y :| P(y);
}
Some examples are found in this Dafny Power User note. Some advanced technical information about the :| operators are found in this paper.
I wanted to encode partial maps in Z3, with support for asking whether the map is defined for a certain key.
It should also support the operation (update_map m1 m2), which updates m1 with the mappings in m2 such that the mappings of m2 override those of m1.
I tried to encode it using arrays, and a custom option datatype, and I axiomatically specified the update_map function. However, it seems that Z3 can not even deal with my specification of update_map: It returns unknown on the following code:
(declare-datatypes (T) ((option None (Some (option-value T)))))
(define-sort map (K V) (Array K (option V)))
(declare-sort V)
(declare-sort K)
(define-const empty_map (map K V)
((as const (Array K (option V))) None))
(define-fun get ((m (map K V)) (k K)) (option V)
(select m k))
(define-fun put ((m (map K V)) (k K) (v V)) (map K V)
(store m k (Some v)))
(declare-fun update_map ((map K V) (map K V)) (map K V))
(assert (forall ((m1 (map K V)) (m2 (map K V)) (k K))
(=> (= (get m2 k) None)
(= (get (update_map m1 m2) k) (get m1 k)))))
(assert (forall ((m1 (map K V)) (m2 (map K V)) (k K) (v V))
(=> (= (get m2 k) (Some v))
(= (get (update_map m1 m2) k) (Some v)))))
(check-sat)
So I have two questions:
Is there a better way to encode maps which would enable me to prove facts about update_map?
Can anyone share some intuition on why Z3 doesn't understand this specification? As in, what's the "core feature" making this hard for Z3? [I already know that quantifiers are considered hard and that first-order logic is undecidable, I'm looking for something more specific ;-)]
Z3 fails to prove satisfiability, i.e. it fails to construct a model for your formula. I unfortunately don't know a more precise reason for this — might be a limitation of Z3's model finding abilities for arrays, quantifiers, or the combination therefore.
If you are ultimately not interested in finding models, but rather counterexamples (unsat cores), then add on an unsatisfiable formula and try to get unsat instead. I.e. try something such as the following (disclaimer: I didn't actually try it, but I am sure you get the idea):
(assert (not
(=
(get
(update_map
(put empty_map 2 -2)
(put empty_map 1 -1))
-1))))
As an alternative to axiomatising maps on top of arrays, have a look at Dafny's map axiomatisation. The axioms are provided in the Boogie language, but a translation to Z3 is typically straight-forward.
We know that, we can prove validity of a theorem by saying :
let Demorgan(x, y) = formula1(x,y) iff formula2(x,y)
assert ( forall (x,y) . Demorgan(x,y) )
Alternatively we can eliminate the forall quantifier by saying that :
let Demorgan(x, y) = formula1(x,y) iff formula2(x,y)
( assert (not Demorgan(x,y) ) )
So if it returns unsat, then we can say the above formula is valid.
Now I want to use this idea to eliminate the forall quantifier from the following assertion:
assert ( exists x1,x2,x3 st .( forall y . formula1(x1,y) iff
formula2(x2,y) iff
formula3(x3,y) ) )
So is there any way in Z3(using C++ API or SMT-LIB2.0) that I can assert something like the following :
assert (exists x1,x2,x3 st. ( and ((not ( formula1(x1,y) iff formula2(x2,y) )) == unsat)
((not ( formula2(x2,y) iff formula3(x3,y) )) == unsat)))
Yes, when we can prove the validity of a formula by showing its negation to be unsatisfiable.
For example, to show that Forall X. F(X) is valid, we just have to show that not (Forall X. F(X)) is unsatisfiable. The formula not (Forall X. F(X)) is equivalent to (Exists X. not F(X)). The formula (Exists X. not F(X)) is equisatisfiable to the formula not F(X) where the bound variable X is replaced by a fresh constant X. By equisatisfiable, I mean that the first one is satisfiable iff the second one is. This step that removes existential quantifiers is usually called skolemization.
Note that these last two formulas are not equivalent.
For example, consider the interpretation { X -> 2 } that assigns X to 2. The formula Exists X. not (X = 2) still evaluates to true in this interpretation because we can choose X to be 3. On the other hand, the formula not (X = 2) evaluates to false in this interpretation.
We usually use the term quantifier elimination procedure for a procedure that given a formula F produces an equivalent quantifier-free formula F'. So, skolemization is not considered a quantifier elimination procedure because the result is not an equivalent formula.
That being said, we don't have to apply the skolemization step by hand. Z3 can do it for us. Here is an example (also available online here).
(declare-sort S)
(declare-fun F (S) Bool)
(declare-fun G (S) Bool)
(define-fun Conjecture () Bool
(forall ((x S)) (= (and (F x) (G x)) (not (or (not (F x)) (not (G x)))))))
(assert (not Conjecture))
(check-sat)
Now, let us consider a formula of the form Exists X. Forall Y. F(X, Y). To prove the validity of this formula, we can show the negation not Exists X. Forall Y. F(X, Y) to be unsatisfiable. The negation is equivalent to Forall X. Exists Y. not F(X, Y). Now, if apply skolemization to this formula, we obtain Forall X. not F(X, Y(X)). In this case, the bound variable Y was replaced with Y(X), where Y is a fresh function symbol in the resultant formula. The intuition is that the function Y is the "choice function". For each X, we can choose a different value to satisfy the formula F. Z3 performs all these steps automatically for us. We don't need to apply skolemization by hand. However, in this case, the resultant formula is usually harder to solve because it contains a universal quantifier after the skolemization step.
Suppose we have two uninterpreted functions func1 and func2:
stuct_sort func1(struct_sort);
stuct_sort func2(struct_sort ,int).
And they have the relationship:
func2(p,n)=func1(p) if n==1
func2(p,n)=func1(func2(p,n-1)) if n>1
What I want to know is that if the following proposition :
((forall i:[1,m].func2(p,i)==Z)&&(q==func1(p))) implies (forall i:[1,m-1].func2(q,i)==Z)
can be proved to be true in Z3?
In my program, the prove result is Z3_L_UNDEF.
When I assign m with a value such as 3, the proposition now is
((forall i:[1,3].func2(p,i)==Z)&&(q==func1(p))) implies (forall i:[1,3-1].func2(q,i)==Z);
the result is Z3_L_UNDEF.
But when I rewrite the case separately(not using forall) as follows, the result is true.
(func2(p,1)==Z)&&(func2(p,2)==Z)&&(func2(p,3)==Z)&&(q==func1(p)) implies (func2(q,1))&&(func2(q,2)).
I can't find out the reason and looking forward to your answer
I encoded your problem using the Z3 Python interface, and Z3 solved it. It found a counterexample for the conjecture.
Of course, I may have made a mistake when I encoded the problem. The Python code is in the end of the post. We can try it online at rise4fun. BTW, which version of Z3 are you using? I'm assuming you are using the C API. If that is the case, could you provide the C code you used to create the Z3 formulas? Another possibility is to create a log that records the interaction of your application and Z3. To create a log file, we have to execute Z3_open_log("z3.log"); before you execute any other Z3 API. We can use the log file to replay all the interaction between your application and Z3.
from z3 import *
# Declare stuct_sort
S = DeclareSort('stuct_sort')
I = IntSort()
# Declare functions func1 and func2
func1 = Function('func1', S, S)
func2 = Function('func2', S, I, S)
# More declarations
p = Const('p', S)
n = Int('n')
m = Int('m')
i = Int('i')
q = Const('q', S)
Z = Const('Z', S)
# Encoding of the relations
# func2(p,n)=func1(p) if n==1
# func2(p,n)=func1(func2(p,n-1)) if n>1
Relations = And(func2(p, 1) == func1(p),
ForAll([n], Implies(n > 1, func2(p, n) == func1(func2(p, n - 1)))))
# Increase the maximum line width for the Z3 Python formula pretty printer
set_option(max_width=120)
print Relations
# Encoding of the conjecture
# ((forall i:[1,m].func2(p,i)==Z)&&(q==func1(p))) implies (forall i:[1,m-1].func2(q,i)==Z)
Conjecture = Implies(And(q == func1(p), ForAll([i], Implies(And(1 <= i, i <= m), func2(p, i) == Z))),
ForAll([i], Implies(And(1 <= i, i <= m - 1), func2(q, i) == Z)))
print Conjecture
prove(Implies(Relations, Conjecture))
I have a function calculate binomial expansion with optional parameters to specify the beginning and ending term:
(defun comb-index (s k)
(let ((combinations nil))
(labels ((rec (s k offset entry)
(cond ((equal s k)
(push (reverse (loop
for i from 1 to s
do (push (1- (+ i offset)) entry)
finally (return entry)))
combinations))
((equal k 0)
(push (reverse entry) combinations))
(t (rec (1- s) (1- k) (1+ offset) (cons offset entry))
(rec (1- s) k (1+ offset) entry)))))
(rec s k 0 nil))
(nreverse combinations)))
(defun binomial (k &key (start 1) end)
(let ((b start)
(e (if (null end) k end)))
(labels ((rec (i)
(cond ((equal i e)
(comb-index k e))
(t
(append (comb-index k i) (rec (1+ i)))))))
(rec b))
)
)
When I compile and run this code, it will yield the following run time error:
Unhandled memory fault at #x18.
[Condition of type SB-SYS:MEMORY-FAULT-ERROR]
This is caused by e, but I'm not sure why. I can avoid this problem by assigning 'e' with either 'k' or 'end', or simply using a (when ... ) to set 'end' to 'k' if it's nil, but I'm not sure why this doesn't work.
Looks like a memory overflow...
Ever thought about the memory efficiency of your code?
The problem in the code isn't obvious; it just seems to be a real overflow. Is there any way you can adjust the memory available to the underlying Lisp system? Command line flags or something?
Do you have the option of implementing this in a language like Clojure that can create lazy sequences? As you are probably aware, this type of calculation has the potential to create extremely large results. I seem to recall something in one of the Clojure/contrib libraries that did just this calculation.