I wanted to encode partial maps in Z3, with support for asking whether the map is defined for a certain key.
It should also support the operation (update_map m1 m2), which updates m1 with the mappings in m2 such that the mappings of m2 override those of m1.
I tried to encode it using arrays, and a custom option datatype, and I axiomatically specified the update_map function. However, it seems that Z3 can not even deal with my specification of update_map: It returns unknown on the following code:
(declare-datatypes (T) ((option None (Some (option-value T)))))
(define-sort map (K V) (Array K (option V)))
(declare-sort V)
(declare-sort K)
(define-const empty_map (map K V)
((as const (Array K (option V))) None))
(define-fun get ((m (map K V)) (k K)) (option V)
(select m k))
(define-fun put ((m (map K V)) (k K) (v V)) (map K V)
(store m k (Some v)))
(declare-fun update_map ((map K V) (map K V)) (map K V))
(assert (forall ((m1 (map K V)) (m2 (map K V)) (k K))
(=> (= (get m2 k) None)
(= (get (update_map m1 m2) k) (get m1 k)))))
(assert (forall ((m1 (map K V)) (m2 (map K V)) (k K) (v V))
(=> (= (get m2 k) (Some v))
(= (get (update_map m1 m2) k) (Some v)))))
(check-sat)
So I have two questions:
Is there a better way to encode maps which would enable me to prove facts about update_map?
Can anyone share some intuition on why Z3 doesn't understand this specification? As in, what's the "core feature" making this hard for Z3? [I already know that quantifiers are considered hard and that first-order logic is undecidable, I'm looking for something more specific ;-)]
Z3 fails to prove satisfiability, i.e. it fails to construct a model for your formula. I unfortunately don't know a more precise reason for this — might be a limitation of Z3's model finding abilities for arrays, quantifiers, or the combination therefore.
If you are ultimately not interested in finding models, but rather counterexamples (unsat cores), then add on an unsatisfiable formula and try to get unsat instead. I.e. try something such as the following (disclaimer: I didn't actually try it, but I am sure you get the idea):
(assert (not
(=
(get
(update_map
(put empty_map 2 -2)
(put empty_map 1 -1))
-1))))
As an alternative to axiomatising maps on top of arrays, have a look at Dafny's map axiomatisation. The axioms are provided in the Boogie language, but a translation to Z3 is typically straight-forward.
Related
I'm trying to create a function in Z3 that is transitive but not reflexive. I.e. if (transitive a b) and (transitive b c)hold then (transitive a c) should hold, but (transitive a a) should not.
I've tried to do it the following way, with 5 "tests". The first does what I expect, but the second one fails and results in unknown.
(declare-datatypes () ((T T1 T2 T3)))
(declare-fun f (T T) Bool)
(assert(f T1 T2))
(assert(f T2 T3))
; Make sure that f is not reflexive
(assert
(forall ((x T))
(not (f x x))))
; Now we create the transitivity function ourselves
(define-fun-rec transitive ((x T) (y T)) Bool
(or
(f x y)
(exists ((z T))
(and
(f x z)
(transitive z y)))))
; This works and gives sat
(push)
(assert (not (transitive T1 T1)))
(assert (not (transitive T2 T2)))
(assert (not (transitive T3 T3)))
(check-sat)
(pop)
; This fails with "unknown" and the verbose flag gives: (smt.mbqi "max instantiations 1000 reached")
(push)
(assert
(forall ((x T))
(not (transitive x x))))
(check-sat)
(pop)
My question is: how does the second test differ from the first? Why does the last one give unknown, whereas the one before that works just fine?
The "verbose" message is a hint here. mbqi stands for model-based-quantifier-instantiation. It's a method of dealing with quantifiers in SMT solving. In the first case, MBQI manages to find a model. But your transitive function is just too complicated for MBQI to handle, and thus it gives up. Increasing the limit will not likely address the problem, nor it's a long term solution.
Short story long, recursive definitions are difficult to deal with, and recursive definitions with quantifiers are even harder. The logic becomes semi-decidable, and you're at the mercy of heuristics. Even if you found a way to make z3 compute a model for this, it would be brittle. These sorts of problems are just not suitable for SMT solving; better use a proper theorem prover like Isabelle, Hol, Coq, Lean. Agda, etc. Almost all these tools offer "tactics" to dispatch subgoals to SMT solvers, so you have the best of both worlds. (Of course you lose full automation, but with quantifiers present, you can't expect any better.)
The following Z3 code times out on the online repl:
; I want a function
(declare-fun f (Int) Int)
; I want it to be linear
(assert (forall ((a Int) (b Int)) (
= (+ (f a) (f b)) (f (+ a b))
)))
; I want f(2) == 4
(assert (= (f 2) 4))
; TIMEOUT :(
(check-sat)
So does this version, where it is looking for a function on the reals:
(declare-fun f (Real) Real)
(assert (forall ((a Real) (b Real)) (
= (+ (f a) (f b)) (f (+ a b))
)))
(assert (= (f 2) 4))
(check-sat)
It's faster when I give it a contradiction:
(declare-fun f (Real) Real)
(assert (forall ((a Real) (b Real)) (
= (+ (f a) (f b)) (f (+ a b))
)))
(assert (= (f 2) 4))
(assert (= (f 4) 7))
(check-sat)
I'm quite unknowledgeable about theorem provers. What is so slow here? Is the prover just having lots of trouble proving that linear functions with f(2) = 4 exist?
The slowness is most likely due to too many quantifier instantiations, caused by problematic patterns/triggers. If you don't know about these yet, have a look at the corresponding section of the Z3 guide.
Bottom line: patterns are a syntactic heuristic, indicating to the SMT solver when to instantiate the quantifier. Patterns must cover all quantified variables and interpreted functions such as addition (+) are not allowed in patterns. A matching loop is a situation in which every quantifier instantiation gives rise to further quantifier instantiations.
In your case, Z3 probably picks the pattern set :pattern ((f a) (f b)) (since you don't explicitly provide patterns). This suggests Z3 to instantiate the quantifier for every a, b for which the ground terms (f a) and (f b) have already occurred in the current proof search. Initially, the proof search contains (f 2); hence, the quantifier can be instantiated with a, b bound to 2, 2. This yields (f (+ 2 2)), which can be used to instantiate the quantifier once more (and also in combination with (f 2)). Z3 is thus stuck in a matching loop.
Here is a snippet arguing my point:
(set-option :smt.qi.profile true)
(declare-fun f (Int) Int)
(declare-fun T (Int Int) Bool) ; A dummy trigger function
(assert (forall ((a Int) (b Int)) (!
(= (+ (f a) (f b)) (f (+ a b)))
:pattern ((f a) (f b))
; :pattern ((T a b))
)))
(assert (= (f 2) 4))
(set-option :timeout 5000) ; 5s is enough
(check-sat)
(get-info :reason-unknown)
(get-info :all-statistics)
With the explicitly provided pattern you'll get your original behaviour (modulo the specified timeout). Moreover, the statistics report lots of instantiations of the quantifier (and more still if you increase the timeout).
If you comment the first pattern and uncomment the second, i.e. if you "guard" the quantifier with a dummy trigger that won't show up in the proof search, then Z3 terminates immediately. Z3 will still report unknown, though, because it "knowns" that it did not account for the quantified constraint (which would be a requirement for sat; and it also cannot show unsat).
It is sometimes possible to rewrite quantifiers in order to have better triggering behaviour. The Z3 guide, for example, illustrates that in the context of injective functions/inverse functions. Maybe you'll be able to perform a similar transformation here.
I'm using muZ engine of Z3. For all SAT cases I would like to see which rules where used in the derivation. Is there any way to extract this information?
For instance the input might look like this:
(declare-rel R1 (Int))
(declare-rel R2 (Int))
(declare-rel q (Int))
(declare-var n Int)
(rule (R1 n) rule_one)
(rule (=> (R1 n) (R2 n)) rule_two)
(rule (=> (and (R2 n) (< n 1)) (q n)) query)
(query q
:print-answer true
)
And I'd be glad to know which rules were triggered, something like
q is SAT, used rules: rule_one->rule_two->query
One can use the following option
(set-option :fixedpoint.generate_proof_trace true)
Does z3 provide a cross product function for two lists? If not is it possible to define one without using higher order functions or using the provided list functions? I have been having trouble trying to define one. I know how to define one using map but I don't think that is supported in z3.
You can declare the cross product function in SMT 2.0.
However, any non-trivial property will require a proof by induction. Z3 currently does not support proofs by induction. Thus, it will only be able to prove very simple facts.
BTW, by cross-product of lists, I'm assuming you want a function that given the lists [a, b] and [c, d] returns the list or pairs [(a, c), (a, d), (b, c), (b, d)].
Here is a script that defines the product function.
The script also demonstrates some limitations of the SMT 2.0 language. For example, SMT 2.0 does not support the definition of parametric axioms or functions. So, I used uninterpreted sorts to "simulate" that. I also had to define the auxiliary functions append and product-aux. You can try this example online at: http://rise4fun.com/Z3/QahiP
The example also proves the following trivial fact that if l = product([a], [b]), then first(head(l)) must be a.
If you are insterested in proving non-trivial properties. I see two options. We can try to prove the base case and inductive cases using Z3. The main disadvantage in this approach is that we have to manually create these cases, and mistakes can be made. Another option is to use an interactive theorem prover such as Isabelle. BTW, Isabelle has a much richer input language, and provides tactics for invoking Z3.
For more information about algebraic datatypes in Z3, go to the online tutorial http://rise4fun.com/Z3/tutorial/guide (Section Datatypes).
;; List is a builtin datatype in Z3
;; It has the constructors insert and nil
;; Declaring Pair type using algebraic datatypes
(declare-datatypes (T1 T2) ((Pair (mk-pair (first T1) (second T2)))))
;; SMT 2.0 does not support parametric function definitions.
;; So, I'm using two uninterpreted sorts.
(declare-sort T1)
(declare-sort T2)
;; Remark: We can "instantiate" these sorts to interpreted sorts (Int, Real) by replacing the declarations above
;; with the definitions
;; (define-sort T1 () Int)
;; (define-sort T2 () Real)
(declare-fun append ((List (Pair T1 T2)) (List (Pair T1 T2))) (List (Pair T1 T2)))
;; Remark: I'm using (as nil (Pair T1 T2)) because nil is overloaded. So, I must tell which one I want.
(assert (forall ((l (List (Pair T1 T2))))
(= (append (as nil (List (Pair T1 T2))) l) l)))
(assert (forall ((h (Pair T1 T2)) (t (List (Pair T1 T2))) (l (List (Pair T1 T2))))
(= (append (insert h t) l) (insert h (append t l)))))
;; Auxiliary definition
;; Given [a, b, c], d returns [(a, d), (b, d), (c, d)]
(declare-fun product-aux ((List T1) T2) (List (Pair T1 T2)))
(assert (forall ((v T2))
(= (product-aux (as nil (List T1)) v)
(as nil (List (Pair T1 T2))))))
(assert (forall ((h T1) (t (List T1)) (v T2))
(= (product-aux (insert h t) v)
(insert (mk-pair h v) (product-aux t v)))))
(declare-fun product ((List T1) (List T2)) (List (Pair T1 T2)))
(assert (forall ((l (List T1)))
(= (product l (as nil (List T2))) (as nil (List (Pair T1 T2))))))
(assert (forall ((l (List T1)) (h T2) (t (List T2)))
(= (product l (insert h t))
(append (product-aux l h) (product l t)))))
(declare-const a T1)
(declare-const b T2)
(declare-const l (List (Pair T1 T2)))
(assert (= (product (insert a (as nil (List T1))) (insert b (as nil (List T2))))
l))
(assert (not (= (first (head l)) a)))
(check-sat)
There isn't an #include directive for the smt-lib format.
Z3 provides several other ways to provide input.
The Python input format leverages all of Python, so importing files is naturally supported.
There is a tutorial on Z3Py on http://rise4fun.com/z3py
Is there a way to concat two lists in z3? Similar to the # operator in ML? I was thinking of defining it myself but I don't think z3 supports recursive function definitions, i.e.,
define-fun concat ( (List l1) (List l2) List
(ite (isNil l1) (l2) (concat (tail l1) (insert (head l1) l2)) )
)
2021 Update
Below answer was written in 2012; 9 years ago. It largely remains still correct; except SMTLib now explicitly allows for recursive-function definitions, via define-fun-rec construct. However, solver support is still very weak, and most properties of interest regarding such functions can still not be proven out-of-the-box. Bottom line remains that such recursive definitions lead to inductive proofs, and SMT-solvers are simply not equipped to do induction. Perhaps in another 9 years they will be able to do so, presumably allowing users to specify their own invariants. For the time being, theorem-provers such as Isabelle, Coq, ACL2, HOL, Lean, etc., remain the best tools to handle these sorts of problems.
Answer from 2012
You are correct that SMT-Lib2 does not allow recursive function definitions. (In SMT-Lib2, function definitions are more like macros, they are good for abbreviations.)
The usual trick is to declare such symbols as uninterpreted functions, and then assert the defining equations as quantified axioms. Of course, as soon as quantifiers come into play the solver can start returning unknown or timeout for "difficult" queries. However, Z3 is pretty good at many goals arising from typical software verification tasks, so it should be able to prove many properties of interest.
Here's an example illustrating how you can define len and append over lists, and then prove some theorems about them. Note that if a proof requires induction, then Z3 is likely to time-out (as in the second example below), but future versions of Z3 might be able to handle inductive proofs as well.
Here's the permalink for this example on the Z3 web-site if you'd like to play around: http://rise4fun.com/Z3/RYmx
; declare len as an uninterpreted function
(declare-fun len ((List Int)) Int)
; assert defining equations for len as an axiom
(assert (forall ((xs (List Int)))
(ite (= nil xs)
(= 0 (len xs))
(= (+ 1 (len (tail xs))) (len xs)))))
; declare append as an uninterpreted function
(declare-fun append ((List Int) (List Int)) (List Int))
; assert defining equations for append as an axiom
(assert (forall ((xs (List Int)) (ys (List Int)))
(ite (= nil xs)
(= (append xs ys) ys)
(= (append xs ys) (insert (head xs) (append (tail xs) ys))))))
; declare some existential constants
(declare-fun x () Int)
(declare-fun xs () (List Int))
(declare-fun ys () (List Int))
; prove len (insert x xs) = 1 + len xs
; note that we assert the negation, so unsat means the theorem is valid
(push)
(assert (not (= (+ 1 (len xs)) (len (insert x xs)))))
(check-sat)
(pop)
; prove (len (append xs ys)) = len xs + len ys
; note that Z3 will time out since this proof requires induction
; future versions might very well be able to deal with it..
(push)
(assert (not (= (len (append xs ys)) (+ (len xs) (len ys)))))
(check-sat)
(pop)
While Levent's code works, if you're willing to set a bound on the recursion depth, Z3 normally has much less trouble with your assertions. You don't even need to rely on MBQI, which often takes far too much time to be practical. Conceptually, you'll want to do:
; the macro finder can figure out when universal declarations are macros
(set-option :macro-finder true)
(declare-fun len0 ((List Int)) Int)
(assert (forall ((xs (List Int))) (= (len0 xs) 0)))
(declare-fun len1 ((List Int)) Int)
(assert (forall ((xs (List Int))) (ite (= xs nil)
0
(+ 1 (len0 (tail xs))))))
(declare-fun len2 ((List Int)) Int)
(assert (forall ((xs (List Int))) (ite (= xs nil)
0
(+ 1 (len1 (tail xs))))))
... and so on. Writing all of this down manually will probably be a pain, so I'd recommend using a programmatic API. (Shameless plug: I've been working on Racket bindings and here's how you'd do it there.)