I would like to express this problem in the SMTLib Format and evaluate it using Z3.
edge("som1","som3").
edge("som2","som4").
edge("som4","som1").
edge("som3","som4").
path(x,y) :- edge(x,y). % x and y are strings
path(x,z) :- edge(x,y), path(y,z).
:- path(x,y), path(y,x). %cyclic path.
My question is how to write the rule (or query) which detect the existence of a cycle in the relation path (this rule in basic datalog : :- path(x,y), path(y,x) ).
The tutorial Levent Erkok pointed out actually contains all the right information (I think). Knowing neither Datalog nor Z3's fixpoint features, I was still able to piece together the following:
(set-option :fixedpoint.engine datalog)
(define-sort s () Int)
(declare-rel edge (s s))
(declare-rel path (s s))
(declare-var a s)
(declare-var b s)
(declare-var c s)
(rule (=> (edge a b) (path a b)) P-1)
(rule (=> (and (path a b) (path b c)) (path a c)) P-2)
(rule (edge 1 2) E-1)
(rule (edge 2 3) E-2)
(rule (edge 3 1) E-3)
(declare-rel cycle (s))
(rule (=> (path a a) (cycle a)))
(query cycle :print-answer true)
Z3 4.8.0 nightly reports sat, indicating that there is a cycle, but unsat if any of the E-rules is removed.
I had to use ints instead of strings, though, since (my version of) Z3 aborts with the error Rule contains infinite sorts in rule P-1 if strings are used.
Related
The following Z3 code times out on the online repl:
; I want a function
(declare-fun f (Int) Int)
; I want it to be linear
(assert (forall ((a Int) (b Int)) (
= (+ (f a) (f b)) (f (+ a b))
)))
; I want f(2) == 4
(assert (= (f 2) 4))
; TIMEOUT :(
(check-sat)
So does this version, where it is looking for a function on the reals:
(declare-fun f (Real) Real)
(assert (forall ((a Real) (b Real)) (
= (+ (f a) (f b)) (f (+ a b))
)))
(assert (= (f 2) 4))
(check-sat)
It's faster when I give it a contradiction:
(declare-fun f (Real) Real)
(assert (forall ((a Real) (b Real)) (
= (+ (f a) (f b)) (f (+ a b))
)))
(assert (= (f 2) 4))
(assert (= (f 4) 7))
(check-sat)
I'm quite unknowledgeable about theorem provers. What is so slow here? Is the prover just having lots of trouble proving that linear functions with f(2) = 4 exist?
The slowness is most likely due to too many quantifier instantiations, caused by problematic patterns/triggers. If you don't know about these yet, have a look at the corresponding section of the Z3 guide.
Bottom line: patterns are a syntactic heuristic, indicating to the SMT solver when to instantiate the quantifier. Patterns must cover all quantified variables and interpreted functions such as addition (+) are not allowed in patterns. A matching loop is a situation in which every quantifier instantiation gives rise to further quantifier instantiations.
In your case, Z3 probably picks the pattern set :pattern ((f a) (f b)) (since you don't explicitly provide patterns). This suggests Z3 to instantiate the quantifier for every a, b for which the ground terms (f a) and (f b) have already occurred in the current proof search. Initially, the proof search contains (f 2); hence, the quantifier can be instantiated with a, b bound to 2, 2. This yields (f (+ 2 2)), which can be used to instantiate the quantifier once more (and also in combination with (f 2)). Z3 is thus stuck in a matching loop.
Here is a snippet arguing my point:
(set-option :smt.qi.profile true)
(declare-fun f (Int) Int)
(declare-fun T (Int Int) Bool) ; A dummy trigger function
(assert (forall ((a Int) (b Int)) (!
(= (+ (f a) (f b)) (f (+ a b)))
:pattern ((f a) (f b))
; :pattern ((T a b))
)))
(assert (= (f 2) 4))
(set-option :timeout 5000) ; 5s is enough
(check-sat)
(get-info :reason-unknown)
(get-info :all-statistics)
With the explicitly provided pattern you'll get your original behaviour (modulo the specified timeout). Moreover, the statistics report lots of instantiations of the quantifier (and more still if you increase the timeout).
If you comment the first pattern and uncomment the second, i.e. if you "guard" the quantifier with a dummy trigger that won't show up in the proof search, then Z3 terminates immediately. Z3 will still report unknown, though, because it "knowns" that it did not account for the quantified constraint (which would be a requirement for sat; and it also cannot show unsat).
It is sometimes possible to rewrite quantifiers in order to have better triggering behaviour. The Z3 guide, for example, illustrates that in the context of injective functions/inverse functions. Maybe you'll be able to perform a similar transformation here.
I'm using muZ engine of Z3. For all SAT cases I would like to see which rules where used in the derivation. Is there any way to extract this information?
For instance the input might look like this:
(declare-rel R1 (Int))
(declare-rel R2 (Int))
(declare-rel q (Int))
(declare-var n Int)
(rule (R1 n) rule_one)
(rule (=> (R1 n) (R2 n)) rule_two)
(rule (=> (and (R2 n) (< n 1)) (q n)) query)
(query q
:print-answer true
)
And I'd be glad to know which rules were triggered, something like
q is SAT, used rules: rule_one->rule_two->query
One can use the following option
(set-option :fixedpoint.generate_proof_trace true)
my question is that i do not see that the unsat_core tracks any assertions that are provided in an entire chunk using the api
f = Z3_parse_smtlib2_string(c, "unsat_core_example1.smt2",0,0,0,0,0,0);
params p(c);
p.set(":unsat-core", true);
s.set(p);
// enabling unsat core tracking
expr r = to_expr(c, f);
unsat_core_example1.smt2:
(declare-fun p () Bool)
(declare-fun q () Bool)
(declare-fun r () Bool)
(declare-fun s () Bool)
(assert (! (or p q) :named a1))
(assert (! (implies r s) :named a2))
(assert (! (implies s (iff q r)) :named a3))
(assert (! (or r p) :named a4))
(assert (! (or r s) :named a5))
(assert (! (not (and r q)) :named a6))
(assert (! (not (and s p)) :named a7))
it seems like the annotation :named is not being processed, as the unsat_core vector returned is always empty.
However, this is not the case if i were to use z3.exe and input the file in.
Any idea what could be the cause?
The Z3_parse_smtlib2_file function does not support all of SMTLIB2, it's really just a convenience function; it's output is not guaranteed to completely cover all of the language (e.g., it doesn't execute commands like check-sat or some set-option commands). It was also written as an extension of a previous SMTLIB1 parser which was written long before the goal/tactic/solver architecture was introduced and consequently not all information is carried over to this new architecture.
In this particular case, the assertion names are indeed saved inside the context, but Z3_parse_smtlib2_file does not return a set of assertions and names; it returns a single expression which is unnamed. To accurately represent an SMTLIB2 benchmark the signature of the function would have to change significantly.
In the example given, the expression which is asserted into the solver is r, i.e., we would have something like
s.add(r);
this essentially asks the solver the assert the unnamed assertion r, but there is no support for "subnames" for expressions inside of r. We can still name r itself however, e.g., by calling
s.add(r, "top")
which results in the correct unsat core.
Is there a way to concat two lists in z3? Similar to the # operator in ML? I was thinking of defining it myself but I don't think z3 supports recursive function definitions, i.e.,
define-fun concat ( (List l1) (List l2) List
(ite (isNil l1) (l2) (concat (tail l1) (insert (head l1) l2)) )
)
2021 Update
Below answer was written in 2012; 9 years ago. It largely remains still correct; except SMTLib now explicitly allows for recursive-function definitions, via define-fun-rec construct. However, solver support is still very weak, and most properties of interest regarding such functions can still not be proven out-of-the-box. Bottom line remains that such recursive definitions lead to inductive proofs, and SMT-solvers are simply not equipped to do induction. Perhaps in another 9 years they will be able to do so, presumably allowing users to specify their own invariants. For the time being, theorem-provers such as Isabelle, Coq, ACL2, HOL, Lean, etc., remain the best tools to handle these sorts of problems.
Answer from 2012
You are correct that SMT-Lib2 does not allow recursive function definitions. (In SMT-Lib2, function definitions are more like macros, they are good for abbreviations.)
The usual trick is to declare such symbols as uninterpreted functions, and then assert the defining equations as quantified axioms. Of course, as soon as quantifiers come into play the solver can start returning unknown or timeout for "difficult" queries. However, Z3 is pretty good at many goals arising from typical software verification tasks, so it should be able to prove many properties of interest.
Here's an example illustrating how you can define len and append over lists, and then prove some theorems about them. Note that if a proof requires induction, then Z3 is likely to time-out (as in the second example below), but future versions of Z3 might be able to handle inductive proofs as well.
Here's the permalink for this example on the Z3 web-site if you'd like to play around: http://rise4fun.com/Z3/RYmx
; declare len as an uninterpreted function
(declare-fun len ((List Int)) Int)
; assert defining equations for len as an axiom
(assert (forall ((xs (List Int)))
(ite (= nil xs)
(= 0 (len xs))
(= (+ 1 (len (tail xs))) (len xs)))))
; declare append as an uninterpreted function
(declare-fun append ((List Int) (List Int)) (List Int))
; assert defining equations for append as an axiom
(assert (forall ((xs (List Int)) (ys (List Int)))
(ite (= nil xs)
(= (append xs ys) ys)
(= (append xs ys) (insert (head xs) (append (tail xs) ys))))))
; declare some existential constants
(declare-fun x () Int)
(declare-fun xs () (List Int))
(declare-fun ys () (List Int))
; prove len (insert x xs) = 1 + len xs
; note that we assert the negation, so unsat means the theorem is valid
(push)
(assert (not (= (+ 1 (len xs)) (len (insert x xs)))))
(check-sat)
(pop)
; prove (len (append xs ys)) = len xs + len ys
; note that Z3 will time out since this proof requires induction
; future versions might very well be able to deal with it..
(push)
(assert (not (= (len (append xs ys)) (+ (len xs) (len ys)))))
(check-sat)
(pop)
While Levent's code works, if you're willing to set a bound on the recursion depth, Z3 normally has much less trouble with your assertions. You don't even need to rely on MBQI, which often takes far too much time to be practical. Conceptually, you'll want to do:
; the macro finder can figure out when universal declarations are macros
(set-option :macro-finder true)
(declare-fun len0 ((List Int)) Int)
(assert (forall ((xs (List Int))) (= (len0 xs) 0)))
(declare-fun len1 ((List Int)) Int)
(assert (forall ((xs (List Int))) (ite (= xs nil)
0
(+ 1 (len0 (tail xs))))))
(declare-fun len2 ((List Int)) Int)
(assert (forall ((xs (List Int))) (ite (= xs nil)
0
(+ 1 (len1 (tail xs))))))
... and so on. Writing all of this down manually will probably be a pain, so I'd recommend using a programmatic API. (Shameless plug: I've been working on Racket bindings and here's how you'd do it there.)
how can I make a datatype that contains a set of another objects. Basically, I am doing the following code:
(define-sort Set(T) (Array Int T))
(declare-datatypes () ((A f1 (cons (value Int) (b (Set B))))
(B f2 (cons (id Int) (a (Set A))))
))
But Z3 tells me unknown sort for A and B. If I remove "Set" it works just as the guide states.
I was trying to use List instead but it does not work. Anyone knows how to make it work?
You are addressing a question that comes up on a regular basis:
how can I mix data-types and arrays (as sets, multi-sets or
data-types in the range)?
As stated above Z3 does not support mixing data-types
and arrays in a single declaration.
A solution is to develop a custom solver for the
mixed datatype + array theory. Z3 contains programmatic
APIs for developing custom solvers.
It is still useful to develop this example
to illustrate the capabilities and limitations
of encoding theories with quantifiers and triggers.
Let me simplify your example by just using A.
As a work-around you can define an auxiliary sort.
The workaround is not ideal, though. It illustrates some
axiom 'hacking'. It relies on the operational semantics
of how quantifiers are instantiated during search.
(set-option :model true) ; We are going to display models.
(set-option :auto-config false)
(set-option :mbqi false) ; Model-based quantifier instantiation is too powerful here
(declare-sort SetA) ; Declare a custom fresh sort SetA
(declare-datatypes () ((A f1 (cons (value Int) (a SetA)))))
(define-sort Set (T) (Array T Bool))
Then define bijections between (Set A), SetA.
(declare-fun injSA ((Set A)) SetA)
(declare-fun projSA (SetA) (Set A))
(assert (forall ((x SetA)) (= (injSA (projSA x)) x)))
(assert (forall ((x (Set A))) (= (projSA (injSA x)) x)))
This is almost what the data-type declaration states.
To enforce well-foundedness you can associate an ordinal with members of A
and enforce that members of SetA are smaller in the well-founded ordering:
(declare-const v Int)
(declare-const s1 SetA)
(declare-const a1 A)
(declare-const sa1 (Set A))
(declare-const s2 SetA)
(declare-const a2 A)
(declare-const sa2 (Set A))
With the axioms so far, a1 can be a member of itself.
(push)
(assert (select sa1 a1))
(assert (= s1 (injSA sa1)))
(assert (= a1 (cons v s1)))
(check-sat)
(get-model)
(pop)
We now associate an ordinal number with the members of A.
(declare-fun ord (A) Int)
(assert (forall ((x SetA) (v Int) (a A))
(=> (select (projSA x) a)
(> (ord (cons v x)) (ord a)))))
(assert (forall ((x A)) (> (ord x) 0)))
By default quantifier instantiation in Z3 is pattern-based.
The first quantified assert above will not be instantiated on all
relevant instances. One can instead assert:
(assert (forall ((x1 SetA) (x2 (Set A)) (v Int) (a A))
(! (=> (and (= (projSA x1) x2) (select x2 a))
(> (ord (cons v x1)) (ord a)))
:pattern ((select x2 a) (cons v x1)))))
Axioms like these, that use two patterns (called a multi-pattern)
are quite expensive. They produce instantiations for every pair
of (select x2 a) and (cons v x1)
The membership constraint from before is now unsatisfiable.
(push)
(assert (select sa1 a1))
(assert (= s1 (injSA sa1)))
(assert (= a1 (cons v s1)))
(check-sat)
(pop)
but models are not necessarily well formed yet.
the default value of the set is 'true', which
would mean that the model implies there is a membership cycle
when there isn't one.
(push)
(assert (not (= (cons v s1) a1)))
(assert (= (projSA s1) sa1))
(assert (select sa1 a1))
(check-sat)
(get-model)
(pop)
We can approximate more faithful models by using
the following approach to enforce that sets that are
used in data-types are finite.
For example, whenever there is a membership check on a set x2,
we enforce that the 'default' value of the set is 'false'.
(assert (forall ((x2 (Set A)) (a A))
(! (not (default x2))
:pattern ((select x2 a)))))
Alternatively, whenever a set occurs in a data-type constructor
it is finite
(assert (forall ((v Int) (x1 SetA))
(! (not (default (projSA x1)))
:pattern ((cons v x1)))))
(push)
(assert (not (= (cons v s1) a1)))
(assert (= (projSA s1) sa1))
(assert (select sa1 a1))
(check-sat)
(get-model)
(pop)
Throughout the inclusion of additional axioms,
Z3 produces the answer 'unknown' and furthermore
the model that is produced indicates that the domain SetA
is finite (a singleton). So while we could patch the defaults
this model still does not satisfy the axioms. It satisfies
the axioms modulo instantiation only.
This is not supported in Z3. You can use arrays in datatype declarations, but they can't contain "references" to the datatypes you are declaring. For example, it is ok to use (Set Int).