I'm curious what the limitations on z3's map operator are. According to the z3 tutorial (http://rise4fun.com/z3/tutorial), "Z3 provides a parametrized map function on arrays. It allows applying arbitrary functions to the range of arrays."
Map appears to work when I'm mapping built-in functions or functions declared with (declare-fun ...) syntax. When I attempt to use map with function (really macros) defined with (define-fun ...) syntax, I receive the error invalid function declaration reference, named expressions (aka macros) cannot be referenced.
Is there a standard way to map user-defined functions over arrays?
Here is some code that illustrates my confusion:
;simple function, equivalent to or
(define-fun my-or ((x Bool) (y Bool)) Bool (or x y))
(assert (forall ((x Bool) (y Bool)) (= (my-or x y) (or x y))))
(check-sat)
;mapping or with map works just fine
(define-sort Set () (Array Int Bool))
(declare-const a Set)
(assert ( = a ((_ map or) a a) ))
(check-sat)
;but this fails with error
(assert ( = a ((_ map my-or) a a) ))
I'm currently hacking around the problem like this:
(define-fun my-or-impl ((x Bool) (y Bool)) Bool (or x y))
(declare-fun my-or (Bool Bool) Bool)
(assert (forall ((x Bool) (y Bool)) (= (my-or x y) (my-or-impl x y))))
(check-sat)
But I'm hoping that there's a way to solve this which doesn't involve universal quantifiers.
Unfortunately, define-fun is just a macro definition in Z3. They are implemented in the Z3 SMT 2.0 parser. They are not part of the Z3 kernel. That is, Z3 solvers do not even "see" these definitions.
The approach using declare-fun and quantifiers works, but as you said we should avoid quantifiers since they create performance problems, and it is really easy to create problems with quantifiers that Z3 can't solve.
The best option is to use (_ map or).
The best option is to use (_ map or).
Unless one wants to map a non-built-in function over an array... I guess declare-fun plus assert is the only way to go?
Related
I have switched from using Int to Bit Vectors in SMT. However, the logic QF_BV does not allow the use of any quantifiers in your script, and I need to define FOL rules.
I know how to eliminate existential quantifiers, but universal quantifiers? How to do that?
Imagine a code like that:
(set-logic QF_AUFBV)
(define-sort Index () (_ BitVec 3))
(declare-fun P (Index) Bool)
(assert (forall ((i Index)) (= (P (bvadd i #b001)) (not (P i)) ) ) )
Strictly speaking, you're out-of-luck. According to http://smtlib.cs.uiowa.edu/logics.shtml, there's no logic that contains quantifiers and bit-vectors at the same time.
Having said that, most solvers will allow non-standard combinations. Simply leave out the set-logic command, and you might get lucky. For instance, Z3 takes your query just fine without the set-logic part; I just tried..
In Non-linear arithmetic and uninterpreted functions, Leonardo de Moura states that the qfnra-nlsat tactic hasn't been fully integrated with the rest of Z3 yet. I thought that the situation has changed in two years, but apparently the integration is still not very complete.
In the example below, I use datatypes purely for "software engineering" purposes: to organize my data into records. Even though there are no uninterpreted functions, Z3 still fails to give me a solution:
(declare-datatypes () (
(Point (point (point-x Real) (point-y Real)))
(Line (line (line-a Real) (line-b Real) (line-c Real)))))
(define-fun point-line-subst ((p Point) (l Line)) Real
(+ (* (line-a l) (point-x p)) (* (line-b l) (point-y p)) (line-c l)))
(declare-const p Point)
(declare-const l Line)
(assert (> (point-y p) 20.0))
(assert (= 0.0 (point-line-subst p l)))
(check-sat-using qfnra-nlsat)
(get-model)
> unknown
(model
)
However, if I manually inline all the functions, Z3 finds a model instantly:
(declare-const x Real)
(declare-const y Real)
(declare-const a Real)
(declare-const b Real)
(declare-const c Real)
(assert (> y 20.0))
(assert (= 0.0 (+ (* a x) (* b y) c)))
(check-sat-using qfnra-nlsat)
(get-model)
> sat
(model
(define-fun y () Real
21.0)
(define-fun a () Real
0.0)
(define-fun x () Real
0.0)
(define-fun b () Real
0.0)
(define-fun c () Real
0.0)
)
My question is, is there a way to perform such an inlining automatically? I'm fine with either one of these workflows:
Launch Z3 with a tactic that says "Inline first, then apply qfnra-nlsat. I haven't found a way to do so, but maybe I wasn't looking well enough.
Launch Z3 using some version of simplify to do the inlining. Launch Z3 the second time on the result of the first invocation (the inlined version).
In other words, how to make qfnra-nlsat work with tuples?
Thank you!
That's correct, the NLSAT solver is still not integrated with the other theories. At the moment, we can only use it if we eliminate all datatypes (or elements of other theories) before running it. I believe there is no useful existing tactic inside of Z3 at the moment though, so this would have to be done beforehand. In general it's not hard to compose tactics, e.g., like this:
(check-sat-using (and-then simplify qfnra-nlsat))
but the simplifier is not strong enough to eliminate the datatype constants in this problem. (The respective implementation files are datatype_rewriter.cpp and datatype_simplifier_plugin.cpp.)
I noticed some strange behavior with Z3 4.3.1 when working with .smt2 files.
If I do (assert (= 0 0.5)) it will be satisfiable. However, if I switch the order and do (assert (= 0.5 0)) it's not satisfiable.
My guess as to what is happening is that if the first parameter is an integer, it casts both of them to integers (rounding 0.5 down to 0), then does the comparison. If I change "0" to "0.0" it works as expected. This is in contrast to most programming languages I've worked with where if either of the parameters is a floating-point number, they are both cast to floating-point numbers and compared. Is this really the expected behavior in Z3?
I think this is a consequence of lack of type-checking; z3 is being too lenient. It should simply reject such queries as they are simply not well formed.
According to the SMT-Lib standard, v2 (http://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.0-r10.12.21.pdf); page 30; the core theory is defined thusly:
(theory Core
:sorts ((Bool 0))
:funs ((true Bool) (false Bool) (not Bool Bool)
(=> Bool Bool Bool :right-assoc) (and Bool Bool Bool :left-assoc)
(or Bool Bool Bool :left-assoc) (xor Bool Bool Bool :left-assoc)
(par (A) (= A A Bool :chainable))
(par (A) (distinct A A Bool :pairwise))
(par (A) (ite Bool A A A))
)
:definition
"For every expanded signature Sigma, the instance of Core with that signature
is the theory consisting of all Sigma-models in which:
- the sort Bool denotes the set {true, false} of Boolean values;
- for all sorts s in Sigma,
- (= s s Bool) denotes the function that
returns true iff its two arguments are identical;
- (distinct s s Bool) denotes the function that
returns true iff its two arguments are not identical;
- (ite Bool s s) denotes the function that
returns its second argument or its third depending on whether
its first argument is true or not;
- the other function symbols of Core denote the standard Boolean operators
as expected.
"
:values "The set of values for the sort Bool is {true, false}."
)
So, by definition equality requires the input sorts to be the same; and hence the aforementioned query should be rejected as invalid.
There might be a switch to z3 or some other setting that forces more strict type-checking than it does by default; but I would've expected this case to be caught even with the most relaxed of the implementations.
Do not rely on the implicit type conversion of any solver. Instead,
use to_real and to_int to do explicit type conversions. Only send
well-typed formulas to the solver. Then Mohamed Iguernelala's examples become the following.
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= (to_real x) 1.5))
(check-sat)
(exit)
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= 1.5 (to_real x)))
(check-sat)
(exit)
Both of these return UNSAT in Z3 and CVC4. If instead, you really
wanted to find the model where x = 1 you should have instead used one
of the following.
(set-option :produce-models true)
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= (to_int 1.5) x))
(check-sat)
(get-model)
(exit)
(set-option :produce-models true)
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= x (to_int 1.5)))
(check-sat)
(get-model)
(exit)
Both of these return SAT with x = 1 in Z3 and CVC4.
Once you make all the type conversions explicit and deal only in well-typed formulas, the order of arguments to equality no longer matters (for correctness).
One of our interns, who worked on a conservative extension of SMT2 with polymorphism has noticed the same strange behavior, when he tried the understand how formulas mixing integers and reals are type-checked:
z3 (http://rise4fun.com/z3) says that the following example is SAT, and finds a model x = 1
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= x 1.5))
(check-sat)
(get-model)
(exit)
But, it says that the following "equivalent" example in UNSAT
(set-logic AUFLIRA)
(declare-fun x () Int)
(assert (= 1.5 x))
(check-sat)
(exit)
So, this does not comply with the symmetric property of equality predicate. So, I think it's a bug.
Strictly speaking, Z3 is not SMT 2.0 compliant by default, and this is one of those cases. We can add
(set-option :smtlib2-compliant true)
and then this query is indeed rejected correctly.
Z3 is not the unique SMT solver that type-checks these examples:
CVC4 accepts them as well (even with option --smtlib-strict), and answers UNSAT in both cases of my formulas above.
Yices accepts them and answers UNSAT (after changing the logic to QF_LIA, because it does not support AUFLIRA).
With (set-logic QF_LIA), Z3 emits an error: (error "line 3 column 17: logic does not support reals").
Alt-Ergo says "typing error: Int and Real cannot be unified" in both cases. But Alt-Ergo's SMT2 parser is very limited and not heavily tested, as we concentrated on its native polymorphic language. So, it should not be taken as a reference.
I think that developers usually assume an "implicit" sub-typing relation between Int and Real. This is why these examples are successfully type-checked by Z3, CVC4 and Yices (and probably others as well).
Jochen Hoenicke gived the answer (on SMT-LIB mailing list) regarding "mixing reals and integers". Here it is:
I just wanted to point out, that the syntax may be officially correct.
There is an extension in AUFLIRA and AUFNIRA.
From http://smtlib.cs.uiowa.edu/logics/AUFLIRA.smt2
"For every operator op with declaration (op Real Real s) for some
sort s, and every term t1, t2 of sort Int and t of sort Real, the
expression
- (op t1 t) is syntactic sugar for (op (to_real t1) t)
- (op t t1) is syntactic sugar for (op t (to_real t1))
- (/ t1 t2) is syntactic sugar for (/ (to_real t1) (to_real t2)) "
One possible solution is
(declare-fun x () Real)
(declare-fun y () Real)
(assert (= x 0))
(assert (= y 0.5))
(check-sat)
(push)
(assert (= x y) )
(check-sat)
(pop)
and the output is
sat
unsat
In the following example, I tried to use uninterpreted Boolean function like "(declare-const p (Int) Bool)" rather than single Boolean constant for each assumption. But it does not work (it gives compilation error).
(set-option :produce-unsat-cores true)
(set-option :produce-models true)
(declare-fun p (Int) Bool)
;(declare-const p1 Bool)
;(declare-const p2 Bool)
; (declare-const p3 Bool)
;; We assert (=> p C) to track C using p
(declare-const x Int)
(declare-const y Int)
(assert (=> (p 1) (> x 10)))
;; An Boolean constant may track more than one formula
(assert (=> (p 1) (> y x)))
(assert (=> (p 2) (< y 5)))
(assert (=> (p 3) (> y 0)))
(check-sat (p 1) (p 2) (p 3))
(get-unsat-core)
Output
Z3(18, 16): ERROR: invalid check-sat command, 'not' expected, assumptions must be Boolean literals
Z3(19, 19): ERROR: unsat core is not available
I understand that it is not possible (unsupported) to use Boolean function. Is there any reason behind that? Is there different way to do that?
We have this restriction because Z3 applies many simplifications before it solves a problem. Some of them will rewrite formulas and terms. The problem that is actually solved by Z3 is very often quite different from the input problem. We would have trace back the simplified assumptions to the original assumptions, or introduce auxiliary variables. Restricting to Boolean literals avoids this issue, and makes the interface very clean. Note that this restriction does not limit the expressiveness. If you think it is too annoying to declare many Boolean variables to track different assertions. I suggest you take a look at the new Python front-end for Z3 called Z3Py. It is much more convenient to use than SMT 2.0. Here is your example in Z3Py: http://rise4fun.com/Z3Py/cL
In this example, instead of creating an uninterpreted predicate p, a "vector" (actually, it is a Python list) o Boolean constants is created.
The Z3Py online tutorial contains many examples.
It is also possible to implement in Z3Py the approach that creates auxiliary variables.
Here is the script that does the trick. I defined a function check_ext that does all the plumbing. http://rise4fun.com/Z3Py/B4
I'm trying to encode a QBF in smt-lib 2 syntax for z3. Running z3 results in a warning
WARNING: failed to find a pattern for quantifier (quantifier id: k!14)
and the satisfiability result is "unknown".
The code is as follows:
(declare-fun R (Bool Bool Bool Bool) Bool)
(assert
(forall ((x2 Bool) (x3 Bool))
(exists ((y Bool))
(forall ((x1 Bool))
(R x1 x2 x3 y)
)
)
)
)
(check-sat)
I got rid of the warning by rewriting the code to
(set-option :auto-config false)
(set-option :mbqi false)
(declare-fun R (Bool Bool Bool Bool) Bool)
(declare-fun x1 () Bool)
(declare-fun x2 () Bool)
(declare-fun x3 () Bool)
(declare-fun y () Bool)
(assert
(forall ((x2 Bool) (x3 Bool))
(!
(exists ((y Bool))
(!
(forall ((x1 Bool))
(!
(R x1 x2 x3 y)
:pattern((R x1 x2 x3 y)))
)
:pattern((R x1 x2 x3 y)))
)
:pattern((R x1 x2 x3 y)))
)
)
(check-sat)
The result for the sat-query, however, remains "unknown".
I'm guessing that I need to get the patterns right? How do I specify them for nested quantifiers? Simpler examples with quantifiers seem to work without pattern annotation, though.
The answer to What is the reason behind the warning message in Z3: "failed to find a pattern for quantifier (quantifier id: k!18) " and the z3 guide didn't help me too much, unfortunately.
This warning message can be ignored. It is just informing you that the E-matching engine will not be able to process this quantified formula.
E-matching is only effective for showing that a problem is unsatisfiable. Since your example is satisfiable, E-matching will not be very useful. That is, Z3 will not be able to return sat using the E-matching engine. Model based quantifier instantiation (MBQI) is the only engine in Z3 that is capable of showing that problems containing quantifiers are satisfiable.
Using the default configuration, Z3 will return sat for your example. It returns unknown because you disabled the MBQI module.
The MBQI engine guarantees that Z3 is a decision procedure for many fragments (see http://rise4fun.com/Z3/tutorial/guide). However, it is very expensive in general, and should be disabled when quick and approximated answers are sufficient. In this case, unknown may be read as probably sat. Verification tools such as VCC disable MBQI module since it is incapable of deciding formulas produced by them. That is, the formulas produced by VCC are not in any of the fragments that can be decided by the MBQI engine.
We say a fragment can be decided by Z3 when for any formula in the fragment Z3 will return sat or unsat (i.e., it does not return unknown). Of course, this claim assumes we have a unlimited amount of resources. That is, Z3 may also fail (i.e., return unknown) for decidable fragments when it runs out of memory, or a timeout was specified by the user.
Finally, Z3 3.2 has a bug in the MBQI engine. The bug has been fixed, and it does not affect your problem. If you need I can give you a pre-release version of Z3 4.0 which contains the bug fix.