I am using the ELIM_QUANTIFIERS feature of Z3 to eliminate quantifiers from formulas over bit vectors. I ran into the following situation where Z3 produces a correct, yet overly complicated result, and I was wondering whether there exists a way to re-write my problem or perhaps a configuration option that would lead to the simple result I expect.
First, here is an example that works as expected. It states that for a bit vector of length 4 there exists a bit vector that is equal to it.
(set-option ELIM_QUANTIFIERS true)
(declare-fun a () BitVec[4])
(simplify (exists ((x BitVec[4]))
(= a x)))
Z3 generates the following output for this example.
success
success
true
However, if I add a negation:
(set-option ELIM_QUANTIFIERS true)
(declare-fun a () BitVec[4])
(simplify (exists ((x BitVec[4]))
(not (= a x))))
then Z3 produces the following output, which lists all possible values of the vector instead of just returning "true".
success
success
(let (($x54 (= (_ bv0 4) a)))
(let (($x55 (not $x54)))
(let (($x61 (= (_ bv2 4) a)))
(let (($x62 (not $x61)))
(let (($x68 (= (_ bv6 4) a)))
(let (($x69 (not $x68)))
(let (($x75 (= (_ bv4 4) a)))
(let (($x76 (not $x75)))
(let (($x82 (= (_ bv12 4) a)))
(let (($x83 (not $x82)))
(let (($x89 (= (_ bv8 4) a)))
(let (($x90 (not $x89)))
(let (($x95 (= (_ bv1 4) a)))
(let (($x96 (not $x95)))
(let (($x102 (= (_ bv5 4) a)))
(let (($x103 (not $x102)))
(let (($x109 (= (_ bv13 4) a)))
(let (($x110 (not $x109)))
(let (($x116 (= (_ bv9 4) a)))
(let (($x117 (not $x116)))
(let (($x123 (= (_ bv3 4) a)))
(let (($x124 (not $x123)))
(let (($x130 (= (_ bv7 4) a)))
(let (($x131 (not $x130)))
(let (($x137 (= (_ bv14 4) a)))
(let (($x138 (not $x137)))
(let (($x144 (= (_ bv10 4) a)))
(let (($x145 (not $x144)))
(let (($x151 (= (_ bv11 4) a)))
(let (($x152 (not $x151)))
(let (($x158 (= (_ bv15 4) a)))
(let (($x159 (not $x158)))
(or $x159 $x152 $x145 $x138 $x131 $x124 $x117 $x110 $x103 $x96 $x90 $x83 $x76 $x69 $x62 $x55)))))))))))))))))))))))))))))))))
For longer bit vectors, e.g., of size 32 or more, Z3 does not produce a result in a reasonable time, as it is presumably enumerating all possible values of the 32-bit variable.
Note that in this particular case I could just use check-sat to check the validity of the formula; however in the general case I am interested in obtaining a quantifier-free expression equivalent to the original formula, rather than just checking its validity.
I am using Z3 v3.2 for Linux.
Thanks this is a good simple example showing a limitation with the approach used for bit-vectors.
It is really only appropriate for formulas where the number of solutions is relatively small.
There is no way to toggle bit-vectors on/off during elimination this way.
It is a valid point to improve on in Z3.
There are a couple of potential work-arounds:
First of all, the Boolean quantifier elimination, while it is also rudimentary (it just enumerates satisfiable instances and eliminates the Boolean variable) it does produce the simpler result 'true'.
when you bit-blast your bit-vector from this example. A second avenue is to consider whether your problems can be reformulated in the UFBV fragment where satisfiability does not require quantifier elimination.
Related
I'm trying to use the SMTLIB format to express set membership in Z3:
(declare-const a (Set Int))
;; the next two lines parse correctly in CVC4, but not Z3:
(assert (= a (as emptyset (Set Int))))
(assert (member 12 a))
;; these work in both solvers
(assert (= a (union a a)))
(assert (subset a a))
(assert (= a (intersection a a)))
(check-sat)
The functions emptyset and member seem to parse as expected in CVC4, but not in Z3.
From checking the API (e.g., here: https://z3prover.github.io/api/html/group__capi.html), Z3 does support empty sets and membership programatically, but how does one express these in SMTLIB syntax?
It's indeed annoying z3 and CVC4 use slightly different notations for sets. In z3, a set is essentially an array with the range of bool. Based on this analogy, your program is coded as:
(declare-const a (Set Int))
(assert (= a ((as const (Set Int)) false)))
(assert (select a 12))
(assert (= a (union a a)))
(assert (subset a a))
(assert (= a (intersection a a)))
(check-sat)
which z3 accepts as is and produces unsat. But you'll find that CVC4 doesn't like this program now.
It would be great if the SMTLib movement standardized the theory of sets (http://smtlib.cs.uiowa.edu/) and there has indeed been a proposal along these lines (https://www.kroening.com/smt-lib-lsm.pdf) but I don't think it has been adopted by solvers nor sanctioned by the SMTLib committee yet.
I'm trying to use the z3 smt solver to allocate values to variables subject to constraints. As well as hard constraints I have some soft constraints (e.g. a != c). I expected to be able to specify the hard constraints with assert and the soft constraints as soft-assert and this works if I solve with (check-sat).
Some of the files are large and complex and only solve in a reasonable time if I turn on bit-blasting using (check-sat-using (then simplify solve-eqs bit-blast sat)). When I do this the soft asserts seem to be ignored (example below or at rise4fun). Is this expected? Is it possible to use both bit-blast solving and soft-assert at the same time?
The following SMT code defines 4 bitvectors, a, b, c & d which should all be able to take unique values but are only forced to do so by soft asserts. Using the check-sat (line 39) works as expected but the check-sat-using (line 38) assigns b and d to the same value.
(set-option :produce-models true)
(set-logic QF_BV)
;; Declaring all the variables
(declare-const a (_ BitVec 2))
(declare-const b (_ BitVec 2))
(declare-const c (_ BitVec 2))
(declare-const d (_ BitVec 2))
(assert (or (= a #b00)
(= a #b01)
(= a #b10)
(= a #b11)))
(assert (or (= b #b00)
(= b #b01)
(= b #b10)
(= b #b11)))
(assert (or (= c #b00)
(= c #b01)
(= c #b10)
(= c #b11)))
(assert (or (= d #b00)
(= d #b01)
(= d #b10)
(= d #b11)))
;; Soft constraints to limit reuse
(assert-soft (not (= a b)))
(assert-soft (not (= a c)))
(assert-soft (not (= a d)))
(assert-soft (not (= b c)))
(assert-soft (not (= b d)))
(assert-soft (not (= c d)))
(check-sat-using (then simplify solve-eqs bit-blast sat))
;;(check-sat)
(get-value (a
b
c
d))
Great question! When you use assert-soft the optimization engine kicks in by default. You can see this by using your program with the (check-sat) clause, and running with higher verbosity. I've put your program in a file called a.smt2:
$ z3 -v:3 a.smt2
(optimize:check-sat)
(sat.solver)
(optimize:sat)
(maxsmt)
(opt.maxres [0:6])
(sat.solver)
(opt.maxres [0:0])
found optimum
sat
((a #b01)
(b #b00)
(c #b11)
(d #b10))
So, we can see z3 is treating this as an optimization problem, which takes soft-constraints into account and gives you the "disjointness" you're seeking.
Let's do the same, but this time we'll use the check-sat call that specifies the tactics to use. We get:
$ z3 -v:3 a.smt2
(smt.searching)
sat
((a #b11)
(b #b11)
(c #b11)
(d #b10))
And this confirms your suspicion: When you tell z3 exactly what to do, it doesn't do the optimization pass. In hindsight, this is to be expected, but I do agree that it's rather surprising.
The question is then whether we can tell z3 to do the optimization explicitly. However I'm not sure if this is even possible within the tactic language. I think this question is well worthy of asking at their issues site (https://github.com/Z3Prover/z3/issues) and see if there's a magic incantation you can use to kick off the maxres engine from the tactic language. (This may not be possible due to a number of reasons, but there's no reason to speculate here.) Please report back here what you find out!
I'm new to Z3 and I'm trying to understand how it works, and what it can and cannot do. I know that Z3 has at least some support for exponentials through the power (^) operator (see Z3py returns unknown for equation using pow() function, How to represent logarithmic formula in z3py, and Use Z3 and SMT-LIB to define sqrt function with a real number). What I'm unclear on is how extensive this support is, and what kind of inferences z3 can make about exponentials.
Here's a simple example involving exponentials which z3 can analyze. We define an exponential function, and then ask it to verify that exp(0) == 1:
(define-fun exp ((x Real)) Real
(^ 2.718281828459045 x))
(declare-fun x1 () Real)
(declare-fun y1 () Real)
(assert (= y1 (exp x1)))
(assert (not (=> (= x1 0.0) (= y1 1.0))))
(check-sat)
(exit)
Z3 returns unsat, as expected. On the other hand, here's a simple example which Z3 can't analyze:
(define-fun exp ((x Real)) Real
(^ 2.718281828459045 x))
(declare-fun x1 () Real)
(declare-fun y1 () Real)
(assert (= y1 (exp x1)))
(assert (not (< y1 0.0)))
(check-sat)
(exit)
This should be satisfiable, since literally any value for x1 would give y1 > 0. However, Z3 returns unknown. Naively I might have expected that Z3 would be able to analyze this, given that it could analyze the first example.
I realize this question is a bit broad, but: can anyone give me any insight into how Z3 handles exponentials, and (more specifically) why it can solve the first example I gave but not the second?
It is hard to say in general, since non-linear solving is challenging, but the case you presented is actually not so mysterious. You wrote:
(assert (= y (exp x)))
(assert (not (=> (= x 0) (= y 1))))
Z3 is going to simplify the second assertion, yielding:
(assert (= y (exp x)))
(assert (= x 0))
(assert (not (= y 1)))
Then it will propagate the first equality, yielding:
(assert (= y (exp 0)))
(assert (not (= y 1)))
Now when exp is expanded, you have a case of constant^constant, which Z3 can handle (for integer exponents, etc).
For the second case, you are asking it a very very basic question about variable exponents, and Z3 immediately barfs. That's not too odd, since so many questions about variable exponents are either known uncomputable or unknown but hard.
I'm trying to use Why3's Z3 back-end in order to retrieve models that can then be used to derive test cases exhibiting bugs in programs. However, Z3 version 4.3.2 seems unable to answer sat for any Why3 goal. It looks like some of the axiomatic definitions used by Why3 somehow confuse Z3. For instance, the following example (which is a tiny part of what Why3 generates)
(declare-fun abs1 (Int) Int)
;; abs_def
(assert
(forall ((x Int)) (ite (<= 0 x) (= (abs1 x) x) (= (abs1 x) (- x)))))
(check-sat)
results in timeout with the following command line:
z3 -smt2 model.partial=true file.smt2 -T:10
On the other hand, changing the definition to
(declare-fun abs1 (Int) Int)
;; abs_def
(assert
(forall ((x Int)) (=> (<= 0 x) (= (abs1 x) x))))
(assert
(forall ((x Int)) (=> (> 0 x) (= (abs1 x) (- x)))))
will get me a model (which looks pretty reasonable)
(model
(define-fun abs1 ((x!1 Int)) Int
(ite (>= x!1 0) x!1 (* (- 1) x!1)))
)
but if I try to add the next axiom present in the original Why3 file, namely
;; Abs_pos
(assert (forall ((x Int)) (<= 0 (abs1 x))))
again Z3 answers timeout.
Is there something I'm missing in the configuration of Z3? Moreover, in previous versions of Why3, there was an option MODEL_ON_TIMEOUT, which allowed to retrieve a model in such circumstances. Even though there was no guarantee that this was a real model since Z3 could not finish to check it, in practice such models generally contained all the information I needed. However, I haven't found a similar option in 4.3.2. Does it still exist?
Update The last axiom Abs_pos was wrong (I toyed a bit with Why3's output before posting here and ended up pasting an incorrect version of the issue). This is now fixed.
The additional axiom
(assert (not (forall ((x Int)) (<= 0 (abs1 x)))))
makes the problem unsatisfiable, since abs1 always returns a non-negative integer and with the additional axiom you require the existence of a negative result for abs1 for some x. The web version of Z3 returns unsat as expected, see here.
I've two SMT2-Lib scripts using reals, which are morally equivalent. The only difference is that one also uses bit-vectors while the other does not.
Here's the version that uses both reals and bit-vectors:
; uses both reals and bit-vectors
(set-option :produce-models true)
(define-fun s2 () Real (root-obj (+ (^ x 2) (- 2)) 2))
(define-fun s3 () Real 0.0)
(define-fun s6 () Real (/ 1.0 1.0))
(declare-fun s0 () (_ BitVec 1))
(declare-fun s1 () (_ BitVec 1))
(assert
(let ((s4 (- s3 s2)))
(let ((s5 (ite (= #b1 s1) s2 s4)))
(let ((s7 (+ s5 s6)))
(let ((s8 (- s5 s6)))
(let ((s9 (ite (= #b1 s0) s7 s8)))
(let ((s10 (ite (>= s9 s3) #b1 #b0)))
(= s10 #b1))))))))
(check-sat)
(get-model)
Here's the morally equivalent script, using Bool instead of a bit-vector of size 1, otherwise it's essentially the same:
; uses reals only
(set-option :produce-models true)
(define-fun s2 () Real (root-obj (+ (^ x 2) (- 2)) 2))
(define-fun s3 () Real 0.0)
(define-fun s6 () Real (/ 1.0 1.0))
(declare-fun s0 () (Bool))
(declare-fun s1 () (Bool))
(assert
(let ((s4 (- s3 s2)))
(let ((s5 (ite s1 s2 s4)))
(let ((s7 (+ s5 s6)))
(let ((s8 (- s5 s6)))
(let ((s9 (ite s0 s7 s8)))
(let ((s10 (ite (>= s9 s3) #b1 #b0)))
(= s10 #b1))))))))
(check-sat)
(get-model)
For the former I get unknown from z3 (v4.1 on Mac), while the latter nicely produces sat and a model.
While SMT-Lib2 doesn't allow mixing reals and bit-vectors, I thought Z3 handled these combinations just fine. Am I mistaken? Is there a workaround?
(Note that these are generated scripts, so just using Bool instead of (_ BitVec 1) is rather costly, as it requires quite a bit of changes elsewhere.)
The new nonlinear solver is not integrated with other theories yet. It supports only real variables and Booleans. Actually, it also allows integer variables, but it is very limited support for them. It actually solves nonlinear integer problems as real problems, and just checks in the end whether each integer variable is assigned to an integer value. Moreover, this solver is the only complete procedure for nonlinear (real) arithmetic available in Z3.
Since your first problem contains Bit-vectors, the nonlinear solver is not used by Z3. Instead, Z3 uses a general purpose solver that combines many theories, but it is incomplete for nonlinear arithmetic.
That being said, I understand this is a limitation, and I'm working on that. In the (not so near) future, Z3 will have a new solver that integrates nonlinear arithmetic, arrays, bit-vectors, etc.
Finally, the bit-vector theory is a very special case, since we can easily reduce it to propositional logic in Z3.
Z3 has tactic bit-blast that applies this reduction. This tactic can reduce any nonlinear+bit-vector problem into a problem that contains only reals and Booleans. Here is an example (http://rise4fun.com/Z3/0xl).
; uses both reals and bit-vectors
(set-option :produce-models true)
(define-fun s2 () Real (root-obj (+ (^ x 2) (- 2)) 2))
(define-fun s3 () Real 0.0)
(define-fun s6 () Real (/ 1.0 1.0))
(declare-fun s0 () (_ BitVec 1))
(declare-fun s1 () (_ BitVec 1))
(declare-fun v2 () (_ BitVec 8))
(assert
(let ((s4 (- s3 s2)))
(let ((s5 (ite (= #b1 s1) s2 s4)))
(let ((s7 (+ s5 s6)))
(let ((s8 (- s5 s6)))
(let ((s9 (ite (= #b1 s0) s7 s8)))
(let ((s10 (ite (>= s9 s3) #b1 #b0)))
(= s10 #b1))))))))
(assert (or (and (not (= v2 #x00)) (not (= v2 #x01))) (bvslt v2 #x00)))
(assert (distinct (bvnot v2) #x00))
(check-sat-using (then simplify bit-blast qfnra))
(get-model)