I'm trying to use the z3 smt solver to allocate values to variables subject to constraints. As well as hard constraints I have some soft constraints (e.g. a != c). I expected to be able to specify the hard constraints with assert and the soft constraints as soft-assert and this works if I solve with (check-sat).
Some of the files are large and complex and only solve in a reasonable time if I turn on bit-blasting using (check-sat-using (then simplify solve-eqs bit-blast sat)). When I do this the soft asserts seem to be ignored (example below or at rise4fun). Is this expected? Is it possible to use both bit-blast solving and soft-assert at the same time?
The following SMT code defines 4 bitvectors, a, b, c & d which should all be able to take unique values but are only forced to do so by soft asserts. Using the check-sat (line 39) works as expected but the check-sat-using (line 38) assigns b and d to the same value.
(set-option :produce-models true)
(set-logic QF_BV)
;; Declaring all the variables
(declare-const a (_ BitVec 2))
(declare-const b (_ BitVec 2))
(declare-const c (_ BitVec 2))
(declare-const d (_ BitVec 2))
(assert (or (= a #b00)
(= a #b01)
(= a #b10)
(= a #b11)))
(assert (or (= b #b00)
(= b #b01)
(= b #b10)
(= b #b11)))
(assert (or (= c #b00)
(= c #b01)
(= c #b10)
(= c #b11)))
(assert (or (= d #b00)
(= d #b01)
(= d #b10)
(= d #b11)))
;; Soft constraints to limit reuse
(assert-soft (not (= a b)))
(assert-soft (not (= a c)))
(assert-soft (not (= a d)))
(assert-soft (not (= b c)))
(assert-soft (not (= b d)))
(assert-soft (not (= c d)))
(check-sat-using (then simplify solve-eqs bit-blast sat))
;;(check-sat)
(get-value (a
b
c
d))
Great question! When you use assert-soft the optimization engine kicks in by default. You can see this by using your program with the (check-sat) clause, and running with higher verbosity. I've put your program in a file called a.smt2:
$ z3 -v:3 a.smt2
(optimize:check-sat)
(sat.solver)
(optimize:sat)
(maxsmt)
(opt.maxres [0:6])
(sat.solver)
(opt.maxres [0:0])
found optimum
sat
((a #b01)
(b #b00)
(c #b11)
(d #b10))
So, we can see z3 is treating this as an optimization problem, which takes soft-constraints into account and gives you the "disjointness" you're seeking.
Let's do the same, but this time we'll use the check-sat call that specifies the tactics to use. We get:
$ z3 -v:3 a.smt2
(smt.searching)
sat
((a #b11)
(b #b11)
(c #b11)
(d #b10))
And this confirms your suspicion: When you tell z3 exactly what to do, it doesn't do the optimization pass. In hindsight, this is to be expected, but I do agree that it's rather surprising.
The question is then whether we can tell z3 to do the optimization explicitly. However I'm not sure if this is even possible within the tactic language. I think this question is well worthy of asking at their issues site (https://github.com/Z3Prover/z3/issues) and see if there's a magic incantation you can use to kick off the maxres engine from the tactic language. (This may not be possible due to a number of reasons, but there's no reason to speculate here.) Please report back here what you find out!
Related
I'm trying to use the SMTLIB format to express set membership in Z3:
(declare-const a (Set Int))
;; the next two lines parse correctly in CVC4, but not Z3:
(assert (= a (as emptyset (Set Int))))
(assert (member 12 a))
;; these work in both solvers
(assert (= a (union a a)))
(assert (subset a a))
(assert (= a (intersection a a)))
(check-sat)
The functions emptyset and member seem to parse as expected in CVC4, but not in Z3.
From checking the API (e.g., here: https://z3prover.github.io/api/html/group__capi.html), Z3 does support empty sets and membership programatically, but how does one express these in SMTLIB syntax?
It's indeed annoying z3 and CVC4 use slightly different notations for sets. In z3, a set is essentially an array with the range of bool. Based on this analogy, your program is coded as:
(declare-const a (Set Int))
(assert (= a ((as const (Set Int)) false)))
(assert (select a 12))
(assert (= a (union a a)))
(assert (subset a a))
(assert (= a (intersection a a)))
(check-sat)
which z3 accepts as is and produces unsat. But you'll find that CVC4 doesn't like this program now.
It would be great if the SMTLib movement standardized the theory of sets (http://smtlib.cs.uiowa.edu/) and there has indeed been a proposal along these lines (https://www.kroening.com/smt-lib-lsm.pdf) but I don't think it has been adopted by solvers nor sanctioned by the SMTLib committee yet.
I wonder why Z3 is able to demonstrate some trivial equalities by applying associativity and commutativity (AC) axioms in some cases but not in others. For instance,
(simplify (= (bvadd x (bvadd y z)) (bvadd z (bvadd y x))))
reduces to true, but
(simplify (= (bvxor x (bvxor y z)) (bvxor z (bvxor y x))))
does not (Z3 just flattens bvxor applications).
I took a look at the source code (src/ast/bv_decl_plugin.cpp) and both bvadd and bvxor are declared as AC symbols. Is it related with the rewriting rules that are applied to each of those symbols? In particular, mk_bv_add (src/ast/rewriter/bv_rewriter.cpp) calls mk_add_core (src/ast/simplifier/poly_simplifier_plugin.cpp) which process a bvadd term as a sum of monomials.
Yes, it is related to the rewriting rules that are applied to the symbols.
Z3 has two expression simplifiers: src/ast/rewriter and src/ast/simplifier. The src/ast/simplifier is legacy and it is not used in new code. The simplify command in the SMT-LIB front-end is based on src/ast/rewriter. The mk_bv_add is actually using mk_add_core in src/ast/rewriter/poly_rewriter.h.
It is not hard to change the code to force Z3 to simplify the bvxor expression in your question to true. We just have to add the following line of code at src/ast/rewriter/bv_rewriter.h.
The new line is simply sorting the bvxor arguments. This is a correct rewrite for any AC operator.
br_status bv_rewriter::mk_bv_xor(unsigned num, expr * const * args, expr_ref & result) {
SASSERT(num > 0);
...
default:
// NEW LINE
std::sort(new_args.begin(), new_args.end(), ast_to_lt());
//
result = m_util.mk_bv_xor(new_args.size(), new_args.c_ptr());
return BR_DONE;
}
}
That being said, the Z3 rewriters are not supposed to apply every possible simplification and/or produce canonical normal forms. Their main goal is to produce formulas that are possibly simpler to solver. The rules grow based on demand (e.g., Z3 is too slow in example X, and the performance problem can be "fixed" by applying a new preprocessing rule), or based on user request. So, if you think this is an useful feature, we can add an option that will sort the arguments of every AC operator.
EDIT
Correction: we also have to modify the following statement
if (!merged && !flattened && (num_coeffs == 0 || (num_coeffs == 1 && !v1.is_zero() && v1 != (rational::power_of_two(sz) - numeral(1))))
return BR_FAILED;
This statement interrupts the execution of mk_bv_xor, when none of the existing rewriting rules are applicable. We also have to modify it. I implemented these modifications here. We can activate them by using the new option :bv-sort-ac. This option is not enabled by default. The new option is available in the unstable (work-in-progress) branch. When set to true, it will sort bit-vector AC operators.
Note that, the unstable branch uses the new parameter setting framework that will be available in the next official release. Here are instructions on how to build the unstable branch.
These modifications will also be available this week on the nightly builds.
Here are some examples that use the new option:
(declare-const a (_ BitVec 8))
(declare-const b (_ BitVec 8))
(declare-const c (_ BitVec 8))
(declare-const d (_ BitVec 8))
(simplify (= (bvxor a b c) (bvxor b a c)))
(simplify (= (bvxor a b c) (bvxor b a c)) :bv-sort-ac true)
(simplify (= (bvxor a (bvxor b c)) (bvxor b a c)) :bv-sort-ac true)
(simplify (= (bvor a b c) (bvor b a c)))
(simplify (= (bvor a b c) (bvor b a c)) :bv-sort-ac true)
(simplify (= (bvor a (bvor b c)) (bvor b a c)) :bv-sort-ac true)
(simplify (= (bvand a b c) (bvand b a c)))
(simplify (= (bvand a b c) (bvand b a c)) :bv-sort-ac true)
(simplify (= (bvand a (bvand b c)) (bvand b a c)) :bv-sort-ac true)
; In the new parameter setting framework, each module has its own parameters.
; The bv-sort-ac is a parameter of the "rewriter" framework.
(set-option :rewriter.bv-sort-ac true)
; Now, Z3 will rewrite the following formula to true even when we do not provide
; the option :bv-sort-ac true
(simplify (= (bvand a b c) (bvand b a c)))
; It will also simplify the following assertion.
(assert (= (bvand a b c) (bvand b a c)))
(check-sat)
END EDIT
(declare-datatypes () ((SE BROKEN ON OFF)))
(declare-const s SE)
(declare-const a Int)
(simplify (or (= s ON) (= s OFF) (= s BROKEN)))
(simplify (and (> a 0) (> a 1)))
The result is:
(or (= s ON) (= s OFF) (= s BROKEN))
(and (not (<= a 0)) (not (<= a 1)))
But the expected result was:
1
> a 1
Is it possible to simplify such expressions (the combinations of such expressions) in Z3?
Thank you!
The simplify command is just a bottom-up rewriter. It is fast, but will fail to simplify expressions such as the ones in your post. Z3 allows users to define their own simplification strategies using tactics. They are described in this article, and the Z3 tutorials (Python and SMT 2.0). The following posts also have additional information:
t>=1 or t>=2 => t>=1
Asymmetric behavior in ctx-solver-simplify
what's the difference between "simplify" and "ctx-solver-simplify" in z3
The first query in your example can be simplified using the tactic ctx-solver-simplify (also available online here).
(declare-datatypes () ((SE BROKEN ON OFF)))
(declare-const s SE)
(declare-const a Int)
(assert (or (= s ON) (= s OFF) (= s BROKEN)))
(assert (and (> a 0) (> a 1)))
(apply ctx-solver-simplify)
The command apply applies the tactic ctx-solver-simplify over the set of assertions, and displays the resulting set of goals. Note that, this tactic is way more expensive than the command simplify.
Is there a way to use z3 to convert a formula to CNF (using Tseitsin-style encoding)? I am looking for something like the simplify command, but guaranteeing that the returned formula is CNF.
You can use the apply command for doing it. We can provide arbitrary tactics/strategies to this command. For more information about tactics and strategies in Z3 4.0, check the tutorial http://rise4fun.com/Z3/tutorial/strategies
The command (help-tactic) can be used to display all available tactics in Z3 4.0 and their parameters. The programmatic is more convenient to use and flexible. Here is a tutorial based on the new Python API: http://rise4fun.com/Z3Py/tutorial/strategies.
The same capabilities are available in the .Net and C/C++ APIs.
The following script demonstrates how to convert a formula into CNF using this framework:
http://rise4fun.com/Z3/TEu6
The example link #Leonardo provided is broken now. Found the code using wayback machine. Posting it here so future seekers may make use of it:
(declare-const x Int)
(declare-const y Int)
(declare-const z Int)
(assert (iff (> x 0) (> y 0)))
(assert (or (and (= x 0) (= y 0)) (and (= x 1) (= y 1)) (and (= x 2) (= y 2))))
(assert (if (> x 0) (= y x) (= y (- x 1))))
(assert (> z (if (> x 0) (- x) x)))
(apply (then (! simplify :elim-and true) elim-term-ite tseitin-cnf))
(echo "Trying again without using distributivity...")
(apply (then (! simplify :elim-and true) elim-term-ite (! tseitin-cnf :distributivity false)))
I am using the ELIM_QUANTIFIERS feature of Z3 to eliminate quantifiers from formulas over bit vectors. I ran into the following situation where Z3 produces a correct, yet overly complicated result, and I was wondering whether there exists a way to re-write my problem or perhaps a configuration option that would lead to the simple result I expect.
First, here is an example that works as expected. It states that for a bit vector of length 4 there exists a bit vector that is equal to it.
(set-option ELIM_QUANTIFIERS true)
(declare-fun a () BitVec[4])
(simplify (exists ((x BitVec[4]))
(= a x)))
Z3 generates the following output for this example.
success
success
true
However, if I add a negation:
(set-option ELIM_QUANTIFIERS true)
(declare-fun a () BitVec[4])
(simplify (exists ((x BitVec[4]))
(not (= a x))))
then Z3 produces the following output, which lists all possible values of the vector instead of just returning "true".
success
success
(let (($x54 (= (_ bv0 4) a)))
(let (($x55 (not $x54)))
(let (($x61 (= (_ bv2 4) a)))
(let (($x62 (not $x61)))
(let (($x68 (= (_ bv6 4) a)))
(let (($x69 (not $x68)))
(let (($x75 (= (_ bv4 4) a)))
(let (($x76 (not $x75)))
(let (($x82 (= (_ bv12 4) a)))
(let (($x83 (not $x82)))
(let (($x89 (= (_ bv8 4) a)))
(let (($x90 (not $x89)))
(let (($x95 (= (_ bv1 4) a)))
(let (($x96 (not $x95)))
(let (($x102 (= (_ bv5 4) a)))
(let (($x103 (not $x102)))
(let (($x109 (= (_ bv13 4) a)))
(let (($x110 (not $x109)))
(let (($x116 (= (_ bv9 4) a)))
(let (($x117 (not $x116)))
(let (($x123 (= (_ bv3 4) a)))
(let (($x124 (not $x123)))
(let (($x130 (= (_ bv7 4) a)))
(let (($x131 (not $x130)))
(let (($x137 (= (_ bv14 4) a)))
(let (($x138 (not $x137)))
(let (($x144 (= (_ bv10 4) a)))
(let (($x145 (not $x144)))
(let (($x151 (= (_ bv11 4) a)))
(let (($x152 (not $x151)))
(let (($x158 (= (_ bv15 4) a)))
(let (($x159 (not $x158)))
(or $x159 $x152 $x145 $x138 $x131 $x124 $x117 $x110 $x103 $x96 $x90 $x83 $x76 $x69 $x62 $x55)))))))))))))))))))))))))))))))))
For longer bit vectors, e.g., of size 32 or more, Z3 does not produce a result in a reasonable time, as it is presumably enumerating all possible values of the 32-bit variable.
Note that in this particular case I could just use check-sat to check the validity of the formula; however in the general case I am interested in obtaining a quantifier-free expression equivalent to the original formula, rather than just checking its validity.
I am using Z3 v3.2 for Linux.
Thanks this is a good simple example showing a limitation with the approach used for bit-vectors.
It is really only appropriate for formulas where the number of solutions is relatively small.
There is no way to toggle bit-vectors on/off during elimination this way.
It is a valid point to improve on in Z3.
There are a couple of potential work-arounds:
First of all, the Boolean quantifier elimination, while it is also rudimentary (it just enumerates satisfiable instances and eliminates the Boolean variable) it does produce the simpler result 'true'.
when you bit-blast your bit-vector from this example. A second avenue is to consider whether your problems can be reformulated in the UFBV fragment where satisfiability does not require quantifier elimination.