For this benchmark:
(set-option :produce-models true)
(declare-fun s0 () Real)
(declare-fun s1 () Real)
(assert (> s0 s1))
(check-sat)
(get-value (s0))
(get-value (s1))
I'm getting:
sat
((s0 0.0))
((s1 0.0))
Is this a known issue? (This is using Z3 V3.2 both on Linux and Mac.)
Yes, this was a bug in the model generator. The bug has been fixed.
The bug can be avoided using
(set-option :auto-config false)
If Z3 becomes too slow in quantifier free problems, then we can also add
(set-option :relevancy 0)
Related
We have encountered constraints for which Z3 returns sat but if we then add a certain named assertion then Z3 returns unknown. One example is:
(declare-fun a () Real)
(declare-fun b () Bool)
(assert (and (<= 0.1 a) (<= a 10.0)))
(assert (= b (= 1.0 (/ 1.0 a))))
(check-sat)
Z3 reports that this is satisfiable as expected. We can additionally assert that b is either true or false, both of which are satisfiable as expected. However, if we use a named assertion for the value of b then the result of the satisfiability check can become unknown depending on the value. With the value true, Z3 still returns sat:
(set-option :produce-unsat-cores true)
(declare-fun a () Real)
(declare-fun b () Bool)
(assert (and (<= 0.1 a) (<= a 10.0)))
(assert (= b (= 1.0 (/ 1.0 a))))
(assert (! (= b true) :named c))
(check-sat)
Using the value false, Z3 returns unknown:
(set-option :produce-unsat-cores true)
(declare-fun a () Real)
(declare-fun b () Bool)
(assert (and (<= 0.1 a) (<= a 10.0)))
(assert (= b (= 1.0 (/ 1.0 a))))
(assert (! (= b false) :named c))
(check-sat)
Checking the reason for the unknown result (with (get-info :reason-unknown)) returns (incomplete (theory arithmetic)).
It appears that z3 is just picking the wrong tactic in the latter case. Replace your check-sat call with:
(check-sat-using qfnra-nlsat)
And z3 says sat in both cases.
Why z3 ends up picking a different tactic is a difficult question to answer; I'd recommend filing this as a ticket at their github site; they might be missing a heuristic.
Side note: I found this by running both cases with -v:10 and skimming through the verbose output. It's not a bad way to see what z3 is doing for short-enough benchmarks.
I wonder, if there is a possibility in a SMT-LIB 2.0 script to access the last satisfiability decision of a solver (sat, unsat, ...). For example, the following code:
(set-option :produce-unsat-cores true)
(set-option :produce-models true)
(set-logic QF_UF)
(declare-fun p () Bool)
(declare-fun q () Bool)
(declare-fun r () Bool)
(assert (! (=> p q) :named PQ))
(assert (! (=> q r) :named QR))
(assert (! (not (=> p r)) :named nPR))
(check-sat)
(get-model)
(get-unsat-core)
ran in Z3 returns:
unsat
(error "line 15 column 10: model is not available")
(PQ QR nPR)
and ran in MathSAT returns:
unsat
(error "model generation not enabled")
In MathSAT 5 it just breaks on (get-model) and doesn't even reach (get-unsat-core).
Is there any way in SMT-LIB 2.0 language to get-model iff the decision was SAT and unsat-core iff the decision was UNSAT? Solution could for example look like this:
(check-sat)
(ite (= (was-sat) true) (get-model) (get-unsat-core))
I searched SMT-LIB 2.0 language documentation, but I did not found any hint.
EDIT:
I also tried the code below, and unfortunately it did not work.
(ite (= (check-sat) "sat") (get-model) (get-unsat-core))
The SMT language does not let you write commands like this.
The way that tools, such as Boogie, deal with this is to use
a two-way text pipe: It reads back the result from (check-sat).
If the resulting string is "unsat" models are not available, but
cores would be if the check uses asssumptions. If the resulting
string is "sat" the tool can expect that a (get-model) command
succeeds.
As Nikolaj said in his answer, the right way to do this is to parse the solver output and conditionally generate either a (get-model) or a (get-unsat-core) statement.
However, with mathsat you can use the code without the (get-model) statement, and call mathsat with the -model option. For example:
$ cat demo_sat.smt2
(set-option :produce-unsat-cores true)
(set-option :produce-models true)
(set-logic QF_UF)
(declare-fun p () Bool)
(declare-fun q () Bool)
(declare-fun r () Bool)
(assert (! (=> p q) :named PQ))
(assert (! (=> q r) :named QR))
; (assert (! (not (=> p r)) :named nPR))
(check-sat)
(get-unsat-core)
$ mathsat -model demo_sat.smt2
sat
( (p false)
(q false)
(r false) )
(error "no unsatisfiability proof, impossible to compute unsat core")
And in the unsat case:
$ cat demo_unsat.smt2
(set-option :produce-unsat-cores true)
(set-option :produce-models true)
(set-logic QF_UF)
(declare-fun p () Bool)
(declare-fun q () Bool)
(declare-fun r () Bool)
(assert (! (=> p q) :named PQ))
(assert (! (=> q r) :named QR))
(assert (! (not (=> p r)) :named nPR))
(check-sat)
(get-unsat-core)
$ mathsat -model demo_unsat.smt2
unsat
( PQ
QR
nPR )
Unfortunately there does not seem to exist an option like -model for producing unsat cores. So this hack won't work if you want to use it with an incremental problem, unless you are OK with the solver terminating after the first sat result. (Because at the first sat result the solver will exit on the error for (get-unsat-core).)
While trying to solve large nonlinear real arithmetic problems, I track every assertion using answer literals and explicit implications, as recommended in other posts. It should be equivalent to using the (! (...) :named p1) syntax of the SMT2 format. It seems, though, that both methods are handled differently internally.
The following SMT2 code gives an UNKNOWN result, with explanation "(incomplete (theory arithmetic))":
(set-option :print-success false)
(set-option :produce-unsat-cores true) ; enable generation of unsat cores
(set-option :produce-models true) ; enable model generation
(declare-const p1 Bool)
(declare-const p2 Bool)
(declare-const p3 Bool)
(declare-const p4 Bool)
(declare-const p5 Bool)
(declare-const x1 Real)
(declare-const x2 Real)
(declare-const x3 Real)
(assert (=> p1 (= x1 (/ 1.0 (* x2 x2)))))
(assert (=> p2 (not (= x2 0.0))))
(assert (=> p3 (= x3 (* 2.0 x1))))
(assert (=> p4 (= x3 5.0)))
(assert (=> p5 (< x3 0.0)))
(check-sat p1 p2 p3)
(get-info:reason-unknown)
On the other hand, the following SMT2 code gives the correct answer, UNSAT, and produces an informative unsat core (p4, p5):
(set-option :print-success false)
(set-option :produce-unsat-cores true) ; enable generation of unsat cores
(set-option :produce-models true) ; enable model generation
(declare-const x1 Real)
(declare-const x2 Real)
(declare-const x3 Real)
(assert (! (= x1 (/ 1.0 (* x2 x2))) :named p1))
(assert (! (not (= x2 0.0)) :named p2))
(assert (! (= x3 (* 2.0 x1)) :named p3))
(assert (! (= x3 5.0) :named p4))
(assert (! (< x3 0) :named p5))
(check-sat)
(get-unsat-core)
;(get-model)
My specific questions are:
How can this differing behavior be explained? What is recommended practice for tracking nonlinear real equations and inequalities?
What would be the equivalent OCaml API call for the (! (...) :named p1) syntax of SMT2? Is it assert_and_track?
I am using Z3 version 4.3.2 from the ml-ng branch under Linux.
Many thanks!
The new ML API has been integrated into the unstable branch a couple months ago, and the ml-ng branch has been removed. A few bugfixes/extensions were added to it's worth updating.
assert_and_track does exactly what you suspect and it is internally translated to the first example given.
The difference in behavior is explained by (check-sat p1 p2 p3) which is missing p4 and p5. Once those are added, the two versions behave exactly the same and they produce the same unsat core.
Given the following simplified quantifiers, with the Z3 options set according to those generated by Boogie (full details below), I get "unknown" as a result:
(declare-fun F (Int) Bool)
(declare-fun G (Int) Bool)
(assert (forall ((x Int)) (! (and
(F x) (G x))
:pattern ((F x))
)))
(assert (not (forall ((x Int)) (! (and
(G x) (F x))
:pattern ((F x))
))))
(check-sat)
My understanding for what (I think) Z3 would do with this problem, is skolemise the existential (not forall), which would yield ground instances of both F and G. Given these in the e-graph, we should be able to instantiate the other quantifier, and get unsat. I can see that Z3 probably has to case-split to do this, but I would expect this case-splitting to take place after removing the quantifier and populating the e-graph.
Instead, the first quantifier doesn't get instantiated in the above problem. I've made a number of observations:
Swapping the order of the (F x) and (G x) terms in the first quantifier results in "unsat" without any quantifier instantiations (I suppose some simplification spots the similarity between the two quantified assertions?).
Swapping the order of the (G x) and (F x) terms in the second quantifier (as well as those in the first) results in "unsat" with a single quantifier instantiation (which is the behaviour I'd expect in general).
Changing the smt.case_split option affects the behaviour. Set to 3 (as chosen by Boogie) or 5, we get "unknown". Set to 0,1,2 or 4, I get "unsat".
It would be great to understand the scenarios above, and why (in the failing cases) these terms don't always make it to the e-graph after skolemisation. I'm not sure what the effects of changing the case_split option are in general. At the moment, I don't think Boogie allows that to be changed (and overrides any choice made on the command-line). But I have the feeling that the e-graph should get the information in all cases, ideally.
Here's the full file (removing most of the options set doesn't seem to make a difference to the failing cases, except for the smt.case_split one):
(set-option :print-success false)
(set-info :smt-lib-version 2.0)
(set-option :AUTO_CONFIG false)
;(set-option :MODEL.V2 true)
(set-option :smt.PHASE_SELECTION 0)
(set-option :smt.RESTART_STRATEGY 0)
(set-option :smt.RESTART_FACTOR |1.5|)
(set-option :smt.ARITH.RANDOM_INITIAL_VALUE true)
(set-option :smt.DELAY_UNITS true)
(set-option :NNF.SK_HACK true)
(set-option :smt.MBQI false)
(set-option :smt.QI.EAGER_THRESHOLD 100)
(set-option :smt.QI.COST |"(+ weight generation)"|)
(set-option :TYPE_CHECK true)
(set-option :smt.BV.REFLECT true)
(set-option :TIMEOUT 0)
(set-option :smt.QI.PROFILE true)
(set-option :smt.CASE_SPLIT 3)
; done setting options
(declare-fun F (Int) Bool)
(declare-fun G (Int) Bool)
(assert (forall ((x Int)) (! (and
(F x) (G x))
:pattern ((F x))
)))
(assert (not (forall ((x Int)) (! (and
(G x) (F x))
:pattern ((F x))
))))
(check-sat)
This is settled by the answer from https://stackoverflow.com/users/1096362/nikolaj-bjorner to this question:
Surprising behaviour when trying to prove a forall
The translation of the proof obligation into a disjunction, followed by corresponding relevancy of the clauses, explains why Z3 doesn't see both conjuncts as potential triggers.
Here is the input:
(set-option :auto-config false)
(set-option :mbqi false)
(declare-sort T6)
(declare-sort T7)
(declare-fun set23 (T7 T7) Bool)
(assert (forall ((bv1 T7) (bv0 T7))
(= (set23 bv0 bv1) (= bv0 bv1))))
(check-sat)
Note that Z3 does not timeout. It returns unknown almost instantaneously.
This returns unknown because you've disabled the quantifier handling mechanism (e.g., you disabled the model based quantifier instantiation module, MBQI), and it appears that with the options you've specified, Z3 does not have any other mechanisms enabled for deciding quantified formulas, so it figures out that it cannot decide your formula and returns unknown.
If you change that to true to enable MBQI, you will receive sat as expected. There are a variety of techniques (MBQI, quantifier elimination, etc.) for deciding quantified formulas in Z3 (see in the manual here for an overview: http://rise4fun.com/z3/tutorialcontent/guide#h28 ), e.g., for your example, you can also get sat as expected using the macro finder module (rise4fun link: http://rise4fun.com/Z3/NuQg ):
(set-option :auto-config false)
(set-option :mbqi false)
(set-option :macro-finder true)
(declare-sort T6)
(declare-sort T7)
(declare-fun set23 (T7 T7) Bool)
(assert (forall ((bv1 T7) (bv0 T7))
(= (set23 bv0 bv1) (= bv0 bv1))))
(check-sat) ; sat