I have a question regarding how Z3 incrementally solves problems. After reading through some answers here, I found the following:
There are two ways to use Z3 for incremental solving: one is push/pop(stack) mode, the other is using assumptions. Soft/Hard constraints in Z3.
In stack mode, z3 will forget all learned lemmas in global (am I right?) scope even after one local "pop" Efficiency of constraint strengthening in SMT solvers
In assumptions mode (I don't know the name, that is the name that comes to my mind), z3 will not simplify some formulas, e.g. value propagation. z3 behaviour changing on request for unsat core
I did some comparison (you are welcome to ask for the formulas, they are just too large to put on the rise4fun), but here are my observations: On some formulas, including quantifiers, the assumptions mode is faster. On some formulas with lots of boolean variables (assumptions variables), stack mode is faster than assumptions mode.
Are they implemented for specific purposes? How does incremental solving work in Z3?
Yes, there are essentially two incremental modes.
Stack based: using push(), pop() you create a local context, that follows a stack discipline. Assertions added under a push() are removed after a matching pop(). Furthermore, any lemmas that are derived under a push are removed. Use push()/pop() to emulate freezing a state and adding additional constraints over the frozen state, then resume to the frozen state. It has the advantage that any additional memory overhead (such as learned lemmas) built up within the scope of a push() is released. The working assumption is that learned lemmas under a push would not be useful any longer.
Assumption based: using additional assumption literals passed to check()/check_sat() you can (1) extract unsatisfiable cores over the assumption literals, (2) gain local incrementality without garbage collecting lemmas that get derived independently of the assumptions. In other words, if Z3 learns a lemma that does not contain any of the assumption literals it expects to not garbage collect them. To use assumption literals effectively, you would have to add them to formulas too. So the tradeoff is that clauses used with assumptions contain some amount of bloat. For example if you want to locally assume some formula (<= x y), then you add a clause (=> p (<= x y)), and assume p when calling check_sat(). Note that the original assumption was a unit. Z3 propagates units efficiently. With the formulation that uses assumption literals it is no longer a unit at the base level of search. This incurs some extra overhead. Units become binary clauses, binary clauses become ternary clauses, etc.
The differentiation between push/pop functionality holds for Z3's default SMT engine. This is the engine most formulas will be using. Z3 contains some portfolio of engines. For example, for pure bit-vector problems, Z3 may end up using the sat based engine. Incrementality in the sat based engine is implemented differently from the default engine. Here incrementality is implemented using assumption literals. Any assertion you add within the scope of a push is asserted as an implication (=> scope_literals formula). check_sat() within such a scope will have to deal with assumption literals. On the flip-side, any consequence (lemma) that does not depend on the current scope is not garbage collected on pop().
In optimization mode, when you assert optimization objectives, or when you use the optimization objects over the API, you can also invoke push/pop. Likewise with fixedpoints. For these two features, push/pop are essentially for user-convenience. There is no internal incrementality. The reason is that these two modes use substantial pre-processing that is super non-incremental.
Related
In Z3 (Python) my SAT queries inside a loop are slowing down, can I use incremental SAT to overcome this problem?
The problem is the following: I am performing a concrete SAT search inside a loop. On each iteration, I get a model (of course, I store the negation of the model in order not to explore the same model again). And also, if that model satisfies a certain property, then I also add a subquery of it and add other restrictions to the formula. And iterate again, until UNSAT (i.e. "no more models") is obtained.
I offer an orientative snapshot of the code:
...
s = Solver()
s.add(True)
while s.check() == sat:
s.check()
m = s.model()
phi = add_modelNegation(m)
s.add(phi) #in order not to explore the same model again
if holds_property(m): #if the model holds a property
s = add_moreConstraints(s,m) #add other constrains to the formula
...
The question is that, as the formula that s has to solve gets greater, Z3 is starting to have more trouble to find those models. That is okay: this should happen, since finding a model is now more difficult because of the added restrictions. However, in my case, it is happening too much: the computation speed has been even halved; i.e. the time that the solver needs to find a new model is the double after some iterations.
Thus, I would like to implement some kind of incremental solving and wondered whether there are native methods in Z3 to do so.
I have been reading about this in many pages (see, for instance, How incremental solving works in Z3?), but only found this response in How to use incremental solving with z3py interesting:
The Python API is automatically "incremental". This simply means the ability to call the command check() multiple times, without the solver forgetting what it has seen before (i.e., call check(), assert more facts, call check() again; the second check() will take into account all the assertions from the very beginning).
I am not sure I understand, thus I make a simple question: that the response mean that the incremental SAT is indeed used in Z3's SAT? The point I think I am looking for another incrementality; for example: if in the SAT iteration number 230 it is inevitable that a variable (say b1) is true, then that is a fact that will not change afterwards, you can set it to 1, simplify the formula and not re-reason anything to do with b1, because all models if any will have b1. Is this incremental SAT of Z3 considering these kind of cases?
In case not, how could I implement this?
I know there are some implementations in PySat or in MiniSat, but I would like to do it in Z3.
As with anything related to performance of z3 solving, there's no one size fits all. Each specific problem can benefit from different ideas.
Incremental Solving The term "incremental solving" has a very specific meaning in the SAT/SMT context. It means that you can continue to add assertions to the system after a call to check, without it forgetting the assertions you added before hand. This is what makes it incremental. Additionally, you can set jump-points; i.e., you can tell the solver to "forget" the assertions you put in after a certain point in your program, essentially moving through a stack of assertions. For details, see Section 3.9 of https://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.6-r2021-05-12.pdf, specifically the part where it talks about the "Assertion Stack."
And, as noted before, you don't have to do anything specific for z3 to be incremental. It is incremental by default, i.e., you can simply add new assertions after calling check, or use push/pop calls etc. (Compare this to, for instance, CVC4; which is by default not incremental. If you want to use CVC4 in incremental mode, you have to pass a specific command line argument.) The main reason for this is that incremental mode requires extra bookkeeping, which CVC4 isn't willing to pay for unless you explicitly ask it to do so. For z3, the developers decided to always make it incremental without any command line switches.
Regarding your particular question about what happens if b1 is true: Well, if you somehow determined b1 is always true, simply assert it. And the solver will automatically take advantage of this; nothing special needs to be done. Note that z3 learns a ton of lemmas as it works through your program such as these and adds them to its internal database anyhow. But if you have some external mechanism that lets you deduce a particular constraint, just go ahead and add it. (Of course, the soundness of this will be on you, not on z3; but that's a different question.)
One specific "trick" in speeding up enumerating "find me all-solutions" loops like you are doing is to do a divide-and-conquer approach, instead of the "add the negation of the previous model and iterate." In practice this can make a significant difference in performance. I think you should try this idea. It's explained here: https://theory.stanford.edu/~nikolaj/programmingz3.html#sec-blocking-evaluations As you can see, the all_smt function defined at the end of that section takes specific advantage of incrementality (note the calls to push/pop) to speed up the model-search process, by carefully dividing the search space into disjoint segments, instead of doing a random-walk. Using this might give you the speed-up you need. But, again, as with anything performance specific, you'll need to tell us more about exactly what problem you are solving: None of these methods can avoid performance problems caused by modeling issues. (For instance, using integers to model booleans is one common pitfall.) See this answer for some generic advice: https://stackoverflow.com/a/57661441/936310
The SMT-LIB standard enables arbitrary attributes, but prescribes only very few, e.g. :pattern. Z3, on the other hand, currently only supports a few selected attributes, and issues a warning for unrecognised ones.
Which attributes are supported, and what is their typical use case?
Attributes
:named: named terms can be included in unsat cores
:weight: heavier quantifiers make Z3 reach its quantifier instantiation depth threshold more quickly
:qid: identify quantifiers, e.g. when obtaining instantiation statistics
:pattern: syntactic hints for when to instantiate a quantifier in e-matching
:no-pattern: prevent Z3 from using certain terms when inferring patterns
:ex-act: seems like dead code, to be removed
:skolemid: specific to VCC/Boogie use
:lblneg, :lblpos: associated with Boogie labels to track the source of counter-examples
Notes
Information partly provided by Z3's very own Nikolaj Bjorner, see also Z3 issue #4536.
See also Z3 source file smt2parser.cpp
since current version, there is some problem in "ctx-solver-simplify"like in the example http://rise4fun.com/Z3/CqRv z3 gives the wrong answer. I replace "ctx-solver-simplify" by "simplify" like http://rise4fun.com/Z3/x9X4
I am wondering, what's the difference between these two tactic "simplify" and "ctx-solver-simplify"?
The tactic simplify only performs "local simplifications". For every term t, we have that simplify(t) is a new term equivalent to t. Moreover, the result of simplify(t) does not depend on the context where t occurs. By context, I meant the assertion F where t occurs and all other assertions. Since, simplify is local, it is very efficient. The implementation is essentially based on a bottom up application of simplification rules. Moreover, since the result of simplify(t) does not depend on contextual information, we can cache it. Thus, even if t occurs N times in a formula F, we only need to simplify it once. All builtin solvers in Z3 apply this kind of simplification. Thus, tactics such as simplify have been extensively tested.
The tactic ctx-solver-simplify uses the context where t occurs to apply simplifications. The basic idea is to simplify a formula F by traversing it using a solver S. The solver S essentially contains the "context". Whenever S.check() returns unsat, we know the current context is inconsistent, then we can replace the current formula by false. The ctx-solver-simplify is much more expensive. First, it performs many calls to S.check(). Each one of these calls is potentially very expensive. It is also much harder to cache intermediate results. Z3 may have to simplify a subformula t many times because it occurs in different contexts.
The bug you reported in your question have been fixed. The fix will be available in the next release (version 4.1). If you need we can provide you a pre-release version of Z3 4.1
I am running Z3 on UFBV queries. Currently the query contains 2 calls check-sat.
If I put push 1 just after check-sat, Z3 solves the query in 30sec. If I don't put any push 1 at all, thus have two calls check-sat without any push 1 between them, then Z3 solves it in 200sec.
Interesting. Any specific reasons or just a coincidence?
Z3 3.x has a "strategy specification language" based on tactics and tacticals. I'm not "advertising" that yet because it is working in progress.
The basic idea is described in this slide deck.
We have a different built-in strategy for each logic. The strategies usually do not support incremental solving, because they may apply transformations that use a "closed-world" assumption. Example, we have transformations that map 0-1 linear integer arithmetic into SAT. Whenever Z3 detects that the user "wants" incremental solving (e.g., multiple check-sat commands, push&pop commands), it switches to a general purpose solver. In future versions, we will provide more features for controlling Z3 behavior.
BTW, if you have two consecutive (check-sat) (check-sat) commands, Z3 not necessarily enters in incremental mode. It will enter only if there is an assert or push command between the two calls.
Now, suppose your query is of the form: (check-sat) <assertions> (check-sat), and your second query is of the form (check-sat) <assertions> (push) (check-sat). In both cases, Z3 will be in incremental mode in the second (check-sat). However, the behavior is still not the same. The incremental solver "compiles" the asserted formulas to an internal format, and its behavior is affected if a push command has been executed. For example, it will use a more efficient encoding of binary clauses only if there is no user scope. By user scope, I mean, the number of push commands - number of pop commands. It does that because the data-structure used in the more efficient encoding does not have an efficient undo/inverse operation.
I am currently experimenting with Z3 as bounded engine for specifications written in Alloy (a relational logic/language). I am using the UFBV as target language.
I detect a problem using the Z3 option (set-option :pull-nested-quantifiers true).
For two semantically identical SMT specifications Spec1 and Spec2, Z3 times out (200 sec) for proving Spec1 but proves Spec2.
The only different between Spec1 and Spec2 is that they have different function identifiers (because I use java hash names). Can this be related to a bug?
The second observation I would like to share and discuss, is the "iff" operator in the context of UFBV. This operator is not supported, if (set-logic UFBV) is set. My solution was to use "=" instead but this do not work well if the operands contains deeply nested quantifiers and the "pull-nested-quantifiers" is set. The other saver solution is to use double implication.
Now the question:
Is there any other better solution for model "iff" in UFBV, because I think, that using double implication will in general loose maybe useable semantic information for improvement/simplifications.
http://i12www.ira.uka.de/~elghazi/tmp/
you can find: spec1 and spec2 the tow (I think) semantically identical SMT specifications, and spec3 an SMT specification using "=" to model "iff", for which z3 times out.
The default strategy for the UFBV logic is not effective for your problems. Actually, the default strategy solves all of them in less than 1 sec. To force Z3 to use the default strategy, you just need to comment the following lines in your script.
; (set-logic UFBV)
; (set-option :pull-nested-quantifiers true)
; (set-option :macro-finder true)
If the warning messages are bothering you, you can add:
(set-option :print-warning false)
That being said, I will try to address the issues you raised.
Does identifier names affect the behavior of Z3? Yes, they do.
Starting at version 3.0, we started using a total order on Z3 expressions for performing operations such as: sorting the arguments of associative-commutative operators.
This total order is based on the identifier names.
Ironically, this modification was motivated by user feedback. In previous versions, we used an internal ID for performing operations such as sorting, and breaking ties in many different heuristics. However, these IDs are based on the order Z3 creates/deletes expressions, which is based on the order users declare symbols. So, Z3 2.x behavior would be affected by trivial modifications such as removing unused declarations.
Regarding iff, it is not part of SMT-LIB 2.0 standard. In SMT-LIB 2.0, = is used for formulas and terms. To make sure Z3 is fully compliant with the SMT-LIB 2.0 standard, whenever users specify a SMT-LIB supported logic (or soon to be supported such as UFBV), Z3 only "loads" the symbols defined in it. When, a logic is not specified, Z3 assumes the user is using the "Z3 logic" that contains all supported theories in Z3, and many extra aliases such as: iff for Boolean =, if for ite, etc.