In Z3 (Python) my SAT queries inside a loop are slowing down, can I use incremental SAT to overcome this problem?
The problem is the following: I am performing a concrete SAT search inside a loop. On each iteration, I get a model (of course, I store the negation of the model in order not to explore the same model again). And also, if that model satisfies a certain property, then I also add a subquery of it and add other restrictions to the formula. And iterate again, until UNSAT (i.e. "no more models") is obtained.
I offer an orientative snapshot of the code:
...
s = Solver()
s.add(True)
while s.check() == sat:
s.check()
m = s.model()
phi = add_modelNegation(m)
s.add(phi) #in order not to explore the same model again
if holds_property(m): #if the model holds a property
s = add_moreConstraints(s,m) #add other constrains to the formula
...
The question is that, as the formula that s has to solve gets greater, Z3 is starting to have more trouble to find those models. That is okay: this should happen, since finding a model is now more difficult because of the added restrictions. However, in my case, it is happening too much: the computation speed has been even halved; i.e. the time that the solver needs to find a new model is the double after some iterations.
Thus, I would like to implement some kind of incremental solving and wondered whether there are native methods in Z3 to do so.
I have been reading about this in many pages (see, for instance, How incremental solving works in Z3?), but only found this response in How to use incremental solving with z3py interesting:
The Python API is automatically "incremental". This simply means the ability to call the command check() multiple times, without the solver forgetting what it has seen before (i.e., call check(), assert more facts, call check() again; the second check() will take into account all the assertions from the very beginning).
I am not sure I understand, thus I make a simple question: that the response mean that the incremental SAT is indeed used in Z3's SAT? The point I think I am looking for another incrementality; for example: if in the SAT iteration number 230 it is inevitable that a variable (say b1) is true, then that is a fact that will not change afterwards, you can set it to 1, simplify the formula and not re-reason anything to do with b1, because all models if any will have b1. Is this incremental SAT of Z3 considering these kind of cases?
In case not, how could I implement this?
I know there are some implementations in PySat or in MiniSat, but I would like to do it in Z3.
As with anything related to performance of z3 solving, there's no one size fits all. Each specific problem can benefit from different ideas.
Incremental Solving The term "incremental solving" has a very specific meaning in the SAT/SMT context. It means that you can continue to add assertions to the system after a call to check, without it forgetting the assertions you added before hand. This is what makes it incremental. Additionally, you can set jump-points; i.e., you can tell the solver to "forget" the assertions you put in after a certain point in your program, essentially moving through a stack of assertions. For details, see Section 3.9 of https://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.6-r2021-05-12.pdf, specifically the part where it talks about the "Assertion Stack."
And, as noted before, you don't have to do anything specific for z3 to be incremental. It is incremental by default, i.e., you can simply add new assertions after calling check, or use push/pop calls etc. (Compare this to, for instance, CVC4; which is by default not incremental. If you want to use CVC4 in incremental mode, you have to pass a specific command line argument.) The main reason for this is that incremental mode requires extra bookkeeping, which CVC4 isn't willing to pay for unless you explicitly ask it to do so. For z3, the developers decided to always make it incremental without any command line switches.
Regarding your particular question about what happens if b1 is true: Well, if you somehow determined b1 is always true, simply assert it. And the solver will automatically take advantage of this; nothing special needs to be done. Note that z3 learns a ton of lemmas as it works through your program such as these and adds them to its internal database anyhow. But if you have some external mechanism that lets you deduce a particular constraint, just go ahead and add it. (Of course, the soundness of this will be on you, not on z3; but that's a different question.)
One specific "trick" in speeding up enumerating "find me all-solutions" loops like you are doing is to do a divide-and-conquer approach, instead of the "add the negation of the previous model and iterate." In practice this can make a significant difference in performance. I think you should try this idea. It's explained here: https://theory.stanford.edu/~nikolaj/programmingz3.html#sec-blocking-evaluations As you can see, the all_smt function defined at the end of that section takes specific advantage of incrementality (note the calls to push/pop) to speed up the model-search process, by carefully dividing the search space into disjoint segments, instead of doing a random-walk. Using this might give you the speed-up you need. But, again, as with anything performance specific, you'll need to tell us more about exactly what problem you are solving: None of these methods can avoid performance problems caused by modeling issues. (For instance, using integers to model booleans is one common pitfall.) See this answer for some generic advice: https://stackoverflow.com/a/57661441/936310
Related
I am trying to find an optimal solution using the Z3 API for python. I have used set_option("verbose", 1) to print statements that Z3 generates while checking for sat. One of the statements it prints is pb.conflict statements. The statements look something like this -
pb.conflict statements.
I want to know what exactly is pb.conflict. What do these statements signify? Also, what are the two numbers that get printed along with it?
pb stands for Pseudo-boolean. A pseudo-boolean function is a function from booleans to some other domain, usually Real. A conflict happens when the choice of a variable leads to an unsatisfiable clause set, at which point the solver has to backtrack. Keeping the backtracking to a minimum is essential for efficiency, and many of the SAT engines carefully track that number. While the details are entirely solver specific (i.e., those two numbers you're asking about), in general the higher the numbers, the more conflict cases the solver met, and hence might decide to reset the state completely or take some other action. Often, there are parameters that users can set to specify when such actions are taken and exactly what those are. But again, this is entirely solver and implementation specific.
A google search on pseudo-boolean optimization will result in a bunch of scholarly articles that you might want to peruse.
If you really want to find Z3's treatment of pseudo-booleans, then your best bet is probably to look at the implementation itself: https://github.com/Z3Prover/z3/blob/master/src/smt/theory_pb.cpp
I am using the python API of the Z3 solver to search for optimized schedules. It works pretty well apart from that it sometimes is very slow even for small graphs (sometimes its very quick though). The reason for that is probably that the constraints of my scheduling problem are quite complex.
I am trying to speed things up and stumbled on some articles about incremental solving.
As far I understood, you can use incremental solving to prune some of the search space by only applying parts of the constraints.
So my original code was looking like that:
for constraint in constraint_set:
self._opt_solver.add(constraint)
self._opt_solver.minimize(some_objective)
self._opt_solver.check()
model = self._opt_solver.mode()
I changed it now to the following:
for constraint in constraint_set:
self._opt_solver.push(constraint)
self._opt_solver.check()
self._opt_solver.minimize(some_objective)
self._opt_solver.check()
model = self._opt_solver.mode()
I basically substituted the "add" command by the "push" command and added a check() after each push.
So first of all: is my general approach correct?
Furthermore, I get an exception which I can't get rid of:
self._opt_solver.push(constraint) TypeError: push() takes 1 positional argument but 2 were given
Can anyone give me a hint, what I am doing wrong. Also is there maybe a z3py tutorial that explains (with some examples maybe) how to use incremental solving with the python api.
My last question is: Is that at all the right way of minimizing the execution time of the solver or is there a different/better way?
The function push doesn't take an argument. It creates a "backtracking" point that you can pop to later on. See here: http://z3prover.github.io/api/html/classz3py_1_1_solver.html#abc4ae989afee7ad164844640537107d9
So, it seems push isn't really what you want/need here at all. You should simply add your constraints one-by-one and call check. However, I very much doubt checking after each addition is going to speed anything up significantly. The optimizing solver (as opposed to the regular one), in particular, usually solves everything from scratch. (See the relevant discussion here: https://github.com/Z3Prover/z3/issues/1577)
Regarding incremental: The python API is automatically "incremental." Incremental simply means the ability to call the command check() multiple times, without the solver forgetting what it has seen before. (i.e., call check, assert more facts, call check again; the second check will take into account all the assertions from the very beginning.) You shouldn't make any assumptions regarding this will give you speed over calling check just once at the very end: It entirely depends on the heuristics and the decision procedures involved, which is dependent on the problem at hand.
Is there an incremental SMT solver or an API for some incremental SMT solver where I can add constraints incrementally, where I can uniquely identify each constraint by some label/name?
The reason I want to identify the constraints uniquely is so that I can drop them later by specifying that label/name.
The need for dropping constraints is due to the fact that my earlier constraints become irrelevant with time.
I see that with Z3 I cannot use the push/pop based incremental approach because it follows a stack based idea whereas my requirement is to drop specific earlier/old constraints.
With the other incremental approach of Z3 based on assumptions, I would have to perform check-sat of the format "(check-sat p1 p2 p3)" i.e. if I had three assertions to check then I would require three boolean constants p1,p2,p3, but in my implementation I would have thousands of assertions to check at a time, indirectly requiring thousands of boolean constants.
I also checked JavaSMT, a Java API for SMT solvers, to see if the API provides some better way of handling this requirement, but I see only way to add constraints by "addConstraint" or "push" and was unable to find any way of dropping or removing specific constraints since the pop is the only option available.
I would like to know if there is any incremental solver where I can add or drop constraints uniquely identified by names, or an API where there is an alternative way to handle it. I would appreciate any suggestion or comments.
The "stack" based approach is pretty much ingrained into SMTLib, so I think it'll be tough to find a solver that does exactly what you want. Although I do agree it would be a nice feature.
Having said that, I can think of two solutions. But neither will serve your particular use-case well, though they will both work. It comes down to the fact that you want to be able to cherry-pick your constraints at each call to check-sat. Unfortunately this is going to be expensive. Each time the solver does a check-sat it learns a lot of lemmas based on all the present assertions, and a lot of internal data-structures are correspondingly modified. The stack-based approach essentially allows the solver to "backtrack" to one of those learned states. But of course, that does not allow cherry-picking as you observed.
So, I think you're left with one of the following:
Using check-sat-assuming
This is essentially what you described already. But to recap, instead of asserting booleans, you simply give them names. So, this:
(assert complicated_expression)
becomes
; for each constraint ci, do this:
(declare-const ci Bool)
(assert (= ci complicated_expression))
; then, check with whatever subset you want
(check-sat-assuming (ci cj ck..))
This does increase the number of boolean constants you have to manage, but in a sense these are the "names" you want anyhow. I understand you do not like this as it introduces a lot of variables; and that is indeed the case. And there's a good reason for that. See this discussion here: https://github.com/Z3Prover/z3/issues/1048
Using reset-assertions and :global-declarations
This is the variant that allows you to arbitrarily cherry-pick the assertions at each call to check-sat. But it will not be cheap. In particular, the solver will forget everything it learned each time you follow this recipe. But it will do precisely what you wanted. First issue:
(set-option :global-declarations true)
And somehow keep track of all these yourself in your wrapper. Now, if you want to arbitrarily "add" a constraint, you don't need to do anything. Just add it. If you want to remove something, then you say:
(reset-assertions)
(assert your-tracked-assertion-1)
(assert your-tracked-assertion-2)
;(assert your-tracked-assertion-3) <-- Note the comment, we're skipping
(assert your-tracked-assertion-4)
..etc
etc. That is, you "remove" the ones you don't want. Note that the :global-declarations call is important since it'll make sure all your data-declarations and other bindings stay intact when you call reset-assertions, which tells the solver to start from a clean-slate of what it assumed and learned.
Effectively, you're managing your own constraints, as you wanted in the first place.
Summary
Neither of these solutions is precisely what you wanted, but they will work. There's simply no SMTLib compliant way to do what you want without resorting to one of these two solutions. Individual solvers, however, might have other tricks up their sleeve. You might want to check with their developers to see if they might have something custom for this use case. While I doubt that is the case, it would be nice to find out!
Also see this previous answer from Nikolaj which is quite related: How incremental solving works in Z3?
I am currently dealing with a situation where the assertions given to Z3 contain a large number of inequalities and equalities. They are dependent on each other in a way that it is most efficient to start solving the formula by assigning values to the variables used in the equalities.
Is there a way to alter the heuristics of Z3 such that the solver always chooses to "start" at these formulas?
My guess would be to use a tactic which initially processes a goal containing the mentioned equalities. It would then continue with the other assertions, restarting the whole process if necessary.
However, I'm not sure how to go about implementing this - how can I create custom goals from sets of formulas?
You can try asserting the first set of formulas you want, then issue check-sat, issue the next set, issue check-sat; repeat as necessary. You can also use push-pop to go back to these points if you want.
By issuing multiple check-sats this way, you would be forcing the solver to explore the formulas you asserted up to that point. Whether this would actually achieve what you want of course depends on exactly what your formulas look like and how much the solver can derive at each check-sat call.
since current version, there is some problem in "ctx-solver-simplify"like in the example http://rise4fun.com/Z3/CqRv z3 gives the wrong answer. I replace "ctx-solver-simplify" by "simplify" like http://rise4fun.com/Z3/x9X4
I am wondering, what's the difference between these two tactic "simplify" and "ctx-solver-simplify"?
The tactic simplify only performs "local simplifications". For every term t, we have that simplify(t) is a new term equivalent to t. Moreover, the result of simplify(t) does not depend on the context where t occurs. By context, I meant the assertion F where t occurs and all other assertions. Since, simplify is local, it is very efficient. The implementation is essentially based on a bottom up application of simplification rules. Moreover, since the result of simplify(t) does not depend on contextual information, we can cache it. Thus, even if t occurs N times in a formula F, we only need to simplify it once. All builtin solvers in Z3 apply this kind of simplification. Thus, tactics such as simplify have been extensively tested.
The tactic ctx-solver-simplify uses the context where t occurs to apply simplifications. The basic idea is to simplify a formula F by traversing it using a solver S. The solver S essentially contains the "context". Whenever S.check() returns unsat, we know the current context is inconsistent, then we can replace the current formula by false. The ctx-solver-simplify is much more expensive. First, it performs many calls to S.check(). Each one of these calls is potentially very expensive. It is also much harder to cache intermediate results. Z3 may have to simplify a subformula t many times because it occurs in different contexts.
The bug you reported in your question have been fixed. The fix will be available in the next release (version 4.1). If you need we can provide you a pre-release version of Z3 4.1