I'm having trouble figuring out how to debug z3. Is there a way to see what the SMT engine is "trying" to make it easier to understand why it's failing to see a solution that seems obvious and where it's devoting it's time instead?
As an example in my particular circumstance, I'm working with a recursive function and setting z3 to find inputs where the function has a certain result. SMT was timing out, yadda yadda yadda, turns out the thing I was recursing on had a base case of 0, but if it ever went negative, it'd recurse forever. Z3 didn't know not to pick a negative number, so it'd get stuck. I figured that out by staring at the code, but if I had some output somewhere that said "trying i == -10, trying i == -11, etc" it'd be very obvious what was going wrong.
I'm continuing to have less obvious issues, and I suspect Z3 is still getting stuck in loops. How can I see the loop it's getting stuck in?
It is unfortunately very difficult to find out why exactly Z3 is running forever, but typical culprits are matching loops due to bad patterns (a quantifier instantiations yields new ground terms that trigger another instantiations, and so on) and non-linear arithmetic.
The Z3 axiom profiler, described in this paper can help with identifying problems due to too many quantifier instantiations.
Related
My understanding from reading the literature is that assert statements are only for the user to be able to see what the verifier thinks a certain point. However in my own experience I have seen that in the case of lemmas, an assert statement actually changes what the verifier thinks and allows a lemma to be proved.
Can anyone clarify?
Thank you
Yes, adding assert statements can help Dafny prove lemmas, and this is typically how a programmer guides Dafny's proof search. In particular, Dafny might be able to prove a given intermediate statement, and it might be able to use that statement to prove a desired lemma, but it wouldn't know to look for that statement without the programmer providing it.
Experience with Dafny helps build intuition about what kind of assertions are helpful to Dafny. (I suggest reading about triggers, which will illuminate how Dafny proof search works. Triggers are patterns that Dafny uses as hints to know when to instantiate a quantified variable; the art of using assertions to guide Dafny is often about finding the right triggers.)
I am trying to find an optimal solution using the Z3 API for python. I have used set_option("verbose", 1) to print statements that Z3 generates while checking for sat. One of the statements it prints is pb.conflict statements. The statements look something like this -
pb.conflict statements.
I want to know what exactly is pb.conflict. What do these statements signify? Also, what are the two numbers that get printed along with it?
pb stands for Pseudo-boolean. A pseudo-boolean function is a function from booleans to some other domain, usually Real. A conflict happens when the choice of a variable leads to an unsatisfiable clause set, at which point the solver has to backtrack. Keeping the backtracking to a minimum is essential for efficiency, and many of the SAT engines carefully track that number. While the details are entirely solver specific (i.e., those two numbers you're asking about), in general the higher the numbers, the more conflict cases the solver met, and hence might decide to reset the state completely or take some other action. Often, there are parameters that users can set to specify when such actions are taken and exactly what those are. But again, this is entirely solver and implementation specific.
A google search on pseudo-boolean optimization will result in a bunch of scholarly articles that you might want to peruse.
If you really want to find Z3's treatment of pseudo-booleans, then your best bet is probably to look at the implementation itself: https://github.com/Z3Prover/z3/blob/master/src/smt/theory_pb.cpp
I am using the python API of the Z3 solver to search for optimized schedules. It works pretty well apart from that it sometimes is very slow even for small graphs (sometimes its very quick though). The reason for that is probably that the constraints of my scheduling problem are quite complex.
I am trying to speed things up and stumbled on some articles about incremental solving.
As far I understood, you can use incremental solving to prune some of the search space by only applying parts of the constraints.
So my original code was looking like that:
for constraint in constraint_set:
self._opt_solver.add(constraint)
self._opt_solver.minimize(some_objective)
self._opt_solver.check()
model = self._opt_solver.mode()
I changed it now to the following:
for constraint in constraint_set:
self._opt_solver.push(constraint)
self._opt_solver.check()
self._opt_solver.minimize(some_objective)
self._opt_solver.check()
model = self._opt_solver.mode()
I basically substituted the "add" command by the "push" command and added a check() after each push.
So first of all: is my general approach correct?
Furthermore, I get an exception which I can't get rid of:
self._opt_solver.push(constraint) TypeError: push() takes 1 positional argument but 2 were given
Can anyone give me a hint, what I am doing wrong. Also is there maybe a z3py tutorial that explains (with some examples maybe) how to use incremental solving with the python api.
My last question is: Is that at all the right way of minimizing the execution time of the solver or is there a different/better way?
The function push doesn't take an argument. It creates a "backtracking" point that you can pop to later on. See here: http://z3prover.github.io/api/html/classz3py_1_1_solver.html#abc4ae989afee7ad164844640537107d9
So, it seems push isn't really what you want/need here at all. You should simply add your constraints one-by-one and call check. However, I very much doubt checking after each addition is going to speed anything up significantly. The optimizing solver (as opposed to the regular one), in particular, usually solves everything from scratch. (See the relevant discussion here: https://github.com/Z3Prover/z3/issues/1577)
Regarding incremental: The python API is automatically "incremental." Incremental simply means the ability to call the command check() multiple times, without the solver forgetting what it has seen before. (i.e., call check, assert more facts, call check again; the second check will take into account all the assertions from the very beginning.) You shouldn't make any assumptions regarding this will give you speed over calling check just once at the very end: It entirely depends on the heuristics and the decision procedures involved, which is dependent on the problem at hand.
I'm experimenting with Z3 (using the python api) where I'm building up a scheduling model for a class assignment, where I have to use modulo quite often (because its periodic). Modulo seems already to slow down z3 by a lot, but if I try do some optimization on top (minimize a cost function which is a sum), then it takes quite some time for fairly small problems.
Without optimization it works okayish (few seconds for a smaller problem). So that being said, I have now 2 questions:
1) Is there any trick with the modulo function of how to use it? I already assign the modulo value to a function. Or is there any other way to express periodic/ring behavior?
2) I am not interested in finding THE best solution. A good one, will be good enough. Is there a way to set some bounds for the cost function. Like, if know the upper and lower bound of it? Any other tricks, where I could use domain knowledge to find solutions fast.
Furthermore, I thought that if I ll use timeout option solver.set("timeout" 10000), then the solver would time out with the best solution so far. That doesnt seem to be the case. It just times out.
Impossible to comment on the mod function without seeing some code. But as a rule of thumb, division is difficult. Are you using Int values, or bit-vectors? If you are using unbounded integers, you might want to try bit-vectors which might benefit from better internal heuristics. Or try Real values, and then do a proper reduction.
Regarding how to get the "best so far" optimal value, see this answer: Finding suboptimal solution (best solution so far) with Z3 command line tool and timeout
Instead of using the built-in modulo and division, you could introduce uninterpreted functions mymod and mydiv, for which you only provide the necessary axioms of division and modulo that your problem requires. If I remember correctly, Microsoft's Ironclad and/or Ironfleet team did that when they had modulo/division-related performance problems (using the pipeline Dafny -> Boogie -> Z3).
I'm trying to make z3 (I'm using z3py) to simplify some formulas for me (so that I can have more or less human-readable output). Using ctx-solver-simplify tactic seemed a good choice for me since in a couple of passes it would produce nice compact formulas. But soon I've run into a situation when the output of ctx-solver-simplify does not seem to be equivalent to the original formula (it looks more like being satisfiability-equivalent or so). Also, it might be the case that I'm not dealing with tactics correctly.
Here's what I was trying to do: http://rise4fun.com/Z3Py/g5sX. So, I construct a formula Set2 (everything before the definition of Set2 is just a setup needed to define it) which has a particular satisfying assignment. After applying ctx-solver-simplify, I get a single formula (as a goal) for which this assignment is not satisfying. So what am I dong wrong?
Am I wrong assuming that ctx-solver-simplify would produce an equivalent formula?
Am I handling the tactics and their output in the wrong way?
Anything else?
Thanks.
I have been looking into this, but have so far been unable to reproduce the bug directly
with our current branch. A bug in the context simplifier was fixed a little while ago, and it could
be manifesting itself with the online version of Z3.
There are still a few things I can do to double check if we can reproduce the bug
and I will update this post with what I find.