In many programming languages, branching efficiency is dependent on the order in which the clauses are provided. E.g., in Python,
if p or q :
will branch into the if statement as soon as p evaluates to true, so it is generally a good idea to provide computationally light clauses first. I'm wondering if the same is true for satisfiability checking in Z3. In other words, is there any difference between checking And(P, Q) and And(Q, P), provided that one of the formulas is significantly more complex than the other?
Yes, it make can make a difference what order you specify the clauses. However, there is probably not much to be gained by rearranging the clauses when it comes to Z3. Also, it might be difficult (and perhaps not worth your while) to determine a generic "optimal order" beforehand.
With other SMT solvers, I have observed what you are describing in an extreme fashion. With some solvers, it can make or break the computation, in the sense that I have seen certain cases where a solver times out when the clauses are presented in one order, but quickly finds a solution when the same clauses are presented in a different order. However, to me, this is an indication of a poorly designed solver. Z3 and other modern SMT solvers such as CVC4 are much less vulnerable to this kind of problem than older or less robust SMT solvers.
Regarding actually finding the optimal ordering, what constitutes "computationally light" or "more complex" for an SMT solver is different than a traditional computer language. What really matters is whether or not a certain clause in a conjunction is likely to lead to a conflict, or whether a particular clause in a disjunction is likely to lead to a solution. This is akin to trying to find your way out of a maze -- if you make a wrong turn early on, then you might spend a long time following a dead end, before you finally decided to go back to the beginning and take a different branch. Again, modern SMT and SAT solvers have techniques for mitigating this.
Also, When it comes to Z3 specifically, if Z3 could be made more efficient in real-world applications by sorting the clauses in terms of some human-understandable and generic measure of complexity, then that has probably already added as a pre-processing step to Z3. In general, for such complicated optimization questions where there are multiple factors at play (e.g. the particulars of the Z3 implementation) you have to try multiple things and benchmark over a suite of test examples relevant for your application.
As a bit of evidence for some of my claims above, I wrote this little example SMT problem:
(declare-fun p () Int)
(declare-fun q () Int)
(declare-fun n () Int)
(assert (> p 1))
(assert (> q 1))
(assert (and (= n 18679565357) (= n (* p q))))
(check-sat)
(get-value (p q n))
(exit)
Swapping the order of the clauses in the and statement makes very little difference on my system (less than 1%). If I break it up into two separate assertions, then the difference is larger, but to know if this difference holds in general (across different SMT problems of this class) I would have to run a benchmark suite with many such problems.
Related
In general, First Order logic is Undecidable. However, Some fragments of first-order logic as Monadic logics, BSR Fragments, Separated Fragments are decidable.
There exist tools to solve SAT/SMT Solvers as Z3.
Is there any tool/Language which checks the satisfiability of FOL formulas?
SMT solvers, like Z3, can attempt to check satisfiability of FOL (even 2nd order logic!), though performance might not be great (depending on how the problem looks like)
There are also dedicated FOL provers (aka TPTP solvers), like Vampire, E, iProver, etc. See more here: https://en.wikipedia.org/wiki/Automated_theorem_proving
is there a way to express assumptions in Z3 (I am using the Z3Py library) such that the engine does not check their validity but takes them as underlying theories, just like in theorem proving?
For example, lets say that I have two unary functions with argument of type Real. I would like to tell the Z3 engine that for all input values, f1(t) is equal to f2(t).
Encoded in Z3Py that would look something like the following:
t = Real("t")
assumption1 = ForAll(t, f1(t) = f2(t)).
The problem with the presented code is that my assertion set is quite big and I use quantifiers (I am trying to prove satisfiability of a real-time system). If I add the above assertion to the set of the other assertions the checking procedure does not terminate.
is there a way to express assumptions in Z3 (I am using the Z3Py library) such that the engine does not check their validity but takes them as underlying theories, just like in theorem proving?
In fact, all assertions you add to Z3 are treated as what you call assumptions. Z3 checks satisfiability of the assertions, it does not check validity. To check validity of a formula F, you assert (not F), and check for satisfiability of (not F). If (not F) is unsat, then F is valid. If you have background axioms, you are essentially checking validity of Background => F, so you can check satisifiability of Background & (not F).
Whether Z3 terminates on your query depends on which combination of theories and quantifiers you use. The more features your queries combine the tougher it is.
For formulas over pure linear arithmetic or polynomial real arithmetic,
these are called LRA, LIA and NRA in the SMT-LIB classification (see smtlib.org) Z3 uses specialized decision procedures that have recently been added.
Yes, that's possible just as you describe it, but you will end up with quantifiers, which does of course mean that you're solving a harder problem and Z3 will behave differently (it's possible you end up using completely different solvers that don't even share much source code).
For the particular example given, it's possible to eliminate the quantifier cheaply because it has the form of a function definition (ForAll x . f(x) = ...), i.e., we can just replace all occurrences of f with the right hand side and then the quantifier is trivially satisfied. In Z3, this is done by the macro finder, which may be applied as a tactic (with name "macro-finder"), or if you are using the "smt" tactic (implicitly via others or directly), then you can set smt.macro_finder=true.
I'm experimenting with Z3 tactics. I noticed that the solver returned by
context.MkSimpleSolver()
has a different performance profile than the solver
context.MkSolver(context.MkTactic("smt"))
I understand that the "smt" tactic is supposed to perform the most general algorithm that Z3 has. So both these solvers should have identical output. Given this, why do they have different performance profiles?
I am testing this on a formula that is quantified over a bitvector. The quantifier body makes use of integers and bitvectors. The simple solver returns immediately with sat. The other one takes longer than I was willing to wait.
(If it helps here is a little background: I am trying to make Z3 bit-blast all integer terms to bitvectors hoping that this speeds things up. All integers involved are constrained to [0, 3]. I'm trying to get the bit-blast tactic to "take".)
My program, bounded synthesizer of reactive finite state systems, produces SMT queries to annotate a product automaton of the (uninterpreted) system and a specification. Essentially it is a model checking with uninterpreted functions. If the annotation exists => the model found by Z3 satisfies the spec. The queries contain:
datatype (to encode states of a system and of a specification automaton)
>= (greater), > (strictly) (to specify ranking function of states of automaton system*spec, which is used to search lassos with bad states)or in other words, ordering of states of that automaton, which
uninterpreted functions with boolean domain and range
all clauses are horn clauses
An example is https://dl.dropboxusercontent.com/u/444947/posts/full_arbiter2.smt2
('forall' are used to encode "don't care" inputs to functions)
Currently queries take strictly greater > operator from integers arithmetic (that is a ranking function has Int range).
Question: is it worth developing a custom theory solver in Z3 for such queries? It could exploit DFS based search of lassos which might be faster than integers theory solver (or diff-neg tactic).
Or Z3 already efficiently handles this? (efficiently means "comparable to graph-based search of lassos").
Arithmetic is not the bottleneck of your benchmark.
We can check that by using
valgrind --tool=callgrind z3 full_arbiter2.smt2
kcachegrind
Valgrind and kcachegrind are available in most Linux distros.
So, I don't think you will get a significant performance improvement if you implement a solver for order theory.
One bottleneck is the datatype theory. You may get a performance boost if you encode the types Q and T using Bit-vectors. Another bottleneck is quantifier reasoning. Have you tried to expand them before invoking Z3?
In Z3, the qe (quantifier elimination) tactic will essentially expand Boolean quantifiers.
I got a small speedup by replacing
(check-sat)
with
(check-sat-using (then qe smt))
I am experimenting with optimizing the use of Z3 for proving facts about a first-order theory. Currently, I specify a first-order theory in Python, ground the quantifiers there and send all the clauses along with the negation of the proof goal to Z3. I have the following idea that I hope could optimize the outcome: I only want to send the formulas in the theory to Z3 that are relevant to the proof goal. I will not discuss this concept in detail, but I think the intuition is simple: my theory is a conjunction of formulas, and I only want to send conjuncts that can possibly affect the truth value of the proof goal.
My question is the following: can this lead to an improvement in efficiency, or does Z3 already use a similar method? I would guess not, because I don't think that Z3 always assumes that the last assertion is the proof goal, so it has no way of optimizing this.
Yes, removing irrelevant facts can make a big difference. Suppose that we have a unsatisfiable formula of the form F_1 and F_2 and (not G). Moreover, let us assume that F_1 and (not G) is unsatisfiable, and F_2 is satisfiable. F_2 is what you call irrelevant. If there is a cheap way to remove F_2 before sending the formulat to Z3, it will probable make a big difference.
Z3 has heuristics for "ignoring" irrelevant facts, but they are just heuristics. For our example, the worst case scenario is a F_2 that is really hard for Z3 to satisfy. Z3 is essentially trying to build an interpretation/solution that satisfies the input formula (the formula F_1 an F_2 and (not G) in our working example). A formula is unsatisfiable when Z3 can show it is impossible to build the interpretation. In practice, the formula F_2 is irrelevant for Z3 only if it can quickly show it to be satisfiable, and the interpretation/solution for F_2 does not conflicts F_1 and (not G). If that is not the case, Z3 can waste a lot of resources with F_2.