After working with νZ, an extension within the SMT solver Z3 to make use of objective functions, I was surprised to find that the used optimization primitives are not part of the SMT-LIB2 syntax. These primitives are:
(maximize t) - instruct the solver to maximize t.
(minimize t) - instruct the solver to minimize t.
(assert-soft F :weight n) - assert soft constraint F, optionally with weight n.
This extension was introduced in 2014. The latest version of the SMT-LIB standard, version 2.6, still doesn't seem to introduce any kind of syntax to support objective functions. Is this really the case? If so, is there any work being done to introduce such a standard?
I'm not aware of any work to standardize Max-SAT (i.e., optimization) in SMTLib. Your best bet is to ask at the SMTLib mailing list (https://groups.google.com/g/smt-lib), to call for attention to this matter.
Related
In many programming languages, branching efficiency is dependent on the order in which the clauses are provided. E.g., in Python,
if p or q :
will branch into the if statement as soon as p evaluates to true, so it is generally a good idea to provide computationally light clauses first. I'm wondering if the same is true for satisfiability checking in Z3. In other words, is there any difference between checking And(P, Q) and And(Q, P), provided that one of the formulas is significantly more complex than the other?
Yes, it make can make a difference what order you specify the clauses. However, there is probably not much to be gained by rearranging the clauses when it comes to Z3. Also, it might be difficult (and perhaps not worth your while) to determine a generic "optimal order" beforehand.
With other SMT solvers, I have observed what you are describing in an extreme fashion. With some solvers, it can make or break the computation, in the sense that I have seen certain cases where a solver times out when the clauses are presented in one order, but quickly finds a solution when the same clauses are presented in a different order. However, to me, this is an indication of a poorly designed solver. Z3 and other modern SMT solvers such as CVC4 are much less vulnerable to this kind of problem than older or less robust SMT solvers.
Regarding actually finding the optimal ordering, what constitutes "computationally light" or "more complex" for an SMT solver is different than a traditional computer language. What really matters is whether or not a certain clause in a conjunction is likely to lead to a conflict, or whether a particular clause in a disjunction is likely to lead to a solution. This is akin to trying to find your way out of a maze -- if you make a wrong turn early on, then you might spend a long time following a dead end, before you finally decided to go back to the beginning and take a different branch. Again, modern SMT and SAT solvers have techniques for mitigating this.
Also, When it comes to Z3 specifically, if Z3 could be made more efficient in real-world applications by sorting the clauses in terms of some human-understandable and generic measure of complexity, then that has probably already added as a pre-processing step to Z3. In general, for such complicated optimization questions where there are multiple factors at play (e.g. the particulars of the Z3 implementation) you have to try multiple things and benchmark over a suite of test examples relevant for your application.
As a bit of evidence for some of my claims above, I wrote this little example SMT problem:
(declare-fun p () Int)
(declare-fun q () Int)
(declare-fun n () Int)
(assert (> p 1))
(assert (> q 1))
(assert (and (= n 18679565357) (= n (* p q))))
(check-sat)
(get-value (p q n))
(exit)
Swapping the order of the clauses in the and statement makes very little difference on my system (less than 1%). If I break it up into two separate assertions, then the difference is larger, but to know if this difference holds in general (across different SMT problems of this class) I would have to run a benchmark suite with many such problems.
is there a way to express assumptions in Z3 (I am using the Z3Py library) such that the engine does not check their validity but takes them as underlying theories, just like in theorem proving?
For example, lets say that I have two unary functions with argument of type Real. I would like to tell the Z3 engine that for all input values, f1(t) is equal to f2(t).
Encoded in Z3Py that would look something like the following:
t = Real("t")
assumption1 = ForAll(t, f1(t) = f2(t)).
The problem with the presented code is that my assertion set is quite big and I use quantifiers (I am trying to prove satisfiability of a real-time system). If I add the above assertion to the set of the other assertions the checking procedure does not terminate.
is there a way to express assumptions in Z3 (I am using the Z3Py library) such that the engine does not check their validity but takes them as underlying theories, just like in theorem proving?
In fact, all assertions you add to Z3 are treated as what you call assumptions. Z3 checks satisfiability of the assertions, it does not check validity. To check validity of a formula F, you assert (not F), and check for satisfiability of (not F). If (not F) is unsat, then F is valid. If you have background axioms, you are essentially checking validity of Background => F, so you can check satisifiability of Background & (not F).
Whether Z3 terminates on your query depends on which combination of theories and quantifiers you use. The more features your queries combine the tougher it is.
For formulas over pure linear arithmetic or polynomial real arithmetic,
these are called LRA, LIA and NRA in the SMT-LIB classification (see smtlib.org) Z3 uses specialized decision procedures that have recently been added.
Yes, that's possible just as you describe it, but you will end up with quantifiers, which does of course mean that you're solving a harder problem and Z3 will behave differently (it's possible you end up using completely different solvers that don't even share much source code).
For the particular example given, it's possible to eliminate the quantifier cheaply because it has the form of a function definition (ForAll x . f(x) = ...), i.e., we can just replace all occurrences of f with the right hand side and then the quantifier is trivially satisfied. In Z3, this is done by the macro finder, which may be applied as a tactic (with name "macro-finder"), or if you are using the "smt" tactic (implicitly via others or directly), then you can set smt.macro_finder=true.
Can SMT solver efficiently find a solution (or an assignment) for the pseudo-Boolean problem as described as follows:
\sum {i..m} f_i x1 x2.. xn *w_i
where f_i x1 x2 .. xn is a Boolean function, and w_i is a weight of Int type.
For your convenience, I highlight the contents in page 1 and 3, which is enough for specifying
the pseudo-Boolean problem.
SMT solvers typically address the question: given a logical formula, optionally using functions and predicates from underlying theories (such as the theory of arithmetic, the theory of bit-vectors, arrays), is the formula satisfiable or not.
They typically don't expose a way for you specify objective functions
and typically don't have built-in optimization procedures.
Some special cases are formulas that only use Booleans or a combination of Booleans and either bit-vectors or integers. Pseudo Boolean constraints can be formulated with either integers or encoded (with some care taking overflow semantics into account) using bit-vectors, or they can be encoded directly into SAT. For some formulas using bounded integers that fall in the class of psuedo-boolean problems, Z3 will try automatic reductions into bit-vectors. This applies only to benchmkars in the SMT-LIB2 format tagged as QF_LIA or applies if you explicitly invoke a tactic that performs this reduction (the "qflia" tactic should apply).
While Z3 does not directly expose objective functions, the question of augmenting
SMT solvers with objective functions is actively pursued in the research community.
One approach suggested by Nieuwenhuis and Oliveras in SAT 2006 was to build in
solving for the "weighted max SMT" problem as a custom theory. Yices comes with built-in
features for weighted max SMT, Z3 does not, but it is possible to write a custom
theory that performs the backtracking search of a weighted max SMT solver, but nothing
out of the box.
Sometimes people try to specify objective functions using quantified formulas.
In theory one could hope that quantifier elimination procedures then can solve
for the objective.
This is generally pretty bad when it comes to performance. Quantifier elimination
is an overfit and the routines (that we have) will not be efficient.
For your problem, if you want to find an optimized (maximum or minimum) result from the sum, yes Z3 has this ability. You can use the Optimize class of Z3 library instead of Solver class. The class provides two methods for 'maximization' and 'minimization' respectively. You can pass the SMT variable that is needed to be optimized and Optimization class model will give the solution for you. It actually worked with C# API using Microsoft.Z3 library. For your inconvenience, I am attaching a snippet:
Optimize opt; // initializing object
opt.MkMaximize(*your variable*);
opt.MkMinimize(*your variable*);
opt.Assert(*anything you need to do*);
I'm doing quantifier elimination on LIA using F# and Z3 3.2 API.
Z3 used to have QUANT_ARITH configuration which indicates the use of Cooper's method or the Omega test for LIA quantifier elimination. But that option was replaced by ELIM_QUANTIFIERS in Z3 2.6 (see Z3 release notes).
I want to ask internally how Z3 3.2 knows which method to use for quantifier elimination? Can users affect the choice of method like QUANT_ARITH before?
Furthermore, with the introduction of strategy specification language, will Z3 allow us to customize quantifier elimination by extending or combining these methods?
The quantifier elimination module was re-implemented. The new implementation should be faster and correct.
The latest Z3 does not have a implementation of Cooper’s method or Omega test.
You can find more details about the actual quantifier elimination procedure used in Z3 at:
“Linear Quantifier Elimination as an Abstract Decision Procedure, Nikolaj Bjørner, IJCAR 2010”.
Regarding the strategy specification language, we will eventually expose tactics for performing quantifier elimination.
We are currently working on this infrastructure, more news are coming soon.
Has anyone tried proving Z3 with Z3 itself?
Is it even possible, to prove that Z3 is correct, using Z3?
More theoretical, is it possible to prove that tool X is correct, using X itself?
The short answer is: “no, nobody tried to prove Z3 using Z3 itself” :-)
The sentence “we proved program X to be correct” is very misleading.
The main problem is: what does it mean to be correct.
In the case of Z3, one could say that Z3 is correct if, at least, it never returns “sat” for an unsatisfiable problem, and “unsat” for a satisfiable one.
This definition may be improved by also including additional properties such as: Z3 should not crash; the function X in the Z3 API has property Y, etc.
After we agree on what we are supposed to prove, we have to create models of the runtime, programming language semantics (C++ in the case of Z3), etc.
Then, a tool (aka verifier) is used to convert the actual code into a set of formulas that we should check using a theorem prover such as Z3.
We need the verifier because Z3 does not “understand” C++.
The Verifying C Compiler (VCC) is an example of this kind of tool.
Note that, proving Z3 to be correct using this approach does not provide a definitive guarantee that Z3 is really correct since our models may be incorrect, the verifier may be incorrect, Z3 may be incorrect, etc.
To use verifiers, such as VCC, we need to annotate the program with the properties we want to verify, loop invariants, etc. Some annotations are used to specify what code fragments are supposed to do. Other annotations are used to "help/guide" the theorem prover. In some cases, the amount of annotations is bigger than the program being verified. So, the process is not completely automatic.
Another problem is cost, the process would be very expensive. It would be much more time consuming than implementing Z3.
Z3 has 300k lines of code, some of this code is based on very subtle algorithms and implementation tricks.
Another problem is maintenance, we are regularly adding new features and improving performance. These modifications would affect the proof.
Although the cost may be very high, VCC has been used to verify nontrivial pieces of code such as the Microsoft Hyper-V hypervisor.
In theory, any verifier for programming language X can be used to prove itself if it is also implemented in language X.
The Spec# verifier is an example of such tool.
Spec# is implemented in Spec#, and several parts of Spec# were verified using Spec#.
Note that, Spec# uses Z3 and assumes it is correct. Of course, this is a big assumption.
You can find more information about these issues and Z3 applications on the paper:
http://research.microsoft.com/en-us/um/people/leonardo/ijcar10.pdf
No, it is not possible to prove that a nontrivial tool is correct using the tool itself. This was basically stated in Gödel's second incompleteness theorem:
For any formal effectively generated theory T including basic arithmetical truths and also certain truths about formal provability, if T includes a statement of its own consistency then T is inconsistent.
Since Z3 includes arithmetic, it cannot prove its own consistency.
Because it was mentioned in a comment above: Even if the user provides invariants, Gödels's theorem still applies. This is not a question of computability. The theorem states that no such prove can exist in a consistent system.
However you could verify parts of Z3 with Z3.
Edit after 5 years:
Actually the argument is easier than Gödel's incompleteness theorem.
Let's say Z3 is correct if it only returns UNSAT for unsatisfiable formulas.
Assume we find a formula A, such that if A is unsatisfiable then Z3 is correct (and we somehow have proven this relation).
We can give this formula to Z3, but
if Z3 returns UNSAT it could be because Z3 is correct or because of a bug in Z3. So we have not verified anything.
if Z3 returns SAT and a countermodel, we might be able to find a bug in Z3 by analyzing the model
otherwise we don't know anything.
So we can use Z3 to find bugs in Z3 and to improve confidence about Z3 (to an extremely high level), but not to formally verify it.