When using Z3 on TPTP files (e.g. http://www.cs.miami.edu/~tptp/cgi-bin/SeeTPTP?Category=Problems&Domain=SYN&File=SYN054+1.p) is there a way to find out which axioms were used to prove the conjecture?
Alternatively, can Z3 produce TPTP proofs?
Cheers
Z3 includes limited TPTP support.
It does not track axiom names or produce proofs in the TPTP format.
Z3 offers rich support for SMT-LIB2 format and it produces proofs
in a format that can be digested by SMT-LIB2 parsers.
Related
After working with νZ, an extension within the SMT solver Z3 to make use of objective functions, I was surprised to find that the used optimization primitives are not part of the SMT-LIB2 syntax. These primitives are:
(maximize t) - instruct the solver to maximize t.
(minimize t) - instruct the solver to minimize t.
(assert-soft F :weight n) - assert soft constraint F, optionally with weight n.
This extension was introduced in 2014. The latest version of the SMT-LIB standard, version 2.6, still doesn't seem to introduce any kind of syntax to support objective functions. Is this really the case? If so, is there any work being done to introduce such a standard?
I'm not aware of any work to standardize Max-SAT (i.e., optimization) in SMTLib. Your best bet is to ask at the SMTLib mailing list (https://groups.google.com/g/smt-lib), to call for attention to this matter.
I am trying to devise ways to improve performance of z3 on my problems. I am aware of the the CAV'06 paper and the tech report . Do relevant parts of z3 v4.3.1 differ from what is described in these documents, and if so in what ways? Also, what is the strategy followed by default in z3 for deciding when to check for consistency in Linear Real Arithmetic, of the theory atoms corresponding to the decided (and propagated) propositional literals?
Linear arithmetic is implemented in the files at src/smt/theory_arith*.
See http://z3.codeplex.com/SourceControl/latest#src/smt/theory_arith_core.h
Regarding the paper you pointed out, the ideas are used in the implementation. However, the actual code contains many extensions for linear integer, nonlinear arithmetic and proof generation. If you only care about linear real arithmetic, you should focus only on theory_arith.h, theory_arith_core.h. The file theory_arith_aux.h also contains useful functionality.
My program, bounded synthesizer of reactive finite state systems, produces SMT queries to annotate a product automaton of the (uninterpreted) system and a specification. Essentially it is a model checking with uninterpreted functions. If the annotation exists => the model found by Z3 satisfies the spec. The queries contain:
datatype (to encode states of a system and of a specification automaton)
>= (greater), > (strictly) (to specify ranking function of states of automaton system*spec, which is used to search lassos with bad states)or in other words, ordering of states of that automaton, which
uninterpreted functions with boolean domain and range
all clauses are horn clauses
An example is https://dl.dropboxusercontent.com/u/444947/posts/full_arbiter2.smt2
('forall' are used to encode "don't care" inputs to functions)
Currently queries take strictly greater > operator from integers arithmetic (that is a ranking function has Int range).
Question: is it worth developing a custom theory solver in Z3 for such queries? It could exploit DFS based search of lassos which might be faster than integers theory solver (or diff-neg tactic).
Or Z3 already efficiently handles this? (efficiently means "comparable to graph-based search of lassos").
Arithmetic is not the bottleneck of your benchmark.
We can check that by using
valgrind --tool=callgrind z3 full_arbiter2.smt2
kcachegrind
Valgrind and kcachegrind are available in most Linux distros.
So, I don't think you will get a significant performance improvement if you implement a solver for order theory.
One bottleneck is the datatype theory. You may get a performance boost if you encode the types Q and T using Bit-vectors. Another bottleneck is quantifier reasoning. Have you tried to expand them before invoking Z3?
In Z3, the qe (quantifier elimination) tactic will essentially expand Boolean quantifiers.
I got a small speedup by replacing
(check-sat)
with
(check-sat-using (then qe smt))
I'm doing quantifier elimination on LIA using F# and Z3 3.2 API.
Z3 used to have QUANT_ARITH configuration which indicates the use of Cooper's method or the Omega test for LIA quantifier elimination. But that option was replaced by ELIM_QUANTIFIERS in Z3 2.6 (see Z3 release notes).
I want to ask internally how Z3 3.2 knows which method to use for quantifier elimination? Can users affect the choice of method like QUANT_ARITH before?
Furthermore, with the introduction of strategy specification language, will Z3 allow us to customize quantifier elimination by extending or combining these methods?
The quantifier elimination module was re-implemented. The new implementation should be faster and correct.
The latest Z3 does not have a implementation of Cooper’s method or Omega test.
You can find more details about the actual quantifier elimination procedure used in Z3 at:
“Linear Quantifier Elimination as an Abstract Decision Procedure, Nikolaj Bjørner, IJCAR 2010”.
Regarding the strategy specification language, we will eventually expose tactics for performing quantifier elimination.
We are currently working on this infrastructure, more news are coming soon.
Has anyone tried proving Z3 with Z3 itself?
Is it even possible, to prove that Z3 is correct, using Z3?
More theoretical, is it possible to prove that tool X is correct, using X itself?
The short answer is: “no, nobody tried to prove Z3 using Z3 itself” :-)
The sentence “we proved program X to be correct” is very misleading.
The main problem is: what does it mean to be correct.
In the case of Z3, one could say that Z3 is correct if, at least, it never returns “sat” for an unsatisfiable problem, and “unsat” for a satisfiable one.
This definition may be improved by also including additional properties such as: Z3 should not crash; the function X in the Z3 API has property Y, etc.
After we agree on what we are supposed to prove, we have to create models of the runtime, programming language semantics (C++ in the case of Z3), etc.
Then, a tool (aka verifier) is used to convert the actual code into a set of formulas that we should check using a theorem prover such as Z3.
We need the verifier because Z3 does not “understand” C++.
The Verifying C Compiler (VCC) is an example of this kind of tool.
Note that, proving Z3 to be correct using this approach does not provide a definitive guarantee that Z3 is really correct since our models may be incorrect, the verifier may be incorrect, Z3 may be incorrect, etc.
To use verifiers, such as VCC, we need to annotate the program with the properties we want to verify, loop invariants, etc. Some annotations are used to specify what code fragments are supposed to do. Other annotations are used to "help/guide" the theorem prover. In some cases, the amount of annotations is bigger than the program being verified. So, the process is not completely automatic.
Another problem is cost, the process would be very expensive. It would be much more time consuming than implementing Z3.
Z3 has 300k lines of code, some of this code is based on very subtle algorithms and implementation tricks.
Another problem is maintenance, we are regularly adding new features and improving performance. These modifications would affect the proof.
Although the cost may be very high, VCC has been used to verify nontrivial pieces of code such as the Microsoft Hyper-V hypervisor.
In theory, any verifier for programming language X can be used to prove itself if it is also implemented in language X.
The Spec# verifier is an example of such tool.
Spec# is implemented in Spec#, and several parts of Spec# were verified using Spec#.
Note that, Spec# uses Z3 and assumes it is correct. Of course, this is a big assumption.
You can find more information about these issues and Z3 applications on the paper:
http://research.microsoft.com/en-us/um/people/leonardo/ijcar10.pdf
No, it is not possible to prove that a nontrivial tool is correct using the tool itself. This was basically stated in Gödel's second incompleteness theorem:
For any formal effectively generated theory T including basic arithmetical truths and also certain truths about formal provability, if T includes a statement of its own consistency then T is inconsistent.
Since Z3 includes arithmetic, it cannot prove its own consistency.
Because it was mentioned in a comment above: Even if the user provides invariants, Gödels's theorem still applies. This is not a question of computability. The theorem states that no such prove can exist in a consistent system.
However you could verify parts of Z3 with Z3.
Edit after 5 years:
Actually the argument is easier than Gödel's incompleteness theorem.
Let's say Z3 is correct if it only returns UNSAT for unsatisfiable formulas.
Assume we find a formula A, such that if A is unsatisfiable then Z3 is correct (and we somehow have proven this relation).
We can give this formula to Z3, but
if Z3 returns UNSAT it could be because Z3 is correct or because of a bug in Z3. So we have not verified anything.
if Z3 returns SAT and a countermodel, we might be able to find a bug in Z3 by analyzing the model
otherwise we don't know anything.
So we can use Z3 to find bugs in Z3 and to improve confidence about Z3 (to an extremely high level), but not to formally verify it.