I'm evaluating the use Dafny, compared to the direct use of z3 for encoding a problem we have in my company.
While trying out the z3 API I noticed that there is an Array type, which seems to resemble a lot the way Z3 implements total functions. It seems like there is not an easy way to reason on the Lenght of such arrays in Z3.
If we look at Dafny, it can easily prove the following correct
method Find(a: array<int>) returns (index: int)
ensures index == a.Length
{
index := a.Length;
}
Now, for me the question is:
How is the array used by Dafny encoded in Z3?
Let's say that Dafny doesn't actually use Z3 arrays, where can I find some documentation on the way Dafny abstract things that do not identify a bounded number of variables in Z3 (such as array)? Same question applies for any object that is heap allocated, does it use allocation-site abstractions or something else?
There's really no magic, nor anything "implicit" that guarantees all array accesses are within bounds. You can surely write Dafny programs that index out-of-bounds, you just cannot prove useful things for them. Dafny makes you do this, by generating the necessary "in-bounds" check; but it's still up to you to have the proof go through by putting in the necessary annotations. Is that what you're looking for? This is also explained here: https://rise4fun.com/Dafny/tutorialcontent/guide#h27
Related
I am wondering what datatype I should be using for my z3 application. My understanding is that the only options for integer-array-like data structures are Array(IntSort(), IntSort()) and IntVector().
Reasons I think Arrays are overkill: Each array element is only written once, I'm not doing anything like Store((Store(X, y, z1)), y, z2). In addition, each array has a predefined length of <= 256 (and each integer in the array is between 0 and 63).
Reasons I think BitVectors won't work: I want to use Int variables to index into the arrays. For instance, I might have z = Int('z'), some clauses constraining z, and then Or(arr[z] == 2, arr[z + 1] == 2). My understanding after playing around with z3 and reading up is that vectors don't support this.
Is there a way I can get the power of variable-indexing without having to use expensive Array operations?
If you have small arrays of fixed-length with no symbolic index access, then I'd strongly recommend using an IntVector (See https://z3prover.github.io/api/html/namespacez3py.html#a7e166f891df8f17fd23290bec963b03c)
Note the important thing here is whether you need access with a symbolic index. (That is, do you always address your array with known constant indices, or do you need the ability to read/write to a symbolically addressed location.) From your description, it appears you always statically know the address, so IntVector is your best choice. If addresses can be symbolic, then you have to use good old SMTLib arrays, which are more costly.
For learning dependent types, I'm rewriting my old Haskell game in Idris. Currently the game „engine“ uses builtin integral types, such as Word8. I'd want to prove some lemmas involving numerical properties of those numbers (such as, that double negation is identity). However, it's not possible to say something about behaviour of primitive arithmetic operations. What would be better, to just use believe_me or other handwaving (at least for the most basic properties), or to rewrite my code using Nat, Fin and other „high-level“ numerical types?
I'd suggest using postulate for any of the primitive properties you need, being careful only to use things which are actually true for the numerical types in question, of course (which basically just means being careful about overflow). So you can say things like:
postulate add_commutes : (x, y : Int) -> x + y = y + x
believe_me is best avoided unless you need some computation behaviour of the proof. But, to be honest, we're still trying to work this stuff out when reasoning about primitives...
I believe for the moment it's generally better to use Nat when you can. The Idris devs are planning, eventually, to implement a general mechanism for replacing proof-friendly types with fast primitive ones in compilation, but for now that only happens for Nat. You could believe_me through if you really wanted, but you'd end up with functions that are not so easy to work with in proofs. Note that if you decide to play with believe_me, then you should consider also really_believe_me, which apparently makes it more believable to the type checker.
As previously asked Z3 can not provide a model for recursive functions. However, is it possible to tell Z3 (preferably via the Java API) that a model without a definition for recursive functions is sufficient (or indeed choose the functions I am interested in, essentially I do not need a model for non-constant functions)? Obviously, that might lead to queries that return sat although some functions do not have a model. Basically, the user then has to ensure that these functions do have some model.
However, I would assume this is not really realizable the way Z3 or SMT solvers work, i.e., I would assume Z3 needs a (partial) model of recursive functions to evalutate expressions.
The MBQI implementation for quantifiers doesn't really work nicely with recursive functions.
There is one thing that might do the trick for you: replace recursive function definitions by bounded-recursive functions. You add an extra argument that counts the number of unfoldings you are willing to explore of the function. Whenever you apply the recursive function in the rest of the input, set the depth parameter. You can use either algebraic data-type Peano numbers or integers.
The symbolic automata toolkit uses this approach, for example.
I found one paper that explores the idea given in Nikolaj Bjorner's answer further: "Computing with an SMT solver" by Amin, Leino and Rompf (2014).
Furthermore, I found "Satisfiability Modulo Recursive Programs" by Suter, Köksal and Kuncak (2011) which unfolds calls to recursive functions successively until the satisfiability can be decided.
Dart has both:
an equality operator == and
a top-level function named identical().
By the choice of syntax, it feels natural to want to use Dart's == operator more frequently than identical(), and I like that. In fact, the Section on Equality of the Idiomatic Dart states that "in practice, you will rarely need to use" identical().
In a recent answer to one of my questions concerning custom filters, it seems that Angular Dart favors use of identical() rather than == when trying to determine whether changes to a model have reached a steady state. (Which can make sense, I suppose, for large models for reasons of efficiency.)
This got me to thinking about identity of int's and so I wrote some tests of identical() over ints. While I expected that small ints might be "interned/cached" (e.g. similar to what is done by Java's Integer.valueOf()), to my surprise, I can't seem to generate two ints that are equal but not identical. I get similar results for double.
Are int and double values being interned/cached? Or maybe identical() is treating them specially? Coming from a Java background, I used to equate equate Dart's:
== to Java's equal() method and
identical() to Java's equality test ==.
But that now seems wrong. Anyone know what is going on?
Numbers are treated specially. If their bit-pattern is the same they must be identical (although it is still debated if this includes the different versions of NaNs).
The main reasons are expectations, leaking of internal details and efficiency.
Expectations: users expect numbers to be identical. It goes against common sense that x == y (for two integers) but not identical(x, y).
Leaking of internal details: the VM uses SMIs (SMall Integers) to represent integers in a specific range (31 bits on 32-bit machines, 63 on 64-bit machines). These are canonicalized and are always identical. Exposing this internal implementation detail would lead to inconsistent results depending on which platform you run.
Efficiency: the VM wants to unbox numbers wherever it can. For example, inside a method doubles are frequently moved into registers. However, keeping track of the original box can be cumbersome and difficult.
foo(x, y) {
var result = x;
while(y-- > 0) {
result += x;
}
return result;
}
Suppose, that the VM optimizes this function and moves result into a register (unboxing x in the process). This allows for a tight loop where result is then efficiently modified. The difficult case happens, when y is 0. The loop wouldn't execute and foo would return x directly. In other words, the following would need to be true:
var x = 5.0;
identical(x, foo(x, 0)); // should be true.
If the VM unboxed the result variable in the method foo it would need to allocate a fresh box for the result and the identical call would therefore return false.
By modifying the definition of identical all these problems are avoided. It comes with a small cost to the identical check, though.
Seems like I posted too quickly. I just stumbled on Dart Issue 13084: Spec says identical(1.0, 1) is true, even if they have different types which led me to the Dart section on Object Identity of the language spec. (I had previously search for equality in the spec but not object identity.)
Here is an excerpt:
The predefined dart function identical() is defined such that identical(c1, c2) iff:
- c1 evaluates to either null or an instance of
bool and c1 == c2, OR
- c1 and c2 are instances of int and c1 == c2, OR
- c1 and c2 are constant strings and c1 == c2, OR
- c1 and c2 are instances of double and one of the following holds: ...
and there are more clauses dealing with lists, maps and constant objects. See the language spec for the full details. Hence, identical() is much more than just a simple test for reference equality.
I can't remember the source for this, but somewhere on dartlang.org or the issue tracker it was said that num, int and double are indeed getting special treatment. One of those special treatments is that you can't subclass those types for performance reasons, but there may be more. What exactly this special treatment entails can probably only be answered by the developers, or maybe someone who knows the specification by heart, but one thing can be inferred:
The numeric types are dart objects - they have methods you can call on their instances. But they also have qualities of primitive data types, as you can do int i = 3;, while a pure object should have a new keyword somewhere. This is different from Java, where there are real primitive types and real objects wrapping them and exposing instance methods.
While the technical details certainly are more complex, if you think about dart numerics as a blend of object and primitive, your comparison to Java still makes sense. In Java, new Integer(5).equals(new Integer(5)) evaluates to true, and so does 5==5.
I am aware this is not a very technically correct answer, but I hope it's still useful to make sense of the behaviour of dart numerics when coming from a Java background.
I know that Z3 cannot check the satisfiability of formulas that contain recursive functions. But, I wonder if Z3 can handle such formulas over bounded data structures. For example, I've defined a list of length at most two in my Z3 program and a function, called last, to return the last element of the list. However, Z3 does not terminate when asked to check the satisfiability of a formula that contains last.
Is there a way to use recursive functions over bounded lists in Z3?
(Note that this related to your other question as well.) We looked at such cases as part of the Leon verifier project. What we are doing there is avoiding the use of quantifiers and instead "unrolling" the recursive function definitions: if we see the term length(lst) in the formula, we expand it using the definition of length by introducing a new equality: length(lst) = if(isNil(lst)) 0 else 1 + length(tail(lst)). You can view this as a manual quantifier instantiation procedure.
If you're interested in lists of length at most two, doing the manual instantiation for all terms, then doing it once more for the new list terms should be enough, as long as you add the term:
isCons(lst) => ((isCons(tail(lst)) => isNil(tail(tail(lst))))
for each list. In practice you of course don't want to generate these equalities and implications manually; in our case, we wrote a program that is essentially a loop around Z3 adding more such axioms when needed.
A very interesting property (very related to your question) is that it turns out that for some functions (such as length), using successive unrollings will give you a complete decision procedure. Ie. even if you don't constrain the size of the datastructures, you will eventually be able to conclude SAT or UNSAT (for the quantifier-free case).
You can find more details in our paper Satisfiability Modulo Recursive Programs, or I'm happy to give more here.
You may be interested in the work of Erik Reeber on SULFA, the ``Subclass of Unrollable List Formulas in ACL2.'' He showed in his PhD thesis how a large class of list-oriented formulas can be proven by unrolling function definitions and applying SAT-based methods. He proved decidability for the SULFA class using these methods.
See, e.g., http://www.cs.utexas.edu/~reeber/IJCAR-2006.pdf .