As previously asked Z3 can not provide a model for recursive functions. However, is it possible to tell Z3 (preferably via the Java API) that a model without a definition for recursive functions is sufficient (or indeed choose the functions I am interested in, essentially I do not need a model for non-constant functions)? Obviously, that might lead to queries that return sat although some functions do not have a model. Basically, the user then has to ensure that these functions do have some model.
However, I would assume this is not really realizable the way Z3 or SMT solvers work, i.e., I would assume Z3 needs a (partial) model of recursive functions to evalutate expressions.
The MBQI implementation for quantifiers doesn't really work nicely with recursive functions.
There is one thing that might do the trick for you: replace recursive function definitions by bounded-recursive functions. You add an extra argument that counts the number of unfoldings you are willing to explore of the function. Whenever you apply the recursive function in the rest of the input, set the depth parameter. You can use either algebraic data-type Peano numbers or integers.
The symbolic automata toolkit uses this approach, for example.
I found one paper that explores the idea given in Nikolaj Bjorner's answer further: "Computing with an SMT solver" by Amin, Leino and Rompf (2014).
Furthermore, I found "Satisfiability Modulo Recursive Programs" by Suter, Köksal and Kuncak (2011) which unfolds calls to recursive functions successively until the satisfiability can be decided.
Related
I'm using the Z3_parse_smtlib2_string function from the Z3 C API (via Haskell's Z3 lib) to parse an SMTLIB file and apply some tactics to simplify its content, however I notice that any push, pop and check-sat commands appear to be swallowed by this function and do not appear in the resulting AST.
Is there anyway that I can parse this without losing these commands (and then apply the required tactics, once again without losing them)?
I don't think it's possible to do this with Z3_parse_smtlib2_string. As you can see in the documentation "It returns a formula comprising of the conjunction of assertions in the scope (up to push/pop) at the end of the string." See: https://z3prover.github.io/api/html/group__capi.html#ga7905ebec9289b9fe5debcad965f6267e
Note that the reason for this is not just mere "not-implemented" or "buggy." Look at the return type of the function you're using. It returns a Z3_ast_vector, and Z3_ast only captures "expressions" in the SMTLib language. But push/pop etc. are not considered expressions by Z3, but rather commands; i.e., they are internally represented differently. (Whether this was a conscious choice or historical is something I'm not sure about.)
I don't think there's a function to do what you're asking; i.e., can return both expressions and commands. You can ask at https://github.com/Z3Prover/z3/discussions to see if the developers can provide an alternative API, or if they already have something exposed to the users that achieves this.
Say for modeling the haskell map function, which takes in a "mapper" function that is applied to all elements of a list. How can I declare map in smtlib?
No; SMTLib is essentially a first-order theory; higher order functions are simply not supported.
Z3, however, allows mapping functions over arrays, with the (_ map f) extension. See https://rise4fun.com/Z3/tutorial/guide, search for "Mapping Functions on Arrays." This doesn't give you arbitrary higher-order functions, but can be used to simulate those that operate on SMTLib arrays.
If you do intend to reason about higher-order functions, then SMTLib is arguably the wrong logic for you. Use of a more traditional theorem prover like HOL/Isabelle, or modern incarnations in Agda/Coq would be more suitable. You can also take a look at Lean, which has a good compromise of features and automation.
In z3 is it possible to declare a function that takes another function as an argument? For instance, this
(declare-fun foo ( ((Int) Bool) ) Int)
doesn't quite seem to work. Thanks.
As Leonardo mentioned, SMT-Lib does not allow higher-order functions. This is not merely a syntactic restriction: Reasoning with higher-order functions is (generally) beyond what SMT solvers can deal with. (Although uninterpreted functions can be used in some special cases.)
If you do need to reason with higher-order functions, then interactive theorem provers are the main weapon of choice: Isabelle, HOL, Coq being some of the examples.
However, sometimes you want the higher-order functions not to reason about them, but rather merely to simplify programming tasks. SMT-Lib input language is not suitable for high-level programming that end-users typically need in practical situations. If that is your use case, then I'd recommend not using SMT-Lib directly, but rather working with a programming language that gives you access to Z3 (or other SMT solvers). There are several choices, depending on what host language is most suitable for your use case:
If you are a Python user, Z3Py that just shipped with Z3 4.0 is the way to go,
If you are a Scala user, then look into Scala^Z3.
If Haskell is your preferred language, then take a look at SBV.
Each binding has its own feature set, Z3Py probably being the most versatile since it's directly supported by the Z3 folks. (It also provides access to Z3 internals that remain inaccessible for the other choices, at least for the time being.)
No, this is not possible. However, you can define a function that takes an array as an argument.
(declare-fun foo ((Array Int Bool)) Int)
You can use this trick to simulate high-order functions like the one in your question.
Here is an example: http://rise4fun.com/Z3/qsED
The Z3 guide contains more information about Z3 and SMT.
I know that Z3 cannot check the satisfiability of formulas that contain recursive functions. But, I wonder if Z3 can handle such formulas over bounded data structures. For example, I've defined a list of length at most two in my Z3 program and a function, called last, to return the last element of the list. However, Z3 does not terminate when asked to check the satisfiability of a formula that contains last.
Is there a way to use recursive functions over bounded lists in Z3?
(Note that this related to your other question as well.) We looked at such cases as part of the Leon verifier project. What we are doing there is avoiding the use of quantifiers and instead "unrolling" the recursive function definitions: if we see the term length(lst) in the formula, we expand it using the definition of length by introducing a new equality: length(lst) = if(isNil(lst)) 0 else 1 + length(tail(lst)). You can view this as a manual quantifier instantiation procedure.
If you're interested in lists of length at most two, doing the manual instantiation for all terms, then doing it once more for the new list terms should be enough, as long as you add the term:
isCons(lst) => ((isCons(tail(lst)) => isNil(tail(tail(lst))))
for each list. In practice you of course don't want to generate these equalities and implications manually; in our case, we wrote a program that is essentially a loop around Z3 adding more such axioms when needed.
A very interesting property (very related to your question) is that it turns out that for some functions (such as length), using successive unrollings will give you a complete decision procedure. Ie. even if you don't constrain the size of the datastructures, you will eventually be able to conclude SAT or UNSAT (for the quantifier-free case).
You can find more details in our paper Satisfiability Modulo Recursive Programs, or I'm happy to give more here.
You may be interested in the work of Erik Reeber on SULFA, the ``Subclass of Unrollable List Formulas in ACL2.'' He showed in his PhD thesis how a large class of list-oriented formulas can be proven by unrolling function definitions and applying SAT-based methods. He proved decidability for the SULFA class using these methods.
See, e.g., http://www.cs.utexas.edu/~reeber/IJCAR-2006.pdf .
I'm learning F# (new to functional programming in general though used functional aspects of C# for years but let's face it, that's pretty different) and one of the things that I've read is that the F# compiler identifies tail recursion and compiles it into a while loop (see http://thevalerios.net/matt/2009/01/recursion-in-f-and-the-tail-recursion-police/).
What I don't understand is why you would write a recursive function instead of a while loop if that's what it's going to turn into anyway. Especially considering that you need to do some extra work to make your function recursive.
I have a feeling someone might say that the while loop is not particularly functional and you want to act all functional and whatnot so you use recursion but then why is it sufficient for the compiler to turn it into a while loop?
Can someone explain this to me?
You could use the same argument for any transformation that the compiler performs. For instance, when you're using C#, do you ever use lambda expressions or anonymous delegates? If the compiler is just going to turn those into classes and (non-anonymous) delegates, then why not just use those constructions yourself? Likewise, do you ever use iterator blocks? If the compiler is just going to turn those into state machines which explicitly implement IEnumerable<T>, then why not just write that code yourself? Or if the C# compiler is just going to emit IL anyway, why bother writing C# instead of IL in the first place? And so on.
One obvious answer to all of these questions is that we want to write code which allows us to express ourselves clearly. Likewise, there are many algorithms which are naturally recursive, and so writing recursive functions will often lead to a clear expression of those algorithms. In particular, it is arguably easier to reason about the termination of a recursive algorithm than a while loop in many cases (e.g. is there a clear base case, and does each recursive call make the problem "smaller"?).
However, since we're writing code and not mathematics papers, it's also nice to have software which meets certain real-world performance criteria (such as the ability to handle large inputs without overflowing the stack). Therefore, the fact that tail recursion is converted into the equivalent of while loops is critical for being able to use recursive formulations of algorithms.
A recursive function is often the most natural way to work with certain data structures (such as trees and F# lists). If the compiler wants to transform my natural, intuitive code into an awkward while loop for performance reasons that's fine, but why would I want to write that myself?
Also, Brian's answer to a related question is relevant here. Higher-order functions can often replace both loops and recursive functions in your code.
The fact that F# performs tail optimization is just an implementation detail that allows you to use tail recursion with the same efficiency (and no fear of a stack overflow) as a while loop. But it is just that - an implementation detail - on the surface your algorithm is still recursive and is structured that way, which for many algorithms is the most logical, functional way to represent it.
The same applies to some of the list handling internals as well in F# - internally mutation is used for a more efficient implementation of list manipulation, but this fact is hidden from the programmer.
What it comes down to is how the language allows you to describe and implement your algorithm, not what mechanics are used under the hood to make it happen.
A while loop is imperative by its nature. Most of the time, when using while loops, you will find yourself writing code like this:
let mutable x = ...
...
while someCond do
...
x <- ...
This pattern is common in imperative languages like C, C++ or C#, but not so common in functional languages.
As the other posters have said some data structures, more exactly recursive data structures, lend themselves to recursive processing. Since the most common data structure in functional languages is by far the singly linked list, solving problems by using lists and recursive functions is a common practice.
Another argument in favor of recursive solutions is the tight relation between recursion and induction. Using a recursive solution allows the programmer to think about the problem inductively, which arguably helps in solving it.
Again, as other posters said, the fact that the compiler optimizes tail-recursive functions (obviously, not all functions can benefit from tail-call optimization) is an implementation detail which lets your recursive algorithm run in constant space.