I'm trying to model the following expression using SMT a + b - c - d, all constants a,b,c,d are bitvecs of the same size, constraint with the following assert a + b >= c + d. I want to model it in such a way that overflows/underflows don't happen.
This is what I have tried so far:
(declare-const a (_ BitVec 4))
(declare-const b (_ BitVec 4))
(declare-const c (_ BitVec 4))
(declare-const d (_ BitVec 4))
(assert (bvuge (bvadd a b) (bvadd c d)))
; this is an inaccurate, only checks the last operation for underflow
; (a+b-c-d) >= d
; (assert (bvuge (bvsub (bvsub (bvadd a b) c) d) d))
;
; this should model that either both sides overflow, neither,
; or only the expression to the left of the inequality
; (
; (a + b <= a and c + d <= c and a + b >= c + d) or
; (a + b >= a and c + d <= c) or
; (a + b >= a and c + d >= c and a + b >= c + d)
; )
;(assert (or
; (and (bvule (bvadd a b) a)
; (bvule (bvadd c d) c)
; (bvuge (bvadd a b) (bvadd c d)))
; (and (bvuge (bvadd a b) a)
; (bvule (bvadd c d) c))
; (and (bvuge (bvadd a b) a)
; (bvuge (bvadd c d) c)
; (bvuge (bvadd a b) (bvadd c d)))))
(assert (bvuge (bvsub (bvsub (bvadd a b) c) d) #x0))
(check-sat)
(get-model)
But I'm not sure if the constraint are sufficient.
A better strategy is to "extend" to larger bit-vectors and check that the result fits in the bit-vector size you want. Note that proper underflow/overflow checking for bit-vectors can be surprisingly tricky, especially in the presence of multiplication.
Luckily, there's an excellent paper that describes how to do this properly: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/z3prefix.pdf
(There are a few minor issues with the paper: a few typos here and there, and the signed-multiplication overflow/underflow formula is outdated. But it's an excellent resource to start with!)
Furthermore, Z3 also provides out-of-the-box checker predicates for multiplication under/over-flows.
Once you review the paper, feel free to ask further questions regarding its contents if you still have issues!
Related
I am just starting to play with Z3 on my own and I thought one interesting experiment would be to construct a 3-element field.
So I declared my field S to be a scalar enumeration of three elements, A, B, C, and started gradually adding field axioms, asking Z3 for a model after each step, just to see what is going on. All goes well until I assert the possibility of subtraction, ∀ab.(∃x.a+x=b):
(declare-datatypes () ((S A B C)))
; there exist three distinct elements in S
(declare-const someA S)
(declare-const someB S)
(declare-const someC S)
(assert (distinct someA someB someC))
(declare-fun ADD (S S) S)
(declare-fun MUL (S S) S)
; commutative
(assert (forall ((x S) (y S)) (= (ADD x y) (ADD y x))))
(assert (forall ((x S) (y S)) (= (MUL x y) (MUL y x))))
; associative
(assert (forall ((x S) (y S) (z S)) (= (ADD x (ADD y z)) (ADD (ADD x y) z))))
(assert (forall ((x S) (y S) (z S)) (= (MUL x (MUL y z)) (MUL (MUL x y) z))))
; subtractivity
(assert (forall ((a S) (b S)) (exists ((x S)) (= (ADD a x) b))))
(check-sat)
(get-model)
This results in Z3 looping forever. I am surprised. I mean yes, I understand why FOL is in general undecidable, but I'd think this would be one of those "easy" cases because the space of all possible values for a, b and ADD is finite (and in this case even very small)? Why does it loop? and what is the correct way to express the subtractability axiom preferably in a way that doesn't lose being perceivable as its intended intuitive meaning?
While #alias is generally right with his suggestion, you might nevertheless be able to use Z3 if you're willing to weaken your axiomatisation. E.g.:
...
; subtractivity
(declare-fun SUB (S S) S)
(assert (forall ((a S) (b S) (x S)) (iff (= (ADD a x) b) (= (SUB b x) a))))
(check-sat)
(get-model)
I use Z3 with :fixedpoint.engine set to datalog.
I have an enumerated filter relation (f pos min max). Let us say that we have (f #x10 #x100000 #x200000), (f #x20 #x150000 #x200000) and (f #x20 #x300000 #x500000).
For a given x, I search the greatest pos such that (f pos min max) and min <= x <= max (Here I use intervals but filters can be arbitrarily complex). Priority (sort t) and values (sort s) are BitVectors but x, min and max are on a rather large space (eg. 24 bits).
; configuration
(set-option :fixedpoint.engine datalog)
; sorts
(define-sort s () (_ BitVec 24))
(define-sort t () (_ BitVec 8))
; Relations
(declare-rel f (t s s))
(declare-rel match (t s))
(declare-rel better (t s))
(declare-rel best (t s))
(declare-rel a (t))
(declare-rel b ())
(declare-rel c ())
(declare-var x s)
(declare-var xmin s)
(declare-var xmax s)
(declare-var p t)
(declare-var q t)
; Facts (EDB)
(rule (f #x10 #x100000 #x200000))
(rule (f #x20 #x150000 #x200000))
(rule (f #x20 #x300000 #x500000))
; Rules
(rule (=> (and (f p xmin xmax) (bvule xmin x) (bvule x xmax))
(match p x)))
(rule (=> (and (match q x) (bvugt q p))
(better p x)))
(rule (=> (and (match p x) (not (better p x)))
(best p x)))
; Queries
(rule (=> (match p #x170000) (a p)))
(rule (=> (better #x10 #x170000) b))
(rule (=> (best #x10 #x170000) c))
; output: sat
; (or (= (:var 0) #x20) (= (:var 0) #x10))
(query (a p) :print-answer true)
; output: sat
(query b)
; Output 'WARNING: creating large table of size 16777216 for relation better' and fails
(query c)
(match p x) codes the fact that a filter at priority p filters x.
(better p x) if a rule with a better priority than p filters x.
(best p x) codes that the best filter matching x has priority p.
If I query (match p #x170000), I quickly get #x10 and #x20. If I ask (better #x10 #x170000) I quickly get
an answer same for priority #20. But the query on (best p #x170000) fails to execute in reasonable time and reasonable space.
It seems that (not (better p x)) is computed independently of (match p x) and so is represented by a very large table (the possible values of x are not forwarded). In some cases I can restrict x by some tricks in better (sometimes I know that I am only interested by x that explicitly appear in other relations) so that the space is reduced but this is not a real generic solutions and sometimes I am stuck.
How should I rephrase the problem or which options should I use to avoid this problem ?
Z3's default Datalog tables are over concrete values, so if you use large bit-vectors, Z3 may end-up creating huge tables.
You can try a simpler table data-structure, that supports fewer operations but it's sparse (uses "don't care" bits).
You can try it out with: z3 fixedpoint.engine=datalog fixedpoint.datalog.default_relation=doc file.smt2
Given the following code:
from z3 import *
a,b,c = BitVecs('a b c', 32)
f1 = Exists([a, b, c], And(a + b == c, a < b, c == 1337))
f2 = And(a + b == c, a < b, c == 1337)
prove(f1 == f2)
I would assume that z3 implicitly existential quantifies a, b and c, in this example. Why aren't the two formulas equal, what is the difference?
The way you formulated your query doesn't really check whether f1 equals f2. Your query is essentially asking the solver to find a, b, c such that the following fails to hold:
Exists([a, b, c], And(a + b == c, a < b, c == 1337))
=
And(a + b == c, a < b, c == 1337))
And indeed, you can instantiate the outer a, b, and c such that the right hand-side is false; but the left hand side is an existential which is true; thus failing the equivalence you asked for.
It might be easier to see this with a simpler example; with just one boolean variable. You're essentially asking:
x == (Exists [x], x)
You see that those xs are actually different, so we can rename the inner one to (say) y; we get:
x == (Exist [y]. y)
Now, the right-hand-side is clearly true since there is a y that makes (Exist [y]. y) true. So, you are essentially asking the prover to establish that no matter what x you pick, it is true. Which is definitely not the case when you pick x to be false.
Indeed, you can ask Z3 to give you the formula it's trying to prove, and this is what it returns for your original query:
(set-info :status unknown)
(declare-fun c () (_ BitVec 32))
(declare-fun b () (_ BitVec 32))
(declare-fun a () (_ BitVec 32))
(assert
(let (($x24 (exists ((a (_ BitVec 32))
(b (_ BitVec 32))
(c (_ BitVec 32)) )
(and (= (bvadd a b) c) (bvslt a b) (= c (_ bv1337 32))))))
(let (($x57 (= $x24 (and (= (bvadd a b) c) (bvslt a b) (= c (_ bv1337 32))))))
(and $x57))))
(check-sat)
Which is clearly satisfiable, by the above reasoning.
(See Z3: convert Z3py expression to SMT-LIB2 from Solver object for the code that converts a z3-python query to smt-lib.)
I'm trying to use Z3 to solve equations involving unknown projection functions, to find a valid interpretation of the functions that satisfy the equation. So for example for the equation: snd . f = g . fst a valid interpretation would be f = \(x,y) -> (y,x) and g = id. I know that Z3 isn't higher order so I've been trying to encode the problem in first order form. So for example for f = g.fst I use:
(declare-datatypes (T) ((Tree (leaf (value T)) (node (children TreeList)))
(TreeList nil (cons (head Tree) (tail TreeList)))))
(define-fun fst ((x (Tree Int))) (Tree Int) (head (children x)))
(define-fun snd ((x (Tree Int))) (Tree Int) (head (tail (children x))))
(declare-fun f ((Tree Int)) (Tree Int))
(declare-fun g ((Tree Int)) (Tree Int))
(assert (forall ((x (Tree Int))) (= (f x) (g (fst x)))))
(check-sat)
(get-model)
Which sort of works returning:
(define-fun g ((x!1 (Tree Int))) (Tree Int)
(leaf 0))
(define-fun f ((x!1 (Tree Int))) (Tree Int)
(g (head (children x!1))))
However for snd . f = g . fst (I've simplified trees to pairs to try and help):
(declare-datatypes (T) ((Pair (leaf (value T)) (pair (fst Pair) (snd Pair)))))
(declare-fun f ((Pair Int)) (Pair Int))
(declare-fun g ((Pair Int)) (Pair Int))
(assert (forall ((x (Pair Int))) (= (snd (f x)) (g (fst x)))))
I get unknown.
I've also tried to encode a similar problem without the ADT just using booleans or ints as parameters, but then the model just assigns constant values to the functions. I've also tried to define a simple ADT for function constants, the identity function, and pairwise and sequential composition, and then define an "equals" function that can simplify expressions like f.id = f, but this either involves a recursive function like:
(declare-datatypes () (
(Fun id
(fun (funnum Int))
(seq (after Fun) (before Fun))
(pair (fst Fun) (snd Fun)) )))
(define-fun eq ((x Fun) (y Fun)) Bool (or
(= x y)
(eq x (seq y id)) ; id neutral for seq
(eq x (seq id y))
(eq y (seq x id))
(eq y (seq id x))))
(declare-const f Fun)
(declare-const g Fun)
(assert (eq f (seq id g)))
(check-sat)
(get-model)
Which is obviously invalid. Or if I use an uninterpretted function, it makes "eq" a constant function i.e.
(declare-fun eq (Fun Fun) Bool)
(assert (forall ((x Fun) (y Fun)) ; semantic equality
(= (eq x y) (or
(= x y) ; syntactic eq
(eq x (seq y id)) ; id neutral for seq
(eq x (seq id y))
(eq y (seq x id))
(eq y (seq id x))
))
))
=>
(define-fun eq ((x!1 Fun) (x!2 Fun)) Bool
true)
And similarly with equations involving functions with type Int -> Int, this returns constant functions for f and g:
(declare-fun f (Int) Int)
(declare-fun g (Int) Int)
(assert (forall ((x Int)) (= (+ (f x) 1) (+ (g x) 2)) ))
and adding these times out:
(assert (forall ((x Int) (y Int)) (=>
(not (= x y))
(not (= (g x) (g y))))))
(assert (forall ((x Int) (y Int)) (=>
(not (= x y))
(not (= (f x) (f y))))))
Any ideas how I can get this sort of thing to work?
Many thanks!
Z3 searches for essentially finite models so isn't well suited for solving functional equations directly. The main trick for finding models of these kinds is to strengthen the formulas by providing a finite set of alternative interpretations of functions that can be composed. For example, you can allow f(x) to be either identity, permutation, or repeat x or y, or return a constant value in one of the fields, This may be composed with functions that perform simple arithmetical operations. You will have to bound the number of compositions you are willing to admit. You assert a similar set of templates for g. So far this has worked best for bit-vectors.
The search space for such interpretations can easily get overwhelming. I tried your example with algebraic data-types and templates. Z3 is not able to find an interpretation in this case, at least not by stating the problem directly as a template search.
I am trying to prove with Z3 the theorem in general topology given at
TPTP-Topology
I am translating the code given there using the following Z3-SMT-LIB code
;; File : TOP001-2 : TPTP v6.0.0. Released v1.0.0.
;; Domain : Topology
;; Problem : Topology generated by a basis forms a topological space, part 1
(declare-sort S)
(declare-sort Q)
(declare-sort P)
(declare-fun elemcoll (S Q) Bool)
(declare-fun elemset (P S) Bool)
(declare-fun unionmemb (Q) S)
(declare-fun f1 (Q P) S)
(declare-fun f11 (Q S) P)
(declare-fun basis (S Q) Bool)
(declare-fun Subset (S S) Bool)
(declare-fun topbasis (Q) Q)
;; union of members axiom 1.
(assert (forall ((U P) (Vf Q)) (or (not (elemset U (unionmemb Vf)))
(elemset U (f1 Vf U) ) ) ))
;; union of members axiom 2.
(assert (forall ((U P) (Vf Q)) (or (not (elemset U (unionmemb Vf)))
(elemcoll (f1 Vf U) Vf ) ) ))
;; basis for topology, axiom 28
(assert (forall ((X S) (Vf Q)) (or (not (basis X Vf)) (= (unionmemb Vf) X ) ) ))
;; Topology generated by a basis, axiom 40.
(assert (forall ((Vf Q) (U S)) (or (elemcoll U (topbasis Vf))
(elemset (f11 Vf U) U)) ))
;; Set theory, axiom 7.
(assert (forall ((X S) (Y Q)) (or (not (elemcoll X Y)) (Subset X (unionmemb Y) ) ) ))
;; Set theory, axiom 8.
(assert (forall ((X S) (Y S) (U P)) (or (not (Subset X Y)) (not (elemset U X))
(elemset U Y) )))
;; Set theory, axiom 9.
(assert (forall ((X S)) (Subset X X ) ))
;; Set theory, axiom 10.
(assert (forall ((X S) (Y S) (Z S)) (or (not (= X Y)) (not (Subset Z X)) (Subset Z Y) ) ))
;; Set theory, axiom 11.
(assert (forall ((X S) (Y S) (Z S)) (or (not (= X Y)) (not (Subset X Z)) (Subset Y Z) ) ))
(check-sat)
(push)
(declare-fun cx () S)
(declare-fun f () Q)
(assert (basis cx f))
(assert (not (elemcoll cx (topbasis f))))
(check-sat)
(pop)
(push)
(assert (basis cx f))
(assert (elemcoll cx (topbasis f)))
(check-sat)
(pop)
The corresponding output is
sat
sat
sat
Please run this example online here
The first sat is correct; but the second sat is wrong, it must be unsat. In other words, Z3 is saying that the theorem and its negation are true simultaneously.
Please let me know what happens in this case. Many thanks. All the best.
It is possible that both a formula and the negation of the formula is consistent with respect to a background theory T. In particular, when T is not complete, then there are sentences that are neither consequences of T nor inconsistent with T. In your case the theory T is the set of topology axioms.
You can use the command (get-model) to obtain a model that satisfies the axioms and the sentence.