Let's assume that we have the following files:
func.smt
(declare-datatypes (T) ((AVL leafA (nodeA (val T) (alt Int) (izq AVL) (der AVL)))))
espec.smt
(declare-const t (AVL Int))
And the following code:
from z3 import *
s = Solver()
s.from_file("func.smt")
s.from_file("espec.smt")
When instruction "s.from_file("espec.smt")" is executed, z3 throws the next exception:
z3.z3types.Z3Exception: b'(error "line 1 column 18: unknown sort \'AVL\'")\n'
It seems that the Solver "s" doesn't save the information of datatypes (and functions too). That is why he throws the exception (I guess).
The following code is a way to solve it:
s = Solver()
file = open("func.smt", "r")
func = file.read()
file.close()
file = open("espec.smt", "r")
espec = file.read()
file.close()
s.from_string(func + espec)
But this way it's not efficient.
Is there another more efficient way to solve this and make z3 save datatypes and functions for future asserts and declarations?
Edit:
In my real case for example, the file "func.smt" has a total of 54 declarations between functions and datatypes (some quite complex). The "spec.smt" file has few declarations and asserts. And finally I have a file in which there are many asserts which I have to read little by little and generate a model.
That is, imagine that I have 600 lines of asserts, and I read from 10 to 10, so every 10 lines I generate a model. That is, if we assume that those asserts that I have read are stored in a variable called "aux", every time I want to get a new model, I will have to do "s.from_string (func + spec + aux)" 60 times in total, and I don't know if that could be made more efficient.
Separately reading the files and loading to solver is your best bet, as you found out.
Unless efficiency is of paramount importance, I would not worry about trying to optimize this any further. For any reasonable problem, your solver time will dominate any time spent in loading the assertions from a few (or a few thousand!) files. Do you have a use case that really requires this part to be significantly faster?
Related
I have tried to implement Floor and Ceiling Function as defined in the following link
https://math.stackexchange.com/questions/3619044/floor-or-ceiling-function-encoding-in-first-order-logic/3619320#3619320
But Z3 query returning counterexample.
Floor Function
_X=Real('_X')
_Y=Int('_Y')
_W=Int('_W')
_n=Int('_n')
_Floor=Function('_Floor',RealSort(),IntSort())
..
_s.add(_X>=0)
_s.add(_Y>=0)
_s.add(Implies(_Floor(_X)==_Y,And(Or(_Y==_X,_Y<_X),ForAll(_W,Implies(And(_W>=0,_W<_X),And(_W ==_Y,_W<_Y))))))
_s.add(Implies(And(Or(_Y==_X,_Y<_X),ForAll(_W,Implies(And(_W>=0,_W<_X),And(_W==_Y,_W<_Y))),_Floor(_X)==_Y))
_s.add(Not(_Floor(0.5)==0))
Expected Result - Unsat
Actual Result - Sat
Ceiling Function
_X=Real('_X')
_Y=Int('_Y')
_W=Int('_W')
_Ceiling=Function('_Ceiling',RealSort(),IntSort())
..
..
_s.add(_X>=0)
_s.add(_Y>=0)
_s.add(Implies(_Ceiling(_X)==_Y,And(Or(_Y==_X,_Y<_X),ForAll(_W,Implies(And(_W>=0,_W<_X),And(_W ==_Y,_Y<_W))))))
_s.add(Implies(And(Or(_Y==_X,_Y<_X),ForAll(_W,Implies(And(_W>=0,_W<_X),And(_W==_Y,_Y<_W)))),_Ceiling(_X)==_Y))
_s.add(Not(_Ceilng(0.5)==1))
Expected Result - Unsat
Actual Result - Sat
[Your encoding doesn't load to z3, it gives a syntax error even after eliminating the '..', as your call to Implies needs an extra argument. But I'll ignore all that.]
The short answer is, you can't really do this sort of thing in an SMT-Solver. If you could, then you can solve arbitrary Diophantine equations. Simply cast it in terms of Reals, solve it (there is a decision procedure for Reals), and then add the extra constraint that the result is an integer by saying Floor(solution) = solution. So, by this argument, you can see that modeling such functions will be beyond the capabilities of an SMT solver.
See this answer for details: Get fractional part of real in QF_UFNRA
Having said that, this does not mean you cannot code this up in Z3. It just means that it will be more or less useless. Here's how I would go about it:
from z3 import *
s = Solver()
Floor = Function('Floor',RealSort(),IntSort())
r = Real('R')
f = Int('f')
s.add(ForAll([r, f], Implies(And(f <= r, r < f+1), Floor(r) == f)))
Now, if I do this:
s.add(Not(Floor(0.5) == 0))
print(s.check())
you'll get unsat, which is correct. If you do this instead:
s.add(Not(Floor(0.5) == 1))
print(s.check())
you'll see that z3 simply loops forever. To make this usefull, you'd want the following to work as well:
test = Real('test')
s.add(test == 2.4)
result = Int('result')
s.add(Floor(test) == result)
print(s.check())
but again, you'll see that z3 simply loops forever.
So, bottom line: Yes, you can model such constructs, and z3 will correctly answer the simplest of queries. But with anything interesting, it'll simply loop forever. (Essentially whenever you'd expect sat and most of the unsat scenarios unless they can be constant-folded away, I'd expect z3 to simply loop.) And there's a very good reason for that, as I mentioned: Such theories are just not decidable and fall well out of the range of what an SMT solver can do.
If you are interested in modeling such functions, your best bet is to use a more traditional theorem prover, like Isabelle, Coq, ACL2, HOL, HOL-Light, amongst others. They are much more suited for working on these sorts of problems. And also, give a read to Get fractional part of real in QF_UFNRA as it goes into some of the other details of how you can go about modeling such functions using non-linear real arithmetic.
Can I save the constraints I created for a z3 solver and later reload them to continue looking for more solutions?
I have learned there is the SMT-LIB2 format for such things and that z3 and z3py have an API for saving and loading in that format. Unfortunately I cannot make it work.
Here's my example program which pointlessly saves and reloads:
import z3
filename = 'z3test.smt'
# Make a solver with some arbitrary useless constraint
solver = z3.Solver()
solver.add(True)
# Save to file
smt2 = solver.sexpr()
with open(filename, mode='w', encoding='ascii') as f: # overwrite
f.write(smt2)
f.close()
# Load from file
solver.reset()
solver.from_file(filename)
It fails with:
Exception has occurred: ctypes.ArgumentError
argument 3: <class 'TypeError'>: wrong type
File "C:\Users\Marian Aldenhövel\Desktop\FridgeIQ\z3\z3-4.8.4.d6df51951f4c-x64-win\bin\python\z3\z3core.py", line 3449, in Z3_solver_from_file
_elems.f(a0, a1, _to_ascii(a2))
File "C:\Users\Marian Aldenhövel\Desktop\FridgeIQ\z3\z3-4.8.4.d6df51951f4c-x64-win\bin\python\z3\z3.py", line 6670, in from_file
_handle_parse_error(e, self.ctx)
File "C:\Users\Marian Aldenhövel\Desktop\FridgeIQ\src\z3test.py", line 17, in <module>
solver.from_file(filename)
Is this a problem with my understanding or my code? Can it be done like this? Are sexpr() and from_file() the right pair of API calls?
I am using z3 and z3py 4.8.4 from https://github.com/z3prover/z3/releases on Windows 10 64bit.
More detail if required:
I am playing with z3 in Python to find solutions for a large disection-puzzle.
To find all solutions I am calling solver.check(). When it returns a sat verdict I interpret the model as image of my puzzle solution. I then add a blocking clause ruling out that specific solution and call solver.check() again.
This works fine and I am delighted.
The runtime to find all solutions will be on the order of many days or until I get bored. I am concerned that my machine will not be running continuously for that long. It may crash, run out of power, or be rebooted for other reasons.
I can easily recreate the initial constraints which is the whole point of the program. But the blocking clauses are a runtime product and a function of how far we have gotten.
I thought I could save the state of the solver and if at runtime I find such a file restart by loading that with the blocking clauses intact and go on finding more solutions instead of having to start over.
Thank you for taking your time reading and thinking.
Marian
With z3 4.4.1 and z3 4.8.5, I would dump (and reload) the constraints in smt2 format as follows:
import z3
filename = "z3test.smt2"
x1 = z3.Real("x1")
x2 = z3.Real("x2")
solver = z3.Solver()
solver.add(x1 != x2)
#
# STORE
#
with open(filename, mode='w') as f:
f.write(solver.to_smt2())
#
# RELOAD
#
solver.reset()
constraints = z3.parse_smt2_file(filename, sorts={}, decls={})
solver.add(constraints)
print(solver)
output:
~$ python t.py
[And(x1 != x2, True)]
file z3test.smt2:
(set-info :status unknown)
(declare-fun x2 () Real)
(declare-fun x1 () Real)
(assert
(and (distinct x1 x2) true))
(check-sat)
I have no idea whether the API changed in the version you are using. Feedback is welcome.
As per my knowledge, since z3 doesn't recognize transcendental functions its throwing me an error while conversion using following code.
def convertor(f, status="unknown", name="benchmark", logic=""):
v = (Ast * 0)()
if isinstance(f, Solver):
a = f.assertions()
if len(a) == 0:
f = BoolVal(True)
else:
f = And(*a)
return Z3_benchmark_to_smtlib_string(f.ctx_ref(), name, logic, status, "", 0, v, f.as_ast())
pi, EI, kA , kB, N = Reals('pi EI kA kB N')
s= Solver()
s.add(pi == 3.1415926525)
s.add(EI == 175.2481)
s.add(kA>= 0)
s.add(kA<= 100)
s.add(kB>= 0)
s.add(kB<= 100)
s.add(N>= 100)
s.add(N<= 200)
please change the path of the input file "beamfinv3.bch", which can be found at: link
continue_read=False
input_file = open('/home/mani/downloads/new_z3/beamfinv3.bch', 'r')
for line in input_file:
if line.strip()=="Constraints":
continue_read=True
continue
if line.strip()=="end":
continue_read=False
if continue_read==True:
parts = line.split(';')
if (parts[0]!="end"):
#print parts[0]
s.add(eval(parts[0]))
input_file.close()
file=open('cyber.smt2','w')
result=convertor(s, logic="None")
file.write (result)
error:
File "<string>", line 1, in <module>
NameError: name 'sin' is not defined
Any way out? or help?
Thanks.
The core of this problem is that eval tries to execute a Python script, i.e., all functions that occur within parts[0] must have a corresponding Python function of the same name, which is not the case for the trigonometric functions (the are neither in the Python API nor the C API, the former being based on the latter). For now you could try to add those functions yourself, perhaps with an implementation based on parse_smt2_string, or perhaps by replacing the Python strings with SMT2 strings altogether.
Z3 can represent expressions containing trigonometric functions, but it will refuse to do so when the logic is set to something; see arith_decl_plugin. I don't know Python well enough, but it might have to be None instead of "".
While Z3 can represent these expressions, it's probably not very good at solving them. See comments on the limitations in Can Z3 handle sinusoidal and exponential functions, Z3 supports for nonlinear arithmetics, and Z3 Performance with Non-Linear Arithmetic.
I'm actually using Z3py for scheduling solving problems and I'm trying to represent a 2 processors system where 4 process of different execution time must be done.
My actual data are :
Process 1 : Arrival at 0 and execution time of 4
Process 2 : Arrival at 1 and execution time of 3
Process 3 : Arrival at 3 and execution time of 5
Process 4 : Arrival at 1 and execution time of 2
I'm actually trying to represent each process while decomposing each in subprocess of equal time so my datatypes are like this :
Pn = Datatype('Pn')
Pn.declare('1')
Pn.declare('2')
Pn.declare('3')
Pn.declare('4')
Pt = Datatype('Pt')
Pt.declare('1')
Pt.declare('2')
Pt.declare('3')
Pt.declare('4')
Pt.declare('5')
Process = Datatype('Process')
Process.declare('cons' , ('name',Pn) , ('time', Pt))
Process.declare('idle')
where pn and pt are the process name and the part of the process (process 1 is in 4 parts, ...)
But now I don't know how I can represent my processors to add 3 rules I need : unicity (each sub process must be done 1 and only 1 time by only 1 processor) check arrival (the first part of a process can't be processed before it arrived) and order (each part of a process must be processed after the precedent)
So I was thinking of using arrays to represent my 2 processors with this kind of declaration :
P = Array('P', IntSort() , Process)
But when I tried to execute it I got an error message saying :
Traceback (most recent call last):
File "C:\Users\Alexis\Desktop\test.py", line 16, in <module>
P = Array('P', IntSort() , Process)
File "src/api/python\z3.py", line 3887, in Array
File "src/api/python\z3.py", line 3873, in ArraySort
File "src/api/python\z3.py", line 56, in _z3_assert
Z3Exception: 'Z3 sort expected'
And know I don't know how handle it... must I create a new datatype and figure a way to add my rules ? or Is there a way to add datatypes to array which would let me create rules like this :
unicity = ForAll([x,y] , (Implies(x!=y,P[x]!=P[y])))
Thanks in advance
There is a tutorial on using Datatypes from the Python API. A link to the tutorial is:
http://rise4fun.com/Z3Py/tutorialcontent/advanced#h22
It shows how to create a list data-type and use the "create()" method to instantiate a Sort object from the object used when declaring the data-type. For your example, it suffices to add calls to "create()" in the places where you want to use the declared type as a sort.
See: http://rise4fun.com/Z3Py/rQ7t
Regarding the rest of the case study you are looking at: it is certainly possible to express the constrsaints you describe using quantifiers and arrays. You could also consider somewhat more efficient encodings:
Instead of using an array, use a function declaration. So P would be declared as a unary function:
P = Function('P', IntSort(), Process.create()).
Using quantifiers for small finite domain problems may be more of an overhead than a benefit. Writing down the constraints directly as a finite conjunction saves the overhead of instantiating quantifiers possibly repeatedly. That said, some quantified axioms can also be optimized. Z3 automatically compiles axioms of the form: ForAll([x,y], Implies(x != y, P(x) != P(y))) into
an axioms of the form Forall([x], Pinv(P(x)) == x), where "Pinv" is a fresh function. The new axiom still enforces that P is injective but requires only a linear number of instantiations, linear in the number of occurrences of P(t) for some term 't'.
Have fun!
Note: This is a sequel to my previous question about powersets.
I have got a nice Ruby solution to my previous question about generating a powerset of a set without a need to keep a stack:
class Array
def powerset
return to_enum(:powerset) unless block_given?
1.upto(self.size) do |n|
self.combination(n).each{|i| yield i}
end
end
end
# demo
['a', 'b', 'c'].powerset{|item| p item} # items are generated one at a time
ps = [1, 2, 3, 4].powerset # no block, so you'll get an enumerator
10.times.map{ ps.next } # 10.times without a block is also an enumerator
It does the job and works nice.
However, I would like to try to rewrite the same solution in Erlang, because for the {|item| p item} block I have a big portion of working code already written in Erlang (it does some stuff with each generated subset).
Although I have some experience with Erlang (I have read all 2 books :)), I am pretty confused with the example and the comments that sepp2k kindly gave me to my previous question about powersets. I do not understand the last line of the example - the only thing that I know is that is is a list comprehension. I do not understand how I can modify it to be able to do something with each generated subset (then throw it out and continue with the next subset).
How can I rewrite this Ruby iterative powerset generation in Erlang? Or maybe the provided Erlang example already almost suits the need?
All the given examples have O(2^N) memory complexity, because they return whole result (the first example). Two last examples use regular recursion so that stack raises. Below code which is modification and compilation of the examples will do what you want:
subsets(Lst) ->
N = length(Lst),
Max = trunc(math:pow(2, N)),
subsets(Lst, 0, N, Max).
subsets(Lst, I, N, Max) when I < Max ->
_Subset = [lists:nth(Pos+1, Lst) || Pos <- lists:seq(0, N-1), I band (1 bsl Pos) =/= 0],
% perform some actions on particular subset
subsets(Lst, I+1, N, Max);
subsets(_, _, _, _) ->
done.
In the above snippet tail recursion is used which is optimized by Erlang compiler and converted to simple iteration under the covers. Recursion may be optimized this way only if recursive call is the last one within function execution flow. Note also that each generated Subset may be used in place of the comment and it will be forgotten (garbage collected) just after that. Thanks to that neither stack nor heap won't grow, but you also have to perform operation on subsets one after another as there's no final result containing all of them.
My code uses same names for analogous variables like in the examples to make it easier to compare both of them. I'm sure it could be refined for performance, but I only want to show the idea.