Retrieve the bounds of an Extract node in z3py - z3

I am using z3py. My question is, how do I retrieve the bounds of an Extract node? I thought Extract would be a function with arity three, but it isn't:
>>> x = BitVecVal(3, 32)
>>> e = Extract(15, 0, x)
>>> e.decl()
Extract
>>> e.decl().arity()
1
>>> e2 = Extract(7, 0, x)
>>> e2.decl()
Extract
>>> e.decl() == e2.decl()
False
Each Extract operation is typed (apparently) by the first two arguments (I infer this because the decls aren't equal).
If I'm given a BitVecRef which is an Extract operation, how can I tell the bounds of the operation? So for Extract(i, j, x) I want a function that gives me back i and j.

The bounds are encoded as "parameters" together with the term. These parameters don't get passed as regular arguments. The python API does not expose access to parameters, but the C API does, and you can call these functions from Python (it is just a little more work).
The function you need is Z3_get_decl_int_parameter.
Here is a sample using the function: http://rise4fun.com/Z3Py/Rsl8
x = BitVec('x',32)
t = Extract(10,5,x)
f = t.decl()
print Z3_get_decl_int_parameter(t.ctx.ref(), f.ast, 0)
print Z3_get_decl_int_parameter(t.ctx.ref(), f.ast, 1)

Related

Does Z3 have troubles with division?

During a project, I encountered a phenomenon I can't explain. It is boiled down in the following minimal example (using Z3Py) which should be unsat but returns a model.
from z3 import *
solver = Solver()
x = Int("x")
# The next proposition is clearly unsatisfiable
solver.add(And(x > 0, (x + 1) * (1 / (x + 1)) != 1))
solver.add(x == 1)
# Still, Z3 provides a "model"
print(solver.check())
print(solver.model())
Is there anything I am missing? Is it forbidden to use division in certain situations?
Z3 version: 4.8.14
Python version: 3.9
Note that when you use / over integers, than the result is an integer itself. That is, we use integer division; which in SMTLib is defined to be the Euclidian one. See https://smtlib.cs.uiowa.edu/theories-Ints.shtml for details.
This means that the expression 1 / (1+1) reduces to 0, since division produces an integer. So you end up with 0 != 1, which is true, and thus your problem is satisfiable.
If you want division to produce reals, then you should either start with a real x (i.e., x = Real("x")) in which case your problem will be unsat; or convert the arguments to be over the reals, via an expression of the form ToReal(x); which again will produce unsat for your problem.
So, in the first case, your modification will be:
x = Real('x')
casting the problem over reals. Or, if you want to keep x an integer, yet the division to be over the reals, you should use:
solver.add(And(x > 0, ToReal(x + 1) * (1 / ToReal(x + 1)) != 1))
which does the conversion.
I should note that mixing integers and reals in this way can lead to hard problems in general, and the solver might end up reporting unknown. But that's besides the point for your particular question here.

arguments and function call of LSTM in pytorch

Could anyone please explain me the below code:
import torch
import torch.nn as nn
input = torch.randn(5, 3, 10)
h0 = torch.randn(2, 3, 20)
c0 = torch.randn(2, 3, 20)
rnn = nn.LSTM(10,20,2)
output, (hn, cn) = rnn(input, (h0, c0))
print(input)
While calling rnn rnn(input, (h0, c0)) we gave arguments h0 and c0 in parenthesis. What is it supposed to mean? if (h0, c0) represents a single value then what is that value and what is the third argument passed here?
However, in the line rnn = nn.LSTM(10,20,2) we are passing arguments in LSTM function without paranthesis.
Can anyone explain me how this function call is working?
The assignment rnn = nn.LSTM(10, 20, 2) instanciates a new nn.Module using the nn.LSTM class. It's first three arguments are input_size (here 10), hidden_size (here 20) and num_layers (here 2).
On the other hand rnn(input, (h0, c0)) corresponds to actually calling the class instance, i.e. running __call__ which is roughly equivalent to the forward function of that module. The __call__ method of nn.LSTM takes in two parameters: input (shaped (sequnce_length, batch_size, input_size), and a tuple of two tensors (h_0, c_0) (both shaped (num_layers, batch_size, hidden_size) in the basic use case of nn.LSTM)
Please refer to the PyTorch documentation whenever using builtins, you will find the exact definition of the parameters list (the arguments used to initialize the class instance) as well as the input/outputs specifications (whenever inferring with that said module).
You might be confused with the notation, here's a small example that could help:
tuple as input:
def fn1(x, p):
a, b = p # unpack input
return a*x + b
>>> fn1(2, (3, 1))
>>> 7
tuple as output
def fn2(x):
return x, (3*x, x**2) # actually output is a tuple of int and tuple
>>> x, (a, b) = fn2(2) # unpacking
(2, (6, 4))
>>> x, a, b
(2, 6, 4)

Is there a way to get the final system of equations sent by cvxpy to the solver?

If I understand correctly, cvxpy converts our high-level problem description to the standard canonical form before it is sent to a solver.
By the standard form I mean the form that can be used for the descent algorithms, so, for instance, it would convert all the absolute values in the objective to be a difference of two positive numbers with some new constraints, etc.
Wondering if its possible to see what the reduction looked like for a problem I specify in cvxpy?
For instance, lets say I have the following problem:
import numpy as np
import cvxpy as cp
x = cp.Variable(2)
L = np.asarray([[1,2],[2,3]])
P = L.T # L
constraints = []
constraints.append(x >= [-10, -10])
constraints.append(x <= [10, 10])
obj = cp.Minimize(cp.quad_form(x, P) - [1, 2] * x)
prob = cp.Problem(obj, constraints)
prob.solve(), prob.solver_stats.solver_name
(-0.24999999999999453, 'OSQP')
So, I would like to see the actual arguments (P, q, A, l, u) being sent to the OSQP solver https://github.com/oxfordcontrol/osqp-python/blob/master/module/interface.py#L278
Any help is greatly appreciated!
From looking at the documentation, it seems you can do this using the command get_problem_data as follows:
data, chain, inverse_data = prob.get_problem_data(prob.solver_stats.solver_name)
I have not tried it, and it says it output depends on the particular solver and the solver chain, but it may help you!

linear-regression with torch7 demo

I am following this demo-
https://github.com/torch/demos/blob/master/linear-regression/example-linear-regression.lua
feval = function(x_new)
-- set x to x_new, if differnt
-- (in this simple example, x_new will typically always point to x,
-- so the copy is really useless)
if x ~= x_new then
x:copy(x_new)
end
-- select a new training sample
_nidx_ = (_nidx_ or 0) + 1
if _nidx_ > (#data)[1] then _nidx_ = 1 end
local sample = data[_nidx_]
local target = sample[{ {1} }] -- this funny looking syntax allows
local inputs = sample[{ {2,3} }] -- slicing of arrays.
dl_dx:zero()
local loss_x = criterion:forward(model:forward(inputs), target)
model:backward(inputs, criterion:backward(model.output, target))
return loss_x, dl_dx
end
I have a few doubts in this function
Where is the argument x_new (or its copy x) used in the code?
What does _nidx_ = (_nidx_ or 0) + 1 mean?
what is the value of nidx when the function is first called?
Where is dl_dx updated? Ideally it should have been just after local loss_x is updated, but it isnt written explicitly
EDIT:
My point#4 is very clear now. For those who are interested-
(source- deep learning, oxford, practical 3 lab sheet)
Where is the argument x_new (or its copy x) used in the code?
x is the tensor of parameters of your model. It was previously acquired via x, dl_dx = model:getParameters(). model:forward() and model:backward() automatically use this parameter tensor. x_new is a new set of parameters for your model and is provided by the optimizer (SGD). If it is ever different from your model's parameter tensor, your model's parameters will be set to these new parameters via x:copy(x_new) (in-place copy of tensor's x_new values to x).
What does nidx = (nidx or 0) + 1 mean?
It increases the value of _nidx_ by 1 ((_nidx_) + 1) or sets it to 1 ((0) + 1) if _nidx_ was not yet defined.
what is the value of nidx when the function is first called?
It is never set before that function. Variables which were not yet set have the value nil in lua.
Where is dl_dx updated? Ideally it should have been just after local loss_x is updated, but it isnt written explicitly
dl_dx is the model's tensor of gradients. model:backward() computes the gradient per parameter given a loss and adds it to the model's gradient tensor. As dl_dx is the model's gradient tensor, its values will be increases. Notice that the gradient values are added, which is why you need to call dl_dx:zero() (sets the values of dl_dx in-place to zero), otherwise your gradient values would keep increasing with every call of feval.
x is a global variable, see line 126. The function only seems to update it, not to use it.
This is a common lua idiom: you set things to a parameter or a default value if it is not present. Typical use in functions:
function foo(a, b)
local a = a or 0
local b = b or "foo"
end
The idea is that an expression using and or or evaluates to the first or the second argument, according to the values. x and y yields y if x is not nil or false and x (nil or false) otherwise.
x or y yields y if x is not present (nil or false) and x otherwise. Therefore, or is used for default arguments.
The two can be rewritten the following way:
-- x and y
if x then
return y
else
return x
end
-- x or y
if x then
return x
else
return y
end
you have _nidx_ = (_nidx or 0) + 1, so at the first call of the function, _nidx_ is nil, since it has been defined nowhere. After that, it is (globally) set to 1 (0 + 1)
I'm not sure what you mean exactly. It is reset in line 152 and returned by the function itself. It is a global variable, so maybe there is an outer use for it?

Minimum and maximum values of integer variable

Let's assume a very simple constraint: solve(x > 0 && x < 5).
Can Z3 (or any other SMT solver, or any other automatic technique)
compute the minimum and maximum values of (integer) variable x that satisfies the given constraints?
In our case, the minimum is 1 and the maximum is 4.
Z3 has not support for optimizing (maximizing/minimizing) objective functions or variables.
We plan to add this kind of capability, but it will not happen this year.
In the current version, we can "optimize" an objective function by solving several problems where in each iteration we add additional constraints. We know we found the optimal when the problem becomes unsatisfiable. Here is a small Python script that illustrates the idea. The script maximizes the value of a variable X. For minimization, we just have to replace s.add(X > last_model[X]) with s.add(X < last_model[X]). This script is very naive, it performs a "linear search". It can be improved in many ways, but it demonstrates the basic idea.
You can also try the script online at: http://rise4fun.com/Z3Py/KI1
See the following related question: Determine upper/lower bound for variables in an arbitrary propositional formula
from z3 import *
# Given formula F, find the model the maximizes the value of X
# using at-most M iterations.
def max(F, X, M):
s = Solver()
s.add(F)
last_model = None
i = 0
while True:
r = s.check()
if r == unsat:
if last_model != None:
return last_model
else:
return unsat
if r == unknown:
raise Z3Exception("failed")
last_model = s.model()
s.add(X > last_model[X])
i = i + 1
if (i > M):
raise Z3Exception("maximum not found, maximum number of iterations was reached")
x, y = Ints('x y')
F = [x > 0, x < 10, x == 2*y]
print max(F, x, 10000)
As Leonardo pointed out, this was discussed in detail before: Determine upper/lower bound for variables in an arbitrary propositional formula. Also see: How to optimize a piece of code in Z3? (PI_NON_NESTED_ARITH_WEIGHT related).
To summarize, one can either use a quantified formula, or go iteratively. Unfortunately, these techniques are not equivalent:
Quantified approach needs no iteration, and can find global min/max in a single call to the solver; at least in theory. However, it does give rise to harder formulas. So, the backend solver can time-out, or simply return "unknown".
Iterative approach creates simple formulas for the backend solver to deal with, but it can loop forever if there's no optimal value; simplest example being trying to find the largest Int value. Quantified version can solve this problem nicely by quickly telling you that there is no such value, while the iterative version would go on indefinitely. This can be a problem if you don't know ahead of time that your constraints do have an optimal solution. (Needless to say, the "sufficient" iteration count is typically hard to guess, and might depend on random factors, like the seed used by the solver.)
Also keep in mind that if there is a custom optimization algorithm for the problem domain at hand, it's unlikely that a general purpose SMT solver can outperform it.
z3 now supports optimization.
from z3 import *
o = Optimize()
x = Int( 'x' )
o.add(And(x > 0, x < 5))
o.maximize(x)
print(o.check()) # prints sat
print(o.model()) # prints [x = 4]
This particular problem is an integer program.

Resources