Based on some non-linear constraints on $a_k$,$b_k$, I have to find feasible set of the following fourier series expression:
$ x(t)= {a_0+ \sum_{k=1}^{\infty} (a_k\cos(2\pi f_0 kt)+(b_k\sin(2\pi f_0 kt))}
Whereas constraints on $a_k$,$b_k$ and $a_0$ are
$ L \leq a_0 \leq U $
$ Lower_bound \leq a_k^2+b_k^2 \leq Upper_bound
Can I do this using Z3?
In addition to this can I use Z3 for exponential functions having complex powers, e.g. in fourier transform expression.
Unfortunately, Z3 does not have support for transcendental functions such as sin, cos and exponential yet. The current version can only handle nonlinear polynomial constraints.
You may consider the MetiTarski theorem prover. BTW, MetiTarski uses Z3 to discharge nonlinear polynomial constraints.
Related
I need some help.
I have an equation:
f(x) = acos(x) + (bsqrt(x) + ctg(x))^2,
where a, b, c are unknown parameters.
Also I have a few pairs of x -> f(x) like in Supervised ML problem.
How can I find parameters? I'm thinking of some numerical methods and linear regression, but actually I don't know what to do.
The unknown parameters would minimize the sum of squared differences between computed function values and observed function values.
You could define such a sum of squared errors in Excel and use Excel Solver to minimize it.
From a Python program, you could try Scipy fsolve.
I understand that Z3 has some supports for nonlinear arith but wondering to what extends ? Is it possible to specify what classes of nonlinear arithmetics are supported and are not (or likely to give time out) ? Know these in advances will help me abort my task early.
Seems like power related stuff is not supported as shown below
def pow2(x):
k=Int('k')
return Exists(k, And(k>=0,2**k==x))
prove(pow2(7))
failed to prove
Z3 supports nonlinear polynomial Real arithmetic. So, there is no support for transcendental functions (e.g., sine and cosine), and exponential (e.g., 2^x). Actually, for the exponential, Z3 can handle exponents that can be simplified to numerals. Here is an example,
x = Real('x')
y = Real('y')
solve(y == 3, x**y == 2)
In this example, the y in x**y is rewritten to 3 during a preprocessing step. After preprocessing, the nlsat solver for nonlinear polynomial real arithmetic is invoked.
Regarding nonlinear integer arithmetic, see this related post.
This is a follow-up to my previous question on Z3's Model-based
Quantifier Instantiation (MBQI) and the stratified sorts fragment (thanks
again to Leonardo de Moura for the quick answer).
In their paper on decidable fragments of many-sorted logic [Abadi et
al., Decidable fragments of many-sorted logic, LPAR 2007], the authors
describe a fragment St1 of many-sorted logic that is decidable with a
finite model property.
This fragment requires the sorts to be stratified and the formula F to be in (skolemized) prenex normal form as described in the Z3
documentation, but allows an additional atomic formula
y in Im[f]
to occur in F, which is a "shorthand" for
exists x1 : A1, ..., xn : An . y = f(x1,...,xn)
where f is a function with a signature f : A1 x ... x An -> B, and f must be the only function with range B. Thus, the St1 fragment allows (in a very restricted way) to violate the stratification, e.g., in order to assert that f is surjective.
I am not sure if this could be an open research question:
Does someone know whether the MBQI decision procedure for Z3 is complete
for the St1 fragment? Will Z3 produce (theoretically) either SAT or
UNSAT for F after a finite time?
First, one clarification, in principle, MBQI can decide the stratified multi-sorted fragment. The justification is given in Section 4.1 of http://research.microsoft.com/en-us/um/people/leonardo/ci.pdf (*). However, Z3 4.0 does not support implement the additional rules suggested in Section 4.1. So, Z3 4.0, may fail (return unknown) on formulas that are in this fragment. I just want to make clear a distinction between the algorithm and the actual implementation using the current Z3.
Regarding your question, yes MBQI framework can decide the stratified formulas containing the expanded predicate y in Im[f]. I'm assuming this predicate occurs only positively.
That is, we do not have not y in Im[f] which is equivalent to
forall x1:A1, ...,xn:An. y != f(x1, ... xn)
If y in Im[f] occurs only positively, then it can be expanded, and after skolemization we have a ground formula of the form y = f(k1, ..., kn).
MBQI is still a decision procedure because the set F* defined in (*) will still be finite. F* may become infinite only if the stratification is broken inside of a universal formula.
Many thanks Josh and Leonardo for answering the previous question.
I have few more questions.
<1> Consider another example.
(exists k) i * k > = 4 and k > 1.
This has a simple solution i > 0. (both for Int and Real case)
However, when I tried following,
(declare-const i Int)
(assert (exists ((k Int)) (and (>= (* i k) 4) (> k 1))))
(apply (using-params qe :qe-nonlinear true))
Z3 Could not eliminate quantifier here.
However, it could eliminate for a Real case. (when i and k are both reals)
Is Quantifier Elimination more difficult for integers?
<2> I am using Z3 C API in my system. I am adding some non-linear constraints on Integers with quantifiers in my system.
Z3 currently checks for satisfiability and gives me a correct model when the system is satisfiable.
I know that after quantifier elimination, these constraints get reduced to linear constraints.
I thought that z3 does quantifier elimination automatically before checking satisfiability. But since, it couldn't do that in case 1 above, I now think, that it usually finds a model without Quantifier Elimination. Am I correct?
Currently z3 can solve the constraints in my system. But it may fail on complex systems.
In such case, is it a good idea to do quantifier elimination by some other method without z3 and add constraints to z3 later?
<3> I can think of adding Real non-linear constraints instead of Integer non-linear constraints in my system. In that case, how can I enforce z3 to do Quantifier Elimination using C-API ?
<4> Finally, is this a good idea to enforce z3 to do Quantifier Elimination? Or it usually finds a model more intelligently without doing Quantifier Elimination?
Thanks.
<1> The theory of nonlinear integer arithmetic does not admit quantifier elimination (qe).
Moreover, the decision problem for nonlinear integer arithmetic is undecidable.
Recall that, Z3 has limited support for quantifier elimination of nonlinear real arithmetic formulas. The current procedure is based on virtual term substitution. Future versions, may have full support for nonlinear real arithmetic.
<2> Quantifier elimination is not enabled by default. The user must request it.
Z3 may find models for satisfiable formulas even when quantifier elimination is not enabled.
It uses a technique called model-based quantifier instantiation (MBQI). The Z3 online tutorial has several examples describing capabilities and limitations of this technique.
<3> You have to enable it when you create the Z3_context object.
Any option that is set in the command line, can be provided during Z3_context object creation. Here is an example, that enables model construction and quantifier elimination:
Z3_config cfg = Z3_mk_config();
Z3_context ctx;
Z3_set_param_value(cfg, "MODEL", "true");
Z3_set_param_value(cfg, "ELIM_QUANTIFIERS", "true");
Z3_set_param_value(cfg, "ELIM_NLARITH_QUANTIFIERS", "true");
ctx = mk_context_custom(cfg, throw_z3_error);
Z3_del_config(cfg);
After that, ctx is pointing to a Z3 context object that supports model construction and quantifier elimination.
<4> The MBQI module is not complete even for the linear arithmetic fragment.
The Z3 online tutorial describes the fragments it is complete. MBQI module is a good option for problems that contain uninterpreted functions. If your problems only use arithmetic, then quantifier elimination is usually better and more efficient. That being said, several problems can be quickly solved using MBQI.
Does anyone has an idea how to calculate the Inverse of a 2-D filter?
Let's say I have a 3x3 filter:
0 1 0
1 1 1
0 1 0
I want to find it's inverse.
It's easy to do using DFT.
But let's say I want to do it by convolution.
Now, that's the problem, Matlab symbolic isn't my specialty.
Assuming there's a 3X3 Inverse Filter it means convolution of the two will result in:
0 0 0
0 1 0
0 0 0
The problem is to create the right set of equations for that and solving it.
Doing it with symbols is easy to think yet I couldn't do it.
Any ideas?
Thanks.
P.S.
I'm not sure there an Inverse Filter for this one as it has zeros in its DTFT.
Moreover, someone should allow Latex in this forum the way MathOverflow has.
Let h[n] be the finite impulse response of a 1D filter. What does that imply about its inverse filter? The inverse cannot possibly be FIR.
Proof: Let H(omega) G(omega) = 1 for all omega, where H is the DTFT of h[n], and G is the DTFT of g[n]. If h[n] is FIR, then g[n] must be IIR.
Of course, there are ways to approximate the inverse IIR filter with an FIR filter. A basic method is adaptive filtering, e.g., Least Mean Squares (LMS) algorithm. Or just truncate the IIR filter. You still need to worry about stability, though.
For practical purposes, there probably is no desirable solution to your specific question. Especially if, for example, this is in image processing and you are trying to inverse an FIR blur filter with FIR sharpening filter. The final image will not look that good, period, unless your sharpening filter is really, really large.
EDIT: Let y[n] = b0 x[n-0] + b1 x[n-1] + ... + bN x[n-N]. Let this equation characterize the forward system, where y is the output and x is the input. By definition, the impulse response is the output when the input is an impulse: h[n] = b0 d[n-0] + b1 d[n-1] + ... + bN d[n-N]. This impulse response has finite length N+1.
Now, consider the inverse system where x is the output and y is the input. Then the impulse response is described by the recurrence equation d[n] = b0 h[n] + b1 h[n-1] + ... + bN h[n-N]. Equivalently, b0 h[n] = d[n] - b1 h[n-1] - ... - bN h[n-N].
Without loss of generality, assume that b0 and bN are both nonzero. For any m, if h[m] is nonzero, then h[m+N] is also nonzero. Because this system has feedback, its impulse response is infinitely long. QED.
Causality does not matter. The inverse of a delay is an advance, and vice versa. Neither a delay or an advance alter the finiteness of an impulse response. Shift an infinite impulse response left or right; it is still infinite.
EDIT 2: For clarification, this proof is not related to my original "proof". One was in frequency domain, the other in time domain.
In practice, one useful solution is Wiener deconvolution http://en.wikipedia.org/wiki/Wiener_deconvolution which basically, in Fourier space, divides by the spectrum of the given filter. Zeros are handled by adding a fudge term: instead of 1/H(w), use H(w)/( abs(H(w))^2 + c) where H(w) is the discrete Fourier transform of the filter h(x) (add a 2nd dimension as you like) and "w" is supposed to be omega. The constant is chosen based on the noise level of the signal or image.
For a separable filter (i.e., one in which you can pull out a horizontal and vertical filter which can be applied in any order and give the same result as the composite 2-D filter), you could attempt to calculate the inverse of each on individually.
The inverse of a filter H(w) is G(w)=1/H(w), so one way to do it is to take the impulse response (the h[n] time-domain coeffs) and inverse-DFT them. It's not always easy to get an analytic expression for such a filter, so you could either compute it numerically (approximating to the desired precision) or do adaptive inverse filtering, as Steve suggested. See Widrow and Stearn's Adaptive Signal Processing for more info on this latter method.
One way is to write a matrix representation of convolution with your filter and then try to find some (regularized) inverse of this matrix.
For example the 1D [1,-1] filter is maybe the simplest discrete differential approximation that exists.
Its matrix will have two diagonals 1 on main diagonal and -1 on first off diagonal.
It's inverse will be integral filter. An IIR filter with infinite trail of 1s. Since our matrix has limited size this will in practice mean filling up the matrix with 1s everywhere on one side of the diagonal.
Deriving the Inverse Kernel of a Given 2D Convolution Kernel
This is basically a generalization of the question - Deriving the Inverse Filter of Image Convolution Kernel.
Problem Formulation
Given a Convolution Kernel $ f \in \mathbb{R}^{m \times n} $ find its inverse kernel, $ g \in \mathbb{R}^{p \times q} $ such that $ f \ast g = h = \delta $.
Solution I
One could build the Matrix Form of the Convolution Operator.
The Matrix form can replicate various mode of Image Filtering (Exclusive):
Boundary Conditions (Applied as padding the image and apply Convolution in Valid mode).
Zero Padding
Padding the image with zeros.
Circular
Using Circular / Periodic continuation of the image.
Match Frequency Domain convolution.
Replicate
Replicating the edge values (Nearest Neighbor).
Usually creates the least artifacts in real world.
Symmetric
Mirroring the image along the edges.
Convolution Shape
Full
The output size is to full extent - $ \left( m + p - 1 \right) \times \left( n + q - 1 \right) $.
Same
Output size is the same as the input size ("Image").
Valid
Output size is the size of full overlap of the image and kernel.
When using Image Filtering the output size matches the input size hence the Matrix Form is square and the inverse is defined.
Using Convolution Matrix the matrix might not be square (Unless "Same" is chosen) hence the Pseudo Inverse should be derived.
One should notice that while the input matrix should be sparse the inverse matrix isn't.
Moreover while the Convolution Matrix will have special form (Toeplitz neglecting the Boundary Conditions) the inverse won't be.
Hence this solution is more accurate than the next one yet it also use higher degree of freedom (The solution isn't necessarily in Toeplitz form).
Solution II
In this solution the inverse is derived using minimization of the following cost function:
$$ \arg \min_{g} \frac{1}{2} {\left| f \ast g - h \right|}_{2}^{2} $$
The derivative is given by:
$$ \frac{\partial \frac{1}{2} {\left| f \ast g - h \right|}_{2}^{2} }{\partial g} = f \star \left( f \ast g - h \right) $$
Where $ \star $ is the Correlation operation.
In practice the convolution in the Objective Function is done in full mode (MATLAB idiom).
In practice it is done:
hObjFun = #(mG) 0.5 * sum((conv2(mF, mG, 'full') - mH) .^ 2, 'all');
mObjFunGrad = conv2(conv2(mF, mG, 'full') - mH, mF(end:-1:1, end:-1:1), 'valid');
Where the code flips mF for Correaltion and use valid (In matrix form it is the Adjoint / Transpose -> the output is smaller).
The optimization problem is Strictly Convex and can be solved easily using Gradient Descent:
for ii = 1:numIteraions
mObjFunGrad = conv2(conv2(mF, mG, 'full') - mH, mF(end:-1:1, end:-1:1), 'valid');
mG = mG - (stepSize * mObjFunGrad);
end
Full code is available in my StackExchnage Code StackOverflow Q2080835 GitHub Repository (Have a look at StackOverflow\Q2080835 folder).