I am thinking of process an image to generate in Mathematica given its powerful image processing capabilities. Could anyone give some idea as to how to do this?
Thanks a lot.
Here's one version, using a textures. It of course doesn't act as a real lens, just repeats portions of the image in an overlapping fashion.
t = CurrentImage[];
(* square off the image to avoid distortion *)
t = ImageCrop[t, {240,240}];
n = 20;
Graphics[{Texture[t],
Table[
Polygon[
Table[h*{Sqrt[3]/2, 0} + (g - h)*{Sqrt[3]/4, 3/4} + {Sin[t], Cos[t]},
{t, 0., 2*Pi - Pi/3, Pi/3}
],
VertexTextureCoordinates -> Transpose[{
Rescale[
(1/4)*Sqrt[3]*(g - h) + (Sqrt[3]*h)/2.,
{-n/2, n/2},
{0, 1}
] + {0, Sqrt[3]/2, Sqrt[3]/2, 0, -(Sqrt[3]/2), -(Sqrt[3]/2)}/(n/2),
Rescale[
(3.*(g - h))/4,
{-n/2, n/2},
{0, 1}
] + {1, 1/2, -(1/2), -1, -(1/2), 1/2}/(n/2)
}]
],
{h, -n, n, 2},
{g, -n, n, 2}
]
},
PlotRange -> n/2 - 1
]
Here's the above code applied to the standard image test (Lena)
"I think this could be well approximated with a calculated offset for the image in each cell" - Mr.Wizard
Exactly! As you can see from reconstructed image there is no lens effect and tiles are just displacements.
What you need is a Hexagonal_tessellation and a simple algorithm to calculate displacement for each hexagon from some chosen central point (weight/2, height/2).
Related
I'm using Pytorch to experiment image segmentation task. I found input and output shape are often inconsistent after applying Conv2d() and Convtranspose2d() to my image data of shape [1,1,height,width]). How to fix it the issue for arbitrary height and width?
Best regards
import torch
data = torch.rand(1,1,16,26)
a = torch.nn.Conv2d(1,1,kernel_size=3, stride=2)
b = a(data)
print(b.shape)
c = torch.nn.ConvTranspose2d(1,1,kernel_size=3, stride=2)
d = c(b)
print(d.shape) # torch.Size([1, 1, 15, 25])
TLDR; Given the same parameters nn.ConvTranspose2d is not the invert operation of nn.Conv2d in terms of dimension shape conservation.
From an input with spatial dimension x_in, nn.Conv2d will output a tensor with respective spatial dimension x_out:
x_out = [(x_in + 2p - d*(k-1) - 1)/s + 1]
Where [.] is the whole part function, p the padding, d the dilation, k the kernel size, and s the stride.
In your case: k=3, s=2, while other parameters default to p=0 and d=1. In other words x_out = [(x_in - 3)/2 + 1]. So given x_in=16, you get x_out = [7.5] = 7.
On the other hand, we have for nn.ConvTranspose2d:
x_out = (x_in-1)*s - 2p + d*(k-1) + op + 1
Where [.] is the whole part function, p the padding, d the dilation, k the kernel size, s the stride, and op the output padding.
In your case: k=3, s=2, while other parameters default to p=0, d=1, and op=0. You get x_out = (x_in-1)*2 + 3. So given x_in=7, you get x_out = 15.
However, if you apply an output padding on your transpose convolution, you will get the desired shape:
>>> conv = nn.Conv2d(1,1, kernel_size=3, stride=2)
>>> convT = nn.ConvTranspose2d(1, 1, kernel_size=3, stride=2, output_padding=1)
>>> convT(conv(data)).shape
torch.Size([1, 1, 16, 26])
I am trying to solved a mixed-integer non-linear program with Drake using the Snopt solver, but I encounter the following error:
ValueError: The capabilities of drake::solvers::SnoptSolver do not meet the requirements of the MathematicalProgram ({ProgramAttributes: GenericConstraint, QuadraticCost, LinearConstraint, LinearEqualityConstraint, BinaryVariable})
What is the recommended solver/alternative approach in my case?
Mixed integer nonlinear optimizations are very hard problems and only a handful of optimization solvers exist for that, e.g.: Couenne, Baron. Drake does not support any of these solvers.
Drake supports solvers for most common mixed-integer convex programs (such as MILPs, MIQPs, MISOCs). But for using these, your optimization problem must only have convex constraints (e.g. linear equalities and inequalities).
One way to proceed using a nonlinear solver such as SNOPT to solve your optimization problem is to enforce binary constraints as follow. Say you want x to be binary, then you can add the smooth constraint x * (x - 1) = 0. (Note that the latter is verified iff x = 0 or x = 1.) This however is a very "tough" constraint, and can lead to numeric issues. Note also that doing this you do not have any guarantees in terms of finding a feasible solution if one exists (guarantees that you do have in mixed-integer convex programming), and the solver might converge to a local minimum.
As you mentioned that your constraint is
x[k+1] = A * ((1-b[k]) * x1[k] + b[k] * x2[k])
where b[k] is a binary variable, x1[k], x2[k], x[k+1] are continuous variables.
This constraint could be reformulated as a mixed-integer linear constraint. What you want is
b[k] = 1, then x[k+1] = A * x2[k]
b[k] = 0, then x[k+1] = A * x1[k]
Generally, if by fixing the binary variable to either 0 or 1, the remaining constraint is linear, then the constraints can be reformulated as mixed-integer linear constraints. There are two approaches to convert your constraint to mixed-integer linear constraint, namely the big-M approach and the convex hull approach, as explained in this tutorial.
As a quick demo on the convex hull approach, suppose that your variable are bounded
x1[k] ∈ ConvexHull(v₁, v₂, ..., vₘ)
x2[k] ∈ ConvexHull(w₁, w₂, ..., wₙ)
where v₁, v₂, ..., vₘ, w₁, w₂, ..., wₙ are all given points.
We introduce two slack variables s1, s2. We intend to impose the following constraints
b[k] = 1, then s1 = 0, s2 = A * x2[k]
b[k] = 0, then s2 = 0, s1 = A * x1[k]
x[k+1] = s1 + s2
The constraints above means that the convex hull of the point (b[k], s2, x2[k]) is the polytope with vertices
(1, A * w₁, w₁)
(1, A * w₂, w₂)
...
(1, A * wₙ, wₙ)
(0, 0, w₁)
(0, 0, w₂)
...
(0, 0, wₙ)
With these vertices, the polytope is written in the V-representation. You could convert this V-representation to an H-representation. This H-representation is denoted as
H * [b[k], s2, x2[k]] <= h
Where each row of H is a face normal of the polytope. This H-representation is a (mixed-integer) linear constraint on the variables b[k], s2, x2[k]. An alternative formulation to constrain that b[k], s2, x2[k] to lie within the polytope is to write (b[k], s2, x2[k]) as a convex combination of the polytope vertices (You will need to introduce the convex combination weights as decision variables also, with all the weights being non-negative, and the sum of the weights equal to 1). Using this convex combination approach you won't need to convert the V-representation to H-representation. Here is the mixed-integer linear constraints on b[k], s2, x2[k] using the convex combination approach.
b[k] = λ₁ + λ₂ + ... + λₙ
s2 = A*(λ₁w₁ + λ₂w₂ + ... + λₙwₙ)
x2[k] = λ₁w₁ + λ₂w₂ + ... + λₙwₙ + λₙ₊₁w₁ + λₙ₊₂w₂ + ... + λ₂ₙwₙ
1 = λ₁ + λ₂ + ... + λₙ + λₙ₊₁ + λₙ₊₂ + ... + λ₂ₙ
λᵢ ≥ 0 ∀ i = 1, ..., 2n
b[k] is binary
where the weights λᵢ, i = 1, ..., 2n are new slack variables representing the convex combination weights. You could verify that these constraints enforce that
if b[k] = 0, then s2 = 0.
if b[k] = 1, then s2 = A*x2[k]
Similarly we know that the convex hull of the point (b[k], s1, x1[k]) is a polytope with the following vertices
(1, 0, v₁)
(1, 0, v₂)
...
(1, 0, vₘ)
(0, A*v₁, v₁)
(0, A*v₂, v₂)
...
(0, A*vₘ, vₘ)
and we can write the linear constraints from the H-representation of the polytope. We combine the two sets of linear constraints derived from the polytopes, together with the linear constraint x[k+1] = s1 + s2, we get the mixed-integer linear constraints of your problem. For the detailed explanation of this convex hull approach, you could refer to the linked tutorial above.
I have the logistic map function in Maxima like so:
F(x,r,n):= x[n]=r*x[n-1]*(1-x[n-1]);
And when I input the correct variables it gives me the answer to, for example, x[0]:
(%i15) n:0$
x[n-1]:[0.1]$
F(x, r:3, n);
(%o15) x[0]=[0.27]
However, this answer does not stay memorized and when I enter x[0] I get
x[0];
(%o5) x[0]
How do I write a function that will calculate x[n] for me and store it in memory, so I can use it later? I am trying to make a bifurcation diagram for the logistic map without using any black boxes, i.e., the orbits functions.
Thank you!
There are different ways to go about it. One straightforward way is to create a list and then iterate, computing its elements one by one. E.g.:
(%i4) x: makelist (0, 10);
(%o4) [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
(%i5) x[1]: 0.1;
(%o5) 0.1
(%i6) r: 3;
(%o6) 3
(%i7) for i:2 thru 10 do x[i]: r * x[i - 1] * (1 - x[i - 1]);
(%o7) done
(%i8) x;
(%o8) [0.1, 0.2700000000000001, 0.5913000000000002,
0.7249929299999999, 0.5981345443500454, 0.7211088336156269,
0.603332651091411, 0.7179670896552621, 0.6074710434816448,
0.7153499244388992]
Note that : is the assignment operator, not =.
I am training 1000 images of 28x28 size. But before training, I am performing ZCA whitening on my data by taking the reference from How to implement ZCA Whitening? Python.
Since I have 1000 data images of size 28x28, after flattening, it becomes 1000x784.
But as given in the code below, whether X is my image dataset of 1000x784?
If it is so, then it means the ZCAMatrix size is 1000x1000.
In this case, for prediction I have a image of size 28x28, or we can say, size of 1x784.So it doesn't make sense to multiply ZCAMatrix to the image.
So I think, X is the transpose of image data set. Am I right?
If I am right, then the size of ZCAMatrix is 784x784.
Now how should I calculate the ZCA whitened image, whether I should use np.dot(ZCAMatrix, transpose_of_image_to_be_predict) or np.dot(image_to_be_predict, ZCAMatrix)?
Suggestion would be greatly appreciate.
def zca_whitening_matrix(X):
"""
Function to compute ZCA whitening matrix (aka Mahalanobis whitening).
INPUT: X: [M x N] matrix.
Rows: Variables
Columns: Observations
OUTPUT: ZCAMatrix: [M x M] matrix
"""
# Covariance matrix [column-wise variables]: Sigma = (X-mu)' * (X-mu) / N
sigma = np.cov(X, rowvar=True) # [M x M]
# Singular Value Decomposition. X = U * np.diag(S) * V
U,S,V = np.linalg.svd(sigma)
# U: [M x M] eigenvectors of sigma.
# S: [M x 1] eigenvalues of sigma.
# V: [M x M] transpose of U
# Whitening constant: prevents division by zero
epsilon = 1e-5
# ZCA Whitening matrix: U * Lambda * U'
ZCAMatrix = np.dot(U, np.dot(np.diag(1.0/np.sqrt(S + epsilon)), U.T)) # [M x M]
return ZCAMatrix
And an example of the usage:
X = np.array([[0, 2, 2], [1, 1, 0], [2, 0, 1], [1, 3, 5], [10, 10, 10] ]) # Input: X [5 x 3] matrix
ZCAMatrix = zca_whitening_matrix(X) # get ZCAMatrix
ZCAMatrix # [5 x 5] matrix
xZCAMatrix = np.dot(ZCAMatrix, X) # project X onto the ZCAMatrix
xZCAMatrix # [5 x 3] matrix
I got the reference from the Keras code available here.
It is very clear that in my case the co-variance matrix will give 784x784 matrix, on which Singular Value Decomposition is performed. It gives 3 matrix that is used to calculate the principal_components, and that principal_components is used to find the ZCA whitened data.
Now my question was
how should I calculate the ZCA whitened image, whether I should use
np.dot(ZCAMatrix, transpose_of_image_to_be_predict) or
np.dot(image_to_be_predict, ZCAMatrix)? Suggestion would be greatly
appreciate.
For this I got the reference from here.
Here I need to use np.dot(image_to_be_predict, ZCAMatrix) to calculate the ZCA whitened image.
I'm experimenting with the ImageTransformation function to try to make anamorphic versions of images, but with limited progress so far. I'm aiming for the results you get using the image reflected in a cylindrical mirror, where the image curves around the central mirror for about 270 degrees. The wikipedia article has a couple of neat examples (and I borrowed Holbein's skull from them too).
i = Import["../Desktop/Holbein_Skull.jpg"];
i = ImageResize[i, 120]
f[x_, y_] := {(2 (y - 0.3) Cos [1.5 x]), (2 (y - 0.3) Sin [1.5 x])};
ImageTransformation[i, f[#[[1]], #[[2]]] &, Padding -> White]
But I can't persuade Mathematica to show me the entire image, or to bend it correctly. The anamorphic image should wrap right round the mirror placed "inside" the centre of the image, but it won't. I found suitable values for constants by putting it inside a manipulate (and turning the resolution down :). I'm using the formula:
x1 = a(y + b) cos(kx)
y1 = a(y + b) sin(kx)
Any help producing a better result would be greatly appreciated!
In ImageTransformation[f,img], the function f is such that a point {x,y} in the resulting image corresponds to f[{x,y}] in img. Since the resulting image is basically the polar transformation of img, f should be the inverse polar transformation, so you could do something like
anamorphic[img_, angle_: 270 Degree] :=
Module[{dim = ImageDimensions[img], rInner = 1, rOuter},
rOuter = rInner (1 + angle dim[[2]]/dim[[1]]);
ImageTransformation[img,
Function[{pt}, {ArcTan[-#2, #1] & ## pt, Norm[pt]}],
DataRange -> {{-angle/2, angle/2}, {rInner, rOuter}},
PlotRange -> {{-rOuter, rOuter}, {-rOuter, rOuter}},
Padding -> White
]
]
The resulting image looks something like
anamorphic[ExampleData[{"TestImage", "Lena"}]]
Note that you can a similar result with ParametricPlot and TextureCoordinateFunction, e.g.
anamorphic2[img_Image, angle_: 270 Degree] :=
Module[{rInner = 1,rOuter},
rOuter = rInner (1 + angle #2/#1 & ## ImageDimensions[img]);
ParametricPlot[{r Sin[t], -r Cos[t]}, {t, -angle/2, angle/2},
{r, rInner, rOuter},
TextureCoordinateFunction -> ({#3, #4} &),
PlotStyle -> {Opacity[1], Texture[img]},
Mesh -> None, Axes -> False,
BoundaryStyle -> None,
Frame -> False
]
]
anamorphic2[ExampleData[{"TestImage", "Lena"}]]
Edit
In answer to Mr.Wizard's question, if you don't have access to ImageTransformation or Texture you could transform the image data by hand by doing something like
anamorph3[img_, angle_: 270 Degree, imgWidth_: 512] :=
Module[{data, f, matrix, dim, rOuter, rInner = 1.},
dim = ImageDimensions[img];
rOuter = rInner (1 + angle #2/#1 & ## dim);
data = Table[
ListInterpolation[#[[All, All, i]],
{{rOuter, rInner}, {-angle/2, angle/2}}], {i, 3}] &#ImageData[img];
f[i_, j_] := If[Abs[j] <= angle/2 && rInner <= i <= rOuter,
Through[data[i, j]], {1., 1., 1.}];
Image#Table[f[Sqrt[i^2 + j^2], ArcTan[i, -j]],
{i, -rOuter, rOuter, 2 rOuter/(imgWidth - 1)},
{j, -rOuter, rOuter, 2 rOuter/(imgWidth - 1)}]]
Note that this assumes that img has three channels. If the image has fewer or more channels, you need to adapt the code.