I'd like to plot the following piecewise function with Fourier series in wxMaxima:
for given values of constants.
Here's my current input in wxMaxima:
a_1(t):=A_0+sum(A_n*cos(n*ω*(t-t_0))+B_n*sin(n*ω*(t-t_0)), n, 1, N);
a_2(t):=A_0;
a(t):=if(is(t>=t_0)) then a_1(t) else a_2(t);
N=2$
ω=31.416$
t_0=-0.1614$
A_0=0$
A_1=0.227$
B_1=0$
A_2=0.413$
B_2=0$
plot2d([a(t)], [t,0,0.5])$
Unfortunately, it doesn't work. I get the expression evaluates to non-numeric value everywhere in plotting range error. What can I do to make it work? Is it possible to plot this function in wxMaxima?
UPDATE: It works with modifications suggested by Robert Dodier:
a_1(t):=A[0]+sum(A[n]*cos(n*ω*(t-t_0))+B[n]*sin(n*ω*(t-t_0)), n, 1, N);
a_2(t):=A[0];
a(t):=if t>=t_0 then a_1(t) else a_2(t);
N:2$
ω:31.416$
t_0:-0.1614$
A[0]:0$
A[1]:0.227$
B[1]:0$
A[2]:0.413$
B[2]:0$
wxplot2d([a(t)], [t,0,0.5], [ylabel,"a"])$
I'm trying to understand RNNs and I would like to find a simple example that actually shows the one hot vectors and the numerical operations. Preferably conceptual since actual code may make it even more confusing. Most examples I google just show boxes with loops coming out of them and its really difficult to understand what exactly is going on. In the rare case where they do show the vectors its still difficult to see how they are getting the values.
for example I don't know where the values are coming from in this picture https://i1.wp.com/karpathy.github.io/assets/rnn/charseq.jpeg
If the example could integrate LSTMs and other popular extensions that would be cool too.
In the simple RNN case, a network accepts an input sequence x and produces an output sequence y while a hidden sequence h stores the network's dynamic state, such that at timestep i: x(i) ∊ ℝM, h(i) ∊ ℝN, y(i) ∊ ℝP the real valued vectors of M/N/P dimensions corresponding to input, hidden and output values respectively. The RNN changes its state and omits output based on the state equations:
h(t) = tanh(Wxh ∗ [x(t); h(t-1)]), where Wxh a linear map: ℝM+N ↦ ℝN, * the matrix multiplication and ; the concatenation operation. Concretely, to obtain h(t) you concatenate x(t) with h(t-1), you apply matrix multiplication between Wxh (of shape (M+N, N)) and the concatenated vector (of shape M+N) , and you use a tanh non-linearity on each element of the resulting vector (of shape N).
y(t) = sigmoid(Why * h(t)), where Why a linear map: ℝN ↦ ℝP. Concretely, you apply matrix multiplication between Why (of shape (N, P)) and h(t) (of shape N) to obtain a P-dimensional output vector, on which the sigmoid function is applied.
In other words, obtaining the output at time t requires iterating through the above equations for i=0,1,...,t. Therefore, the hidden state acts as a finite memory for the system, allowing for context-dependent computation (i.e. h(t) fully depends on both the history of the computation and the current input, and so does y(t)).
In the case of gated RNNs (GRU or LSTM), the state equations get somewhat harder to follow, due to the gating mechanisms which essentially allow selection between the input and the memory, but the core concept remains the same.
Numeric Example
Let's follow your example; we have M = 4, N = 3, P = 4, so Wxh is of shape (7, 3) and Why of shape (3, 4). We of course do not know the values of either W matrix, so we cannot reproduce the same results; we can still follow the process though.
At timestep t<0, we have h(t) = [0, 0, 0].
At timestep t=0, we receive input x(0) = [1, 0, 0, 0]. Concatenating x(0) with h(0-), we get [x(t); h(t-1)] = [1, 0, 0 ..., 0] (let's call this vector u to ease notation). We apply u * Wxh (i.e. multiplying a 7-dimensional vector with a 7 by 3 matrix) and get a vector v = [v1, v2, v3], where vi = Σj uj Wji = u1 W1i + u2 W2i + ... + u7 W7i. Finally, we apply tanh on v, obtaining h(0) = [tanh(v1), tanh(v2), tanh(v3)] = [0.3, -0.1, 0.9]. From h(0) we can also get y(0) via the same process; multiply h(0) with Why (i.e. 3 dimensional vector with a 3 by 4 matrix), get a vector s = [s1, s2, s3, s4], apply sigmoid on s and get σ(s) = y(0).
At timestep t=1, we receive input x(1) = [0, 1, 0, 0]. We concatenate x(1) with h(0) to get a new u = [0, 1, 0, 0, 0.3, -0.1, 0.9]. u is again multiplied with Wxh, and tanh is again applied on the result, giving us h(1) = [1, 0.3, 1]. Similarly, h(1) is multiplied by Why, giving us a new s vector on which we apply the sigmoid to obtain σ(s) = y(1).
This process continues until the input sequence finishes, ending the computation.
Note: I have ignored bias terms in the above equations because they do not affect the core concept and they make notation impossible to follow
In LATCH_match.cpp in opencv_3.1.0 the homography matrix is defined and used as:
Mat homography;
FileStorage fs("../data/H1to3p.xml", FileStorage::READ);
...
fs.getFirstTopLevelNode() >> homography;
...
Mat col = Mat::ones(3, 1, CV_64F);
col.at<double>(0) = matched1[i].pt.x;
col.at<double>(1) = matched1[i].pt.y;
col = homography * col;
...
Why H1to3p.xml is:
<opencv_storage><H13 type_id="opencv-matrix"><rows>3</rows><cols>3</cols><dt>d</dt><data>
7.6285898e-01 -2.9922929e-01 2.2567123e+02
3.3443473e-01 1.0143901e+00 -7.6999973e+01
3.4663091e-04 -1.4364524e-05 1.0000000e+00 </data></H13></opencv_storage>
With which criteria these numbers were chosen? They can be used for any other homography test for filtering keypoints (as in LATCH_match.cpp)?
I assume that your "LATCH_match.cpp in opencv_3.1.0" is
https://github.com/Itseez/opencv/blob/3.1.0/samples/cpp/tutorial_code/xfeatures2D/LATCH_match.cpp
In that file, you find:
// If you find this code useful, please add a reference to the following paper in your work:
// Gil Levi and Tal Hassner, "LATCH: Learned Arrangements of Three Patch Codes", arXiv preprint arXiv:1501.03719, 15 Jan. 2015
And so, looking at http://arxiv.org/pdf/1501.03719v1.pdf you will find
For each set, we compare the first image against each of the remaining
five and check for correspondences. Performance is measured using the
code from [16, 17]1 , which computes recall and 1-precision
using known ground truth homographies between the images.
I think that the image ../data/graf1.png is https://github.com/Itseez/opencv/blob/3.1.0/samples/data/graf1.png that I show here:
According to the comment Homography matrix in Opencv? by Catree the original dataset is at http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/graf.tar.gz where it is said that
Homographies between image pairs included.
So I think that the homography stored in file ../data/H1to3p.xml is the homography between image 1 and image 3.
I want to verify that homography matrix will give good results and this this answer
has an answer for it - but, I don't know how to implement the answer. So can anyone recommend how I may use OpenCV to compute SVD and and verify that the ratio of the first-to-last singular value is sane?
There are several ways to compute the SVD in OpenCV:
cv::SVD homographySVD(homography, cv::SVD::FULL_UV); // constructor
// or:
homographySVD(newHomography, cv::SVD::FULL_UV); // operator ()
homographySVD.w.at<double>(0, 0); // access the first singular value
// alternatives:
cv::SVD::compute(homography, w); // compute just the singular values
cv::eigen(homography, w);
Have a look at the documentation for cv::SVD and cv::eigen for more details.
You can compute SVD in python using numpy.
For example:
import numpy as np
U, s, V = np.linalg.svd(a, full_matrices=True)
which factors the matrix a of dimension M x N as u * np.diag(s) * v, where u and v are unitary and s is a 1-d array of a's singular values.
More can be found here.
Assuming "A" is a real vector packed (with vDSP_ctoz) in the proper way, doing:
vDSP_fft_zrip(setupReal, &A, 1, FFT_LENGTH_LOG2, FFT_FORWARD);
Will transform my real content to it's frequency representation.
What do then the following values represents ?:
A.realp[0];
A.imagp[0];
A.realp[i];
A.imagp[i];
A.realp[N-1];
A.imagp[N-1];
Actually I'm wondering where is the DC and Nyquist components stored. Also is the A.imagp[j] the imaginary part of A.realp[j] ?
Let H be the vector that is the mathematical result of the FFT, so that Hk is the kth element of the vector. H0 is the DC component, and HN/2 is the Nyquist component. Then:
A.realp[0] contains H0.
A.imagp[0] contains HN/2.
For 0 < k < N/2, A.realp[k] and A.imagp[k] combined contain Hk. Specifically, A.realp[k] contains the real part of Hk, and A.imagp[k] contains the imaginary part of Hk. Equivalently, Hk = A.realp[k] + i • A.imagp[k].
Some documentation about the vDSP FFTs is here.