This plots but the result of conv is a vector of a new length and so t is usless to include in plot like plot(t, z1) %doesn't work!.
t = [-5:.1:10];
unit = #(t) 1.*(t>=0);
h1 = #(t) (3*t + 2).*exp(-3*t).*unit(t);
z1 = conv(unit(t), h1(t));
plot(z1);
I want a plot of the convolved signal as a function of time.
You need to add the shape argument. Here's the spec:
— Function File: conv (a, b) — Function File: conv (a, b, shape)
Convolve two vectors a and b.
The output convolution is a vector with length equal to length (a) +
length (b) - 1. When a and b are the coefficient vectors of two
polynomials, the convolution represents the coefficient vector of the
product polynomial.
The optional shape argument may be
shape = "full" Return the full convolution. (default) shape = "same"
Return the central part of the convolution with the same size as a.
so convolve like this:
z1 = conv(unit(t), h1(t), "same");
And you'll get the same time units as the original.
Related
In layer normalization, we compute mean and variance across the input layer (instead of across batch which is what we do in batch normalization). And then normalize the input layer according to mean and variance, and then return gamma times normalized layer plus beta.
My question is, are the gamma and beta scalars with shape (1, 1) and (1, 1) respectively or their shapes are (1, number of hidden units) and (1, number of hidden units) respectively.
Here is how I have implemented the layer normalization, is this correct!
def layernorm(layer, gamma, beta):
mean = np.mean(layer, axis = 1, keepdims = True)
variance = np.mean((layer - mean) ** 2, axis=1, keepdims = True)
layer_hat = (layer - mean) * 1.0 / np.sqrt(variance + 1e-8)
outpus = gamma * layer_hat + beta
return outpus
where gamma and beta are defined as below:
gamma = np.random.normal(1, 128)
beta = np.random.normal(1, 128)
According to the Tensorflow's implementation, assume the input has shape [B, rest], gamma and beta are of shape rest. rest could be (h, ) for a 2-dimensional input or (h, w, c) for a 4-dimensional input.
I would like to generate a polynomial 'fit' to the cluster of colored pixels in the image here
(The point being that I would like to measure how much that cluster approximates an horizontal line).
I thought of using grabit or something similar and then treating this as a cloud of points in a graph. But is there a quicker function to do so directly on the image file?
thanks!
Here is a Python implementation. Basically we find all (xi, yi) coordinates of the colored regions, then set up a regularized least squares system where the we want to find the vector of weights, (w0, ..., wd) such that yi = w0 + w1 xi + w2 xi^2 + ... + wd xi^d "as close as possible" in the least squares sense.
import numpy as np
import matplotlib.pyplot as plt
def rgb2gray(rgb):
return np.dot(rgb[...,:3], [0.299, 0.587, 0.114])
def feature(x, order=3):
"""Generate polynomial feature of the form
[1, x, x^2, ..., x^order] where x is the column of x-coordinates
and 1 is the column of ones for the intercept.
"""
x = x.reshape(-1, 1)
return np.power(x, np.arange(order+1).reshape(1, -1))
I_orig = plt.imread("2Md7v.jpg")
# Convert to grayscale
I = rgb2gray(I_orig)
# Mask out region
mask = I > 20
# Get coordinates of pixels corresponding to marked region
X = np.argwhere(mask)
# Use the value as weights later
weights = I[mask] / float(I.max())
# Convert to diagonal matrix
W = np.diag(weights)
# Column indices
x = X[:, 1].reshape(-1, 1)
# Row indices to predict. Note origin is at top left corner
y = X[:, 0]
We want to find vector w that minimizes || Aw - y ||^2
so that we can use it to predict y = w . x
Here are 2 versions. One is a vanilla least squares with l2 regularization and the other is weighted least squares with l2 regularization.
# Ridge regression, i.e., least squares with l2 regularization.
# Should probably use a more numerically stable implementation,
# e.g., that in Scikit-Learn
# alpha is regularization parameter. Larger alpha => less flexible curve
alpha = 0.01
# Construct data matrix, A
order = 3
A = feature(x, order)
# w = inv (A^T A + alpha * I) A^T y
w_unweighted = np.linalg.pinv( A.T.dot(A) + alpha * np.eye(A.shape[1])).dot(A.T).dot(y)
# w = inv (A^T W A + alpha * I) A^T W y
w_weighted = np.linalg.pinv( A.T.dot(W).dot(A) + alpha * \
np.eye(A.shape[1])).dot(A.T).dot(W).dot(y)
The result
# Generate test points
n_samples = 50
x_test = np.linspace(0, I_orig.shape[1], n_samples)
X_test = feature(x_test, order)
# Predict y coordinates at test points
y_test_unweighted = X_test.dot(w_unweighted)
y_test_weighted = X_test.dot(w_weighted)
# Display
fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.imshow(I_orig)
ax.plot(x_test, y_test_unweighted, color="green", marker='o', label="Unweighted")
ax.plot(x_test, y_test_weighted, color="blue", marker='x', label="Weighted")
fig.legend()
fig.savefig("curve.png")
For simple straight line fit, set the argument order of feature to 1. You can then use the gradient of the line to get a sense of how close it is to a horizontal line (e.g., by checking the angle of its slope).
It is also possible to set this to any degree of polynomial you want. I find that degree 3 looks pretty good. In this case, the 6 times the absolute value of the coefficient corresponding to x^3 (w_unweighted[3] or w_weighted[3]) is one measure of the curvature of the line.
See A measure for the curvature of a quadratic polynomial in Matlab for additional details.
I am trying to implement a convolutional neural network from scratch and I am not able to figure out how to perform (vectorized)operations on multi-channel images like rgb, which have 3 dimensions. On following the articles and tutorials such as this CS231n tutorial , it's pretty clear to implement a network for a single input as the input layer will be a 3d matrix but there are always multiple data points in a dataset. so, I cannot figure out how to implement these networks for vectorized operation on entire datsets.
I have implemented a network which takes a 3d matrix as input but now I have realized that It will not work on entire dataset but I will have to propagate one input at a time.I don't really know whether conv nets are vectorized over entire dataset or not .But if they are, how can I vectorize my convolutional network for multi-channel images ?
If I got your question right, you're basically asking how to do convolutional layer for a mini-batch, which will be a 4-D tensor.
To put it simply, you want to treat each input in a batch independently and apply convolution to each one. It's fairly straightforward to code without vectorization using a loop.
A vectorization implementation is often based on im2col technique, which basically transforms the 4-D input tensor into a giant matrix and performs a matrix multiplication. Here's an implementation of a forward pass using numpy.lib.stride_tricks in python:
import numpy as np
def conv_forward(x, w, b, stride, pad):
N, C, H, W = x.shape
F, _, HH, WW = w.shape
# Check dimensions
assert (W + 2 * pad - WW) % stride == 0, 'width does not work'
assert (H + 2 * pad - HH) % stride == 0, 'height does not work'
# Pad the input
p = pad
x_padded = np.pad(x, ((0, 0), (0, 0), (p, p), (p, p)), mode='constant')
# Figure out output dimensions
H += 2 * pad
W += 2 * pad
out_h = (H - HH) / stride + 1
out_w = (W - WW) / stride + 1
# Perform an im2col operation by picking clever strides
shape = (C, HH, WW, N, out_h, out_w)
strides = (H * W, W, 1, C * H * W, stride * W, stride)
strides = x.itemsize * np.array(strides)
x_stride = np.lib.stride_tricks.as_strided(x_padded,
shape=shape, strides=strides)
x_cols = np.ascontiguousarray(x_stride)
x_cols.shape = (C * HH * WW, N * out_h * out_w)
# Now all our convolutions are a big matrix multiply
res = w.reshape(F, -1).dot(x_cols) + b.reshape(-1, 1)
# Reshape the output
res.shape = (F, N, out_h, out_w)
out = res.transpose(1, 0, 2, 3)
out = np.ascontiguousarray(out)
return out
Note that it uses some non-trivial features of linear algebra library, which are implemented in numpy, but may be not in your library.
BTW, you generally don't want to push the entire data set as one batch - split it into several batches.
How to adapt the Gaussian Kernel to discontinuous spaces,such as that of strings over a finite alphabet,for which we already have a kernel (say K(s, t)) defined ?
The Gaussian Kernel can be stated by :
G(x, y) = e^(−(||x−y||^2)/σ^2)
If you want to build a Gaussian kernel on top of the Hilbert vector space induced by your Kernel K you could put something like
G_K(x, y) = e^(−(K(x, x)-2K(x, y)+K(y,y))/σ^2)
why is it ok?
G(x, y) = e^(−(||x−y||^2)/σ^2) = G(x, y)
= e^(−(<x-y, x-y>)/σ^2)
= e^(−(<x, x>-<x, y>-<y,x>+<y,y>)/σ^2)
= e^(−(<x, x>-2<x, y>+<y,y>)/σ^2)
thus given a kernel K which is a dot product in some space, meaning that K(x,y) = <phi(x), phi(y)> you get
G(phi(x), phi(y) = e^(−(<phi(x), phi(x)>-2<phi(x), phi(y)>+<phi(y),phi(y)>)/σ^2)
= e^(−(K(x, x)-2K(x, y)+K(y,y))/σ^2)
consequently as G is a valid kernel, also G_K(x,y) is (as it is just a scalar product over a transformation through both gaussian projection and the kernel induced one).
The following piece of python code works well for finding gradient descent:
def gradientDescent(x, y, theta, alpha, m, numIterations):
xTrans = x.transpose()
for i in range(0, numIterations):
hypothesis = np.dot(x, theta)
loss = hypothesis - y
cost = np.sum(loss ** 2) / (2 * m)
print("Iteration %d | Cost: %f" % (i, cost))
gradient = np.dot(xTrans, loss) / m
theta = theta - alpha * gradient
return theta
Here, x = m*n (m = no. of sample data and n = total features) feature matrix.
However, if my features are non-numerical (say, director and genre) of '2' movies then my feature matrix may look like:
['Peter Jackson', 'Action'
Sergio Leone', 'Comedy']
In such a case, how can I map these features to numerical values and apply gradient descent ?
You can map your features to numerical value of your choice and then apply gradient descent the usual way.
In python you can use panda to do that easily:
import pandas as pd
df = pd.DataFrame(X, ['director', 'genre'])
df.director = df.director.map({'Peter Jackson': 0, 'Sergio Leone': 1})
df.genre = df.genre.map({'Action': 0, 'Comedy': 1})
As you can see, this way can become pretty complicated and it might be better to write a piece of code doing that dynamically.