I guess that PCA cannot be applied on vectors, however I found some papers that apply PCA on each of the wavelet subbands as in this paper and this. As wavelet subbands are vectors, how is it possible to apply PCA on them?
Thank you in advance
The papers you mention are about EEG and ECG signals, which are also (1D) vectors. Multiple signals together for one subject (or a group) are a matrix. That's what how the PCA runs on input EEG signals.
You can do the same with a wavelet transform. A wavelet subband of a 1D signal is still a 1D signal, but you can group them together in matrices. Then you can run PCA in the same way as on the input data.
Related
I have some 2d data which looks like it could be well classified by an area described by two intersecting straight lines. These lines won't necessarily by at right angles to each other. Here is a simple example where the two lines would be more or less at right angles:
Is there a suitable classifier for this? Logistic regression will give me one straight line but I am not sure what will give me two as a decision boundary. A decision tree will give me two that are axis parallel which isn't really want I want.
You can give Support Vector Machine (SVM) a try. There are multiple kernels that can be used with SVM, like
Linear
Polynomial
RBF (Radial Basis Function)
Sigmoid
You can even specify custom kernels as mentioned in this list.
Here is an image of decision boundaries over Iris dataset, taken from this example
References
Difference between various SVM kernels
Selecting Kernels for SVM
Custom kernel SVM
I would like the mathematical proof of it. does anyone know a paper for it. or can workout the math?
https://pvirie.wordpress.com/2016/03/29/linear-autoencoders-do-pca/
PCA is restricted to a linear map, while auto encoders can have nonlinear enoder/decoders.
A single layer auto encoder with linear transfer function is nearly equivalent to PCA, where nearly means that the WW found by AE and PCA won't be the same--but the subspace spanned by the respective WW's will.
Here is a detailed mathematical explanation:
https://arxiv.org/abs/1804.10253
This paper also shows that using a linear autoencoder, it is possible not only to compute the subspace spanned by the PCA vectors, but it is actually possible to compute the principal components themselves.
i'm wondering why in a Wavelet Neural Network, there is no Inverse Transform that recompose the signal?
How come only the wavelet coefficients are enough to find the wanted signal?
Wouldn't it be better to have the IWT?
Jeff
The use of wavelets in neural network is just for features extractions.
And from wavelets decompositions you can restore the original signal.
Another point about your question is.
Are you using the WNN for regression or classification?
In case of classification there's no need for IWT.
In case of regression i don't know how to fit WNN.
sugestion:
Wavelet Neural Networks
It seems to me that many advanced graph analysis algorithm are based on spectral graph analysis, relying more specifically on the Laplacian matrix properties.
I know there are some alternative for clustering that are based on random-walk type algorithms, that make no use of the Laplacian matrix factorization.
I am curious if there exists anything to go a bit further and determine the Laplacian matrix eigenvalues (especially the second one), without using spectral method, but more like wandering on the graph.
In OpenCV, there is a CvSVM class which takes in a matrix of samples to train the SVM. The matrix is 2D, with the samples in the rows.
I created my own method to generate a histogram of oriented gradients (HOG) off of a video feed. To do this, I created a 9 channeled matrix to store the HOG, where each channel corresponds to an orientation bin. So in the end I have a 40x30 matrix of type CV_32FC(9).
Also made a visualisation for the HOG and it's working.
I don't see how I'm supposed to feed this matrix into the OpenCV SVM, because if I flatten it, I don't see how the SVM is supposed to learn a 9D hyperplane from 1D input data.
The SVM always takes in a single row of data per feature vector. The dimensionality of the feature vector is thus the length of the row. If you're dealing with 2D data, then there are 2 items per feature vector. Example of 2D data is on this webpage:
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
code of an equivalent demo in OpenCV http://sites.google.com/site/btabibian/labbook/svmusingopencv
The point is that even though you're thinking of the histogram as 2D with 9-bin cells, the feature vector is in fact the flattened version of this. So it's correct to flatten it out into a long feature vector. The result for me was a feature vector of length 2304 (16x16x9) and I get 100% prediction accuracy on a small test set (i.e. it's probably slightly less than 100% but it's working exceptionally well).
The reason this works is that the SVM is working on a system of weights per item of the feature vector. So it doesn't have anything to do with the problem's dimension, the hyperplane is always in the same dimension as the feature vector. Another way of looking at it is to forget about the hyperplane and just view it as a bunch of weights for each item in the feature vector. In this case, it needs one weighting for every item, then it multiplies each item by its weighting and outputs the result.