Where and why is linear and non-linear transformations is useful. What are use cases in Machine learning and in deep learning especially for computer vision.
A neural net can be seen as framework for linear transformation.
Think of the math operations which are defining the linear transformation and compare it with a neural net.
Related
The question seems simple, can we find a neural network for every classical Machine Learning model?
For example:
Linear regression is a perceptron.
PCA is an auto-encoder with a single intermediate layer
We can approximate Ridge or Lasso by adding some decay at the time of the construction of the network
If the answer to the first question is yes then how can I find equivalents to decision trees and SVMs?
I am trying to see applications of grover's algorithm. I have seen it can be applied in the DNA sequence alignment. I was wondering where in machine learning (deep learning, NLP and reinforcement learning) can i use the grover's algorithm.
There is a Grover-augmented Viterbi algorithm with a claimed quadratic runtime speedup. Methods have also been proposed for Quantum Reinforcement Learning. More relevant than the search algorithm itself is the iterative process used to rotate the state vector, which has applications in algorithms in a number of domains (most prominently these days in quantum cryptography).
Is Gradient Descent algorithm ever used during training of any unsupervised training like clustering, collaborative filtering, etc..?
Gradient descent can be used for a whole bunch of unsupervised learning tasks. In fact, neural networks, which use the Gradient Descent algorithm are used widely for unsupervised learning tasks, like representations of text or natural language in vector space (word2vec).
You can also think of dimensionality reduction techniques like autoencoders, which use Gradient Descent as well.
I am not aware of GD being directly used in clustering, but this link discusses an approach that utilizes Autoencoders with Kmeans, which use GD.
Read this link as well, which discusses a similar question.
In unsupervised algorithms, you don't need to do this. For example, in k-Means, where you are trying to minimize the mean squared error (MSE), you can minimize the error directly at each step given the assignments; no gradients needed.
Is this process correct?
Suppose We have a bunch of data such as MNIST.
We just feed all these data(without label) to RBM and resample each data from trained model.
Then output can be treated as new data for classification.
Do I understand it correctly?
What is the purpose of using RBM?
You are correct, RBMs are a form of unsupervised learning algorithm that are commonly used to reduce the dimensionality of your feature space. Another common approach is to use autoencoders.
RBMs are trained using the contrastive divergence algorithm. The best overview of this algorithm comes from Geoffrey Hinton who came up with it.
https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf
A great paper about how unsupervised learning improves performance can be found at http://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf. The paper shows that unsupervised learning provides better generalization and filters (if using CRBMs)
If I were to train an artificial neural network's weights using a genetic algorithm what type of learning would this be classed as? I believe it's unsupervised but does it have a name? It seems like it could fall under reinforcement learning but I'm not too sure.
Genetic algorithm itself is an optimization algorithm rather than a learning algorithm. And you probably don't want to ignore the performance of neural network and only consider the weight changes. So what type of learning is a combination of neural network and genetic algorithm depends on the learning type of neural network. A neural network can be used for supervised learning, reinforcement learning, and even unsupervised learning.
It seems such a combination applies more in reinforcement, because genetic algorithm is slower than most backpropagation-based optimization algorithms with gradient information. Updating the weights with genetic algorithm is also called neuroevolution. This post cited many research on neuroevolution over traditional neural network in case you may be interested in.
It is also possible to apply genetic algorithm to unsupervised neural network. In Shibata et al's Nonlinear backlash compensation using recurrent neural network. Unsupervised learning by genetic algorithm, the authors applied the genetic algorithm to determine the weights of the recurrent neural networks, and the approach does not need the teaching signals.