If I were to train an artificial neural network's weights using a genetic algorithm what type of learning would this be classed as? I believe it's unsupervised but does it have a name? It seems like it could fall under reinforcement learning but I'm not too sure.
Genetic algorithm itself is an optimization algorithm rather than a learning algorithm. And you probably don't want to ignore the performance of neural network and only consider the weight changes. So what type of learning is a combination of neural network and genetic algorithm depends on the learning type of neural network. A neural network can be used for supervised learning, reinforcement learning, and even unsupervised learning.
It seems such a combination applies more in reinforcement, because genetic algorithm is slower than most backpropagation-based optimization algorithms with gradient information. Updating the weights with genetic algorithm is also called neuroevolution. This post cited many research on neuroevolution over traditional neural network in case you may be interested in.
It is also possible to apply genetic algorithm to unsupervised neural network. In Shibata et al's Nonlinear backlash compensation using recurrent neural network. Unsupervised learning by genetic algorithm, the authors applied the genetic algorithm to determine the weights of the recurrent neural networks, and the approach does not need the teaching signals.
Related
The question seems simple, can we find a neural network for every classical Machine Learning model?
For example:
Linear regression is a perceptron.
PCA is an auto-encoder with a single intermediate layer
We can approximate Ridge or Lasso by adding some decay at the time of the construction of the network
If the answer to the first question is yes then how can I find equivalents to decision trees and SVMs?
Where and why is linear and non-linear transformations is useful. What are use cases in Machine learning and in deep learning especially for computer vision.
A neural net can be seen as framework for linear transformation.
Think of the math operations which are defining the linear transformation and compare it with a neural net.
In Wikipedia I found that machine learning is a subsection of neural networks science. So, does it mean that work with machine learning is itself implies working with neural networks or not?
What will be better to use for pattern recognition tasks in terms of efficiency and complexity?
Machine learning is a part of neural network? I'd be surprised because machine learning includes dozen of techniques that have nothing to do with neural network. It's most likely the other way around.
The exact pattern recognition algorithm depends on your requirement and data set. There're many such algorithms, for example, SVM, linear models for classification, HMM, PCA etc. Note that the phase "pattern recognition" is a very general term, there is no algorithm that always work. It all depends on what patterns you are looking and what kind of assumption you can make.
I recommend Dr Bishop's "Pattern Recognition and Machine Learning" book, you'll learn a lot from the book.
Is this process correct?
Suppose We have a bunch of data such as MNIST.
We just feed all these data(without label) to RBM and resample each data from trained model.
Then output can be treated as new data for classification.
Do I understand it correctly?
What is the purpose of using RBM?
You are correct, RBMs are a form of unsupervised learning algorithm that are commonly used to reduce the dimensionality of your feature space. Another common approach is to use autoencoders.
RBMs are trained using the contrastive divergence algorithm. The best overview of this algorithm comes from Geoffrey Hinton who came up with it.
https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf
A great paper about how unsupervised learning improves performance can be found at http://jmlr.org/papers/volume11/erhan10a/erhan10a.pdf. The paper shows that unsupervised learning provides better generalization and filters (if using CRBMs)
recently I started learning pattern recognition and I'm very interested in Perceptron algorithm. But sometimes I heard other students talking about Perceptron network, that is a type of neutral networks. What is the relation ship between them?
I think you should review your terminology first.
It's MULTILAYER, PERCEPTRON, NEURAL network.
That aside, wikipedia is always your friend: MLP
At the highest level, you can think of a MLP as a network of perceptron units (although I'm probably oversimplifying it).