recently I started learning pattern recognition and I'm very interested in Perceptron algorithm. But sometimes I heard other students talking about Perceptron network, that is a type of neutral networks. What is the relation ship between them?
I think you should review your terminology first.
It's MULTILAYER, PERCEPTRON, NEURAL network.
That aside, wikipedia is always your friend: MLP
At the highest level, you can think of a MLP as a network of perceptron units (although I'm probably oversimplifying it).
Related
As a beginner in ML and AI, I have come across ANN, RNN and LSTMs, however I would like to know what is the classification among neural networks ranging from the simplest single perceptron feedforward network to the most complicated one.
Maybe you'll be interested in looking at the neural network zoo. It's not a hierarchical classification of all the different neural network types, but it does graphically show a lot of types and also provides short descriptions of them (and also some other models that are not typically considered to be neural networks). I haven't read through it all in detail, so I can't personally vouch for the page's correctness, but it looks good.
In Wikipedia I found that machine learning is a subsection of neural networks science. So, does it mean that work with machine learning is itself implies working with neural networks or not?
What will be better to use for pattern recognition tasks in terms of efficiency and complexity?
Machine learning is a part of neural network? I'd be surprised because machine learning includes dozen of techniques that have nothing to do with neural network. It's most likely the other way around.
The exact pattern recognition algorithm depends on your requirement and data set. There're many such algorithms, for example, SVM, linear models for classification, HMM, PCA etc. Note that the phase "pattern recognition" is a very general term, there is no algorithm that always work. It all depends on what patterns you are looking and what kind of assumption you can make.
I recommend Dr Bishop's "Pattern Recognition and Machine Learning" book, you'll learn a lot from the book.
I've often read, that there are fundamental differences between feed-forward and recurrent neural networks (RNNs), due to the lack of an internal state and a therefore short-term memory in feed-forward networks. This seemed plausible to me at first sight.
However when learning a recurrent neural network with the Backpropagation through time algorithm recurrent networks are transformed into equivalent feed forward networks, if I understand correctly.
This would imply, that there is in fact no fundamental difference. Why do RNNs perform better in certain tasks (image recognition, time-series prediction, ...) than deep feed forward networks?
The fact that training is done using some trick, does not change the fact, that there is a fundamental difference in the preservation of the network state, which is absent in the feed-forward network.
The "unrolled" feed forward network is not equivalent to the recurrent network. It is only a markov approximation (to the level given by the number of "unrolled" levels). So you just "simulate" the recurrent network with k step memory, while the actual recurrent neural network has (in theory) unlimited memory.
If I were to train an artificial neural network's weights using a genetic algorithm what type of learning would this be classed as? I believe it's unsupervised but does it have a name? It seems like it could fall under reinforcement learning but I'm not too sure.
Genetic algorithm itself is an optimization algorithm rather than a learning algorithm. And you probably don't want to ignore the performance of neural network and only consider the weight changes. So what type of learning is a combination of neural network and genetic algorithm depends on the learning type of neural network. A neural network can be used for supervised learning, reinforcement learning, and even unsupervised learning.
It seems such a combination applies more in reinforcement, because genetic algorithm is slower than most backpropagation-based optimization algorithms with gradient information. Updating the weights with genetic algorithm is also called neuroevolution. This post cited many research on neuroevolution over traditional neural network in case you may be interested in.
It is also possible to apply genetic algorithm to unsupervised neural network. In Shibata et al's Nonlinear backlash compensation using recurrent neural network. Unsupervised learning by genetic algorithm, the authors applied the genetic algorithm to determine the weights of the recurrent neural networks, and the approach does not need the teaching signals.
I am wondering how to go about training a neural network without providing it with training values. My premise for this is that the neural network(s) will be used on a robot that can receive positive/negative feedback from sensors. IE, in order to train it to roam freely without bumping into things, a positive feedback occurs when no collision sensors or proximity sensors are triggered. A negative feedback occurs when the collision/proximity sensors ARE triggered. How can the neural network be trained using this method?
I am writing this in C++
What you describe is called reinforcement learning. It could be applied to neural networks, but does not require them in general. The canonical textbook to read on the subject is Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto. The connection between neural networks and reinforcement learning is explored in a bit more detail in the PDP Handbook by James McClelland.
Have you taken a look at SLAM? It's a technique robots can use to navigate an area while simultaneously building up and keeping a map of that area.