Performance Surfaces and Optimum Points

Preliminaries Perceptron learning algorithm Hebbian learning algorithm Linear algebra Neural Network Training Technique1 Several architectures of the neural networks had been introduced. And each neural network had its own learning rule, like, the perceptron learning algorithm, and the Hebbian learning algorithm. When more and more neural network architectures were designed, some general training methods were necessary. Up to now, we can classify all training rules in three categories in a general way:...

December 19, 2019 · (Last Modification: May 1, 2022) · Anthony Tan

Supervised Hebbian Learning

Preliminaries Linear algebra Hebb Rule1 Hebb rule is one of the earliest neural network learning laws. It was published in 1949 by Donald O. Hebb, a Canadian psychologist, in his work ’ The Organization of Behavior’. In this great book, he proposed a possible mechanism for synaptic modification in the brain. And this rule then was used in training the artificial neural networks for pattern recognition. ’ The Organization of Behavior’ The main premise of the book is that behavior could be explained by the action of a neuron....

December 17, 2019 · (Last Modification: May 1, 2022) · Anthony Tan

Implement of Perceptron

Preliminaries An Introduction to Neural Networks Neuron Model and Network Architecture Perceptron Learning Rule Implement of Perceptron1 What we need to do next is to implement the algorithm described in ‘Perceptron Learning Rule’ and observe the effect of 1. different parameters, 2. different training sets, 3. and different transfer functions. A single neuron perceptron consists of a linear combination and a threshold operation simply. So we note its capacity is close to a linear classification....

December 12, 2019 · (Last Modification: April 30, 2022) · Anthony Tan

Learning Rules and Perceptron Learning Rule

Preliminaries supervised learning unsupervised learning reinforcement learning ‘An Introduction to Neural Networks’ Learning Rules1 We have built some neural network models in the post ‘An Introduction to Neural Networks’ and as we know architectures and learning rules are two main aspects of designing a useful network. The architectures we have introduced could not be used yet. What we are going to do is to investigate the learning rules for different architectures....

December 11, 2019 · (Last Modification: May 3, 2022) · Anthony Tan

Neuron Model and Network Architecture

Preliminaries linear classifier An Introduction to Neural Networks Theory and Notation1 We are not able to build any artificial cells up to now. It seems impossible to build a neuron network through biological materials manually, either. To investigate the ability of neurons we have built mathematical models of the neuron. These models have been assigned a number of neuron-like properties. However, there must be a balance between the number of properties contained by the mathematical models and the current computational abilities of the machines....

December 10, 2019 · (Last Modification: April 29, 2022) · Anthony Tan

An Introduction to Neural Networks

Preliminaries Nothing Neural Networks1 Neural Networks are a model of our brain that is built with neurons and is considered the source of intelligence. There are almost \(10^{11}\) neurons in the human brain and \(10^4\) connections of each neuron to other neurons. Some of these brilliant structures were given when we were born. Some other structures could be established by experience, and this progress is called learning. Learning is also considered as the establishment or modification of the connections between neurons....

December 8, 2019 · (Last Modification: April 30, 2022) · Anthony Tan