Download to read offline
Artificial Neural Network Learning Algorithm.ppt
- 1. Name Ammar Muhammad SubjectIntelligent Control Application Topic Neural Network & Learning Algorithm Professor Chan Zhou
- 2. What is theArtificial Neural Network? • An Artificial Neural Network (ANN) is a computational model inspired by the structure and function of the human brain's neural networks. • It resembles the brain in two respects. It comprises interconnected nodes (neurons) that process information, learn patterns from data. Make predictions or classifications by adjusting connections (synaptic weights ) between nodes. • ANN are considered nonlinear statistical data modeling tools where complex relationships between inputs and outputs are modeled or patterns are found.
- 3. Biological Inspiration • TheBrain is a massively parallel information processing system. • Our brains are a huge network of processing elements. • A typical brain contains a network of 10 billion neurons.
- 4. CNS- Brain andNeuron • The brain is highly complex nonlinear and massively parallel computing machine.
- 5. Motivation behind Artificial NeuralNetwork A neuron is a basic unit of the brain processes and transmitting the information. • Dendrite: Receive signal from other neurons. • Soma(Cell body): sums all the inputs. • Axon: it is used to transmit the electric signal to other neurons. • Synaptic Terminal: Release the neurotransmitter to transmit information to dendrites.
- 6.
- 7. Association with theBiological neurons and Artificial neurons. Inputs Output Liner / nonlinear model BNN ANN Soma Node Dendrites Inputs Synapse Weight or interconnection /Link Axon Output
- 8. Perceptron • A perceptronis a most fundamental unit of neural network (ANN) that does certain computations to detect feature or business intelligence in the input data. • The perceptron is linear model for the supervised learning used for the binary classification, • A perceptron is a most fundamental unit of neural network (ANN) that does certain computations to detect feature or business intelligence in the input data. • The perceptron is linear model for the supervised learning used for the binary classification, • Perceptron consist of 4 parts • Inputs • weights & bias • summation function Activation function • Output • Perceptron learning rule. • Perceptron learns the weight for the input single sin order to draw a linear decision boundary. Types of perceptron • Single layer perceptron • Multi layer perceptron
- 9. Activation Function The functionAN receives the net input signal and bias and determines the output of the neuron. This function is referred as the activation function.
- 10. Network Architecture Network Architecture SingleLayer ANN Architecture Feedforward ANN Architectures Unsupervised (Kohonen) Supervised (MLP, RBF) Recurrent ANN Architectures Unsupervised (ART) Supervised (Elman, Jordan, Hopfield) • The way the neurons of a neural networks are structured is intimately linked with the learning algorithm used to train the network. • Generally, three fundamentally different classes of the network architecture.
- 11. Single Layer ANNArchitecture • It is one of the oldest and first introduced neural networks. • It was proposed by Frank Rosenblatt in 1958. Perceptron is also known as an artificial neural network. • Perceptron is mainly used to compute the logical gate like AND, OR, and XOR which has binary input and binary output.
- 12. Feedforward ANN Architectures •It is also known as the Multi- Layered Neural Network. • A multi-layer perceptron has one input layer and for each input, there is one neuron(or node), it has one output layer with a single node for each output and it can have any number of hidden layers and each hidden layer can have any number of nodes. • Capable of handling the non-linearly separable data.
- 13. Recurrent ANN Architectures •Recurrent Neural Network(RNN) is a type of Neural Network where the output from the previous step is fed as input to the current step. • In traditional neural networks, all the inputs and outputs are independent of each other. • Two types of recurrent network • Without hidden layers • With Hidden layers
- 14. ANN Learning Techniques Depending on types and characteristics of the encoding and recall process, ANN learning techniques are categorized as Supervised learning Un supervised learning Reinforcement learning Competitive learning
- 15. Supervised Learning • Inthe supervise learning system there is a trainer(teacher) who provides the input and the corresponding target (output ) patterns. • A learning algorithm is employed to determine a unique set of the network parameters that jointly satisfy the input–output interrelationship of patterns(encoding) • Then excitation with an unknown input pattern can generate its corresponding output pattern. • ANN trained by supervised learning algorithm behaves like a multi-input-multi-output function approximator. • There are many supervised learning using ANN. • Most popular is back propagation algorithm.
- 16. Unsupervised Learning • Anunsupervised learning system employs no teacher. • Interrelation among patterns is not known. • One or more input patterns is automatically mapped to one pattern cluster. • Most systems employ recursive leaning rule that automatically adjusts the network parameters for attaining some criteria like minimization f the network energy states. • Imagine you have a folder filled with various pictures from a trip, but these images aren't categorized or labeled. You want to organize them based on similarities without manually labeling each photo. • Hopfield nets and associative memory are most popular unsupervised learning system.
- 17. Reinforcement Learning • ReinforcementLearning bridges the gap between the supervised and unsupervised learning. • Learning scheme employs internal critic that examines the response of the environment in turn of the action of the learning system on the environment. • If the response is in favor of the goal, then action is rewarded otherwise it is penalized. • Determination of the status of the action: reward or penalty may require quite a long time. • Imagine you're teaching a dog a new trick using reinforcement learning concepts: like fetching a ball. • Q-learning is the most common reinforcement learning technique.
- 18. Competitive learning • InCompetitive learning scheme neurons compete with one another to satisfy the given goal. • The output neurons compete among themselves to become active. • Only as single output neuron is active at any one time. • The neuron that wins the competition is called winner takes all neurons. • Imagine you have a basket of fruits (apples, oranges, and bananas) that vary in color, and you want to sort them into different bins based on their colors using a competitive learning approach: • You have three bins labeled "Red," "Orange," and "Yellow," representing the colors of fruits (like apples, oranges, and bananas).
- 19. Hebb’s Postulate ofLearning • Donald Hebb the origination of the behavior (1949) • Hebb’s Postulate “ When an axon of cell A is near enough to excite a cell B and repeatedly or presently takes part in firing it, some growth process or metabolic change takes place in one or both cell such that A’s efficiency as one pf the cell firing B is increased. • Mathematically, • Where xi is the input and y is the output. • Bipolar input or output (-1 or +1) • Limitation –can classify linearly separable patterns only.
- 21. Advantages / Disadvantages Advantages Adapt to unknown situations Powerful, it can model complex functions. Ease of use, learns by example, and very little user domain‐specific expertise needed Disadvantages Forgets Not exact Large complexity of the network structure
- 22. Conclusion Artificial NeuralNetworks are an imitation of the biological neural networks, but much simpler ones. The computing would have a lot to gain from neural networks. Their ability to learn by example makes them very flexible and powerful furthermore there is need to device an algorithm to perform a specific task.