Training Multilayer Perceptrons with the Extended Kalman Algorithm

Sharad Singhal, Lance Wu

Advances in Neural Information Processing Systems 1 (NIPS 1988)

trained with

A large fraction of recent work in artificial neural nets uses multilayer perceptrons the back-propagation algorithm described by Rumelhart et. a1. This algorithm converges slowly for large or complex problems such as speech recognition, where thousands of iterations may be needed for convergence even with small data sets. In this paper, we show that training multilayer perceptrons is an identification problem for a nonlinear dynamic system which can be solved using the Extended Kalman Algorithm. Although computationally complex, the Kalman algorithm usually converges in a few the algorithm and compare it with back-propagation using two(cid:173) dimensional examples.

iterations. We describe