Part of Advances in Neural Information Processing Systems 6 (NIPS 1993)
Bill Baird, Todd Troyer, Frank Eeckman
We show how an "Elman" network architecture, constructed from recurrently connected oscillatory associative memory network mod(cid:173) ules, can employ selective "attentional" control of synchronization to direct the flow of communication and computation within the architecture to solve a grammatical inference problem. Previously we have shown how the discrete time "Elman" network algorithm can be implemented in a network completely described by continuous ordinary differential equations. The time steps (ma(cid:173) chine cycles) of the system are implemented by rhythmic variation (clocking) of a bifurcation parameter. In this architecture, oscilla(cid:173) tion amplitude codes the information content or activity of a mod(cid:173) ule (unit), whereas phase and frequency are used to "softwire" the network. Only synchronized modules communicate by exchang(cid:173) ing amplitude information; the activity of non-resonating modules contributes incoherent crosstalk noise. Attentional control is modeled as a special subset of the hidden modules with ouputs which affect the resonant frequencies of other hidden modules. They control synchrony among the other mod(cid:173) ules and direct the flow of computation (attention) to effect transi(cid:173) tions between two subgraphs of a thirteen state automaton which the system emulates to generate a Reber grammar. The internal crosstalk noise is used to drive the required random transitions of the automaton.