Part of Advances in Neural Information Processing Systems 6 (NIPS 1993)
Christoph Bregler, Stephen Omohundro
Most connectionist research has focused on learning mappings from one space to another (eg. classification and regression). This paper introduces the more general task of learning constraint surfaces. It describes a simple but powerful architecture for learning and manipulating nonlinear surfaces from data. We demonstrate the technique on low dimensional synthetic surfaces and compare it to nearest neighbor approaches. We then show its utility in learning the space of lip images in a system for improving speech recognition by lip reading. This learned surface is used to improve the visual tracking performance during recognition.
1 Surface Learning
Mappings are an appropriate representation for systems whose variables naturally decompose into "inputs" and "outputs)). To use a learned mapping, the input vari(cid:173) ables must be known and error-free and a single output value must be estimated for each input. Many tasks in vision, robotics, and control must maintain relationships between variables which don't naturally decompose in this way. Instead, there is a nonlinear constraint surface on which the values of the variables are jointly re(cid:173) stricted to lie. We propose a representation for such surfaces which supports a wide range of queries and which can be naturally learned from data.
The simplest queries are "completion queries)). In these queries, the values of certain variables are specified and the values (or constraints on the values) of remaining