Mean Field Methods for Classification with Gaussian Processes

Part of Advances in Neural Information Processing Systems 11 (NIPS 1998)

Bibtex Metadata Paper

Authors

Manfred Opper, Ole Winther

Abstract

We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classifi(cid:173) cation models with Gaussian processes. In contrast to previous ap(cid:173) proaches, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.

1 Modeling with Gaussian Processes

Bayesian models which are based on Gaussian prior distributions on function spaces are promising non-parametric statistical tools. They have been recently introduced into the Neural Computation community (Neal 1996, Williams & Rasmussen 1996, Mackay 1997). To give their basic definition, we assume that the likelihood of the output or target variable T for a given input s E RN can be written in the form p(Tlh(s)) where h : RN --+ R is a priori assumed to be a Gaussian random field. If we assume fields with zero prior mean, the statistics of h is entirely defined by the second order correlations C(s, S') == E[h(s)h(S')], where E denotes expectations

310

M Opper and 0. Winther

with respect to the prior. Interesting examples are