Information Maximization in Noisy Channels : A Variational Approach

David Barber, Felix V. Agakov

Advances in Neural Information Processing Systems 16 (NIPS 2003)

The maximisation of information transmission over noisy channels is a common, albeit generally computationally difficult problem. We approach the difficulty of computing the mutual information for noisy channels by using a variational approximation. The re- sulting IM algorithm is analagous to the EM algorithm, yet max- imises mutual information, as opposed to likelihood. We apply the method to several practical examples, including linear compression, population encoding and CDMA.