NIPS Proceedingsβ

On Sparsity and Overcompleteness in Image Models

Part of: Advances in Neural Information Processing Systems 20 (NIPS 2007)

[PDF] [BibTeX]



Computational models of visual cortex, and in particular those based on sparse coding, have enjoyed much recent attention. Despite this currency, the question of how sparse or how over-complete a sparse representation should be, has gone without principled answer. Here, we use Bayesian model-selection methods to address these questions for a sparse-coding model based on a Student-t prior. Having validated our methods on toy data, we find that natural images are indeed best modelled by extremely sparse distributions; although for the Student-t prior, the associated optimal basis size is only modestly overcomplete.