Part of Advances in Neural Information Processing Systems 9 (NIPS 1996)
Brendan J. Frey
Real-valued random hidden variables can be useful for modelling latent structure that explains correlations among observed vari(cid:173) ables. I propose a simple unit that adds zero-mean Gaussian noise to its input before passing it through a sigmoidal squashing func(cid:173) tion. Such units can produce a variety of useful behaviors, ranging from deterministic to binary stochastic to continuous stochastic. I show how "slice sampling" can be used for inference and learning in top-down networks of these units and demonstrate learning on two simple problems.