Regularized Greedy Importance Sampling

Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)

Bibtex Metadata Paper


Finnegan Southey, Dale Schuurmans, Ali Ghodsi


Greedy importance sampling is an unbiased estimation technique that re- duces the variance of standard importance sampling by explicitly search- ing for modes in the estimation objective. Previous work has demon- strated the feasibility of implementing this method and proved that the technique is unbiased in both discrete and continuous domains. In this paper we present a reformulation of greedy importance sampling that eliminates the free parameters from the original estimator, and introduces a new regularization strategy that further reduces variance without com- promising unbiasedness. The resulting estimator is shown to be effective for difficult estimation problems arising in Markov random field infer- ence. In particular, improvements are achieved over standard MCMC estimators when the distribution has multiple peaked modes.