Part of Advances in Neural Information Processing Systems 18 (NIPS 2005)
Masashi Sugiyama
expressed as
Active learning is the problem in supervised learning to design the loca- tions of training input points so that the generalization error is minimized. Existing active learning methods often assume that the model used for learning is correctly specified, i.e., the learning target function can be ex- pressed by the model at hand. In many practical situations, however, this assumption may not be fulfilled. In this paper, we first show that the ex- isting active learning method can be theoretically justified under slightly weaker condition: the model does not have to be correctly specified, but slightly misspecified models are also allowed. However, it turns out that the weakened condition is still restrictive in practice. To cope with this problem, we propose an alternative active learning method which can be theoretically justified for a wider class of misspecified models. Thus, the proposed method has a broader range of applications than the exist- ing method. Numerical studies show that the proposed active learning method is robust against the misspecification of models and is thus reli- able.
Let us discuss the regression problem of learning a real-valued functionfx defined on Rd from training examplesfxi;yijyi=fxi(cid:15)igi=1; wheref(cid:15)igi=1 are i.i.d. noise with mean zero and unknown variance(cid:27)2 bfx=Xi=1(cid:11)iix; wherefixgi=1 are fixed linearly independent functions and(cid:11)=(cid:11)1;(cid:11)2;:::;(cid:11)> We evaluate the goodness of the learned functionbfx by the expected squared test error are drawn independently from a distribution with densityx, the generalization error is G=E(cid:15)Z(cid:16)bfxfx(cid:17)2xdx;
over test input points and noise (i.e., the generalization error). When the test input points
1
Introduction and Problem Formulation
are parameters to be learned.
lowing linear regression model for learning.