Pascal Vincent, Yoshua Bengio
The similarity between objects is a fundamental element of many learn- ing algorithms. Most non-parametric methods take this similarity to be ﬁxed, but much recent work has shown the advantages of learning it, in particular to exploit the local invariances in the data or to capture the possibly non-linear manifold on which most of the data lies. We propose a new non-parametric kernel density estimation method which captures the local structure of an underlying manifold through the leading eigen- vectors of regularized local covariance matrices. Experiments in density estimation show signiﬁcant improvements with respect to Parzen density estimators. The density estimators can also be used within Bayes classi- ﬁers, yielding classiﬁcation rates similar to SVMs and much superior to the Parzen classiﬁer.