Part of Advances in Neural Information Processing Systems 9 (NIPS 1996)
Christopher Merz, Michael Pazzani
When combining a set of learned models to form an improved es(cid:173) timator, the issue of redundancy or multicollinearity in the set of models must be addressed. A progression of existing approaches and their limitations with respect to the redundancy is discussed. A new approach, PCR , based on principal components regres(cid:173) sion is proposed to address these limitations. An evaluation of the new approach on a collection of domains reveals that: 1) PCR was the most robust combination method as the redundancy of the learned models increased, 2) redundancy could be handled without eliminating any of the learned models, and 3) the principal compo(cid:173) nents of the learned models provided a continuum of "regularized" weights from which PCR * could choose.