Combining Neural Network Regression Estimates Using Principal Components

Christopher J. Merz, Michael J. Pazzani
Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, PMLR R1:363-370, 1997.

Abstract

Combining a set of learned models1 to improve classification and regression estimates has been an area ofmuch research in machine learning and neural net- works ([Wolpert92, Merz95 , PerroneCooper92 , LeblancTibshirani93, Breiman92, Meir95, Krogh95, Tresp95, ChanStolfo95]). The challenge of this problem is to decide which models to rely on for prediction and how much weight to give each. The goal of combining learned models is to obtain a more accurate predic- tion than can be obtained from any single source alone. One major issue in combining a set of learned models is redundancy. Redundancy refers to the amount of agreement or linear dependence between models when making a set of predictions  The more the set agrees, the more redundancy is present. In statistical terms, this is referred to as the multicollinearity problem. The focus of this paper is to describe and evaluate an approach for combining regression estimates based on principal components regression. The method, called PCR*, is then evaluated on several real-world domains to demonstrate its robustness versus a collection of existing techniques.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR1-merz97a, title = {Combining Neural Network Regression Estimates Using Principal Components}, author = {Merz, Christopher J. and Pazzani, Michael J.}, booktitle = {Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics}, pages = {363--370}, year = {1997}, editor = {Madigan, David and Smyth, Padhraic}, volume = {R1}, series = {Proceedings of Machine Learning Research}, month = {04--07 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r1/merz97a/merz97a.pdf}, url = {https://proceedings.mlr.press/r1/merz97a.html}, abstract = {Combining a set of learned models1 to improve classification and regression estimates has been an area ofmuch research in machine learning and neural net- works ([Wolpert92, Merz95 , PerroneCooper92 , LeblancTibshirani93, Breiman92, Meir95, Krogh95, Tresp95, ChanStolfo95]). The challenge of this problem is to decide which models to rely on for prediction and how much weight to give each. The goal of combining learned models is to obtain a more accurate predic- tion than can be obtained from any single source alone. One major issue in combining a set of learned models is redundancy. Redundancy refers to the amount of agreement or linear dependence between models when making a set of predictions  The more the set agrees, the more redundancy is present. In statistical terms, this is referred to as the multicollinearity problem. The focus of this paper is to describe and evaluate an approach for combining regression estimates based on principal components regression. The method, called PCR*, is then evaluated on several real-world domains to demonstrate its robustness versus a collection of existing techniques.}, note = {Reissued by PMLR on 30 March 2021.} }
Endnote
%0 Conference Paper %T Combining Neural Network Regression Estimates Using Principal Components %A Christopher J. Merz %A Michael J. Pazzani %B Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 1997 %E David Madigan %E Padhraic Smyth %F pmlr-vR1-merz97a %I PMLR %P 363--370 %U https://proceedings.mlr.press/r1/merz97a.html %V R1 %X Combining a set of learned models1 to improve classification and regression estimates has been an area ofmuch research in machine learning and neural net- works ([Wolpert92, Merz95 , PerroneCooper92 , LeblancTibshirani93, Breiman92, Meir95, Krogh95, Tresp95, ChanStolfo95]). The challenge of this problem is to decide which models to rely on for prediction and how much weight to give each. The goal of combining learned models is to obtain a more accurate predic- tion than can be obtained from any single source alone. One major issue in combining a set of learned models is redundancy. Redundancy refers to the amount of agreement or linear dependence between models when making a set of predictions  The more the set agrees, the more redundancy is present. In statistical terms, this is referred to as the multicollinearity problem. The focus of this paper is to describe and evaluate an approach for combining regression estimates based on principal components regression. The method, called PCR*, is then evaluated on several real-world domains to demonstrate its robustness versus a collection of existing techniques. %Z Reissued by PMLR on 30 March 2021.
APA
Merz, C.J. & Pazzani, M.J.. (1997). Combining Neural Network Regression Estimates Using Principal Components. Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R1:363-370 Available from https://proceedings.mlr.press/r1/merz97a.html. Reissued by PMLR on 30 March 2021.

Related Material