A Bayesian approach to CART

Hugh Chipman, Edward I. George, Robert E. McCulloch
Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, PMLR R1:91-102, 1997.

Abstract

A Bayesian approach for finding classification and regression tree (CART) models is proposed. By putting an appropriate prior distribution on the space of CART models, the resulting posterior will put higher probability on the more "promising trees". In particular, priors are proposed which penalize complexity by putting higher probability on trees with fewer nodes. Metropolis-Hastings algorithms are used to rapidly grow trees in such a way that the high posterior probability trees are more likely to be obtained. In effect, the algorithm performs a stochastic search for promising trees. Examples are used to illustrate the potential superiority of this approach over conventional greedy methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR1-chipman97a, title = {A {B}ayesian approach to {CART}}, author = {Chipman, Hugh and George, Edward I. and McCulloch, Robert E.}, booktitle = {Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics}, pages = {91--102}, year = {1997}, editor = {Madigan, David and Smyth, Padhraic}, volume = {R1}, series = {Proceedings of Machine Learning Research}, month = {04--07 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r1/chipman97a/chipman97a.pdf}, url = {https://proceedings.mlr.press/r1/chipman97a.html}, abstract = {A Bayesian approach for finding classification and regression tree (CART) models is proposed. By putting an appropriate prior distribution on the space of CART models, the resulting posterior will put higher probability on the more "promising trees". In particular, priors are proposed which penalize complexity by putting higher probability on trees with fewer nodes. Metropolis-Hastings algorithms are used to rapidly grow trees in such a way that the high posterior probability trees are more likely to be obtained. In effect, the algorithm performs a stochastic search for promising trees. Examples are used to illustrate the potential superiority of this approach over conventional greedy methods.}, note = {Reissued by PMLR on 30 March 2021.} }
Endnote
%0 Conference Paper %T A Bayesian approach to CART %A Hugh Chipman %A Edward I. George %A Robert E. McCulloch %B Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 1997 %E David Madigan %E Padhraic Smyth %F pmlr-vR1-chipman97a %I PMLR %P 91--102 %U https://proceedings.mlr.press/r1/chipman97a.html %V R1 %X A Bayesian approach for finding classification and regression tree (CART) models is proposed. By putting an appropriate prior distribution on the space of CART models, the resulting posterior will put higher probability on the more "promising trees". In particular, priors are proposed which penalize complexity by putting higher probability on trees with fewer nodes. Metropolis-Hastings algorithms are used to rapidly grow trees in such a way that the high posterior probability trees are more likely to be obtained. In effect, the algorithm performs a stochastic search for promising trees. Examples are used to illustrate the potential superiority of this approach over conventional greedy methods. %Z Reissued by PMLR on 30 March 2021.
APA
Chipman, H., George, E.I. & McCulloch, R.E.. (1997). A Bayesian approach to CART. Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R1:91-102 Available from https://proceedings.mlr.press/r1/chipman97a.html. Reissued by PMLR on 30 March 2021.

Related Material