Bayesian and Empirical Bayesian Forests

Taddy Matthew, Chun-Sheng Chen, Jun Yu, Mitch Wyle
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:967-976, 2015.

Abstract

We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view such ensembles as samples from a posterior distribution. This insight motivates a class of Bayesian Forest (BF) algorithms that provide small gains in performance and large gains in interpretability. Based on the BF framework, we are able to show that high-level tree hierarchy is stable in large samples. This motivates an empirical Bayesian Forest (EBF) algorithm for building approximate BFs on massive distributed datasets and we show that EBFs outperform sub-sampling based alternatives by a large margin.

Cite this Paper

BibTeX
@InProceedings{pmlr-v37-matthew15, title = {Bayesian and Empirical Bayesian Forests}, author = {Matthew, Taddy and Chen, Chun-Sheng and Yu, Jun and Wyle, Mitch}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {967--976}, year = {2015}, editor = {Bach, Francis and Blei, David}, volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/matthew15.pdf}, url = {https://proceedings.mlr.press/v37/matthew15.html}, abstract = {We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view such ensembles as samples from a posterior distribution. This insight motivates a class of Bayesian Forest (BF) algorithms that provide small gains in performance and large gains in interpretability. Based on the BF framework, we are able to show that high-level tree hierarchy is stable in large samples. This motivates an empirical Bayesian Forest (EBF) algorithm for building approximate BFs on massive distributed datasets and we show that EBFs outperform sub-sampling based alternatives by a large margin.} }
Endnote
%0 Conference Paper %T Bayesian and Empirical Bayesian Forests %A Taddy Matthew %A Chun-Sheng Chen %A Jun Yu %A Mitch Wyle %B Proceedings of the 32nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2015 %E Francis Bach %E David Blei %F pmlr-v37-matthew15 %I PMLR %P 967--976 %U https://proceedings.mlr.press/v37/matthew15.html %V 37 %X We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view such ensembles as samples from a posterior distribution. This insight motivates a class of Bayesian Forest (BF) algorithms that provide small gains in performance and large gains in interpretability. Based on the BF framework, we are able to show that high-level tree hierarchy is stable in large samples. This motivates an empirical Bayesian Forest (EBF) algorithm for building approximate BFs on massive distributed datasets and we show that EBFs outperform sub-sampling based alternatives by a large margin.
RIS
TY - CPAPER TI - Bayesian and Empirical Bayesian Forests AU - Taddy Matthew AU - Chun-Sheng Chen AU - Jun Yu AU - Mitch Wyle BT - Proceedings of the 32nd International Conference on Machine Learning DA - 2015/06/01 ED - Francis Bach ED - David Blei ID - pmlr-v37-matthew15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 37 SP - 967 EP - 976 L1 - http://proceedings.mlr.press/v37/matthew15.pdf UR - https://proceedings.mlr.press/v37/matthew15.html AB - We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view such ensembles as samples from a posterior distribution. This insight motivates a class of Bayesian Forest (BF) algorithms that provide small gains in performance and large gains in interpretability. Based on the BF framework, we are able to show that high-level tree hierarchy is stable in large samples. This motivates an empirical Bayesian Forest (EBF) algorithm for building approximate BFs on massive distributed datasets and we show that EBFs outperform sub-sampling based alternatives by a large margin. ER -
APA
Matthew, T., Chen, C., Yu, J. & Wyle, M.. (2015). Bayesian and Empirical Bayesian Forests. Proceedings of the 32nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 37:967-976 Available from https://proceedings.mlr.press/v37/matthew15.html.

Related Material