Binary Space Partitioning Forest
[edit]
Proceedings of Machine Learning Research, PMLR 89:30223031, 2019.
Abstract
The Binary Space Partitioning (BSP)Tree process is proposed to produce flexible 2D partition structures which are originally used as a Bayesian nonparametric prior for relational modelling. It can hardly be applied to other learning tasks such as regression trees because extending the BSPTree process to a higher dimensional space is nontrivial. This paper is the first attempt to extend the BSPTree process to a ddimensional ($d>2$) space. We propose to generate a cutting hyperplane, which is assumed to be parallel to $d2$ dimensions, to cut each node in the ddimensional BSPtree. By designing a subtle strategy to sample two free dimensions from d dimensions, the extended BSPTree process can inherit the essential selfconsistency property from the original version. Based on the extended BSPTree process, an ensemble model, which is named the BSPForest, is further developed for regression tasks. Thanks to the retained selfconsistency property, we can thus significantly reduce the geometric calculations in the inference stage. Compared to its counterpart, the Mondrian Forest, the BSPForest can achieve similar performance with fewer cuts due to its flexibility. The BSPForest also outperforms other (Bayesian) regression forests on a number of realworld data sets.
Related Material


