Adversarial Random Forests for Density Estimation and Generative Modeling

David S. Watson, Kristin Blesch, Jan Kapar, Marvin N. Wright
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:5357-5375, 2023.

Abstract

We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying $R$ package, $arf$, is available on $CRAN$.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-watson23a, title = {Adversarial Random Forests for Density Estimation and Generative Modeling}, author = {Watson, David S. and Blesch, Kristin and Kapar, Jan and Wright, Marvin N.}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {5357--5375}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/watson23a/watson23a.pdf}, url = {https://proceedings.mlr.press/v206/watson23a.html}, abstract = {We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying $R$ package, $arf$, is available on $CRAN$.} }
Endnote
%0 Conference Paper %T Adversarial Random Forests for Density Estimation and Generative Modeling %A David S. Watson %A Kristin Blesch %A Jan Kapar %A Marvin N. Wright %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-watson23a %I PMLR %P 5357--5375 %U https://proceedings.mlr.press/v206/watson23a.html %V 206 %X We propose methods for density estimation and data synthesis using a novel form of unsupervised random forests. Inspired by generative adversarial networks, we implement a recursive procedure in which trees gradually learn structural properties of the data through alternating rounds of generation and discrimination. The method is provably consistent under minimal assumptions. Unlike classic tree-based alternatives, our approach provides smooth (un)conditional densities and allows for fully synthetic data generation. We achieve comparable or superior performance to state-of-the-art probabilistic circuits and deep learning models on various tabular data benchmarks while executing about two orders of magnitude faster on average. An accompanying $R$ package, $arf$, is available on $CRAN$.
APA
Watson, D.S., Blesch, K., Kapar, J. & Wright, M.N.. (2023). Adversarial Random Forests for Density Estimation and Generative Modeling. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:5357-5375 Available from https://proceedings.mlr.press/v206/watson23a.html.

Related Material