MMD-based Variable Importance for Distributional Random Forest

Clément Bénard, Jeffrey Näf, Julie Josse
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1324-1332, 2024.

Abstract

Distributional Random Forest (DRF) is a flexible forest-based method to estimate the full conditional distribution of a multivariate output of interest given input variables. In this article, we introduce a variable importance algorithm for DRFs, based on the well-established drop and relearn principle and MMD distance. While traditional importance measures only detect variables with an influence on the output mean, our algorithm detects variables impacting the output distribution more generally. We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors. In particular, our algorithm is highly efficient to select variables through recursive feature elimination, and can therefore provide small sets of variables to build accurate estimates of conditional output distributions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-benard24a, title = {{MMD}-based Variable Importance for Distributional Random Forest}, author = {B\'{e}nard, Cl\'{e}ment and N\"{a}f, Jeffrey and Josse, Julie}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1324--1332}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/benard24a/benard24a.pdf}, url = {https://proceedings.mlr.press/v238/benard24a.html}, abstract = {Distributional Random Forest (DRF) is a flexible forest-based method to estimate the full conditional distribution of a multivariate output of interest given input variables. In this article, we introduce a variable importance algorithm for DRFs, based on the well-established drop and relearn principle and MMD distance. While traditional importance measures only detect variables with an influence on the output mean, our algorithm detects variables impacting the output distribution more generally. We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors. In particular, our algorithm is highly efficient to select variables through recursive feature elimination, and can therefore provide small sets of variables to build accurate estimates of conditional output distributions.} }
Endnote
%0 Conference Paper %T MMD-based Variable Importance for Distributional Random Forest %A Clément Bénard %A Jeffrey Näf %A Julie Josse %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-benard24a %I PMLR %P 1324--1332 %U https://proceedings.mlr.press/v238/benard24a.html %V 238 %X Distributional Random Forest (DRF) is a flexible forest-based method to estimate the full conditional distribution of a multivariate output of interest given input variables. In this article, we introduce a variable importance algorithm for DRFs, based on the well-established drop and relearn principle and MMD distance. While traditional importance measures only detect variables with an influence on the output mean, our algorithm detects variables impacting the output distribution more generally. We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors. In particular, our algorithm is highly efficient to select variables through recursive feature elimination, and can therefore provide small sets of variables to build accurate estimates of conditional output distributions.
APA
Bénard, C., Näf, J. & Josse, J.. (2024). MMD-based Variable Importance for Distributional Random Forest. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1324-1332 Available from https://proceedings.mlr.press/v238/benard24a.html.

Related Material