Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence

Joseph Paillard, Angel David Reyero Lobo, Vitaliy Kolodyazhniy, Bertrand Thirion, Denis-Alexander Engemann
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:47456-47477, 2025.

Abstract

Causal machine learning (ML) promises to provide powerful tools for estimating individual treatment effects. While causal methods have placed some emphasis on heterogeneity in treatment response, it is of paramount importance to clarify the nature of this heterogeneity, by highlighting which variables drive it. We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method, for statistically rigorous global variable importance assessment in the estimation of the Conditional Average Treatment Effect (CATE). Theoretical analysis of the finite sample regime and empirical studies show that PermuCATE has lower variance than the Leave-One-Covariate-Out (LOCO) method and provides a reliable measure of variable importance. This property increases statistical power, which is crucial for causal inference applications with finite sample sizes. We empirically demonstrate the benefits of PermuCATE in simulated and real datasets, including complex settings with high-dimensional, correlated variables.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-paillard25a, title = {Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence}, author = {Paillard, Joseph and Lobo, Angel David Reyero and Kolodyazhniy, Vitaliy and Thirion, Bertrand and Engemann, Denis-Alexander}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {47456--47477}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/paillard25a/paillard25a.pdf}, url = {https://proceedings.mlr.press/v267/paillard25a.html}, abstract = {Causal machine learning (ML) promises to provide powerful tools for estimating individual treatment effects. While causal methods have placed some emphasis on heterogeneity in treatment response, it is of paramount importance to clarify the nature of this heterogeneity, by highlighting which variables drive it. We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method, for statistically rigorous global variable importance assessment in the estimation of the Conditional Average Treatment Effect (CATE). Theoretical analysis of the finite sample regime and empirical studies show that PermuCATE has lower variance than the Leave-One-Covariate-Out (LOCO) method and provides a reliable measure of variable importance. This property increases statistical power, which is crucial for causal inference applications with finite sample sizes. We empirically demonstrate the benefits of PermuCATE in simulated and real datasets, including complex settings with high-dimensional, correlated variables.} }
Endnote
%0 Conference Paper %T Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence %A Joseph Paillard %A Angel David Reyero Lobo %A Vitaliy Kolodyazhniy %A Bertrand Thirion %A Denis-Alexander Engemann %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-paillard25a %I PMLR %P 47456--47477 %U https://proceedings.mlr.press/v267/paillard25a.html %V 267 %X Causal machine learning (ML) promises to provide powerful tools for estimating individual treatment effects. While causal methods have placed some emphasis on heterogeneity in treatment response, it is of paramount importance to clarify the nature of this heterogeneity, by highlighting which variables drive it. We propose PermuCATE, an algorithm based on the Conditional Permutation Importance (CPI) method, for statistically rigorous global variable importance assessment in the estimation of the Conditional Average Treatment Effect (CATE). Theoretical analysis of the finite sample regime and empirical studies show that PermuCATE has lower variance than the Leave-One-Covariate-Out (LOCO) method and provides a reliable measure of variable importance. This property increases statistical power, which is crucial for causal inference applications with finite sample sizes. We empirically demonstrate the benefits of PermuCATE in simulated and real datasets, including complex settings with high-dimensional, correlated variables.
APA
Paillard, J., Lobo, A.D.R., Kolodyazhniy, V., Thirion, B. & Engemann, D.. (2025). Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:47456-47477 Available from https://proceedings.mlr.press/v267/paillard25a.html.

Related Material