Evaluation of Causal Structure Learning Algorithms via Risk Estimation

Marco Eigenmann, Sach Mukherjee, Marloes Maathuis
Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR 124:151-160, 2020.

Abstract

Recent years have seen many advances in methods for causal structure learning from data. The empirical assessment of such methods, however, is much less developed. Motivated by this gap, we pose the following question: how can one assess, in a given problem setting, the practical efficacy of one or more causal structure learning methods? We formalize the problem in a decision-theoretic framework, via a notion of expected loss or risk for the causal setting. We introduce a theoretical notion of causal risk as well as sample quantities that can be computed from data, and study the relationship between the two, both theoretically and through an extensive simulation study. Our results provide an assumptions-light framework for assessing causal structure learning methods that can be applied in a range of practical use-cases.

Cite this Paper

BibTeX
@InProceedings{pmlr-v124-eigenmann20a, title = {Evaluation of Causal Structure Learning Algorithms via Risk Estimation}, author = {Eigenmann, Marco and Mukherjee, Sach and Maathuis, Marloes}, booktitle = {Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)}, pages = {151--160}, year = {2020}, editor = {Peters, Jonas and Sontag, David}, volume = {124}, series = {Proceedings of Machine Learning Research}, month = {03--06 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v124/eigenmann20a/eigenmann20a.pdf}, url = {https://proceedings.mlr.press/v124/eigenmann20a.html}, abstract = {Recent years have seen many advances in methods for causal structure learning from data. The empirical assessment of such methods, however, is much less developed. Motivated by this gap, we pose the following question: how can one assess, in a given problem setting, the practical efficacy of one or more causal structure learning methods? We formalize the problem in a decision-theoretic framework, via a notion of expected loss or risk for the causal setting. We introduce a theoretical notion of causal risk as well as sample quantities that can be computed from data, and study the relationship between the two, both theoretically and through an extensive simulation study. Our results provide an assumptions-light framework for assessing causal structure learning methods that can be applied in a range of practical use-cases.} }
Endnote
%0 Conference Paper %T Evaluation of Causal Structure Learning Algorithms via Risk Estimation %A Marco Eigenmann %A Sach Mukherjee %A Marloes Maathuis %B Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) %C Proceedings of Machine Learning Research %D 2020 %E Jonas Peters %E David Sontag %F pmlr-v124-eigenmann20a %I PMLR %P 151--160 %U https://proceedings.mlr.press/v124/eigenmann20a.html %V 124 %X Recent years have seen many advances in methods for causal structure learning from data. The empirical assessment of such methods, however, is much less developed. Motivated by this gap, we pose the following question: how can one assess, in a given problem setting, the practical efficacy of one or more causal structure learning methods? We formalize the problem in a decision-theoretic framework, via a notion of expected loss or risk for the causal setting. We introduce a theoretical notion of causal risk as well as sample quantities that can be computed from data, and study the relationship between the two, both theoretically and through an extensive simulation study. Our results provide an assumptions-light framework for assessing causal structure learning methods that can be applied in a range of practical use-cases.
APA
Eigenmann, M., Mukherjee, S. & Maathuis, M.. (2020). Evaluation of Causal Structure Learning Algorithms via Risk Estimation. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), in Proceedings of Machine Learning Research 124:151-160 Available from https://proceedings.mlr.press/v124/eigenmann20a.html.

Related Material