Validating Causal Inference Models via Influence Functions

Ahmed Alaa, Mihaela Van Der Schaar
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:191-201, 2019.

Abstract

The problem of estimating causal effects of treatments from observational data falls beyond the realm of supervised learning {—} because counterfactual data is inaccessible, we can never observe the true causal effects. In the absence of "supervision", how can we evaluate the performance of causal inference methods? In this paper, we use influence functions {—} the functional derivatives of a loss function {—} to develop a model validation procedure that estimates the estimation error of causal inference methods. Our procedure utilizes a Taylor-like expansion to approximate the loss function of a method on a given dataset in terms of the influence functions of its loss on a "synthesized", proximal dataset with known causal effects. Under minimal regularity assumptions, we show that our procedure is consistent and efficient. Experiments on 77 benchmark datasets show that using our procedure, we can accurately predict the comparative performances of state-of-the-art causal inference methods applied to a given observational study.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-alaa19a, title = {Validating Causal Inference Models via Influence Functions}, author = {Alaa, Ahmed and Van Der Schaar, Mihaela}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {191--201}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/alaa19a/alaa19a.pdf}, url = {https://proceedings.mlr.press/v97/alaa19a.html}, abstract = {The problem of estimating causal effects of treatments from observational data falls beyond the realm of supervised learning {—} because counterfactual data is inaccessible, we can never observe the true causal effects. In the absence of "supervision", how can we evaluate the performance of causal inference methods? In this paper, we use influence functions {—} the functional derivatives of a loss function {—} to develop a model validation procedure that estimates the estimation error of causal inference methods. Our procedure utilizes a Taylor-like expansion to approximate the loss function of a method on a given dataset in terms of the influence functions of its loss on a "synthesized", proximal dataset with known causal effects. Under minimal regularity assumptions, we show that our procedure is consistent and efficient. Experiments on 77 benchmark datasets show that using our procedure, we can accurately predict the comparative performances of state-of-the-art causal inference methods applied to a given observational study.} }
Endnote
%0 Conference Paper %T Validating Causal Inference Models via Influence Functions %A Ahmed Alaa %A Mihaela Van Der Schaar %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-alaa19a %I PMLR %P 191--201 %U https://proceedings.mlr.press/v97/alaa19a.html %V 97 %X The problem of estimating causal effects of treatments from observational data falls beyond the realm of supervised learning {—} because counterfactual data is inaccessible, we can never observe the true causal effects. In the absence of "supervision", how can we evaluate the performance of causal inference methods? In this paper, we use influence functions {—} the functional derivatives of a loss function {—} to develop a model validation procedure that estimates the estimation error of causal inference methods. Our procedure utilizes a Taylor-like expansion to approximate the loss function of a method on a given dataset in terms of the influence functions of its loss on a "synthesized", proximal dataset with known causal effects. Under minimal regularity assumptions, we show that our procedure is consistent and efficient. Experiments on 77 benchmark datasets show that using our procedure, we can accurately predict the comparative performances of state-of-the-art causal inference methods applied to a given observational study.
APA
Alaa, A. & Van Der Schaar, M.. (2019). Validating Causal Inference Models via Influence Functions. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:191-201 Available from https://proceedings.mlr.press/v97/alaa19a.html.

Related Material