Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding

Graham Van Goffrier, Lucas Maystre, Ciarán Mark Gilligan-Lee
Proceedings of the Second Conference on Causal Learning and Reasoning, PMLR 213:791-813, 2023.

Abstract

Understanding and quantifying cause and effect relationships is an important problem in many domains. The generally-agreed standard solution to this problem is to perform a randomised controlled trial. However, even when randomised controlled trials can be performed, they usually have relatively short duration’s due to cost considerations. This makes learning long-term causal effects a very challenging task in practice, since the long-term outcome is only observed after a long delay. In this paper, we study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Previous work provided an estimation strategy to determine long-term causal effects from such data regimes. However, this strategy only works if one assumes there are no unobserved confounders in the observational data. In this paper, we specifically address the challenging case where unmeasured confounders are present in the observational data. Our long-term causal effect estimator is obtained by combining regression residuals with short-term experimental outcomes in a specific manner to create an instrumental variable, which is then used to quantify the long-term causal effect through instrumental variable regression. We prove this estimator is unbiased, and analytically study its variance. Finally, we empirically test our approach on synthetic data, as well as real-data from the International Stroke Trial. Relevant source code and documentation has been made freely available in our \href{https://github.com/vangoffrier/UnConfounding}{online repository}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v213-goffrier23a, title = {Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding}, author = {Goffrier, Graham Van and Maystre, Lucas and Gilligan-Lee, Ciar\'an Mark}, booktitle = {Proceedings of the Second Conference on Causal Learning and Reasoning}, pages = {791--813}, year = {2023}, editor = {van der Schaar, Mihaela and Zhang, Cheng and Janzing, Dominik}, volume = {213}, series = {Proceedings of Machine Learning Research}, month = {11--14 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v213/goffrier23a/goffrier23a.pdf}, url = {https://proceedings.mlr.press/v213/goffrier23a.html}, abstract = {Understanding and quantifying cause and effect relationships is an important problem in many domains. The generally-agreed standard solution to this problem is to perform a randomised controlled trial. However, even when randomised controlled trials can be performed, they usually have relatively short duration’s due to cost considerations. This makes learning long-term causal effects a very challenging task in practice, since the long-term outcome is only observed after a long delay. In this paper, we study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Previous work provided an estimation strategy to determine long-term causal effects from such data regimes. However, this strategy only works if one assumes there are no unobserved confounders in the observational data. In this paper, we specifically address the challenging case where unmeasured confounders are present in the observational data. Our long-term causal effect estimator is obtained by combining regression residuals with short-term experimental outcomes in a specific manner to create an instrumental variable, which is then used to quantify the long-term causal effect through instrumental variable regression. We prove this estimator is unbiased, and analytically study its variance. Finally, we empirically test our approach on synthetic data, as well as real-data from the International Stroke Trial. Relevant source code and documentation has been made freely available in our \href{https://github.com/vangoffrier/UnConfounding}{online repository}.} }
Endnote
%0 Conference Paper %T Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding %A Graham Van Goffrier %A Lucas Maystre %A Ciarán Mark Gilligan-Lee %B Proceedings of the Second Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2023 %E Mihaela van der Schaar %E Cheng Zhang %E Dominik Janzing %F pmlr-v213-goffrier23a %I PMLR %P 791--813 %U https://proceedings.mlr.press/v213/goffrier23a.html %V 213 %X Understanding and quantifying cause and effect relationships is an important problem in many domains. The generally-agreed standard solution to this problem is to perform a randomised controlled trial. However, even when randomised controlled trials can be performed, they usually have relatively short duration’s due to cost considerations. This makes learning long-term causal effects a very challenging task in practice, since the long-term outcome is only observed after a long delay. In this paper, we study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Previous work provided an estimation strategy to determine long-term causal effects from such data regimes. However, this strategy only works if one assumes there are no unobserved confounders in the observational data. In this paper, we specifically address the challenging case where unmeasured confounders are present in the observational data. Our long-term causal effect estimator is obtained by combining regression residuals with short-term experimental outcomes in a specific manner to create an instrumental variable, which is then used to quantify the long-term causal effect through instrumental variable regression. We prove this estimator is unbiased, and analytically study its variance. Finally, we empirically test our approach on synthetic data, as well as real-data from the International Stroke Trial. Relevant source code and documentation has been made freely available in our \href{https://github.com/vangoffrier/UnConfounding}{online repository}.
APA
Goffrier, G.V., Maystre, L. & Gilligan-Lee, C.M.. (2023). Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding. Proceedings of the Second Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 213:791-813 Available from https://proceedings.mlr.press/v213/goffrier23a.html.

Related Material