Regularizing towards Causal Invariance: Linear Models with Proxies

Michael Oberst, Nikolaj Thams, Jonas Peters, David Sontag
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:8260-8270, 2021.

Abstract

We propose a method for learning linear models whose predictive performance is robust to causal interventions on unobserved variables, when noisy proxies of those variables are available. Our approach takes the form of a regularization term that trades off between in-distribution performance and robustness to interventions. Under the assumption of a linear structural causal model, we show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength. This strength depends on the magnitude of the measurement noise in the proxy, which is, in general, not identifiable. In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength. We further show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training. We evaluate our theoretical findings in synthetic experiments and using real data of hourly pollution levels across several cities in China.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-oberst21a, title = {Regularizing towards Causal Invariance: Linear Models with Proxies}, author = {Oberst, Michael and Thams, Nikolaj and Peters, Jonas and Sontag, David}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {8260--8270}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/oberst21a/oberst21a.pdf}, url = {https://proceedings.mlr.press/v139/oberst21a.html}, abstract = {We propose a method for learning linear models whose predictive performance is robust to causal interventions on unobserved variables, when noisy proxies of those variables are available. Our approach takes the form of a regularization term that trades off between in-distribution performance and robustness to interventions. Under the assumption of a linear structural causal model, we show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength. This strength depends on the magnitude of the measurement noise in the proxy, which is, in general, not identifiable. In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength. We further show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training. We evaluate our theoretical findings in synthetic experiments and using real data of hourly pollution levels across several cities in China.} }
Endnote
%0 Conference Paper %T Regularizing towards Causal Invariance: Linear Models with Proxies %A Michael Oberst %A Nikolaj Thams %A Jonas Peters %A David Sontag %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-oberst21a %I PMLR %P 8260--8270 %U https://proceedings.mlr.press/v139/oberst21a.html %V 139 %X We propose a method for learning linear models whose predictive performance is robust to causal interventions on unobserved variables, when noisy proxies of those variables are available. Our approach takes the form of a regularization term that trades off between in-distribution performance and robustness to interventions. Under the assumption of a linear structural causal model, we show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength. This strength depends on the magnitude of the measurement noise in the proxy, which is, in general, not identifiable. In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength. We further show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training. We evaluate our theoretical findings in synthetic experiments and using real data of hourly pollution levels across several cities in China.
APA
Oberst, M., Thams, N., Peters, J. & Sontag, D.. (2021). Regularizing towards Causal Invariance: Linear Models with Proxies. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:8260-8270 Available from https://proceedings.mlr.press/v139/oberst21a.html.

Related Material