On the Identifiability and Estimation of Causal Location-Scale Noise Models

Alexander Immer, Christoph Schultheiss, Julia E Vogt, Bernhard Schölkopf, Peter Bühlmann, Alexander Marx
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:14316-14332, 2023.

Abstract

We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect $Y$ can be written as a function of the cause $X$ and a noise source $N$ independent of $X$, which may be scaled by a positive function $g$ over the cause, i.e., $Y = f(X) + g(X)N$. Despite the generality of the model class, we show the causal direction is identifiable up to some pathological cases. To empirically validate these theoretical findings, we propose two estimators for LSNMs: an estimator based on (non-linear) feature maps, and one based on neural networks. Both model the conditional distribution of $Y$ given $X$ as a Gaussian parameterized by its natural parameters. When the feature maps are correctly specified, we prove that our estimator is jointly concave, and a consistent estimator for the cause-effect identification task. Although the the neural network does not inherit those guarantees, it can fit functions of arbitrary complexity, and reaches state-of-the-art performance across benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-immer23a, title = {On the Identifiability and Estimation of Causal Location-Scale Noise Models}, author = {Immer, Alexander and Schultheiss, Christoph and Vogt, Julia E and Sch\"{o}lkopf, Bernhard and B\"{u}hlmann, Peter and Marx, Alexander}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {14316--14332}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/immer23a/immer23a.pdf}, url = {https://proceedings.mlr.press/v202/immer23a.html}, abstract = {We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect $Y$ can be written as a function of the cause $X$ and a noise source $N$ independent of $X$, which may be scaled by a positive function $g$ over the cause, i.e., $Y = f(X) + g(X)N$. Despite the generality of the model class, we show the causal direction is identifiable up to some pathological cases. To empirically validate these theoretical findings, we propose two estimators for LSNMs: an estimator based on (non-linear) feature maps, and one based on neural networks. Both model the conditional distribution of $Y$ given $X$ as a Gaussian parameterized by its natural parameters. When the feature maps are correctly specified, we prove that our estimator is jointly concave, and a consistent estimator for the cause-effect identification task. Although the the neural network does not inherit those guarantees, it can fit functions of arbitrary complexity, and reaches state-of-the-art performance across benchmarks.} }
Endnote
%0 Conference Paper %T On the Identifiability and Estimation of Causal Location-Scale Noise Models %A Alexander Immer %A Christoph Schultheiss %A Julia E Vogt %A Bernhard Schölkopf %A Peter Bühlmann %A Alexander Marx %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-immer23a %I PMLR %P 14316--14332 %U https://proceedings.mlr.press/v202/immer23a.html %V 202 %X We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect $Y$ can be written as a function of the cause $X$ and a noise source $N$ independent of $X$, which may be scaled by a positive function $g$ over the cause, i.e., $Y = f(X) + g(X)N$. Despite the generality of the model class, we show the causal direction is identifiable up to some pathological cases. To empirically validate these theoretical findings, we propose two estimators for LSNMs: an estimator based on (non-linear) feature maps, and one based on neural networks. Both model the conditional distribution of $Y$ given $X$ as a Gaussian parameterized by its natural parameters. When the feature maps are correctly specified, we prove that our estimator is jointly concave, and a consistent estimator for the cause-effect identification task. Although the the neural network does not inherit those guarantees, it can fit functions of arbitrary complexity, and reaches state-of-the-art performance across benchmarks.
APA
Immer, A., Schultheiss, C., Vogt, J.E., Schölkopf, B., Bühlmann, P. & Marx, A.. (2023). On the Identifiability and Estimation of Causal Location-Scale Noise Models. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:14316-14332 Available from https://proceedings.mlr.press/v202/immer23a.html.

Related Material