An Analysis of the Adaptation Speed of Causal Models

Rémi Le Priol, Reza Babanezhad, Yoshua Bengio, Simon Lacoste-Julien
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:775-783, 2021.

Abstract

Consider a collection of datasets generated by unknown interventions on an unknown structural causal model $G$. Recently, Bengio et al. (2020) conjectured that among all candidate models, $G$ is the fastest to adapt from one dataset to another, along with promising experiments. Indeed, intuitively $G$ has less mechanisms to adapt, but this justification is incomplete. Our contribution is a more thorough analysis of this hypothesis. We investigate the adaptation speed of cause-effect SCMs. Using convergence rates from stochastic optimization, we justify that a relevant proxy for adaptation speed is distance in parameter space after intervention. Applying this proxy to categorical and normal cause-effect models, we show two results. When the intervention is on the cause variable, the SCM with the correct causal direction is advantaged by a large factor. When the intervention is on the effect variable, we characterize the relative adaptation speed. Surprisingly, we find situations where the anticausal model is advantaged, falsifying the initial hypothesis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-le-priol21a, title = { An Analysis of the Adaptation Speed of Causal Models }, author = {Le Priol, R{\'e}mi and Babanezhad, Reza and Bengio, Yoshua and Lacoste-Julien, Simon}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {775--783}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/le-priol21a/le-priol21a.pdf}, url = {https://proceedings.mlr.press/v130/le-priol21a.html}, abstract = { Consider a collection of datasets generated by unknown interventions on an unknown structural causal model $G$. Recently, Bengio et al. (2020) conjectured that among all candidate models, $G$ is the fastest to adapt from one dataset to another, along with promising experiments. Indeed, intuitively $G$ has less mechanisms to adapt, but this justification is incomplete. Our contribution is a more thorough analysis of this hypothesis. We investigate the adaptation speed of cause-effect SCMs. Using convergence rates from stochastic optimization, we justify that a relevant proxy for adaptation speed is distance in parameter space after intervention. Applying this proxy to categorical and normal cause-effect models, we show two results. When the intervention is on the cause variable, the SCM with the correct causal direction is advantaged by a large factor. When the intervention is on the effect variable, we characterize the relative adaptation speed. Surprisingly, we find situations where the anticausal model is advantaged, falsifying the initial hypothesis. } }
Endnote
%0 Conference Paper %T An Analysis of the Adaptation Speed of Causal Models %A Rémi Le Priol %A Reza Babanezhad %A Yoshua Bengio %A Simon Lacoste-Julien %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-le-priol21a %I PMLR %P 775--783 %U https://proceedings.mlr.press/v130/le-priol21a.html %V 130 %X Consider a collection of datasets generated by unknown interventions on an unknown structural causal model $G$. Recently, Bengio et al. (2020) conjectured that among all candidate models, $G$ is the fastest to adapt from one dataset to another, along with promising experiments. Indeed, intuitively $G$ has less mechanisms to adapt, but this justification is incomplete. Our contribution is a more thorough analysis of this hypothesis. We investigate the adaptation speed of cause-effect SCMs. Using convergence rates from stochastic optimization, we justify that a relevant proxy for adaptation speed is distance in parameter space after intervention. Applying this proxy to categorical and normal cause-effect models, we show two results. When the intervention is on the cause variable, the SCM with the correct causal direction is advantaged by a large factor. When the intervention is on the effect variable, we characterize the relative adaptation speed. Surprisingly, we find situations where the anticausal model is advantaged, falsifying the initial hypothesis.
APA
Le Priol, R., Babanezhad, R., Bengio, Y. & Lacoste-Julien, S.. (2021). An Analysis of the Adaptation Speed of Causal Models . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:775-783 Available from https://proceedings.mlr.press/v130/le-priol21a.html.

Related Material