Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge

Shahine Bouabid, Jake Fawkes, Dino Sejdinovic
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:2885-2913, 2023.

Abstract

A directed acyclic graph (DAG) provides valuable prior knowledge that is often discarded in regression tasks in machine learning. We show that the independences arising from the presence of collider structures in DAGs provide meaningful inductive biases, which constrain the regression hypothesis space and improve predictive performance. We introduce collider regression, a framework to incorporate probabilistic causal knowledge from a collider in a regression problem. When the hypothesis space is a reproducing kernel Hilbert space, we prove a strictly positive generalisation benefit under mild assumptions and provide closed-form estimators of the empirical risk minimiser. Experiments on synthetic and climate model data demonstrate performance gains of the proposed methodology.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-bouabid23a, title = {Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge}, author = {Bouabid, Shahine and Fawkes, Jake and Sejdinovic, Dino}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {2885--2913}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/bouabid23a/bouabid23a.pdf}, url = {https://proceedings.mlr.press/v202/bouabid23a.html}, abstract = {A directed acyclic graph (DAG) provides valuable prior knowledge that is often discarded in regression tasks in machine learning. We show that the independences arising from the presence of collider structures in DAGs provide meaningful inductive biases, which constrain the regression hypothesis space and improve predictive performance. We introduce collider regression, a framework to incorporate probabilistic causal knowledge from a collider in a regression problem. When the hypothesis space is a reproducing kernel Hilbert space, we prove a strictly positive generalisation benefit under mild assumptions and provide closed-form estimators of the empirical risk minimiser. Experiments on synthetic and climate model data demonstrate performance gains of the proposed methodology.} }
Endnote
%0 Conference Paper %T Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge %A Shahine Bouabid %A Jake Fawkes %A Dino Sejdinovic %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-bouabid23a %I PMLR %P 2885--2913 %U https://proceedings.mlr.press/v202/bouabid23a.html %V 202 %X A directed acyclic graph (DAG) provides valuable prior knowledge that is often discarded in regression tasks in machine learning. We show that the independences arising from the presence of collider structures in DAGs provide meaningful inductive biases, which constrain the regression hypothesis space and improve predictive performance. We introduce collider regression, a framework to incorporate probabilistic causal knowledge from a collider in a regression problem. When the hypothesis space is a reproducing kernel Hilbert space, we prove a strictly positive generalisation benefit under mild assumptions and provide closed-form estimators of the empirical risk minimiser. Experiments on synthetic and climate model data demonstrate performance gains of the proposed methodology.
APA
Bouabid, S., Fawkes, J. & Sejdinovic, D.. (2023). Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:2885-2913 Available from https://proceedings.mlr.press/v202/bouabid23a.html.

Related Material