Continuous Bayesian Model Selection for Multivariate Causal Discovery

Anish Dhir, Ruby Sedgwick, Avinash Kori, Ben Glocker, Mark Van Der Wilk
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:13486-13512, 2025.

Abstract

Current causal discovery approaches require restrictive model assumptions in the absence of interventional data to ensure structure identifiability. These assumptions often do not hold in real-world applications leading to a loss of guarantees and poor performance in practice. Recent work has shown that, in the bivariate case, Bayesian model selection can greatly improve performance by exchanging restrictive modelling for more flexible assumptions, at the cost of a small probability of making an error. Our work shows that this approach is useful in the important multivariate case as well. We propose a scalable algorithm leveraging a continuous relaxation of the discrete model selection problem. Specifically, we employ the Causal Gaussian Process Conditional Density Estimator (CGP-CDE) as a Bayesian non-parametric model, using its hyperparameters to construct an adjacency matrix. This matrix is then optimised using the marginal likelihood and an acyclicity regulariser, giving the maximum a posteriori causal graph. We demonstrate the competitiveness of our approach, showing it is advantageous to perform multivariate causal discovery without infeasible assumptions using Bayesian model selection.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-dhir25a, title = {Continuous {B}ayesian Model Selection for Multivariate Causal Discovery}, author = {Dhir, Anish and Sedgwick, Ruby and Kori, Avinash and Glocker, Ben and Van Der Wilk, Mark}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {13486--13512}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/dhir25a/dhir25a.pdf}, url = {https://proceedings.mlr.press/v267/dhir25a.html}, abstract = {Current causal discovery approaches require restrictive model assumptions in the absence of interventional data to ensure structure identifiability. These assumptions often do not hold in real-world applications leading to a loss of guarantees and poor performance in practice. Recent work has shown that, in the bivariate case, Bayesian model selection can greatly improve performance by exchanging restrictive modelling for more flexible assumptions, at the cost of a small probability of making an error. Our work shows that this approach is useful in the important multivariate case as well. We propose a scalable algorithm leveraging a continuous relaxation of the discrete model selection problem. Specifically, we employ the Causal Gaussian Process Conditional Density Estimator (CGP-CDE) as a Bayesian non-parametric model, using its hyperparameters to construct an adjacency matrix. This matrix is then optimised using the marginal likelihood and an acyclicity regulariser, giving the maximum a posteriori causal graph. We demonstrate the competitiveness of our approach, showing it is advantageous to perform multivariate causal discovery without infeasible assumptions using Bayesian model selection.} }
Endnote
%0 Conference Paper %T Continuous Bayesian Model Selection for Multivariate Causal Discovery %A Anish Dhir %A Ruby Sedgwick %A Avinash Kori %A Ben Glocker %A Mark Van Der Wilk %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-dhir25a %I PMLR %P 13486--13512 %U https://proceedings.mlr.press/v267/dhir25a.html %V 267 %X Current causal discovery approaches require restrictive model assumptions in the absence of interventional data to ensure structure identifiability. These assumptions often do not hold in real-world applications leading to a loss of guarantees and poor performance in practice. Recent work has shown that, in the bivariate case, Bayesian model selection can greatly improve performance by exchanging restrictive modelling for more flexible assumptions, at the cost of a small probability of making an error. Our work shows that this approach is useful in the important multivariate case as well. We propose a scalable algorithm leveraging a continuous relaxation of the discrete model selection problem. Specifically, we employ the Causal Gaussian Process Conditional Density Estimator (CGP-CDE) as a Bayesian non-parametric model, using its hyperparameters to construct an adjacency matrix. This matrix is then optimised using the marginal likelihood and an acyclicity regulariser, giving the maximum a posteriori causal graph. We demonstrate the competitiveness of our approach, showing it is advantageous to perform multivariate causal discovery without infeasible assumptions using Bayesian model selection.
APA
Dhir, A., Sedgwick, R., Kori, A., Glocker, B. & Van Der Wilk, M.. (2025). Continuous Bayesian Model Selection for Multivariate Causal Discovery. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:13486-13512 Available from https://proceedings.mlr.press/v267/dhir25a.html.

Related Material