Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders

Christian Toth, Christian Knoll, Franz Pernkopf, Robert Peharz
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:4240-4248, 2025.

Abstract

The traditional two-stage approach to causal inference first identifies a \emph{single} causal model (or equivalence class of models), which is then used to answer causal queries. However, this neglects any epistemic model uncertainty. In contrast, \emph{Bayesian} causal inference does incorporate epistemic uncertainty into query estimates via Bayesian marginalisation (posterior averaging) over \emph{all} causal models. While principled, this marginalisation over entire causal models, i.e., both causal structures (graphs) and mechanisms, poses a tremendous computational challenge. In this work, we address this challenge by decomposing structure marginalisation into the marginalisation over (i) causal orders and (ii) directed acyclic graphs (DAGs) given an order. We can marginalise the latter in closed form by limiting the number of parents per variable and utilising Gaussian Processes to model mechanisms. To marginalise over orders, we use a sampling-based approximation, for which we devise a novel auto-regressive distribution over causal orders (ARCO). Our method outperforms state-of-the-art in structure learning on simulated non-linear additive noise benchmarks, and yields competitive results on real-world data. Furthermore, we can accurately infer interventional distributions and average causal effects.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-toth25a, title = {Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders}, author = {Toth, Christian and Knoll, Christian and Pernkopf, Franz and Peharz, Robert}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {4240--4248}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/toth25a/toth25a.pdf}, url = {https://proceedings.mlr.press/v258/toth25a.html}, abstract = {The traditional two-stage approach to causal inference first identifies a \emph{single} causal model (or equivalence class of models), which is then used to answer causal queries. However, this neglects any epistemic model uncertainty. In contrast, \emph{Bayesian} causal inference does incorporate epistemic uncertainty into query estimates via Bayesian marginalisation (posterior averaging) over \emph{all} causal models. While principled, this marginalisation over entire causal models, i.e., both causal structures (graphs) and mechanisms, poses a tremendous computational challenge. In this work, we address this challenge by decomposing structure marginalisation into the marginalisation over (i) causal orders and (ii) directed acyclic graphs (DAGs) given an order. We can marginalise the latter in closed form by limiting the number of parents per variable and utilising Gaussian Processes to model mechanisms. To marginalise over orders, we use a sampling-based approximation, for which we devise a novel auto-regressive distribution over causal orders (ARCO). Our method outperforms state-of-the-art in structure learning on simulated non-linear additive noise benchmarks, and yields competitive results on real-world data. Furthermore, we can accurately infer interventional distributions and average causal effects.} }
Endnote
%0 Conference Paper %T Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders %A Christian Toth %A Christian Knoll %A Franz Pernkopf %A Robert Peharz %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-toth25a %I PMLR %P 4240--4248 %U https://proceedings.mlr.press/v258/toth25a.html %V 258 %X The traditional two-stage approach to causal inference first identifies a \emph{single} causal model (or equivalence class of models), which is then used to answer causal queries. However, this neglects any epistemic model uncertainty. In contrast, \emph{Bayesian} causal inference does incorporate epistemic uncertainty into query estimates via Bayesian marginalisation (posterior averaging) over \emph{all} causal models. While principled, this marginalisation over entire causal models, i.e., both causal structures (graphs) and mechanisms, poses a tremendous computational challenge. In this work, we address this challenge by decomposing structure marginalisation into the marginalisation over (i) causal orders and (ii) directed acyclic graphs (DAGs) given an order. We can marginalise the latter in closed form by limiting the number of parents per variable and utilising Gaussian Processes to model mechanisms. To marginalise over orders, we use a sampling-based approximation, for which we devise a novel auto-regressive distribution over causal orders (ARCO). Our method outperforms state-of-the-art in structure learning on simulated non-linear additive noise benchmarks, and yields competitive results on real-world data. Furthermore, we can accurately infer interventional distributions and average causal effects.
APA
Toth, C., Knoll, C., Pernkopf, F. & Peharz, R.. (2025). Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:4240-4248 Available from https://proceedings.mlr.press/v258/toth25a.html.

Related Material