BNqMark: Bayesian-Network Inference in Large Language Models

Fernando Rodriguez, Bojan Mihaljević
Proceedings of the 13th International Conference on Probabilistic Graphical Models, PMLR 343:725-757, 2026.

Abstract

We present the first systematic evaluation of exact Bayesian-network inference in frontier reasoning large language models (LLMs). We introduce BNqMark, a benchmark-generation framework for exact inference in discrete Bayesian networks, and evaluate BNqMark-20, a benchmark instance comprising 78 procedurally generated binary networks and 434 conditional probability queries with exact ground truth. Across 9 frontier reasoning LLMs and 2 evaluation protocols, we find that current models can solve many exact inference queries on small and medium Bayesian networks, especially by externalizing inference as code. Several models correctly answered queries whose exact solution would require more than 13,000 arithmetic operations under variable elimination, rising to 68,000 in the code-generation protocol. Model capability degrades as inference complexity increases: graph and query complexity are both useful predictors of success after controlling for prompt length, network size, and model. These results position Bayesian networks as a controlled testbed for studying probabilistic reasoning in LLMs and motivate future work on richer and more challenging probabilistic inference settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v343-rodriguez26a, title = {BNqMark: {Bayesian}-Network Inference in Large Language Models}, author = {Rodriguez, Fernando and Mihaljevi\'c, Bojan}, booktitle = {Proceedings of the 13th International Conference on Probabilistic Graphical Models}, pages = {725--757}, year = {2026}, editor = {Camps-Valls, Gustau and Leonelli, Manuele and Varando, Gherardo}, volume = {343}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v343/main/assets/rodriguez26a/rodriguez26a.pdf}, url = {https://proceedings.mlr.press/v343/rodriguez26a.html}, abstract = {We present the first systematic evaluation of exact Bayesian-network inference in frontier reasoning large language models (LLMs). We introduce BNqMark, a benchmark-generation framework for exact inference in discrete Bayesian networks, and evaluate BNqMark-20, a benchmark instance comprising 78 procedurally generated binary networks and 434 conditional probability queries with exact ground truth. Across 9 frontier reasoning LLMs and 2 evaluation protocols, we find that current models can solve many exact inference queries on small and medium Bayesian networks, especially by externalizing inference as code. Several models correctly answered queries whose exact solution would require more than 13,000 arithmetic operations under variable elimination, rising to 68,000 in the code-generation protocol. Model capability degrades as inference complexity increases: graph and query complexity are both useful predictors of success after controlling for prompt length, network size, and model. These results position Bayesian networks as a controlled testbed for studying probabilistic reasoning in LLMs and motivate future work on richer and more challenging probabilistic inference settings.} }
Endnote
%0 Conference Paper %T BNqMark: Bayesian-Network Inference in Large Language Models %A Fernando Rodriguez %A Bojan Mihaljević %B Proceedings of the 13th International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2026 %E Gustau Camps-Valls %E Manuele Leonelli %E Gherardo Varando %F pmlr-v343-rodriguez26a %I PMLR %P 725--757 %U https://proceedings.mlr.press/v343/rodriguez26a.html %V 343 %X We present the first systematic evaluation of exact Bayesian-network inference in frontier reasoning large language models (LLMs). We introduce BNqMark, a benchmark-generation framework for exact inference in discrete Bayesian networks, and evaluate BNqMark-20, a benchmark instance comprising 78 procedurally generated binary networks and 434 conditional probability queries with exact ground truth. Across 9 frontier reasoning LLMs and 2 evaluation protocols, we find that current models can solve many exact inference queries on small and medium Bayesian networks, especially by externalizing inference as code. Several models correctly answered queries whose exact solution would require more than 13,000 arithmetic operations under variable elimination, rising to 68,000 in the code-generation protocol. Model capability degrades as inference complexity increases: graph and query complexity are both useful predictors of success after controlling for prompt length, network size, and model. These results position Bayesian networks as a controlled testbed for studying probabilistic reasoning in LLMs and motivate future work on richer and more challenging probabilistic inference settings.
APA
Rodriguez, F. & Mihaljević, B.. (2026). BNqMark: Bayesian-Network Inference in Large Language Models. Proceedings of the 13th International Conference on Probabilistic Graphical Models, in Proceedings of Machine Learning Research 343:725-757 Available from https://proceedings.mlr.press/v343/rodriguez26a.html.

Related Material