Transformer Based Bayesian Network Structure Learning from an Information Theory Perspective

Zhiwei Qi, Kun Yue, Zhu Yang, Jiahui Wang
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:5541-5555, 2026.

Abstract

The pivotal challenge of handling uncertainty in artificial intelligence has prompted a growing focus on learning {Bayesian} network ({BN}) structures from data in recent years. Nevertheless, most of the existing methods such as hill-climbing, variational auto-encoder and graph neural network-based algorithms continue to encounter the issues like local optimum and exponential computational complexity, etc. In response to these challenges, we propose a Transformer-based approach for learning the {BN} structure (T-{BNSL}) from an information theory perspective. Specifically, we first establish the graph skeleton of a {BN} by employing conditional independence tests grounded in mutual information. Subsequently, we propose a hill-climbing method with decomposable BIC scoring function to generate potential directed acyclic graphs (DAGs) and evaluate the BIC scores of a subset of these DAGs. Lastly, we use the Transformer framework with a refined attention module to predict the BIC scores of the remaining DAGs, enabling the efficient identification of the {DAG} with the highest score. Experimental findings demonstrate that our approach significantly surpasses state-of-the-art competitors in terms of accuracy and efficiency by several orders of magnitude when learning the {BN} structure.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-qi26a, title = {Transformer Based {Bayesian} Network Structure Learning from an Information Theory Perspective}, author = {Qi, Zhiwei and Yue, Kun and Yang, Zhu and Wang, Jiahui}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {5541--5555}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/qi26a/qi26a.pdf}, url = {https://proceedings.mlr.press/v337/qi26a.html}, abstract = {The pivotal challenge of handling uncertainty in artificial intelligence has prompted a growing focus on learning {Bayesian} network ({BN}) structures from data in recent years. Nevertheless, most of the existing methods such as hill-climbing, variational auto-encoder and graph neural network-based algorithms continue to encounter the issues like local optimum and exponential computational complexity, etc. In response to these challenges, we propose a Transformer-based approach for learning the {BN} structure (T-{BNSL}) from an information theory perspective. Specifically, we first establish the graph skeleton of a {BN} by employing conditional independence tests grounded in mutual information. Subsequently, we propose a hill-climbing method with decomposable BIC scoring function to generate potential directed acyclic graphs (DAGs) and evaluate the BIC scores of a subset of these DAGs. Lastly, we use the Transformer framework with a refined attention module to predict the BIC scores of the remaining DAGs, enabling the efficient identification of the {DAG} with the highest score. Experimental findings demonstrate that our approach significantly surpasses state-of-the-art competitors in terms of accuracy and efficiency by several orders of magnitude when learning the {BN} structure.} }
Endnote
%0 Conference Paper %T Transformer Based Bayesian Network Structure Learning from an Information Theory Perspective %A Zhiwei Qi %A Kun Yue %A Zhu Yang %A Jiahui Wang %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-qi26a %I PMLR %P 5541--5555 %U https://proceedings.mlr.press/v337/qi26a.html %V 337 %X The pivotal challenge of handling uncertainty in artificial intelligence has prompted a growing focus on learning {Bayesian} network ({BN}) structures from data in recent years. Nevertheless, most of the existing methods such as hill-climbing, variational auto-encoder and graph neural network-based algorithms continue to encounter the issues like local optimum and exponential computational complexity, etc. In response to these challenges, we propose a Transformer-based approach for learning the {BN} structure (T-{BNSL}) from an information theory perspective. Specifically, we first establish the graph skeleton of a {BN} by employing conditional independence tests grounded in mutual information. Subsequently, we propose a hill-climbing method with decomposable BIC scoring function to generate potential directed acyclic graphs (DAGs) and evaluate the BIC scores of a subset of these DAGs. Lastly, we use the Transformer framework with a refined attention module to predict the BIC scores of the remaining DAGs, enabling the efficient identification of the {DAG} with the highest score. Experimental findings demonstrate that our approach significantly surpasses state-of-the-art competitors in terms of accuracy and efficiency by several orders of magnitude when learning the {BN} structure.
APA
Qi, Z., Yue, K., Yang, Z. & Wang, J.. (2026). Transformer Based Bayesian Network Structure Learning from an Information Theory Perspective. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:5541-5555 Available from https://proceedings.mlr.press/v337/qi26a.html.

Related Material