Augmenting Bayesian Networks with Categorical Principal Components

Ashley Bojko, Marie Melı́nová, Jiřı́ Vomlel
Proceedings of the 13th International Conference on Probabilistic Graphical Models, PMLR 343:57-72, 2026.

Abstract

We propose a method for learning interpretable Bayesian networks (BNs) over high-dimensional categorical data by augmenting the original variable set with discrete components derived from expert-defined groups of variables. Because the components are computed deterministically and entered as fully observed columns, the augmented dataset defines a BN in a curved exponential family, making standard BIC asymptotically valid for joint model scoring. To enable a fair comparison of models with different numbers of component nodes — which induce different model classes over the original variables — we introduce a modified BIC that evaluates marginal likelihood over the original variables while retaining the full parameter penalty. We show that this criterion is conservative relative to the true Geiger-Heckerman effective dimension and establish when the marginal and joint criteria agree asymptotically. We compare PCA-derived components (applied to ordinally coded variables) with Multiple Correspondence Analysis (MCA), the discrete analog of PCA, and show theoretically and empirically that MCA is the more natural choice. Experiments on the Czech Attitude Barometer dataset demonstrate that MCA models achieve the best marginal BIC as well as the best held-out marginal log-likelihood across all augmented and non-augmented models, yielding a sparse, stable, and sociologically interpretable network.

Cite this Paper

BibTeX
@InProceedings{pmlr-v343-bojko26a, title = {Augmenting {Bayesian} Networks with Categorical Principal Components}, author = {Bojko, Ashley and Mel\'{\i}nov\'{a}, Marie and Vomlel, Ji\v{r}\'{\i}}, booktitle = {Proceedings of the 13th International Conference on Probabilistic Graphical Models}, pages = {57--72}, year = {2026}, editor = {Camps-Valls, Gustau and Leonelli, Manuele and Varando, Gherardo}, volume = {343}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v343/main/assets/bojko26a/bojko26a.pdf}, url = {https://proceedings.mlr.press/v343/bojko26a.html}, abstract = {We propose a method for learning interpretable Bayesian networks (BNs) over high-dimensional categorical data by augmenting the original variable set with discrete components derived from expert-defined groups of variables. Because the components are computed deterministically and entered as fully observed columns, the augmented dataset defines a BN in a curved exponential family, making standard BIC asymptotically valid for joint model scoring. To enable a fair comparison of models with different numbers of component nodes — which induce different model classes over the original variables — we introduce a modified BIC that evaluates marginal likelihood over the original variables while retaining the full parameter penalty. We show that this criterion is conservative relative to the true Geiger-Heckerman effective dimension and establish when the marginal and joint criteria agree asymptotically. We compare PCA-derived components (applied to ordinally coded variables) with Multiple Correspondence Analysis (MCA), the discrete analog of PCA, and show theoretically and empirically that MCA is the more natural choice. Experiments on the Czech Attitude Barometer dataset demonstrate that MCA models achieve the best marginal BIC as well as the best held-out marginal log-likelihood across all augmented and non-augmented models, yielding a sparse, stable, and sociologically interpretable network.} }
Endnote
%0 Conference Paper %T Augmenting Bayesian Networks with Categorical Principal Components %A Ashley Bojko %A Marie Melı́nová %A Jiřı́ Vomlel %B Proceedings of the 13th International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2026 %E Gustau Camps-Valls %E Manuele Leonelli %E Gherardo Varando %F pmlr-v343-bojko26a %I PMLR %P 57--72 %U https://proceedings.mlr.press/v343/bojko26a.html %V 343 %X We propose a method for learning interpretable Bayesian networks (BNs) over high-dimensional categorical data by augmenting the original variable set with discrete components derived from expert-defined groups of variables. Because the components are computed deterministically and entered as fully observed columns, the augmented dataset defines a BN in a curved exponential family, making standard BIC asymptotically valid for joint model scoring. To enable a fair comparison of models with different numbers of component nodes — which induce different model classes over the original variables — we introduce a modified BIC that evaluates marginal likelihood over the original variables while retaining the full parameter penalty. We show that this criterion is conservative relative to the true Geiger-Heckerman effective dimension and establish when the marginal and joint criteria agree asymptotically. We compare PCA-derived components (applied to ordinally coded variables) with Multiple Correspondence Analysis (MCA), the discrete analog of PCA, and show theoretically and empirically that MCA is the more natural choice. Experiments on the Czech Attitude Barometer dataset demonstrate that MCA models achieve the best marginal BIC as well as the best held-out marginal log-likelihood across all augmented and non-augmented models, yielding a sparse, stable, and sociologically interpretable network.
APA
Bojko, A., Melı́nová, M. & Vomlel, J.. (2026). Augmenting Bayesian Networks with Categorical Principal Components. Proceedings of the 13th International Conference on Probabilistic Graphical Models, in Proceedings of Machine Learning Research 343:57-72 Available from https://proceedings.mlr.press/v343/bojko26a.html.

Related Material