Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference

Mykola Lukashchuk, Kyrylo Yemets, Wouter M. Kouw, Dmitry Bagaev, Ismail Senoz, Jeff Beck, Bert de Vries
Proceedings of the 2nd International Conference on Probabilistic Numerics, PMLR 341:102-123, 2026.

Abstract

Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be preserved. We identify five factor-graph primitives: a bilinear factor, an exponential link, a Gamma prior, a Gaussian likelihood, and an equality node, and prove that any model composed from them admits closed-form variational message passing. The construction works because each primitive preserves a small set of message families: under mean-field factorization, messages on Gaussian variables remain Gaussian and messages on precision variables remain Gamma, while the only non-conjugate interface, the exponential link, remains tractable through the Gaussian moment-generating function and the sufficient statistics of the Gamma family. We demonstrate composition at increasing depth, from static ensembles through input-dependent gating to split-branch routing, and show that stacking routing layers encodes arbitrary decision trees, establishing universal function approximation with closed-form inference. Applied to ensemble time-series forecasting, the framework yields a Bayesian mixture of experts in which gating functions are inferred rather than learned, providing calibrated uncertainty over expert selection across five benchmark datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v341-lukashchuk26a, title = {Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference}, author = {Lukashchuk, Mykola and Yemets, Kyrylo and Kouw, Wouter M. and Bagaev, Dmitry and Senoz, Ismail and Beck, Jeff and de Vries, Bert}, booktitle = {Proceedings of the 2nd International Conference on Probabilistic Numerics}, pages = {102--123}, year = {2026}, editor = {Karvonen, Toni and Bosch, Nathanael and Cockayne, Jon and Gessner, Alexandra and Hennig, Philipp and Kouw, Wouter}, volume = {341}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v341/main/assets/lukashchuk26a/lukashchuk26a.pdf}, url = {https://proceedings.mlr.press/v341/lukashchuk26a.html}, abstract = {Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be preserved. We identify five factor-graph primitives: a bilinear factor, an exponential link, a Gamma prior, a Gaussian likelihood, and an equality node, and prove that any model composed from them admits closed-form variational message passing. The construction works because each primitive preserves a small set of message families: under mean-field factorization, messages on Gaussian variables remain Gaussian and messages on precision variables remain Gamma, while the only non-conjugate interface, the exponential link, remains tractable through the Gaussian moment-generating function and the sufficient statistics of the Gamma family. We demonstrate composition at increasing depth, from static ensembles through input-dependent gating to split-branch routing, and show that stacking routing layers encodes arbitrary decision trees, establishing universal function approximation with closed-form inference. Applied to ensemble time-series forecasting, the framework yields a Bayesian mixture of experts in which gating functions are inferred rather than learned, providing calibrated uncertainty over expert selection across five benchmark datasets.} }
Endnote
%0 Conference Paper %T Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference %A Mykola Lukashchuk %A Kyrylo Yemets %A Wouter M. Kouw %A Dmitry Bagaev %A Ismail Senoz %A Jeff Beck %A Bert de Vries %B Proceedings of the 2nd International Conference on Probabilistic Numerics %C Proceedings of Machine Learning Research %D 2026 %E Toni Karvonen %E Nathanael Bosch %E Jon Cockayne %E Alexandra Gessner %E Philipp Hennig %E Wouter Kouw %F pmlr-v341-lukashchuk26a %I PMLR %P 102--123 %U https://proceedings.mlr.press/v341/lukashchuk26a.html %V 341 %X Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show that closed-form inference can be preserved. We identify five factor-graph primitives: a bilinear factor, an exponential link, a Gamma prior, a Gaussian likelihood, and an equality node, and prove that any model composed from them admits closed-form variational message passing. The construction works because each primitive preserves a small set of message families: under mean-field factorization, messages on Gaussian variables remain Gaussian and messages on precision variables remain Gamma, while the only non-conjugate interface, the exponential link, remains tractable through the Gaussian moment-generating function and the sufficient statistics of the Gamma family. We demonstrate composition at increasing depth, from static ensembles through input-dependent gating to split-branch routing, and show that stacking routing layers encodes arbitrary decision trees, establishing universal function approximation with closed-form inference. Applied to ensemble time-series forecasting, the framework yields a Bayesian mixture of experts in which gating functions are inferred rather than learned, providing calibrated uncertainty over expert selection across five benchmark datasets.
APA
Lukashchuk, M., Yemets, K., Kouw, W.M., Bagaev, D., Senoz, I., Beck, J. & de Vries, B.. (2026). Composing Non-Conjugate Factor Graphs with Closed-Form Variational Inference. Proceedings of the 2nd International Conference on Probabilistic Numerics, in Proceedings of Machine Learning Research 341:102-123 Available from https://proceedings.mlr.press/v341/lukashchuk26a.html.

Related Material