On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors

Julius Kobialka, Emanuel Sommer, Chris Kolb, Juntae Kwon, Daniel Dold, David Rügamer
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:19-27, 2026.

Abstract

Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: layer balancedness, weight distribution on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-kobialka26a, title = { On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors }, author = {Kobialka, Julius and Sommer, Emanuel and Kolb, Chris and Kwon, Juntae and Dold, Daniel and R{\"u}gamer, David}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {19--27}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/kobialka26a/kobialka26a.pdf}, url = {https://proceedings.mlr.press/v300/kobialka26a.html}, abstract = { Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: layer balancedness, weight distribution on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions. } }
Endnote
%0 Conference Paper %T On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors %A Julius Kobialka %A Emanuel Sommer %A Chris Kolb %A Juntae Kwon %A Daniel Dold %A David Rügamer %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-kobialka26a %I PMLR %P 19--27 %U https://proceedings.mlr.press/v300/kobialka26a.html %V 300 %X Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: layer balancedness, weight distribution on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.
APA
Kobialka, J., Sommer, E., Kolb, C., Kwon, J., Dold, D. & Rügamer, D.. (2026). On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:19-27 Available from https://proceedings.mlr.press/v300/kobialka26a.html.

Related Material