Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means

Raphaël Razafindralambo, Rémy Sun, Damien Garreau, Frederic Precioso, Pierre-Alexandre Mattei
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:5677-5699, 2026.

Abstract

Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order $r \in \mathbb{R} \cup \{-\infty,+\infty\}$ through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime $r \in [0,1]$ is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear ($r=1$) and geometric ($r=0$) pooling. In contrast, we show that aggregation rules with $r \notin [0,1]$ may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-razafindralambo26a, title = {Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means}, author = {Razafindralambo, Rapha\"{e}l and Sun, R\'{e}my and Garreau, Damien and Precioso, Frederic and Mattei, Pierre-Alexandre}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {5677--5699}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/razafindralambo26a/razafindralambo26a.pdf}, url = {https://proceedings.mlr.press/v337/razafindralambo26a.html}, abstract = {Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order $r \in \mathbb{R} \cup \{-\infty,+\infty\}$ through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime $r \in [0,1]$ is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear ($r=1$) and geometric ($r=0$) pooling. In contrast, we show that aggregation rules with $r \notin [0,1]$ may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.} }
Endnote
%0 Conference Paper %T Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means %A Raphaël Razafindralambo %A Rémy Sun %A Damien Garreau %A Frederic Precioso %A Pierre-Alexandre Mattei %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-razafindralambo26a %I PMLR %P 5677--5699 %U https://proceedings.mlr.press/v337/razafindralambo26a.html %V 337 %X Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order $r \in \mathbb{R} \cup \{-\infty,+\infty\}$ through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime $r \in [0,1]$ is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear ($r=1$) and geometric ($r=0$) pooling. In contrast, we show that aggregation rules with $r \notin [0,1]$ may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.
APA
Razafindralambo, R., Sun, R., Garreau, D., Precioso, F. & Mattei, P.. (2026). Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:5677-5699 Available from https://proceedings.mlr.press/v337/razafindralambo26a.html.

Related Material