A Divergence-Based Method for Weighting and Averaging Model Predictions

Olav Benjamin Vassend
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:1855-1863, 2026.

Abstract

This paper uses a minimum divergence framework to introduce a new way of calculating model weights that can be used to average probabilistic predictions from statistical and machine learning models. The method is general and can be applied regardless of whether the models under consideration are fit to data using frequentist, Bayesian, or some other fitting method. The proposed method is motivated in two different ways and is shown empirically to perform better than or on a par with standard model averaging methods, including model stacking and model averaging that relies on Akaike-style negative exponentiated model weighting, especially when the sample size is small. Our theoretical analysis explains why the method has a small-sample advantage.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-vassend26a, title = { A Divergence-Based Method for Weighting and Averaging Model Predictions }, author = {Vassend, Olav Benjamin}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {1855--1863}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/vassend26a/vassend26a.pdf}, url = {https://proceedings.mlr.press/v300/vassend26a.html}, abstract = { This paper uses a minimum divergence framework to introduce a new way of calculating model weights that can be used to average probabilistic predictions from statistical and machine learning models. The method is general and can be applied regardless of whether the models under consideration are fit to data using frequentist, Bayesian, or some other fitting method. The proposed method is motivated in two different ways and is shown empirically to perform better than or on a par with standard model averaging methods, including model stacking and model averaging that relies on Akaike-style negative exponentiated model weighting, especially when the sample size is small. Our theoretical analysis explains why the method has a small-sample advantage. } }
Endnote
%0 Conference Paper %T A Divergence-Based Method for Weighting and Averaging Model Predictions %A Olav Benjamin Vassend %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-vassend26a %I PMLR %P 1855--1863 %U https://proceedings.mlr.press/v300/vassend26a.html %V 300 %X This paper uses a minimum divergence framework to introduce a new way of calculating model weights that can be used to average probabilistic predictions from statistical and machine learning models. The method is general and can be applied regardless of whether the models under consideration are fit to data using frequentist, Bayesian, or some other fitting method. The proposed method is motivated in two different ways and is shown empirically to perform better than or on a par with standard model averaging methods, including model stacking and model averaging that relies on Akaike-style negative exponentiated model weighting, especially when the sample size is small. Our theoretical analysis explains why the method has a small-sample advantage.
APA
Vassend, O.B.. (2026). A Divergence-Based Method for Weighting and Averaging Model Predictions . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:1855-1863 Available from https://proceedings.mlr.press/v300/vassend26a.html.

Related Material