Posterior Uncertainty Quantification in Neural Networks using Data Augmentation

Luhuan Wu, Sinead A Williamson
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:3376-3384, 2024.

Abstract

In this paper, we approach the problem of uncertainty quantification in deep learning through a predictive framework, which captures uncertainty in model parameters by specifying our assumptions about the predictive distribution of unseen future data. Under this view, we show that deep ensembling (Lakshminarayanan et al., 2017) is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only—a situation rarely encountered in practice. To address this limitation, we propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques. MixupMP operates as a drop-in replacement for deep ensembles, where each ensemble member is trained on a random simulation from this predictive distribution. Grounded in the recently-proposed framework of Martingale posteriors (Fong et al., 2023), MixupMP returns samples from an implicitly defined Bayesian posterior. Our empirical analysis showcases that MixupMP achieves superior predictive per- formance and uncertainty quantification on various image classification datasets, when compared with existing Bayesian and non-Bayesian approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-wu24e, title = {Posterior Uncertainty Quantification in Neural Networks using Data Augmentation}, author = {Wu, Luhuan and A Williamson, Sinead}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {3376--3384}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/wu24e/wu24e.pdf}, url = {https://proceedings.mlr.press/v238/wu24e.html}, abstract = {In this paper, we approach the problem of uncertainty quantification in deep learning through a predictive framework, which captures uncertainty in model parameters by specifying our assumptions about the predictive distribution of unseen future data. Under this view, we show that deep ensembling (Lakshminarayanan et al., 2017) is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only—a situation rarely encountered in practice. To address this limitation, we propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques. MixupMP operates as a drop-in replacement for deep ensembles, where each ensemble member is trained on a random simulation from this predictive distribution. Grounded in the recently-proposed framework of Martingale posteriors (Fong et al., 2023), MixupMP returns samples from an implicitly defined Bayesian posterior. Our empirical analysis showcases that MixupMP achieves superior predictive per- formance and uncertainty quantification on various image classification datasets, when compared with existing Bayesian and non-Bayesian approaches.} }
Endnote
%0 Conference Paper %T Posterior Uncertainty Quantification in Neural Networks using Data Augmentation %A Luhuan Wu %A Sinead A Williamson %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-wu24e %I PMLR %P 3376--3384 %U https://proceedings.mlr.press/v238/wu24e.html %V 238 %X In this paper, we approach the problem of uncertainty quantification in deep learning through a predictive framework, which captures uncertainty in model parameters by specifying our assumptions about the predictive distribution of unseen future data. Under this view, we show that deep ensembling (Lakshminarayanan et al., 2017) is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only—a situation rarely encountered in practice. To address this limitation, we propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques. MixupMP operates as a drop-in replacement for deep ensembles, where each ensemble member is trained on a random simulation from this predictive distribution. Grounded in the recently-proposed framework of Martingale posteriors (Fong et al., 2023), MixupMP returns samples from an implicitly defined Bayesian posterior. Our empirical analysis showcases that MixupMP achieves superior predictive per- formance and uncertainty quantification on various image classification datasets, when compared with existing Bayesian and non-Bayesian approaches.
APA
Wu, L. & A Williamson, S.. (2024). Posterior Uncertainty Quantification in Neural Networks using Data Augmentation. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:3376-3384 Available from https://proceedings.mlr.press/v238/wu24e.html.

Related Material