Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles

Tárik S. Salem, Helge Langseth, Heri Ramampiaro
Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR 124:1179-1187, 2020.

Abstract

Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.

Cite this Paper

BibTeX
@InProceedings{pmlr-v124-saleh-salem20a, title = {Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles}, author = {S. Salem, T\'arik and Langseth, Helge and Ramampiaro, Heri}, booktitle = {Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)}, pages = {1179--1187}, year = {2020}, editor = {Peters, Jonas and Sontag, David}, volume = {124}, series = {Proceedings of Machine Learning Research}, month = {03--06 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v124/saleh-salem20a/saleh-salem20a.pdf}, url = {https://proceedings.mlr.press/v124/saleh-salem20a.html}, abstract = {Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates. } }
Endnote
%0 Conference Paper %T Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles %A Tárik S. Salem %A Helge Langseth %A Heri Ramampiaro %B Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) %C Proceedings of Machine Learning Research %D 2020 %E Jonas Peters %E David Sontag %F pmlr-v124-saleh-salem20a %I PMLR %P 1179--1187 %U https://proceedings.mlr.press/v124/saleh-salem20a.html %V 124 %X Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.
APA
S. Salem, T., Langseth, H. & Ramampiaro, H.. (2020). Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), in Proceedings of Machine Learning Research 124:1179-1187 Available from https://proceedings.mlr.press/v124/saleh-salem20a.html.

Related Material