Rethinking Aleatoric and Epistemic Uncertainty

Freddie Bickford Smith, Jannik Kossen, Eleanor Trollope, Mark Van Der Wilk, Adam Foster, Tom Rainforth
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:4345-4359, 2025.

Abstract

The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all the distinct quantities that researchers are interested in. To address this we present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance and statistical dispersion in data. This serves to support clearer thinking as the field moves forward. Additionally we provide insights into popular information-theoretic quantities, showing they can be poor estimators of what they are often purported to measure, while also explaining how they can still be useful in guiding data acquisition.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-bickford-smith25a, title = {Rethinking Aleatoric and Epistemic Uncertainty}, author = {Bickford Smith, Freddie and Kossen, Jannik and Trollope, Eleanor and Van Der Wilk, Mark and Foster, Adam and Rainforth, Tom}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {4345--4359}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/bickford-smith25a/bickford-smith25a.pdf}, url = {https://proceedings.mlr.press/v267/bickford-smith25a.html}, abstract = {The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all the distinct quantities that researchers are interested in. To address this we present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance and statistical dispersion in data. This serves to support clearer thinking as the field moves forward. Additionally we provide insights into popular information-theoretic quantities, showing they can be poor estimators of what they are often purported to measure, while also explaining how they can still be useful in guiding data acquisition.} }
Endnote
%0 Conference Paper %T Rethinking Aleatoric and Epistemic Uncertainty %A Freddie Bickford Smith %A Jannik Kossen %A Eleanor Trollope %A Mark Van Der Wilk %A Adam Foster %A Tom Rainforth %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-bickford-smith25a %I PMLR %P 4345--4359 %U https://proceedings.mlr.press/v267/bickford-smith25a.html %V 267 %X The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all the distinct quantities that researchers are interested in. To address this we present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance and statistical dispersion in data. This serves to support clearer thinking as the field moves forward. Additionally we provide insights into popular information-theoretic quantities, showing they can be poor estimators of what they are often purported to measure, while also explaining how they can still be useful in guiding data acquisition.
APA
Bickford Smith, F., Kossen, J., Trollope, E., Van Der Wilk, M., Foster, A. & Rainforth, T.. (2025). Rethinking Aleatoric and Epistemic Uncertainty. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:4345-4359 Available from https://proceedings.mlr.press/v267/bickford-smith25a.html.

Related Material