Learning with Confidence

Oliver Ethan Richardson
Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, PMLR 286:3552-3569, 2025.

Abstract

We characterize a notion of confidence that arises in learning or updating beliefs: the amount of trust one has in incoming information and its impact on the belief state. This *learner’s confidence* can be used alongside (and is easily mistaken for) probability or likelihood, but it is fundamentally a different concept—one that captures many familiar concepts in the literature, including learning rates and number of training epochs, Shafer’s weight of evidence, and Kalman gain. We formally axiomatize what it means to learn with confidence, give two canonical ways of measuring confidence on a continuum, and prove that confidence can always be represented in this way. Under additional assumptions, we derive more compact representations of confidence-based learning in terms of vector fields and loss functions. These representations induce an extended language of compound "parallel" observations. We characterize *Bayesian* learning as the special case of an optimizing learner whose loss representation is a linear expectation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v286-richardson25a, title = {Learning with Confidence}, author = {Richardson, Oliver Ethan}, booktitle = {Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence}, pages = {3552--3569}, year = {2025}, editor = {Chiappa, Silvia and Magliacane, Sara}, volume = {286}, series = {Proceedings of Machine Learning Research}, month = {21--25 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v286/main/assets/richardson25a/richardson25a.pdf}, url = {https://proceedings.mlr.press/v286/richardson25a.html}, abstract = {We characterize a notion of confidence that arises in learning or updating beliefs: the amount of trust one has in incoming information and its impact on the belief state. This *learner’s confidence* can be used alongside (and is easily mistaken for) probability or likelihood, but it is fundamentally a different concept—one that captures many familiar concepts in the literature, including learning rates and number of training epochs, Shafer’s weight of evidence, and Kalman gain. We formally axiomatize what it means to learn with confidence, give two canonical ways of measuring confidence on a continuum, and prove that confidence can always be represented in this way. Under additional assumptions, we derive more compact representations of confidence-based learning in terms of vector fields and loss functions. These representations induce an extended language of compound "parallel" observations. We characterize *Bayesian* learning as the special case of an optimizing learner whose loss representation is a linear expectation.} }
Endnote
%0 Conference Paper %T Learning with Confidence %A Oliver Ethan Richardson %B Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2025 %E Silvia Chiappa %E Sara Magliacane %F pmlr-v286-richardson25a %I PMLR %P 3552--3569 %U https://proceedings.mlr.press/v286/richardson25a.html %V 286 %X We characterize a notion of confidence that arises in learning or updating beliefs: the amount of trust one has in incoming information and its impact on the belief state. This *learner’s confidence* can be used alongside (and is easily mistaken for) probability or likelihood, but it is fundamentally a different concept—one that captures many familiar concepts in the literature, including learning rates and number of training epochs, Shafer’s weight of evidence, and Kalman gain. We formally axiomatize what it means to learn with confidence, give two canonical ways of measuring confidence on a continuum, and prove that confidence can always be represented in this way. Under additional assumptions, we derive more compact representations of confidence-based learning in terms of vector fields and loss functions. These representations induce an extended language of compound "parallel" observations. We characterize *Bayesian* learning as the special case of an optimizing learner whose loss representation is a linear expectation.
APA
Richardson, O.E.. (2025). Learning with Confidence. Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 286:3552-3569 Available from https://proceedings.mlr.press/v286/richardson25a.html.

Related Material