Multicalibration for Confidence Scoring in LLMs

Gianluca Detommaso, Martin Andres Bertran, Riccardo Fogliato, Aaron Roth
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:10624-10641, 2024.

Abstract

This paper proposes the use of "multicalibration": to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs). Multicalibration asks for calibration not just marginally, but simultaneously across various intersecting groupings of the data. We show how to form groupings for prompt/completion pairs that are correlated with the probability of correctness via two techniques: clustering within an embedding space, and "self-annotation" - querying the LLM by asking it various yes-or-no questions about the prompt. We also develop novel variants of multicalibration algorithms that offer performance improvements by reducing their tendency to overfit. Through systematic benchmarking across various question answering datasets and LLMs, we show how our techniques can yield confidence scores that provide substantial improvements in fine-grained measures of both calibration and accuracy compared to existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-detommaso24a, title = {Multicalibration for Confidence Scoring in {LLM}s}, author = {Detommaso, Gianluca and Bertran, Martin Andres and Fogliato, Riccardo and Roth, Aaron}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {10624--10641}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/detommaso24a/detommaso24a.pdf}, url = {https://proceedings.mlr.press/v235/detommaso24a.html}, abstract = {This paper proposes the use of "multicalibration": to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs). Multicalibration asks for calibration not just marginally, but simultaneously across various intersecting groupings of the data. We show how to form groupings for prompt/completion pairs that are correlated with the probability of correctness via two techniques: clustering within an embedding space, and "self-annotation" - querying the LLM by asking it various yes-or-no questions about the prompt. We also develop novel variants of multicalibration algorithms that offer performance improvements by reducing their tendency to overfit. Through systematic benchmarking across various question answering datasets and LLMs, we show how our techniques can yield confidence scores that provide substantial improvements in fine-grained measures of both calibration and accuracy compared to existing methods.} }
Endnote
%0 Conference Paper %T Multicalibration for Confidence Scoring in LLMs %A Gianluca Detommaso %A Martin Andres Bertran %A Riccardo Fogliato %A Aaron Roth %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-detommaso24a %I PMLR %P 10624--10641 %U https://proceedings.mlr.press/v235/detommaso24a.html %V 235 %X This paper proposes the use of "multicalibration": to yield interpretable and reliable confidence scores for outputs generated by large language models (LLMs). Multicalibration asks for calibration not just marginally, but simultaneously across various intersecting groupings of the data. We show how to form groupings for prompt/completion pairs that are correlated with the probability of correctness via two techniques: clustering within an embedding space, and "self-annotation" - querying the LLM by asking it various yes-or-no questions about the prompt. We also develop novel variants of multicalibration algorithms that offer performance improvements by reducing their tendency to overfit. Through systematic benchmarking across various question answering datasets and LLMs, we show how our techniques can yield confidence scores that provide substantial improvements in fine-grained measures of both calibration and accuracy compared to existing methods.
APA
Detommaso, G., Bertran, M.A., Fogliato, R. & Roth, A.. (2024). Multicalibration for Confidence Scoring in LLMs. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:10624-10641 Available from https://proceedings.mlr.press/v235/detommaso24a.html.

Related Material