Adaptive Conformal Inference by Betting

Aleksandr Podkopaev, Dong Xu, Kuang-Chih Lee
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:40886-40907, 2024.

Abstract

Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeability, a condition which is often not met in real-world scenarios. In this paper, we consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. A notable shortcoming of such approaches is in their explicit dependence on and sensitivity to the choice of the learning rates. In this paper, we propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques. We prove that our method controls long-term miscoverage frequency at a nominal level and demonstrate its convincing empirical performance without any need of performing cumbersome parameter tuning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-podkopaev24a, title = {Adaptive Conformal Inference by Betting}, author = {Podkopaev, Aleksandr and Xu, Dong and Lee, Kuang-Chih}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {40886--40907}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/podkopaev24a/podkopaev24a.pdf}, url = {https://proceedings.mlr.press/v235/podkopaev24a.html}, abstract = {Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeability, a condition which is often not met in real-world scenarios. In this paper, we consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. A notable shortcoming of such approaches is in their explicit dependence on and sensitivity to the choice of the learning rates. In this paper, we propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques. We prove that our method controls long-term miscoverage frequency at a nominal level and demonstrate its convincing empirical performance without any need of performing cumbersome parameter tuning.} }
Endnote
%0 Conference Paper %T Adaptive Conformal Inference by Betting %A Aleksandr Podkopaev %A Dong Xu %A Kuang-Chih Lee %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-podkopaev24a %I PMLR %P 40886--40907 %U https://proceedings.mlr.press/v235/podkopaev24a.html %V 235 %X Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeability, a condition which is often not met in real-world scenarios. In this paper, we consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. A notable shortcoming of such approaches is in their explicit dependence on and sensitivity to the choice of the learning rates. In this paper, we propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques. We prove that our method controls long-term miscoverage frequency at a nominal level and demonstrate its convincing empirical performance without any need of performing cumbersome parameter tuning.
APA
Podkopaev, A., Xu, D. & Lee, K.. (2024). Adaptive Conformal Inference by Betting. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:40886-40907 Available from https://proceedings.mlr.press/v235/podkopaev24a.html.

Related Material