When Is Conformal Coverage Free? Switching Thresholds for Predict-then-Optimize

Chandra Dronavajjala
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:548-575, 2026.

Abstract

Machine learning increasingly drives operational decisions (dispatching power plants, routing vehicles, allocating medical supplies), yet its forecasts are uncertain. Conformal prediction turns a forecast into a calibrated uncertainty set with distribution-free guarantees, and a robust optimizer can then hedge its decision against that set. Before adopting this, a practitioner wants to know: will maintaining the uncertainty set actually change the decision the system makes, or will it leave the decision untouched and merely add an audit trail? We answer with a single quantity, the switching threshold: the point at which calibrated uncertainty begins to alter the chosen action. While the uncertainty stays below this threshold, coverage is free, and the system gains calibrated, auditable uncertainty without changing what it does. We show how to read this threshold from the structure of the decision problem, characterize the fluctuations of the online quantile that decide which side of it a problem falls on, and reduce these to a single pre-deployment safety margin that labels a problem as free, borderline, or costly. The label is set by the decision problem’s structure rather than the choice of predictor. Across real benchmarks in energy, routing, public health, and logistics, dispatching power under real market prices is free (coverage never changes which generators run and adds no cost), whereas routing on a dense city map changes the route about half the time. The certificate stays reliable at city scale, on road networks with tens of thousands of streets, and even when the predictor is a vision model reading images. The result is a simple test for whether adding conformal coverage will cost a decision system anything.

Cite this Paper


BibTeX
@InProceedings{pmlr-v329-dronavajjala26a, title = {When Is Conformal Coverage Free? Switching Thresholds for Predict-then-Optimize}, author = {Dronavajjala, Chandra}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {548--575}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/dronavajjala26a/dronavajjala26a.pdf}, url = {https://proceedings.mlr.press/v329/dronavajjala26a.html}, abstract = {Machine learning increasingly drives operational decisions (dispatching power plants, routing vehicles, allocating medical supplies), yet its forecasts are uncertain. Conformal prediction turns a forecast into a calibrated uncertainty set with distribution-free guarantees, and a robust optimizer can then hedge its decision against that set. Before adopting this, a practitioner wants to know: will maintaining the uncertainty set actually change the decision the system makes, or will it leave the decision untouched and merely add an audit trail? We answer with a single quantity, the switching threshold: the point at which calibrated uncertainty begins to alter the chosen action. While the uncertainty stays below this threshold, coverage is free, and the system gains calibrated, auditable uncertainty without changing what it does. We show how to read this threshold from the structure of the decision problem, characterize the fluctuations of the online quantile that decide which side of it a problem falls on, and reduce these to a single pre-deployment safety margin that labels a problem as free, borderline, or costly. The label is set by the decision problem’s structure rather than the choice of predictor. Across real benchmarks in energy, routing, public health, and logistics, dispatching power under real market prices is free (coverage never changes which generators run and adds no cost), whereas routing on a dense city map changes the route about half the time. The certificate stays reliable at city scale, on road networks with tens of thousands of streets, and even when the predictor is a vision model reading images. The result is a simple test for whether adding conformal coverage will cost a decision system anything.} }
Endnote
%0 Conference Paper %T When Is Conformal Coverage Free? Switching Thresholds for Predict-then-Optimize %A Chandra Dronavajjala %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-dronavajjala26a %I PMLR %P 548--575 %U https://proceedings.mlr.press/v329/dronavajjala26a.html %V 329 %X Machine learning increasingly drives operational decisions (dispatching power plants, routing vehicles, allocating medical supplies), yet its forecasts are uncertain. Conformal prediction turns a forecast into a calibrated uncertainty set with distribution-free guarantees, and a robust optimizer can then hedge its decision against that set. Before adopting this, a practitioner wants to know: will maintaining the uncertainty set actually change the decision the system makes, or will it leave the decision untouched and merely add an audit trail? We answer with a single quantity, the switching threshold: the point at which calibrated uncertainty begins to alter the chosen action. While the uncertainty stays below this threshold, coverage is free, and the system gains calibrated, auditable uncertainty without changing what it does. We show how to read this threshold from the structure of the decision problem, characterize the fluctuations of the online quantile that decide which side of it a problem falls on, and reduce these to a single pre-deployment safety margin that labels a problem as free, borderline, or costly. The label is set by the decision problem’s structure rather than the choice of predictor. Across real benchmarks in energy, routing, public health, and logistics, dispatching power under real market prices is free (coverage never changes which generators run and adds no cost), whereas routing on a dense city map changes the route about half the time. The certificate stays reliable at city scale, on road networks with tens of thousands of streets, and even when the predictor is a vision model reading images. The result is a simple test for whether adding conformal coverage will cost a decision system anything.
APA
Dronavajjala, C.. (2026). When Is Conformal Coverage Free? Switching Thresholds for Predict-then-Optimize. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:548-575 Available from https://proceedings.mlr.press/v329/dronavajjala26a.html.

Related Material