Rethinking Calibration for Early-Exit Neural Networks

Piotr Kubaty, Filip Szatkowski, Grzegorz Choczyński, Eric Nalisnick, Bartosz Wójcik
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:61372-61389, 2026.

Abstract

Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confidence thresholds for exiting, and consequently, improving classifier calibration is widely assumed to improve performance. In this work, we challenge this assumption and show that calibration alone is not sufficient for EENNs to exploit adaptive computation. To address this insufficiency, we introduce Early-Exit Failure Prediction (EEFP), which accounts for both prediction correctness and the cost of further computation. We also propose a lightweight, EEFP-motivated procedure to improve the intermediate classifiers, which can directly replace calibration in EENNs. Extensive experiments demonstrate that our approach achieves superior cost-accuracy trade-offs compared to calibration, and EEFP more reliably reflects overall EENN performance. Our code is available at https://github.com/gmum/rethinking-calibration-for-eenns.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-kubaty26a, title = {Rethinking Calibration for Early-Exit Neural Networks}, author = {Kubaty, Piotr and Szatkowski, Filip and Choczy\'{n}ski, Grzegorz and Nalisnick, Eric and W\'{o}jcik, Bartosz}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {61372--61389}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/kubaty26a/kubaty26a.pdf}, url = {https://proceedings.mlr.press/v306/kubaty26a.html}, abstract = {Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confidence thresholds for exiting, and consequently, improving classifier calibration is widely assumed to improve performance. In this work, we challenge this assumption and show that calibration alone is not sufficient for EENNs to exploit adaptive computation. To address this insufficiency, we introduce Early-Exit Failure Prediction (EEFP), which accounts for both prediction correctness and the cost of further computation. We also propose a lightweight, EEFP-motivated procedure to improve the intermediate classifiers, which can directly replace calibration in EENNs. Extensive experiments demonstrate that our approach achieves superior cost-accuracy trade-offs compared to calibration, and EEFP more reliably reflects overall EENN performance. Our code is available at https://github.com/gmum/rethinking-calibration-for-eenns.} }
Endnote
%0 Conference Paper %T Rethinking Calibration for Early-Exit Neural Networks %A Piotr Kubaty %A Filip Szatkowski %A Grzegorz Choczyński %A Eric Nalisnick %A Bartosz Wójcik %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-kubaty26a %I PMLR %P 61372--61389 %U https://proceedings.mlr.press/v306/kubaty26a.html %V 306 %X Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confidence thresholds for exiting, and consequently, improving classifier calibration is widely assumed to improve performance. In this work, we challenge this assumption and show that calibration alone is not sufficient for EENNs to exploit adaptive computation. To address this insufficiency, we introduce Early-Exit Failure Prediction (EEFP), which accounts for both prediction correctness and the cost of further computation. We also propose a lightweight, EEFP-motivated procedure to improve the intermediate classifiers, which can directly replace calibration in EENNs. Extensive experiments demonstrate that our approach achieves superior cost-accuracy trade-offs compared to calibration, and EEFP more reliably reflects overall EENN performance. Our code is available at https://github.com/gmum/rethinking-calibration-for-eenns.
APA
Kubaty, P., Szatkowski, F., Choczyński, G., Nalisnick, E. & Wójcik, B.. (2026). Rethinking Calibration for Early-Exit Neural Networks. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:61372-61389 Available from https://proceedings.mlr.press/v306/kubaty26a.html.

Related Material