Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning

Hen Davidov, Nachshon Cohen, Oren Kalinsky, Yaron Fairstein, Guy Kushilevitz, Ram Yazdi, Patrick Rebeschini
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:23121-23158, 2026.

Abstract

Large language models (LLMs) using chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning tasks support our theory and demonstrate improved selective accuracy over existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-davidov26a, title = {Knowing When to Quit: A Principled Framework for Dynamic Abstention in {LLM} Reasoning}, author = {Davidov, Hen and Cohen, Nachshon and Kalinsky, Oren and Fairstein, Yaron and Kushilevitz, Guy and Yazdi, Ram and Rebeschini, Patrick}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {23121--23158}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/davidov26a/davidov26a.pdf}, url = {https://proceedings.mlr.press/v306/davidov26a.html}, abstract = {Large language models (LLMs) using chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning tasks support our theory and demonstrate improved selective accuracy over existing methods.} }
Endnote
%0 Conference Paper %T Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning %A Hen Davidov %A Nachshon Cohen %A Oren Kalinsky %A Yaron Fairstein %A Guy Kushilevitz %A Ram Yazdi %A Patrick Rebeschini %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-davidov26a %I PMLR %P 23121--23158 %U https://proceedings.mlr.press/v306/davidov26a.html %V 306 %X Large language models (LLMs) using chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning tasks support our theory and demonstrate improved selective accuracy over existing methods.
APA
Davidov, H., Cohen, N., Kalinsky, O., Fairstein, Y., Kushilevitz, G., Yazdi, R. & Rebeschini, P.. (2026). Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:23121-23158 Available from https://proceedings.mlr.press/v306/davidov26a.html.

Related Material