Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning

Renos Zabounidis, Aditya Golatkar, Michael Kleinman, Alessandro Achille, Wei Xia, Stefano Soatto
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:152695-152715, 2026.

Abstract

We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning and larger models. Re-FORC enables: 1) early stopping of unpromising reasoning chains, reducing compute by up to 26% compared to fixed-budget cutoffs, while maintaining accuracy, 2) optimized model and thinking length selection that outperforms the largest model alone— reaching 1.7 percentage points higher peak accuracy while needing up to 12% less compute to match the largest model’s accuracy, 3) adaptive test-time scaling, which increases accuracy by 9.9 percentage points (on average at maximum compute) over confidence-based baselines. Re-FORC allows dynamic reasoning with length control via cost-per-token thresholds while estimating computation time upfront.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zabounidis26a, title = {Re-{FORC}: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning}, author = {Zabounidis, Renos and Golatkar, Aditya and Kleinman, Michael and Achille, Alessandro and Xia, Wei and Soatto, Stefano}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {152695--152715}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zabounidis26a/zabounidis26a.pdf}, url = {https://proceedings.mlr.press/v306/zabounidis26a.html}, abstract = {We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning and larger models. Re-FORC enables: 1) early stopping of unpromising reasoning chains, reducing compute by up to 26% compared to fixed-budget cutoffs, while maintaining accuracy, 2) optimized model and thinking length selection that outperforms the largest model alone— reaching 1.7 percentage points higher peak accuracy while needing up to 12% less compute to match the largest model’s accuracy, 3) adaptive test-time scaling, which increases accuracy by 9.9 percentage points (on average at maximum compute) over confidence-based baselines. Re-FORC allows dynamic reasoning with length control via cost-per-token thresholds while estimating computation time upfront.} }
Endnote
%0 Conference Paper %T Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning %A Renos Zabounidis %A Aditya Golatkar %A Michael Kleinman %A Alessandro Achille %A Wei Xia %A Stefano Soatto %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zabounidis26a %I PMLR %P 152695--152715 %U https://proceedings.mlr.press/v306/zabounidis26a.html %V 306 %X We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning and larger models. Re-FORC enables: 1) early stopping of unpromising reasoning chains, reducing compute by up to 26% compared to fixed-budget cutoffs, while maintaining accuracy, 2) optimized model and thinking length selection that outperforms the largest model alone— reaching 1.7 percentage points higher peak accuracy while needing up to 12% less compute to match the largest model’s accuracy, 3) adaptive test-time scaling, which increases accuracy by 9.9 percentage points (on average at maximum compute) over confidence-based baselines. Re-FORC allows dynamic reasoning with length control via cost-per-token thresholds while estimating computation time upfront.
APA
Zabounidis, R., Golatkar, A., Kleinman, M., Achille, A., Xia, W. & Soatto, S.. (2026). Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:152695-152715 Available from https://proceedings.mlr.press/v306/zabounidis26a.html.

Related Material