Latent Reasoning in TRMs is Secretly a Policy Improvement Operator

Arip Asadulaev, Rayan Banerjee, Fakhri Karray, Martin Takáč
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:4035-4047, 2026.

Abstract

Recently, small models with latent recursion have obtained promising results on complex reasoning tasks. These results are typically explained by the theory that such recursion increases a network’s depth, allowing it to compactly emulate the capacity of larger models. However, the performance of recursively added layers remains behind the capabilities of one-pass models with the same feed-forward depth. This means that in the looped version, not every recursive step effectively contributes to depth. This raises the question: when and why does latent reasoning improve performance, and when does it result in dead compute? In our work, we analyze the algorithms that latent reasoning provides answer to this question. We show that latent reasoning can be formalized as a classifier-free guidance and policy improvement algorithm. Building on these insights, we propose to use a training schemes from RL and diffusion methods for latent reasoning modles. Using the Tiny Recursive Model as our testbed, we show that with our modifications we can avoid dead compute steps and reduce the total number of forward passes by 18$\times$ while maintaining performance. Broadly speaking, we show how a policy improvement perspective on recursive steps can explain model behavior and provide insights for further improvements.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-asadulaev26a, title = {Latent Reasoning in {TRM}s is Secretly a Policy Improvement Operator}, author = {Asadulaev, Arip and Banerjee, Rayan and Karray, Fakhri and Tak\'{a}\v{c}, Martin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {4035--4047}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/asadulaev26a/asadulaev26a.pdf}, url = {https://proceedings.mlr.press/v306/asadulaev26a.html}, abstract = {Recently, small models with latent recursion have obtained promising results on complex reasoning tasks. These results are typically explained by the theory that such recursion increases a network’s depth, allowing it to compactly emulate the capacity of larger models. However, the performance of recursively added layers remains behind the capabilities of one-pass models with the same feed-forward depth. This means that in the looped version, not every recursive step effectively contributes to depth. This raises the question: when and why does latent reasoning improve performance, and when does it result in dead compute? In our work, we analyze the algorithms that latent reasoning provides answer to this question. We show that latent reasoning can be formalized as a classifier-free guidance and policy improvement algorithm. Building on these insights, we propose to use a training schemes from RL and diffusion methods for latent reasoning modles. Using the Tiny Recursive Model as our testbed, we show that with our modifications we can avoid dead compute steps and reduce the total number of forward passes by 18$\times$ while maintaining performance. Broadly speaking, we show how a policy improvement perspective on recursive steps can explain model behavior and provide insights for further improvements.} }
Endnote
%0 Conference Paper %T Latent Reasoning in TRMs is Secretly a Policy Improvement Operator %A Arip Asadulaev %A Rayan Banerjee %A Fakhri Karray %A Martin Takáč %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-asadulaev26a %I PMLR %P 4035--4047 %U https://proceedings.mlr.press/v306/asadulaev26a.html %V 306 %X Recently, small models with latent recursion have obtained promising results on complex reasoning tasks. These results are typically explained by the theory that such recursion increases a network’s depth, allowing it to compactly emulate the capacity of larger models. However, the performance of recursively added layers remains behind the capabilities of one-pass models with the same feed-forward depth. This means that in the looped version, not every recursive step effectively contributes to depth. This raises the question: when and why does latent reasoning improve performance, and when does it result in dead compute? In our work, we analyze the algorithms that latent reasoning provides answer to this question. We show that latent reasoning can be formalized as a classifier-free guidance and policy improvement algorithm. Building on these insights, we propose to use a training schemes from RL and diffusion methods for latent reasoning modles. Using the Tiny Recursive Model as our testbed, we show that with our modifications we can avoid dead compute steps and reduce the total number of forward passes by 18$\times$ while maintaining performance. Broadly speaking, we show how a policy improvement perspective on recursive steps can explain model behavior and provide insights for further improvements.
APA
Asadulaev, A., Banerjee, R., Karray, F. & Takáč, M.. (2026). Latent Reasoning in TRMs is Secretly a Policy Improvement Operator. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:4035-4047 Available from https://proceedings.mlr.press/v306/asadulaev26a.html.

Related Material