A Formal Comparison Between Chain of Thought and Latent Thought

Kevin Xu, Issei Sato
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:141657-141685, 2026.

Abstract

Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that latent thought admits efficient parallel computation, in contrast to the inherently sequential nature of CoT. Conversely, CoT enables approximate counting through stochastic decoding. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-xu26ar, title = {A Formal Comparison Between Chain of Thought and Latent Thought}, author = {Xu, Kevin and Sato, Issei}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {141657--141685}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/xu26ar/xu26ar.pdf}, url = {https://proceedings.mlr.press/v306/xu26ar.html}, abstract = {Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that latent thought admits efficient parallel computation, in contrast to the inherently sequential nature of CoT. Conversely, CoT enables approximate counting through stochastic decoding. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms.} }
Endnote
%0 Conference Paper %T A Formal Comparison Between Chain of Thought and Latent Thought %A Kevin Xu %A Issei Sato %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-xu26ar %I PMLR %P 141657--141685 %U https://proceedings.mlr.press/v306/xu26ar.html %V 306 %X Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that latent thought admits efficient parallel computation, in contrast to the inherently sequential nature of CoT. Conversely, CoT enables approximate counting through stochastic decoding. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms.
APA
Xu, K. & Sato, I.. (2026). A Formal Comparison Between Chain of Thought and Latent Thought. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:141657-141685 Available from https://proceedings.mlr.press/v306/xu26ar.html.

Related Material