Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents

Ruihan Yang, Fanghua Ye, Xiang Wei, Ruoqing Zhao, Kang Luo, Xinbo Xu, Bo Zhao, Ruotian Ma, Shanyi Wang, Zhaopeng Tu, Xiaolong Li, Deqing Yang, Liefeng Bo
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:146766-146790, 2026.

Abstract

Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CogSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-yang26cz, title = {Think Fast and Slow: Step-Level Cognitive Depth Adaptation for {LLM} Agents}, author = {Yang, Ruihan and Ye, Fanghua and Wei, Xiang and Zhao, Ruoqing and Luo, Kang and Xu, Xinbo and Zhao, Bo and Ma, Ruotian and Wang, Shanyi and Tu, Zhaopeng and Li, Xiaolong and Yang, Deqing and Bo, Liefeng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {146766--146790}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/yang26cz/yang26cz.pdf}, url = {https://proceedings.mlr.press/v306/yang26cz.html}, abstract = {Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CogSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency.} }
Endnote
%0 Conference Paper %T Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents %A Ruihan Yang %A Fanghua Ye %A Xiang Wei %A Ruoqing Zhao %A Kang Luo %A Xinbo Xu %A Bo Zhao %A Ruotian Ma %A Shanyi Wang %A Zhaopeng Tu %A Xiaolong Li %A Deqing Yang %A Liefeng Bo %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-yang26cz %I PMLR %P 146766--146790 %U https://proceedings.mlr.press/v306/yang26cz.html %V 306 %X Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CogSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency.
APA
Yang, R., Ye, F., Wei, X., Zhao, R., Luo, K., Xu, X., Zhao, B., Ma, R., Wang, S., Tu, Z., Li, X., Yang, D. & Bo, L.. (2026). Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:146766-146790 Available from https://proceedings.mlr.press/v306/yang26cz.html.

Related Material