Cycle-of-Science: Reliable Reasoning through Counterfactual Verification for Agent Decision Making

Ruojie Zhang, Wencheng Zhu, Peiyuan Jiang, Dayong Zhu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:161035-161053, 2026.

Abstract

Large Language Models have significantly advanced autonomous agents through their sophisticated perception and execution capabilities. Despite effective, agents still struggle with robust decision-making due to passive learning from similar experiences that often confound correlation with causality. Inspired by the Scientific Method, we propose a Cycle-of-Science framework that autonomously explores potential causal pathways through an iterative loop of Hypothesis, Experiment, and Validation, enabling agents to identify truly effective causal dependencies. To be specific, we first leverage causal knowledge to guide the initial hypotheses generation. These hypotheses are then analyzed through experiments using counterfactual samples. Afterward, we perform causal analysis to quantify effects of interventions, deriving well-validated hypotheses for next agent steps. To train our policy, we further introduce a two-stage pipeline that integrates supervised fine-tuning with Counterfactual Preference Optimization, which constructs preference signals from intervention outcomes to reinforce validated reasoning chains. Experiments on benchmarks demonstrate that our method achieves superior performance over state-of-the-art approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zhang26jk, title = {Cycle-of-Science: Reliable Reasoning through Counterfactual Verification for Agent Decision Making}, author = {Zhang, Ruojie and Zhu, Wencheng and Jiang, Peiyuan and Zhu, Dayong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {161035--161053}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhang26jk/zhang26jk.pdf}, url = {https://proceedings.mlr.press/v306/zhang26jk.html}, abstract = {Large Language Models have significantly advanced autonomous agents through their sophisticated perception and execution capabilities. Despite effective, agents still struggle with robust decision-making due to passive learning from similar experiences that often confound correlation with causality. Inspired by the Scientific Method, we propose a Cycle-of-Science framework that autonomously explores potential causal pathways through an iterative loop of Hypothesis, Experiment, and Validation, enabling agents to identify truly effective causal dependencies. To be specific, we first leverage causal knowledge to guide the initial hypotheses generation. These hypotheses are then analyzed through experiments using counterfactual samples. Afterward, we perform causal analysis to quantify effects of interventions, deriving well-validated hypotheses for next agent steps. To train our policy, we further introduce a two-stage pipeline that integrates supervised fine-tuning with Counterfactual Preference Optimization, which constructs preference signals from intervention outcomes to reinforce validated reasoning chains. Experiments on benchmarks demonstrate that our method achieves superior performance over state-of-the-art approaches.} }
Endnote
%0 Conference Paper %T Cycle-of-Science: Reliable Reasoning through Counterfactual Verification for Agent Decision Making %A Ruojie Zhang %A Wencheng Zhu %A Peiyuan Jiang %A Dayong Zhu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhang26jk %I PMLR %P 161035--161053 %U https://proceedings.mlr.press/v306/zhang26jk.html %V 306 %X Large Language Models have significantly advanced autonomous agents through their sophisticated perception and execution capabilities. Despite effective, agents still struggle with robust decision-making due to passive learning from similar experiences that often confound correlation with causality. Inspired by the Scientific Method, we propose a Cycle-of-Science framework that autonomously explores potential causal pathways through an iterative loop of Hypothesis, Experiment, and Validation, enabling agents to identify truly effective causal dependencies. To be specific, we first leverage causal knowledge to guide the initial hypotheses generation. These hypotheses are then analyzed through experiments using counterfactual samples. Afterward, we perform causal analysis to quantify effects of interventions, deriving well-validated hypotheses for next agent steps. To train our policy, we further introduce a two-stage pipeline that integrates supervised fine-tuning with Counterfactual Preference Optimization, which constructs preference signals from intervention outcomes to reinforce validated reasoning chains. Experiments on benchmarks demonstrate that our method achieves superior performance over state-of-the-art approaches.
APA
Zhang, R., Zhu, W., Jiang, P. & Zhu, D.. (2026). Cycle-of-Science: Reliable Reasoning through Counterfactual Verification for Agent Decision Making. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:161035-161053 Available from https://proceedings.mlr.press/v306/zhang26jk.html.

Related Material