From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework

Yufei Chen, Yao Wang, Haibin Zhang, Hualin Zhou, Tao Gu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:17308-17328, 2026.

Abstract

Retrieval-augmented generation (RAG) systems suffer from a fundamental functional misalignment where retrievers optimize for semantic relevance, often recalling documents with high background utility but factually erroneous answer spans that generators blindly adopt as cognitive shortcuts. To resolve this, we propose the collaborative Critic-Reasoner framework that shifts robustness control from coarse-grained filtering to fine-grained cognitive decoupling. We disentangle the generation process into two serialized roles by deploying a Critic to perform surgical evidence purification through identifying and masking misleading entities while preserving supportive background context, followed by a Reasoner that switches from rote extraction to deductive reasoning based on the residual evidence. We operationalize this framework via a two-stage alignment strategy combining supervised fine-tuning (SFT) with path-aware direct preference optimization (DPO) to enforce strict behavioral synergy. Experimental results on adversarial benchmarks such as ConFiQA demonstrate that our method significantly outperforms baselines, achieving a 25.99% accuracy gain in conflicting scenarios and effectively resolving the trust bias dilemma in real-world RAG.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26et, title = {From Extraction to Deduction: Resolving Functional Misalignment in {RAG} via a Collaborative Critic-Reasoner Framework}, author = {Chen, Yufei and Wang, Yao and Zhang, Haibin and Zhou, Hualin and Gu, Tao}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {17308--17328}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26et/chen26et.pdf}, url = {https://proceedings.mlr.press/v306/chen26et.html}, abstract = {Retrieval-augmented generation (RAG) systems suffer from a fundamental functional misalignment where retrievers optimize for semantic relevance, often recalling documents with high background utility but factually erroneous answer spans that generators blindly adopt as cognitive shortcuts. To resolve this, we propose the collaborative Critic-Reasoner framework that shifts robustness control from coarse-grained filtering to fine-grained cognitive decoupling. We disentangle the generation process into two serialized roles by deploying a Critic to perform surgical evidence purification through identifying and masking misleading entities while preserving supportive background context, followed by a Reasoner that switches from rote extraction to deductive reasoning based on the residual evidence. We operationalize this framework via a two-stage alignment strategy combining supervised fine-tuning (SFT) with path-aware direct preference optimization (DPO) to enforce strict behavioral synergy. Experimental results on adversarial benchmarks such as ConFiQA demonstrate that our method significantly outperforms baselines, achieving a 25.99% accuracy gain in conflicting scenarios and effectively resolving the trust bias dilemma in real-world RAG.} }
Endnote
%0 Conference Paper %T From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework %A Yufei Chen %A Yao Wang %A Haibin Zhang %A Hualin Zhou %A Tao Gu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26et %I PMLR %P 17308--17328 %U https://proceedings.mlr.press/v306/chen26et.html %V 306 %X Retrieval-augmented generation (RAG) systems suffer from a fundamental functional misalignment where retrievers optimize for semantic relevance, often recalling documents with high background utility but factually erroneous answer spans that generators blindly adopt as cognitive shortcuts. To resolve this, we propose the collaborative Critic-Reasoner framework that shifts robustness control from coarse-grained filtering to fine-grained cognitive decoupling. We disentangle the generation process into two serialized roles by deploying a Critic to perform surgical evidence purification through identifying and masking misleading entities while preserving supportive background context, followed by a Reasoner that switches from rote extraction to deductive reasoning based on the residual evidence. We operationalize this framework via a two-stage alignment strategy combining supervised fine-tuning (SFT) with path-aware direct preference optimization (DPO) to enforce strict behavioral synergy. Experimental results on adversarial benchmarks such as ConFiQA demonstrate that our method significantly outperforms baselines, achieving a 25.99% accuracy gain in conflicting scenarios and effectively resolving the trust bias dilemma in real-world RAG.
APA
Chen, Y., Wang, Y., Zhang, H., Zhou, H. & Gu, T.. (2026). From Extraction to Deduction: Resolving Functional Misalignment in RAG via a Collaborative Critic-Reasoner Framework. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:17308-17328 Available from https://proceedings.mlr.press/v306/chen26et.html.

Related Material