The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes

Marina Mancoridis, Zoe Hitzig
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:86097-86142, 2026.

Abstract

Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model applies relevant concepts the same way when it generates an output and later evaluates that output. We propose a new measure, generator–evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts. We find, first, that there is substantial variation in self-consistency. Second, we find that in a clinical setting with physician-validated mistakes (Proniakin et al., 2025), across models, those with higher self-consistency are linked to greater vulnerability to mistakes. Thus, even when models consistently apply concepts they may not be safe to deploy. This is evidence of a consistency dilemma in LLMs: self-consistency is operationally useful, but models that are more consistent are also more prone to mistakes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-mancoridis26a, title = {The Consistency Dilemma in {LLM}s: Generator-Evaluator Agreement and Vulnerability to Mistakes}, author = {Mancoridis, Marina and Hitzig, Zoe}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {86097--86142}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/mancoridis26a/mancoridis26a.pdf}, url = {https://proceedings.mlr.press/v306/mancoridis26a.html}, abstract = {Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model applies relevant concepts the same way when it generates an output and later evaluates that output. We propose a new measure, generator–evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts. We find, first, that there is substantial variation in self-consistency. Second, we find that in a clinical setting with physician-validated mistakes (Proniakin et al., 2025), across models, those with higher self-consistency are linked to greater vulnerability to mistakes. Thus, even when models consistently apply concepts they may not be safe to deploy. This is evidence of a consistency dilemma in LLMs: self-consistency is operationally useful, but models that are more consistent are also more prone to mistakes.} }
Endnote
%0 Conference Paper %T The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes %A Marina Mancoridis %A Zoe Hitzig %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-mancoridis26a %I PMLR %P 86097--86142 %U https://proceedings.mlr.press/v306/mancoridis26a.html %V 306 %X Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model applies relevant concepts the same way when it generates an output and later evaluates that output. We propose a new measure, generator–evaluator self-consistency, to test this assumption directly and apply it to 10 frontier models across 491 concepts. We find, first, that there is substantial variation in self-consistency. Second, we find that in a clinical setting with physician-validated mistakes (Proniakin et al., 2025), across models, those with higher self-consistency are linked to greater vulnerability to mistakes. Thus, even when models consistently apply concepts they may not be safe to deploy. This is evidence of a consistency dilemma in LLMs: self-consistency is operationally useful, but models that are more consistent are also more prone to mistakes.
APA
Mancoridis, M. & Hitzig, Z.. (2026). The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:86097-86142 Available from https://proceedings.mlr.press/v306/mancoridis26a.html.

Related Material