Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads

Artem Vazhentsev, Lyudmila Rvanova, Gleb Kuzmin, Ekaterina Fadeeva, Ivan Lazichny, Alexander Panchenko, Maxim Panov, Mrinmaya Sachan, Preslav Nakov, Timothy Baldwin, Artem Shelmanov
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:123596-123627, 2026.

Abstract

While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ) offers a promising way to mitigate this issue, but most existing methods are computationally intensive and/or require supervision. In this work, we propose Recurrent Attention-based Uncertainty Quantification (RAUQ), an unsupervised and efficient framework for identifying hallucinations. The method leverages an observation about transformer attention behavior: when incorrect information is generated, certain "uncertainty-aware" attention heads tend to reduce their focus on preceding tokens. RAUQ automatically detects these attention heads and combines their activation patterns with token-level confidence measures in a recurrent scheme, producing a sequence-level uncertainty estimate in just a single forward pass. Through experiments on twelve datasets spanning question answering, summarization, and translation across nine different LLMs, we show that RAUQ consistently outperforms state-of-the-art UQ baselines. Importantly, it incurs minimal overhead, requiring less than 1% additional computation. Since it requires neither labeled data nor extensive parameter tuning, RAUQ serves as a lightweight, plug-and-play solution for real-time hallucination detection in white-box LLMs.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-vazhentsev26a, title = {Efficient Hallucination Detection for {LLM}s Using Uncertainty-Aware Attention Heads}, author = {Vazhentsev, Artem and Rvanova, Lyudmila and Kuzmin, Gleb and Fadeeva, Ekaterina and Lazichny, Ivan and Panchenko, Alexander and Panov, Maxim and Sachan, Mrinmaya and Nakov, Preslav and Baldwin, Timothy and Shelmanov, Artem}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {123596--123627}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/vazhentsev26a/vazhentsev26a.pdf}, url = {https://proceedings.mlr.press/v306/vazhentsev26a.html}, abstract = {While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ) offers a promising way to mitigate this issue, but most existing methods are computationally intensive and/or require supervision. In this work, we propose Recurrent Attention-based Uncertainty Quantification (RAUQ), an unsupervised and efficient framework for identifying hallucinations. The method leverages an observation about transformer attention behavior: when incorrect information is generated, certain "uncertainty-aware" attention heads tend to reduce their focus on preceding tokens. RAUQ automatically detects these attention heads and combines their activation patterns with token-level confidence measures in a recurrent scheme, producing a sequence-level uncertainty estimate in just a single forward pass. Through experiments on twelve datasets spanning question answering, summarization, and translation across nine different LLMs, we show that RAUQ consistently outperforms state-of-the-art UQ baselines. Importantly, it incurs minimal overhead, requiring less than 1% additional computation. Since it requires neither labeled data nor extensive parameter tuning, RAUQ serves as a lightweight, plug-and-play solution for real-time hallucination detection in white-box LLMs.} }
Endnote
%0 Conference Paper %T Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads %A Artem Vazhentsev %A Lyudmila Rvanova %A Gleb Kuzmin %A Ekaterina Fadeeva %A Ivan Lazichny %A Alexander Panchenko %A Maxim Panov %A Mrinmaya Sachan %A Preslav Nakov %A Timothy Baldwin %A Artem Shelmanov %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-vazhentsev26a %I PMLR %P 123596--123627 %U https://proceedings.mlr.press/v306/vazhentsev26a.html %V 306 %X While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ) offers a promising way to mitigate this issue, but most existing methods are computationally intensive and/or require supervision. In this work, we propose Recurrent Attention-based Uncertainty Quantification (RAUQ), an unsupervised and efficient framework for identifying hallucinations. The method leverages an observation about transformer attention behavior: when incorrect information is generated, certain "uncertainty-aware" attention heads tend to reduce their focus on preceding tokens. RAUQ automatically detects these attention heads and combines their activation patterns with token-level confidence measures in a recurrent scheme, producing a sequence-level uncertainty estimate in just a single forward pass. Through experiments on twelve datasets spanning question answering, summarization, and translation across nine different LLMs, we show that RAUQ consistently outperforms state-of-the-art UQ baselines. Importantly, it incurs minimal overhead, requiring less than 1% additional computation. Since it requires neither labeled data nor extensive parameter tuning, RAUQ serves as a lightweight, plug-and-play solution for real-time hallucination detection in white-box LLMs.
APA
Vazhentsev, A., Rvanova, L., Kuzmin, G., Fadeeva, E., Lazichny, I., Panchenko, A., Panov, M., Sachan, M., Nakov, P., Baldwin, T. & Shelmanov, A.. (2026). Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:123596-123627 Available from https://proceedings.mlr.press/v306/vazhentsev26a.html.

Related Material