From Hints to Answers: Uncertainty-Aware LLM-Guided Retrieval for Multi-Hop Question Answering

Mahdiyar Ali Akbar Alavi, Bita Azad, Julien Serbanescu, Fattane Zarrinkalam, Faezeh Ensan
Proceedings of the The 39th Canadian Conference on Artificial Intelligence, PMLR 318:1060-1067, 2026.

Abstract

We propose Generate-Retrieve-Generate (GReG), a training-free pipeline for multi-hop open-domain question answering. GReG uses a strong LLM to generate multiple long-form “hints” that expose implicit intermediate facts in the question, and uses the selected hint as a retrieval query for gathering supporting evidence. To choose among candidate hints, we introduce an uncertainty-aware selection method, which favors lower-entropy generations. By improving retrieval quality, GReG enables a smaller, cost-efficient answer generator to answer complex multi-hop questions more accurately. Experiments on HotpotQA and 2WikiMultihopQA show that GReG achieves state-of-the-art performance under identical retrieval and generation settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v318-alavi26a, title = {From Hints to Answers: Uncertainty-Aware LLM-Guided Retrieval for Multi-Hop Question Answering}, author = {Alavi, Mahdiyar Ali Akbar and Azad, Bita and Serbanescu, Julien and Zarrinkalam, Fattane and Ensan, Faezeh}, booktitle = {Proceedings of the The 39th Canadian Conference on Artificial Intelligence}, pages = {1060--1067}, year = {2026}, editor = {Bouzar-Benlabiod, Lydia and Leung, Carson}, volume = {318}, series = {Proceedings of Machine Learning Research}, month = {25--29 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v318/main/assets/alavi26a/alavi26a.pdf}, url = {https://proceedings.mlr.press/v318/alavi26a.html}, abstract = {We propose Generate-Retrieve-Generate (GReG), a training-free pipeline for multi-hop open-domain question answering. GReG uses a strong LLM to generate multiple long-form “hints” that expose implicit intermediate facts in the question, and uses the selected hint as a retrieval query for gathering supporting evidence. To choose among candidate hints, we introduce an uncertainty-aware selection method, which favors lower-entropy generations. By improving retrieval quality, GReG enables a smaller, cost-efficient answer generator to answer complex multi-hop questions more accurately. Experiments on HotpotQA and 2WikiMultihopQA show that GReG achieves state-of-the-art performance under identical retrieval and generation settings.} }
Endnote
%0 Conference Paper %T From Hints to Answers: Uncertainty-Aware LLM-Guided Retrieval for Multi-Hop Question Answering %A Mahdiyar Ali Akbar Alavi %A Bita Azad %A Julien Serbanescu %A Fattane Zarrinkalam %A Faezeh Ensan %B Proceedings of the The 39th Canadian Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Lydia Bouzar-Benlabiod %E Carson Leung %F pmlr-v318-alavi26a %I PMLR %P 1060--1067 %U https://proceedings.mlr.press/v318/alavi26a.html %V 318 %X We propose Generate-Retrieve-Generate (GReG), a training-free pipeline for multi-hop open-domain question answering. GReG uses a strong LLM to generate multiple long-form “hints” that expose implicit intermediate facts in the question, and uses the selected hint as a retrieval query for gathering supporting evidence. To choose among candidate hints, we introduce an uncertainty-aware selection method, which favors lower-entropy generations. By improving retrieval quality, GReG enables a smaller, cost-efficient answer generator to answer complex multi-hop questions more accurately. Experiments on HotpotQA and 2WikiMultihopQA show that GReG achieves state-of-the-art performance under identical retrieval and generation settings.
APA
Alavi, M.A.A., Azad, B., Serbanescu, J., Zarrinkalam, F. & Ensan, F.. (2026). From Hints to Answers: Uncertainty-Aware LLM-Guided Retrieval for Multi-Hop Question Answering. Proceedings of the The 39th Canadian Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 318:1060-1067 Available from https://proceedings.mlr.press/v318/alavi26a.html.

Related Material