IDCR: Information-Directed Conformal Retrieval

Manas Nanivadekar, Jatin Khanijoan, Swayam Kothekar, Mohd Amaan Khan
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:4712-4732, 2026.

Abstract

Retrieval-augmented prediction systems select documents by semantic similarity, ignoring their effect on downstream predictive uncertainty. In high-stakes domains such as clinical diagnosis, this can yield overconfident or needlessly imprecise predictions. We propose Information-Directed Conformal Retrieval, a framework that selects documents to minimize the volume of conformal prediction sets while preserving distribution-free coverage. Modeling each document as a {Bayesian} precision update, we show that minimizing conformal volume is exactly equivalent to maximizing a log-determinant objective, and prove this objective monotone submodular, so greedy selection inherits the constant-factor $(1-1/e)$ guarantee. A Document Interaction Tensor characterizes corpus-level interaction structure, and a lightweight marginal-gain-separation gate routes uncertain retrieval steps to lookahead search. On MIMIC-IV clinical diagnosis (275 admissions from 100 patients, 9273 PubMed abstracts), greedy retrieval attains a mean greedy-to-optimal ratio of 0.9999, produces conformal ellipsoids 4.8 times smaller than random retrieval and 5.8 times smaller than cosine retrieval while retaining the distribution-free coverage guarantee, and two-step lookahead closes 77 percent of the remaining gap. Gains generalize to the SciQ and GoEmotions benchmarks, and on GoEmotions and LexGLUE the method yields tighter posterior uncertainty than learned and uncertainty-aware retrieval baselines. Complex multi-morbid patients benefit most, with twice the synergy-gap rate, precisely where prediction is hardest.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-nanivadekar26a, title = {{IDCR}: Information-Directed Conformal Retrieval}, author = {Nanivadekar, Manas and Khanijoan, Jatin and Kothekar, Swayam and Khan, Mohd Amaan}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {4712--4732}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/nanivadekar26a/nanivadekar26a.pdf}, url = {https://proceedings.mlr.press/v337/nanivadekar26a.html}, abstract = {Retrieval-augmented prediction systems select documents by semantic similarity, ignoring their effect on downstream predictive uncertainty. In high-stakes domains such as clinical diagnosis, this can yield overconfident or needlessly imprecise predictions. We propose Information-Directed Conformal Retrieval, a framework that selects documents to minimize the volume of conformal prediction sets while preserving distribution-free coverage. Modeling each document as a {Bayesian} precision update, we show that minimizing conformal volume is exactly equivalent to maximizing a log-determinant objective, and prove this objective monotone submodular, so greedy selection inherits the constant-factor $(1-1/e)$ guarantee. A Document Interaction Tensor characterizes corpus-level interaction structure, and a lightweight marginal-gain-separation gate routes uncertain retrieval steps to lookahead search. On MIMIC-IV clinical diagnosis (275 admissions from 100 patients, 9273 PubMed abstracts), greedy retrieval attains a mean greedy-to-optimal ratio of 0.9999, produces conformal ellipsoids 4.8 times smaller than random retrieval and 5.8 times smaller than cosine retrieval while retaining the distribution-free coverage guarantee, and two-step lookahead closes 77 percent of the remaining gap. Gains generalize to the SciQ and GoEmotions benchmarks, and on GoEmotions and LexGLUE the method yields tighter posterior uncertainty than learned and uncertainty-aware retrieval baselines. Complex multi-morbid patients benefit most, with twice the synergy-gap rate, precisely where prediction is hardest.} }
Endnote
%0 Conference Paper %T IDCR: Information-Directed Conformal Retrieval %A Manas Nanivadekar %A Jatin Khanijoan %A Swayam Kothekar %A Mohd Amaan Khan %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-nanivadekar26a %I PMLR %P 4712--4732 %U https://proceedings.mlr.press/v337/nanivadekar26a.html %V 337 %X Retrieval-augmented prediction systems select documents by semantic similarity, ignoring their effect on downstream predictive uncertainty. In high-stakes domains such as clinical diagnosis, this can yield overconfident or needlessly imprecise predictions. We propose Information-Directed Conformal Retrieval, a framework that selects documents to minimize the volume of conformal prediction sets while preserving distribution-free coverage. Modeling each document as a {Bayesian} precision update, we show that minimizing conformal volume is exactly equivalent to maximizing a log-determinant objective, and prove this objective monotone submodular, so greedy selection inherits the constant-factor $(1-1/e)$ guarantee. A Document Interaction Tensor characterizes corpus-level interaction structure, and a lightweight marginal-gain-separation gate routes uncertain retrieval steps to lookahead search. On MIMIC-IV clinical diagnosis (275 admissions from 100 patients, 9273 PubMed abstracts), greedy retrieval attains a mean greedy-to-optimal ratio of 0.9999, produces conformal ellipsoids 4.8 times smaller than random retrieval and 5.8 times smaller than cosine retrieval while retaining the distribution-free coverage guarantee, and two-step lookahead closes 77 percent of the remaining gap. Gains generalize to the SciQ and GoEmotions benchmarks, and on GoEmotions and LexGLUE the method yields tighter posterior uncertainty than learned and uncertainty-aware retrieval baselines. Complex multi-morbid patients benefit most, with twice the synergy-gap rate, precisely where prediction is hardest.
APA
Nanivadekar, M., Khanijoan, J., Kothekar, S. & Khan, M.A.. (2026). IDCR: Information-Directed Conformal Retrieval. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:4712-4732 Available from https://proceedings.mlr.press/v337/nanivadekar26a.html.

Related Material