Human-AI Collaborative Uncertainty Quantification

Sima Noorani, Shayan Kiyani, George J. Pappas, Hamed Hassani
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:93777-93800, 2026.

Abstract

AI predictive systems increasingly support high-stakes decision making, yet robust decisions under uncertainty often rely on human capabilities beyond AI alone. This motivates collaborative approaches that combine human judgment with AI predictions. We study this problem through the lens of uncertainty quantification and introduce Human-AI Collaborative Uncertainty Quantification, a framework in which an AI system refines a human expert’s proposed prediction set subject to two principles: counterfactual harm, requiring that the AI not degrade correct human judgments, and complementarity, requiring recovery of correct outcomes the human missed. At the population level, we show that the optimal collaborative prediction set has a simple two-threshold structure over a single score function, governing pruning and augmentation relative to the human proposal. Building on this characterization, we develop offline and online calibration algorithms with distribution-free finite-sample guarantees. The online algorithm adapts to arbitrary distribution shifts, including settings where human behavior evolves through interaction with the AI. Empirically, we show that collaborative prediction sets outperform human-only and AI-only baselines, achieving improved coverage–efficiency tradeoffs across image classification, regression, and text-based medical decision making.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-noorani26b, title = {Human-{AI} Collaborative Uncertainty Quantification}, author = {Noorani, Sima and Kiyani, Shayan and Pappas, George J. and Hassani, Hamed}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {93777--93800}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/noorani26b/noorani26b.pdf}, url = {https://proceedings.mlr.press/v306/noorani26b.html}, abstract = {AI predictive systems increasingly support high-stakes decision making, yet robust decisions under uncertainty often rely on human capabilities beyond AI alone. This motivates collaborative approaches that combine human judgment with AI predictions. We study this problem through the lens of uncertainty quantification and introduce Human-AI Collaborative Uncertainty Quantification, a framework in which an AI system refines a human expert’s proposed prediction set subject to two principles: counterfactual harm, requiring that the AI not degrade correct human judgments, and complementarity, requiring recovery of correct outcomes the human missed. At the population level, we show that the optimal collaborative prediction set has a simple two-threshold structure over a single score function, governing pruning and augmentation relative to the human proposal. Building on this characterization, we develop offline and online calibration algorithms with distribution-free finite-sample guarantees. The online algorithm adapts to arbitrary distribution shifts, including settings where human behavior evolves through interaction with the AI. Empirically, we show that collaborative prediction sets outperform human-only and AI-only baselines, achieving improved coverage–efficiency tradeoffs across image classification, regression, and text-based medical decision making.} }
Endnote
%0 Conference Paper %T Human-AI Collaborative Uncertainty Quantification %A Sima Noorani %A Shayan Kiyani %A George J. Pappas %A Hamed Hassani %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-noorani26b %I PMLR %P 93777--93800 %U https://proceedings.mlr.press/v306/noorani26b.html %V 306 %X AI predictive systems increasingly support high-stakes decision making, yet robust decisions under uncertainty often rely on human capabilities beyond AI alone. This motivates collaborative approaches that combine human judgment with AI predictions. We study this problem through the lens of uncertainty quantification and introduce Human-AI Collaborative Uncertainty Quantification, a framework in which an AI system refines a human expert’s proposed prediction set subject to two principles: counterfactual harm, requiring that the AI not degrade correct human judgments, and complementarity, requiring recovery of correct outcomes the human missed. At the population level, we show that the optimal collaborative prediction set has a simple two-threshold structure over a single score function, governing pruning and augmentation relative to the human proposal. Building on this characterization, we develop offline and online calibration algorithms with distribution-free finite-sample guarantees. The online algorithm adapts to arbitrary distribution shifts, including settings where human behavior evolves through interaction with the AI. Empirically, we show that collaborative prediction sets outperform human-only and AI-only baselines, achieving improved coverage–efficiency tradeoffs across image classification, regression, and text-based medical decision making.
APA
Noorani, S., Kiyani, S., Pappas, G.J. & Hassani, H.. (2026). Human-AI Collaborative Uncertainty Quantification. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:93777-93800 Available from https://proceedings.mlr.press/v306/noorani26b.html.

Related Material