Robust Human-AI Complementarity under Uncertainty

Yewon Byun, Bryan Wilder
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:10420-10449, 2026.

Abstract

Machine learning models are often intended to augment rather than replace human decision-makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI’s prediction errors are negatively correlated with those of the human, the decision-maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world forecasting benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-byun26b, title = {Robust Human-{AI} Complementarity under Uncertainty}, author = {Byun, Yewon and Wilder, Bryan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {10420--10449}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/byun26b/byun26b.pdf}, url = {https://proceedings.mlr.press/v306/byun26b.html}, abstract = {Machine learning models are often intended to augment rather than replace human decision-makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI’s prediction errors are negatively correlated with those of the human, the decision-maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world forecasting benchmarks.} }
Endnote
%0 Conference Paper %T Robust Human-AI Complementarity under Uncertainty %A Yewon Byun %A Bryan Wilder %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-byun26b %I PMLR %P 10420--10449 %U https://proceedings.mlr.press/v306/byun26b.html %V 306 %X Machine learning models are often intended to augment rather than replace human decision-makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI’s prediction errors are negatively correlated with those of the human, the decision-maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world forecasting benchmarks.
APA
Byun, Y. & Wilder, B.. (2026). Robust Human-AI Complementarity under Uncertainty. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:10420-10449 Available from https://proceedings.mlr.press/v306/byun26b.html.

Related Material