Online Learning-to-Defer with Varying Experts

Yannis Montreuil, Hoang Duy Dang, Maxime Meyer, Lai Xing Ng, Axel Carlier, Wei Tsang Ooi
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2701-2709, 2026.

Abstract

Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. While existing work studies this problem in batch settings, real-world deployments require handling streaming data, changing expert availability, and shifting expert distribution. We introduce the first online L2D algorithm for multiclass classification with bandit feedback and a dynamically varying pool of experts. Our method achieves regret guarantees of $O((n+n_e)T^{2/3})$ in general and $O((n+n_e)\sqrt{T})$ under a low-noise condition, where $T$ is the time horizon, $n$ the number of labels, and $n_e$ the number of distinct experts observed across rounds. The analysis builds on novel $\mathcal{H}$-consistency bounds for the online framework, combined with first-order methods for online convex optimization. Experiments on synthetic and real-world datasets demonstrate that our approach effectively extends standard Learning-to-Defer to settings with varying expert availability and reliability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-montreuil26b, title = { Online Learning-to-Defer with Varying Experts }, author = {Montreuil, Yannis and Dang, Hoang Duy and Meyer, Maxime and Ng, Lai Xing and Carlier, Axel and Ooi, Wei Tsang}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2701--2709}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/montreuil26b/montreuil26b.pdf}, url = {https://proceedings.mlr.press/v300/montreuil26b.html}, abstract = { Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. While existing work studies this problem in batch settings, real-world deployments require handling streaming data, changing expert availability, and shifting expert distribution. We introduce the first online L2D algorithm for multiclass classification with bandit feedback and a dynamically varying pool of experts. Our method achieves regret guarantees of $O((n+n_e)T^{2/3})$ in general and $O((n+n_e)\sqrt{T})$ under a low-noise condition, where $T$ is the time horizon, $n$ the number of labels, and $n_e$ the number of distinct experts observed across rounds. The analysis builds on novel $\mathcal{H}$-consistency bounds for the online framework, combined with first-order methods for online convex optimization. Experiments on synthetic and real-world datasets demonstrate that our approach effectively extends standard Learning-to-Defer to settings with varying expert availability and reliability. } }
Endnote
%0 Conference Paper %T Online Learning-to-Defer with Varying Experts %A Yannis Montreuil %A Hoang Duy Dang %A Maxime Meyer %A Lai Xing Ng %A Axel Carlier %A Wei Tsang Ooi %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-montreuil26b %I PMLR %P 2701--2709 %U https://proceedings.mlr.press/v300/montreuil26b.html %V 300 %X Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. While existing work studies this problem in batch settings, real-world deployments require handling streaming data, changing expert availability, and shifting expert distribution. We introduce the first online L2D algorithm for multiclass classification with bandit feedback and a dynamically varying pool of experts. Our method achieves regret guarantees of $O((n+n_e)T^{2/3})$ in general and $O((n+n_e)\sqrt{T})$ under a low-noise condition, where $T$ is the time horizon, $n$ the number of labels, and $n_e$ the number of distinct experts observed across rounds. The analysis builds on novel $\mathcal{H}$-consistency bounds for the online framework, combined with first-order methods for online convex optimization. Experiments on synthetic and real-world datasets demonstrate that our approach effectively extends standard Learning-to-Defer to settings with varying expert availability and reliability.
APA
Montreuil, Y., Dang, H.D., Meyer, M., Ng, L.X., Carlier, A. & Ooi, W.T.. (2026). Online Learning-to-Defer with Varying Experts . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2701-2709 Available from https://proceedings.mlr.press/v300/montreuil26b.html.

Related Material