General Quantification of Covariate and Concept Shifts

Hongbo Chen, Li C Xia
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:17603-17640, 2026.

Abstract

Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing definition of concept shift breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose a key notion: $\gamma^\ast$-concept shifts, and derive a general error bound unifying covariate and $\gamma^\ast$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications - a rigorous and general tool for analyzing learning error under distribution shift.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26fg, title = {General Quantification of Covariate and Concept Shifts}, author = {Chen, Hongbo and Xia, Li C}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {17603--17640}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26fg/chen26fg.pdf}, url = {https://proceedings.mlr.press/v306/chen26fg.html}, abstract = {Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing definition of concept shift breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose a key notion: $\gamma^\ast$-concept shifts, and derive a general error bound unifying covariate and $\gamma^\ast$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications - a rigorous and general tool for analyzing learning error under distribution shift.} }
Endnote
%0 Conference Paper %T General Quantification of Covariate and Concept Shifts %A Hongbo Chen %A Li C Xia %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26fg %I PMLR %P 17603--17640 %U https://proceedings.mlr.press/v306/chen26fg.html %V 306 %X Generalization under distribution shift remains a core challenge in modern machine learning, yet existing learning bound theory is limited to narrow, idealized settings and is non-estimable from samples. In this paper, we bridge the gap between theory and practical applications. We first show that existing definition of concept shift breaks when the source and target supports mismatch. Leveraging entropic optimal transport, we propose a key notion: $\gamma^\ast$-concept shifts, and derive a general error bound unifying covariate and $\gamma^\ast$-concept shifts, which applies to broad loss functions, label spaces, and stochastic labeling. We further develop estimators for these shifts with concentration guarantees, and the DataShifts algorithm, which can quantify distribution shifts and estimate the error bound in most applications - a rigorous and general tool for analyzing learning error under distribution shift.
APA
Chen, H. & Xia, L.C.. (2026). General Quantification of Covariate and Concept Shifts. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:17603-17640 Available from https://proceedings.mlr.press/v306/chen26fg.html.

Related Material