The Interplay Between Interpolation and Aggregation in Regression: Optimal Sample Complexity

Mikael Møller Høgsgaard, Kasper Green Larsen, Liang-Yu Zou
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:43351-43387, 2026.

Abstract

This work investigates theoretically the interplay between interpolation and aggregation in regression. We establish that the $\gamma$-graph dimension characterizes learnability for a broad class of natural aggregation procedures. Furthermore, we prove that an extremely simple aggregation procedure, combining three interpolating hypotheses via the median, is optimal among all these aggregation procedures, and is strictly more powerful than proper learning. Finally, we show that some hypothesis classes are learnable only by aggregating infinitely many hypotheses or by using non-interpolating aggregation rules (which may predict outside the range of their inputs), and any finite interpolating aggregation fails to achieve even trivial performance.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-hogsgaard26a, title = {The Interplay Between Interpolation and Aggregation in Regression: Optimal Sample Complexity}, author = {H{\o}gsgaard, Mikael M{\o}ller and Larsen, Kasper Green and Zou, Liang-Yu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {43351--43387}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/hogsgaard26a/hogsgaard26a.pdf}, url = {https://proceedings.mlr.press/v306/hogsgaard26a.html}, abstract = {This work investigates theoretically the interplay between interpolation and aggregation in regression. We establish that the $\gamma$-graph dimension characterizes learnability for a broad class of natural aggregation procedures. Furthermore, we prove that an extremely simple aggregation procedure, combining three interpolating hypotheses via the median, is optimal among all these aggregation procedures, and is strictly more powerful than proper learning. Finally, we show that some hypothesis classes are learnable only by aggregating infinitely many hypotheses or by using non-interpolating aggregation rules (which may predict outside the range of their inputs), and any finite interpolating aggregation fails to achieve even trivial performance.} }
Endnote
%0 Conference Paper %T The Interplay Between Interpolation and Aggregation in Regression: Optimal Sample Complexity %A Mikael Møller Høgsgaard %A Kasper Green Larsen %A Liang-Yu Zou %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-hogsgaard26a %I PMLR %P 43351--43387 %U https://proceedings.mlr.press/v306/hogsgaard26a.html %V 306 %X This work investigates theoretically the interplay between interpolation and aggregation in regression. We establish that the $\gamma$-graph dimension characterizes learnability for a broad class of natural aggregation procedures. Furthermore, we prove that an extremely simple aggregation procedure, combining three interpolating hypotheses via the median, is optimal among all these aggregation procedures, and is strictly more powerful than proper learning. Finally, we show that some hypothesis classes are learnable only by aggregating infinitely many hypotheses or by using non-interpolating aggregation rules (which may predict outside the range of their inputs), and any finite interpolating aggregation fails to achieve even trivial performance.
APA
Høgsgaard, M.M., Larsen, K.G. & Zou, L.. (2026). The Interplay Between Interpolation and Aggregation in Regression: Optimal Sample Complexity. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:43351-43387 Available from https://proceedings.mlr.press/v306/hogsgaard26a.html.

Related Material