Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator

Zeng Li, Chuanlong Xie, Qinwen Wang
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6533-6542, 2021.

Abstract

This paper quantifies the uncertainty of prediction risk for the min-norm least squares estimator in high-dimensional linear regression models. We establish the asymptotic normality of prediction risk when both the sample size and the number of features tend to infinity. Based on the newly established central limit theorems(CLTs), we derive the confidence intervals of the prediction risk under various scenarios. Our results demonstrate the sample-wise non-monotonicity of the prediction risk and confirm “more data hurt" phenomenon. Furthermore, the width of confidence intervals indicates that over-parameterization would enlarge the randomness of prediction performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-li21x, title = {Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator}, author = {Li, Zeng and Xie, Chuanlong and Wang, Qinwen}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {6533--6542}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/li21x/li21x.pdf}, url = {https://proceedings.mlr.press/v139/li21x.html}, abstract = {This paper quantifies the uncertainty of prediction risk for the min-norm least squares estimator in high-dimensional linear regression models. We establish the asymptotic normality of prediction risk when both the sample size and the number of features tend to infinity. Based on the newly established central limit theorems(CLTs), we derive the confidence intervals of the prediction risk under various scenarios. Our results demonstrate the sample-wise non-monotonicity of the prediction risk and confirm “more data hurt" phenomenon. Furthermore, the width of confidence intervals indicates that over-parameterization would enlarge the randomness of prediction performance.} }
Endnote
%0 Conference Paper %T Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator %A Zeng Li %A Chuanlong Xie %A Qinwen Wang %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-li21x %I PMLR %P 6533--6542 %U https://proceedings.mlr.press/v139/li21x.html %V 139 %X This paper quantifies the uncertainty of prediction risk for the min-norm least squares estimator in high-dimensional linear regression models. We establish the asymptotic normality of prediction risk when both the sample size and the number of features tend to infinity. Based on the newly established central limit theorems(CLTs), we derive the confidence intervals of the prediction risk under various scenarios. Our results demonstrate the sample-wise non-monotonicity of the prediction risk and confirm “more data hurt" phenomenon. Furthermore, the width of confidence intervals indicates that over-parameterization would enlarge the randomness of prediction performance.
APA
Li, Z., Xie, C. & Wang, Q.. (2021). Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:6533-6542 Available from https://proceedings.mlr.press/v139/li21x.html.

Related Material