Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent

Yichao Lu, Dean Foster
Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, PMLR R12:471-478, 2014.

Abstract

We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regres- sion under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency compared with other large scale regression al- gorithms like Gradient Descent, Stochastic Gra- dient Descent and Principal Component Regres- sion on both simulated and real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR12-lu14a, title = {Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent}, author = {Lu, Yichao and Foster, Dean}, booktitle = {Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence}, pages = {471--478}, year = {2014}, editor = {Zhang, Nevin L. and Tian, Jin}, volume = {R12}, series = {Proceedings of Machine Learning Research}, month = {23--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r12/main/assets/lu14a/lu14a.pdf}, url = {https://proceedings.mlr.press/r12/lu14a.html}, abstract = {We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regres- sion under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency compared with other large scale regression al- gorithms like Gradient Descent, Stochastic Gra- dient Descent and Principal Component Regres- sion on both simulated and real datasets.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent %A Yichao Lu %A Dean Foster %B Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2014 %E Nevin L. Zhang %E Jin Tian %F pmlr-vR12-lu14a %I PMLR %P 471--478 %U https://proceedings.mlr.press/r12/lu14a.html %V R12 %X We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regres- sion under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency compared with other large scale regression al- gorithms like Gradient Descent, Stochastic Gra- dient Descent and Principal Component Regres- sion on both simulated and real datasets. %Z Reissued by PMLR on 04 October 2026.
APA
Lu, Y. & Foster, D.. (2014). Fast Ridge Regression with Randomized Principal Component Analysis and Gradient Descent. Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R12:471-478 Available from https://proceedings.mlr.press/r12/lu14a.html. Reissued by PMLR on 04 October 2026.

Related Material