Stochastic and Private Nonconvex Outlier-Robust PCAs

Tyler Maunu, Chenyu Yu, Gilad Lerman
Proceedings of Mathematical and Scientific Machine Learning, PMLR 190:173-188, 2022.

Abstract

We develop theoretically guaranteed stochastic methods for outlier-robust PCA. Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset that is corrupted with outliers. We are able to show that our methods, which involve stochastic geodesic gradient descent over the Grassmannian manifold, converge and recover an underlying subspace in various regimes through the development of a novel convergence analysis. The main application of this method is an effective differentially private algorithm for outlier-robust PCA that uses a Gaussian noise mechanism within the stochastic gradient method. Our results emphasize the advantages of the nonconvex methods over another convex approach to solving this problem in the differentially private setting. Experiments on synthetic and stylized data verify these results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v190-maunu22a, title = {Stochastic and Private Nonconvex Outlier-Robust PCAs}, author = {Maunu, Tyler and Yu, Chenyu and Lerman, Gilad}, booktitle = {Proceedings of Mathematical and Scientific Machine Learning}, pages = {173--188}, year = {2022}, editor = {Dong, Bin and Li, Qianxiao and Wang, Lei and Xu, Zhi-Qin John}, volume = {190}, series = {Proceedings of Machine Learning Research}, month = {15--17 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v190/maunu22a/maunu22a.pdf}, url = {https://proceedings.mlr.press/v190/maunu22a.html}, abstract = {We develop theoretically guaranteed stochastic methods for outlier-robust PCA. Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset that is corrupted with outliers. We are able to show that our methods, which involve stochastic geodesic gradient descent over the Grassmannian manifold, converge and recover an underlying subspace in various regimes through the development of a novel convergence analysis. The main application of this method is an effective differentially private algorithm for outlier-robust PCA that uses a Gaussian noise mechanism within the stochastic gradient method. Our results emphasize the advantages of the nonconvex methods over another convex approach to solving this problem in the differentially private setting. Experiments on synthetic and stylized data verify these results.} }
Endnote
%0 Conference Paper %T Stochastic and Private Nonconvex Outlier-Robust PCAs %A Tyler Maunu %A Chenyu Yu %A Gilad Lerman %B Proceedings of Mathematical and Scientific Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Bin Dong %E Qianxiao Li %E Lei Wang %E Zhi-Qin John Xu %F pmlr-v190-maunu22a %I PMLR %P 173--188 %U https://proceedings.mlr.press/v190/maunu22a.html %V 190 %X We develop theoretically guaranteed stochastic methods for outlier-robust PCA. Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset that is corrupted with outliers. We are able to show that our methods, which involve stochastic geodesic gradient descent over the Grassmannian manifold, converge and recover an underlying subspace in various regimes through the development of a novel convergence analysis. The main application of this method is an effective differentially private algorithm for outlier-robust PCA that uses a Gaussian noise mechanism within the stochastic gradient method. Our results emphasize the advantages of the nonconvex methods over another convex approach to solving this problem in the differentially private setting. Experiments on synthetic and stylized data verify these results.
APA
Maunu, T., Yu, C. & Lerman, G.. (2022). Stochastic and Private Nonconvex Outlier-Robust PCAs. Proceedings of Mathematical and Scientific Machine Learning, in Proceedings of Machine Learning Research 190:173-188 Available from https://proceedings.mlr.press/v190/maunu22a.html.

Related Material