Optimal AverageCase Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness
[edit]
Proceedings of the ThirtySecond Conference on Learning Theory, PMLR 99:469470, 2019.
Abstract
In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statisticalcomputational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the averagecase complexity of sparse PCA through reductions from the planted clique ($\textsc{pc}$) conjecture. All previous reductions either fail to show tight computational lower bounds matching existing algorithms or show lower bounds for formulations of sparse PCA other than its canonical generative model, the spiked covariance model. Also, these lower bounds all quickly degrade given weak forms of the $\textsc{pc}$ conjecture. We give a reduction from $\textsc{pc}$ that yields the first full characterization of the computational barrier in the spiked covariance model, providing tight lower bounds at all sparsities $k$. We also show the surprising result that even a mild improvement in the signal strength needed by the best known polynomialtime sparse PCA algorithms would imply that the hardness threshold for $\textsc{pc}$ is $\text{polylog}(N)$, rather than on the order $N^{1/2}$ as is widely conjectured. This is the first instance of a suboptimal hardness assumption implying optimal lower bounds for another problem in unsupervised learning.
Related Material


