Provable Guarantees for Estimating Covariances between Latent Variables with Application to Precision Matrix Estimation

Haichi Long, Qifan Song, Jean Honorio
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:532-540, 2026.

Abstract

In many scientific fields, key variables of interest are latent—either because they cannot be measured directly or because doing so is prohibitively expensive. As a result, researchers often rely on high-dimensional surrogate observations and must infer relationships among the unobserved quantities. In this work, we address a fundamental challenge: How can one estimate the covariance between variables that are not directly observable? We consider a model where each latent variable elicits high-dimensional observable covariates. Under our model, we propose a method that estimates several spiked covariances from the observed variables and then reconstructs the covariance matrix among the latent variables. Our estimator achieves quadratic-time complexity with respect to the number of latent variables and only requires the sample size to be logarithmic in the number of latent variables. As an immediate application, our procedure can be leveraged to recover the conditional independence structure among the latent variables, providing interpretable insights. Extensive synthetic experiments validate our theory, demonstrating accurate estimation in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-long26a, title = { Provable Guarantees for Estimating Covariances between Latent Variables with Application to Precision Matrix Estimation }, author = {Long, Haichi and Song, Qifan and Honorio, Jean}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {532--540}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/long26a/long26a.pdf}, url = {https://proceedings.mlr.press/v300/long26a.html}, abstract = { In many scientific fields, key variables of interest are latent—either because they cannot be measured directly or because doing so is prohibitively expensive. As a result, researchers often rely on high-dimensional surrogate observations and must infer relationships among the unobserved quantities. In this work, we address a fundamental challenge: How can one estimate the covariance between variables that are not directly observable? We consider a model where each latent variable elicits high-dimensional observable covariates. Under our model, we propose a method that estimates several spiked covariances from the observed variables and then reconstructs the covariance matrix among the latent variables. Our estimator achieves quadratic-time complexity with respect to the number of latent variables and only requires the sample size to be logarithmic in the number of latent variables. As an immediate application, our procedure can be leveraged to recover the conditional independence structure among the latent variables, providing interpretable insights. Extensive synthetic experiments validate our theory, demonstrating accurate estimation in practice. } }
Endnote
%0 Conference Paper %T Provable Guarantees for Estimating Covariances between Latent Variables with Application to Precision Matrix Estimation %A Haichi Long %A Qifan Song %A Jean Honorio %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-long26a %I PMLR %P 532--540 %U https://proceedings.mlr.press/v300/long26a.html %V 300 %X In many scientific fields, key variables of interest are latent—either because they cannot be measured directly or because doing so is prohibitively expensive. As a result, researchers often rely on high-dimensional surrogate observations and must infer relationships among the unobserved quantities. In this work, we address a fundamental challenge: How can one estimate the covariance between variables that are not directly observable? We consider a model where each latent variable elicits high-dimensional observable covariates. Under our model, we propose a method that estimates several spiked covariances from the observed variables and then reconstructs the covariance matrix among the latent variables. Our estimator achieves quadratic-time complexity with respect to the number of latent variables and only requires the sample size to be logarithmic in the number of latent variables. As an immediate application, our procedure can be leveraged to recover the conditional independence structure among the latent variables, providing interpretable insights. Extensive synthetic experiments validate our theory, demonstrating accurate estimation in practice.
APA
Long, H., Song, Q. & Honorio, J.. (2026). Provable Guarantees for Estimating Covariances between Latent Variables with Application to Precision Matrix Estimation . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:532-540 Available from https://proceedings.mlr.press/v300/long26a.html.

Related Material