Scalable Algorithms for Learning High-Dimensional Linear Mixed Models

Zilong Tan, Kimberly Roche, Xiang Zhou, Sayan Mukherjee
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:258-267, 2018.

Abstract

Linear mixed models (LMMs) are used extensively to model observations that are not in-dependent. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends at least linearly on the dimension p of the covariates, and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational complexity dependence on p. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on the subsampled randomized Hadamard transform. We provide theoretical guarantees for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-tan18a, title = {Scalable Algorithms for Learning High-Dimensional Linear Mixed Models}, author = {Tan, Zilong and Roche, Kimberly and Zhou, Xiang and Mukherjee, Sayan}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {258--267}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/tan18a/tan18a.pdf}, url = {https://proceedings.mlr.press/r16/tan18a.html}, abstract = {Linear mixed models (LMMs) are used extensively to model observations that are not in-dependent. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends at least linearly on the dimension p of the covariates, and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational complexity dependence on p. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on the subsampled randomized Hadamard transform. We provide theoretical guarantees for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Scalable Algorithms for Learning High-Dimensional Linear Mixed Models %A Zilong Tan %A Kimberly Roche %A Xiang Zhou %A Sayan Mukherjee %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-tan18a %I PMLR %P 258--267 %U https://proceedings.mlr.press/r16/tan18a.html %V R16 %X Linear mixed models (LMMs) are used extensively to model observations that are not in-dependent. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends at least linearly on the dimension p of the covariates, and often use heuristics that do not offer theoretical guarantees. We present scalable algorithms for learning high-dimensional LMMs with sublinear computational complexity dependence on p. Key to our approach are novel dual estimators which use only kernel functions of the data, and fast computational techniques based on the subsampled randomized Hadamard transform. We provide theoretical guarantees for our learning algorithms, demonstrating the robustness of parameter estimation. Finally, we complement the theory with experiments on large synthetic and real data. %Z Reissued by PMLR on 04 October 2026.
APA
Tan, Z., Roche, K., Zhou, X. & Mukherjee, S.. (2018). Scalable Algorithms for Learning High-Dimensional Linear Mixed Models. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:258-267 Available from https://proceedings.mlr.press/r16/tan18a.html. Reissued by PMLR on 04 October 2026.

Related Material