Robust Inference for High-Dimensional Linear Models via Residual Randomization

Y. Samuel Wang, Si Kai Lee, Panos Toulis, Mladen Kolar
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10805-10815, 2021.

Abstract

We propose a residual randomization procedure designed for robust inference using Lasso estimates in the high-dimensional setting. Compared to earlier work that focuses on sub-Gaussian errors, the proposed procedure is designed to work robustly in settings that also include heavy-tailed covariates and errors. Moreover, our procedure can be valid under clustered errors, which is important in practice, but has been largely overlooked by earlier work. Through extensive simulations, we illustrate our method’s wider range of applicability as suggested by theory. In particular, we show that our method outperforms state-of-art methods in challenging, yet more realistic, settings where the distribution of covariates is heavy-tailed or the sample size is small, while it remains competitive in standard, “well behaved" settings previously studied in the literature.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-wang21m, title = {Robust Inference for High-Dimensional Linear Models via Residual Randomization}, author = {Wang, Y. Samuel and Lee, Si Kai and Toulis, Panos and Kolar, Mladen}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {10805--10815}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/wang21m/wang21m.pdf}, url = {https://proceedings.mlr.press/v139/wang21m.html}, abstract = {We propose a residual randomization procedure designed for robust inference using Lasso estimates in the high-dimensional setting. Compared to earlier work that focuses on sub-Gaussian errors, the proposed procedure is designed to work robustly in settings that also include heavy-tailed covariates and errors. Moreover, our procedure can be valid under clustered errors, which is important in practice, but has been largely overlooked by earlier work. Through extensive simulations, we illustrate our method’s wider range of applicability as suggested by theory. In particular, we show that our method outperforms state-of-art methods in challenging, yet more realistic, settings where the distribution of covariates is heavy-tailed or the sample size is small, while it remains competitive in standard, “well behaved" settings previously studied in the literature.} }
Endnote
%0 Conference Paper %T Robust Inference for High-Dimensional Linear Models via Residual Randomization %A Y. Samuel Wang %A Si Kai Lee %A Panos Toulis %A Mladen Kolar %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-wang21m %I PMLR %P 10805--10815 %U https://proceedings.mlr.press/v139/wang21m.html %V 139 %X We propose a residual randomization procedure designed for robust inference using Lasso estimates in the high-dimensional setting. Compared to earlier work that focuses on sub-Gaussian errors, the proposed procedure is designed to work robustly in settings that also include heavy-tailed covariates and errors. Moreover, our procedure can be valid under clustered errors, which is important in practice, but has been largely overlooked by earlier work. Through extensive simulations, we illustrate our method’s wider range of applicability as suggested by theory. In particular, we show that our method outperforms state-of-art methods in challenging, yet more realistic, settings where the distribution of covariates is heavy-tailed or the sample size is small, while it remains competitive in standard, “well behaved" settings previously studied in the literature.
APA
Wang, Y.S., Lee, S.K., Toulis, P. & Kolar, M.. (2021). Robust Inference for High-Dimensional Linear Models via Residual Randomization. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:10805-10815 Available from https://proceedings.mlr.press/v139/wang21m.html.

Related Material