Generalized Correlation Shifting for Lasso

Izuru Miyazaki, Hironori Fujisawa
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2494-2502, 2026.

Abstract

The Lasso has been widely used in a high-dimensional setting, but its estimation accuracy may become inadequate when the covariates are highly correlated or when the number of covariates is extremely large. To overcome this problem, we propose a novel preconditioner that adaptively induces a low-rank structure in the design matrix. The proposed preconditioner achieves a higher probability of sign correctness under some conditions. We establish theoretical guarantees showing that our method dominates the standard Lasso, and we further demonstrate its superiority over the correlation shifting. To validate its practical effectiveness, we conducted numerical experiments on synthetic and semi-real datasets, and the proposed method presented better performance than existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-miyazaki26a, title = { Generalized Correlation Shifting for Lasso }, author = {Miyazaki, Izuru and Fujisawa, Hironori}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2494--2502}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/miyazaki26a/miyazaki26a.pdf}, url = {https://proceedings.mlr.press/v300/miyazaki26a.html}, abstract = { The Lasso has been widely used in a high-dimensional setting, but its estimation accuracy may become inadequate when the covariates are highly correlated or when the number of covariates is extremely large. To overcome this problem, we propose a novel preconditioner that adaptively induces a low-rank structure in the design matrix. The proposed preconditioner achieves a higher probability of sign correctness under some conditions. We establish theoretical guarantees showing that our method dominates the standard Lasso, and we further demonstrate its superiority over the correlation shifting. To validate its practical effectiveness, we conducted numerical experiments on synthetic and semi-real datasets, and the proposed method presented better performance than existing methods. } }
Endnote
%0 Conference Paper %T Generalized Correlation Shifting for Lasso %A Izuru Miyazaki %A Hironori Fujisawa %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-miyazaki26a %I PMLR %P 2494--2502 %U https://proceedings.mlr.press/v300/miyazaki26a.html %V 300 %X The Lasso has been widely used in a high-dimensional setting, but its estimation accuracy may become inadequate when the covariates are highly correlated or when the number of covariates is extremely large. To overcome this problem, we propose a novel preconditioner that adaptively induces a low-rank structure in the design matrix. The proposed preconditioner achieves a higher probability of sign correctness under some conditions. We establish theoretical guarantees showing that our method dominates the standard Lasso, and we further demonstrate its superiority over the correlation shifting. To validate its practical effectiveness, we conducted numerical experiments on synthetic and semi-real datasets, and the proposed method presented better performance than existing methods.
APA
Miyazaki, I. & Fujisawa, H.. (2026). Generalized Correlation Shifting for Lasso . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2494-2502 Available from https://proceedings.mlr.press/v300/miyazaki26a.html.

Related Material