Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions

Liangchen Liu, Juncai He, Yen-Hsi Tsai
Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML), PMLR 221:557-576, 2023.

Abstract

In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold’s extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold’s curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the manifold is flat. Otherwise, the manifold’s curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.

Cite this Paper

BibTeX
@InProceedings{pmlr-v221-liu23b, title = {Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions}, author = {Liu, Liangchen and He, Juncai and Tsai, Yen-Hsi}, booktitle = {Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML)}, pages = {557--576}, year = {2023}, editor = {Doster, Timothy and Emerson, Tegan and Kvinge, Henry and Miolane, Nina and Papillon, Mathilde and Rieck, Bastian and Sanborn, Sophia}, volume = {221}, series = {Proceedings of Machine Learning Research}, month = {28 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v221/liu23b/liu23b.pdf}, url = {https://proceedings.mlr.press/v221/liu23b.html}, abstract = {In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold’s extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold’s curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the manifold is flat. Otherwise, the manifold’s curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.} }
Endnote
%0 Conference Paper %T Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions %A Liangchen Liu %A Juncai He %A Yen-Hsi Tsai %B Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML) %C Proceedings of Machine Learning Research %D 2023 %E Timothy Doster %E Tegan Emerson %E Henry Kvinge %E Nina Miolane %E Mathilde Papillon %E Bastian Rieck %E Sophia Sanborn %F pmlr-v221-liu23b %I PMLR %P 557--576 %U https://proceedings.mlr.press/v221/liu23b.html %V 221 %X In this paper, we study linear regression applied to data structured on a manifold. We assume that the data manifold is smooth and is embedded in a Euclidean space, and our objective is to reveal the impact of the data manifold’s extrinsic geometry on the regression. Specifically, we analyze the impact of the manifold’s curvatures (or higher order nonlinearity in the parameterization when the curvatures are locally zero) on the uniqueness of the regression solution. Our findings suggest that the corresponding linear regression does not have a unique solution when the manifold is flat. Otherwise, the manifold’s curvature (or higher order nonlinearity in the embedding) may contribute significantly, particularly in the solution associated with the normal directions of the manifold. Our findings thus reveal the role of data manifold geometry in ensuring the stability of regression models for out-of-distribution inferences.
APA
Liu, L., He, J. & Tsai, Y.. (2023). Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions. Proceedings of 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML), in Proceedings of Machine Learning Research 221:557-576 Available from https://proceedings.mlr.press/v221/liu23b.html.

Related Material