Lorentzian Distance Learning for Hyperbolic Representations

Marc Law, Renjie Liao, Jake Snell, Richard Zemel
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:3672-3681, 2019.

Abstract

We introduce an approach to learn representations based on the Lorentzian distance in hyperbolic geometry. Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world. Current hyperbolic representation learning methods compare examples with the Poincaré distance. They try to minimize the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes. This formulation produces node representations close to the centroid of their descendants. To obtain efficient and interpretable algorithms, we exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form. We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases. This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children. Our approach obtains state-of-the-art results in retrieval and classification tasks on different datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-law19a, title = {Lorentzian Distance Learning for Hyperbolic Representations}, author = {Law, Marc and Liao, Renjie and Snell, Jake and Zemel, Richard}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3672--3681}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/law19a/law19a.pdf}, url = {https://proceedings.mlr.press/v97/law19a.html}, abstract = {We introduce an approach to learn representations based on the Lorentzian distance in hyperbolic geometry. Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world. Current hyperbolic representation learning methods compare examples with the Poincaré distance. They try to minimize the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes. This formulation produces node representations close to the centroid of their descendants. To obtain efficient and interpretable algorithms, we exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form. We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases. This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children. Our approach obtains state-of-the-art results in retrieval and classification tasks on different datasets.} }
Endnote
%0 Conference Paper %T Lorentzian Distance Learning for Hyperbolic Representations %A Marc Law %A Renjie Liao %A Jake Snell %A Richard Zemel %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-law19a %I PMLR %P 3672--3681 %U https://proceedings.mlr.press/v97/law19a.html %V 97 %X We introduce an approach to learn representations based on the Lorentzian distance in hyperbolic geometry. Hyperbolic geometry is especially suited to hierarchically-structured datasets, which are prevalent in the real world. Current hyperbolic representation learning methods compare examples with the Poincaré distance. They try to minimize the distance of each node in a hierarchy with its descendants while maximizing its distance with other nodes. This formulation produces node representations close to the centroid of their descendants. To obtain efficient and interpretable algorithms, we exploit the fact that the centroid w.r.t the squared Lorentzian distance can be written in closed-form. We show that the Euclidean norm of such a centroid decreases as the curvature of the hyperbolic space decreases. This property makes it appropriate to represent hierarchies where parent nodes minimize the distances to their descendants and have smaller Euclidean norm than their children. Our approach obtains state-of-the-art results in retrieval and classification tasks on different datasets.
APA
Law, M., Liao, R., Snell, J. & Zemel, R.. (2019). Lorentzian Distance Learning for Hyperbolic Representations. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:3672-3681 Available from https://proceedings.mlr.press/v97/law19a.html.

Related Material