Local Distance Preserving Auto-encoders using Continuous kNN Graphs

Nutan Chen, Patrick van der Smagt, Botond Cseke
Proceedings of Topological, Algebraic, and Geometric Learning Workshops 2022, PMLR 196:55-66, 2022.

Abstract

Auto-encoder models that preserve similarities in the data are a popular tool in representation learning.In this paper we introduce several auto-encoder models that preserve local distances when mapping from the data space to the latent space. We use a local distance-preserving loss that is based on the continuous k-nearest neighbours graph which is known to capture topological features at all scales simultaneously. To improve training performance, we formulate learning as a constraint optimisation problem with local distance preservation as the main objective and reconstruction accuracy as a constraint. We generalise this approach to hierarchical variational auto-encoders thus learning generative models with geometrically consistent latent and data spaces. Our method provides state-of-the-art performance across several standard datasets and evaluation metrics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v196-chen22b, title = {Local Distance Preserving Auto-encoders using Continuous kNN Graphs}, author = {Chen, Nutan and van der Smagt, Patrick and Cseke, Botond}, booktitle = {Proceedings of Topological, Algebraic, and Geometric Learning Workshops 2022}, pages = {55--66}, year = {2022}, editor = {Cloninger, Alexander and Doster, Timothy and Emerson, Tegan and Kaul, Manohar and Ktena, Ira and Kvinge, Henry and Miolane, Nina and Rieck, Bastian and Tymochko, Sarah and Wolf, Guy}, volume = {196}, series = {Proceedings of Machine Learning Research}, month = {25 Feb--22 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v196/chen22b/chen22b.pdf}, url = {https://proceedings.mlr.press/v196/chen22b.html}, abstract = {Auto-encoder models that preserve similarities in the data are a popular tool in representation learning.In this paper we introduce several auto-encoder models that preserve local distances when mapping from the data space to the latent space. We use a local distance-preserving loss that is based on the continuous k-nearest neighbours graph which is known to capture topological features at all scales simultaneously. To improve training performance, we formulate learning as a constraint optimisation problem with local distance preservation as the main objective and reconstruction accuracy as a constraint. We generalise this approach to hierarchical variational auto-encoders thus learning generative models with geometrically consistent latent and data spaces. Our method provides state-of-the-art performance across several standard datasets and evaluation metrics.} }
Endnote
%0 Conference Paper %T Local Distance Preserving Auto-encoders using Continuous kNN Graphs %A Nutan Chen %A Patrick van der Smagt %A Botond Cseke %B Proceedings of Topological, Algebraic, and Geometric Learning Workshops 2022 %C Proceedings of Machine Learning Research %D 2022 %E Alexander Cloninger %E Timothy Doster %E Tegan Emerson %E Manohar Kaul %E Ira Ktena %E Henry Kvinge %E Nina Miolane %E Bastian Rieck %E Sarah Tymochko %E Guy Wolf %F pmlr-v196-chen22b %I PMLR %P 55--66 %U https://proceedings.mlr.press/v196/chen22b.html %V 196 %X Auto-encoder models that preserve similarities in the data are a popular tool in representation learning.In this paper we introduce several auto-encoder models that preserve local distances when mapping from the data space to the latent space. We use a local distance-preserving loss that is based on the continuous k-nearest neighbours graph which is known to capture topological features at all scales simultaneously. To improve training performance, we formulate learning as a constraint optimisation problem with local distance preservation as the main objective and reconstruction accuracy as a constraint. We generalise this approach to hierarchical variational auto-encoders thus learning generative models with geometrically consistent latent and data spaces. Our method provides state-of-the-art performance across several standard datasets and evaluation metrics.
APA
Chen, N., van der Smagt, P. & Cseke, B.. (2022). Local Distance Preserving Auto-encoders using Continuous kNN Graphs. Proceedings of Topological, Algebraic, and Geometric Learning Workshops 2022, in Proceedings of Machine Learning Research 196:55-66 Available from https://proceedings.mlr.press/v196/chen22b.html.

Related Material