Geometric Autoencoders - What You See is What You Decode

Philipp Nazari, Sebastian Damrich, Fred A Hamprecht
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:25834-25857, 2023.

Abstract

Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding’s distortion, and second a new regularizer mitigating such distortion. Our “Geometric Autoencoder” avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-nazari23a, title = {Geometric Autoencoders - What You See is What You Decode}, author = {Nazari, Philipp and Damrich, Sebastian and Hamprecht, Fred A}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {25834--25857}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/nazari23a/nazari23a.pdf}, url = {https://proceedings.mlr.press/v202/nazari23a.html}, abstract = {Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding’s distortion, and second a new regularizer mitigating such distortion. Our “Geometric Autoencoder” avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.} }
Endnote
%0 Conference Paper %T Geometric Autoencoders - What You See is What You Decode %A Philipp Nazari %A Sebastian Damrich %A Fred A Hamprecht %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-nazari23a %I PMLR %P 25834--25857 %U https://proceedings.mlr.press/v202/nazari23a.html %V 202 %X Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding’s distortion, and second a new regularizer mitigating such distortion. Our “Geometric Autoencoder” avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.
APA
Nazari, P., Damrich, S. & Hamprecht, F.A.. (2023). Geometric Autoencoders - What You See is What You Decode. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:25834-25857 Available from https://proceedings.mlr.press/v202/nazari23a.html.

Related Material