Autoencoder Image Interpolation by Shaping the Latent Space

Alon Oring, Zohar Yakhini, Yacov Hel-Or
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:8281-8290, 2021.

Abstract

One of the fascinating properties of deep learning is the ability of the network to reveal the underlying factors characterizing elements in datasets of different types. Autoencoders represent an effective approach for computing these factors. Autoencoders have been studied in the context of enabling interpolation between data points by decoding convex combinations of latent vectors. However, this interpolation often leads to artifacts or produces unrealistic results during reconstruction. We argue that these incongruities are due to the structure of the latent space and to the fact that such naively interpolated latent vectors deviate from the data manifold. In this paper, we propose a regularization technique that shapes the latent representation to follow a manifold that is consistent with the training images and that forces the manifold to be smooth and locally convex. This regularization not only enables faithful interpolation between data points, as we show herein but can also be used as a general regularization technique to avoid overfitting or to produce new samples for data augmentation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-oring21a, title = {Autoencoder Image Interpolation by Shaping the Latent Space}, author = {Oring, Alon and Yakhini, Zohar and Hel-Or, Yacov}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {8281--8290}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/oring21a/oring21a.pdf}, url = {https://proceedings.mlr.press/v139/oring21a.html}, abstract = {One of the fascinating properties of deep learning is the ability of the network to reveal the underlying factors characterizing elements in datasets of different types. Autoencoders represent an effective approach for computing these factors. Autoencoders have been studied in the context of enabling interpolation between data points by decoding convex combinations of latent vectors. However, this interpolation often leads to artifacts or produces unrealistic results during reconstruction. We argue that these incongruities are due to the structure of the latent space and to the fact that such naively interpolated latent vectors deviate from the data manifold. In this paper, we propose a regularization technique that shapes the latent representation to follow a manifold that is consistent with the training images and that forces the manifold to be smooth and locally convex. This regularization not only enables faithful interpolation between data points, as we show herein but can also be used as a general regularization technique to avoid overfitting or to produce new samples for data augmentation.} }
Endnote
%0 Conference Paper %T Autoencoder Image Interpolation by Shaping the Latent Space %A Alon Oring %A Zohar Yakhini %A Yacov Hel-Or %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-oring21a %I PMLR %P 8281--8290 %U https://proceedings.mlr.press/v139/oring21a.html %V 139 %X One of the fascinating properties of deep learning is the ability of the network to reveal the underlying factors characterizing elements in datasets of different types. Autoencoders represent an effective approach for computing these factors. Autoencoders have been studied in the context of enabling interpolation between data points by decoding convex combinations of latent vectors. However, this interpolation often leads to artifacts or produces unrealistic results during reconstruction. We argue that these incongruities are due to the structure of the latent space and to the fact that such naively interpolated latent vectors deviate from the data manifold. In this paper, we propose a regularization technique that shapes the latent representation to follow a manifold that is consistent with the training images and that forces the manifold to be smooth and locally convex. This regularization not only enables faithful interpolation between data points, as we show herein but can also be used as a general regularization technique to avoid overfitting or to produce new samples for data augmentation.
APA
Oring, A., Yakhini, Z. & Hel-Or, Y.. (2021). Autoencoder Image Interpolation by Shaping the Latent Space. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:8281-8290 Available from https://proceedings.mlr.press/v139/oring21a.html.

Related Material