Autoencoding Under Normalization Constraints

Sangwoong Yoon, Yung-Kyun Noh, Frank Park
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:12087-12097, 2021.

Abstract

Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-yoon21c, title = {Autoencoding Under Normalization Constraints}, author = {Yoon, Sangwoong and Noh, Yung-Kyun and Park, Frank}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {12087--12097}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/yoon21c/yoon21c.pdf}, url = {https://proceedings.mlr.press/v139/yoon21c.html}, abstract = {Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.} }
Endnote
%0 Conference Paper %T Autoencoding Under Normalization Constraints %A Sangwoong Yoon %A Yung-Kyun Noh %A Frank Park %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-yoon21c %I PMLR %P 12087--12097 %U https://proceedings.mlr.press/v139/yoon21c.html %V 139 %X Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.
APA
Yoon, S., Noh, Y. & Park, F.. (2021). Autoencoding Under Normalization Constraints. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:12087-12097 Available from https://proceedings.mlr.press/v139/yoon21c.html.

Related Material