On autoencoder scoring

Hanna Kamyshanska, Roland Memisevic
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):720-728, 2013.

Abstract

Autoencoders are popular feature learning models because they are conceptually simple, easy to train and allow for efficient inference and training. Recent work has shown how certain autoencoders can assign an unnormalized “score” to data which measures how well the autoencoder can represent the data. Scores are commonly computed by using training criteria that relate the autoencoder to a probabilistic model, such as the Restricted Boltzmann Machine. In this paper we show how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system. We discuss how, and under which conditions, running the dynamical system can be viewed as performing gradient descent in an energy function, which in turn allows us to derive a score via integration. We also show how one can combine multiple, unnormalized scores into a generative classifier.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-kamyshanska13, title = {On autoencoder scoring}, author = {Kamyshanska, Hanna and Memisevic, Roland}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {720--728}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/kamyshanska13.pdf}, url = {https://proceedings.mlr.press/v28/kamyshanska13.html}, abstract = {Autoencoders are popular feature learning models because they are conceptually simple, easy to train and allow for efficient inference and training. Recent work has shown how certain autoencoders can assign an unnormalized “score” to data which measures how well the autoencoder can represent the data. Scores are commonly computed by using training criteria that relate the autoencoder to a probabilistic model, such as the Restricted Boltzmann Machine. In this paper we show how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system. We discuss how, and under which conditions, running the dynamical system can be viewed as performing gradient descent in an energy function, which in turn allows us to derive a score via integration. We also show how one can combine multiple, unnormalized scores into a generative classifier.} }
Endnote
%0 Conference Paper %T On autoencoder scoring %A Hanna Kamyshanska %A Roland Memisevic %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-kamyshanska13 %I PMLR %P 720--728 %U https://proceedings.mlr.press/v28/kamyshanska13.html %V 28 %N 3 %X Autoencoders are popular feature learning models because they are conceptually simple, easy to train and allow for efficient inference and training. Recent work has shown how certain autoencoders can assign an unnormalized “score” to data which measures how well the autoencoder can represent the data. Scores are commonly computed by using training criteria that relate the autoencoder to a probabilistic model, such as the Restricted Boltzmann Machine. In this paper we show how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system. We discuss how, and under which conditions, running the dynamical system can be viewed as performing gradient descent in an energy function, which in turn allows us to derive a score via integration. We also show how one can combine multiple, unnormalized scores into a generative classifier.
RIS
TY - CPAPER TI - On autoencoder scoring AU - Hanna Kamyshanska AU - Roland Memisevic BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-kamyshanska13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 720 EP - 728 L1 - http://proceedings.mlr.press/v28/kamyshanska13.pdf UR - https://proceedings.mlr.press/v28/kamyshanska13.html AB - Autoencoders are popular feature learning models because they are conceptually simple, easy to train and allow for efficient inference and training. Recent work has shown how certain autoencoders can assign an unnormalized “score” to data which measures how well the autoencoder can represent the data. Scores are commonly computed by using training criteria that relate the autoencoder to a probabilistic model, such as the Restricted Boltzmann Machine. In this paper we show how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system. We discuss how, and under which conditions, running the dynamical system can be viewed as performing gradient descent in an energy function, which in turn allows us to derive a score via integration. We also show how one can combine multiple, unnormalized scores into a generative classifier. ER -
APA
Kamyshanska, H. & Memisevic, R.. (2013). On autoencoder scoring. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):720-728 Available from https://proceedings.mlr.press/v28/kamyshanska13.html.

Related Material