Gated Autoencoders with Tied Input Weights

Droniou Alain, Sigaud Olivier
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(2):154-162, 2013.

Abstract

The semantic interpretation of images is one of the core applications of deep learning. Several techniques have been recently proposed to model the relation between two images, with application to pose estimation, action recognition or invariant object recognition. Among these techniques, higher-order Boltzmann machines or relational autoencoders consider projections of the images on different subspaces and intermediate layers act as transformation specific detectors. In this work, we extend the mathematical study of (Memisevic, 2012b) to show that it is possible to use a unique projection for both images in a way that turns intermediate layers as spectrum encoders of transformations. We show that this results in networks that are easier to tune and have greater generalization capabilities.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-alain13, title = {Gated Autoencoders with Tied Input Weights}, author = {Alain, Droniou and Olivier, Sigaud}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {154--162}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/alain13.pdf}, url = {https://proceedings.mlr.press/v28/alain13.html}, abstract = {The semantic interpretation of images is one of the core applications of deep learning. Several techniques have been recently proposed to model the relation between two images, with application to pose estimation, action recognition or invariant object recognition. Among these techniques, higher-order Boltzmann machines or relational autoencoders consider projections of the images on different subspaces and intermediate layers act as transformation specific detectors. In this work, we extend the mathematical study of (Memisevic, 2012b) to show that it is possible to use a unique projection for both images in a way that turns intermediate layers as spectrum encoders of transformations. We show that this results in networks that are easier to tune and have greater generalization capabilities.} }
Endnote
%0 Conference Paper %T Gated Autoencoders with Tied Input Weights %A Droniou Alain %A Sigaud Olivier %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-alain13 %I PMLR %P 154--162 %U https://proceedings.mlr.press/v28/alain13.html %V 28 %N 2 %X The semantic interpretation of images is one of the core applications of deep learning. Several techniques have been recently proposed to model the relation between two images, with application to pose estimation, action recognition or invariant object recognition. Among these techniques, higher-order Boltzmann machines or relational autoencoders consider projections of the images on different subspaces and intermediate layers act as transformation specific detectors. In this work, we extend the mathematical study of (Memisevic, 2012b) to show that it is possible to use a unique projection for both images in a way that turns intermediate layers as spectrum encoders of transformations. We show that this results in networks that are easier to tune and have greater generalization capabilities.
RIS
TY - CPAPER TI - Gated Autoencoders with Tied Input Weights AU - Droniou Alain AU - Sigaud Olivier BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-alain13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 2 SP - 154 EP - 162 L1 - http://proceedings.mlr.press/v28/alain13.pdf UR - https://proceedings.mlr.press/v28/alain13.html AB - The semantic interpretation of images is one of the core applications of deep learning. Several techniques have been recently proposed to model the relation between two images, with application to pose estimation, action recognition or invariant object recognition. Among these techniques, higher-order Boltzmann machines or relational autoencoders consider projections of the images on different subspaces and intermediate layers act as transformation specific detectors. In this work, we extend the mathematical study of (Memisevic, 2012b) to show that it is possible to use a unique projection for both images in a way that turns intermediate layers as spectrum encoders of transformations. We show that this results in networks that are easier to tune and have greater generalization capabilities. ER -
APA
Alain, D. & Olivier, S.. (2013). Gated Autoencoders with Tied Input Weights. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(2):154-162 Available from https://proceedings.mlr.press/v28/alain13.html.

Related Material