Better Mixing via Deep Representations

Yoshua Bengio, Gregoire Mesnil, Yann Dauphin, Salah Rifai
; Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):552-560, 2013.

Abstract

It has been hypothesized, and supported with experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related conjecture: better representations, in the sense of better disentangling, can be exploited to produce Markov chains that mix faster between modes. Consequently, mixing between modes would be more efficient at higher levels of representation. To better understand this, we propose a secondary conjecture: the higher-level samples fill more uniformly the space they occupy and the high-density manifolds tend to unfold when represented at higher levels. The paper discusses these hypotheses and tests them experimentally through visualization and measurements of mixing between modes and interpolating between samples.

Cite this Paper


BibTeX
@InProceedings{pmlr-v28-bengio13, title = {Better Mixing via Deep Representations}, author = {Yoshua Bengio and Gregoire Mesnil and Yann Dauphin and Salah Rifai}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {552--560}, year = {2013}, editor = {Sanjoy Dasgupta and David McAllester}, volume = {28}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/bengio13.pdf}, url = {http://proceedings.mlr.press/v28/bengio13.html}, abstract = {It has been hypothesized, and supported with experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related conjecture: better representations, in the sense of better disentangling, can be exploited to produce Markov chains that mix faster between modes. Consequently, mixing between modes would be more efficient at higher levels of representation. To better understand this, we propose a secondary conjecture: the higher-level samples fill more uniformly the space they occupy and the high-density manifolds tend to unfold when represented at higher levels. The paper discusses these hypotheses and tests them experimentally through visualization and measurements of mixing between modes and interpolating between samples.} }
Endnote
%0 Conference Paper %T Better Mixing via Deep Representations %A Yoshua Bengio %A Gregoire Mesnil %A Yann Dauphin %A Salah Rifai %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-bengio13 %I PMLR %J Proceedings of Machine Learning Research %P 552--560 %U http://proceedings.mlr.press %V 28 %N 1 %W PMLR %X It has been hypothesized, and supported with experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related conjecture: better representations, in the sense of better disentangling, can be exploited to produce Markov chains that mix faster between modes. Consequently, mixing between modes would be more efficient at higher levels of representation. To better understand this, we propose a secondary conjecture: the higher-level samples fill more uniformly the space they occupy and the high-density manifolds tend to unfold when represented at higher levels. The paper discusses these hypotheses and tests them experimentally through visualization and measurements of mixing between modes and interpolating between samples.
RIS
TY - CPAPER TI - Better Mixing via Deep Representations AU - Yoshua Bengio AU - Gregoire Mesnil AU - Yann Dauphin AU - Salah Rifai BT - Proceedings of the 30th International Conference on Machine Learning PY - 2013/02/13 DA - 2013/02/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-bengio13 PB - PMLR SP - 552 DP - PMLR EP - 560 L1 - http://proceedings.mlr.press/v28/bengio13.pdf UR - http://proceedings.mlr.press/v28/bengio13.html AB - It has been hypothesized, and supported with experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related conjecture: better representations, in the sense of better disentangling, can be exploited to produce Markov chains that mix faster between modes. Consequently, mixing between modes would be more efficient at higher levels of representation. To better understand this, we propose a secondary conjecture: the higher-level samples fill more uniformly the space they occupy and the high-density manifolds tend to unfold when represented at higher levels. The paper discusses these hypotheses and tests them experimentally through visualization and measurements of mixing between modes and interpolating between samples. ER -
APA
Bengio, Y., Mesnil, G., Dauphin, Y. & Rifai, S.. (2013). Better Mixing via Deep Representations. Proceedings of the 30th International Conference on Machine Learning, in PMLR 28(1):552-560

Related Material