Learning to Disentangle Factors of Variation with Manifold Interaction

Scott Reed, Kihyuk Sohn, Yuting Zhang, Honglak Lee
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1431-1439, 2014.

Abstract

Many latent factors of variation interact to generate sensory data; for example pose, morphology and expression in face images. We propose to learn manifold coordinates for the relevant factors of variation and to model their joint interaction. Most existing feature learning algorithms focus on a single task and extract features that are sensitive to the task-relevant factors and invariant to all others. However, models that just extract a single set of invariant features do not exploit the relationships among the latent factors. To address this we propose a higher-order Boltzmann machine that incorporates multiplicative interactions among groups of hidden units that each learn to encode a factor of variation. Furthermore, we propose a manifold-based training strategy that allows effective disentangling, meaning that units in each group encode a distinct type of variation. Our model achieves state-of-the-art emotion recognition and face verification performance on the Toronto Face Database, and we also demonstrate disentangled features learned on the CMU Multi-PIE dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-reed14, title = {Learning to Disentangle Factors of Variation with Manifold Interaction}, author = {Reed, Scott and Sohn, Kihyuk and Zhang, Yuting and Lee, Honglak}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1431--1439}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/reed14.pdf}, url = {https://proceedings.mlr.press/v32/reed14.html}, abstract = {Many latent factors of variation interact to generate sensory data; for example pose, morphology and expression in face images. We propose to learn manifold coordinates for the relevant factors of variation and to model their joint interaction. Most existing feature learning algorithms focus on a single task and extract features that are sensitive to the task-relevant factors and invariant to all others. However, models that just extract a single set of invariant features do not exploit the relationships among the latent factors. To address this we propose a higher-order Boltzmann machine that incorporates multiplicative interactions among groups of hidden units that each learn to encode a factor of variation. Furthermore, we propose a manifold-based training strategy that allows effective disentangling, meaning that units in each group encode a distinct type of variation. Our model achieves state-of-the-art emotion recognition and face verification performance on the Toronto Face Database, and we also demonstrate disentangled features learned on the CMU Multi-PIE dataset.} }
Endnote
%0 Conference Paper %T Learning to Disentangle Factors of Variation with Manifold Interaction %A Scott Reed %A Kihyuk Sohn %A Yuting Zhang %A Honglak Lee %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-reed14 %I PMLR %P 1431--1439 %U https://proceedings.mlr.press/v32/reed14.html %V 32 %N 2 %X Many latent factors of variation interact to generate sensory data; for example pose, morphology and expression in face images. We propose to learn manifold coordinates for the relevant factors of variation and to model their joint interaction. Most existing feature learning algorithms focus on a single task and extract features that are sensitive to the task-relevant factors and invariant to all others. However, models that just extract a single set of invariant features do not exploit the relationships among the latent factors. To address this we propose a higher-order Boltzmann machine that incorporates multiplicative interactions among groups of hidden units that each learn to encode a factor of variation. Furthermore, we propose a manifold-based training strategy that allows effective disentangling, meaning that units in each group encode a distinct type of variation. Our model achieves state-of-the-art emotion recognition and face verification performance on the Toronto Face Database, and we also demonstrate disentangled features learned on the CMU Multi-PIE dataset.
RIS
TY - CPAPER TI - Learning to Disentangle Factors of Variation with Manifold Interaction AU - Scott Reed AU - Kihyuk Sohn AU - Yuting Zhang AU - Honglak Lee BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-reed14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1431 EP - 1439 L1 - http://proceedings.mlr.press/v32/reed14.pdf UR - https://proceedings.mlr.press/v32/reed14.html AB - Many latent factors of variation interact to generate sensory data; for example pose, morphology and expression in face images. We propose to learn manifold coordinates for the relevant factors of variation and to model their joint interaction. Most existing feature learning algorithms focus on a single task and extract features that are sensitive to the task-relevant factors and invariant to all others. However, models that just extract a single set of invariant features do not exploit the relationships among the latent factors. To address this we propose a higher-order Boltzmann machine that incorporates multiplicative interactions among groups of hidden units that each learn to encode a factor of variation. Furthermore, we propose a manifold-based training strategy that allows effective disentangling, meaning that units in each group encode a distinct type of variation. Our model achieves state-of-the-art emotion recognition and face verification performance on the Toronto Face Database, and we also demonstrate disentangled features learned on the CMU Multi-PIE dataset. ER -
APA
Reed, S., Sohn, K., Zhang, Y. & Lee, H.. (2014). Learning to Disentangle Factors of Variation with Manifold Interaction. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1431-1439 Available from https://proceedings.mlr.press/v32/reed14.html.

Related Material