Quantifying and Learning Linear Symmetry-Based Disentanglement

Loek Tonnaer, Luis Armando Perez Rey, Vlado Menkovski, Mike Holenderski, Jim Portegies
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:21584-21608, 2022.

Abstract

The definition of Linear Symmetry-Based Disentanglement (LSBD) formalizes the notion of linearly disentangled representations, but there is currently no metric to quantify LSBD. Such a metric is crucial to evaluate LSBD methods and to compare them to previous understandings of disentanglement. We propose D_LSBD, a mathematically sound metric to quantify LSBD, and provide a practical implementation for SO(2) groups. Furthermore, from this metric we derive LSBD-VAE, a semi-supervised method to learn LSBD representations. We demonstrate the utility of our metric by showing that (1) common VAE-based disentanglement methods don’t learn LSBD representations, (2) LSBD-VAE, as well as other recent methods, can learn LSBD representations needing only limited supervision on transformations, and (3) various desirable properties expressed by existing disentanglement metrics are also achieved by LSBD representations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-tonnaer22a, title = {Quantifying and Learning Linear Symmetry-Based Disentanglement}, author = {Tonnaer, Loek and Rey, Luis Armando Perez and Menkovski, Vlado and Holenderski, Mike and Portegies, Jim}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {21584--21608}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/tonnaer22a/tonnaer22a.pdf}, url = {https://proceedings.mlr.press/v162/tonnaer22a.html}, abstract = {The definition of Linear Symmetry-Based Disentanglement (LSBD) formalizes the notion of linearly disentangled representations, but there is currently no metric to quantify LSBD. Such a metric is crucial to evaluate LSBD methods and to compare them to previous understandings of disentanglement. We propose D_LSBD, a mathematically sound metric to quantify LSBD, and provide a practical implementation for SO(2) groups. Furthermore, from this metric we derive LSBD-VAE, a semi-supervised method to learn LSBD representations. We demonstrate the utility of our metric by showing that (1) common VAE-based disentanglement methods don’t learn LSBD representations, (2) LSBD-VAE, as well as other recent methods, can learn LSBD representations needing only limited supervision on transformations, and (3) various desirable properties expressed by existing disentanglement metrics are also achieved by LSBD representations.} }
Endnote
%0 Conference Paper %T Quantifying and Learning Linear Symmetry-Based Disentanglement %A Loek Tonnaer %A Luis Armando Perez Rey %A Vlado Menkovski %A Mike Holenderski %A Jim Portegies %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-tonnaer22a %I PMLR %P 21584--21608 %U https://proceedings.mlr.press/v162/tonnaer22a.html %V 162 %X The definition of Linear Symmetry-Based Disentanglement (LSBD) formalizes the notion of linearly disentangled representations, but there is currently no metric to quantify LSBD. Such a metric is crucial to evaluate LSBD methods and to compare them to previous understandings of disentanglement. We propose D_LSBD, a mathematically sound metric to quantify LSBD, and provide a practical implementation for SO(2) groups. Furthermore, from this metric we derive LSBD-VAE, a semi-supervised method to learn LSBD representations. We demonstrate the utility of our metric by showing that (1) common VAE-based disentanglement methods don’t learn LSBD representations, (2) LSBD-VAE, as well as other recent methods, can learn LSBD representations needing only limited supervision on transformations, and (3) various desirable properties expressed by existing disentanglement metrics are also achieved by LSBD representations.
APA
Tonnaer, L., Rey, L.A.P., Menkovski, V., Holenderski, M. & Portegies, J.. (2022). Quantifying and Learning Linear Symmetry-Based Disentanglement. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:21584-21608 Available from https://proceedings.mlr.press/v162/tonnaer22a.html.

Related Material