DUET: 2D Structured and Approximately Equivariant Representations

Xavier Suau, Federico Danieli, T. Anderson Keller, Arno Blaas, Chen Huang, Jason Ramapuram, Dan Busbridge, Luca Zappella
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:32749-32769, 2023.

Abstract

Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-suau23a, title = {{DUET}: 2{D} Structured and Approximately Equivariant Representations}, author = {Suau, Xavier and Danieli, Federico and Keller, T. Anderson and Blaas, Arno and Huang, Chen and Ramapuram, Jason and Busbridge, Dan and Zappella, Luca}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {32749--32769}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/suau23a/suau23a.pdf}, url = {https://proceedings.mlr.press/v202/suau23a.html}, abstract = {Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.} }
Endnote
%0 Conference Paper %T DUET: 2D Structured and Approximately Equivariant Representations %A Xavier Suau %A Federico Danieli %A T. Anderson Keller %A Arno Blaas %A Chen Huang %A Jason Ramapuram %A Dan Busbridge %A Luca Zappella %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-suau23a %I PMLR %P 32749--32769 %U https://proceedings.mlr.press/v202/suau23a.html %V 202 %X Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.
APA
Suau, X., Danieli, F., Keller, T.A., Blaas, A., Huang, C., Ramapuram, J., Busbridge, D. & Zappella, L.. (2023). DUET: 2D Structured and Approximately Equivariant Representations. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:32749-32769 Available from https://proceedings.mlr.press/v202/suau23a.html.

Related Material