Robust Representation Learning via Perceptual Similarity Metrics

Saeid A Taghanaki, Kristy Choi, Amir Hosein Khasahmadi, Anirudh Goyal
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10043-10053, 2021.

Abstract

A fundamental challenge in artificial intelligence is learning useful representations of data that yield good performance on a downstream classification task, without overfitting to spurious input features. Extracting such task-relevant predictive information becomes particularly difficult for noisy and high-dimensional real-world data. In this work, we propose Contrastive Input Morphing (CIM), a representation learning framework that learns input-space transformations of the data to mitigate the effect of irrelevant input features on downstream performance. Our method leverages a perceptual similarity metric via a triplet loss to ensure that the transformation preserves task-relevant information. Empirically, we demonstrate the efficacy of our approach on various tasks which typically suffer from the presence of spurious correlations: classification with nuisance information, out-of-distribution generalization, and preservation of subgroup accuracies. We additionally show that CIM is complementary to other mutual information-based representation learning techniques, and demonstrate that it improves the performance of variational information bottleneck (VIB) when used in conjunction.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-taghanaki21a, title = {Robust Representation Learning via Perceptual Similarity Metrics}, author = {Taghanaki, Saeid A and Choi, Kristy and Khasahmadi, Amir Hosein and Goyal, Anirudh}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {10043--10053}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/taghanaki21a/taghanaki21a.pdf}, url = {https://proceedings.mlr.press/v139/taghanaki21a.html}, abstract = {A fundamental challenge in artificial intelligence is learning useful representations of data that yield good performance on a downstream classification task, without overfitting to spurious input features. Extracting such task-relevant predictive information becomes particularly difficult for noisy and high-dimensional real-world data. In this work, we propose Contrastive Input Morphing (CIM), a representation learning framework that learns input-space transformations of the data to mitigate the effect of irrelevant input features on downstream performance. Our method leverages a perceptual similarity metric via a triplet loss to ensure that the transformation preserves task-relevant information. Empirically, we demonstrate the efficacy of our approach on various tasks which typically suffer from the presence of spurious correlations: classification with nuisance information, out-of-distribution generalization, and preservation of subgroup accuracies. We additionally show that CIM is complementary to other mutual information-based representation learning techniques, and demonstrate that it improves the performance of variational information bottleneck (VIB) when used in conjunction.} }
Endnote
%0 Conference Paper %T Robust Representation Learning via Perceptual Similarity Metrics %A Saeid A Taghanaki %A Kristy Choi %A Amir Hosein Khasahmadi %A Anirudh Goyal %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-taghanaki21a %I PMLR %P 10043--10053 %U https://proceedings.mlr.press/v139/taghanaki21a.html %V 139 %X A fundamental challenge in artificial intelligence is learning useful representations of data that yield good performance on a downstream classification task, without overfitting to spurious input features. Extracting such task-relevant predictive information becomes particularly difficult for noisy and high-dimensional real-world data. In this work, we propose Contrastive Input Morphing (CIM), a representation learning framework that learns input-space transformations of the data to mitigate the effect of irrelevant input features on downstream performance. Our method leverages a perceptual similarity metric via a triplet loss to ensure that the transformation preserves task-relevant information. Empirically, we demonstrate the efficacy of our approach on various tasks which typically suffer from the presence of spurious correlations: classification with nuisance information, out-of-distribution generalization, and preservation of subgroup accuracies. We additionally show that CIM is complementary to other mutual information-based representation learning techniques, and demonstrate that it improves the performance of variational information bottleneck (VIB) when used in conjunction.
APA
Taghanaki, S.A., Choi, K., Khasahmadi, A.H. & Goyal, A.. (2021). Robust Representation Learning via Perceptual Similarity Metrics. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:10043-10053 Available from https://proceedings.mlr.press/v139/taghanaki21a.html.

Related Material