Learning Stable Classifiers by Transferring Unstable Features

Yujia Bao, Shiyu Chang, Dr.Regina Barzilay
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:1483-1507, 2022.

Abstract

While unbiased machine learning models are essential for many applications, bias is a human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases – an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task for both synthetically generated environments and real-world environments. Our code is available at https://github.com/YujiaBao/Tofu.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-bao22a, title = {Learning Stable Classifiers by Transferring Unstable Features}, author = {Bao, Yujia and Chang, Shiyu and Barzilay, Dr.Regina}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {1483--1507}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/bao22a/bao22a.pdf}, url = {https://proceedings.mlr.press/v162/bao22a.html}, abstract = {While unbiased machine learning models are essential for many applications, bias is a human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases – an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task for both synthetically generated environments and real-world environments. Our code is available at https://github.com/YujiaBao/Tofu.} }
Endnote
%0 Conference Paper %T Learning Stable Classifiers by Transferring Unstable Features %A Yujia Bao %A Shiyu Chang %A Dr.Regina Barzilay %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-bao22a %I PMLR %P 1483--1507 %U https://proceedings.mlr.press/v162/bao22a.html %V 162 %X While unbiased machine learning models are essential for many applications, bias is a human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases – an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task for both synthetically generated environments and real-world environments. Our code is available at https://github.com/YujiaBao/Tofu.
APA
Bao, Y., Chang, S. & Barzilay, D.. (2022). Learning Stable Classifiers by Transferring Unstable Features. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:1483-1507 Available from https://proceedings.mlr.press/v162/bao22a.html.

Related Material