Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize

Tianren Zhang, Chujie Zhao, Guanyu Chen, Yizhou Jiang, Feng Chen
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:60446-60495, 2024.

Abstract

Learning representations that generalize under distribution shifts is critical for building robust machine learning models. However, despite significant efforts in recent years, algorithmic advances in this direction have been limited. In this work, we seek to understand the fundamental difficulty of out-of-distribution generalization with deep neural networks. We first empirically show that perhaps surprisingly, even allowing a neural network to explicitly fit the representations obtained from a teacher network that can generalize out-of-distribution is insufficient for the generalization of the student network. Then, by a theoretical study of two-layer ReLU networks optimized by stochastic gradient descent (SGD) under a structured feature model, we identify a fundamental yet unexplored feature learning proclivity of neural networks, feature contamination: neural networks can learn uncorrelated features together with predictive features, resulting in generalization failure under distribution shifts. Notably, this mechanism essentially differs from the prevailing narrative in the literature that attributes the generalization failure to spurious correlations. Overall, our results offer new insights into the non-linear feature learning dynamics of neural networks and highlight the necessity of considering inductive biases in out-of-distribution generalization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-zhang24cj, title = {Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize}, author = {Zhang, Tianren and Zhao, Chujie and Chen, Guanyu and Jiang, Yizhou and Chen, Feng}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {60446--60495}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/zhang24cj/zhang24cj.pdf}, url = {https://proceedings.mlr.press/v235/zhang24cj.html}, abstract = {Learning representations that generalize under distribution shifts is critical for building robust machine learning models. However, despite significant efforts in recent years, algorithmic advances in this direction have been limited. In this work, we seek to understand the fundamental difficulty of out-of-distribution generalization with deep neural networks. We first empirically show that perhaps surprisingly, even allowing a neural network to explicitly fit the representations obtained from a teacher network that can generalize out-of-distribution is insufficient for the generalization of the student network. Then, by a theoretical study of two-layer ReLU networks optimized by stochastic gradient descent (SGD) under a structured feature model, we identify a fundamental yet unexplored feature learning proclivity of neural networks, feature contamination: neural networks can learn uncorrelated features together with predictive features, resulting in generalization failure under distribution shifts. Notably, this mechanism essentially differs from the prevailing narrative in the literature that attributes the generalization failure to spurious correlations. Overall, our results offer new insights into the non-linear feature learning dynamics of neural networks and highlight the necessity of considering inductive biases in out-of-distribution generalization.} }
Endnote
%0 Conference Paper %T Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize %A Tianren Zhang %A Chujie Zhao %A Guanyu Chen %A Yizhou Jiang %A Feng Chen %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-zhang24cj %I PMLR %P 60446--60495 %U https://proceedings.mlr.press/v235/zhang24cj.html %V 235 %X Learning representations that generalize under distribution shifts is critical for building robust machine learning models. However, despite significant efforts in recent years, algorithmic advances in this direction have been limited. In this work, we seek to understand the fundamental difficulty of out-of-distribution generalization with deep neural networks. We first empirically show that perhaps surprisingly, even allowing a neural network to explicitly fit the representations obtained from a teacher network that can generalize out-of-distribution is insufficient for the generalization of the student network. Then, by a theoretical study of two-layer ReLU networks optimized by stochastic gradient descent (SGD) under a structured feature model, we identify a fundamental yet unexplored feature learning proclivity of neural networks, feature contamination: neural networks can learn uncorrelated features together with predictive features, resulting in generalization failure under distribution shifts. Notably, this mechanism essentially differs from the prevailing narrative in the literature that attributes the generalization failure to spurious correlations. Overall, our results offer new insights into the non-linear feature learning dynamics of neural networks and highlight the necessity of considering inductive biases in out-of-distribution generalization.
APA
Zhang, T., Zhao, C., Chen, G., Jiang, Y. & Chen, F.. (2024). Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:60446-60495 Available from https://proceedings.mlr.press/v235/zhang24cj.html.

Related Material