Dimensionality-Driven Learning with Noisy Labels

Xingjun Ma, Yisen Wang, Michael E. Houle, Shuo Zhou, Sarah Erfani, Shutao Xia, Sudanthi Wijewickrema, James Bailey
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:3355-3364, 2018.

Abstract

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-ma18d, title = {Dimensionality-Driven Learning with Noisy Labels}, author = {Ma, Xingjun and Wang, Yisen and Houle, Michael E. and Zhou, Shuo and Erfani, Sarah and Xia, Shutao and Wijewickrema, Sudanthi and Bailey, James}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {3355--3364}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/ma18d/ma18d.pdf}, url = {https://proceedings.mlr.press/v80/ma18d.html}, abstract = {Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.} }
Endnote
%0 Conference Paper %T Dimensionality-Driven Learning with Noisy Labels %A Xingjun Ma %A Yisen Wang %A Michael E. Houle %A Shuo Zhou %A Sarah Erfani %A Shutao Xia %A Sudanthi Wijewickrema %A James Bailey %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-ma18d %I PMLR %P 3355--3364 %U https://proceedings.mlr.press/v80/ma18d.html %V 80 %X Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.
APA
Ma, X., Wang, Y., Houle, M.E., Zhou, S., Erfani, S., Xia, S., Wijewickrema, S. & Bailey, J.. (2018). Dimensionality-Driven Learning with Noisy Labels. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:3355-3364 Available from https://proceedings.mlr.press/v80/ma18d.html.

Related Material