Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels

Songhua Wu, Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Nannan Wang, Haifeng Liu, Gang Niu
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:11285-11295, 2021.

Abstract

Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in \emph{pointwise} manners. Meanwhile, \emph{pairwise} manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner \emph{mitigate} label noise? To give an affirmative answer, in this paper, we propose a framework called \emph{Class2Simi}: it transforms data points with noisy \emph{class labels} to data pairs with noisy \emph{similarity labels}, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the \emph{reduction of the noise rate} is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the \emph{clean} class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is \emph{computationally efficient} because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-wu21f, title = {Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels}, author = {Wu, Songhua and Xia, Xiaobo and Liu, Tongliang and Han, Bo and Gong, Mingming and Wang, Nannan and Liu, Haifeng and Niu, Gang}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {11285--11295}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/wu21f/wu21f.pdf}, url = {https://proceedings.mlr.press/v139/wu21f.html}, abstract = {Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in \emph{pointwise} manners. Meanwhile, \emph{pairwise} manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner \emph{mitigate} label noise? To give an affirmative answer, in this paper, we propose a framework called \emph{Class2Simi}: it transforms data points with noisy \emph{class labels} to data pairs with noisy \emph{similarity labels}, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the \emph{reduction of the noise rate} is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the \emph{clean} class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is \emph{computationally efficient} because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.} }
Endnote
%0 Conference Paper %T Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels %A Songhua Wu %A Xiaobo Xia %A Tongliang Liu %A Bo Han %A Mingming Gong %A Nannan Wang %A Haifeng Liu %A Gang Niu %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-wu21f %I PMLR %P 11285--11295 %U https://proceedings.mlr.press/v139/wu21f.html %V 139 %X Learning with noisy labels has attracted a lot of attention in recent years, where the mainstream approaches are in \emph{pointwise} manners. Meanwhile, \emph{pairwise} manners have shown great potential in supervised metric learning and unsupervised contrastive learning. Thus, a natural question is raised: does learning in a pairwise manner \emph{mitigate} label noise? To give an affirmative answer, in this paper, we propose a framework called \emph{Class2Simi}: it transforms data points with noisy \emph{class labels} to data pairs with noisy \emph{similarity labels}, where a similarity label denotes whether a pair shares the class label or not. Through this transformation, the \emph{reduction of the noise rate} is theoretically guaranteed, and hence it is in principle easier to handle noisy similarity labels. Amazingly, DNNs that predict the \emph{clean} class labels can be trained from noisy data pairs if they are first pretrained from noisy data points. Class2Simi is \emph{computationally efficient} because not only this transformation is on-the-fly in mini-batches, but also it just changes loss computation on top of model prediction into a pairwise manner. Its effectiveness is verified by extensive experiments.
APA
Wu, S., Xia, X., Liu, T., Han, B., Gong, M., Wang, N., Liu, H. & Niu, G.. (2021). Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:11285-11295 Available from https://proceedings.mlr.press/v139/wu21f.html.

Related Material