Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates

Yang Liu, Hongyi Guo
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:6226-6236, 2020.

Abstract

Learning with noisy labels is a common challenge in supervised learning. Existing approaches often require practitioners to specify noise rates, i.e., a set of parameters controlling the severity of label noises in the problem, and the specifications are either assumed to be given or estimated using additional steps. In this work, we introduce a new family of loss functions that we name as peer loss functions, which enables learning from noisy labels and does not require a priori specification of the noise rates. Peer loss functions work within the standard empirical risk minimization (ERM) framework. We show that, under mild conditions, performing ERM with peer loss functions on the noisy data leads to the optimal or a near-optimal classifier as if performing ERM over the clean training data, which we do not have access to. We pair our results with an extensive set of experiments. Peer loss provides a way to simplify model development when facing potentially noisy training labels, and can be promoted as a robust candidate loss function in such situations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-liu20e, title = {Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates}, author = {Liu, Yang and Guo, Hongyi}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {6226--6236}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/liu20e/liu20e.pdf}, url = {https://proceedings.mlr.press/v119/liu20e.html}, abstract = {Learning with noisy labels is a common challenge in supervised learning. Existing approaches often require practitioners to specify noise rates, i.e., a set of parameters controlling the severity of label noises in the problem, and the specifications are either assumed to be given or estimated using additional steps. In this work, we introduce a new family of loss functions that we name as peer loss functions, which enables learning from noisy labels and does not require a priori specification of the noise rates. Peer loss functions work within the standard empirical risk minimization (ERM) framework. We show that, under mild conditions, performing ERM with peer loss functions on the noisy data leads to the optimal or a near-optimal classifier as if performing ERM over the clean training data, which we do not have access to. We pair our results with an extensive set of experiments. Peer loss provides a way to simplify model development when facing potentially noisy training labels, and can be promoted as a robust candidate loss function in such situations.} }
Endnote
%0 Conference Paper %T Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates %A Yang Liu %A Hongyi Guo %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-liu20e %I PMLR %P 6226--6236 %U https://proceedings.mlr.press/v119/liu20e.html %V 119 %X Learning with noisy labels is a common challenge in supervised learning. Existing approaches often require practitioners to specify noise rates, i.e., a set of parameters controlling the severity of label noises in the problem, and the specifications are either assumed to be given or estimated using additional steps. In this work, we introduce a new family of loss functions that we name as peer loss functions, which enables learning from noisy labels and does not require a priori specification of the noise rates. Peer loss functions work within the standard empirical risk minimization (ERM) framework. We show that, under mild conditions, performing ERM with peer loss functions on the noisy data leads to the optimal or a near-optimal classifier as if performing ERM over the clean training data, which we do not have access to. We pair our results with an extensive set of experiments. Peer loss provides a way to simplify model development when facing potentially noisy training labels, and can be promoted as a robust candidate loss function in such situations.
APA
Liu, Y. & Guo, H.. (2020). Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:6226-6236 Available from https://proceedings.mlr.press/v119/liu20e.html.

Related Material