SELFIE: Refurbishing Unclean Samples for Robust Deep Learning

Hwanjun Song, Minseok Kim, Jae-Gil Lee
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5907-5915, 2019.

Abstract

Owing to the extremely high expressive power of deep neural networks, their side effect is to totally memorize training data even when the labels are extremely noisy. To overcome overfitting on the noisy labels, we propose a novel robust training method called SELFIE. Our key idea is to selectively refurbish and exploit unclean samples that can be corrected with high precision, thereby gradually increasing the number of available training samples. Taking advantage of this design, SELFIE effectively prevents the risk of noise accumulation from the false correction and fully exploits the training data. To validate the superiority of SELFIE, we conducted extensive experimentation using four real-world or synthetic data sets. The result showed that SELFIE remarkably improved absolute test error compared with two state-of-the-art methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-song19b, title = {{SELFIE}: Refurbishing Unclean Samples for Robust Deep Learning}, author = {Song, Hwanjun and Kim, Minseok and Lee, Jae-Gil}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {5907--5915}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/song19b/song19b.pdf}, url = {https://proceedings.mlr.press/v97/song19b.html}, abstract = {Owing to the extremely high expressive power of deep neural networks, their side effect is to totally memorize training data even when the labels are extremely noisy. To overcome overfitting on the noisy labels, we propose a novel robust training method called SELFIE. Our key idea is to selectively refurbish and exploit unclean samples that can be corrected with high precision, thereby gradually increasing the number of available training samples. Taking advantage of this design, SELFIE effectively prevents the risk of noise accumulation from the false correction and fully exploits the training data. To validate the superiority of SELFIE, we conducted extensive experimentation using four real-world or synthetic data sets. The result showed that SELFIE remarkably improved absolute test error compared with two state-of-the-art methods.} }
Endnote
%0 Conference Paper %T SELFIE: Refurbishing Unclean Samples for Robust Deep Learning %A Hwanjun Song %A Minseok Kim %A Jae-Gil Lee %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-song19b %I PMLR %P 5907--5915 %U https://proceedings.mlr.press/v97/song19b.html %V 97 %X Owing to the extremely high expressive power of deep neural networks, their side effect is to totally memorize training data even when the labels are extremely noisy. To overcome overfitting on the noisy labels, we propose a novel robust training method called SELFIE. Our key idea is to selectively refurbish and exploit unclean samples that can be corrected with high precision, thereby gradually increasing the number of available training samples. Taking advantage of this design, SELFIE effectively prevents the risk of noise accumulation from the false correction and fully exploits the training data. To validate the superiority of SELFIE, we conducted extensive experimentation using four real-world or synthetic data sets. The result showed that SELFIE remarkably improved absolute test error compared with two state-of-the-art methods.
APA
Song, H., Kim, M. & Lee, J.. (2019). SELFIE: Refurbishing Unclean Samples for Robust Deep Learning. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:5907-5915 Available from https://proceedings.mlr.press/v97/song19b.html.

Related Material