Focused Anchors Loss: cost-sensitive learning of discriminative features for imbalanced classification

Bahram K. Baloch, Sateesh Kumar, Sanjay Haresh, Abeerah Rehman, Tahir Syed
Proceedings of The Eleventh Asian Conference on Machine Learning, PMLR 101:822-835, 2019.

Abstract

Deep Neural Networks (DNNs) usually suffer performance penalties when there is a skewed label distribution. This phenomenon, class-imbalance, is most often mitigated peripheral to the classification algorithm itself, usually by modifying the amount of examples per class, for oversampling at the expense of computational efficiency, and for undersampling at the expense of statistical efficiency. In our solution, we combine discriminative feature learning with cost-sensitive learning to tackle the class imbalance problem by using a two step loss function, which we call the Focused Anchors loss (FAL). We evaluate FAL and its variant, Focused Anchor Mean Loss (FAML), on $6$ different datasets in comparison of traditional cross entropy loss and we observe a significant gain in balanced accuracy for all datasets. We also perform better than time-costly re-sampling and ensemble methods like SMOTE and Near Miss in $4$ out of $6$ datasets across F1-score, AUC-ROC and balanced accuracy. We also extend our evaluation to image domain and use long-tailed CIFAR$10$ to evaluate our loss function where we consistently report significant improvement in accuracy. We then go on to test our loss function under extreme imbalance on a propriety dataset and achieve a gain of $0.1$ AUC-ROC over the baseline.

Cite this Paper

BibTeX
@InProceedings{pmlr-v101-baloch19a, title = {Focused Anchors Loss: cost-sensitive learning of discriminative features for imbalanced classification}, author = {Baloch, Bahram K. and Kumar, Sateesh and Haresh, Sanjay and Rehman, Abeerah and Syed, Tahir}, booktitle = {Proceedings of The Eleventh Asian Conference on Machine Learning}, pages = {822--835}, year = {2019}, editor = {Lee, Wee Sun and Suzuki, Taiji}, volume = {101}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v101/baloch19a/baloch19a.pdf}, url = {https://proceedings.mlr.press/v101/baloch19a.html}, abstract = {Deep Neural Networks (DNNs) usually suffer performance penalties when there is a skewed label distribution. This phenomenon, class-imbalance, is most often mitigated peripheral to the classification algorithm itself, usually by modifying the amount of examples per class, for oversampling at the expense of computational efficiency, and for undersampling at the expense of statistical efficiency. In our solution, we combine discriminative feature learning with cost-sensitive learning to tackle the class imbalance problem by using a two step loss function, which we call the Focused Anchors loss (FAL). We evaluate FAL and its variant, Focused Anchor Mean Loss (FAML), on $6$ different datasets in comparison of traditional cross entropy loss and we observe a significant gain in balanced accuracy for all datasets. We also perform better than time-costly re-sampling and ensemble methods like SMOTE and Near Miss in $4$ out of $6$ datasets across F1-score, AUC-ROC and balanced accuracy. We also extend our evaluation to image domain and use long-tailed CIFAR$10$ to evaluate our loss function where we consistently report significant improvement in accuracy. We then go on to test our loss function under extreme imbalance on a propriety dataset and achieve a gain of $0.1$ AUC-ROC over the baseline.} }
Endnote
%0 Conference Paper %T Focused Anchors Loss: cost-sensitive learning of discriminative features for imbalanced classification %A Bahram K. Baloch %A Sateesh Kumar %A Sanjay Haresh %A Abeerah Rehman %A Tahir Syed %B Proceedings of The Eleventh Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Wee Sun Lee %E Taiji Suzuki %F pmlr-v101-baloch19a %I PMLR %P 822--835 %U https://proceedings.mlr.press/v101/baloch19a.html %V 101 %X Deep Neural Networks (DNNs) usually suffer performance penalties when there is a skewed label distribution. This phenomenon, class-imbalance, is most often mitigated peripheral to the classification algorithm itself, usually by modifying the amount of examples per class, for oversampling at the expense of computational efficiency, and for undersampling at the expense of statistical efficiency. In our solution, we combine discriminative feature learning with cost-sensitive learning to tackle the class imbalance problem by using a two step loss function, which we call the Focused Anchors loss (FAL). We evaluate FAL and its variant, Focused Anchor Mean Loss (FAML), on $6$ different datasets in comparison of traditional cross entropy loss and we observe a significant gain in balanced accuracy for all datasets. We also perform better than time-costly re-sampling and ensemble methods like SMOTE and Near Miss in $4$ out of $6$ datasets across F1-score, AUC-ROC and balanced accuracy. We also extend our evaluation to image domain and use long-tailed CIFAR$10$ to evaluate our loss function where we consistently report significant improvement in accuracy. We then go on to test our loss function under extreme imbalance on a propriety dataset and achieve a gain of $0.1$ AUC-ROC over the baseline.
APA
Baloch, B.K., Kumar, S., Haresh, S., Rehman, A. & Syed, T.. (2019). Focused Anchors Loss: cost-sensitive learning of discriminative features for imbalanced classification. Proceedings of The Eleventh Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 101:822-835 Available from https://proceedings.mlr.press/v101/baloch19a.html.

Related Material