Learning Label Shift Correction for Test-Agnostic Long-Tailed Recognition

Tong Wei, Zhen Mao, Zi-Hao Zhou, Yuanyu Wan, Min-Ling Zhang
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:52611-52631, 2024.

Abstract

Long-tail learning primarily focuses on mitigating the label distribution shift between long-tailed training data and uniformly distributed test data. However, in real-world applications, we often encounter a more intricate challenge where the test label distribution is agnostic. To address this problem, we first theoretically establish the substantial potential for reducing the generalization error if we can precisely estimate the test label distribution. Motivated by the theoretical insight, we introduce a simple yet effective solution called label shift correction (LSC). LSC estimates the test label distribution within the proposed framework of generalized black box shift estimation, and adjusts the predictions from a pre-trained model to align with the test distribution. Theoretical analyses confirm that accurate estimation of test label distribution can effectively reduce the generalization error. Extensive experimental results demonstrate that our method significantly outperforms previous state-of-the-art approaches, especially when confronted with non-uniform test label distribution. Notably, the proposed method is general and complements existing long-tail learning approaches, consistently improving their performance. The source code is available at https://github.com/Stomach-ache/label-shift-correction

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-wei24g, title = {Learning Label Shift Correction for Test-Agnostic Long-Tailed Recognition}, author = {Wei, Tong and Mao, Zhen and Zhou, Zi-Hao and Wan, Yuanyu and Zhang, Min-Ling}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {52611--52631}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/wei24g/wei24g.pdf}, url = {https://proceedings.mlr.press/v235/wei24g.html}, abstract = {Long-tail learning primarily focuses on mitigating the label distribution shift between long-tailed training data and uniformly distributed test data. However, in real-world applications, we often encounter a more intricate challenge where the test label distribution is agnostic. To address this problem, we first theoretically establish the substantial potential for reducing the generalization error if we can precisely estimate the test label distribution. Motivated by the theoretical insight, we introduce a simple yet effective solution called label shift correction (LSC). LSC estimates the test label distribution within the proposed framework of generalized black box shift estimation, and adjusts the predictions from a pre-trained model to align with the test distribution. Theoretical analyses confirm that accurate estimation of test label distribution can effectively reduce the generalization error. Extensive experimental results demonstrate that our method significantly outperforms previous state-of-the-art approaches, especially when confronted with non-uniform test label distribution. Notably, the proposed method is general and complements existing long-tail learning approaches, consistently improving their performance. The source code is available at https://github.com/Stomach-ache/label-shift-correction} }
Endnote
%0 Conference Paper %T Learning Label Shift Correction for Test-Agnostic Long-Tailed Recognition %A Tong Wei %A Zhen Mao %A Zi-Hao Zhou %A Yuanyu Wan %A Min-Ling Zhang %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-wei24g %I PMLR %P 52611--52631 %U https://proceedings.mlr.press/v235/wei24g.html %V 235 %X Long-tail learning primarily focuses on mitigating the label distribution shift between long-tailed training data and uniformly distributed test data. However, in real-world applications, we often encounter a more intricate challenge where the test label distribution is agnostic. To address this problem, we first theoretically establish the substantial potential for reducing the generalization error if we can precisely estimate the test label distribution. Motivated by the theoretical insight, we introduce a simple yet effective solution called label shift correction (LSC). LSC estimates the test label distribution within the proposed framework of generalized black box shift estimation, and adjusts the predictions from a pre-trained model to align with the test distribution. Theoretical analyses confirm that accurate estimation of test label distribution can effectively reduce the generalization error. Extensive experimental results demonstrate that our method significantly outperforms previous state-of-the-art approaches, especially when confronted with non-uniform test label distribution. Notably, the proposed method is general and complements existing long-tail learning approaches, consistently improving their performance. The source code is available at https://github.com/Stomach-ache/label-shift-correction
APA
Wei, T., Mao, Z., Zhou, Z., Wan, Y. & Zhang, M.. (2024). Learning Label Shift Correction for Test-Agnostic Long-Tailed Recognition. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:52611-52631 Available from https://proceedings.mlr.press/v235/wei24g.html.

Related Material