Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection

Nadine Chang, Zhiding Yu, Yu-Xiong Wang, Animashree Anandkumar, Sanja Fidler, Jose M Alvarez
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:1463-1472, 2021.

Abstract

Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resampling is typically introduced as a simple but effective approach. However, we observe that long-tailed detection differs from classification since multiple classes may be present in one image. As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object-level. We address object-level resampling by introducing an object-centric sampling strategy based on a dynamic, episodic memory bank. Our proposed strategy has two benefits: 1) convenient object-level resampling without significant extra computation, and 2) implicit feature-level augmentation from model updates. We show that image-level and object-level resamplings are both important, and thus unify them with a joint resampling strategy. Our method achieves state-of-the-art performance on the rare categories of LVIS, with 1.89% and 3.13% relative improvements over Forest R-CNN on detection and instance segmentation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-chang21c, title = {Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection}, author = {Chang, Nadine and Yu, Zhiding and Wang, Yu-Xiong and Anandkumar, Animashree and Fidler, Sanja and Alvarez, Jose M}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {1463--1472}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/chang21c/chang21c.pdf}, url = {https://proceedings.mlr.press/v139/chang21c.html}, abstract = {Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resampling is typically introduced as a simple but effective approach. However, we observe that long-tailed detection differs from classification since multiple classes may be present in one image. As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object-level. We address object-level resampling by introducing an object-centric sampling strategy based on a dynamic, episodic memory bank. Our proposed strategy has two benefits: 1) convenient object-level resampling without significant extra computation, and 2) implicit feature-level augmentation from model updates. We show that image-level and object-level resamplings are both important, and thus unify them with a joint resampling strategy. Our method achieves state-of-the-art performance on the rare categories of LVIS, with 1.89% and 3.13% relative improvements over Forest R-CNN on detection and instance segmentation.} }
Endnote
%0 Conference Paper %T Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection %A Nadine Chang %A Zhiding Yu %A Yu-Xiong Wang %A Animashree Anandkumar %A Sanja Fidler %A Jose M Alvarez %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-chang21c %I PMLR %P 1463--1472 %U https://proceedings.mlr.press/v139/chang21c.html %V 139 %X Training on datasets with long-tailed distributions has been challenging for major recognition tasks such as classification and detection. To deal with this challenge, image resampling is typically introduced as a simple but effective approach. However, we observe that long-tailed detection differs from classification since multiple classes may be present in one image. As a result, image resampling alone is not enough to yield a sufficiently balanced distribution at the object-level. We address object-level resampling by introducing an object-centric sampling strategy based on a dynamic, episodic memory bank. Our proposed strategy has two benefits: 1) convenient object-level resampling without significant extra computation, and 2) implicit feature-level augmentation from model updates. We show that image-level and object-level resamplings are both important, and thus unify them with a joint resampling strategy. Our method achieves state-of-the-art performance on the rare categories of LVIS, with 1.89% and 3.13% relative improvements over Forest R-CNN on detection and instance segmentation.
APA
Chang, N., Yu, Z., Wang, Y., Anandkumar, A., Fidler, S. & Alvarez, J.M.. (2021). Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:1463-1472 Available from https://proceedings.mlr.press/v139/chang21c.html.

Related Material