WS-iFSD: Weakly Supervised Incremental Few-shot Object Detection Without Forgetting

Xinyu Gong, Li Yin, Juan-Manuel Perez-Rua, Zhangyang Wang, Zhicheng Yan
Conference on Parsimony and Learning, PMLR 234:20-38, 2024.

Abstract

Traditional object detection algorithms rely on extensive annotations from a pre-defined set of base categories, leaving them ill-equipped to identify objects from novel classes. We address this limitation by introducing a novel framework for Incremental Few-Shot Object Detection (iFSD). Leveraging a meta-learning approach, our \hypernetwork is designed to generate class-specific codes, enabling object recognition from both base and novel categories. To enhance the \hypernetwork’s generalization performance, we propose a Weakly Supervised Class Augmentation technique that significantly amplifies the training data by merely requiring image-level labels for object localization. Additionally, we stabilize detection performance on base categories by freezing the backbone and detection heads during meta-training. Our model demonstrates significant performance gains on two major benchmarks. Specifically, it outperforms the state-of-the-art ONCE approach on the MS COCO dataset by margins of $2.8%$ and $20.5%$ in box AP for novel and base categories, respectively. When trained on MS COCO and cross-evaluated on PASCAL VOC, our model achieves a four-fold improvement in box AP compared to ONCE.

Cite this Paper

BibTeX
@InProceedings{pmlr-v234-gong24a, title = {WS-iFSD: Weakly Supervised Incremental Few-shot Object Detection Without Forgetting}, author = {Gong, Xinyu and Yin, Li and Perez-Rua, Juan-Manuel and Wang, Zhangyang and Yan, Zhicheng}, booktitle = {Conference on Parsimony and Learning}, pages = {20--38}, year = {2024}, editor = {Chi, Yuejie and Dziugaite, Gintare Karolina and Qu, Qing and Wang, Atlas Wang and Zhu, Zhihui}, volume = {234}, series = {Proceedings of Machine Learning Research}, month = {03--06 Jan}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v234/gong24a/gong24a.pdf}, url = {https://proceedings.mlr.press/v234/gong24a.html}, abstract = {Traditional object detection algorithms rely on extensive annotations from a pre-defined set of base categories, leaving them ill-equipped to identify objects from novel classes. We address this limitation by introducing a novel framework for Incremental Few-Shot Object Detection (iFSD). Leveraging a meta-learning approach, our \hypernetwork is designed to generate class-specific codes, enabling object recognition from both base and novel categories. To enhance the \hypernetwork’s generalization performance, we propose a Weakly Supervised Class Augmentation technique that significantly amplifies the training data by merely requiring image-level labels for object localization. Additionally, we stabilize detection performance on base categories by freezing the backbone and detection heads during meta-training. Our model demonstrates significant performance gains on two major benchmarks. Specifically, it outperforms the state-of-the-art ONCE approach on the MS COCO dataset by margins of $2.8%$ and $20.5%$ in box AP for novel and base categories, respectively. When trained on MS COCO and cross-evaluated on PASCAL VOC, our model achieves a four-fold improvement in box AP compared to ONCE.} }
Endnote
%0 Conference Paper %T WS-iFSD: Weakly Supervised Incremental Few-shot Object Detection Without Forgetting %A Xinyu Gong %A Li Yin %A Juan-Manuel Perez-Rua %A Zhangyang Wang %A Zhicheng Yan %B Conference on Parsimony and Learning %C Proceedings of Machine Learning Research %D 2024 %E Yuejie Chi %E Gintare Karolina Dziugaite %E Qing Qu %E Atlas Wang Wang %E Zhihui Zhu %F pmlr-v234-gong24a %I PMLR %P 20--38 %U https://proceedings.mlr.press/v234/gong24a.html %V 234 %X Traditional object detection algorithms rely on extensive annotations from a pre-defined set of base categories, leaving them ill-equipped to identify objects from novel classes. We address this limitation by introducing a novel framework for Incremental Few-Shot Object Detection (iFSD). Leveraging a meta-learning approach, our \hypernetwork is designed to generate class-specific codes, enabling object recognition from both base and novel categories. To enhance the \hypernetwork’s generalization performance, we propose a Weakly Supervised Class Augmentation technique that significantly amplifies the training data by merely requiring image-level labels for object localization. Additionally, we stabilize detection performance on base categories by freezing the backbone and detection heads during meta-training. Our model demonstrates significant performance gains on two major benchmarks. Specifically, it outperforms the state-of-the-art ONCE approach on the MS COCO dataset by margins of $2.8%$ and $20.5%$ in box AP for novel and base categories, respectively. When trained on MS COCO and cross-evaluated on PASCAL VOC, our model achieves a four-fold improvement in box AP compared to ONCE.
APA
Gong, X., Yin, L., Perez-Rua, J., Wang, Z. & Yan, Z.. (2024). WS-iFSD: Weakly Supervised Incremental Few-shot Object Detection Without Forgetting. Conference on Parsimony and Learning, in Proceedings of Machine Learning Research 234:20-38 Available from https://proceedings.mlr.press/v234/gong24a.html.

Related Material