RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation

Subhankar Roy, Riccardo Volpi, Gabriela Csurka, Diane Larlus
Proceedings of The 2nd Conference on Lifelong Learning Agents, PMLR 232:244-269, 2023.

Abstract

Class-incremental semantic image segmentation assumes multiple model updates, each enriching the model to segment new categories. This is typically carried out by providing expensive pixel-level annotations to the training algorithm for all new objects, limiting the adoption of such methods in practical applications. Approaches that solely require image-level labels offer an attractive alternative, yet, such coarse annotations lack precise information about the location and boundary of the new objects. In this paper we argue that, since classes represent not just indices but semantic entities, the conceptual relationships between them can provide valuable information that should be leveraged. We propose a weakly supervised approach that exploits such semantic relations to transfer objectness prior from the previously learned classes into the new ones, complementing the supervisory signal from image-level labels. We validate our approach on a number of continual learning tasks, and show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes. We show these conclusions still hold for longer and, hence, more realistic sequences of tasks and for a challenging few-shot scenario.

Cite this Paper

BibTeX
@InProceedings{pmlr-v232-roy23a, title = {RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation}, author = {Roy, Subhankar and Volpi, Riccardo and Csurka, Gabriela and Larlus, Diane}, booktitle = {Proceedings of The 2nd Conference on Lifelong Learning Agents}, pages = {244--269}, year = {2023}, editor = {Chandar, Sarath and Pascanu, Razvan and Sedghi, Hanie and Precup, Doina}, volume = {232}, series = {Proceedings of Machine Learning Research}, month = {22--25 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v232/roy23a/roy23a.pdf}, url = {https://proceedings.mlr.press/v232/roy23a.html}, abstract = {Class-incremental semantic image segmentation assumes multiple model updates, each enriching the model to segment new categories. This is typically carried out by providing expensive pixel-level annotations to the training algorithm for all new objects, limiting the adoption of such methods in practical applications. Approaches that solely require image-level labels offer an attractive alternative, yet, such coarse annotations lack precise information about the location and boundary of the new objects. In this paper we argue that, since classes represent not just indices but semantic entities, the conceptual relationships between them can provide valuable information that should be leveraged. We propose a weakly supervised approach that exploits such semantic relations to transfer objectness prior from the previously learned classes into the new ones, complementing the supervisory signal from image-level labels. We validate our approach on a number of continual learning tasks, and show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes. We show these conclusions still hold for longer and, hence, more realistic sequences of tasks and for a challenging few-shot scenario.} }
Endnote
%0 Conference Paper %T RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation %A Subhankar Roy %A Riccardo Volpi %A Gabriela Csurka %A Diane Larlus %B Proceedings of The 2nd Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2023 %E Sarath Chandar %E Razvan Pascanu %E Hanie Sedghi %E Doina Precup %F pmlr-v232-roy23a %I PMLR %P 244--269 %U https://proceedings.mlr.press/v232/roy23a.html %V 232 %X Class-incremental semantic image segmentation assumes multiple model updates, each enriching the model to segment new categories. This is typically carried out by providing expensive pixel-level annotations to the training algorithm for all new objects, limiting the adoption of such methods in practical applications. Approaches that solely require image-level labels offer an attractive alternative, yet, such coarse annotations lack precise information about the location and boundary of the new objects. In this paper we argue that, since classes represent not just indices but semantic entities, the conceptual relationships between them can provide valuable information that should be leveraged. We propose a weakly supervised approach that exploits such semantic relations to transfer objectness prior from the previously learned classes into the new ones, complementing the supervisory signal from image-level labels. We validate our approach on a number of continual learning tasks, and show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes. We show these conclusions still hold for longer and, hence, more realistic sequences of tasks and for a challenging few-shot scenario.
APA
Roy, S., Volpi, R., Csurka, G. & Larlus, D.. (2023). RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation. Proceedings of The 2nd Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 232:244-269 Available from https://proceedings.mlr.press/v232/roy23a.html.

Related Material