Towards realistic incremental scenario in class incremental semantic segmentation

Jihwan Kwak, Sungmin Cha, Taesup Moon
Proceedings of The 3rd Conference on Lifelong Learning Agents, PMLR 274:652-671, 2025.

Abstract

This paper addresses the unrealistic aspect of the overlapped scenario, a commonly adopted incremental learning scenario in Class Incremental Semantic Segmentation (CISS). We highlight that the overlapped scenario allows the same image to reappear in future tasks with different pixel labels, creating unwanted advantage or disadvantage to widely used techniques in CISS, such as pseudo-labeling and data replay from the exemplar memory. Our experiments show that methods trained and evaluated under the overlapped scenario can produce biased results, potentially affecting algorithm adoption in practical applications. To mitigate this, we propose a practical scenario called partitioned, where the dataset is first divided into distinct subsets representing each class, and then these subsets are assigned to corresponding tasks. This efficiently addresses the data reappearance artifact while meeting other requirements of CISS scenario, such as capturing the background shifts. Additionally, we identify and resolve the code implementation issues related to replaying data from the exemplar memory, previously overlooked in other works. Lastly, we introduce a simple yet competitive memory-based baseline, MiB-AugM, that handles background shifts in the exemplar memory. This baseline achieves state-of-the-art results across multiple tasks involving learning many new classes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v274-kwak25a, title = {Towards realistic incremental scenario in class incremental semantic segmentation}, author = {Kwak, Jihwan and Cha, Sungmin and Moon, Taesup}, booktitle = {Proceedings of The 3rd Conference on Lifelong Learning Agents}, pages = {652--671}, year = {2025}, editor = {Lomonaco, Vincenzo and Melacci, Stefano and Tuytelaars, Tinne and Chandar, Sarath and Pascanu, Razvan}, volume = {274}, series = {Proceedings of Machine Learning Research}, month = {29 Jul--01 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v274/main/assets/kwak25a/kwak25a.pdf}, url = {https://proceedings.mlr.press/v274/kwak25a.html}, abstract = {This paper addresses the unrealistic aspect of the overlapped scenario, a commonly adopted incremental learning scenario in Class Incremental Semantic Segmentation (CISS). We highlight that the overlapped scenario allows the same image to reappear in future tasks with different pixel labels, creating unwanted advantage or disadvantage to widely used techniques in CISS, such as pseudo-labeling and data replay from the exemplar memory. Our experiments show that methods trained and evaluated under the overlapped scenario can produce biased results, potentially affecting algorithm adoption in practical applications. To mitigate this, we propose a practical scenario called partitioned, where the dataset is first divided into distinct subsets representing each class, and then these subsets are assigned to corresponding tasks. This efficiently addresses the data reappearance artifact while meeting other requirements of CISS scenario, such as capturing the background shifts. Additionally, we identify and resolve the code implementation issues related to replaying data from the exemplar memory, previously overlooked in other works. Lastly, we introduce a simple yet competitive memory-based baseline, MiB-AugM, that handles background shifts in the exemplar memory. This baseline achieves state-of-the-art results across multiple tasks involving learning many new classes.} }
Endnote
%0 Conference Paper %T Towards realistic incremental scenario in class incremental semantic segmentation %A Jihwan Kwak %A Sungmin Cha %A Taesup Moon %B Proceedings of The 3rd Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2025 %E Vincenzo Lomonaco %E Stefano Melacci %E Tinne Tuytelaars %E Sarath Chandar %E Razvan Pascanu %F pmlr-v274-kwak25a %I PMLR %P 652--671 %U https://proceedings.mlr.press/v274/kwak25a.html %V 274 %X This paper addresses the unrealistic aspect of the overlapped scenario, a commonly adopted incremental learning scenario in Class Incremental Semantic Segmentation (CISS). We highlight that the overlapped scenario allows the same image to reappear in future tasks with different pixel labels, creating unwanted advantage or disadvantage to widely used techniques in CISS, such as pseudo-labeling and data replay from the exemplar memory. Our experiments show that methods trained and evaluated under the overlapped scenario can produce biased results, potentially affecting algorithm adoption in practical applications. To mitigate this, we propose a practical scenario called partitioned, where the dataset is first divided into distinct subsets representing each class, and then these subsets are assigned to corresponding tasks. This efficiently addresses the data reappearance artifact while meeting other requirements of CISS scenario, such as capturing the background shifts. Additionally, we identify and resolve the code implementation issues related to replaying data from the exemplar memory, previously overlooked in other works. Lastly, we introduce a simple yet competitive memory-based baseline, MiB-AugM, that handles background shifts in the exemplar memory. This baseline achieves state-of-the-art results across multiple tasks involving learning many new classes.
APA
Kwak, J., Cha, S. & Moon, T.. (2025). Towards realistic incremental scenario in class incremental semantic segmentation. Proceedings of The 3rd Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 274:652-671 Available from https://proceedings.mlr.press/v274/kwak25a.html.

Related Material