Keypoint Mask-based Local Feature Matching and Keypoint Erasing-based Global Feature Representations for Visible-Infrared Person Re-Identification

Kisung Seo, Soonyong Gwon, Chae Woon
Proceedings of The Workshop on Classifier Learning from Difficult Data, PMLR 263:9-16, 2024.

Abstract

In Visible-Infrared Person Re-Identification, the most crucial challenge is reducing the Modality discrepancy between visible and infrared images. To address this, various approaches such as data augmentation, generative transformation for the opposite modality, and extraction of Modality-Shared and Modality-Specific features have been attempted. While each approach has contributed significantly to performance improvement, re-identification remains particularly challenging when dealing with individuals who have similar clothing or body shapes but are different persons. It is mainly due to the inconsistency in representing identical local features across cross-modalities in existing methods. This paper proposes a novel learning representations of keypoint-based local and global features. Keypoint-based masking for local feature representation learning aims to normalize the representations of each keypoint’s locality, thereby reducing the modality gap at the feature level. Representation learning for local features using keypoint-based masking reduces feature-level modality gaps by identifying the local representation of each keypoint. Representation learning for global features using keypoint-based eraing increases the efficiency and diversity of the global representation by generating images that cover the same area. We compare our proposed methodology and various existing methods for the mAP and Rank-1 performances on the SYSU-MM01 datasets. Experimental results demonstrate that our proposed model effectively solves the existing key and critical problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v263-seo24a, title = {Keypoint Mask-based Local Feature Matching and Keypoint Erasing-based Global Feature Representations for Visible-Infrared Person Re-Identification}, author = {Seo, Kisung and Gwon, Soonyong and Woon, Chae}, booktitle = {Proceedings of The Workshop on Classifier Learning from Difficult Data}, pages = {9--16}, year = {2024}, editor = {Zyblewski, Pawel and Grana, Manuel and Pawel, Ksieniewicz and Minku, Leandro}, volume = {263}, series = {Proceedings of Machine Learning Research}, month = {19--20 Oct}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v263/main/assets/seo24a/seo24a.pdf}, url = {https://proceedings.mlr.press/v263/seo24a.html}, abstract = {In Visible-Infrared Person Re-Identification, the most crucial challenge is reducing the Modality discrepancy between visible and infrared images. To address this, various approaches such as data augmentation, generative transformation for the opposite modality, and extraction of Modality-Shared and Modality-Specific features have been attempted. While each approach has contributed significantly to performance improvement, re-identification remains particularly challenging when dealing with individuals who have similar clothing or body shapes but are different persons. It is mainly due to the inconsistency in representing identical local features across cross-modalities in existing methods. This paper proposes a novel learning representations of keypoint-based local and global features. Keypoint-based masking for local feature representation learning aims to normalize the representations of each keypoint’s locality, thereby reducing the modality gap at the feature level. Representation learning for local features using keypoint-based masking reduces feature-level modality gaps by identifying the local representation of each keypoint. Representation learning for global features using keypoint-based eraing increases the efficiency and diversity of the global representation by generating images that cover the same area. We compare our proposed methodology and various existing methods for the mAP and Rank-1 performances on the SYSU-MM01 datasets. Experimental results demonstrate that our proposed model effectively solves the existing key and critical problems.} }
Endnote
%0 Conference Paper %T Keypoint Mask-based Local Feature Matching and Keypoint Erasing-based Global Feature Representations for Visible-Infrared Person Re-Identification %A Kisung Seo %A Soonyong Gwon %A Chae Woon %B Proceedings of The Workshop on Classifier Learning from Difficult Data %C Proceedings of Machine Learning Research %D 2024 %E Pawel Zyblewski %E Manuel Grana %E Ksieniewicz Pawel %E Leandro Minku %F pmlr-v263-seo24a %I PMLR %P 9--16 %U https://proceedings.mlr.press/v263/seo24a.html %V 263 %X In Visible-Infrared Person Re-Identification, the most crucial challenge is reducing the Modality discrepancy between visible and infrared images. To address this, various approaches such as data augmentation, generative transformation for the opposite modality, and extraction of Modality-Shared and Modality-Specific features have been attempted. While each approach has contributed significantly to performance improvement, re-identification remains particularly challenging when dealing with individuals who have similar clothing or body shapes but are different persons. It is mainly due to the inconsistency in representing identical local features across cross-modalities in existing methods. This paper proposes a novel learning representations of keypoint-based local and global features. Keypoint-based masking for local feature representation learning aims to normalize the representations of each keypoint’s locality, thereby reducing the modality gap at the feature level. Representation learning for local features using keypoint-based masking reduces feature-level modality gaps by identifying the local representation of each keypoint. Representation learning for global features using keypoint-based eraing increases the efficiency and diversity of the global representation by generating images that cover the same area. We compare our proposed methodology and various existing methods for the mAP and Rank-1 performances on the SYSU-MM01 datasets. Experimental results demonstrate that our proposed model effectively solves the existing key and critical problems.
APA
Seo, K., Gwon, S. & Woon, C.. (2024). Keypoint Mask-based Local Feature Matching and Keypoint Erasing-based Global Feature Representations for Visible-Infrared Person Re-Identification. Proceedings of The Workshop on Classifier Learning from Difficult Data, in Proceedings of Machine Learning Research 263:9-16 Available from https://proceedings.mlr.press/v263/seo24a.html.

Related Material