Differencing based Self-supervised pretraining for Scene Change Detection

Vijaya Raghavan T. Ramkumar, Elahe Arani, Bahram Zonooz
Proceedings of The 1st Conference on Lifelong Learning Agents, PMLR 199:952-965, 2022.

Abstract

Scene change detection (SCD), a crucial perception task, identifies changes by comparing scenes captured at different times. SCD is challenging due to noisy changes in illumination, seasonal variations, and perspective differences across a pair of views. Deep neural networks (DNNs)-based solutions require a large quantity of annotated data which is tedious and expensive to obtain. On the other hand, transfer learning from large datasets induces domain shift. To address these challenges, we propose a novel Differencing Self-supervised Pretraining (DSP) method that uses feature differencing to learn discriminatory representations corresponding to the changed regions while simultaneously tackling the noisy changes by enforcing temporal invariance across views. Our experimental results on SCD datasets demonstrate the effectiveness of our method, specifically to differences in camera viewpoints and lighting conditions. Compared against the standard ImageNet pretraining that uses more than a million additional labeled images, DSP can surpass it without using any additional data. Our results also demonstrate the robustness of DSP to natural corruptions, distribution shift, and learning under limited labeled data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v199-ramkumar22a, title = {Differencing based Self-supervised pretraining for Scene Change Detection}, author = {Ramkumar, Vijaya Raghavan T. and Arani, Elahe and Zonooz, Bahram}, booktitle = {Proceedings of The 1st Conference on Lifelong Learning Agents}, pages = {952--965}, year = {2022}, editor = {Chandar, Sarath and Pascanu, Razvan and Precup, Doina}, volume = {199}, series = {Proceedings of Machine Learning Research}, month = {22--24 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v199/ramkumar22a/ramkumar22a.pdf}, url = {https://proceedings.mlr.press/v199/ramkumar22a.html}, abstract = {Scene change detection (SCD), a crucial perception task, identifies changes by comparing scenes captured at different times. SCD is challenging due to noisy changes in illumination, seasonal variations, and perspective differences across a pair of views. Deep neural networks (DNNs)-based solutions require a large quantity of annotated data which is tedious and expensive to obtain. On the other hand, transfer learning from large datasets induces domain shift. To address these challenges, we propose a novel Differencing Self-supervised Pretraining (DSP) method that uses feature differencing to learn discriminatory representations corresponding to the changed regions while simultaneously tackling the noisy changes by enforcing temporal invariance across views. Our experimental results on SCD datasets demonstrate the effectiveness of our method, specifically to differences in camera viewpoints and lighting conditions. Compared against the standard ImageNet pretraining that uses more than a million additional labeled images, DSP can surpass it without using any additional data. Our results also demonstrate the robustness of DSP to natural corruptions, distribution shift, and learning under limited labeled data.} }
Endnote
%0 Conference Paper %T Differencing based Self-supervised pretraining for Scene Change Detection %A Vijaya Raghavan T. Ramkumar %A Elahe Arani %A Bahram Zonooz %B Proceedings of The 1st Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2022 %E Sarath Chandar %E Razvan Pascanu %E Doina Precup %F pmlr-v199-ramkumar22a %I PMLR %P 952--965 %U https://proceedings.mlr.press/v199/ramkumar22a.html %V 199 %X Scene change detection (SCD), a crucial perception task, identifies changes by comparing scenes captured at different times. SCD is challenging due to noisy changes in illumination, seasonal variations, and perspective differences across a pair of views. Deep neural networks (DNNs)-based solutions require a large quantity of annotated data which is tedious and expensive to obtain. On the other hand, transfer learning from large datasets induces domain shift. To address these challenges, we propose a novel Differencing Self-supervised Pretraining (DSP) method that uses feature differencing to learn discriminatory representations corresponding to the changed regions while simultaneously tackling the noisy changes by enforcing temporal invariance across views. Our experimental results on SCD datasets demonstrate the effectiveness of our method, specifically to differences in camera viewpoints and lighting conditions. Compared against the standard ImageNet pretraining that uses more than a million additional labeled images, DSP can surpass it without using any additional data. Our results also demonstrate the robustness of DSP to natural corruptions, distribution shift, and learning under limited labeled data.
APA
Ramkumar, V.R.T., Arani, E. & Zonooz, B.. (2022). Differencing based Self-supervised pretraining for Scene Change Detection. Proceedings of The 1st Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 199:952-965 Available from https://proceedings.mlr.press/v199/ramkumar22a.html.

Related Material